Tracking and positioning for digital radiographic images

Abstract

A radiographic imaging system includes a movable tube head, a camera, and a movable detector mount. A marker is disposed on the detector holder, and a plurality of digital images of the detector holder and the marker are captured by a camera. Each of the digital images is captured at a different relative position between the camera and the movable detector mount. Spatial coordinates of the camera are determined for each of the plurality of captured digital images, and a mathematical best fit function for the spatial coordinates is calculated and stored.

Description

Background Technology

[0001] The topics disclosed in this paper relate to digital radiography (DR). In particular, they relate to systems and methods for visual tracking and positioning in digital radiography imaging systems.

[0002] In DR imaging, accurate patient positioning has always been essential for radiology technicians and is crucial for generating acceptable X-ray images. The positioning process is highly dependent on the technician's clinical experience. Typical reference lines on the bucky, such as the detector and AEC boundary lines, or the crosshairs projected from the top of the tube head, can be used to assist in the positioning process. Once the patient is positioned on the bucky, some reference lines may no longer be clearly visible, and it may be difficult for the technician to determine whether the patient's positioning relative to the bucky is acceptable. When the patient is lying on the table, even fewer visual positioning references may be available. Even highly trained technicians may arrange inappropriate positioning due to a lack of visual references for comparison or due to complex placement requirements. In these cases, the resulting images may fail to meet diagnostic requirements.

[0003] Based on current general DR systems, manual positioning relies on experience and skill. There is room to improve positioning accuracy to reduce failure rates. Therefore, visual tracking and positioning assistance systems and methods (which provide visual guidance and functionality for tracking and positioning the spatial location and distance between the DR system and the patient) can not only facilitate the positioning process but also enable other intelligent and automated features.

[0004] Previously developed systems have incorporated real-time location marker detection and body feature analysis using 3D depth cameras. Such systems offer automated positioning capabilities to assist in the X-ray projection imaging process. A drawback of some previous solutions focusing on walking-standing chest AP / PA and lateral X-ray imaging is that, due to limitations in the method of tracking and positioning spatial information between the DR system and the patient, this technology cannot support imaging using a table or when the X-ray tube head is angled. This limitation also makes it difficult to extend the method to all common clinical scenarios.

[0005] Another drawback of some previous solutions based on real-time location marker detection is that the technology becomes ineffective once the location markers are covered by the patient or cannot be detected by the system due to image noise, distortion, or blurring. This makes it difficult to support imaging of various body types and body parts. Another drawback of some previous solutions is that accuracy is highly affected by errors caused by depth measurement, camera mounting, and mechanical movement of the DR system, which may be difficult to eliminate or reduce within the established framework.

[0006] As described in U.S. Patent No. 9,895,131, a prior solution discloses a system for the automatic control of an X-ray tube scanner using a 3D camera. Automatic control is achieved by performing real-time calibration using RGB images between the stage and the X-ray tube. Due to the calibration method disclosed therein, this solution has limited application scenarios and is susceptible to factors arising from both mechanical and patient-side factors.

[0007] The above discussion provides only general background information and is not intended to be used as an aid in determining the scope of the subject matter for which protection is claimed. Summary of the Invention

[0008] The X-ray imaging system includes a movable tube head, a camera, and a movable detector support. Position markers or targets are placed on the detector support, and multiple digital images of the detector support and markers are captured by the camera. Each digital image is captured at a different relative position between the movable tube head and the camera. The spatial coordinates of the camera are determined for each of the multiple captured digital images, and a best-fit function (such as a polynomial function) is calculated and stored, which can then be used to determine the spatial coordinates of other camera positions.

[0009] In one embodiment, a method of operating a radiographic imaging system is disclosed, the system having a camera attached to a movable tube head and a movable detector holder. A marker is attached to or positioned on the movable detector holder, and multiple digital images of the movable detector holder and the marker are captured using the camera. And so on, captured. The spatial coordinates of the camera are determined for each of the multiple captured digital images, and a function (such as a polynomial function) that best fits the spatial coordinates of the multiple cameras is stored.

[0010] This invention addresses the tracking and positioning of a DR system relative to a patient and / or filters using a proposed labelless motion model and keypoint detection, which provide information and automated guidance to assist in the positioning operation of the DR system. This technique addresses most of the problems of the prior solutions described above. In one embodiment, augmented reality is employed to provide real-time visual guidance, making the workflow compatible with current clinical applications. A visual tracking and positioning assistance system is introduced to overcome the shortcomings described herein. The disclosed system tracks and positions the spatial location between a wall filter, table, X-ray tube, and patient using the proposed labelless motion model and keypoint detection. This system provides both manual and automated positioning functionality, wherein augmented reality is implemented to visually display positioning information to the user, such as, but not limited to, detector boundaries, AEC boundaries, and collimator light boundaries.

[0011] In one embodiment, markers may be mounted on a filter, such that the angles of the markers are at fixed, known distances from a designated origin of the world coordinate system and from a digital ray imaging detector fixed within the filter. In one embodiment, the origin of the world coordinate system may be the center of the filter. To locate the detector, a digital video camera is used to locate at least two markers in the transmitted camera image. Typical marker-based solutions cannot be applied to situations where markers are completely covered or unavailable (e.g., tabletop examinations). When markers are obscured by the patient, a camera projection model can be used to estimate the position of the markers in the video camera image. Based on the camera projection model, matrix multiplication can be used to link world coordinates or DR system coordinates to pixel coordinates in the video image. A popular method for determining extrinsic parameters is called the Perspective n-Point (PnP) algorithm. PnP addresses the following problem: given n 3D points in the world (i.e., markers on the filter), their corresponding 2D projections in the image, and a set of camera intrinsic parameters, it estimates the 3D spatial pose, which includes the calibrated camera's 3D spatial coordinates and azimuth, i.e., 3D angular azimuth (pitch, translation, and roll), which can also be collectively referred to as camera extrinsic parameters. Given a set of corresponding data (such as the position coordinates of the filter and nozzle where the camera is mounted), rotation can be calculated. R coordinates and xyz Translation T Coordinates. This method directly solves the mapping between camera image coordinates and camera position coordinates. In practice, camera intrinsic parameters such as focal length and principal point are inherent characteristics of the camera and do not change. Camera extrinsic parameters (including xyz position coordinates) can change with rotation angles and translations, and will also depend on the construction of the world coordinate system (i.e., in DR system coordinates).

[0012] As described herein, a video camera is mounted on a movable tube head. When the tube head is moved, the camera's extrinsic parameters are changed. It is required to adjust the camera's current extrinsic parameters, and a method for adjusting the camera's current extrinsic parameters without requiring the detection of filter markers in the camera image is described herein. The camera's extrinsic parameters are updated based on the tube head movement and the camera projection model disclosed herein. The tube head movement xyz coordinates are transmitted via an encoder in the DR imaging system, and filter coordinates, based on the center position of the filter, are transmitted using an encoder in the filter system. In one embodiment, the calibration workflow uses tube head and ledge movement in automatic tracking mode. The world coordinate system is based on the center (0,0,0, origin coordinates) of the ledge filter, and the world coordinates of the marker angles are determined based on their known positions relative to the center of the filter. The camera captures a visible image of the markers, and the markers in the image are then detected according to a computer program, and a correspondence between 2D and 3D points is established. The basic extrinsic parameters of the camera for this spatial relationship between the wall filter and the tube head are established and represented as... R basic and T basic .

[0013] In one embodiment, the tube head may have a translational movement in the y-direction relative to the center of the wall-mounted filter, denoted as Δ. trans_y The new camera extrinsic parameters can be calculated as follows: in Where Δ trans_y This information is obtained from feedback from the mechanical encoder of the DR system. A similar transformation regarding the camera's rotation can also be calculated.

[0014] In one embodiment, based on camera extrinsic parameters calculated from the spatial relationship between the wall-mounted filter and the tube head described herein, the position of the marker can be estimated using a camera projection model, and the position of the detector can be derived from the positions of one or more markers. In one embodiment, translation values ​​when updating camera extrinsic parameters can be compensated for using a linear fitting method. As described herein, when the wall-mounted filter and the tube head are in different predefined spatial relationships, by obtaining multiple... R basic and T basic The method of calculating linear fitting using sample data points.

[0015] The general description above is not intended to suggest that the elements are separate, non-interchangeable embodiments. In fact, many of the elements described in relation to a particular embodiment can be used together with and may be interchanged with elements of other described embodiments. Many changes and modifications can be made within the scope of this invention without departing from its spirit, and this invention includes all such modifications.

[0016] This brief description of the invention is intended only to provide a concise overview of the subject matter disclosed herein according to one or more illustrative embodiments, and is not intended to guide the interpretation of the claims, nor to define or limit the scope of the invention, which is defined solely by the appended claims. This brief description is provided to introduce illustrative choices of concepts in a simplified form, which are further described in the detailed description below. This brief description is not intended to identify key or essential features of the claimed subject matter, nor is it intended to serve as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that address any or all of the shortcomings pointed out in the background. Attached Figure Description

[0017] To enable understanding of the features of the invention, a detailed description of the invention can be given by reference to certain embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only certain embodiments of the invention and should therefore not be considered as limiting its scope, as the scope of the invention encompasses other equally effective embodiments. The drawings below are not intended to be drawn at any precise scale relative to relative sizes, angular relationships, relative positions, or timing relationships, nor are they intended to be drawn in any combination of representations of interchangeability, substitution, or claimed implementations; the emphasis is generally placed on illustrating features of certain embodiments of the invention. In the drawings, the same numerals are used throughout the various views to indicate the same parts. Therefore, for a further understanding of the invention, reference can be made to the following detailed description taken in conjunction with the accompanying drawings, wherein: Figure 1 This is a schematic diagram of an exemplary wall-mounted filter type X-ray imaging system; Figure 2 This is a schematic diagram of an exemplary tabletop X-ray imaging system; Figure 3 This is a flowchart of an exemplary initialization process for markerless methods and systems; Figure 4 This is a schematic diagram of an exemplary filter with marking and DR detector; Figures 5A-5D This is a schematic diagram of an exemplary calibration method for an exemplary wall-mounted filter type X-ray imaging system; and Figure 6This is a schematic diagram of an exemplary calibration method for an exemplary stage-type X-ray imaging system. Detailed Implementation

[0018] refer to Figure 1 and Figure 2 Two embodiments of a radiographic imaging system are described, the radiographic imaging system including a vertical wall or column radiographic imaging system 100 ( Figure 1 ) and water platform-based X-ray imaging system 200 ( Figure 2 For ease of illustration, similar components in each of the X-ray imaging systems 100 and 200 may not be listed or shown. Each of the X-ray imaging systems 100 and 200 includes a movable tube head 109 containing a X-ray imaging energy source (such as an X-ray source) and a collimator for controlling the shape of the X-ray beam emitted by the X-ray source. As shown, the movable tube head 109 may be attached to the ceiling of the overhead pipe crane system 117, or alternatively, it may be mounted to a support column (not shown) or other rigid fixture, such as a wall mount. The tube head 109 may be rotatable about one or more axes x, y, z by means of an electronic control device connected to at least one electric motor M, and may be translatable along one or more axes x, y, z (as indicated by exemplary arrow 111). At least one electric motor M may control the translation of the tube head 109 along a horizontal axis via a ceiling mount traversing the pipe crane system 117. At least one motor M can control the translation of the tube head 109 along the vertical axis by extending and retracting the telescopic column 121. At least one motor M can control the rotation of the tube head 109 about the horizontal axis by rotating the tube head 109 about the rotatable connection between the telescopic column 121 and the tube head 109. At least one motor M can control the rotation of the tube head 109 about the vertical axis by rotating the telescopic column 121.

[0019] The movable digital X-ray imaging detector holder or filter 107 can be similarly wall-mounted or mounted on the column 105 and can be controllably moved by a motor M at least in the vertical direction 112. The filter includes a digital X-ray imaging detector 108 fixed therein in a known position relative to the boundary of the filter 107. As shown, the central X-ray 119 of the X-ray beam emitted by the X-ray source in the tube head 109 strikes the center of the filter 107 and the center of the detector 108 fixed therein. In a wall-mounted filter embodiment of the X-ray imaging system 100, as... Figure 1 As shown, a patient standing between the wall filter 107 and the tube head 109 can be imaged by exposing the patient with an X-ray source in the tube head 109 and capturing the patient's X-ray image in the DR detector 108. In such cases... Figure 2In the benchtop embodiment of the X-ray imaging system 200 shown, a table 201 is configured to support a patient lying on it so that X-ray images of the patient exposed by an X-ray source in a tube head 109 can be captured in a movable DR detector 108. The table-configured DR detector 108 can be moved via motor control in a direction 203 along and parallel to the underside of the table 201 using a motor M attached to a filter 107.

[0020] A video camera 115 is mounted on the tube head 109 and aimed in the same direction as the X-ray source so as to capture real-time video or still images of the patient positioned for X-ray imaging using the X-ray imaging systems 100, 200. As described herein, the camera 115 can also be used for spatial calibration and positioning relative to the X-ray imaging systems 100, 200.

[0021] The X-ray imaging systems 100 and 200 are controlled by a computer system 101, which may include a PC console with a processor and electronic memory for programmably controlling the operation of the X-ray imaging systems 100 and 200 (as described herein). Operator O The X-ray imaging systems 100 and 200 can be selectively controlled via input to the computer system 101 (via a keyboard and / or mouse (not shown)). Images captured by the X-ray imaging systems 100 and 200 can be displayed on a digital monitor 102 for use by the operator. O Viewing is possible. Furthermore, the status of the operating components of the X-ray imaging systems 100 and 200 can be displayed on the monitor 102. Additionally, live video or still images captured by the camera 115 can be displayed on the monitor 102. The computer system 101 can exchange control commands and data with the X-ray imaging systems 100 and 200 via a conductive communication cable 103 electrically connected to the X-ray imaging systems 100 and 200. In an alternative embodiment, the computer system 101, the overhead pipe crane system 117, and the DR detector 108 can wirelessly exchange control commands and data.

[0022] Each of the X-ray imaging systems 100 and 200 can use a three-dimensional (3D) coordinate system to record the position of the tube head 109 in three-dimensional space. Typically, xyz The coordinate system can be used by computer system 101 to identify and / or record the 3D spatial positions of tube head 109, filter 108, and camera 115. Although xyz The coordinate labels are arbitrary, but for the purposes of this description, we will refer to the vertical movement of the tube head in the X-ray imaging system 100 as along... x Axis translation; entering and leaving Figure 1Horizontal movement of the page is called along y Axial translation; and the horizontal movement in which the tube head 109 moves directly toward and away from the filter 107 is called along z Axis translation (e.g.) Figure 1 (As illustrated in the diagram). For descriptive purposes, in Figure 2 Similar markings in China xyz Coordinate systems, which may be referenced from time to time herein. An encoder in at least one motor M controlling the movement of the tube head 109 and a similar encoder in a motor M controlling the movement of the filter 107 can accurately detect, measure, and transmit the 3D position coordinates of the tube head 109 and the filter 107 to the computer system 101.

[0023] Figure 3 This is a flowchart of the method and system described herein, from which the X-ray imaging systems 100 and 200 can be programmed to recognize the camera 115 relative to the filter 107. xyz Location coordinates. Data identifying the 3D spatial positioning of camera 115 relative to the wall / desktop filter and spatial position information of the X-ray imaging systems 100, 200 are represented as an input 301 to computer system 101, which has an instruction program 307 (label-free system) installed therein for performing the label-free method as described herein. The calibration data obtained as described herein is represented as another input to the label-free system in 305. The current positioning of the X-ray imaging systems 100, 200, and the components for acquiring X-ray images of the patient are represented as another input to the label-free system 307 in 303. The information represented by 301, 303, 305 is input to and used by computer system 101 programmed as label-free system 307. xyz Coordinates are generated relative to the current 3D position 309 of camera 115 (camera extrinsic parameters) of filter 107 and patient's current 3D position 311.

[0024] Figure 4The diagram illustrates a filter 107 containing a DR detector 108 fixed therein. The location of the automatic exposure control region 401 is also depicted. One or more position markers 409 (which may include temporarily attached patterns) may be attached to the front side of the filter 107, or, if the filter is not directly accessible (such as in a desktop X-ray imaging system 200), the markers 409 may be attached to a portion of the X-ray imaging system 200 such that the markers are visible in an image of the filter 107 captured by the camera 115. The spatial position of one or more markers 409 relative to the center of the filter 107 is known. In particular, the spatial coordinates of the angles of each of the one or more markers 409 are known and may be recorded in the computer system 101. The center position of the projected crosshairs 403 is also shown and may be generated, for example, using a light source (such as collimator light) positioned in the tube head. As used herein, the term mark means any one or more of a position indicator, landmark, or visible pattern (whether permanently affixed or manufactured, temporarily adhered, or manually marked by a marking instrument), which may be easily removable or may be permanent, mechanically printed, and includes, but is not limited to, pre-designed detectable marks, pre-designed detectable patterns, or any pre-existing feature on the surface of the filter or on the boundary (such as a corner) of the filter, or printing or coloring on the filter that can be digitally identified in a digital image of the filter.

[0025] Figures 5A-5D The illustration depicts a calibration method for a wall-mounted / filter-type X-ray imaging system 100, whereby the tube head 109 is repeatedly and selectively positioned relative to the filter 107. In the basic calibration steps for the wall-mounted / filter-type X-ray imaging system 100, as follows... Figure 5A As shown, when the tube head 109 and the filter 107 are in the initial position I In the middle of the time, filter 107 ( Figure 4 The image was captured by camera 115. Initial position. I This can include a tube head 109 located at a distance of one meter (1m) 501, in a designated centered and vertical position relative to the filter 107, whereby the origin of the coordinate system of the X-ray imaging system 100 ( 0,0,0 The center of the front of the filter 107 is located. Another basic calibration step for the wall-mounted / filter X-ray imaging system 100 may include a second calibration, wherein, along the tube head 109... z The shaft was translated to a position 1.8m away between the filter 107 and the tube head 109. I 2 After that, filter 107 ( Figure 4The image of the filter 107 is captured by camera 115. The image of the filter 107 captured by camera 115 is used by computer system 101 to calculate the basic 3D coordinates of the camera in the 3D coordinate system of the X-ray imaging system 100. The method used to calculate the 3D coordinates of the camera uses the perspective n-point (PnP) algorithm described above herein. Given the known 3D coordinates of the tube head 109 (from an encoder in at least one motor M) and the known 3D coordinates of the filter 107 (from an encoder in the motor M that controls the movement of the filter), the set of corresponding coordinates of camera 115 can be calculated. This method directly obtains the coordinate mapping of the camera. The spatial position data of camera 115 (camera basic) is recorded and stored in computer system 101 along with the spatial coordinate information of tube head 109 and filter 107.

[0026] Reference X-ray imaging system 100, and reference Figure 5A In the second stage of calibration, at a distance 501 of approximately 1 m between filter 107 and tube head 109, tube head 109 is vertically translated within a range (including top position 1 and bottom position 2) to several pre-selected positions, and filter 107 is simultaneously translated within the same range (including top filter position A and bottom filter position B) as examples. Similar to the calibration process described above, images of filter 107 are captured at each pre-selected position, and spatial coordinate position information corresponding to filter 107, tube head 109, and calculated camera extrinsic parameters is recorded and stored in computer system 101 for each pre-selected position. For repeatability, for each pre-selected position, the spatial information of filter 107 and tube head 109 is obtained from an encoder connected to motor M, while the camera extrinsic parameters are calculated using the known PnP algorithm described herein.

[0027] In the second phase of calibration, the tube head 109 is moved further away from the filter 107 (e.g., at a distance of 1.8 m), and again translated between the range of top position 3 and bottom position 4 (and including top position 3 and bottom position 4). The filter 107 is also translated simultaneously and parallel to the tube head 109 by translating within the range of top filter position A and bottom filter position B. Images of the filter 107 are captured by the camera 115 at the top and bottom positions, as well as several pre-selected positions within this range. As described above, the spatial positioning information of the filter and tube head is recorded for each position and stored in the computer system 101 along with the calculated camera extrinsic parameters. The second phase of calibration continues by positioning the tube head 109 from position... I Start and along zThe axis extends further to a location 5, which may include a distance ranging from 1 m 501 (e.g., between tube tip 109 and filter 107) to 1.8 m, for example, where images of filter 107 are captured by camera 115 at every 10 cm of translation, and spatial coordinate information is similarly recorded at each pre-selected location (as described herein). It should be noted that in the calibration method described herein, the pre-selected distance interval between tube tip 109 and filter 107 can vary as desired, and the distances and locations described herein are exemplary. The distances, translation intervals, and angular intervals for the calibration steps described herein can be selected, for example, based on the expected anatomy of the patient to be imaged and its typical source-to-image distance (SID) parameter, or they can be selected according to other preferences.

[0028] The second stage of calibration of the X-ray imaging system 100 may further include capturing an image of the filter 107 while angled the tube head 109. (Go to...) Figure 5B The second-stage calibration procedure is described, in which the tube head is rotated about the y-axis at angular intervals of approximately five degrees within a range of approximately 70° to 110°, where the 90° position is the vertical position of the tube head 109 relative to the filter 107 (such as...). Figure 5A The filter 107 remains at its initial filter position I. For example, the image of filter 107 is captured by camera 115 in five-degree increments, and spatial coordinate information is similarly recorded (as described herein); however, in this example, the spatial coordinate information includes translation. xyz Coordinates and angular coordinates relative to camera extrinsic parameters. Go to... Figure 5C The second-stage calibration step is described, in which the tube head 109 is rotated downwards by approximately fifteen degrees from the vertical direction, and while maintaining this downward fifteen-degree rotation, the tube head is then vertically translated to several positions between a selected top position and a selected bottom position (as described herein). The filter 107 is positioned to appear in the center of the camera view and is moved vertically simultaneously and parallel to the tube head 109. Images of the filter 107 are captured by the camera 115 at several selected positions between the selected top and bottom positions, and spatial coordinate information is similarly recorded (as described herein), but in this example, the spatial coordinate information includes translation. xyz Coordinates and parallel to xz The coordinates are 15 degrees downwards from the plane. Go to... Figure 5DThe second-stage calibration step is described, in which the tube head 109 is rotated upwards by approximately fifteen degrees from the vertical direction, and while maintaining this upward rotation, the tube head is then vertically translated to several positions between a selected top position and a selected bottom position (as described herein). The filter 107 is positioned to appear in the center of the camera view and moves vertically simultaneously and parallel to the tube head 109. Images of the filter 107 are captured by the camera 115 at several selected positions between the selected top and bottom positions, and spatial coordinate information is similarly recorded (as described herein), but in this example, the spatial coordinate information includes translation. xyz Coordinates and parallel to xz The 15-degree upward angular coordinates of the plane.

[0029] refer to Figure 6 In the basic calibration steps for the benchtop / filter X-ray imaging system 200, when the tube head 109 and filter 107 are in the initial position... I In the middle of the time, filter 107 ( Figure 4 The image was captured by camera 115. Initial position. I This can include the tube head 109 at a distance of one meter (1m) 601 relative to the designated center position of the filter 107, whereby the origin of the coordinate system of the X-ray imaging system 200 ( 0,0,0 The filter 107 is located at the center of the front part of the filter 107. The image of the filter 107 captured by the camera 115 is used by the computer system 101 (a markerless system) to calculate the 3D spatial coordinates of the camera in the 3D coordinate system of the X-ray imaging system 200 (using the PnP algorithm described herein). The spatial coordinate position data of the camera 115 (camera basic), as well as the spatial coordinate information of the tube head 109 and the spatial coordinate information of the filter 107, are recorded and stored in the computer system 101.

[0030] Continue to refer to Figure 6 In the second-stage calibration of the X-ray imaging system 200, the tube head 109 is moved to several pre-selected positions between the leftmost position 1 and the rightmost position 2 (and including the leftmost position 1 and the rightmost position 2), while the filter 107 is moved simultaneously and parallel to the tube head 109 between the leftmost position A and the rightmost position B, respectively. The image of the filter 107 is captured again at each pre-selected position, and the same spatial coordinate information corresponding to the camera's external parameters, the tube head 109, and the filter 107 is recorded and stored in the computer system 101. A similar second-stage calibration process (as described above with respect to the X-ray imaging system 100) continues to angle the tube head 109 (e.g., taking angular positions 3 and 4 of the tube head 109 as examples).

[0031] While the calibration process described herein uses examples of distances such as 1 m and 1.8 m, and angles ranging from 70° to 110°, these examples illustrate the spatial nature of the calibration process, which may involve greater distances, ranging from approximately 0.4 m to approximately 2 m. Similarly, angled processes can range from approximately 45° to approximately 135° and be calibrated in various angular increments. This process can create a dense, markerless calibration manifold (MCM) from which the intermediate position of camera 115 can be interpolated, wherein the manifold describes a high-dimensional space of many parameters captured and representing the system's parameters. Performing and recording such a calibration process enables the X-ray imaging systems 100, 200 to accurately predict the position of the system in its 3D operating space, regardless of where the filter 107, tube head 109, and camera 115 are placed. This capability is crucial because when a radiographer or technician positions the filter 107 and tube tip 109 for imaging, their positions and angles may differ from any of the specific positions used during the calibration process. By performing the calibration process as described herein and specifying the position and orientation of the camera in the MCM, it is possible to correctly recover the extrinsic parameters of the camera 115 and thus correlate the image coordinates with world coordinates.

[0032] During normal operation of the X-ray imaging systems 100 and 200, mechanical components undergo wear over time. This can eventually lead to malfunction of the system, and its movement may become distorted beyond acceptable thresholds. Timely detection of changes in system performance can be advantageous. In one embodiment, spatial coordinate position data collected during basic and second-stage calibration can be stored and used as a baseline. At a later time, basic and second-stage calibration data can be collected again and compared with the earlier stored data to determine whether any mechanical deviations and offsets have occurred in the X-ray imaging systems 100 and 200. The recorded data may include, but is not limited to, imaging data, external data, marked spatial coordinates, system coordinate data, and any other system characteristics and states recorded as part of the system's normal operation. In one embodiment, mechanical deviations and offsets can be identified as the difference between the initial state after setup (data collected as the system baseline) and its state after a period of use. In one embodiment, to obtain measurements of deviations and offsets, the wall / desktop filter 107 and tube head 109 can be moved to the same positions used during the initial basic and second-stage calibrations to collect basic and second-stage calibration data. The same data (including, but not limited to, imaging data, camera extrinsic parameters, spatial coordinates of the markers, and system coordinates for each selected location) can be recorded during the movement. In one embodiment, the location of the markers in the newly captured image can be predicted by using the corresponding camera extrinsic parameters previously stored in the initial calibration step, or the location of the markers can be detected directly in the newly captured image. The difference between the location of the markers and the camera extrinsic parameters can be used to represent and quantify any mechanical deviations and offsets in the DR imaging system. In one embodiment, the location can be recorded during the translation of the tube head 109 along the z-axis as described above (starting from position). I And along z The initial spatial coordinate data obtained by extending the axis linearly away from the filter 107 to position 5) is compared with the images of the filter 107 captured by the camera 115 at 10cm translation intervals. This is used to compare the images of the filter 107 obtained from position 5. I Start and along z The axis is extended further to position 5 to collect similar data. A second calibration performed later can determine the spatial coordinates of the newly collected data. x The dimensional deviation has exceeded an acceptable threshold, possibly due to wear and tear on the ceiling mounts of the tubular crane system 117, which have sagged over time, causing friction with the surrounding structure. zAn unacceptable deviation from the true linear path of the axis. This deviation information can be used, but is not limited to, to trigger audible and / or visual warning notifications to indicate a deterioration in the availability and reliability of the X-ray imaging systems 100 and 200. This information can therefore be used to assist service engineers in maintaining and repairing the X-ray imaging systems 100 and 200.

[0033] In one embodiment, calibration datasets collected at different time points can be implemented to quantify and calculate the deviation trend of the DR mechanical system. As described above, differences in camera intrinsic parameters and camera extrinsic parameters, as well as the location of the markers, can be used to represent and quantify any mechanical deviations and offsets between any two sets of calibration data. The deviation trend can be estimated by quantifying and comparing the detected deviations and offsets over time between each calibration dataset. In one embodiment, the deviation trend can be calculated along tube head 109 as described above. z Translation of the axis during (starting from position) I And along z The initial spatial coordinate data obtained by extending the axis linearly away from filter 107 to position 5 is compared with images of filter 107 captured by camera 115 at 10 cm translation intervals. Calibration can be performed multiple times after different time periods to start from position 5. I And along z The axis is extended further along until position 5 to collect similar data. The differences between these calibration datasets can determine the collection of spatial coordinates. x The increasing trend in deviation and offset is evident in the dimensions, likely due to wear on the ceiling mounts of the tubular crane system 117, which have sagged over time. This could indicate further deterioration along the lines in the near future. z Unacceptable deviations from the true linear path of the axis. This deviation trend information can be used, but is not limited to, to trigger early warning notifications to indicate a deteriorating trend in the availability and reliability of X-ray imaging systems 100 and 200. Therefore, this information can be used to help service engineers monitor X-ray imaging systems 100 and 200 and establish daily maintenance and repair schedules.

[0034] As described herein, in one embodiment, the camera module is mounted on the DR system. A primary calibration is performed to identify the basic spatial relationships between the wall / desktop filter, the X-ray tube, and the camera module. Errors introduced by mechanical movement and mounting are compensated for in this step to improve tracking and positioning accuracy. A markerless motion model is developed to update and locate the spatial position by comparing the basic coordinate values ​​recorded during the primary calibration with those recorded after relative movement. The camera module can also be used to acquire and analyze the patient's body features, such as body size and key body points. The patient's spatial position can be located by converting positional information from imaging coordinates to DR system coordinates using the markerless motion model. The application of spatial information and body features enables automated positioning and pose recognition, thereby assisting in the operation of the X-ray imaging system.

[0035] Using a single calibration, particularly the camera module calibration embodiment of this disclosure, the basic spatial position between the wall / desktop filter and the X-ray tube can be identified using only a single RGB image, significantly reducing service workload. In particular, utilizing a single calibration, the mechanical calibration embodiment of this disclosure can compensate for errors caused by the mechanical movement and rotation of the DR system (including slide rail translation and X-ray tube head rotation), enabling the disclosed solution to achieve high accuracy in various clinical scenarios.

[0036] In one embodiment using a label-free motion model, the DR imaging system will be able to track the spatial relationship between the DR system and the target patient in real time without additional calibration or position marker detection, making the disclosed solution applicable to a wider range of clinical scenarios and patient populations. Using statistical methods and detection of patient height and width, embodiments of the invention can be used to calculate the appropriate positions of wall / desktop filters and X-ray tubes based on the specific patient and body part. The system can automatically control the movement of the wall / desktop filters and X-ray tubes toward the correct positions.

[0037] Using machine learning-based methods, embodiments of the present invention can be used to identify and classify patient postures according to recommended radiographic positioning criteria. These results can be presented and displayed to the user. Using augmented reality technology, embodiments of the present invention can be able to render and display the target patient position, wall / desktop detector and AEC position, and posture determination results in real time, making the X-ray image acquisition process more intuitive.

[0038] In one embodiment, the visual tracking and positioning assistance system includes a camera module, a system calibration module, a feature analysis module, a motion control module, and an augmented reality module. The camera module collects image data for several functional modules. The system calibration module provides primary calibration functionality to determine basic spatial relationship data between the wall / desktop filter, X-ray tube, and camera module, and to compensate for errors caused by mechanical movement and installation. The feature analysis module can extract target information from the acquired image data, including but not limited to patient height, width, body joints, and the relative position of the patient to the wall / desktop filter, and then applies statistical and machine learning methods to estimate motion control parameters and output positioning guidance information. The motion control module can use the motion parameters to move the wall / desktop filter and X-ray tube to desired positions. The augmented reality module can be applied to display any desired information, including but not limited to wall / filter boundary positions, AEC positions, collimator light boundaries, and positioning guidance information. In one embodiment, the augmented reality module also provides augmented reality control functionality, whereby the user can control the DR system through various multi-touch gestures, such as swiping, squeezing, scrolling, and zooming. Control functions include, but are not limited to, collimator beam size adjustment, and wall / desktop filter and tube position correction.

[0039] In one embodiment, the camera module may consist of at least one 2D and / or 3D camera, or multiple 2D cameras and multiple 3D depth cameras. The camera module may be mounted at, but is not limited to, the top / bottom of the X-ray tube head, the right / left side of the X-ray tube head, or the right / left side of a wall-mounted filter (alone or in combination). In one embodiment, the camera module may collect imaging data, wherein the acquired information includes, but is not limited to, marker positions, wall / desktop filter positions, and patient body positions. In one embodiment, a camera module combined with a sound acquisition module may be used to collect both imaging and sound data. In one embodiment, the calibration module may include camera calibration and mechanical calibration functions. A single calibration may be performed manually or semi-automatically. In one embodiment, the calibration module may obtain information from the camera module and then perform calibration using position markers in the image. In one embodiment, the feature analysis module may include a body feature extraction module, a motion parameter calculation module, and a posture determination module. In one embodiment, the body features extracted by the feature analysis module may include body height, body width, body contour, and body key points. In one embodiment, motion parameters calculated by the feature analysis module using a disclosed markerless motion model may be passed to the motion control module.

[0040] In one embodiment, an exemplary DR imaging system can be assembled such that a 3D depth camera module is mounted on top of a collimator included in the tube head. The camera module can be connected to a PC computer system and configured to perform camera calibration and mechanical calibration sequentially via a system calibration module to initialize the camera module. A mixed reality device can be connected to the PC, and guidance and assistance information can then be projected into the real world via the augmented reality device. The workflow may include: inputting patient and examination information on the PC; moving the patient to a waiting area and having them stand still; the camera module capturing image data of the patient standing in the waiting area; the patient moving to a wall-mounted or tabletop filter; the camera module capturing image data of the target patient standing in front of the wall-mounted filter or lying on the tabletop filter; the PC using the collected image data to extract and analyze the patient's body features through the feature analysis module; the PC using the feature analysis module to calculate the desired motion parameters based on the extracted features and current system coordinates; the PC using the desired motion parameters to control the wall-mounted / tabletop filter and X-ray tube to move to the desired position with a specific tube posture; the camera module re-collecting image data and then passing the posture judgment results to the feature analysis module; and guidance and assistance information then being output to the augmented reality module.

[0041] Mixed reality devices will be used to display guidance and assistance information, such as detector boundaries and AEC boundaries, and posture judgment results, such as centering error, hand posture error, and rotation error. Failed positioning rules and their corresponding error values ​​will also be displayed. The user can adjust the positioning results based on the information above, or allow exposure if the positioning is acceptable. Once the patient moves or the user adjusts the positioning, the system can repeat any of the steps described until the positioning is acceptable.

[0042] As will be appreciated by those skilled in the art, aspects of the present invention can be implemented as systems, methods, or computer program products. Therefore, aspects of the present invention can take the form of hardware and software embodiments (including firmware, resident software, microcode, etc.) or embodiments combining software and hardware aspects, which herein may generally be referred to as “services,” “circuits,” “circuit modules,” “modules,” and / or “systems.”

[0043] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples (a non-exclusive list) of computer-readable storage media will include the following: an electrical connection having one or more wires, a portable computer floppy disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium can be any tangible medium that can contain or store a program used by or in conjunction with an instruction execution system, device, or apparatus.

[0044] Program code and / or executable instructions implemented on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, or any suitable combination thereof.

[0045] Computer program code used to perform the operations of various aspects of this invention can be written in any combination of one or more programming languages ​​(such as Java, Python, C++, or the like) and conventional procedural programming languages ​​(such as the "C" programming language or similar programming languages). The program code can execute entirely on the user's computer (device), partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet provided by an Internet service provider).

[0046] Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create components for implementing the functions / actions specified in the flowchart and / or block diagram blocks or blocks.

[0047] These computer program instructions may also be stored in a computer-readable medium that can instruct a computer, other programmable data processing device or other apparatus to operate in a particular manner, such that the instructions stored in the computer-readable medium produce an article of manufacture including the instructions, which implement the functions / actions specified in flowchart and / or block diagram boxes or blocks.

[0048] Computer program instructions may also be loaded onto a computer, other programmable data processing apparatus or otherwise, to cause a series of operational steps to be performed on the computer, other programmable apparatus or other apparatus, thereby producing a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide for implementing the functions / actions specified in the flowchart and / or block diagram boxes or blocks.

[0049] This written description uses examples to disclose the invention, including the best mode, and also enables those skilled in the art to practice the invention, including making and using any device or system and performing any incorporated methods. The patentable scope of the invention is defined by the claims, but may include other examples that would occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if such other examples include equivalent structural elements that are not substantially different from the literal language of the claims.

Claims

1. A method of operating a radiographic imaging system having a movable tube head and a movable detector holder, the method comprising: Attach the camera to the movable tube head; Detect two or more marks on the movable detector bracket; The camera is used to capture multiple digital images of the movable detector bracket and the two or more markers, each of the digital images being captured at a different relative position between the movable tube head and the movable detector bracket; For each of the plurality of captured digital images, a first set of spatial coordinates of the camera in three-dimensional space is determined; as well as Determine and electronically store in three-dimensional space at least one function that best fits the determined spatial coordinates of multiple cameras.

2. The method of claim 1, further comprising: The two or more markers are placed at known locations on the movable detector bracket relative to the location of the digital X-ray imaging detector contained within the movable detector bracket.

3. The method as described in claim 2, wherein, The step of determining the first set of spatial coordinates of the camera includes detecting the two or more markers in each of the plurality of captured digital images.

4. The method of claim 3 further includes specifying a point on the movable detector bracket to define the origin of the three-dimensional spatial coordinates.

5. The method of claim 4, further comprising designating the center of the detector bracket as the origin of the three-dimensional spatial coordinates.

6. The method of claim 1, further comprising determining and electronically storing in three-dimensional space at least one polynomial function that best fits the determined plurality of camera spatial coordinates.

7. The method of claim 1, further comprising: Position the tube head for digital X-ray imaging; as well as The stored functions are used to determine the extrinsic parameters of the camera in three-dimensional space.

8. The method of claim 7, further comprising using the extrinsic parameters of the camera to determine the spatial coordinates of a point on the movable detector bracket.

9. The method of claim 1, further comprising: Repeatedly capture multiple digital images and determine spatial coordinates; as well as The first set of spatial coordinates is compared with the second set of spatial coordinates obtained through repeated steps to determine the level of deviation between the first set of spatial coordinates and the second set of spatial coordinates.

10. The method of claim 9, further comprising: The system stores the determined deviation levels and transmits a notification signal when the determined deviation level exceeds a predetermined threshold.

11. A method of operating a radiographic imaging system, the radiographic imaging system comprising a movable tube head, a movable detector holder having one or more markings thereon, and a camera, the method comprising: The camera is used to capture a first plurality of digital images of the movable detector bracket and the one or more markers, each of the digital images being captured at a different relative position between the camera and the movable detector bracket; Determine a first plurality of spatial poses of the cameras, each corresponding to one of the first plurality of captured digital images; as well as Determine and electronically store at least one function that best fits the first plurality of spatial poses of the determined camera.

12. The method of claim 11, further comprising: The camera is used to capture a second plurality of digital images of the movable detector bracket and the one or more markers, each of the second plurality of digital images being captured at a different relative position between the camera and the movable detector bracket; Determine the second plurality of spatial poses of the cameras, each corresponding to one of the second plurality of captured digital images; as well as The first plurality of spatial poses are compared with the second plurality of spatial poses to determine the magnitude or rate of change of the deviation between the first plurality of spatial poses and the second plurality of spatial poses.

13. The method of claim 12, further comprising: The system stores the magnitude or rate of change of the determined deviation and transmits a notification signal when the magnitude or rate of change of the determined deviation exceeds a predetermined threshold.

14. The method of claim 13, further comprising: In response to transmitting the notification signal, an audible or visual notification device is activated.

15. The method of claim 12, further comprising: Calculate the trend of the deviation; as well as Based on the calculated trend, the expected time for any deviation in the deviation to exceed a predetermined threshold is calculated.

Images