Tracking and positioning for digital radiography

Abstract

A radiographic imaging system includes a movable tube head, a camera and a movable detector holder. A marker is disposed on the detector holder and a plurality of digital images of the detector holder and the marker are captured by the camera. Each of the digital images are captured at a different relative position as between the camera and the movable detector holder. Spatial coordinates of the camera are determined for each of the plurality of captured digital images and a mathematical best fit function for the spatial coordinates is calculated and stored.

Description

TRACKING AND POSITIONING FOR DIGITAL RADIOGRAPHYBACKGROUND OF THE INVENTION

[0001] The subject matter disclosed herein relates to digital radiography (DR). In particular, to systems and methods for visual tracking and positioning of digital radiographic imaging systems. *

[0002] In DR imaging, accurate patient positioning has always been essential for radiologic technologists, which is also the key for generating qualified radiographic images. The positioning process depends highly on the clinical experience of the technologist. Typical reference lines on a bucky, such as the detector and AEC boundary line, or the crosshair laser lines projected from an overhead tube head can be applied to assist the positioning process. Once the patient is positioned over the bucky, some reference lines may no longer be clearly visible and it may be difficult for the technologist to judge if the patient positioning with respect to the bucky is acceptable. When the patient lays on a table, there may be even fewer visual positioning references available. Even a well-trained technologist may arrange an improper positioning due to a lack of sight references for comparison, or due to complex placement requirements. In these cases, the generated images may fail to meet diagnostic requirements.

[0003] Based on current general DR systems, manual positioning is dependent on experience and skill. There is room to improve the accuracy of positioning to reduce the failure rate. Therefore, a visual tracking and positioning assist system and method, that provides visible guidance and the functionality for tracking and locating the spatial position and distance between the DR system and patient, can not only facilitate the positioning process but also empower other smart and automatic features.

[0004] Previously developed systems have included real-time location marker detection and body feature analysis by using a 3D depth camera. Such systems provide the function of automatic positioning to assist the x-ray projection radiography process. One disadvantage of some previous solutions, that focus on thewalk-stand Chest AP / PA and lateral radiograph, is that the technique cannot support radiography using a table or radiography when the x-ray tube head uses angulation due to its limitation on the method of tracking and locating the spatial information between DR system and patient. This limitation also makes the method difficult to extend to all co mon clinical scenarios.

[0005] Another disadvantage of some previous solutions based on real-time location marker detection is that the technique will no longer be valid once the location landmarks are covered by a patient or fail to be detected by the system due to image noise, distortion or blurring. This leads to difficulty in supporting imaging of various body types and body parts. Another disadvantage of some previous solutions is that accuracy is highly affected by the errors caused by depth measurement, camera installation and DR system mechanical movement, which may be difficult to eliminate or reduce under an established framework.

[0006] One previous solution, as described in US Patent No. 9,895,131, discloses a system for x-ray tube scanner automatic control using a 3D camera. The automatic control is realized by performing real-time calibration using an RGB image between table and x-ray tube. Due to the calibration method disclosed therein, the solution has limited application scenarios and is easily affected by the factors arising from the mechanical aspects and from the patient side.

[0007] The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE INVENTION

[0008] A radiographic imaging system includes a movable tube head, a camera, and a movable detector holder. A location landmark, or marker, is disposed on the detector holder and a plurality of digital images of the detector holder and the marker are captured by the camera. Each of the digital images are captured at a different relative position as between the movable tube head and the camera. Spatial coordinates of the camera are determined for each of the plurality of captured digitalimages and a best fit function is calculated and stored, such as a polynomial function, which can then be used to determine spatial coordinates of other camera positions.

[0009] In one embodiment, a method of operating a radiographic imaging system having a camera, attached to a movable tube head, and a movable detector holder is disclosed. Markers are attached to, or positioned on, the movable detector holder and a plurality of digital images of the movable detector holder and the markers are captured using the camera. Spatial coordinates of the camera are determined for each of the plurality of captured digital images, and. a function, such as a polynomial function, that best fits the plurality of camera spatial coordinates is stored.

[0010] The present invention is directed to tracking and locating the spatial position between the DR system and patient and / or bucky by using a proposed marker-less movement model and key point detection, which provides information and automatic guidance to assist the positioning operation of the DR system. This technique may solve most of the problems, described above, of previous solutions. In one embodiment, augmented reality technology is employed to provide real-time visual guidance to make the workflow compatible with current clinical applications. A visual tracking and positioning assistant system is introduced to overcome the disadvantages described herein. The disclosed system tracks and locates the spatial position between wall-bucky, table, x-ray tube and patient by using the proposed marker-less movement model and key point detection. This system offers both manual and automatic positioning functionality, in which augmented reality technology is implemented to visually display positioning information for the user, such as, but not limited to, detector boundary, AEC boundary, and collimator light boundary.

[0011] In one embodiment, markers may be installed on the bucky whereby comers of the markers have a fixed known distance from a designated origin point of a world coordinate system and from the digital radiographic detector which is secured in the bucky. In one embodiment, the origin of the world coordinate system may be the center of the bucky. To locate the detector, a digital video camera is used tolocate at least two markers in the transmitted camera image. Typical marker-based solutions cannot be applied to cases where the markers are either fully covered or the markers are not available (e.g., table exams). To estimate the location of markers in video camera images when the markers are occluded by a patient, a camera projection model can be used. Based on a camera projection model, world coordinates, or DR system coordinates, can be linked to pixel coordinates in the video image using a matrix multiplication. One popular method to determine extrinsic parameters is referred to as a Perspective-n-Point (PnP) algorithm. PnP addresses the problem of estimating the 3D spatial pose, which includes three dimensional spatial coordinates and orientation, i.e., three dimensional angular orientation (tilt, pan and roll), of a calibrated camera, altogether these may also be referred to as the camera extrinsic parameters, given a set of n 3D points in the world, i.e., markers on the bucky, their corresponding 2D projections in the image, and the camera intrinsic. Given a set of correspondence data, such as location coordinates of the bucky and of the tube head where the camera is mounted, rotation R coordinates and xyz translation T coordinates can be calculated. This method solves the mapping between camera image coordinates and the camera location coordinates directly. In practice, camera intrinsic parameters such as focal length and principal points are innate properties of the camera and do not change. Camera extrinsic parameters, including xyz location coordinates, may vary with changing rotation angle and translation and will also depend on construction of the world coordinate system, i.e., on the DR system coordinates.

[0012] As described herein, a video camera is mounted on a movable tube head. When the tube head is moved, the camera's extrinsic parameters are changed. An adjustment of the camera's current extrinsic parameters is required and a method is described herein for adjusting the camera's current extrinsic parameters without requiring the detection of a bucky marker in the camera image. The camera's extrinsic parameters are updated based on movement of the tube head and the camera projection model disclosed herein. Tube head movement xyz coordinates are transmitted by encoders in the DR imaging system as well as the bucky coordinates using encoders in the bucky system, which coordinates are based on a position of thecenter of the bucky. In one embodiment, the calibration workflow uses tube head and wall stand movement in auto tracking mode. The world coordinate system is based on the center of the wall stand bucky (0,0,0, origin coordinates), and the world coordinates of the comers of the markers are determined based on their known position with respect to the center of the bucky. The camera captures a visible image of the markers, then the markers .in the image are detected per computer program and a correspondence between 2D and 3D points are built. The camera basic extrinsic for this spatial relation of wall bucky and tube head is established and is denoted as Rbasic and T basic-

[0013] In one embodiment, the tube head may have a translation movement in the y direction denoted as trans_y relative to the center of the wall stand bucky. The new camera extrinsic parameters can be calculated as(Carrier ax \ / Worldx\ / Tbasic — x \Cameray j = Rbasic I Worldy j + Tnew, where Tnew = I Tbasic — y + &trans_y I Cameras \Worldz ' \ Tbasic — z / where the trans_y is acquired from mechanical encoder feed back of the DR system. Similar transformations with respect to rotation of the camera may be also computed.

[0014] In one embodiment, based on the camera extrinsic parameters calculated from the spatial relation of the wall bucky and the tube head described herein, locations of markers can be estimated using a camera projection model and the location of the detector can be derived from the location of the marker(s). In one embodiment, the translation value when updating the camera extrinsic parameters can be compensated by a line fitting method. As described herein, the line fitting method is calculated by acquiring multiple Rbasic and Tbasic sample data points when wall-stand and tube head are in different predefined spatial relationships.

[0015] The summary descriptions above are not meant to describe individual separate embodiments whose elements are not interchangeable. In fact, many of the elements described as related to a particular embodiment can be used together with, and possibly interchanged with, elements of other described embodiments. Many changes and modifications may be made within the scope of the present inventionwithout departing from the spirit thereof, and the invention includes all such modifications.

[0016] This brief description of the invention is intended only to provide a brief overview of subject matter disclosed herein according to one or more illustrative embodiments, and does not serve as a guide to interpreting the claims or to define or limit the scope of the invention, which is defined only by the appended claims. This brief description is provided to introduce an illustrative selection of concepts in a simplified form that are further described below in the detailed description. This brief description is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.BRIEF DESCRIPTION OF THE DRAWINGS

[0017] So that the manner in which the features of the invention can be understood, a detailed description of the invention may be had by reference to certain embodiments, some of which are illustrated in the accompanying drawings. It is to be noted, however, that the drawings illustrate only certain embodiments of this invention and are therefore not to be considered limiting of its scope, for the scope of the'invention encompasses other equally effective embodiments. The drawings below are intended to be drawn neither to any precise scale with respect to relative size, angular relationship, relative position, or timing relationship, nor to any combinational relationship with respect to interchangeability, substitution, or representation of a required implementation., emphasis generally being placed upon illustrating the features of certain embodiments of the invention. In the drawings, like numerals are used to indicate like parts throughout the various views. Thus, for further understanding of the invention, reference can be made to the following detailed description, read in connection with the drawings in which:

[0018] FIG. 1 is a schematic diagram of an exemplary wall bucky type of radiographic imaging system;

[0019] FIG. 2 is a schematic diagram of an exemplary table type of radiographic imaging system;

[0020] FIG. 3 is a flow diagram of an exemplary initialization for a markerless method and system;

[0021] FIG. 4 is a diagram of an exemplary bucky with markers and DR detector;

[0022] FIGs. 5A-5D are schematic diagrams of an exemplary calibration method for the exemplary wall bucky type of radiographic imaging system; and

[0023] FIG. 6 is a schematic diagram of an exemplary calibration method for the exemplary table type of radiographic imaging system;DETAILED DESCRIPTION

[0024] With reference to FIG. 1 and FIG. 2 there are depicted two embodiments of a radiographic imaging system including a vertical wall or column radiographic imaging system 100 (FIG. 1) and a horizontal table based radiographic imaging system 200 (FIG. 2). Similar components in each of the radiographic imaging systems 100, 200, may not be enumerated or displayed for ease of illustration. The radiographic imaging systems 100, 200, each include a movable tube head 109 which contains a radiographic energy source, such as an x-ray source, and a collimator for controlling a shape of an x-ray beam emitted by the x-ray source. The movable tube head 109 may be attached to a ceiling mounted over-head tube crane system 117, as shown, or, alternatively, it may be mounted to a support column (not shown) or other rigid fixture, such as a wall mount. The tube head 109 may be rotatable about one or more axes x, y, z, and translatable along one or more axes, x, y, z, as shown by exemplary arrows 111 , by means of electronic control connected to at least one electric motor M. The at least one electric motor M may control translation of the tube head 109 along a horizontal axis by traversing a ceiling mount of tube crane system 117. The at least one electric motor M may control translation of the tube head 109 along a vertical axis by extending and retracting a telescoping column121. The at least one electric motor M may control rotation of the tube head 109 about a horizontal axis by rotating the tube head 109 about a rotatable connection between the telescoping column 121 and the tube head 109. The at least one electric motor M may control rotation of the tube head 109 about a vertical axis by rotating the telescoping column 121.

[0025] A movable digital radiographic detector holder, or bucky, 107 may be similarly wall mounted or mounted on a column 105 and controllably movable by an electric motor M at least in the vertical directions 112. The bucky includes a digital radiographic detector 108 secured therein in a known position relative to borders of the bucky 107. As shown, a central x-ray 119 of an x-ray beam emitted by an x-ray source in tube head 109 impacts a center of the bucky 107 and a center of the detector 108 secured therein. In the wall bucky embodiment of radiographic imaging system 100, as shown in FIG. 1, a patient standing between the wall bucky 107 and the tube head 109 may be radiographically imaged by exposing the patient using an x-ray source in the tube head 109 and capturing a radiographic image of the patient in DR detector 108. In the table embodiment of radiographic imaging system 200 as shown in Fig. 2, a table 201 is configured to support a patient lying thereon in order to capture a radiographic image of the patient in movable DR detector 108 exposed by the x-ray source in tube head 109. The DR detector 108 of the table configuration may be moved along and parallel to the underside of table 201 in directions 203 by motor control using motor M attached to the bucky 107.

[0026] A video camera 115 is installed on the tube head 109 aimed in the same direction as the x-ray source in order to capture a live video image, or a still image, of a patient positioned for radiographic imaging using the radiographic imaging systems 100, 200. As described herein, the camera 115 may also be used for spatial calibration and location relative to the radiographic imaging systems 100, 200.

[0027] The radiographic imaging systems 100, 200, are controlled by a computer system 101, which may include a PC console having a processor and electronic memory for programmably controlling operation, as described herein, of the radiographic imaging systems 100, 200. An operator O may selectively controlthe radiographic imaging systems 100, 200, via inputs to the computer system 101 via keyboard and / or mouse (not shown). Images captured by radiographic imaging systems 100, 200, may be displayed on a digital monitor 102 for viewing by operator O. In addition, status of operational components of the radiographic imaging systems 100, 200, may be displayed on monitor 102. In addition, a live video or still image captured by camera 115 may be displayed on monitor 102. Computer system 101 may exchange control instructions and data with radiographic imaging systems 100, 200, via a conductive communication cable 103 electrically connected to radiographic imaging systems 100, 200. In an alternative embodiment, the computer system 101, as well as the over-head tube crane system 117 and DR detector 108, may exchange control instructions and data wirelessly 104.

[0028] Each of the radiographic imaging systems 100, 200, may use a three- dimensional (3D) coordinate system for recording a position of the tube head 109 in three-dimensional space. Typically, an xyz coordinate system may be used by the computer system 101 to identify and / or record a 3D spatial position of the tube head 109, the bucky 108 and the camera 115. Although the xyz coordinate labeling is arbitrary, for purposes of description herein we will refer to a vertical movement of the tube head in radiographic imaging system 100 as translating along an x axis; a horizontal movement into and out of the page of FIG. 1 as translating along ay axis; and a horizontal movement wherein the tube head 109 moves directly toward and away from the bucky 107, as translating along a z axis, as illustrated in FIG. 1. The xyz coordinate system is similarly identified, for purposes of description, in FIG. 2, which coordinate systems may be referenced from time to time herein. An encoder in the at least one motor M controlling movement of the tube head 109, and a similar encoder in the motor M controlling movement of the bucky 107 may accurately detect, measure and transmit 3D location coordinates of the tube head 109 and the bucky 107 to computer system 101.

[0029] FIG. 3 is a flow chart of the method and system described herein whereby radiographic imaging systems 100, 200, may be programmed to identify xyz location coordinates of the camera 115 relative to the bucky 107. Data identifying 3D spatial positioning of the camera 115 relative to the wall / table bucky, and the spatialposition information of the radiographic imaging systems 100, 200, are represented as one input 301 into the computer system 101 having installed therein a program of instructions 307 for performing the markerless method as described herein (Markerless System). Calibration data obtained as described herein is represented in 305 as another input into the Markerless System. Current positioning of the radiographic imaging system 100, 200, components for obtaining a radiographic patient image is represented in 303 as another input into the Markerless System 307. The information represented by 301, 303, 305, is input to, and used by, computer system 101, programmed as the Markerless System 307, which generates a current 3D position 309 of the camera 115 (camera extrinsic parameters), using xyz coordinates, relative to the bucky 107 and the patient current 3D position 311.

[0030] FIG. 4 illustrates a bucky 107, which contains a DR detector 108 secured therein. Locations of automatic exposure control regions 401 are also depicted. One or more location markers 409, which may comprise temporarily attached patterns, may be attached on a front side of the bucky 107 or, if the bucky is not directly accessible such as in the table radiographic imaging system 200, the markers 409 may be attached to a portion of the radiographic imaging system 200 such that the marker is visible in an image of the bucky 107 as captured by the camera 115. The spatial positions of the one or more markers 409 relative to a center of the bucky 107 are known. In particular, spatial coordinates of a comer of each of the one or more markers 409 are known and may be recorded in the computer system 101. A centralized position of projected cross-hairs 403 are also shown, and may be generated using a light source positioned in the tube head, such as a collimator light, for example. As used herein, the term marker refers to any one or more of a location indicator, a landmark, or a visible pattern whether permanently affixed or manufactured, temporarily adhered or manually marked with a marking instrument, which marking may be easily removed or it may be permanent, mechanically printed, and including, but not limited to, a pre-designed detectable marker, pre-designed detectable pattern, or any preexisting feature on a surface of the bucky, or a border of the bucky, such as a comer, or printing or coloring, on the bucky that may be digitally recognized in a digital image of the bucky.

[0031] FIG. 5 A - FIG. 5D illustrate a calibration method for radiographic imaging systems 100, whereby the tube head 109 is repeatedly selectively positioned relative to bucky 107. In a basic calibration step for wall / bucky radiographic imaging system 100, as shown in FIG. 5 A, an image of the bucky 107 (FIG. 4) is captured by the camera 115 when the tube head 109 and the bucky 107 are in initial positions I. Initial positions / may include the tube head 109 in a designated centered and perpendicular position relative to the bucky 107 at a distance of one meter (lm) 501, whereby an origin 0,0,0) of the radiographic imaging system 100 coordinate system is located in the center of the front of the bucky 107. Another basic calibration step for wall / bucky radiographic imaging system 100 may include a second calibration wherein an image of the bucky 107 (FIG. 4) is captured by the camera 115 after the tube head 109 is translated along the z axis to position / ?, at a distance of 1.8m between the bucky 107 and tube head 109. The images of the bucky 107 as captured by the camera 115 is used by the computer system 101 to calculate basic 3D coordinates of the camera in the 3D coordinate system of the radiographic imaging system 100. The method to calculate the 3D coordinates of the camera uses the Perspective-n-Point (PnP) algorithm described herein above. Given the known 3D coordinates of the tube head 109 (from the encoder in the at least one motor M) and the bucky 107 (from the encoder in the motor M controlling bucky movement), a set of corresponding coordinates of the camera 115 can be calculated. This method obtains the coordinate mapping of the camera directly. The spatial location data of camera 115 (camera basic) is recorded and stored in the computer system 101 together with the spatial coordinate information of the tube head 109 and the spatial coordinate information of the bucky 107.

[0032] Referring to radiographic imaging system 100, and in reference to FIG. 5 A, in a second phase of calibration, the tube head 109 is vertically translated to several preselected positions within, and including, the range exemplified by top position 1 and bottom position 2, at a distance of about lm 501 between bucky 107 and tube head 109, and the bucky 107 is simultaneously translated within, and including, the same range exemplified by top bucky position A and bottom bucky position B. Similar to the calibration procedure described above, an image of thebucky 107 is captured at each of the preselected positions and the spatial coordinate position information corresponding to the bucky 107, the tube head 109 and the calculated camera extrinsic parameters are recorded and stored together in computer system 101, for each preselected position. To repeat, for each preselected position, the spatial information of the bucky 107 and tube head 109 are obtained from the encoders connected to the motors M, while the camera extrinsic parameters are calculated using the known PnP algorithm described herein.

[0033] Continuing the second phase of calibration, the tube head 109 is moved further from the bucky 107, such as at a distance of 1.8m, and translated again between, and including, the range of top position 3 and bottom position 4. The bucky 107 is also translated simultaneously and in parallel with the tube head 109 by translating within the range of top bucky position A and bottom bucky position B. Images of the bucky 107 are captured by camera 115 at the top and bottom positions as well as at several preselected positions within the range. Spatial positioning information of the bucky and the tube head is recorded for each position and stored in computer system 101, as described above, together with the calculated camera extrinsic parameters. The second phase of calibration continues with positioning the tube head 109 at a range of positions starting at position Zand extending linearly further along the z axis through position 5, which may include a distance ranging from Im 501, as between the tube head 109 and the bucky 107, to 1 ,8m with an image of the bucky 107 captured by the camera 115 at every 10cm of translation, for example, and spatial coordinate information similarly recorded at each preselected position, as described herein. It should be noted that the preselected distance intervals between tube head 109 and bucky 107, as in the calibration method described herein, may vary as desired, and the distances and positions described herein are exemplary. The distances, translation intervals and angular intervals for the calibration steps described herein may be selected based on an expected anatomy of patients to be imaged and their typical source-to-image distance (SID) parameters, for example, or they may be selected according to other preferences.

[0034] The second phase calibration of the radiographic imaging system 100 may further include capturing images of the bucky 107 while angulating the tube head109. Turning to FIG. 5B, there is depicted a second phase calibration step whereby the tube head is rotated about the axis at angular intervals of about five degrees in a range between about 70° and 110°, where the 90° position is a perpendicular position of the tube head 109 with respect to the bucky 107 (such as tube head position I in FIG. 5 A), while the bucky 107 remains at initial bucky position I. An image of the bucky 107 is captured by the camera 115 at every five degrees of rotation, for example, and spatial coordinate information is similarly recorded, as described herein but, in this example, the spatial coordinate information includes translation xyz coordinates as well as an angular coordinate for the camera extrinsic parameters. Turning to FIG. 5C, there is depicted a second phase calibration step whereby the tube head 109 is rotated downward about fifteen degrees from a perpendicular direction and, while remaining rotated at fifteen degrees downward, the tube head is then translated vertically to several positions between a selected top position and a selected bottom position, as described herein. The bucky 107 is positioned so as to appear in the center of the camera view and is moved vertically simultaneously and in parallel with the tube head 109. An image of the bucky 107 is captured by the camera 115 at several selected positions between the selected top and bottom positions, and spatial coordinate information is similarly recorded, as described herein but, in this example, the spatial coordinate information includes translation xyz coordinates as well as the fifteen degree downward angular coordinate parallel to the xz plane. Turning to FIG. 5D, there is depicted a second phase calibration step whereby the tube head 109 is rotated upward about fifteen degrees from a perpendicular direction and, while remaining rotated at fifteen degrees upward, the tube head is then translated vertically to several positions between a selected top position and a selected bottom position, as described herein. The bucky 107 is positioned so as to appear in the center of the camera view and is moved vertically simultaneously and in parallel with the tube head 109. An image of the bucky 107 is captured by the camera 115 at several selected positions between the selected top and bottom positions, and spatial coordinate information is similarly recorded, as described herein but, in this example, the spatial coordinate information includes translation xyz coordinates as well as the fifteen degree upward angular coordinate parallel to the xz plane.

[0035] With reference to FIG. 6, in a basic calibration step for table / bucky radiographic imaging system 200, an image of the bucky 107 (FIG. 4) is captured by the camera 115 when the tube head 109 and the bucky 107 are in initial positions I. Initial positions I may include the tube head 109 in a designated centered position relative to the bucky 107 at a distance of one meter (Im) 601, whereby an origin (0,0,0 of the radiographic imaging system 200 coordinate system is located in the center of the front of the bucky 107. The image of the bucky 107 as captured by the camera 115 is used by the computer system 101 (Markerless System) to calculate 3D spatial coordinates of the camera in the 3D coordinate system of the radiographic imaging system 200 using the PnP algorithm described herein. The- spatial coordinate location data of camera 115 (camera basic) is recorded and stored in the computer system 101, as well as the spatial coordinate information of the tube head 109 and the spatial coordinate information of the bucky 107.

[0036] Continuing with reference to FIG. 6, in a second phase calibration of the radiographic imaging system 200, the tube head 109 is moved to several preselected positions between, and including, left-most position 1 and right-most position 2, while the bucky 107 is moved simultaneously, and in parallel, with the tube head 109 between left-most position A and right-most position B, respectively, and images of the bucky 107 are again captured at each preselected position and the same spatial coordinate information corresponding to the camera extrinsic parameters, the tube head 109 and the bucky 107 are recorded and stored in computer system 101. A similar second phase calibration procedure, as described above in relation to the radiographic imaging system 100, continues with angulating the tube head 109 as exemplified by angular positions 3, 4, of the tube head 109.

[0037] Although the calibration process described herein uses examples of distances such as Im and 1.8m, as well as angulation angles ranging between 70° to 110°, these are examples that illustrate the space of calibration processes which might include many more distances that could range from about 0.4m to about 2m. Similarly, the angulation process may range from about 45° to about 135° and be calibrated at different various angular increments. This process may create a dense Markerless Calibration Manifold (MCM) from which intermediary positions of thecamera 115 could be interpolated, wherein the manifold describes a high dimensional space that captures and represents many of the system’s parameters. Performing and recording such a calibration process enables the radiographic imaging systems 100, 200, to correctly predict a position in the 3D operation space of the system irrespective of where the bucky 107, tube head 109 and the camera 115 are placed. This capability is critical because, when a radiographer or technician positions the bucky 107 and the tube head 109 for imaging, their position and angulation may differ from any one of the particular positions used in the calibration process. By performing the calibration processes as described herein and specifying locations and orientations of the camera in the MCM, it is possible to correctly recover camera 115 extrinsic parameters and thus relate image coordinates to world coordinates.

[0038] During normal operation of the radiographic imaging systems 100, 200, the mechanical components are subject to wear-and-tear over time. This can eventually lead to improper operation of the system, and its movements may become distorted beyond an acceptable threshold. Timely detection of changes in the system performance can be advantageous. In one embodiment, the spatial coordinate location data collected during the basic and second phase calibration can be stored and used as a baseline. At a later time period, the basic and second phase calibration data can be collected again and compared to the earlier stored data in order to determine if any mechanical deviations and offsets are occurring in the radiographic imaging systems 100, 200. The recorded data may include, but is not limited to, imaging data, extrinsic data, spatial coordinates of the markers, system coordinate data as well as any other system characteristics and status that is logged as part of the usual operation of the system. In one embodiment, mechanical deviations and offsets may be identified as the difference between the initial state after set-up (system baseline collected data) and its state later after a period of use. In one embodiment, in order to obtain a measurement of the deviation and offset, the wall / table bucky 107 and tube head 109 may be moved to the same positions to collect the basic and second phase calibration data as was used when performing the initial basic and second phase calibration. The same data, including, but not limited to, the imaging data, camera extrinsic parameters, spatial coordinate values of the markers and system coordinatedata for each selected position may be logged during the movements. In one embodiment, the locations of the markers in the newly captured images can be predicted by using the corresponding camera extrinsic parameters previously stored in the initial calibration steps, or directly detected in the newly captured image. A difference of the location of markers and the camera extrinsic parameters may be used to represent and quantify any mechanical deviations and offset of the DR imaging system. In one embodiment, a comparison may be made of the initial spatial coordinate data obtained, as described above, during the translation of the tube head 109 along the z axis starting at position / and extending linearly further from the bucky 107 along the z axis through position 5, with an image of the bucky 107 captured by the camera 115 at every 10cm of translation. A second calibration performed later to collect similar data by starting at position I and extending linearly further along the z axis through position 5, may determine that the newly collected spatial coordinates have deviated beyond an acceptable threshold in the x dimension due to, perhaps, wear and tear of the ceiling mount of tube crane system 117 which has sagged over time, thereby causing an unacceptable deviation from a true linear path along the z axis. This deviation information may be used for, but not limited to, triggering an audible and / or visual warning notification to indicate a deteriorated status with regard to usability and reliability of the radiographic imaging systems 100, 20Q. This information may thus be used in helping a service engineer to maintain and repair the radiographic imaging systems 100, 200.

[0039] In one embodiment, the calibration data sets collected at different points in time may be implemented to quantify and compute a deviation trend of the DR mechanical system. As described above, a difference in the location of the markers and the camera extrinsic parameters and camera intrinsic parameters can be used to represent and quantify any mechanical deviation and offset between any two sets of the calibration data. A deviation trend can be estimated by quantifying and comparing the detected deviation and offset between each of the calibration data sets over time. In one embodiment, a comparison may be made of the initial spatial coordinate data obtained, as described above, during the translation of the tube head 109 along the z axis starting at position I and extending linearly further from thebucky 107 along the z axis through position 5, with an image of the bucky 107 captured by the camera 115 at every 10cm of translation. The calibration might be performed multiple times after different periods of time, to collect similar data by starting at position Zand extending linearly further along the z axis through position 5. The difference among these calibration data sets may determine that the collections of spatial coordinates exhibit an increasing trend in the deviation and offset in the x dimension due to, perhaps, wear and tear of the ceiling mount of tube crane system 117, which has sagged over time. This may indicate an unacceptable deviation from a true linear path along the z axis in the near future. This deviation trend information may be used for, but not limited to, triggering an early warning notification to indicate a deteriorated trend with regard to usability and reliability of the radiographic imaging systems 100, 200. This information may thus be used in helping a service engineer to monitor the radiographic imaging systems 100, 200, and establish a routine maintenance and repair schedule.

[0040] As described herein, in one embodiment, a camera module is installed on a DR system. One time calibration is performed to identify the basic spatial relation between wall / table bucky, x-ray tube and the camera module, in which step the errors introduced by the mechanical motion and installation are compensated to improve the tracking and locating accuracy. A marker-less movement model is developed to update and locate spatial positions by comparing the basic coordinate value recorded during the one time calibration with one after relative motion. The camera module may also be applied to acquire and analyze the body features of the patient, e.g., body size and body key points. The spatial position of patient may be located by transforming the position information from the imaging coordinates to the DR system coordinates using the marker-less movement model. Applying the spatial information and body features may realize automatic positioning and posture identification, thereby assisting operation of the x-ray imaging system.

[0041] Using one time calibration, in particular, the camera module calibration embodiments of the present disclosure will be able to identify the basic spatial location between wall / table bucky and x-ray tube using just one RGB image, which greatly reduces the service workload. With one time calibration, in particular,the mechanical calibration embodiments of the present disclosure will be able to compensate for errors caused by DR system mechanical motion and rotation, including the sliding rail translation and the x-ray tube head rotation, which enables the disclosed solution to achieve high accuracy in different clinical scenarios.

[0042] In one embodiment using a marker-less movement model, the DR imaging system will be able to track the spatial relation between DR system and target patient in real-time without additional calibration or location marker detection, which makes the disclosed solution applicable to more clinical scenarios and patient groups. Using statistically based methods and the detection of patient height and width, embodiments of the present invention may be used to calculate the appropriate position of the wall / table bucky and x-ray tube, according to the specific patient and body part. The system can automatically control the wall / table bucky and x-ray tube to move toward the correct positions.

[0043] Using machine learning based methods, embodiments of the present invention may be used to identify and classify the posture of a patient according to the recommended radiographic positioning criteria. These results may be presented and displayed to the user. Using an augmented reality technology, embodiments of the present invention may be able to render and display the target patient position, the wall / table detector and AEC position, posture judgement results in real time, which makes the x-ray image acquisition process more intuitive.

[0044] In one embodiment, the visual tracking and positioning assistant system consists of a camera module, system calibration module, feature analysis module, motion control module and augmented reality module. The camera module is utilized to collect image data for several function modules. The system calibration module provides the one-time calibration functionality to determine the basic spatial relation data between wall / table bucky, x-ray tube and camera module, and to compensate the errors caused by mechanical motion and installation. The feature analysis module can extract the target information from the collected image data, including but not limited to patient height, width, body joint points and relative position between the patient and wall / table bucky, and then apply statistical methodsas well as machine learning methods to estimate the motion control parameters, and output positioning guidance information. The motion control module may use the motion parameters to move the wall / table bucky and x-ray tube to the desired position. The augmented reality module may be applied to display any desired information including, but not restricted to, wall / bucky boundary position, AEC position, collimator light boundary, and positioning guidance info. In one embodiment, the augmented reality module also provides the augmented reality control functionality, in which user can control the DR system through different multi-touch gestures, such as swipe, pinch, scroll and zoom. The control functionality includes, but is not limited to, collimator light size adjustment, and wall / table bucky and tube position correction.

[0045] In one embodiment, the camera module could consist of at least one 2D and / or 3D camera, or a plurality of 2D cameras and a plurality of 3D depth cameras. The installation position of the camera module includes, but is not limited to, the top / bottom of the x-ray tube head, right / left side of the x-ray tube head or right / left side of the wall bucky, alone or in combination. In one embodiment, the camera module can collect the imaging data, in which the acquired information includes, but is not limit to, marker location, wall / table bucky position and patient body position. In one embodiment, the camera module combined with a sound collection module may be used to collect both the imaging and sound data. In one embodiment, the calibration module could include the camera calibration function and mechanical calibration function. The one-time calibration could be performed in both manual and in a semi-automatic way. In one embodiment, the calibration module could obtain the information from the camera module, then use the location marker in the image to perform the calibration. In one embodiment, the feature analysis module could include a body feature extraction module, motion parameter calculation module and a posture judgement module. In one embodiment, the body features extracted by a feature analysis module may include the body height, body width, body contour and body key points. In one embodiment, motion parameters calculated by a feature analysis module using the disclosed marker-less movement model may be passed to the motion control module.

[0046] In one embodiment, an exemplary DR imaging system could be assembled as follows: one 3D depth camera module could be installed on the top side of a collimator included in a tube head. The camera module may be connected to a PC computer system and configured to perform the camera calibration and mechanical calibration in sequence through the system calibration module to initialize the camera module. A mixed reality device may be connected to the PC, then the guidance and assistant information can be projected into the real world through the augmented reality device. The workflow may include: inputting the patient and exam information on the PC; moving the patient to a waiting area and standing still; the camera module captures image data of the patient standing in the waiting area; the patient may move to the wall or table bucky; the camera module captures the image data of the target patient standing in front of the wall bucky or laying on the table bucky; the PC extracts and analyzes the patient body features using the collected image data through a feature analysis module; the PC calculates the desired motion parameters based on the extracted feature and current system coordinate value using the feature analysis module; the PC controls the wall / table bucky and x-ray tube to move to the desired position with specific tube posture through the desired motion parameters; the camera module recollects the image data then passes to the feature analysis module the posture judgement results; and guidance and assistant information are, then output to the augmented reality module.

[0047] The guidance and assistant information, such as detector boundary and AEC boundary, the posture judgement results, such as centering error, hand posture error and rotation error will be displayed using the mixed reality device. The failed positioning rule and its corresponding error value will be displayed as well. The user can adjust the positioning results according to the above information, or allow the exposure if positioning is acceptable. Once the patient moves or the user adjusts the positioning, the system may repeat any of the steps described until positioning is acceptable.

[0048] As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method, or computer program product. Accordingly, aspects of the present invention may take the form of a hardware andsoftware embodiment (including firmware, resident software, micro-code, etc.), or an embodiment combining software and hardware aspects that may all generally be referred to herein as a “service,” “circuit,” “circuitry,” “module,” and / or “system.”

[0049] Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium k or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc readonly memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0050] Program code and / or executable instructions embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0051] Computer program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, such as Java, Python, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer (device), partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may beconnected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0052] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0053] These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function / act specified in the flowchart and / or block diagram block or blocks.

[0054] The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0055] This writen description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal language of the claims.

Claims

CLAIMS:

1. A method of operating a radiographic imaging system having a movable tube head and a movable detector holder, the method comprising: attaching a camera to the movable tube head; detecting two or more markers on the movable detector holder; capturing a plurality of digital images of the movable detector holder and the two or more markers using the camera, each of the digital images captured at a different relative position as between the movable tube head and the movable detector holder; determining a first set of spatial coordinates of the camera in three dimensional space for each of the plurality of captured digital images; and determining and electronically storing at least one function that best fits the determined plurality of camera spatial coordinates in three dimensional space.

2. The method of claim 1, further comprising: disposing the two or more markers on the movable detector holder at known locations with respect to a location of a digital radiographic detector contained in the movable detector holder.

3. The method of claim 2, wherein the step of determining the first set of spatial coordinates of the camera comprises detecting the two or more markers in each of the plurality of captured digital images.

4. The method of claim 3, further comprising designating a point on the movable detector holder to define an origin of the three dimensional spatial coordinates.

5. The method of claim 4, further comprising designating a center of the detector holder as the origin of the three dimensional spatial coordinates.

6. The method of claim 1 , further comprising determining and electronically storing at least one polynomial function that best fits the determined plurality of camera spatial coordinates in three dimensional space.

7. The method of claim 1 , further comprising: positioning the tube head in preparation for digital radiographic imaging; and using the stored function to determine extrinsic parameters of the camera in three dimensional space.

8. The method of claim 7, further comprising using the extrinsic parameters of the camera to determine spatial coordinates of a point on the movable detector holder.

9. The method of claim 1, further comprising: repeating the steps of capturing a plurality of digital images and determining spatial coordinates; and comparing the first set of spatial coordinates with a second set of spafial coordinates obtained by the step of repeating to determine a level of deviation between the first set of spatial coordinates and the second set of spatial coordinates.

10. The method of claim 9, further comprising storing the determined level of deviation and transmitting a notification signal when the determined level of deviation exceeds a predetermined threshold.

11. A method of operating a radiographic imaging system comprising a movable tube head, a movable detector holder having one or more markers thereon, and a camera, the method comprising: capturing a first plurality of digital images of the movable detector holder and the one or more markers using the camera, each of the digital imagescaptured at a different relative position as between the camera and the movable detector holder; determining a first plurality of spatial poses of the camera each corresponding to one of the first plurality of captured digital images; and determining and electronically storing at least one function that best fits the determined first plurality of spatial poses of the camera.

12. The method of claim 11, further comprising: capturing a second plurality of digital images of the movable detector holder and the one or more markers using the camera, each of the second plurality of digital images captured at a different relative position as between the camera and the movable detector holder; determining a second plurality of spatial poses of the camera each corresponding to one of the second plurality of captured digital images; and comparing the first plurality of spatial poses with the second plurality of spatial poses to determine a magnitude or rate of change of deviations between the first plurality of spatial poses and the second plurality of spatial poses.

13. The method of claim 12, further comprising storing the determined magnitude or rate of change of deviations and transmitting a notification signal when the determined magnitude or rate of change of deviations exceed predetermined thresholds.

14. The method of claim 13, further comprising activating an audible or visual notification device in response to transmitting the notification signal.

15. The method of claim 12, further comprising: computing trends of the deviations; andcomputing an expected time when any of the deviations will exceed a predetermined threshold based on the computed trends.

Images