Benchmark detection for movement estimation and guidance

Abstract

An X-ray imaging system (100) includes an X-ray radiation source (105), an X-ray detector (107), and a C-arm (110) having the X-ray radiation source (105) disposed on a first end and the X-ray detector (107) disposed on a second end opposite the first end. The X-ray imaging system (100) is configured to translate the C-arm (110) in a plurality of different directions and to rotate the C-arm (110) about a plurality of different axes. The X-ray imaging system (100) includes an optical camera (162) configured to capture image data of a position of the C-arm (110) relative to a certain position. The X-ray imaging system (100) includes a controller (150) configured to receive image data from the optical camera (162) and to determine an initial position of the C-arm (110) relative to the position using machine vision and to guide the C-arm (110) to return to both the initial position based on the image data after the C-arm (110) has moved from the initial position.

Description

Background Technology

[0001] The subject matter disclosed herein relates to X-ray imaging systems, and more specifically, to benchmark detection used for motion estimation and guidance in X-ray imaging systems with C-arms.

[0002] Medical diagnostic imaging systems generate images of objects (such as patients) by exposing them to an energy source (such as X-rays passing through, for example, a patient). The generated images can be used for a variety of purposes. Typically, when physicians acquire X-rays of a patient, they want to obtain several X-rays of one or more parts of the patient's body from multiple different locations and angles, preferably without frequently repositioning the patient. To meet this need, C-arm X-ray diagnostic equipment has been developed. The term C-arm generally refers to an X-ray imaging device having a rigid and / or hinged structural member with X-ray source and image detector assemblies, each positioned at opposite ends of the structural member, such that the X-ray source and image detector face each other. The structural member is typically "C"-shaped, hence the name C-arm. In this way, X-rays emitted from the X-ray source can be projected onto the image detector, providing an X-ray image of one or more objects placed between the X-ray source and the image detector.

[0003] In many cases, a C-arm is attached to one end of a movable arm. In such cases, the C-arm can typically be raised and lowered, moved from one side to the other, and / or rotated about one or more axes of rotation. Therefore, such a C-arm can be moved and reoriented to allow X-ray images of the patient from several different positions and angles, as well as different parts of the body, without the need for frequent repositioning of the patient.

[0004] A common procedure in surgery involves using a mobile C-arm to perform some initial work before removing it. In later stages of the surgery, the mobile C-arm needs to be returned to the same position it was in before being moved, so that the surgery can continue with its assistance. Returning to the same position can be a relatively time-consuming process, involving trial and error and taking additional X-rays with the mobile C-arm to ensure it has been returned to its correct position.

[0005] One extension of the process of returning the mobile C-arm to the same position is the surgical step of performing a fluoroscopic imaging at the start of the procedure, using this as a baseline relative to the patient's body. Based on the starting position, further X-ray imaging is performed using the mobile C-arm while it is being repositioned to align with the target anatomy of interest. This fluoroscopic navigation step is a time-consuming process for using X-rays to locate the anatomy of interest and may expose the patient to unnecessary radiation. Summary of the Invention

[0006] The following outlines some embodiments commensurate with the scope of the originally claimed subject matter. These embodiments are not intended to limit the scope of the claimed subject matter, but rather to provide only a brief overview of the possible forms of the subject matter. In reality, the subject matter may include many forms that may be similar to or different from the embodiments described below.

[0007] According to one embodiment, an X-ray imaging system is provided. The X-ray imaging system includes an X-ray radiation source. The X-ray imaging system also includes an X-ray detector. The X-ray imaging system further includes a C-arm having an X-ray radiation source disposed at a first end and an X-ray detector disposed at a second end opposite to the first end. The X-ray imaging system is configured to translate the C-arm in multiple different directions and rotate the C-arm about multiple different axes. The X-ray imaging system also includes an optical camera configured to capture image data of the position of the C-arm relative to a given location. The X-ray imaging system also includes a controller including a memory and a processing system including one or more processors, and the controller is configured to receive image data from the optical camera and to determine an initial position of the C-arm relative to the given location using machine vision and to guide the C-arm back to the initial position based on the image data after the C-arm has moved from the initial position.

[0008] According to another embodiment, a computer-implemented method is provided. This computer-implemented method includes receiving image data from an optical camera mounted on a C-arm of an X-ray imaging system at a processing system including one or more processors. The X-ray imaging system includes an X-ray radiation source, an X-ray detector, and a C-arm having an X-ray radiation source disposed at a first end and an X-ray detector disposed at a second end opposite to the first end. The X-ray imaging system is configured to translate the C-arm in multiple different directions and rotate the C-arm about multiple different axes. The optical camera is configured to capture image data of the position of the C-arm relative to a certain location. The computer-implemented method further includes determining an initial position of the C-arm relative to that location using machine vision via the processing system, and guiding the C-arm back to the initial position based on the image data after the C-arm has moved from that initial position.

[0009] In another embodiment, a non-transitory computer-readable medium is provided. This non-transitory computer-readable medium includes processor-executable code that, when executed by a processing system including one or more processors, causes the processing system to perform actions. These actions include receiving image data from an optical camera mounted on a C-arm of an X-ray imaging system. The X-ray imaging system includes an X-ray radiation source, an X-ray detector, and a C-arm having an X-ray radiation source disposed at a first end and an X-ray detector disposed at a second end opposite the first end. The X-ray imaging system is configured to translate the C-arm in multiple different directions and rotate the C-arm about multiple different axes. The optical camera is configured to capture image data of the position of the C-arm relative to a certain location. These actions also include determining an initial position of the C-arm relative to that location using machine vision and guiding the C-arm back to the initial position based on the image data after the C-arm has moved from that initial position. Attached Figure Description

[0010] These and other features, aspects, and advantages of the disclosed subject matter of the invention will be better understood when reading the following detailed description with reference to the accompanying drawings, in which like reference numerals denote like parts throughout the drawings, wherein:

[0011] Figure 1 This is a block diagram illustrating the components of an example X-ray imaging system according to various aspects of this disclosure;

[0012] Figure 2 Based on all aspects of this disclosure Figure 1 A schematic diagram of the implementation scheme of the X-ray imaging system in the diagram;

[0013] Figure 3 Based on all aspects of this disclosure Figure 1 A schematic diagram of the X-ray imaging system in the image shows the degrees of freedom of movement related to the position of the C-arm gantry.

[0014] Figure 4 This is a flowchart of a method for utilizing benchmark detection for motion estimation and guidance, according to various aspects of this disclosure;

[0015] Figure 5A and Figure 5B A more detailed flowchart of the methods for utilizing benchmark detection for motion estimation and guidance, based on various aspects of this disclosure;

[0016] Figure 6 A more detailed flowchart of a method for using a benchmark for motion estimation and (e.g., using a trajectory) guidance, based on various aspects of this disclosure;

[0017] Figure 7It is a graphical user interface displayed on a display according to various aspects of this disclosure (e.g., an edge-enhanced image that highlights the part of interest);

[0018] Figure 8 It is a graphical user interface (e.g., having user-perceptible instructions) displayed on a display according to various aspects of this disclosure;

[0019] Figure 9 It is a graphical user interface illustrating information extracted from image data (e.g., image features such as contrast) according to various aspects of this disclosure; and

[0020] Figure 10 This is a schematic diagram of a process for utilizing a benchmark detection method for motion estimation and guidance, based on various aspects of this disclosure. Detailed Implementation

[0021] One or more specific implementations will be described below. To provide a concise description of these implementations, not all characteristics of the actual implementation may be described in this specification. It should be understood that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's specific objectives, such as complying with system-related and business-related constraints that may differ from implementation to implementation. Furthermore, it should be understood that such development efforts may be complex and time-consuming, but are nonetheless routine tasks of design, fabrication, and manufacturing for those skilled in the art who benefit from this disclosure.

[0022] When describing elements of various embodiments of the subject matter of this invention, the articles “a,” “an,” “the,” and “described” are intended to indicate the presence of one or more elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to the listed elements. Furthermore, any numerical examples in the following discussion are intended to be non-limiting, and therefore the additional values, ranges, and percentages are within the scope of the disclosed embodiments.

[0023] This disclosure relates to systems and methods for reference detection for motion estimation and guidance used in X-ray imaging systems with C-arms. Specifically, it relates to computer or machine vision-based techniques (e.g., utilizing camera image data) for initial positioning of the C-arm and using the C-arm for realignment, repositioning, and / or reconfirmation that the C-arm has later (e.g., after it has been moved) reached (e.g., returned to) the same position during intraoperative setup or surgery.

[0024] The disclosed systems and methods provide an easier and clearer way to return a C-arm to a previous (e.g., initial) position. The disclosed systems and methods provide user instructions for the trajectory required to return the C-arm to the previous position, including but not limited to visual cues related to the direction of travel, distance traveled, and items to be checked for correctness. This user-aided information simplifies the workflow for returning to the same position. The disclosed systems and methods can be used to virtually move an X-ray image to be virtually moved as the C-arm changes position to show an approximate location for the next X-ray imaging. The disclosed systems and methods eliminate the need for fluoroscopic navigation.

[0025] In the disclosed embodiments, the X-ray imaging includes an X-ray radiation source. The X-ray imaging system also includes an X-ray detector. The X-ray imaging system further includes a C-arm having an X-ray radiation source disposed at a first end and an X-ray detector disposed at a second end opposite the first end. The X-ray imaging system is configured to translate the C-arm in multiple different directions and rotate the C-arm about multiple different axes. The X-ray imaging system also includes an optical camera configured to capture image data of the position of the C-arm relative to a given location. The X-ray imaging system also includes a controller comprising a memory and a processing system including one or more processors, and the controller is configured to receive the image data from the optical camera and to determine an initial position of the C-arm relative to the given location using machine vision and to guide the C-arm back to the initial position based on the image data after the C-arm has moved from the initial position.

[0026] In some embodiments, an optical camera is mounted on the C-arm (e.g., on an X-ray detector or X-ray radiation source). In some embodiments, a controller is configured to process initial image data received from the optical camera to identify image features, enhance the identified image features, and utilize the enhanced image features as reference features to return the C-arm to its initial position. In some embodiments, the controller is configured to receive input from a user to activate the use of machine vision, thereby guiding the C-arm back to its initial position. In some embodiments, the controller is configured to process real-time image data received from the optical camera to identify subsequent image features, enhance the identified subsequent image features, and compare the enhanced subsequent image features with reference features.

[0027] In some embodiments, the X-ray imaging system includes a display, and a controller is configured to cause the display to show both real-time image data and a user-perceptible instruction to guide the C-arm back to its initial position based on a comparison of enhanced subsequent image features with reference features. In some embodiments, the user-perceptible instruction includes a direction of movement, distance to a target, or a combination thereof. In some embodiments, the X-ray imaging system includes a display, and a controller is configured to cause the display to show both real-time image data and a reference image having reference features that assist the user in guiding the C-arm back to its initial position. In some embodiments, the X-ray imaging system includes a display, and a controller is configured to cause the display to show real-time image data, a user-perceptible instruction to guide the C-arm back to its initial position based on a comparison of enhanced subsequent image features with reference features, and a reference image having reference features that assist the user in guiding the C-arm back to its initial position.

[0028] In some embodiments, the controller is configured to provide control signals to move the C-arm and / or X-ray imaging system (e.g., automatically) to an initial position based on a comparison of enhanced subsequent image features with reference features. In some embodiments, the initial image data and real-time image data include red-green-blue images and / or depth images. In some embodiments, image features and subsequent image features include sharp edges, significantly bright or dark areas, distinctive shapes or objects, and / or contrast information.

[0029] In some embodiments, the X-ray imaging system includes a mobile base configured to move the X-ray imaging system, and a controller configured to calculate the global positioning of the C-arm during movement of the X-ray imaging system from an initial position via the mobile base, then map the environment based on image data to form a trajectory relative to the floor where the mobile base is located, and utilize the trajectory when guiding the C-arm back to the initial position. In some embodiments, the X-ray imaging system includes sensors disposed on components of the X-ray imaging system for determining the position and movement of the C-arm, and wherein the controller is configured to receive feedback from the sensors to determine the position and movement of the C-arm and estimate the movement of the mobile base. In some embodiments, the sensors include one or more of a C-arm position sensor, an accelerometer, and a gyroscope. In some embodiments, the X-ray imaging system includes one or more known markers disposed within the field of view of an optical camera, and the controller is configured to use the one or more known markers within the image data to determine the corresponding positions of the X-ray imaging system and the C-arm relative to the one or more known markers.

[0030] In the disclosed embodiments, the computer-implemented method includes receiving image data from an optical camera mounted on a C-arm of an X-ray imaging system at a processing system including one or more processors. The X-ray imaging system includes an X-ray radiation source, an X-ray detector, and a C-arm having an X-ray radiation source disposed at a first end and an X-ray detector disposed at a second end opposite the first end. The X-ray imaging system is configured to translate the C-arm in multiple different directions and rotate the C-arm about multiple different axes. The optical camera is configured to capture image data of the position of the C-arm relative to a given location. The computer-implemented method further includes determining an initial position of the C-arm relative to that location using machine vision via the processing system, and guiding the C-arm back to the initial position based on the image data after the C-arm has moved from that initial position.

[0031] In some embodiments, the computer-implemented method further includes processing initial image data received from an optical camera via a processing system to identify image features, enhance the identified image features, and utilize the enhanced image features as reference features for returning the C-arm to its initial position. In some embodiments, the computer-implemented method further includes receiving input from a user at the processing system to activate the use of machine vision to guide the C-arm back to its initial position, and processing real-time image data received from the optical camera via the processing system to identify subsequent image features, enhance the identified subsequent image features, and compare the enhanced subsequent image features with the reference features.

[0032] In the disclosed embodiments, the non-transitory computer-readable medium includes processor-executable code that, when executed by a processing system including one or more processors, causes the processing system to perform actions. These actions include receiving image data from an optical camera mounted on a C-arm of an X-ray imaging system. The X-ray imaging system includes an X-ray radiation source, an X-ray detector, and a C-arm having an X-ray radiation source disposed at a first end and an X-ray detector disposed at a second end opposite to the first end. The X-ray imaging system is configured to translate the C-arm in multiple different directions and rotate the C-arm about multiple different axes. The optical camera is configured to capture image data of the position of the C-arm relative to a certain location. These actions also include determining an initial position of the C-arm relative to that location using machine vision and guiding the C-arm back to the initial position based on the image data after the C-arm has moved from that initial position.

[0033] In some embodiments, these actions also include processing initial image data received from the optical camera to identify image features, enhancing the identified image features, and using the enhanced image features as reference features to return the C-arm to its initial position. These actions also include receiving input from a user to activate machine vision, thereby guiding the C-arm back to its initial position. These actions also include processing real-time image data received from the optical camera to identify subsequent image features, enhancing the identified subsequent image features, and comparing the enhanced subsequent image features with reference features.

[0034] Figure 1 This is a block diagram illustrating the components of an example X-ray imaging system 100 (e.g., a mobile X-ray imaging system). The mobile X-ray imaging system 100 includes an X-ray source 105 and an X-ray detector 107 mounted on a C-arm gantry 110 (e.g., a C-arm).

[0035] C-arm frame 110 includes a C-arm motor 112 for adjusting the position of the C-arm frame 110. More specifically, the C-arm frame 110 is mechanically coupled to a C-arm bracket 111 (e.g., a C-arm swivel device) including the C-arm motor 112, and the C-arm motor 112 can be driven to adjust the position of the C-arm frame 110 relative to the C-arm bracket 111. For example, the C-arm bracket 111 coupled with the C-arm motor 112 is configured to rotate the C-arm frame 110 relative to the C-arm bracket 111 in a track direction. In some embodiments, the C-arm bracket 111 (via a motorization system) is configured to rotate a pivot (e.g., a pivot point), wherein the C-arm bracket 111 is coupled to a mobile base 140 (e.g., an automated guided vehicle) or the end of an L-arm coupled to the mobile base 140. C-arm bracket 111 rotates about the axis of rotation of the pivot (e.g., a horizontal axis). In some embodiments with an L-arm, the L-arm can rotate about the other end of the L-arm (i.e., the end of the L-arm not connected to the pivot) connected to the position of the movable base 140.

[0036] The mobile X-ray imaging system 100 also includes a mobile base 140. A C-arm bracket 111 is coupled to the mobile base 140. The mobile base 140 is configured to move (e.g., transfer) the mobile X-ray imaging system 100 from one location on the floor to another. The mobile base 140 includes a chassis 141. The mobile base 140 includes one or more motors 142 for driving one or more wheels 144 (e.g., drive wheels) to adjust the position of the mobile base 140. Furthermore, the one or more wheels 144 may be free-wheeling or non-motorized.

[0037] The mobile X-ray imaging system 100 also includes a controller 150, which includes a processor 152 and a non-transitory memory 154. Methods for controlling the mobile X-ray imaging system 100 can be stored as executable instructions 155 in the non-transitory memory 154 and executed by the processor 152.

[0038] The mobile X-ray imaging system 100 also includes a user interface 160 for receiving input from a user or operator of the mobile X-ray imaging system 100. The user interface 160 is communicatively coupled to a controller 150 to provide commands input by the user via the user interface 160 to the controller 150. The user interface 160 may include one or more of the following: a keyboard, mouse, trackball, one or more knobs, one or more joysticks, touchpad, touchscreen, one or more hard and / or soft buttons, smartphone, microphone, virtual reality device, etc. Therefore, the user interface 160 can enable voice control and information display, such as interactive display devices (e.g., touchscreens). In some examples, the user interface 160 may be remotely positioned relative to the mobile X-ray imaging system 100. For example, the user interface 160 may be communicatively coupled to the controller 150 and / or the mobile X-ray imaging system 100 via a wired or wireless connection and may be positioned remotely from the mobile base 140.

[0039] As an example, memory 154 may store processor-executable software code or instructions (e.g., firmware or software) tangibly stored on a non-transitory computer-readable medium. Additionally or alternatively, memory 154 may store data. As an example, memory 154 may include volatile memory (such as random access memory (RAM)) and / or non-volatile memory (such as read-only memory (ROM), flash memory, hard disk drive, or any other suitable optical, magnetic, or solid-state storage medium or combinations thereof). Furthermore, processor 152 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more application-specific microprocessors, and / or one or more application-specific integrated circuits (ASICs), or some combination thereof. For example, processor 152 may include one or more Reduced Instruction Set Computing (RISC) or Complex Instruction Set Computing (CISC) processors. Processor 152 may include multiple processors and / or memory 154 may include multiple memory devices.

[0040] The mobile X-ray imaging system 100 includes an optical camera 162 (e.g., a depth camera). The optical camera 162 is configured to acquire image data (e.g., video images including color images (red, green, and blue (RGB) images) and depth images representing three-dimensional (3D) positional information of surfaces within the image). The optical camera 162 is configured to capture image data of the position of the C-arm gantry 110 relative to a location (e.g., a worktable supporting the subject to be imaged). In some embodiments, the optical camera 162 is mounted on the C-arm gantry 110. In some embodiments, the optical camera 162 is mounted on the X-ray detector 107. In some embodiments, the optical camera 162 is mounted on the X-ray source 105. In some embodiments, the optical camera 162 may be mounted at an alternative location within the mobile X-ray imaging system 100. In some embodiments, the optical camera 162 is separate from the mobile X-ray imaging system 100 (i.e., not mounted on the C-arm gantry). For example, the optical camera 162 may be mounted on the ceiling of the room where the mobile X-ray imaging system 100 is located.

[0041] In some embodiments, the mobile X-ray imaging system 100 includes markers 164 (e.g., reference markers). Markers 164 are known (i.e., the location of markers 164 is known). One or more markers 164 may be placed on the worktable where the subject to be imaged is located, within the environment in which imaging occurs (e.g., on objects in the environment, on walls, on the floor, etc.), and / or on one or more components of the mobile X-ray imaging system 100. In some embodiments, markers 164 are reference patterns configured to be rapidly located by a computer vision system when within the field of view of the optical camera 162. Image data with markers 164 can be processed by processor 152 to provide system position information relative to the marker locations. Markers 164 may be ArUco markers, ArTag markers, and / or AprilTag markers.

[0042] The mobile X-ray imaging system 100 includes a C-arm position sensor 166. The C-arm position sensor 166, disposed on various components of the mobile X-ray imaging system 100 (e.g., C-arm gantry 110, lifting column, crossarm, etc.), is configured to detect and track the movement of the C-arm gantry 110. For example, the C-arm position sensor 166 can track the orbital rotation of the lifting column, crossarm, and C-arm gantry 110, and / or the lateral rotation of the C-arm gantry 110.

[0043] The mobile X-ray imaging system 100 includes an inertial measurement unit 168. The inertial measurement unit 168 can be mounted on various components of the mobile X-ray imaging system 100. The inertial measurement unit 168 can be used to monitor the attitude and movement of the C-arm gantry 110. The inertial measurement unit 168 may include a gyroscope and / or an accelerometer.

[0044] Controller 150 is configured to receive image data (e.g., RGB images and / or depth images) from optical camera 162 and is configured to use machine vision to determine the initial position of C-arm gantry 110 relative to that position (e.g., a worktable on which the patient is positioned during an intraoperative procedure) and guide C-arm gantry 110 back to that initial position based on the image data after C-arm gantry 110 has moved from that initial position. Controller 150 is also configured to process the initial image data received from optical camera 162 to identify image features, enhance the identified image features, and utilize the enhanced image features as reference features for returning C-arm gantry 110 to the initial position. In some embodiments, controller 150 is configured to use one or more known markers 164 within the image data to determine the corresponding positions of X-ray imaging system 100 and C-arm gantry 110 relative to one or more known markers 164.

[0045] The controller 150 is also configured to receive input from a user (e.g., via user interface 160) to activate the use of machine vision, thereby guiding the C-arm gantry 110 back to its initial position. The controller 150 is also configured to process real-time image data received from an optical camera to identify subsequent image features, enhance the identified subsequent image features, and compare the enhanced subsequent image features with reference features.

[0046] In some embodiments, controller 150 is configured to display both real-time image data and a user-perceptible instruction to guide the C-arm gantry 110 back to its initial position based on a comparison of enhanced subsequent image features with reference features on a display (e.g., display device 165). In some embodiments, the user-perceptible instruction includes a direction of movement, distance to a target, or a combination thereof. In some embodiments, controller 150 is configured to display both real-time image data and a reference image having reference features that assist the user in guiding the C-arm gantry 110 back to its initial position on a display. In some embodiments, controller 150 is configured to display real-time image data, a user-perceptible instruction to guide the C-arm gantry 110 back to its initial position based on a comparison of enhanced subsequent image features with reference features, and a reference image having reference features that assist the user in guiding the C-arm gantry 110 back to its initial position on a display.

[0047] In some embodiments, controller 150 is configured to provide control signals to move (e.g., automatically) the C-arm gantry 110 and / or X-ray imaging system 100 to an initial position based on a comparison of enhanced subsequent image features with reference features. In some embodiments, the initial image data and real-time image data include red-green-blue images and / or depth images. In some embodiments, image features and subsequent image features include sharp edges, significantly bright or dark areas, distinctive shapes or objects, and / or contrast information.

[0048] In some embodiments, controller 150 is configured to calculate the global positioning of C-arm gantry 110 during movement of the X-ray imaging system from its initial position via movable base 140, then map the environment based on the image data to form a trajectory relative to the floor where movable base 140 is located, and utilize this trajectory when guiding C-arm gantry 110 back to that initial position. In some embodiments, controller 150 is configured to receive feedback from sensor 166 and / or inertial measurement unit 168 to determine the position and movement of C-arm gantry 110 and estimate the movement of movable base 140.

[0049] The controller 150 is further communicatively coupled to a display device 165 for displaying one or more X-ray images acquired via the X-ray detector 107. Furthermore, in some examples, one or more of the controller 150, user interface 160, and display device 165 may be positioned remotely from the rest of the mobile X-ray imaging system 100 (e.g., in its remote location).

[0050] Figure 2 It shows Figure 1 A schematic diagram of an embodiment of the X-ray imaging system 100 is shown. The X-ray imaging system 100 is configured to translate the C-arm gantry 110 in multiple different directions and rotate the C-arm gantry 110 about multiple different axes. Although a moving imaging system is shown, the embodiment described below can be used with any X-ray imaging system (e.g., a fixed imaging system) having a C-arm. The X-ray imaging system 100 can utilize various imaging modalities (e.g., fluoroscopy, computed tomography, tomography, radiography, magnetic resonance imaging, etc.) to acquire two-dimensional 2D and / or 3D image data. The X-ray imaging system 100 can be used for both diagnostic imaging and interventional imaging. Furthermore, the X-ray imaging system 100 can be used for general purposes (e.g., general radiology, orthopedic surgery, etc.) and special purposes (e.g., image-guided surgery).

[0051] The primary function of the mobile X-ray imaging system 100 is to generate X-rays for diagnostic and interventional imaging. The X-ray imaging system 100 includes a support structure or base 140 (e.g., a mobile base), a C-arm gantry 110, an L-arm 170, and a controller 150. The base 140 provides support to the C-arm gantry 110 and holds the C-arm gantry 110 in a suspended position. The lower portion of the base 140 includes wheels or casters 144 for providing mobility to the system 100. The base 140 includes a vertical lifting column 172 that allows the C-arm gantry 110 and the L-arm 170 to move vertically relative to the base 140. The vertical lifting column 172 terminates in the upper housing 174 of the base 140, wherein the horizontal extension arm 176 (e.g., a crossarm) passes through the upper housing 174 and allows the L-shaped arm 170 (and the C-shaped arm frame 110) to move vertically relative to the vertical lifting column 172 by the movement (e.g., horizontal movement) of the horizontal extension arm 176 relative to the upper housing 174. The C-shaped arm frame 110 can move along the axis of the horizontal extension arm 176 to achieve lateral tracking motion. The L-shaped arm 170 is coupled to the horizontal extension arm 176 via an end 178 and is configured to pivot or rotate about the horizontal extension arm 176, such that the L-shaped arm 170 can pivot in a 360-degree arc. The horizontal extension arm 176 is coupled to one end 178 of the L-shaped arm 170, while the outer end 180 of the L-shaped arm 176 is coupled to the C-shaped arm frame 110. The rotation of the L-arm 176 about its position where it is attached to the horizontal extension arm 176 allows the C-arm frame 110 to rotate (e.g., 360 degrees) relative to the base 140 about a lateral axis 184 (e.g., parallel to the horizontal extension arm 176) in a lateral direction 182 (e.g., circumferential direction).

[0052] C-arm frame 110 is coupled to C-arm bracket 111, which is coupled to the end 180 of L-arm 170. C-arm bracket 111 is coupled to an assembly of rollers or wheels (e.g., within a track 185 disposed within C-arm frame 110), which allows C-arm frame 110 to move or rotate relative to C-arm bracket 111 along track 185 about track axis 186 in the track direction.

[0053] In some embodiments, the C-arm bracket 111 also enables the C-arm frame 110 to rotate (e.g., circumferentially) or flip (e.g., as indicated by reference numeral 188) about an axis 190 (e.g., a flip axis), which emanates from the position where the C-arm bracket 111 is connected to the C-arm frame 110 and thus to the base 140. The C-arm bracket 111 enables the C-arm frame 110 to rotate 180 degrees relative to the C-arm bracket 111.

[0054] X-ray detector 107 and X-ray source 105 are coupled to opposite ends 192, 194 of C-arm gantry 110 to form an image chain. C-arm gantry 110 allows X-ray detector 107 and X-ray source 105 to be mounted and positioned around the object to be imaged, such as a patient. For example, C-arm gantry 110 can be a circular C-shaped or arc-shaped component. C-arm gantry 110 allows for selective positioning of X-ray detector 107 and X-ray source 105 relative to the width and length of the patient or other object located within the internal free space of C-arm gantry 110. X-ray detectors and X-ray sources 46 are used to generate diagnostic images representing the object being imaged.

[0055] Rotation about axes 184, 186, and 190 is independent (e.g., separate or different from each other). Rotation of the C-arm gantry 110 relative to these axes 184, 186, and 190 is driven by a motorization system 196. The motorization system 196 may include one or more motors or servo motors to drive rotation about these axes 184, 186, and 190 via automation. The motors or servo motors may be located in different components of the imaging system 100. The motorization system 196 may be coupled to a control system or controller 150 (e.g., located within and / or remote from the imaging system 100). The controller 150 includes a memory 154 and one or more processors 152 to execute code or instructions stored in the memory 154. The controller 150 can control the automatic movement of the C-arm gantry 110 about axes 184, 186, and 190. Figure 2 As depicted, the optical camera 162 is mounted on the C-arm gantry 110. In particular, the optical camera 162 is mounted on the X-ray detector 107.

[0056] Figure 3 yes Figure 1 A schematic diagram of the X-ray imaging system 100 is shown, illustrating the degrees of freedom of movement associated with the position of the C-arm gantry 110. As shown in box 198, the first, second, and third degrees of freedom occur along the floor. The techniques described herein estimate these degrees of freedom because there are no sensors associated with these types of movement on the movable base 140. The first degree of freedom relates to forward and backward movement (i.e., translation) of the X-ray imaging system 100 along the floor via the movable base 140 (as indicated by arrow 1), which affects the positioning of the C-arm gantry 110. The second degree of freedom relates to left and right movement (i.e., translation) of the X-ray imaging system 100 along the floor via the movable base 140 (as indicated by arrow 2), which affects the positioning of the C-arm gantry 110. The third degree of freedom relates to turning (i.e., rotating) the X-ray imaging system 100 about a position on the floor via the movable base 140 (as indicated by arrow 3), which affects the positioning of the C-arm gantry 110.

[0057] The fourth and fifth degrees of freedom relate to the movement of the frame supports of the movable C-arm frame 110. The fourth degree of freedom relates to the forward and backward movement (i.e., translation) of the transverse arm 176 (e.g., a horizontal extension arm) as indicated by arrow 4. The fifth degree of freedom relates to the up and down movement (i.e., translation) of the vertical lifting column 172 as indicated by arrow 5.

[0058] The sixth and seventh degrees of freedom relate to the movement of the C-arm gantry 110. The sixth degree of freedom relates to the orbital rotation of the C-arm gantry 110 as indicated by arrow 6 (e.g., left anterior oblique (LAO) rotation and right anterior oblique (RAO) rotation). The seventh degree of freedom relates to the lateral rotation of the C-arm gantry 110 about the lateral axis 184 as indicated by arrow 7 (e.g., cranial (CRA) rotation and caudal (CAU) rotation).

[0059] Eight degrees of freedom relate to the X-ray detector 107. The X-ray detector 107 can rotate as shown by arrow 8.

[0060] Figure 4 This is a flowchart of a method 200 for utilizing benchmark detection for motion estimation and guidance. One or more steps of method 200 may be derived by... Figure 1 The processing circuit of the X-ray imaging system 100 is executed.

[0061] Method 200 includes receiving image data (e.g., input video) from an optical camera (e.g., mounted on a C-arm of an X-ray imaging system), wherein the optical camera is configured to capture image data of the position of the C-arm relative to a location (e.g., a worktable with a subject being imaged and undergoing intraoperative procedures) (box 202). The image data may include video, which includes color images (e.g., RGB images) and / or depth images. Method 200 also includes determining an initial position of the C-arm relative to the location using machine vision (e.g., computer vision), and guiding the C-arm back to the initial position based on the image data after the C-arm has moved from the initial position (box 204).

[0062] Figure 5A and Figure 5B This is a flowchart of method 208 for utilizing benchmark detection for motion estimation and guidance. One or more steps of method 208 may be derived from... Figure 1 The processing circuit of the X-ray imaging system 100 is executed.

[0063] Method 208 includes receiving image data (e.g., input video) from an optical camera (e.g., mounted on the C-arm of an X-ray imaging system), wherein the optical camera is configured to capture image data of the position of the C-arm relative to a location (e.g., a worktable with a subject being imaged and undergoing intraoperative procedures) (box 210). The image data may include video, which includes color images (e.g., RGB images) and / or depth images.

[0064] Method 208 further includes processing initial image data received from an optical camera (e.g., input video frames) to identify image features of interest (box 212). Identifiable image features include sharp edges, significantly bright or dark areas, distinctive shapes or objects, and / or contrast information. The image may be decomposed into individual colors, converted to grayscale or other color spaces, smoothed, sharpened, or otherwise processed to facilitate feature extraction. Method 208 also includes enhancing image features identified as providing optimal user display to facilitate the return of the C-arm to its initial position (box 214). Method 208 even includes utilizing the enhanced image features as reference features for returning the C-arm to its initial position (box 216).

[0065] Method 208 also includes receiving input from a user to activate the use of machine vision, thereby guiding the C-arm back to its initial position (box 218). Method 208 further includes processing real-time image data received from an optical camera to identify subsequent image features (box 220). The real-time image data may include video, including color images (e.g., RGB images) and / or depth images. Identifiable subsequent image features include sharp edges, significantly bright or dark areas, distinctive shapes or objects, and / or contrast information. Images may be decomposed into individual colors, converted to grayscale or other color spaces, smoothed, sharpened, or otherwise processed to facilitate feature extraction. Method 208 includes enhancing the identified subsequent image features (box 222). Method 208 includes comparing the enhanced subsequent image features with reference features (box 224).

[0066] In some embodiments, method 208 further includes both displaying real-time image data on a display and a user-perceptible instruction to guide the C-arm back to its initial position based on a comparison of enhanced subsequent image features with reference features (box 226). In some embodiments, method 208 further includes both displaying real-time image data on a display and a reference image having reference features that assist the user in guiding the C-arm back to its initial position (box 228). In some embodiments, method 208 further includes displaying real-time image data on a display, a user-perceptible instruction to guide the C-arm back to its initial position based on a comparison of enhanced subsequent image features with reference features, and a reference image having reference features that assist the user in guiding the C-arm back to its initial position (box 229). In some embodiments, the user-perceptible instruction may be a direction of movement, distance to a target, or a combination thereof. In some embodiments, user input may be received to turn the instruction and / or reference features on or off. In some embodiments, method 208 includes receiving input from a user (e.g., via a user interface) to move the C-arm and / or X-ray imaging system, thereby utilizing the provided assistance to guide the C-arm back to its initial position (box 230). In some embodiments, method 208 includes providing control signals to move the C-arm and / or X-ray imaging system (e.g., automatically in some embodiments) to an initial position (box 232) based on a comparison of the enhanced subsequent image features with reference features.

[0067] Figure 6 This is a flowchart of method 234 for utilizing benchmark detection for motion estimation and guidance. One or more steps of method 234 may be derived from... Figure 1 The processing circuit of the X-ray imaging system 100 is executed.

[0068] Method 234 includes receiving image data (e.g., input video) from an optical camera (e.g., mounted on the C-arm of an X-ray imaging system), wherein the optical camera is configured to capture image data of the position of the C-arm relative to a location (e.g., a worktable with a subject being imaged and undergoing intraoperative procedures) (box 236). The image data may include video, which includes color images (e.g., RGB images) and / or depth images.

[0069] Method 234 further includes processing initial image data received from the optical camera (e.g., input video frames) to identify image features of interest (box 238). In some embodiments, the image features may be richer information, such as contrast. In some embodiments, the image features may be known markers set within the field of view of the optical camera. One or more known markers within the image data may be used to determine the corresponding positions of the X-ray imaging system and the C-arm relative to one or more known markers. Method 234 also includes enhancing the image features identified as providing optimal user display to facilitate the return of the C-arm to its initial position (box 240). Method 234 even includes utilizing the enhanced image features as reference features for returning the C-arm to its initial position (box 242). Method 234 utilizes computer vision for boxes 238-242.

[0070] Method 234 includes moving the C-arm from an initial position (box 244). Some of the movement of the C-arm may occur due to movement (e.g., translation and / or rotation) via a movable base. Method 234 includes receiving feedback from sensors disposed on components of the X-ray imaging system to determine the position and movement of the C-arm and to estimate the movement of the movable base (box 246). In some embodiments, any known markers within the image data may also be used to determine the position and movement of the C-arm and to estimate the movement of the movable base.

[0071] Method 234 further includes calculating the global positioning of the C-arm during movement of the X-ray imaging system from its initial position via a movable base, and then mapping the environment based on the image data to form a trajectory relative to the floor where the movable base is located (box 248). Method 234 also includes utilizing the trajectory when guiding the C-arm back to its initial position (box 250). In some embodiments, utilizing the trajectory includes replaying (e.g., on a display) the trajectory used for guidance, and also displaying user-perceptible instructions to assist the user in guiding the C-arm back to its initial position (box 252). In some embodiments, method 234 includes providing control signals to move the C-arm and / or the X-ray imaging system (e.g., automatically in some embodiments) to its initial position based on a comparison of enhanced subsequent image features with reference features (box 254).

[0072] Figure 7 A graphical user interface 256 (e.g., with an edge-enhanced image highlighting the portion of interest) is displayed on the monitor 165. The graphical user interface 256 includes a graphic 258 indicating the current position of the C-arm relative to the worktable (e.g., the worktable where the object of interest is located). The graphical user interface 256 includes a reference image 260 (e.g., when the C-arm is in its initial position), which has already been... Figure 5A and Figure 5BThe process is performed as described in method 208 and obtained from an optical camera (e.g., a detector mounted on a C-arm). The graphical user interface 256 also discloses a real-time image 262 from the optical camera, which has been described as... Figure 5A and Figure 5B The processing is performed as described in method 208. Specifically, for the portion of interest within the camera's field of view, the edges have been enhanced (e.g., highlighted). The portion of interest is the sterilization cover set above the phantom on the worktable. The graphical user interface 256 also includes a graph 264 that includes positional information of the C-arm (e.g., position relative to 90 degrees LAO and 0 degrees CRA). The highlighted edges and reference image 260 are used to provide user assistance when returning the C-arm to its initial position.

[0073] Figure 8 A graphical user interface 266 (e.g., with user-perceptible instructions) is displayed on a monitor 165. The graphical user interface 266 also discloses a real-time image 268 obtained from an optical camera (e.g., a detector mounted on the C-arm), which has been processed as described in method 208 of FIG. 5. The graphical user interface 266 includes a user-perceptible orientation 270 for providing user assistance when returning the C-arm to its initial position. The object within the camera's field of view is a sterilization drape set above the phantom on the worktable.

[0074] Figure 9 A graphical user interface 272 is provided, illustrating information extracted from image data (e.g., image features such as contrast). The graphical user interface 272 also discloses real-time images 274 obtained from an optical camera (e.g., a detector mounted on a C-arm). As depicted, machine vision is used to extract and utilize contrast information from different portions of the image. Analysis of the contrast images is not displayed to the user.

[0075] Figure 10 This is a schematic diagram of process 276 for detecting a reference used for motion estimation and guidance. Process 276 includes recording information (box 278) when or after the C-arm is in a reference position. This information may include an RGB image 280 and a depth image 282 from an optical camera (e.g., mounted on the C-arm). This information may also include feedback 284 from C-arm position sensors located on various components of the X-ray imaging system. This information may also include feedback 286 from accelerometers, gyroscopes, and / or other motion or position sensors. Feedback 284, 286 provides information about position (e.g., pose) and / or the position of the C-arm. The reference information or data is then processed as described above and feature extraction is performed (box 288).

[0076] Process 276 includes obtaining real-time information upon user activation for (via machine vision) returning the C-arm to its reference position or pose (box 290). This real-time information may include RGB image 292 and depth image 294 from an optical camera (e.g., mounted on the C-arm). The real-time information may also include feedback 296 from C-arm position sensors located on various components of the X-ray imaging system. The real-time information may also include feedback 298 from accelerometers, gyroscopes, and / or other motion or position sensors. Feedback 296, 298 provides information about position (e.g., pose) and / or the position of the C-arm. This real-time information or data is then processed as described above and feature extraction is performed (box 300).

[0077] Process 276 includes comparing real-time position / motion information with reference information (box 302). Process 276 also includes merging the desired information with the transmitted image from the real-time optical camera (box 304). In some embodiments, process 276 includes displaying a reference image and highlighting features (e.g., edges) seen within the real-time transmitted image, as shown in graphical user interface 306. In some embodiments, process 276 includes displaying the real-time transmitted image and user-perceptible instructions (e.g., distance and orientation) to assist the user in moving the C-arm as shown in graphical user interface 308. In some embodiments, process 276 includes displaying additional relevant alignment and movement information, as indicated by reference numeral 310.

[0078] The technical effects of the disclosed embodiments include providing an easier and clearer way to return the C-arm to a previous (e.g., initial) position. The technical effects of the disclosed embodiments provide user instructions for the trajectory required to return the C-arm to the previous position, including but not limited to visual cues related to the direction of travel, distance traveled, and items to be checked for correctness. This user-aided information simplifies the workflow for returning to the same position. The technical effects of the disclosed embodiments include using the disclosed technology to virtually move the X-ray image to be virtually moved as the C-arm changes position to show an approximate location for the next X-ray imaging. The technical effects of the disclosed embodiments include eliminating the need for fluoroscopic navigation.

[0079] Referring to the technology presented herein and protected by the claims, and applying it to physical objects and concrete examples of practical nature, which explicitly improves the present art, and therefore is not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to the end of this specification contains one or more elements designated as “means for [performing]…” or “steps for [performing]…”, such elements are intended to be interpreted according to 35U.SC112(f). However, for any claim containing elements designated in any other manner, such elements are not intended to be interpreted according to 35U.SC112(f).

[0080] This written description uses examples to disclose the subject matter of the invention, including best practices, and also enables those skilled in the art to practice the subject matter, including making and using any apparatus or system and performing any included methods. The patent scope of the subject matter is defined by the claims and may include other examples that would occur to those skilled in the art. Such other examples are intended to fall within the scope of the claims if they have structural elements that are not indistinguishable from the literal language of the claims, or if they include equivalent structural elements that have minor differences from the literal language of the claims.

Claims

1. An X-ray imaging system (100), the X-ray imaging system comprising: X-ray radiation source (105); X-ray detector (107); and A C-arm (110) having an X-ray radiation source (105) disposed at a first end and a second end disposed opposite to the first end. X-ray detector (107), wherein the X-ray imaging system (100) is configured to translate the C-arm (110) in multiple different directions and rotate the C-arm (110) about multiple different axes; An optical camera (162) is configured to capture image data of the position of the C-arm (110) relative to a certain location; and A controller (150) includes a memory (154) and a processing system (152) including one or more processors, and the controller (150) is configured to receive the image data from the optical camera (162) and to determine the initial position of the C-arm (110) relative to the position using machine vision and to guide the C-arm (110) back to the initial position based on the image data after the C-arm (110) has moved from the initial position.

2. The X-ray imaging system (100) according to claim 1, wherein the optical camera (162) is mounted on the C-arm (110).

3. The X-ray imaging system (100) according to claim 1, wherein the controller (150) is configured to process initial image data received from the optical camera (162) to identify image features, enhance the identified image features, and utilize the enhanced image features as reference features for returning the C-arm (110) to the initial position.

4. The X-ray imaging system (100) of claim 3, wherein the controller (150) is configured to receive input from a user to activate the use of machine vision, thereby guiding the C-arm (110) back to the initial position.

5. The X-ray imaging system (100) of claim 4, wherein the controller (150) is configured to process real-time image data received from the optical camera (162) to identify subsequent image features, enhance the identified subsequent image features, and compare the enhanced subsequent image features with the reference features.

6. The X-ray imaging system (100) of claim 5, further comprising a display (165), wherein the controller (150) is configured to display both the real-time image data on the display (165) and a user-perceptible command to guide the C-arm (110) back to the initial position based on a comparison of the enhanced subsequent image features with the reference features.

7. The X-ray imaging system (100) of claim 6, wherein the user-perceptible command includes a direction of movement, a distance to a target, or a combination thereof.

8. The X-ray imaging system (100) of claim 5, further comprising a display (165), wherein the controller (150) is configured to display both the real-time image data and a reference image having the reference features that help the user guide the C-arm (110) back to the initial position on the display (165).

9. The X-ray imaging system (100) of claim 5, further comprising a display (165), wherein the controller (150) is configured to display on the display (165) the real-time image data, a user-perceptible instruction to guide the C-arm (110) back to the initial position based on a comparison of the enhanced subsequent image features with the reference features, and a reference image having the reference features that assist the user in guiding the C-arm (110) back to the initial position.

10. The X-ray imaging system (100) of claim 5, wherein the controller (150) is configured to provide control signals to move the C-arm (110) and / or the X-ray imaging system (100) to the initial position based on a comparison of the enhanced subsequent image features with the reference features.

11. The X-ray imaging system (100) according to claim 5, wherein the initial image data and the real-time image data include red-green-blue images and / or depth images.

12. The X-ray imaging system (100) of claim 5, wherein the image features and the subsequent image features include sharp edges, significantly bright or dark areas, distinctive shapes or objects, and / or contrast information.

13. The X-ray imaging system (100) of claim 5, further comprising a movable base (140) configured to move the X-ray imaging system (100), wherein the controller (150) is configured to calculate the global positioning of the C-arm (110) during the movement of the X-ray imaging system (100) from the initial position via the movable base (140), then map the environment based on the image data to form a trajectory relative to the floor where the movable base (140) is located, and utilize the trajectory when guiding the C-arm (110) back to the initial position.

14. The X-ray imaging system (100) of claim 13, further comprising a sensor (166) disposed on a component of the X-ray imaging system (100) for determining the position and movement of the C-arm (110), and wherein the controller (150) is configured to receive feedback from the sensor (166) to determine the position and movement of the C-arm (110) and to estimate the movement of the movable base (140).

15. The X-ray imaging system (100) of claim 14, wherein the sensor (166) comprises one or more of a C-arm position sensor, an accelerometer, and a gyroscope.

Images