Tomosynthesis imaging system comprising guidance system with x-ray tomosynthesis registration and tracking

Abstract

The invention discloses a tomosynthesis imaging system including a guidance system with X-ray tomosynthesis registration and tracking. The imaging system may include a cone-beam tomosynthesis imaging system configured to create a 3D image of the patient's anatomy upon initiation of the input. The system may also include at least one memory device having instructions that, when executed by the at least one processor, cause the tomosynthesis imaging system to perform a plurality of actions including: obtaining a 3D image from the tomosynthesis imaging system; obtaining a previous 3D image of the patient's anatomy; and registering the 3D image with a previous 3D image to produce a synthesized 3D image. The navigation system may also include an interface system that accepts an activation input to obtain a 3D image and presents at least a portion of the synthetic 3D image for review by a supervisor.

Description

Background Technology

[0001] Registration methods in surgical procedures allow 3D images (such as CT, CBCT, or MRI 3D images) to be registered to tracking systems (such as optical tracking systems or surgical robotic systems). This allows for accurate navigation of the tracking system, the operator, and / or the robotic system. Various registration methods exist; however, these methods often suffer from drawbacks such as high computational cost, excessive X-ray exposure, and low accuracy. One registration method is pairwise point registration. In this method, points are selected on the image and then traced by the tracking system. Registration is achieved by finding a transformation that optimally aligns the points in image space with those in tracking space. Another registration method is surface mapping, where a point cloud of points is collected by sliding the tracking instrument across a surface (such as skin or bone). The surface is found in the image via a process such as segmentation, where registration is achieved by finding a transformation that optimally aligns the image surface with the point cloud dataset.

[0002] Other registration methods include optically based surface mapping, 2D / 3D registration, and automatic registration. Optically based surface mapping is similar to surface mapping, but instead of collecting a point cloud of points, the 3D optical system collects a cloud of points on the surface in tracking space to obtain the surface in tracking space. In 2D / 3D registration, a discrete number of 2D images (such as 2D images from fluorescein projections) are used to register the 3D image. In this case, the fluorescein C-arm can be tracked to know the projection geometry of the 2D image in tracking space, where registration is achieved by finding a transformation that best aligns the virtual projection of the 3D image with the fluorescein projection. In automatic registration, the intraoperative 3D imaging system is tracked in camera space, where the position of the image relative to the position of the imaging system is known. Therefore, the position of the image in tracking space is known.

[0003] However, each of these registration methods has certain drawbacks. For example, paired-point registration is a manual process, which can increase surgical time, is prone to errors, and may lead to registration errors when too few points are collected. Furthermore, this method cannot actually be performed in vivo and is therefore unsuitable for minimally invasive surgery near regions of interest. Surface mapping may be inaccurate if the surface is not rigid (i.e., dependent on applied pressure) or if the surface segmented from the 3D image does not match the surface being touched. This method also cannot be performed in vivo and is therefore unsuitable for minimally invasive surgery near regions of interest.

[0004] Similarly, optical surface mapping requires well-cleaned bone, which can be time-consuming. Like other methods, optical surface mapping cannot be practically performed in vivo and is therefore unsuitable for minimally invasive surgery near the region of interest. Furthermore, 2D / 3D registration requires movement of the C-arm at multiple discrete locations to achieve accurate registration, and also requires some ionizing radiation. Finally, while automated registration eliminates the need for manual steps, the quality of intraoperative images may not be as good as that of preoperative CT. Although registration is fast, image acquisition and reconstruction are not, and also require significant ionizing radiation. It is also important to note that all the methods discussed above for registering prior 3D images (such as CT images) suffer from the problem that even if the registration method is perfect, the patient's anatomy may not match the registered image during follow-up. For these and other reasons, improvements in imaging and registration during patient procedures continue to be sought. Summary of the Invention

[0005] An imaging system may include a cone-beam computed tomography (CBCT) system configured to create 3D images of a patient's anatomy upon initiation input. The imaging system may also include at least one memory device having instructions, when executed by at least one processor, to cause the CBCT system to perform a plurality of actions, including: acquiring a 3D image from the CBCT system; acquiring a prior 3D image of the patient's anatomy, wherein the prior 3D image is at least one of a higher quality image produced by the CBCT system, or generated via an imaging system different from the 3D image. The instructions may also cause the system to register the 3D image with the prior 3D image to produce a synthesized 3D image. The navigation system may also include an interface system configured to accept initiation input to acquire the 3D image and to present at least a portion of the synthesized 3D image for review by a supervisor.

[0006] Another example of this disclosure is a method for surgical navigation using a tracked patient reference. The method may include acquiring a 3D image from a tomographic synthesis imaging system and acquiring a prior 3D image of the patient's anatomy, wherein the prior 3D image is at least one of a higher quality image produced by a cone-beam tomography synthesis imaging system, or generated via an imaging system different from the 3D image. The method may also include registering the 3D image with the prior 3D image using the tracked patient reference to produce a synthetic 3D image. Furthermore, the method may include presenting at least a portion of the synthetic 3D image to a supervisor.

[0007] Therefore, more important features of the invention have been outlined rather broadly in order to better understand the following detailed description and its contribution to the art. Other features of the invention will become clearer from the following detailed description of the invention in conjunction with the accompanying drawings and claims, or may be learned through practice of the invention. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of an imaging system according to an example of the present technology, including a patient bed supporting the patient to be imaged.

[0009] Figure 2A This is a schematic diagram of a hybrid imaging system according to an example of this technology.

[0010] Figure 2B It is based on Figure 2A One example shows a schematic diagram of the hybrid imaging system in its maximum upper position.

[0011] Figure 2C It is based on Figure 2A One example shows a schematic diagram of the hybrid imaging system in its maximum lower position.

[0012] Figure 3 This is a perspective view of a stereoscopic camera and a robotic arm including tracking markers, according to an example of this technology.

[0013] Figure 4 This is a flowchart illustrating an example method of surgical navigation for a patient according to an example of this disclosure.

[0014] These figures are provided to illustrate various aspects of the invention and are not intended to limit the scope in terms of size, material, configuration, arrangement, or proportion, unless otherwise specified in the claims. Detailed Implementation

[0015] While these exemplary embodiments have been described in sufficient detail to enable those skilled in the art to practice the invention, it should be understood that other embodiments may be implemented and various changes may be made to the invention without departing from the spirit and scope thereof. Therefore, the following more detailed description of embodiments of the invention is not intended to limit the scope of the claimed invention, but is provided for illustrative and non-limiting purposes only to describe the features and characteristics of the invention, to elucidate the best mode of operation of the invention, and to enable those skilled in the art to fully practice the invention. Therefore, the scope of the invention is defined only by the appended claims.

[0016] definition

[0017] The following terms will be used in describing and claiming protection for this invention.

[0018] Unless the context clearly specifies otherwise, the singular forms “a,” “an,” and “the / said” include plural indicators. Thus, for example, a reference to “x-ray source” includes a reference to one or more such devices, and a reference to “obtain” is a reference to one or more such actions.

[0019] As used herein, "basic" means that the deviation is small enough not to cause a measurable impairment to the identified property or situation. In some cases, the exact degree of deviation that is permissible may depend on the specific context.

[0020] As used herein, the term "approximately" is used to provide the elasticity and imprecision associated with a given term, measure, or value. Those skilled in the art can readily determine the degree of elasticity of a particular variable. However, unless otherwise stated, the term "approximately" generally indicates an elasticity of less than 2%, and typically less than 1%, and in some cases less than 0.01%.

[0021] As used herein, for convenience, multiple items, structural elements, constituent elements, and / or materials may be presented in a public list. However, these lists should be interpreted as each member of the list being individually identified as a separate and unique member. Therefore, without indication to the contrary, a single member in this list should not be interpreted as a de facto equivalent to any other member of the same list simply because they appear in a public group.

[0022] As used herein, the term “at least one of…” is intended to be synonymous with “one or more of…”. For example, “at least one of A, B and C” explicitly includes only A, only B, only C, or a combination of each.

[0023] As used herein, the term "3D image" refers to a volumetric three-dimensional image created by a 3D imaging system. Such images can be displayed or used in whole or in part. For example, a slice of a 3D image can be displayed instead of the entire 3D image.

[0024] As used herein, “degrees of freedom” refers to the independent orientation and orientation of various objects relative to other objects, which can be tracked using the tracking system and reference markers described herein. For example, degrees of freedom may include the six degrees of freedom for moving a rigid object in space, including positional motion along three axes and rotational motion about three axes. Additional degrees of freedom beyond these six may include bending and torsional motions between the bodies of a rigid structure connected by flexible structures.

[0025] Numerical data may be presented in range format in this document. It should be understood that this range format is used solely for convenience and brevity, and should be flexibly interpreted to include not only the values ​​explicitly stated as the limits of the range, but also all individual values ​​or subranges encompassed within that range, as if each value and subrange were explicitly stated. For example, a range of values ​​from approximately 1 to approximately 4.5 should be interpreted to include not only the explicitly stated limits of 1 to approximately 4.5, but also individual numbers (such as 2, 3, 4) and subranges (such as 1 to 3, 2 to 4, etc.). The same principle applies to ranges containing only a single value, such as "less than approximately 4.5," which should be interpreted to include all the values ​​and ranges stated above. Furthermore, this interpretation should be adopted regardless of the breadth or characteristics of the range being described.

[0026] Any step recited in any method or process claim may be performed in any order, and is not limited to the order presented in the claims. For a particular claim limitation, the limitation of device plus function or step plus function is adopted only if all of the following conditions are met: a) it expressly states "device for..." or "step for..."; b) it expressly states the corresponding function. The structures, materials, or actions supporting the device plus function are expressly described in the description herein. Therefore, the scope of the invention should be determined solely by the appended claims and their legal equivalents, and not by the description and examples given herein.

[0027] Example Implementation

[0028] Visual images of a patient's anatomy can be obtained through scanning procedures such as X-rays, computed tomography (CT), cone-beam CT (CBCT), and magnetic resonance imaging (MRI). These methods can be used to image the surgical site before surgery and can be used periodically during surgery. However, these methods cannot be used for continuous monitoring of the patient's anatomy, and the continuous repetition of scanning throughout the surgical procedure is often impractical. Therefore, scanned images can be combined with surgical navigation systems to provide visualization of surgical instruments or implants. Such devices can be displayed as an overlay on images of the patient's anatomy for the surgeon to view, or as input to robotic or intelligent surgical systems to perform, for example, surgical planning or surgical analysis. This overlay can be accurately positioned and represented relative to the image via a registration process. Because the patient's anatomy can move relative to a tracking camera, the anatomy can be tracked to maintain correct registration between the surgical instruments, the displayed images, and the actual anatomy.

[0029] The imaging and registration system proposed in this paper can be used to register 3D images to a tracking system and update them periodically throughout the patient procedure. Specifically, 3D images can be registered to an optical tracking system or a surgical robotic system. This allows the tracking system or surgical robotic system to navigate accurately. In other words, it allows the tracking system to display the orientation of surgical instruments as a superposition on the 3D image.

[0030] This disclosure allows for the joint display of previous 3D images and tomographically synthesized 3D images in an overlay once the images are registered. This allows for the representation of objects present during the procedure, or those not present in previous images, in the images, resulting in more accurate navigation images and a better representation of the current situation. These objects can then be used to exclude certain areas to ensure that registration is performed only on anatomical information and not on foreign bodies that might reduce registration accuracy.

[0031] Now go to Figure 1The description pertains to navigation system 100. During surgical procedures, imaging and navigation system 100 can be used to display one or more surgical instruments 122 as an overlay on a 3D image. This can assist surgeons or robotic systems in performing surgical procedures. Imaging and navigation system 100 may include imaging system 102. Imaging system 102 is a cone-beam computed tomography (CBCT) system, i.e., a tomographic synthesis system capable of generating tomographic images during imaging, utilizing an X-ray source held on one side of a plane (X-ray source hemisphere) where the object being imaged is located, and an X-ray detector held substantially opposite to the X-ray source (X-ray detector hemisphere) during data acquisition, the two hemispheres not intersecting and not overlapping. Exemplary cone-beam computed tomography (CBCT) systems are described in U.S. Patent Nos. 10,070,828; 10,846,860; and 12,062,177, which are incorporated herein by reference. One example of an imaging system is a tomographic synthesis system, which is a hybrid system. Furthermore, if tomographic images are provided, the system can provide one or both of fluorescence microscopy or CBCT imaging. Cone-beam tomography can utilize proprietary reconstruction and visualization techniques to create near real-time 3D images via rapid reconstruction of sequentially detected 2D projections. In some examples, cone-beam tomography is either cone-beam fluorescence microscopy (CBTF) or cone-beam computed tomography (CBCT). CBTF is an imaging technique that provides real-time images of a target anatomical structure. However, the images provided by CBTF lack depth information, or in other words, are 2D. CBCT is a full 3D imaging technique that allows reconstruction using isotropic voxels and high spatial resolution, thereby allowing precise measurement or imaging of selected anatomical structures in all three orthogonal planes. Imaging system 102 can be a 3D C-arm device 104, including an X-ray source assembly 106 attached to the lower part of the C-arm device 104 and a detector array 108 attached to the upper part of the C-arm device 104.

[0032] The imaging system 102 can be oriented to collect 3D images from a target image-forming area 124. This image-forming area 124 can be directed toward a patient lying on a support table 126. For coarse adjustments, one or both of the support table 126 and the imaging system 102 can be moved relative to each other. This can be achieved by manually moving one or both of the support table and the imaging system (e.g., via wheels). For finer adjustments to the orientation and orientation of the image-forming area, the imaging system itself can be adjusted. For example, the C-arm support frame can be rotated, the X-ray source can be tilted, and / or the detector can be adjusted. The support table 126 may optionally include side rails 128, which can serve as mounting orientations for additional equipment, as handles for moving the support table, or as safety guides to prevent patient injury.

[0033] In one example, imaging system 102 can create 3D images of a patient's anatomical structures. These structures can be located in any anatomical region, such as, but not limited to, the spine, hip, knee, hand, wrist, foot, ankle, shoulder, bone fragments, skull, etc. In some examples, imaging system 102 can automatically create 3D images based on user input or by a coded decision maker. Input can be pressing a button, stepping on a foot pedal, a voice command, etc. Furthermore, in examples where 3D image creation is automatically generated by a coded decision maker, input can be generated via a preset list of instructions on when to create the 3D image, a timer system for creating the 3D image at a predetermined time, or by using artificial intelligence (e.g., dynamically as needed based on experience and input criteria). For the actual image created, in some cases, imaging system 102 provides volumetric imaging data, i.e., reconstructing a 3D volume and providing volumetric data. This volumetric data can be presented as various views on a display for a human user, or it can be used directly by an algorithm or artificial intelligence without generating views on a display.

[0034] The imaging and navigation system 100 may also include at least one memory device 110 and at least one processor 112. The memory device 110 may have instructions that, when executed by the processor, cause the imaging system 102 to perform multiple actions. The imaging and navigation systems 100 may be instructed to acquire 3D images of the patient's anatomy from the imaging system 102. The 3D images may be updated or current images, reflecting changes in the position of tissues and / or surgical instruments. In some examples, short-angle acquisition is used to acquire 3D images. This short-angle acquisition may involve rotating the X-ray source along a finite arc, thereby collecting a limited amount of data. This finite arc may vary depending on the desired resolution or acquisition time. However, as a general guideline, the finite arc can range from 5° to less than 360°, in some cases from 10° to 45°, and in others from 15° to 30°. Similarly, the X-ray source may traverse this finite arc once per acquisition, or multiple times per image acquisition. In any case, this may result in lower image quality, but may still provide a coarse update of changes in the image-forming region 124.

[0035] The imaging and navigation system 100 can also be instructed to acquire prior 3D images of the patient's anatomy. The prior 3D images can be higher quality 3D images (e.g., CBCT images, or images with higher resolution or higher power) produced by the imaging system 102, or they can be generated via an imaging system different from the 3D images. For example, the patient can be imaged using a high-resolution imaging modality during an early visit (e.g., on the same day or the previous day). Examples of prior 3D images being higher quality images or having different characteristics from 3D images can include, but are not limited to, CT, CBCT, or MRI images. In some examples, prior 3D images can be obtained from a previous CT imaging session. In other examples, prior 3D images can be synthetic 3D images from earlier iterations using the same imaging system used to generate the 3D images.

[0036] The imaging and navigation system 100 can also be instructed to register a 3D image from the imaging system 102 with a previous 3D image. This produces a synthetic 3D image that represents a combination of features from the previous 3D image and the newly acquired 3D image. In some cases, the synthetic 3D image can be a superposition of the 3D image onto the previous 3D image. In this case, the image may include highlights of different features between the two images, such as the presence of surgical instruments 122 or implants within the patient, changes in tissue position, etc. In other cases, the synthetic 3D image can be an integrated image of the 3D image and the previous 3D image. An integrated synthetic 3D image can be produced by morphing the previous 3D image onto the 3D image to match the patient's updated anatomy. Morphing the previous 3D image onto the 3D image can include isotropic scaling, anisotropic scaling, shape deformation, spline transformation, etc. Furthermore, since registering the 3D image with the previous 3D image uses a very rich intraoperative dataset, the morphing can use at least six degrees of freedom to improve the matching of the previous 3D image with the patient's anatomical position during surgery. In this context, the integrated synthetic 3D image retains the quality of the previous 3D image while modifying the image to reflect new elements and changing features to reflect the current situation.

[0037] In one example, a 3D image can be reconstructed using an iterative reconstruction scheme by using a previous 3D image or a variant thereof as a seed for an iterative reconstruction algorithm to produce an integrated image. This can reduce certain artifacts, such as tomographic artifacts. In some cases, the seed from the previous 3D image is first registered so that the seed is placed in the correct space for reconstruction. For example, such registration can be achieved by storing the position of the imaging system when the first image was captured. When a newer or later tomographic synthetic image is captured, the first image can be used as a seed for the tomographic synthetic reconstruction algorithm by virtually positioning the seed in a new reference frame of the imaging system given its new position. If a patient is being tracked, these positions can be relative to the patient, or they can be relative to the stage. An exemplary reconstruction technique is outlined in U.S. Patent No. 11,610,346, which is incorporated herein by reference, but any suitable image reconstruction technique, such as, but not limited to, filtered backprojection or similar tomographic techniques, can be used.

[0038] In some examples, the imaging and navigation system 100 may also include an interface system 114. The interface system 114 may be configured to accept initiation input, such as initiation input from a user, which causes the imaging system 102 to create a 3D image of the patient's anatomy. The interface system 114 may be any suitable input device. Non-limiting examples of suitable input devices may include a keyboard, a touchscreen, manual buttons, or, in the case of an automated supervisor, the input device may be the processor 112 itself. Furthermore, the interface system 114 may also be configured to acquire 3D images from the imaging system 102 and present at least a portion of the synthesized 3D image for review by a supervisor, thereby allowing intraoperative decision-making. In some cases, the supervisor may be a human user, such as a surgeon, physician assistant, nurse, etc. In other cases, the supervisor may be a coded decision maker, such as an automated computer system, a pre-programmed decision model, artificial intelligence, etc., or a combination thereof. In some examples, the interface system 114 may be a display 130 on which the synthesized 3D image is presented to the user. In other examples, the interface system 114 may directly present the synthesized 3D image to the coded decision maker. In another example, interface system 114 can provide the encoding decision maker with information describing modifications to the previous image without explicitly providing the synthesized image. The interface system (e.g., including a processor, memory device, display, etc.) can be connected to the imaging system and camera via a connector. The connector can be wired or wireless (e.g., Bluetooth, Wi-Fi, Zigbee, etc.).

[0039] An example of an imaging system is a tomographic synthesis imaging system, which is a hybrid system. Furthermore, if tomographic synthesis images are provided, the system can provide one or both of fluorescein or CBCT imaging. A hybrid system can include a single X-ray source or multiple X-ray sources. In one example, a common X-ray source is used to generate both cone-beam computed tomography (CBCT) and fluorescein images. In this way, the common X-ray source is configured to remain stationary during fluorescein imaging. Therefore, a single X-ray source can be used. In another alternative example, the hybrid system can include at least one rotatable tomographic synthesis X-ray source and at least one central X-ray source. In this case, at least one central X-ray source can be used to generate fluorescein images, while at least one rotatable X-ray source can generate tomographic synthesis images. Figure 2A This is an example of such a hybrid imaging system 202, which has a first stationary x-ray source 204 and a second rotating x-ray source 208 as part of an x-ray assembly 209. The hybrid system may include at least one tomographic synthesis x-ray source and at least one central beam x-ray source. The term "stationary" is intended to specify that the x-ray source is stationary relative to the imaging system 202 and the image forming region 214 during imaging to produce a fluorescein image. Therefore, the stationary x-ray source may be permanently fixed in a particular orientation or temporarily locked in a position during imaging. In some examples, the first x-ray source 204 may be stationary and attached to the lower part of a C-arm frame 220. This x-ray source generates a first stationary x-ray beam 216 directed at a detector array 218. Typically, the detector array 218 may be fixed to the upper part of the C-arm frame 220. The second x-ray source 208 may be fixed within a hollow rotary table or bracelet-shaped component 210 with a continuous annular track, thereby allowing the second x-ray source 208 to rotate along the annular track. The rotating second X-ray source 208 enables the imaging of patient anatomy from various angles. In one example, a hollow rotary stage 210 may surround or enclose a stationary first X-ray source 204, which is located within the hollow portion of the hollow rotary stage 21. An optional hollow slip ring 211 may house power transmission and other electronic equipment for controlling the X-ray source. The second X-ray source 208 generates a rotating X-ray beam 222 (e.g., rotating within a plane of rotation) that passes through the image-forming region 214 to produce a tomographic composite image. In another alternative, a high-voltage power supply 224 may be installed alongside the rotating X-ray source 208. The hollow rotary stage allows the X-ray beam of the central X-ray source to be unobstructed by the rotating source.

[0040] It is worth noting that the C-arm frame 220 can be secured to the base 226 via the C-arm attachment 206. The C-arm attachment 206 may include a coupling mechanism that not only holds the C-arm frame 220 in place but also allows the C-arm frame to slide up and down along the frame body. In any of the above configurations, the C-arm frame 220 can slide up and down relative to the C-arm attachment 206 to allow the track to rotate to generate CBCT images. This allows adjustment of the image forming area 214 relative to the patient (e.g., particularly by changing the relative angle) and facilitates data collection for CBCT image acquisition. As a further explanation, Figure 2B The C-arm frame 220 is shown in its maximum upper position, with the C-arm frame 220 sliding upward along the attachment 206 to the base 226. At this time, the attachment 206 is oriented adjacent to the X-ray source assembly 209. Figure 2C The diagram shows the C-arm frame 220 sliding downwards to its maximum lower position, where the attachment 206 is adjacently aligned with the detector array 218. Of course, the C-arm frame 220 can slide and be oriented at any position along its body. When creating CBCT images, multiple projections can be captured as the C-arm slides from the extreme positions shown in 2B and 2C. In one version, when the imaging system has… Figure 2A As shown in the central beam source 204, the central beam source can be used to generate a projection for CBCT imaging. In another version, the rotating source used for tomography can be essentially stationary to generate a CBCT projection during C-arm sliding, thereby creating trajectory acquisition, as... Figure 2B As shown. In another version, two C-arms can be combined for rotation, where the rotating source first stops at one of the two intersection points between the source's plane of rotation and the plane along which the C-arm slides to initiate the first rotation. Figure 2C Position 1), and then the same source stops at another intersection for a second rotation ( Figure 2C Position 2).

[0041] Return to reference Figure 1In some examples, the imaging system 100 may also include a tracking camera 116. The tracking camera 116 may be one or both of a visible light camera and an infrared camera. In other examples, multiple cameras or lenses may be used. In further examples, the tracking camera 116 may be a stereo or stereoscopic camera. The tracking camera 116 may be a standard NDI navigation camera, an RGB-D camera, etc. Regardless of the type of tracking camera 116 used, it can be configured to track at least one of the surgical instruments 122, the patient, or the imaging system 102. In one example, the tracking camera 116 may track these objects by using reference markers 118 (such as luminous LEDs) or via reflector markers located at a fixed orientation on at least one of the patient, surgical instruments, or the imaging system 102. Alternatively, the tracking camera 116 may be used to track these objects via image recognition software (i.e., algorithms, models, etc., including AI-based learning models). In this case, the tracking camera 116 may transmit captured images to a processor, where the objects (i.e., specific parts of the instruments, imaging device 102, and / or patient anatomy) are identified.

[0042] In some cases, such as during lengthy surgical procedures or when additional precision is required, updated 3D images can be acquired to ensure that the current navigation efforts accurately guide the procedure to the desired tissue. Ideally, continuous or near-continuous imaging would provide the greatest accuracy. However, this is often impractical due to computational load, image acquisition time, and could result in undesirable excessive X-ray exposure for the patient. Therefore, in some examples, the imaging system 102 periodically creates new 3D images of the patient's anatomy (e.g., at automatic intervals, dynamically optimized intervals based on modeling, or manually triggered by the clinician). The new 3D images are then re-registered with previous 3D images or previously synthesized 3D images, creating updated registrations. This allows for the tracking of surgical instruments 122 or multiple surgical instruments based on the updated registrations. Thus, the imaging system 102 can create new 3D images from computer input, etc., at predetermined time intervals after an initiation input from the user, using artificial intelligence or similar methods.

[0043] In some examples, the imaging system 102 can be calibrated prior to intraoperative imaging. This allows the patient's image to be in a known position relative to a reference marker 118 of the imaging system 102. In other examples, the patient's position can be provisionally determined by objects of known shape, visible in the image, and tracked by the tracking camera 116 during or immediately before intraoperative imaging. In one example, registering a 3D image with a previous 3D image includes camera-based registration. This includes at least one of pre-calibrated camera position registration and object-based registration utilizing optical images obtained using the tracking camera 116. In some examples, the tracking camera 116 is a stereoscopic tracking camera. In another example, registering a 3D image with a previous 3D image includes image-based registration, where common features between the 3D image and the previous 3D image are correlated. Image-based registration can be achieved via iterative methods, such as gradient descent or other techniques that solve inverse problems. These problems are typically solved by minimizing a cost function, as this cost function may differ for different types of imaging being registered. For example, when registering images from MRI to X-ray-based tomographic synthesis, a similarity metric can be used instead of the traditional LN norm. Registration can be 3D degree-of-freedom registration, rigid registration, or deformable registration with more degrees of freedom.

[0044] In one example, registering a previous 3D image with another 3D image also includes defining a selected local portion of the previous 3D image. In this way, the previous 3D image is modified only within the selected local portion, and a synthetic 3D image is generated using a local matching metric. As a result, the computational load is reduced, and no computational effort is spent updating image portions that are not critical to the procedure. The local matching metric can be dynamically calculated over a local region of the surgical instrument so that the registration changes and updates continuously as the surgical instrument 122 moves. In some examples, the local matching metric is an L1-norm or L2-norm cost function. However, other matching metrics can be used to align the 3D image with the synthetic image.

[0045] In one example, the navigation system 100 also includes a robotic arm 120. The robotic arm can be configured to manipulate surgical instruments 122. It may also include a command input operably connected to a supervisor and configured to receive instructions from the supervisor. In some examples, the surgical instruments 122 may include one or more of implants, drills, cutters, forceps, needle actuators, lasers, scissors, clamp applicators, cauterizers, hooks, etc. Furthermore, the robotic arm 120 can move and position relative to a synthesized 3D image to position the surgical instruments 122 where needed. To further explain this process, Figure 3A close-up of the robotic arm 120 and stereoscopic camera 116 is shown. The robotic arm 120 may have tracking markers 304 attached individually or in an array. In some examples, the robotic arm 120 may have a tracking marker array 118 and an image positioning array 308. The tracking marker array 118 can be tracked using the tracking camera 116. The image positioning array 308 can be positioned via the detector 108 of the imaging system 102 (…). Figure 1 The robot arm 120 can then move and be positioned using the tracking camera 116. In this way, the tracking marker array 118 can be used to register the tool's orientation visually relative to the synthetic 3D image. Therefore, the image positioning array 308, whether alone or attached to the tracking marker array 118, can be used to align the tool's image within the 3D image and / or the 3D synthetic image. The robot can execute a predetermined plan based on a previous 3D image or based on a recent synthetic image. In another version, the predetermined plan based on a previous image can be modified or updated according to the registration process to conform to the new arrangement of the anatomical structures seen in the synthetic image. In many cases, the user will supervise and control this new plan and may allow modifications to it before the robot executes the plan.

[0046] The present invention also describes a supplementary method for patient surgical navigation. Figure 4 This is a flowchart illustrating an example method 400 for surgical navigation of a patient. In some examples, method 400 may use patient reference tracking of the patient's anatomy. Method 400 may include obtaining a 3D image 410 from a tomographic imaging system and obtaining a previous 3D image 420 of the patient's anatomy. In some examples, the imaging system may be a tomographic imaging system, a CT imaging system, an MRI imaging system, or any other type of imaging system that produces a 3D image. In some examples, the 3D image is a previous 3D image, a 3D tomographic image, a computed tomography image, or a combination of these images. In some examples, the previous 3D image may be a higher quality image produced by a cone-beam tomography system than the 3D image, or the previous 3D image may be produced via an imaging system different from the 3D image. An example of a higher quality image is when the tomographic image is first taken at a higher resolution or using additional radiation. An example of a previous 3D image having different characteristics from the 3D image is when it is a CT, CBCT, or MRI image. In some examples, the previous 3D image may be obtained from a previous CT imaging session. In other examples, the previous 3D image may be a synthesized 3D image from an earlier iteration.

[0047] In one example, method 400 may further include registering the 3D image with a previous 3D image 430. Registration can be performed by aligning tracking reference markers at identifiable and / or trackable locations between each of the 3D image and the previous 3D image to produce a synthetic 3D image. This registration can be achieved using one or both of tracking markers and object recognition. When using tracking markers, these devices can be oriented over one or more of an imaging device, a patient, an optical camera, and surgical instruments, as previously described. Tracking reference markers allow the 3D image and the previous 3D image to be correctly aligned, which is necessary for producing an accurate synthetic 3D image. Suitable tracking reference markers may include, but are not limited to, one or more luminous LEDs or reflector markers. Object recognition can also be used alone or in combination with physical tracking markers. For example, object recognition may include image recognition software, AI-driven models, etc., to identify common features in each image. These common features can then be aligned by registering to a common point or a set of points. For example, the Hough transform can be used to identify features, and these features can be analyzed for commonalities between the two images. Similarly, object detection machine learning algorithms can be used. The frame of reference can be relative to the initial image, or relative to the new image, or relative to the physical patient.

[0048] In one example, a composite 3D image is created by overlaying a 3D image onto a previous 3D image. To help clinicians identify differences, the composite 3D image can include highlights of distinct features between the two images, such as the presence of surgical instruments, implants in the patient, or displaced tissue. Composite 3D images can be generated by blending certain elements of the new image, overlaying them, replacing the content of the first image with the content of the second image, or using a function that combines elements of the two images. The composite image can alter the grayscale levels of the new image or add color to highlight elements of the new image within the previous image, using highlights to show differences. In another example, a composite 3D image can be an integrated image of a 3D image and a previous 3D image. This integrated composite 3D image can be generated by warping the previous 3D image to match the patient's updated anatomy. The warping can be done by starting with the previous 3D image and applying a warping algorithm to match the 3D image (i.e., the updated image), or by starting with the 3D image and warping it using the previous 3D image. Deformations of the 3D image to the previous 3D image can include isotropic scaling, anisotropic scaling, shape deformation, spline transformation, manifold transformation, etc. Furthermore, since registering the 3D image to the previous 3D image utilizes a very rich intraoperative dataset, deformations can use at least six degrees of freedom to improve the match between the previous 3D image and the patient's anatomical position during surgery. For example, deformations can be used to interpret brain displacement in cranial nerve procedures or spinal curvature in spinal procedures. The representation of the deformation itself can also be valuable. In this case, vector fields can be used to display the deformation transformation and show the surgeon how the image changes. When using non-rigid registration navigation, the deformed 3D image can be further deformed during navigation, for example, by using distributed patient references, such as those described in U.S. Patent Application Publication No. US-2023-0311014-A1, which is incorporated herein by reference.

[0049] In a further example, method 400 may include presenting the synthetic image 440. In some cases, only a portion of the synthetic image may be presented. In this case, the presented portion may represent a finite subset of the entire synthetic image. In other cases, the entire synthetic image may be presented. In some examples, the synthetic image may be presented to a supervisor, who is one or both a human user and a coded decision maker. In some cases, the supervisor may be a human user, such as a surgeon, physician assistant, nurse, etc. In other cases, the supervisor may be a coded decision maker, such as an automated computer system, a pre-programmed decision model, artificial intelligence, etc., or a combination thereof.

[0050] In another example, method 400 may further include tracking the tomographic synthetic imaging system to generate a tracking reference orientation via one or both of the tracking markers and object identification as described above. More specifically, tracking can be accomplished using a camera. The camera can be one or both of a visible light camera and an infrared camera. In other examples, multiple cameras or lenses may be used. In further examples, the camera may be a stereo or stereoscopic camera. The tracking camera may be a standard NDI navigation camera, an RGB-D camera, etc. In one example, these methods may include stereoscopically tracking the tomographic synthetic imaging system. In some examples, tracking can be performed using reference markers. The reference markers may be attached to one or more of the tomographic synthetic imaging system, the patient, surgical instruments, etc.

[0051] Although the flowchart presented for this technology implies a specific execution order, the execution order may differ from the order shown. For example, the order of two or more boxes may be rearranged relative to the order shown. Furthermore, two or more boxes shown consecutively may be executed in parallel or partially in parallel. In some configurations, one or more boxes shown in the flowchart may be omitted or skipped. For purposes such as enhancing utility, accounting, performance, measurement, troubleshooting, or similar reasons, any number of counters, status variables, warning signs, or messages can be added to the logical flow.

[0052] Certain functional units described in this specification are labeled as modules to more specifically emphasize their implementation independence. For example, modules may be implemented as hardware circuits (including custom VLSI circuits or gate arrays), off-the-shelf semiconductors (such as logic chips, transistors), or other discrete components. Modules may also be implemented in programmable hardware devices such as field-programmable gate arrays, programmable array logic, programmable logic devices, etc.

[0053] Modules can also be implemented in software to be executed by various types of processors. The identified executable code module may, for example, comprise one or more computer instruction frames, which can be organized as objects, programs, or functions. However, the executable files of the identified modules do not need to be physically located together, but can include different instructions stored in different locations containing the modules, and logically linked together to achieve the module's stated purpose.

[0054] In practice, an executable code module can be a single instruction or multiple instructions, and can even be distributed across several different code segments, different programs, and across multiple memory devices. Similarly, operational data can be identified and illustrated within the module herein, and can be represented in any suitable form and organized within any suitable data structure type. Operational data can be collected as a single dataset or can be distributed in different locations, including across different storage devices. Modules can be passive or active, including agents operable to perform desired functions.

[0055] The techniques described herein can also be stored on computer-readable storage media, including volatile and non-volatile, removable and non-removable media, implemented by any technique for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media include, but are not limited to, non-transitory machine-readable storage media, such as RAM, ROM, EEPROM, flash memory or other storage technologies, CD-ROM, Digital Universal Disk (DVD) or other optical storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other computer storage medium used to store the desired information and the techniques described.

[0056] The device described herein may also include a communication connection or networking device and a networking connection that allows the device to communicate with other devices. A communication connection is an example of a communication medium. A communication medium is generally embodied in computer-readable instructions, data structures, program modules, and other data in a modulated data signal (such as a carrier wave or other transmission mechanism), and includes any information delivery medium. A “modulated data signal” means a signal having one or more characteristics set or altered in a manner that encodes information into the signal. By way of example and not limitation, communication media include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, radio frequency, infrared, and other wireless media. As used herein, the term computer-readable medium includes communication media.

[0057] Referring to the examples shown in the accompanying drawings, and in which these examples are described using specific language herein, it will be understood that this is not intended to limit the scope of the technology. Changes and further modifications to the features shown herein, as well as additional applications of the examples shown herein, should be considered within the scope of this specification.

[0058] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more examples. Numerous specific details, such as examples of various configurations, have been provided in the foregoing description to offer a thorough understanding of examples of the described technology. However, it should be recognized that the technology can be practiced without one or more specific details, or using other methods, components, devices, etc. In other instances, well-known structures or operations are not shown or described in detail to avoid obscuring aspects of the technology.

[0059] Although the subject matter has been described in language specific to structural features and / or operations, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features and operations described above. Rather, the specific features and operations described above are disclosed as exemplary forms for implementing the claims. Many modifications and alternative arrangements can be conceived without departing from the spirit and scope of the described technology.

Claims

1. An imaging system, comprising: A cone-beam computed tomography (CBCT) system comprising at least one rotatable tomographic X-ray source and configured to create 3D images of a patient’s anatomy upon activation input; At least one memory device comprising instructions that, when executed by at least one processor, cause the tomographic synthesis imaging system to: The 3D image is obtained from the tomographic synthetic imaging system; Obtain prior 3D images of the patient's anatomy, wherein the prior 3D images are at least one of higher quality images produced by the cone-beam computed tomography system, or produced via an imaging system different from the 3D images; and The 3D image is registered with the previous 3D image to produce a composite 3D image; And an interface system configured to accept the startup input to obtain the 3D image, and to present at least a portion of the synthesized 3D image for review by a supervisor.

2. The imaging system of claim 1, wherein the cone-beam computed tomography (CBCT) system is a hybrid system configured to generate either a fluorescence microscope image or a cone-beam computed tomography (CBCT) image.

3. The imaging system of claim 2, wherein a common X-ray source is used to generate the CBCT image or the fluorescence image, wherein the common X-ray source is configured to remain stationary at a position during at least one of the fluorescence imaging or CBCT imaging.

4. The imaging system of claim 2, wherein the hybrid system further comprises at least one central X-ray source, wherein the at least one central X-ray source is used to generate one or both of the fluorescence microscopy image and the CBCT image.

5. The imaging system of claim 1 further includes a position encoder configured to track the position and orientation of the 3D image relative to the previous 3D image.

6. The imaging system of claim 1, further comprising a stereotactic tracking camera, wherein the stereotactic tracking camera tracks via at least one reference marker located at a fixed orientation on at least one of the patient and the tomographic synthesis imaging system.

7. The imaging system of claim 6, wherein at least one of the following: a. The stereoscopic tracking camera is also configured to track at least one surgical instrument; b. The at least one reference mark is one or more of a light-emitting LED and a reflector mark; and c. The fixed orientation includes both the patient and the tomographic synthetic imaging system.

8. The imaging system of claim 7, wherein the cone-beam computed tomography (CBCT) system periodically creates new 3D images of the patient's anatomy, re-registers the new 3D images with the synthesized 3D images, creates an updated registration, and allows tracking of the at least one surgical instrument based on the updated registration.

9. The imaging system of claim 7, wherein the user interface system displays the orientation of the at least one surgical instrument superimposed on the previous 3D image.

10. The imaging system of claim 7, wherein registering the 3D image with the previous 3D image includes camera-based registration, the camera-based registration including at least one of the following: pre-calibrated camera position registration, and object-based registration using optical images obtained using the stereoscopic tracking camera.

11. The imaging system of claim 1, wherein registering the 3D image with the previous 3D image includes image-based registration, wherein common features between the 3D image and the previous 3D image are correlated.

12. The imaging system of claim 1, wherein the synthesized 3D image is the 3D image superimposed on the previous 3D image, including highlights with different features.

13. The imaging system of claim 1, wherein the synthesized 3D image is an integrated 3D image generated by deforming the previous 3D image onto the 3D image to match the updated anatomy of the patient.

14. The imaging system of claim 13, wherein deforming the previous 3D image onto the 3D image comprises isotropic scaling, anisotropic scaling, shape deformation, spline transformation, or a combination thereof.

15. The imaging system of claim 13, wherein registering the 3D image with the previous 3D image uses at least 6 degrees of freedom to deform the previous 3D image onto the 3D image to produce the synthesized 3D image.

16. The imaging system of claim 1, wherein the prior 3D image is obtained from a previous CT imaging session or from an earlier iteration of the synthetic 3D image; and short-angle acquisition is used to obtain the 3D image from the tomographic synthetic imaging system.

17. The imaging system of claim 1, wherein registering the previous 3D image onto the 3D image further comprises defining a selected local portion of the previous 3D image such that the previous 3D image is modified only within the selected local portion to generate the synthetic 3D image using a local matching metric.

18. The imaging system of claim 1, wherein the supervisor is a user and the interface system is a display on which the synthetic 3D image is presented to the user.

19. The imaging system of claim 1, wherein the supervisor is an coded decision maker, and the interface system presents the synthesized 3D image to the coded decision maker.

20. The imaging system of claim 1, wherein the activation input is configured to be manually generated by the user or automatically generated by the coded decision maker.

21. The imaging system of claim 1, further comprising a robotic arm configured to manipulate surgical instruments, the surgical instruments including a command input operatively connected to the supervisor and configured to receive instructions from the supervisor.

22. A method for surgical navigation of a patient using distributed patient reference tracking of patient anatomy, comprising: Obtain 3D images from a tomographic synthetic imaging system; Obtain a previous 3D image of the patient's anatomy, wherein the previous 3D image is at least one of higher quality images produced by the tomographic synthesis imaging system, or produced via an imaging system different from the 3D image; The 3D image is registered with the previous 3D image using a tracking reference orientation to produce a synthetic 3D image; as well as At least a portion of the synthesized 3D image is presented to the supervisor.

23. The method of claim 22, wherein the 3D image is a previous 3D image, a 3D tomographic image, a computed tomographic image, or a combination thereof.

24. The method of claim 22, further comprising stereotactically tracking the fault synthesis system to generate a tracking reference orientation.

25. An imaging system, comprising: A cone-beam computed tomography (CBCT) system is configured to create 3D images of the patient's anatomy upon initial input. The cone-beam computed tomography synthesis described therein is a hybrid system configured to produce either a fluorescence microscopy image or a cone-beam computed tomography (CBCT) image. A stereotactic tracking camera configured to track at least one of the tomographic synthetic imaging system and at least one surgical instrument; At least one memory device comprising instructions that, when executed by at least one processor, cause the tomographic synthesis imaging system to: The 3D image is obtained from the tomographic synthetic imaging system; Obtain prior 3D images of the patient's anatomy, wherein the prior 3D images are at least one of higher quality images produced by the cone-beam computed tomography system, or produced via an imaging system different from the 3D images; and The 3D image is registered with the previous 3D image to generate a synthetic 3D image, wherein the synthetic 3D image is generated by deforming the previous 3D image onto the 3D image to match the patient's updated anatomy; as well as A user interface system configured to accept the startup input and obtain the 3D image, and present at least a portion of the synthesized 3D image for supervisor review, wherein registering the 3D image with the previous 3D image uses at least six degrees of freedom to deform the previous 3D image onto the 3D image to produce the synthesized 3D image, wherein registering the previous 3D image onto the 3D image is also adaptively adjusted to define selected local portions of the previous 3D image such that the previous 3D image is modified only within the selected local portions to produce the synthesized 3D image using a local matching metric, and wherein at least one tracked surgical instrument is displayed as a superposition of the synthesized image.

Images