Apparatus and method for registering live and scanned images - Patents.com

Abstract

To provide an improved apparatus which enables a surgeon to identify an object and a tissue viewed with a camera during surgery.SOLUTION: The inventive system uses medical imaging and live imaging on non-rigid tissue structures of a patient 12 and increases preciseness and reliability by shaping the patient's body 18 during surgery so that the outer shape of the body of the patient 12 during surgery is identical to the outer shape of the body during imaging. A processing unit 28 will precisely overlay a scan image (or an image or a graphical representation derived from several scan images) and a live image acquired during surgery for enhancing surgeons understanding and orientation during surgery.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an apparatus for surgery on a patient's body and a method for image recording. [Background technology]

[0002] U.S. Patent No. 9,433,387 discloses a system for obtaining cranial or other scans from a patient. The system includes a diagnostic scan table and a mold that conforms to a portion of the patient's body contours. In one embodiment, a mask is provided for encircling the patient's head. A sensor is placed within the mold or mask to obtain pressure readings that can indicate patient movement, such as swallowing. The sensor signal can be used to avoid inaccuracies during scanning due to patient movement. To do so, the sensor is coupled to a scanning system.

[0003] Additionally, U.S. Patent Application Publication No. 2019 / 0105423 discloses a support for a patient's limb that includes a network of interlaced flexible multi-lumen tubing that forms a lattice structure that can be inflated to immobilize a limb placed within the support.

[0004] The article "Quantification of fMRI Artifact Reduction by a Novel Plaster Cast Head Holder" by V. Edward, C. Windishburger et al. (published online in September 2000, Wiley-Liz, Inc.) discloses a plaster cast head holder for immobilizing and repositioning a patient's head during an fMRI scan. The plaster cast head holder reduces the magnitude of unintended head movement and reduces motion artifacts. The plaster cast head holder includes a head mask with a malleable fixation material that is rigid in its fully cured state. Summary of the Invention [Problem to be solved by the invention]

[0005] Although there are a wide variety of techniques for obtaining images from a patient's body before surgery, such as ultrasound imaging, computed tomography (CT), or magnetic resonance imaging (MRI), surgeons may still have problems identifying objects and tissues seen by the camera during surgery. Therefore, there is a strong desire to avoid this problem and help surgeons orient themselves during surgery. [Means for solving the problem]

[0006] This object is also solved by a device according to claim 1 and a method according to claim 15.

[0007] The apparatus of the present invention includes a patient support device that adapts to the shape of a patient's body. The entire support device, or at least a portion or portions adapted to the patient's body, can be used during (preferably preoperative) medical imaging and during surgery. In a first embodiment, the patient support device can be generated as a rigid molded element, for example by three-dimensional printing (3D printing), following a scan of the shape of a patient's body part. A shape scan can be performed before any medical or live scan is performed to obtain the outline of at least a portion of the patient's body. Alternatively, the shape of the patient's body part can be obtained directly from a medical scan without the need for a prior shape scan, for example, by utilizing automatic skin segmentation of the body part in the medical scan. By directly obtaining the shape of the patient's body part from the medical scan, the risk of further changes in the position of the patient's body part between the shape scan and the medical scan can be reduced. A mechanical scanner, a laser scanner, a camera system, or any other suitable shape acquisition device can be used to scan the shape of the body or at least a portion thereof. The data thus obtained characterizing the patient's body shape can be used to generate a rigid molded element that is applied to the patient during medical scans and also during surgery.

[0008] In a second embodiment, the patient support device itself is adapted to capture the contour of at least one region of the patient during a preoperative scan and then maintain this contour. The patient support device, or at least the contour-capturing portion of the device, can have two states: an initial (soft) state that can conform to the shape of the body, and a final (hardened) state that maintains the shape once acquired on or in the patient's body. The patient support device is used to position the patient during the surgical procedure in the same shape as during medical imaging (scanning). At least one tracker element connected to the support device is adapted to indicate the position and orientation of the support device. In other words, the tracker element is adapted to indicate the position (X, Y, Z) of at least one point in space of the support in combination with three orientations (angular directions about axes X, Y, Z). Alternatively, two or more tracker elements may be provided to indicate the spatial position and orientation of the support device.

[0009] The apparatus further comprises means for capturing data indicating the position and / or orientation of any of the tracker elements, e.g., a camera, a CT scanner, an X-ray machine, or any other apparatus or medical imaging device. The medical imaging device is adapted to acquire at least two-dimensional scan images of at least one region of interest of the patient. The medical imaging device is adapted to acquire those images in relation to the position of the at least one tracker element. The images may be acquired in a pre-operative scan or also during surgery. The tracker attached to the support device may be localized together with the support device. The medical imaging device may comprise a processing unit that generates the scan images in a coordinate system in which the tracker elements are located.

[0010] At the surgical site, a location system is provided for capturing data indicative of the position (and / or orientation) of the tracker during surgery. Data regarding the tracker's position can be captured by at least one tracking camera. The camera may not be close to the surgical site but may be kept at a distance. However, line-of-sight visibility should be ensured.

[0011] Alternatively, an active tracking method may be used in which the support device can emit at least one signal, such as a radio frequency (RF), light, or ultrasound signal. The support device signal can be received and used to capture / obtain the position (and / or orientation) of the support device during surgery. For example, an RF tracker can be embedded in the support device. In that case, data is captured / transmitted from the location where the tracker is placed.

[0012] Because the patient is placed within the same individually shaped portion of the support device as the patient was during the medical imaging process, the patient's body assumes the same shape during surgery as it did during the medical scan. Thus, data captured by the localization system accurately indicates the position of the patient support device, and therefore the position of the patient's tissue structures. The shape-adaptive support device now reshapes the patient's body, ensuring that all of the patient's tissues are in the same place as they were during the medical scan. This is particularly important for surgical procedures on or within soft tissue, which are highly flexible and lack specific natural or artificial landmarks.

[0013] The processing unit is able to align and blend the scanned and live images to provide the surgeon with perfect orientation. In particular, the system of the present invention offers the following advantages:

[0014] 1. No pre-operative intervention is required to implant tracking markers or markings within the patient's body.

[0015] 2. There is no need to use screws, pins, etc. in the bone or target structure, thus avoiding further surgical procedures and complications.

[0016] 3. The need for additional medical imaging devices during surgery is reduced or even eliminated.

[0017] 4. The methods and systems of the present invention do not rely on rigid anatomical structures (due to bones) or low deformation organs.

[0018] The present invention avoids changes in the patient's torso contour due to weight redistribution caused by changes in the patient's contour or position. Thus, the present invention allows for the acquisition of medical images and subsequent performance of a surgical procedure, which may be performed in a different patient position than during medical imaging.

[0019] By placing the patient in or on a patient support device that conforms to the patient's body shape, the patient's body shape is transferred to the support device. The support device captures and maintains the patient's contours during preoperative scanning. The support device can include at least one element made of a casting material, such as a plaster cast, fiberglass, resin-based cast, or other material, which is malleable when the patient is initially placed on or within it. After becoming rigid, the molded support device (cast) is reused while repositioning the patient during the surgical procedure. At least one tracker element is placed on or within the support device before or after hardening the malleable hardenable material. Embedded elements such as screws, fiducials, balls, pins, etc. can be used as trackers and act as position markers for aligning the patient to the navigation system. The navigation system can include a processing unit adapted to register at least one scanned image with at least one live image based on markers placed in or on the support device.

[0020] The patient support device may include a movable table on which a deformable element is disposed. The deformable element may be removably connected to the table. The deformable element may have two states: an initial (soft) state that can conform to the shape of the patient's body, and a final (rigid) state that maintains the shape once acquired at or on the patient's body. Therefore, the deformable element can be moved from one table used to scan the patient to another support for surgery. Because at least one tracker element is rigidly connected to the deformable element, and the deformable element is transformed from the deformable state to the rigid state after the patient is positioned therein and before the medical scan is performed, the navigation system locates tissues and organs within the surgical site in the same spatial relationship to the at least one tracker element as they were during medical imaging.

[0021] Preferably, at least one deformable element is open on one side so that the patient can be removed from and reinserted into the deformable and rigidizable element. Furthermore, the support device can include at least two deformable elements that, when placed around a patient or a portion thereof, surround and thus encase the patient or a portion thereof. This is particularly useful when medical imaging and surgery are performed at different times, e.g., on different days. In particular, when surgery is performed on a body part with little or no rigid structure, the adaptive patient support device, after taking the shape of the patient and hardening to this shape, reshapes the patient's body, protecting soft tissues and organs from assuming the same position as in the previous imaging step when the patient re-enters the device.

[0022] The apparatus of the present invention can include an instrument for treating a patient's body within a region of interest. The instrument can be any type of RF surgical, cryosurgical, plasma surgical, or any other instrument. In particular, the instrument is adapted to be placed within the field of view of a live imaging device. The live imaging device can be part of the instrument, a separate camera inserted in a trocar, or a separate camera placed on, on, or within the patient's body.

[0023] The tracker system is placed at the surgical site of the device and includes at least two cameras or other position detectors for triangulating the position (and, if necessary, the orientation) of at least one tracker element in space. Furthermore, the imaging system can include at least one tracker element or any other sensor connected to a detection system adapted to detect the position and orientation of the camera. The detection system can be connected to a processing unit that can register the live image and the scanned image accordingly.

[0024] Furthermore, the processing unit can include a structure detection unit for generating a graphical representation of tissue structures, which can be obtained from the medical image. The graphical representation can be lines, symbols, colored regions, or other structures adapted to indicate areas or structures that the surgeon wants to distinguish from other areas or structures. These graphical structures can be blended into the live image, thereby helping the surgeon find the structure to be treated within the region of interest.

[0025] Further details and features can also be found in the drawings, the description and the claims. [Brief explanation of the drawings]

[0026] [Figure 1] 1 is a schematic diagram of a patient support device with a patient positioned thereon and a medical imaging device. [Figure 2] 1 is a cross-sectional view of a patient support device and a patient positioned thereon. [Figure 3] 1 is a schematic diagram of a scanned image provided by a medical imaging device. [Figure 4] 2 is a diagram of the patient according to FIG. 1 positioned on a patient support device at the operating site of the apparatus of the invention, including a localization system for locating the patient during surgery. [Figure 5] FIG. 1 illustrates scanned, live, and blended images provided to the surgeon. [Figure 6] FIG. 1 is a schematic diagram of a camera for acquiring live images. [Figure 7] 1 shows a scan image, a volumetric model of the patient's anatomy derived from the scan image, and a live image registered to the spatial representation of the anatomy. DETAILED DESCRIPTION OF THE INVENTION

[0027] 1 shows an apparatus 10 for medical imaging, comprising a support device 11 for positioning a patient 12 in a desired position relative to a medical imaging system 13. The support device 11 may consist of a table 14 and at least one deformable element 15 placed thereon. Further deformable elements 16 and 17 may be placed around the body 18 of the patient 12, as can be seen in FIG. 2, which shows a cross-sectional view of the table 14, the deformable elements 15 to 17, and the body 18.

[0028] The deformable elements 15 to 17 may be cushions filled with a malleable, durable material, such as plaster cast, fiberglass, reinforced gypsum cast, or resin-based cast. Alternatively, the malleable material may be placed directly on the patient's skin or on a cloth placed on the skin. The deformable element 15 may be placed between the patient's body 18 and the table 14, while elements 16 and 17 may be placed on the patient 12 and held between walls 19 and 20, for example. At least one of the deformable elements 15 to 17 and / or at least one of the walls 19 is rigidly connected to tracker elements 21, 22, and 23, which in this case are balls fixed to the ends of trees 24 at known distances from each other. The balls 21 to 23 may be light-reflective or light-absorbing balls visible to the imaging system 13 or specific cameras assigned to the imaging system 13. In this case, three tracking elements are provided to clearly indicate the position and orientation of the patient 12. However, although the three balls 21, 22, 23 placed at the end of the tree give a fairly good indication of the position and orientation of the patient 12, it is also possible to place at least three different balls independently of each other at different positions on the deformable elements 15, 16 or 17. If the tracker elements 21, 22, 23 are detected optically, they are placed on the viewing side of the deformable elements 15 to 17.

[0029] Furthermore, it is possible to use only one item as tracker element, for example one cube rigidly connected to support 14 or at least one of deformable elements 15, 16 and 17.

[0030] The imaging system 13 can be any type of medical imaging system for obtaining preoperative medical scans, such as an MRI system or a CT system as shown in FIG. 1. The CT system can include an X-ray source and an X-ray detector adapted to receive X-rays from the X-ray source and send data to a processing unit 28, which generates scan images 29 (29a-29z) as shown in FIG. 3. The processing unit 28 can be any type of computer adapted to process signals provided by the X-ray detector 27. The processing unit 28 is connected to a storage device 30 for storing the scan images 29 therein. Alternatively or additionally, an intraoperative scanning device, such as a C-arm system, an ultrasound imaging device, or any other system suitable for providing medical scan images during surgery, may be provided.

[0031] The medical imaging system 13 may further be used as a means for capturing data 33 indicative of the positions of the tracking elements 21, 22, 23 during operation of the imaging system, i.e., during scanning of the patient's body 18. Alternatively, a separate localization system may be provided for detecting and locating the tracking elements 21, 22, 23 and for generating images 29 in spatial relationship to the tracking elements 21, 22, 23.

[0032] Part of the apparatus 10 of the present invention is a surgical site 34, shown in FIG. 4. The patient 12 is again placed on a table 14a, which may be identical to the table 14 of the imaging site shown in FIG. 1. Typically, however, the table 14a is a different table than those typically used in conventional operating rooms. Whether the tables 14, 14a are identical or not, in either case, the deformable elements 15 through 17 are used to present the patient's body 18 in the same shape as during medical imaging as shown in FIG. 2. Furthermore, the tracking elements 21, 22, and 23 are in the same position relative to the patient's body 18 during imaging and during surgery.

[0033] A localization system 35 is provided at the surgical site 34 for capturing data 36, ​​which is provided to a processing unit 28a connected to the storage device 30. The processing unit 28a may be the same as the processing unit 28 of FIG. 1 or may be a different processing unit. The processing unit 28a may be any type of computer or processor adapted to receive data from the localization system 35 and determine the position and orientation of the tracker elements 21, 22, 23, and thus the position and orientation of the body 18 of the patient 12. The localization system 35 may include at least two cameras 37, 38 oriented so that the tracking elements 21, 22, 23 are within the field of view of the cameras 37, 38. The processing unit 28 or 28a is adapted to locate the tracker elements 21, 22, 23 by triangulation once before surgery begins, if the table 14a remains stationary. If the table 14a is moved, the localization system 35 may repeat the process of locating and determining the orientation of the body 18 of the patient 12. Alternatively, location may be performed permanently by the location system 35 .

[0034] Part of the apparatus is another camera 39 for acquiring live images, as shown separately in Figures 4 and 6. The field of view 40 of the camera 39 is a region of interest 41 on the patient's body 18 where the surgery is to be performed. Figure 5 shows the region of interest 41 covered by the field of view 40. The camera 39 may be a laparoscopic camera, an endoscopic camera, or any other type of camera suitable and adapted to generate a live image 42 of the region of interest 40.

[0035] The live image 42 may be provided to the processing unit 28 or 28a as shown in Figure 4. The processing unit 28a may process any live image 42 shown in the top right diagram of Figure 5. The live image 42 may include the actual tissue structure 43 and the tip of the instrument 44.

[0036] Any type of localization system may be used to detect the position and orientation of the instrument 44 and / or camera 39. The localization system 35 may include a tracker structure 45 connected to the camera 39 and visible by the cameras 37, 38. The tracker structure may include at least one tracker element, for example, three tracker elements 46, 47, 48 as shown in Figure 6, as well as tracker elements 21, 22, 23. Other types of tracking systems may also be used.

[0037] The apparatus 10 described thus far operates as follows.

[0038] Prior to surgery, patient 12 is positioned on support device 11 on table 14, with deformable elements 15 molded to body 18 as shown in FIG. 2. If desired or necessary, one or two additional deformable elements 16, 17 are placed around patient body 18, so that deformable elements 15, 16, 17 take the inverse shape of body 18 and fit closely around body 18. Deformable elements 15, 16, 17 are filled or formed with a malleable material that solidifies over time, e.g., within minutes or tens of minutes. After hardening, i.e., solidification, imaging system 13 can acquire scan images 29a-29z, which are stored in storage device 30 by processing unit 28. Patient 12 can then leave support device 11 and be prepared for surgery, which can occur within a short time, sometimes hours or days.

[0039] For surgery, the patient 12 re-enters the support device 11, as shown in FIG. 4, by placing the patient's body 18 on the table 14a with the deformable, now rigid, elements 15 through 17 positioned around the body 18 as shown in FIG. 2. The surgeon cuts windows 49 in one or more of the elements 15, 16, and 17 to allow access to the body 18 through the windows 49. The windows 49 may also be provided in the support device, particularly in the deformable elements 15, 16, and 17, in accordance with the planned surgery before the patient is placed in the deformable elements. Alternatively, the windows 49 may be cut into the deformable elements 15, 16, and 17 between the medical pre-operative scan and the surgery. This process may be part of the operation planning.

[0040] At or before the start of the procedure, a localization system 35 is activated, which captures the positions of the tracking elements 21, 22, 23. Thus, the processing unit 28a can register the position of the patient's body 18 with the scan images 29a-29z, as shown in Figure 7. Furthermore, the processing unit 28a or the processing unit 28 can generate a volume model 50 of at least a portion of the patient's body, e.g., a region of interest 41.

[0041] The localization system 35 or any other tracking system for determining the position and orientation of the camera 39 continuously generates data from which the processing unit 28a determines the location and orientation of the field of view 40 of the camera 39, and thus the location and viewing direction of the live image 42. As shown in FIG. 7, the live image 42 may intersect the volumetric model 50 in a different way than the scan images 29a-29z. However, the processing unit 28a can generate a composite image of the volumetric model 50, such as that shown in the upper left diagram of FIG. 5, of at least the region of interest 41. To do so, the processing unit can intersect the volumetric model 50 in the same plane as the live image.

[0042] Processing unit 28a then merges or blends live image 42 (top right diagram of FIG. 5) with a volumetric model view derived by intersecting volumetric model 50 in the same location and orientation as live image 42. FIG. 5 shows a blended image 51 of tissue structures 43 seen by camera 39 and specific tissue structures 52 found by imaging and to be treated by instrument 44.

[0043] Furthermore, processing unit 28 or 28a may alternatively or additionally generate graphical representations 52 of tissue structures and blend those graphical representations into the live image. Any of scan image 29, an image obtained by intersecting with volumetric model 50, and graphical representation 52 obtained from at least one of scan images or from volumetric model 50 are considered to be scan images for blending with the "live image" according to claim 1. Apparatus 10 further comprises an image display 53 for playing the blended image. Display 53 may be a screen, a virtual reality headset, or any other means for displaying the blended image.

[0044] The system of the present invention increases accuracy and reliability by using medical imaging and live imaging on the patient's non-rigid tissue structures and shaping the patient's body 18 during surgery so that the contours of the patient's body during surgery are identical to the contours of the body during imaging. Processing unit 28 or 28a accurately overlays the scan images (or an image or graphical representation derived from several scan images) with the live images acquired during surgery to further improve the surgeon's understanding and orientation during surgery. [Explanation of symbols]

[0045] 10 Medical imaging equipment 11 Supporting Devices 12 patients 13 Imaging system 14,14a Table 15-17 Deformable elements 15a~17a Malleable / hardenable materials 18 Patient's Body 19,20 Wall 21-23 Tracker elements 24 Tree 28,28a Processing unit 29 scanned images 30 Storage section 33 Data capture methods 34 Surgery location 35 Location Identification System 36 Data 37,38 Camera 39 Camera 40 Camera 39 field of view 41 Areas of Interest 42 Live Images 43 Organizational Structure 44 Equipment 45 Tracker Structure 46-48 Tracker elements 49 Windows 50 volume model 51 Mixed Images 52 Organizational Structure 53 Display

Claims

1. An apparatus for imaging (10) and surgery (34) on a patient's body (18), comprising: a patient support device (11) adapted to the shape of the patient's body; at least one tracker element (21) connected to said support device (11) and adapted to indicate the position and orientation of said support device (11); means for capturing data (33) indicative of the position and / or orientation of said at least one tracker element (21); a medical imaging system (13) adapted to acquire at least one at least two-dimensional scan image (29) of at least one region of interest of a patient relative to said position of said at least one tracker element (21); a location system (35) for capturing data (36) indicative of the location of said at least one tracker element (21) during surgery; a live imaging device (39) for acquiring live images (42) of the surgical site; a processing unit (28, 28a) adapted to register and blend the scanned images and live images (29, 42, 52) according to the data captured during the medical imaging and live imaging; An apparatus, wherein said patient support device (11) comprises a deformable element (15) comprising or consisting of a malleable material (15a), said malleable material being a hardenable material (15a).

2. The apparatus of claim 1 , wherein the patient support device (11) includes a movable table (14, 14a).

3. 2. The device of claim 1, wherein the at least one tracker element (21) is directly connected to the at least one deformable element (15).

4. The device described in claim 1, wherein at least one of the deformable elements (15) is open on one side so that the patient can be removed from the deformable element (15) and reinserted.

5. The apparatus of claim 1 , wherein the support device comprises at least two deformable elements (15, 16, 17) that surround at least a portion of the patient.

6. An apparatus as described in claim 4, wherein at least one of the deformable elements (15, 16, 17) has a window (49) for providing access to the patient.

7. The apparatus of claim 1, comprising an instrument (44) for treating the patient's body (18) in the region of interest (41).

8. The apparatus of claim 1 , wherein the tracker element (21) is adapted to indicate the position and the orientation of the support device in space.

9. 2. The apparatus of claim 1, wherein the tracker elements (21) comprise spaced apart reflector elements locatable by the location system.

10. 2. The apparatus of claim 1, wherein the localization system comprises at least two cameras (37, 38) for triangulating the position and the orientation of the at least one tracker element (21) in space.

11. The apparatus of claim 1 , wherein the live imaging device comprises at least one camera (39).

12. The apparatus described in Claim 11, wherein the positioning system (35) is provided for detecting the position and orientation of the camera (39) of the live imaging device.

13. 1. A method for image registration, comprising: Adapting the patient support device (11) to the shape of the patient's body (18); connecting at least one tracker element (21) to said support device (11) and capturing data indicative of the position and / or orientation of said at least one tracker element (21); using a medical imaging system (13) to acquire at least one at least two-dimensional scan image (29) of a region of interest (41) of a patient associated with said position of said at least one tracker element (21); capturing data (36) indicative of the position of said at least one tracker element (21) during surgery by a location system (35); acquiring a live image (42) of the surgical site by a live imaging device (39); and registering and blending the scanned image and the live image according to the data (36) captured during the medical imaging and the live imaging; A method, wherein the patient support device (11) comprises a deformable element (15) comprising or consisting of a malleable material (15a), the malleable material being a hardenable material (15a).

14. An apparatus for imaging (10) and surgery (34) on a patient's body (18), comprising: a patient support device (11) adapted to the shape of the patient's body; at least one tracker element (21) connected to said support device (11) and adapted to indicate the position and orientation of said support device (11); means for capturing data (33) indicative of the position and / or orientation of said at least one tracker element (21); a medical imaging system (13) adapted to acquire at least one at least two-dimensional scan image (29) of at least one region of interest of a patient relative to said position of said at least one tracker element (21); a location system (35) for capturing data (36) indicative of the location of said at least one tracker element (21) during surgery; a live imaging device (39) for acquiring live images (42) of the surgical site; a processing unit (28, 28a) adapted to register and blend the scanned images and live images (29, 42, 52) according to the data captured during the medical imaging and live imaging; 10. An apparatus, wherein the patient support device (11) conforms to the shape of the patient's body by 3D printing at least one element (15) according to a scan of at least a portion of the patient's body contour.

15. A method for aligning images, comprising: Adapting the patient support device (11) to the shape of the patient's body (18); connecting at least one tracker element (21) to said support device (11) and capturing data indicative of the position and / or orientation of said at least one tracker element (21); using a medical imaging system (13) to acquire at least one at least two-dimensional scan image (29) of a region of interest (41) of a patient associated with said position of said at least one tracker element (21); capturing data (36) indicative of the position of said at least one tracker element (21) during surgery by a location system (35); acquiring a live image (42) of the surgical site by a live imaging device (39); and registering and blending the scanned image and the live image according to the data (36) captured during the medical imaging and the live imaging; 10. The method of claim 1, wherein the patient support device (11) conforms to the shape of the patient's body by 3D printing at least one element (15) according to a scan of at least a portion of the patient's body contour.

Images