Apparatus and method for registering live and scanned images

Abstract

Apparatus and method for registering live and scanned images. The system of the present invention utilizes medical imaging and live imaging of a patient's non-rigid anatomy to improve accuracy and reliability by shaping the patient's body 18 during surgery so that the patient's body's external shape during surgery is the same as the body's external shape during imaging. A processing unit accurately overlays the scanned image (or an image or graphical representation derived from several scanned images) with the live image acquired during surgery to enhance the surgeon's understanding and orientation during surgery.

Description

[0001] The invention relates to an arrangement for performing surgery on a patient's body and a method for image registration.

[0002] U.S. Patent No. 9,433,387 B2 discloses a system for obtaining a skull or other scan from a patient. The system includes a diagnostic scanning table and a mold that conforms to a portion of the patient's body contours. In one embodiment, a mask is provided for surrounding the patient's head. Sensors are placed within the mold or mask to obtain pressure readings that can indicate patient movement, such as swallowing. The sensor signals can be used to avoid inaccuracies caused by patient movement during the scan. To this end, the sensors are coupled to the scanning system.

[0003] In addition, US 2019 / 0105423 A1 discloses a support for a patient's limb, which includes a flexible multi-lumen tube interlaced network forming a grid structure. By inflating the multi-lumen tube, the limb placed in the support can be fixed.

[0004] The article "Quantification of fMRI Artefact Reduction by a Novel Plaster Cast Head Holder" by V. Edward, C. Windishburger, et al. (published online in September 2000, Wiley-Liz) discloses a plaster cast head holder for securing and repositioning a patient's head during fMRI scanning. The plaster cast head holder reduces the magnitude of unintentional head movement and motion artifacts. The plaster cast head holder includes a head mask with a ductile fixation material that is rigid in its fully hardened state.

[0005] While there are various techniques for acquiring images of a patient's body before surgery, such as ultrasound imaging, radiology, computed tomography (CT), or magnetic resonance imaging (MRI), surgeons may still have problems identifying objects and tissues to be viewed with a camera during surgery. Therefore, it is highly desirable to avoid this problem and help surgeons orient themselves during surgery.

[0006] This object is achieved by an arrangement according to claim 1 and also by a method according to claim 15 .

[0007] The arrangement of the present invention includes a patient support device that adapts to the shape of a patient's body. The support device, as a whole, or at least adapted to one or more parts of the patient's body, can be used during medical imaging (preferably preoperatively) and also during surgical procedures. In a first embodiment, the patient support device can be produced as a rigid body shaping element, for example, by three-dimensional printing (3D printing) based on a scan of the shape of a patient's body part. The shape scan can be performed before any medical scan or live scan and acquires the external shape of at least a part of the patient's body. Alternatively, the shape of the patient's body part can be directly acquired from the medical scan without requiring a prior shape scan, for example, by utilizing automatic skin segmentation of the body part in the medical scan. Determining the shape of the patient's body part directly from the medical scan can reduce the risk of further changes in the position of the patient's body part between the shape scan and the medical scan. The shape of the body, or at least a part thereof, can be scanned using a mechanical scanner, a laser scanner, a camera system, or any other suitable shape acquisition device. The data representing the shape of the patient's body acquired in this manner can be used to generate the rigid body shaping element that is applied to the patient during the medical scan and also during the surgical procedure.

[0008] In a second embodiment, the patient support device itself is adapted to capture the contour of at least one area of the patient during a preoperative scan and to maintain that contour thereafter. The patient support device, or at least the contour-capturing portion of the device, can have two states: an initial (soft) state, in which it can adapt to the shape of the body, and a final (hardened) state, in which it maintains that shape once acquired at or on the patient's body. The patient support device will be used to position the patient in the same shape during surgery as during medical imaging (during scanning). At least one tracker element connected to the support device is adapted to indicate the position and orientation of the support device. In other words, the tracker element is adapted to indicate the position (X, Y, Z) of at least one point in space of the support member in combination with three orientations (angular orientations about axes X, Y, and Z). Alternatively, two or more tracker elements may be provided to indicate the spatial position and orientation of the support device.

[0009] Furthermore, the arrangement comprises means for capturing data indicating the position and / or orientation of any of the tracker elements, such as a camera, a CT scanner, an X-ray device or any other arrangement or medical imaging device. The medical imaging device is adapted to acquire at least one at least two-dimensional scanned image of a region of interest of the patient. The medical imaging device is adapted to acquire those images relating to the position of at least one tracker element. The images may be acquired in a preoperative scan or even during surgery. The tracker attached to the support device may be positioned therewith. The medical imaging device may comprise a processing unit that generates a scanned image in the coordinate system with the tracker elements being located in the same coordinate system.

[0010] A positioning system is provided at the surgical site to capture data indicating the position (and / or orientation) of the tracker during surgery. Data regarding the tracker's position can be captured by at least one tracking camera. The camera may not be located close to the surgical site, but rather at a distance. However, line-of-sight visibility should be ensured.

[0011] Alternatively, active tracking methods can be used, where the support device can emit at least one signal, such as a radio frequency (RF), light, or ultrasound signal. The signal from the support device can be received and used to capture / obtain the position (and / or orientation) of the support device during surgery. For example, an RF tracker can be embedded in the support device. In this case, data is captured / transmitted from the site where the tracker is placed.

[0012] Because the patient is positioned within the same individually shaped portion of the support device as they were during the medical imaging procedure, the patient's body will assume the same shape during surgery as it did during the medical scan. Consequently, the data captured by the positioning system precisely indicates the position of the patient support device, and therefore the position of the patient's tissue structures. The shape-adaptive support device will now reshape the patient's body and ensure that all of the patient's tissues are in the same position as they were during the medical scan. This is particularly important for performing surgical procedures on or in soft tissues that are highly flexible and lack specific natural or artificial landmarks.

[0013] The processing unit can register and blend the scanned image with the live image and provide the surgeon with perfect orientation. In particular, the system of the present invention provides the following advantages:

[0014] 1. No pre-operative intervention is required for implanting tracking markers or markings within the patient's body.

[0015] 2. No screws, pins or the like are required in the bone or target structure, thus avoiding additional surgical procedures and complications.

[0016] 3. Reduce or even eliminate the need for additional medical imaging equipment during surgery.

[0017] 4. The methods and systems of the present invention do not rely on rigid anatomical structures (such as bones) or organs with low deformation.

[0018] The present invention avoids changes in the contour of the patient's torso due to weight redistribution caused by changing the patient's contour or position. Thus, the present invention allows for the acquisition of medical images and subsequent performance of surgical procedures. Surgical procedures can be performed in a patient position that differs from the position used during medical imaging.

[0019] The patient is placed in or on a patient support device that adapts to the shape of the patient's body, which will transfer the shape of the patient's body to the support device. The support device will capture and maintain the patient's contours during the preoperative scan. The support device may include at least one element comprising a casting material, such as a plaster cast, fiberglass, resin-based cast, or anything else, that is malleable when the patient is first placed on or in it. After becoming hardened, the shaped support device (cast) will be reused when repositioning the patient during surgery. Before or after the malleable curable material is cured, at least one tracker element will be placed on or in the support device. Embedded elements such as screws, fiducials, balls, pins, or the like can be used as trackers and will act as position markers to align the patient to the navigation system. The navigation system may include a processing unit that is suitable for aligning at least one scanned image with at least one live image based on markers placed in or on the support device.

[0020] The patient support device may include a movable table upon which a deformable element is positioned. The deformable element may be removably connected to the table. The deformable element may have two states: an initial (soft) state, in which it adapts to the shape of the patient's body, and a final (hardened) state, in which it retains that shape once positioned on or near the patient's body. Thus, the deformable element can be moved from one table used to scan a patient to another support used for a surgical procedure. Because the at least one tracker element is securely connected to the deformable element, and because the deformable element transitions from a deformable state to a rigid state after the patient is positioned therein and before the medical scan is completed, the navigation system will locate tissues and organs within the surgical site in the same spatial relationship to the at least one tracker element as they would during the medical imaging.

[0021] Preferably, the at least one deformable element is open on one side so that the patient can be removed from and reinserted into the deformable and solidifiable element. Furthermore, the support device may include at least two deformable elements that, when placed around the patient or a portion thereof, will surround and, therefore, enclose the patient or a portion thereof. This is particularly useful if the medical imaging and surgery are to be performed at different times, such as on different days. In particular, if the surgery is to be performed on a body part with little or no rigid structure, the adaptable patient support device, after assuming the patient's shape and hardening into that shape, will reshape the patient's body and, if the patient re-enters the device, preserve the flexible tissues and organs in the same position as in the previous imaging step.

[0022] The arrangement of the present invention may include an instrument for treating a patient's body in an area of interest. The instrument may be any type of RF surgical, cryosurgery, plasma surgical, or any other instrument. In particular, the instrument is adapted to be placed within the field of view of a live imaging device. The live imaging device may be part of the instrument, a separate camera inserted into a trocar, or a separate camera placed at, on, or within the patient's body.

[0023] The tracker system is placed at the surgical site of the arrangement and includes at least two cameras or other position detectors for triangulating the position (and, if necessary, the orientation) of at least one tracker element in space. Furthermore, the imaging system may include at least one tracker element or any other sensor connected to a detection system suitable for detecting the position and orientation of the camera. The detection system may be connected to a processing unit that can accordingly register the live image with the scanned image.

[0024] Furthermore, the processing unit may include a structure detection unit for generating a graphical representation of tissue structure that can be obtained from the medical image. The graphical representation may be a line, a symbol, a colored area, or anything else suitable for indicating an area or tissue that the surgeon should distinguish from other areas or tissue. These graphical structures may be blended into the live image, which helps the surgeon find the structure to be treated within the region of interest.

[0025] Additional details and features can be found in the drawings, the description, and the claims.

[0026] Figure 1 is a schematic representation of a patient support device and a medical imaging device with a patient positioned thereon,

[0027] Figure 2 is a cross-sectional view of a patient support device and a patient positioned thereon,

[0028] Figure 3 is a schematic representation of a scanned image provided by a medical imaging device,

[0029] Figure 4 is placed on a patient support device at the surgical site of the arrangement of the present invention according to Figure 1 the patient, the arrangement of the invention comprises a positioning system for positioning the patient during surgery,

[0030] Figure 5 The diagram shows the scanned image, live image and hybrid image provided to the surgeon.

[0031] Figure 6 is a schematic illustration of a camera used to acquire live images, and

[0032] Figure 7 A scanned image, a volumetric model of the patient's tissue structure obtained from the scanned image, and a live image registered to the spatial representation of the tissue structure are illustrated.

[0033] Figure 1 The diagram shows an arrangement 10 for medical imaging, comprising a support device 11 for positioning a patient 12 in a desired position relative to a medical imaging system 13. The support device 11 may consist of a table 14 and at least one deformable element 15 placed thereon. Further deformable elements 16 and 17 may be placed around a body 18 of the patient 12, such as may be seen from the side of the patient. Figure 2 Obtained in Figure 2 A cross-sectional view of the table 14 , the deformable elements 15 to 17 and the body 18 is shown.

[0034] Deformable elements 15-17 may be pads filled with a malleable, durable material, such as plaster cast, fiberglass, reinforced plaster cast, resin-based cast, or the like. Alternatively, the malleable material may be placed directly on the patient's skin or on a cloth placed on the skin. While deformable element 15 may be placed between patient 12's body 18 and table 14, elements 16 and 17 may be placed on patient 12 and, for example, held between walls 19, 20. At least one of deformable elements 15-17 and / or at least one of walls 19 is securely connected to tracker elements 21, 22, 23, which in the present case are balls fixed at known distances from one another at the ends of a tree 24. Balls 21-23 may be light-reflective or light-absorbing balls visible through imaging system 13 or a specific camera assigned to imaging system 13. In the present case, three tracking elements are provided to unambiguously indicate the position and orientation of patient 12. However, although the three balls 21, 22, 23 placed on the ends of the tree give a fairly good indication of the position and orientation of the patient 12, it is also possible to place at least three different balls, independent of each other, at different positions of the deformable element 15, 16 or 17. If the tracker elements 21, 22, 23 are to be detected optically, they will be placed at the visible side of the deformable elements 15 to 17.

[0035] Furthermore, it is possible to use only one item as tracker element, for example a cube firmly connected to the support 14 or at least one of the deformable elements 15 , 16 and 17 .

[0036] The imaging system 13 may be any type of medical imaging system used to obtain preoperative medical scans, such as Figure 1 The MRI system or CT system shown in FIG. 1 may include an X-ray source and an X-ray detector adapted to receive X-rays from the source and deliver data to a processing unit 28 that generates scanned images 29 ( 29 a to 29 z ), as shown in FIG. Figure 3 . The processing unit 28 may be any type of computer suitable for processing the signals provided by the X-ray detector 27. The processing unit 28 is connected to a storage device 30 for storing therein the scanned images 29. Alternatively or additionally, an intraoperative scanning device may be provided, such as a C-arm system, an ultrasound imaging device or any other system suitable for providing medical scanned images during surgery.

[0037] The medical imaging system 13 may additionally serve as a means 33 for capturing data indicative of the position of the tracking elements 21, 22, 23 during operation of the imaging system, i.e., during scanning of the patient's body 18. Alternatively, a separate positioning system may be provided for detecting and positioning the tracking elements 21, 22, 23 and for bringing the image 29 into spatial relationship with the tracking elements 21, 22, 23.

[0038] A portion of the arrangement 10 of the present invention is Figure 4 The patient 12 is again placed on the table 14a, which can be Figure 1 14a will be identical to the imaging site shown in FIG. However, typically, the table 14a will be a different table than that typically used in a conventional operating room. Regardless of whether the tables 14, 14a are identical, in any case, the deformable elements 15 to 17 will be used in a manner similar to that used in a conventional operating room. Figure 2 The patient's body 18 is presented in the same shape it already has during medical imaging when illustrated in . Furthermore, the tracking elements 21 , 22, 23 will be in the same position relative to the patient's body 18 during imaging and as well as during surgery.

[0039] At the surgical site 34, a positioning system 35 is provided for capturing data 36 which is fed to a processing unit 28a connected to a storage device 30. The processing unit 28a may be connected to Figure 1 The processing unit 28 or 28a may be the same as the processing unit 28 of the patient 12, or alternatively it may be a different processing unit. The processing unit 28a may be any type of computer or processor that is suitable for receiving data from the positioning system 35 and determining the position and orientation of the tracker elements 21, 22, 23 and therefore the position and orientation of the body 18 of the patient 12. The positioning system 35 may include at least two cameras 37, 38, which are oriented so that the tracking elements 21, 22, 23 are within the field of view of the cameras 37, 38. If the table 14a remains stationary, the processing unit 28 or 28a is suitable for positioning the tracker elements 21, 22, 23 once before the start of the operation by triangulation. If the table 14a is moved, the positioning system 35 may repeatedly determine the position and orientation of the body 18 of the patient 12. Alternatively, the positioning may be done permanently by the positioning system 35.

[0040] Part of the arrangement is to obtain Figure 4 and Figure 6 Another camera 39 of the live image is illustrated in . The field of view 40 of the camera 39 is a region of interest 41 of the patient's body 18 where the surgery is to be performed. Figure 5The region of interest 41 covered by the field of view 40 is illustrated. The camera 39 may be a laparoscopic camera, an endoscopic camera or any other type of camera suitable and adapted to produce a live image 42 of the region of interest 40.

[0041] The live image 42 may be fed to the processing unit 28 or 28a, such as Figure 4 The processing unit 28a can process Figure 5 Any live image 42 shown in the upper right illustration. The live image 42 may contain real tissue structures 43 and the tip of the instrument 44.

[0042] Any type of positioning system may be used to detect the position and orientation of the instrument 44 and / or the camera 39. The positioning system 35 may include a tracker structure 45 connected to the camera 39 and visible to the cameras 37, 38. The tracker structure may include at least one tracker element, such as Figure 6 The three tracker elements 46, 47, 48 illustrated in FIG, are similar to the tracker elements 21, 22, 23. Other types of tracking systems may be used.

[0043] The arrangement 10 described thus far operates as follows:

[0044] Before the operation, the patient 12 will be placed on the support 11 device of the table 14, wherein the deformable element 15 is shaped by the body 18, such as Figure 2 . If desired or necessary, one or two additional deformable elements 16, 17 are placed around the patient's body 18, such that the deformable elements 15, 16, 17 assume the negative shape of the patient's body 18 and fit snugly around the body 18. Deformable elements 15, 16, 17 are filled with or formed from a malleable material that solidifies over time, for example, within a few minutes or tens of minutes. After solidification, imaging system 13 can acquire scanned images 29a to 29z, which are stored by processing unit 28 in storage device 30. Patient 12 can then be removed from support device 11 and prepared for surgery, which can be performed quickly, sometimes hours or days later.

[0045] For surgical procedures, the patient 12 re-enters the Figure 4 The support device 11 shown in FIG, wherein the once deformable and now rigid elements 15 to 17 are placed around the body 18, as Figure 2. The surgeon may have cut a window 49 in one or more of the elements 15, 16, 17 so that he or she can access the body 18 through the window 49. Window 49 may also be provided in the support device, particularly in the deformable element(s) 15, 16, 17, for a planned procedure before the patient is placed in the deformable element(s). Alternatively, window 49 may be cut into the deformable element(s) 15, 16, 17 between the pre-operative medical scan and the procedure. This process may be part of the planned procedure.

[0046] At the beginning of the operation or before the beginning of the operation, the positioning system 35 will be activated, which captures the position of the tracking elements 21, 22, 23. The processing unit 28a can thus register the position of the patient's body 18 to the scanned images 29a to 29z, as shown in FIG. Figure 7 Furthermore, the processing unit 28a or the processing unit 28 may generate a volume model 50 of at least a portion of the patient's body, eg the region of interest 41 .

[0047] The positioning system 35 or any other tracking system for determining the position and orientation of the camera 39 continuously generates data from which the processing unit 28a determines the position and orientation of the field of view 40 of the camera 39 and, therefore, the position of the live image 42 and the viewing direction of the live image. Figure 7 As illustrated in FIG, the live image 42 may intersect the volume model 50 in a different manner than the scanned images 29a to 29z. However, the processing unit 28a may generate a composite image of the volume model 50, such as Figure 5 As illustrated in the upper left figure at least the region of interest 41 in FIG.

[0048] The processing unit 28a then combines or blends the live images 42 ( Figure 5 The upper right illustration) is an illustration of a volume model obtained by intersecting the volume model 50 at the same position and with the same orientation as the live image 42. Figure 5 A hybrid image 51 is illustrated having the tissue structure 43 seen by the camera 39 and the specific tissue structure 52 found by imaging and treated by the instrument 44 .

[0049] Furthermore, the processing unit 28 or 28a can alternatively or additionally generate graphical representations 52 of the tissue structure and blend those graphical representations into the live image. According to claim 1, any of the scanned images 29, the image obtained by intersecting with the volume model 50, and the graphical representation 52 obtained from at least one of the scanned images or from the volume model 50 are considered "scanned images" for blending with the "live image." The arrangement 10 further includes an image display 53 for reproducing the blended image. The display 53 can be a screen, a virtual reality headset, or any other means for displaying the blended image.

[0050] The system of the present invention uses medical imaging and live imaging of the patient's non-rigid tissue structure and improves accuracy and reliability by shaping the patient's body 18 during surgery so that the external shape of the patient's body during surgery is the same as the external shape of the body during imaging. The processing unit 28 or 28a will accurately overlay the scan image (or image or graphical representation obtained from several scan images) and the live image obtained during surgery to enhance the surgeon's understanding and orientation during surgery.

[0051] Reference Number:

[0052] 10 Arrangements for medical imaging

[0053] 11 Support equipment

[0054] 12 patients

[0055] 13 Imaging System

[0056] 14.14a workbench

[0057] 15-17 Deformable Elements

[0058] 15a-17a ductile / curable materials

[0059] 18Patient's body

[0060] 19, 20 wall

[0061] 21-23 Tracker Components

[0062] 24 Tree-like objects

[0063] 28, 28a processing unit

[0064] 29 scanned images

[0065] 30 storage devices

[0066] 33 Artifacts for capturing data

[0067] 34Surgical site

[0068] 35 positioning system

[0069] 36 data

[0070] 37, 38 cameras

[0071] 39 cameras

[0072] 40 Field of view of camera 39

[0073] 41 Areas of Interest

[0074] 42 live images

[0075] 43Organizational Structure

[0076] 44 instruments

[0077] 45 Tracker Structure

[0078] 46-48 Tracker Components

[0079] 49 windows

[0080] 50 volume model

[0081] 51 mixed images

[0082] 52Organizational Structure

[0083] 53 displays.

Claims

1. An arrangement for imaging (10) and surgery (34) on a patient's body (18), comprising: A patient support device (11) adapted to the shape of a patient's body, wherein the patient support device (11) comprises a deformable element (15), the deformable element (15) comprising or consisting of a stretchable material (15a), and wherein the stretchable material is a solidifiable material (15a), at least one tracker element (21) connected to the support device (11) and adapted to indicate the position and orientation of the support device (11), means (33) for capturing data indicative of a position and / or orientation of at least one tracker element (21), a medical imaging system (13) adapted to acquire at least one at least 2-dimensional scanned image (29) of a region of interest of a patient associated with a position of at least one tracker element (21), a positioning system (35) for capturing data (36) including the position of at least one tracker and thus the position of a tissue structure of the patient during surgery, a live imaging device (39) for acquiring a live image (42) of the surgical site indicating the position and / or orientation of the tracker during surgery, A processing unit (28, 28a) is adapted to register and blend at least one scanned image (29) containing the tracker element (21) and a live image (42, 52) containing the tracker element (21) based on data captured during medical imaging and live imaging, based on a tracker element placed in or on a supporting device.

2. The arrangement according to claim 1, wherein The patient support device (11) comprises a movable table (14, 14a).

3. An arrangement according to claim 1 or 2, wherein The patient support device (11) is adapted to the shape of the patient's body by 3D printing at least one element (15) based on a scan of at least a portion of the external shape of the patient's body.

4. Arrangement according to claim 3, wherein at least one tracker element (21) is directly connected to at least one deformable element (15).

5. An arrangement according to claim 3, wherein at least one deformable element (15) is open at one side, such that a patient can be removed from and reinserted into the deformable element (15).

6. The arrangement according to claim 1 or 2, wherein The support device comprises at least two deformable elements (15, 16, 17) which surround at least a portion of the patient.

7. An arrangement according to claim 1 or 2, wherein at least one deformable element (15, 16, 17) comprises a window (49) for access to the patient.

8. Arrangement according to claim 1 or 2, comprising an instrument (44) for treating the patient's body (18) in the region of interest (41).

9. The arrangement according to claim 1 or 2, wherein: The tracker element (21) is adapted to indicate the position and orientation of the support device in space.

10. The arrangement according to claim 1 or 2, wherein The tracker element (21) comprises spaced apart reflector elements positionable by a positioning system.

11. The arrangement according to claim 1 or 2, wherein The positioning system comprises at least two cameras (37, 38) for triangulating the position and orientation of at least one tracker element (21) in space.

12. The arrangement according to claim 1 or 2, wherein The live imaging device comprises at least one camera (39).

13. An arrangement according to claim 12, wherein a detection system (35) is provided for detecting the position and orientation of a camera (39) of the live imaging device.

14. A method for image registration, comprising: Adapting a patient support device (11) to the shape of a patient's body (18), wherein the patient support device (11) comprises a deformable element (15), the deformable element (15) comprising or consisting of a stretchable material (15a), and wherein the stretchable material is a solidifiable material (15a), connecting at least one tracker element (21) to the support device (11) and capturing data comprising the position and / or orientation of the at least one tracker element (21) and thus the position of the patient's tissue structure, acquiring at least one at least 2-dimensional scan image (29) of a region of interest (41) of a patient associated with a position of a tracker (21) using a medical imaging system (13), capturing data (36) indicating the position of the tracker during surgery by means of a positioning system (35), acquiring a live image (42) of the surgical site indicating the position and / or orientation of the tracker during surgery via a live imaging device (39), At least one scanned image containing a tracker element (21) and a live image containing a tracker element (21) are registered and blended based on data (36) captured during medical imaging and live imaging based on a tracker element placed in or on a support device.

Images