Systems and methods for assisting surgical procedures

Abstract

The present invention relates to a system and method for assisting in image-guided surgical procedures, particularly image-guided percutaneous needle interventions such as, for example, the removal of kidney stones. Using this system and method, it is possible to provide an automatic recommendation for a control scan at critical moments during the intervention. For this purpose, in an initial image, dangerous structures are identified and an intervention path to a target position is provided. Using an evaluation unit, it is possible to evaluate whether the spatial relationship between a) the tracked position of a medical instrument and b) the position of the dangerous structure and / or the intervention path meets a predetermined criterion. If it is found that the predetermined criterion is met, the acquisition of a control scan is automatically recommended.

Description

Technical Field

[0001] The present invention relates to a system for assisting a surgical procedure, a method for assisting a surgical procedure, a computer program, and a non-transitory computer-readable data medium.

Background Art

[0002] In minimally invasive surgery, it is known to empirically plan the trajectory to the incision point and anatomical region in a patient's body. For this purpose, generally, pre-recorded fluoroscopic images or tomographic images showing the patient's anatomical region are used. During the surgery, usually, continuous low-dose X-ray imaging or X-ray imaging at selected times is performed, and while guiding a medical instrument along the planned path to the patient's anatomical region, the position of the medical instrument is confirmed using control images. As a result, during the surgery, the patient, and sometimes even the clinician performing the surgery, are exposed to a certain degree of X-ray irradiation.

[0003] European Patent Application Publication No. EP3797724A1 discloses a system for performing an image-guided medical procedure, which uses a three-dimensional fluoroscopic configuration to plan and navigate a percutaneously inserted device, such as a biopsy tool, from an entry point into a patient and to a target within the patient.

[0004] International Publication No. WO2018 / 005842A1 discloses performing a first fluoroscopic sweep including a target area to obtain a first fluoroscopic image. The position of the target is determined based on this first fluoroscopic image, and the position of the entry point is determined based on this target position and the first fluoroscopic image. Then, a needle is inserted or advanced forward, and then an additional fluoroscopic sweep is performed to obtain an additional fluoroscopic image. The position and orientation of the inserted needle are determined based on this additional fluoroscopic image, and the distance between the tip of the needle and the target is determined. Then, it is checked whether the tip of the needle has reached the target. If the tip of the needle has not reached the target, the needle is further advanced forward, and a further additional fluoroscopic sweep is performed to obtain a further additional fluoroscopic image. If the tip of the needle has reached the target, the method ends.

[0005] European Patent Application Publication No. EP3932357A1 discloses a system for assisting a user in disposing a penetrating device in tissue such that a pedicle screw is disposed in a vertebra's pedicle. This system generates a virtual view in the direction of a path through a model of the tissue from the perspective of the tip of the penetrating device within the tissue. This virtual view is generated based on tracking information indicating the posture, model, and path of the penetrating device, and this virtual view is configured to indicate the direction in which the user should move this penetrating device while disposing the penetrating device within the tissue. For example, a virtual tunnel disposed along the path can be shown.

[0006] International Publication No. 2012 / 056386A1 discloses a system having a shape detection or localization system coupled to an intervention device to measure the spatial characteristics of the intervention device within a subject, and an image module configured to receive the spatial characteristics and generate one or more control signals according to the spatial characteristics. This system also has at least one imaging device configured to image the subject according to the control signal.

[0007] To reduce X-ray exposure, it has been proposed to use a robot to perform a surgical procedure based on an anatomical roadmap. A configuration using an intervention robot is disclosed in International Publication No. WO2019 / 092225A1. Here, when the robot reaches an important navigation position within the anatomical roadmap and / or when the patient's physiological state is in crisis, an intervention controller is provided for robot navigation of the intervention instrument based on a single static anatomical roadmap that is automatically updated by the intervention controller, thereby promoting a reduction in radiation exposure in the intervention configuration. An intervention system is disclosed.

Summary of the Invention

Problems to be Solved by the Invention

[0008] Furthermore, it is still desirable to further improve the safety of the patient during the surgical procedure and at the same time reduce the X-ray exposure of the patient and, if present, the clinician during the operation.

Means for Solving the Problems

[0009] The present invention is based on the object of providing a system, method, computer program and non-transitory computer-readable data medium for assisting in performing a surgical procedure with high reliability. Preferably, using the system, method, computer program and non-transitory computer-readable data medium, it is further possible to perform a surgical procedure with a relatively reduced X-ray exposure of the patient.

[0010] Regarding this system, a system for assisting a surgical procedure is proposed, which has an image providing unit, an access route providing unit, a dangerous structure providing unit, a position providing unit, an evaluation unit, and an output unit. The image providing unit is configured to provide an initial image of a target position within a patient's body. The access route providing unit is configured to provide, in the initial image, an access route along which a medical instrument is guided to the target position. The dangerous structure providing unit is configured to provide, in the initial image, the position of a dangerous structure. The position providing unit is configured to provide the tracking position of the medical instrument. The evaluation unit is configured to evaluate whether the spatial relationship between a) the tracking position and b) the position of the dangerous structure and / or the access route meets a predetermined criterion. The output unit is configured to provide an output signal indicating whether the spatial relationship meets the predetermined criterion.

[0011] The present invention includes the recognition that it is beneficial to empirically plan a surgical procedure using an initial image of a target position within a patient's body in order to enhance the reliability of the surgical procedure. Based on this initial image of the target position, an access route along which a medical instrument is guided to the target position can be planned. Furthermore, it is advantageous if the position of the medical instrument within the patient can be tracked and the tracking position of the medical instrument related to the planned access route can be evaluated.

[0012] Here, the present invention includes the further recognition that, on the way towards a target position within a patient's body, a medical instrument traverses an internal region of the patient such that, even if the medical instrument typically deviates slightly from the intervention path, it probably does not cause any serious damage to the patient. However, the initial image also has one or more dangerous structures located extremely close to the intervention path. The dangerous structure is, for example, a structure that must not be damaged by the medical instrument, for example, due to puncture, in order to avoid more serious damage to the patient. Using this system, it is possible to use the tracked position of this medical instrument to evaluate the spatial relationship between the tracked medical instrument and the position of the dangerous structure and / or the intervention path. In particular, using the evaluation of the system, it is possible to evaluate whether the spatial relationship meets a predetermined criterion. As a result of the evaluated tracked position of the medical instrument and the position of the dangerous structure and / or the intervention path, it is possible to determine whether the predetermined criterion is met. Based on whether the predetermined criterion is met, it is possible to make some modification to the surgical procedure or continue the surgical procedure with high reliability. Therefore, the evaluation of the spatial relationship between the tracked position and the position of the dangerous structure and / or the intervention path with respect to the predetermined criterion improves the reliability of the surgical procedure and reduces the risk of damaging the dangerous structure of the patient. Thereby, the system assists in safely guiding the medical instrument to a target position inside the patient without damaging a nearby dangerous structure. Thereby, the system enables increasing the success rate of the surgical procedure and helps to improve the patient's health condition.

[0013] To provide the initial image, the image providing unit may be configured to receive the initial image from another external device such as an imaging device, for example. The image providing unit may be configured to generate the initial image, for example, the image providing unit itself may be an imaging device.

[0014] The initial image can be a fluoroscopic image, a tomographic image, or an ultrasonic image. Therefore, the initial image may be generated by a fluoroscopic imaging device, such as a C-arm having an X-ray source and an X-ray detector. The initial image may be generated by an X-ray computed tomography (CT) device, such as a cone beam computed tomography (CBCT) device. The initial image may be generated by an ultrasonic device. Preferably, the initial image is preferably captured as a planning image before a surgical procedure. The initial image may be directly captured at the start of a surgical procedure. The initial image can also be recorded during a surgical procedure, for example, to adapt a surgical plan.

[0015] The tracking position of the medical device provided by the system's position providing unit can be generated by the tracking system and provided to the position providing unit. In particular, the position providing unit may be configured to receive the tracking position from the tracking system. The tracking system can be an optical tracking system. The optical tracking system is attached to a medical device and uses a camera to detect active markers such as light-emitting diodes (LEDs), or passive markers such as spheres or disks coated with a highly reflective material, typically using infrared light, to determine the position of the medical device. The tracking system can be an electromagnetic tracking system. The electromagnetic tracking system generally has a magnetic field generator for generating an electromagnetic field and an electromagnetic sensor attached to the medical device for tracking. In order to use the tracking position of the medical device tracked using the tracking system, the tracking position defined in the coordinate system of the tracking system must be converted to the coordinate system of the initial image. This includes patient alignment and calibration of the medical device. More generally, using a non-imaging tracking system to track the position of a marker arranged at the proximal end of a medical device, for example, based on an optical marker or an electromagnetic marker, is achieved by registering the tracking position to a reference X-ray image of the patient. Subsequently, based on the tracking position of the marker, the position of the distal end of the intervention device relative to the dangerous structure in the subject is estimated. It is also possible to track the position using a fixed physical guidance, such as a jig, between the imaging device and the medical device.

[0016] It is also possible to track the position of a medical device and thus provide the tracked position using a mobile application on a standard mobile consumer device such as, for example, a smartphone or a tablet computer, which relies on a database of one or more digital computer-aided design (CAD) geometries of the casing of the medical device and the medical imaging device being used. When using a mobile application, geometric relationships such as, for example, the 3D positions of the imaging device and the medical device can be established, for example, by point cloud matching generated by a LiDAR (light detection and ranging) or similar depth-ranging camera system. Instead of using a LiDAR or the like, the 3D positions of the imaging device and the medical device can be extracted from a 2D image feed such as a web camera. This approach is based on the recognition that, for example, when the shape of the imaging device of a cone beam CT (CBCT) and the shape of the medical device can be recognized in the image feed of the detector, the spatial relationship between this imaging device and the medical device can be extracted. Since the position of the imaging device is known, for example, the position of the gantry of a CBCT is extracted from the control computer of the CBCT and the spatial relationship of the gantry to the captured image is also known, it is also possible to determine the position and orientation of the medical device.

[0017] In one example, the position providing unit is configured to provide, as the tracked position of the medical device, not the position determined by detecting the medical device in the initial image or the control image. In one example, the tracked position is used to define the timing at which the control image should be acquired, but the tracked position used for this purpose is not determined by detecting the medical device in the control image or the initial image.

[0018] The medical instrument whose tracking position is provided by the position providing unit is a needle, a catheter, a guide wire, or any other access channel for an endoscope, for example. In particular, the medical instrument may be an instrument used to insert into a patient as part of a surgical procedure. The medical instrument may be a handheld device guided by a human, or a medical instrument guided by a robot.

[0019] The dangerous structure is a human organ, such as a kidney, a heart, a lung, etc., or a tube such as a blood vessel, such as a vein.

[0020] Generally, the components of the system, that is, in particular, the image providing unit, the intervention path providing unit, the dangerous structure providing unit, the evaluation unit, and the output unit, can be composed of hardware and software adapted to perform their respective functions. For example, the above-described components of the system can be implemented as part of a personal computer or other computing device.

[0021] The system further has a control unit configured to control the image providing unit to provide a control image of the target position when the output signal indicates that the evaluated spatial relationship meets a predetermined criterion.

[0022] Based on this evaluated spatial relationship, the control unit can provide an automatic recommendation for the timing at which a control scan should be performed. The control unit can operate the image providing unit to generate a control image, that is, the control image can be captured by the control unit itself, or by requesting the external imaging device to provide the control image. Thus, the control image is provided only in specific situations, for example, in specific, e.g., critical situations, which require evaluating the position of a specific body part, such as the position of a medical instrument being tracked relative to a dangerous structure. Thereby, the number of control scans is limited to the minimum number required to ensure safely guiding the medical instrument to the target position. Advantageously, using this control unit, it is possible to avoid providing a control scan in situations where a control scan is not actually required. For example, the recommendation to obtain an updated X-ray control image provided by the control unit can be based on the proximity of the estimated tracking position of the distal end of the medical instrument to the position of the dangerous structure.

[0023] Advantageously, using the control unit makes it possible to replace continuous fluoroscopic imaging with intermittent single-view X-ray exposure, i.e., a reduced-dose regime of controlled exposure, at several points in time selected by the control unit, based on an evaluation of the spatial relationship between the tracking position and the position of the intervention path and / or the dangerous structure.

[0024] Preferably, the control unit can control the image providing unit based on an output signal provided by the output unit indicating whether the spatial relationship meets a predetermined criterion. Based on whether the spatial relationship meets the predetermined criterion, the control unit may or may not request the image providing unit to provide a control image.

[0025] As described above, in one example, the control unit can provide an automatic recommendation of the timing at which a control scan should be performed based on the evaluated spatial relationship. Thus, the control unit can be configured to control the image providing unit to provide a control image of the target position by providing the automatic recommendation, and when a user such as a surgeon accepts the automatic recommendation via an input device of the system, the control unit controls the image providing unit to provide a control image of the target position. As an alternative, the control unit can directly provide a control image of the target position without waiting for the user to accept a corresponding recommendation when the output signal indicates that the evaluated spatial relationship meets a predetermined criterion. In other words, in one example, the control unit is configured to control the image providing unit to provide a control image of the target position when the output signal indicates that the evaluated spatial relationship meets a predetermined criterion, and in addition, when the user indicates, for example, by accepting a corresponding recommendation, that the control image should actually be provided, for example, actually acquired. In a further example, the control unit is configured to control the image providing unit to provide a control image of the target position when the output signal indicates that the evaluated spatial relationship meets a predetermined criterion, and in addition, without the need for the user to indicate via an input device of the system that the control image of the target position should actually be provided.

[0026] The control image can be used by the system as a new initial image, i.e., the intervention path providing unit can provide a new intervention path in the new initial image along which the medical instrument is guided to the target position, and the dangerous structure providing unit can provide a new position of the dangerous structure in this new initial image. This new initial image can then be used by the evaluation unit for evaluation. The initial image can be, for example, a fluoroscopic image, and the control image used as the new initial image can be a tomographic image or an ultrasonic image, and vice versa.

[0027] In particular, the control image may show a change in the scene captured in the initial image, for example, because the patient has moved or because the interaction between the medical instrument and the patient's anatomical structure has caused a change in the anatomical structure, for example, because the medical instrument may have pushed aside an organ or another dangerous structure.

[0028] Alternatively or additionally, the control unit may be configured to define a control region that meets a predetermined criterion and to control the image providing unit to provide a control image when the medical instrument reaches that control region.

[0029] Preferably, the control region is defined in the initial image and indicates the position of the medical instrument where a predetermined criterion is met. Thus, the control region indicates a position inside the patient where the spatial relationship meets a predetermined criterion such that a control image should be captured. Thus, the control region indicates the position of the medical instrument where the control image has been or is planned to be captured. The control region is preferably defined in the initial image before the operation. However, it may also be beneficial if the control region is defined during the operation, for example, based on the control image.

[0030] For example, the control region may be defined with respect to a dangerous structure. In particular, when it is expected that the medical instrument to be tracked will reach the control region based on the tracking position, the control unit activates the image providing unit so that the control region is positioned on the intervention path to provide a control image. The control region can have a geometric shape of a point, a sphere or a three-dimensional (3D) volume. Its dimensions can be selected, for example, according to its position relative to the dangerous structure. For example, when the control region is located near the dangerous structure, this control region is defined to have larger dimensions compared to a situation where the dangerous structure is far away from the intervention path. Then, when a critical spatial relationship between the medical instrument and the dangerous structure is expected, the control unit can activate the image providing unit to provide a control image relatively early, that is, when it is far away from the dangerous structure.

[0031] The system may further include a visualization unit configured to visualize the tracking position and the control region in the initial image. The control region preferably indicates the position of a medical instrument for which a control image has been provided or is to be provided. The visualization unit may be further configured to visualize the intervention path, where the control region is on the intervention path. For example, the visualization unit may be a graphical user interface or may have a graphical user interface. The visualization unit may be further configured to visualize the provided dangerous structure. Thus, the visualization unit can visualize the planned intervention path, for example, as provided together with the dangerous structure by the intervention path providing unit. During the operation, the visualization unit can also visualize, for example, the tracking position of the medical instrument relative to the intervention path.

[0032] Preferably, the dangerous structure providing unit is configured to define a safety distance with respect to the dangerous structure. Preferably, the evaluation unit is configured to evaluate that when the tracking position of the medical instrument is closer to the dangerous structure than the defined safety distance, the spatial relationship between a) the tracking position and b) the position of the dangerous structure satisfies a predetermined criterion.

[0033] The specific purpose of the safety distance is to define a buffer zone with respect to the dangerous structure such that when the medical instrument reaches or is expected to reach this safety distance, a control image has already been provided. In this case, the control image is provided before actually reaching the dangerous structure. Thereby, the risk of puncturing the dangerous structure can be reduced. For example, the control region may be arranged at a safety distance such that when it is expected that the medical instrument reaches the control region based on the provided tracking position, the control unit controls the image providing unit to provide a control image.

[0034] This safety distance can be changed to a safety margin around the dangerous structure or, in 3D, a safety envelope surrounding the dangerous structure.

[0035] To define a safety margin, the expected relative movement caused by positioning the patient prior to the intervention and / or the expected movement caused by the patient's respiration during the intervention can be considered. The dangerous structure providing unit may be configured to define a safety margin around the dangerous structure, and the intervention path providing unit may be configured to provide an intervention path such that the intervention path does not intersect the dangerous structure and the defined safety margin.

[0036] Alternatively or additionally, the dangerous structure providing unit may be configured to define a safety distance with respect to the dangerous structure and determine the intersection of the provided intervention path and the defined safety distance. Preferably, the evaluation unit is configured to evaluate that when the tracking position is closer to the intersection than a predetermined threshold crossing distance, a) the tracking position and b) the spatial relationship between the position of the dangerous structure and the intervention path satisfy a predetermined criterion.

[0037] In this case, the intervention path intersects the safety distance and is provided such that the intervening portion is closer to the dangerous structure than the extension of the safety distance. With respect to the intersection of the intervention path and the safety distance, a crossing distance is defined. When the tracking position is closer to the intersection than a predetermined threshold crossing distance, the medical instrument is expected to move relatively close to the dangerous structure, that is, closer than the extension of the safety distance. Further, since the evaluation unit is configured to be used as a predetermined criterion when the tracking position is closer to the intersection than a predetermined threshold crossing distance, an output signal used by the control unit that activates the image providing unit to provide a control image is provided. Therefore, it is possible to evaluate the actual spatial relationship between the medical instrument and the dangerous structure and safely guide the medical instrument without puncturing the dangerous structure. In particular, the intersection is an intersection having a predetermined diameter. The intersection itself may be used as a predetermined or planned control area.

[0038] Preferably, the evaluation unit is configured to evaluate that the spatial relationship between a) the tracking position and b) the intervention path satisfies a predetermined criterion when the deviation of the tracking position from the intervention path exceeds a threshold deviation value indicating the allowable deviation from the intervention path of the tracking position. Thus, the threshold deviation value defines, for example, how much deviation of the tracking position is allowed before it is considered necessary to interrupt the surgical procedure to provide a control image.

[0039] Alternatively or additionally, the evaluation unit is configured to evaluate whether a confidence value associated with the tracking position is below a threshold confidence value of the tracking position, and the output unit is further configured to provide a warning signal when the confidence value is below the threshold confidence value.

[0040] Thus, the threshold confidence value defines how well the tracking position is expected to represent the actual position of the medical device relative to the patient. It is acceptable for the tracking position to deviate to some extent from the actual position, but if the deviation of the tracking position from the actual position is too large, this tracking position can no longer be used with sufficient confidence during the surgical procedure. The transition from an acceptable deviation to an unacceptable deviation is defined by the threshold confidence value and is selected, for example, according to the type of surgical procedure being performed or, for example, according to the specific anatomical structure of the patient.

[0041] To evaluate the reliability of a tracked position, the tracked position provided by a position providing unit further provides a relevant reliability value that can be compared with a threshold reliability value. The reliability value of the tracked position is generated by the position providing unit or by the tracking system. For example, to generate a reliability value, a previous tracked position can be compared with the current tracked position to determine whether the current tracked position is on the same path as the previous tracked position. When the tracked position is determined by an electromagnetic tracking system, the detected distortion in the generated electromagnetic field is also considered to determine the reliability value of the tracked position. Thus, based on the provided reliability value, it is possible to detect whether the tracked position is reliable. If the reliability value indicates that the tracked position is relatively reliable, i.e., above the threshold reliability value, the medical device can be assumed to be on the intervention path. A sufficiently reliable tracked position also means that, using a planned control region, for example, puncturing of a dangerous structure can be avoided. On the other hand, if the comparison between the reliability value of the tracked position and the threshold reliability value results in a high uncertainty regarding this tracked position, an additional control image not induced by a pre-set control region is provided. Based on the control image, the tracking operates to re-calibrate the medical device. The threshold for activating the re-calibration depends, for example, on the distance to a dangerous structure such as an organ. Thus, when the reliability value is below the threshold reliability value, a control image is captured to verify the position of the medical device.

[0042] In particular, the control unit may be further configured to control the image providing unit to provide a control image based on the provided warning signal. The warning signal induces a visual or audible signal indicating that the reliability value is below the threshold reliability value. The warning signal may be used to warn the clinician that the reliability of the tracked position is low.

[0043] Alternatively or additionally, the evaluation unit may be configured to evaluate that the spatial relationship between a) the tracking position and b) the intervention path meets a predetermined criterion when the tracking position is closer to the target position than a threshold target distance to the target position.

[0044] The threshold target distance can also be set to zero (0) such that the spatial relationship between a) the tracking position and b) the intervention path meets a predetermined criterion when the tracking position is at the target position. Furthermore, the threshold target distance is preferably greater than 0. When the tracking position is closer to the target position than the threshold target distance to the target position, a control image is captured before actually reaching the target position. Preferably, the intervention path ends at the target position, i.e., the intervention path can connect the incision point and the target position. Thus, when the medical device reaches the end of the intervention path, at the same time, the medical device reaches the target position. The threshold target distance is preferably defined with respect to the intervention path and, for example, has an end of the intervention path. Thus, before reaching the end of the intervention path, the tracking position may be closer to the target position than the threshold target distance. Thus, a predetermined criterion is met and a control image is provided before actually reaching the target position. For example, the target position may be within a dangerous structure such as an organ, and the threshold target distance may be defined to represent the distance between the target position and the outer surface of the dangerous structure. In this case, the predetermined criterion is met when the medical device enters the dangerous structure before reaching the target position inside. In one example, the target position may be a kidney stone removed from the kidney, which is a dangerous structure, via minimally invasive surgery. The threshold target distance may be selected to represent the distance from the kidney stone to the outer surface of the kidney such that the predetermined criterion is met when the medical device enters the kidney before reaching the kidney stone to be removed.

[0045] Preferably, the dangerous structure providing unit has a neural network trained to receive an initial image as input and provide the position of the dangerous structure in the initial image as output. The trained neural network is a multi-scale neural network or a recurrent neural network (RNN), for example, but not limited to, a recurrent neural network such as a gated recurrent unit (GRU) or a long short-term memory (LSTM) recurrent neural network. Alternatively, the neural network may be a convolutional neural network (CNN). The training data used to train the neural network represents images of previously identified dangerous structures. Thereby, the neural network can be adapted to identify dangerous structures in the provided initial image.

[0046] Furthermore, according to the present invention, a method for assisting a surgical procedure is proposed, the method comprising: providing an initial image of a target position within a patient's body; providing an intervention path in the initial image for guiding a medical instrument to the target position; providing the position of a dangerous structure in the initial image; providing a tracked position of the medical instrument; using a determination unit to determine a spatial relationship between the tracked position and the planned intervention path; evaluating whether a) the tracked position and b) the spatial relationship between the position of the dangerous structure and / or the intervention path satisfy a predetermined criterion; and providing an output signal indicating whether the spatial relationship satisfies the predetermined criterion. The method has.

[0047] providing an intervention path in the initial image for guiding a medical instrument to the target position, and providing the position of a dangerous structure in the initial image The order may be reversed in different variants of the method. The method according to the invention can be carried out using the above-described system for assisting a surgical procedure.

[0048] The method comprises providing a control image of the target position when the output signal indicates that the evaluated spatial relationship meets a predetermined criterion and further comprises.

[0049] Furthermore, according to the invention, there is proposed a computer program comprising instructions for performing the steps of the method defined above when executed on a computer. This computer program can be stored on a non-transitory computer-readable data medium.

[0050] In summary, the invention relates, inter alia, to using the tracked position of a medical instrument to estimate the position of the distal end of the medical instrument relative to a dangerous structure and, when the tracked position of the distal end is too close to the dangerous structure, recommending obtaining an updated control image of the target position.

[0051] In particular, the above-described systems and methods for assisting surgical procedures are used in combination with a plurality of surgical procedures for which preoperative 3D planning imaging examinations are, or are usefully, performed. In particular, they are used in combination with minimally invasive surgery in which surgical access, i.e., the incision point and the trajectory of the surgical instrument within the body, is determined in the initial images acquired during the 3D planning imaging examination. For example, the above-described systems and methods for assisting surgical procedures are used in percutaneous nephrolithotomy (PCNL), which is a surgical kidney stone removal procedure in which the optimal access channel to the renal calyx for reaching the stone is planned on a 3D-CT scan and then, during the surgery, is dilated through the skin and renal parenchyma to allow insertion of an endoscopic device for stone fragmentation and removal. Typically, accessing the kidney should enter the selected renal calyx at the correct angle to avoid bleeding and is technically difficult due to the movement of the kidney with respiration. Furthermore, PCNL procedures are often performed with the patient in the prone position, while diagnostic CT is performed with the patient in the supine position. In such surgeries, the above-described systems and methods for assisting surgical procedures are used to improve patient safety while minimizing the required X-ray exposure.

[0052] It should be understood that the system according to claim 1, the method according to claim 10, and the computer program according to claim 11 have similar and / or identical preferred embodiments, in particular as defined in the dependent claims.

[0053] It should also be understood that the preferred embodiments of the present invention can be in the form of any combination of the dependent claims or of the above embodiments with each of the independent claims. These and other aspects of the present invention will become apparent from, and will be elucidated with reference to, the embodiments described hereinafter.

Brief Description of the Drawings

[0054]

Figure 1

Figure 2

Figure 3

Figure 4

DETAILED DESCRIPTION OF THE INVENTION

[0055] Figure 1 shows a system 100 for assisting in a surgical procedure. In particular, the system 100 can assist a clinician in safely guiding a medical instrument (not part of the system 100), such as a needle, towards a target location, such as a renal calyx of a kidney.

[0056] The system 100 has an image providing unit 102 for providing an initial image of a target location within a patient's body. The image providing unit 102 may be, for example, a stationary or movable C-arm (exemplarily shown in this example) having an X-ray source 113 and an X-ray detector 115 for detecting X-rays 117 that have passed through the patient's body 119, or a CBCT device or an ultrasonic device. The C-arm can also be part of a CBCT device. The image providing unit 102 may be configured to receive image data representing the initial image from an external imaging device. Thus, the imaging device may be either the image providing unit 102 or an external imaging device for recording the initial image, and may be configured to record a projection X-ray image in a single-exposure mode or a fluoroscopy mode, such as a continuous fluoroscopy mode. In particular, the initial image is a planned image showing a target location within the patient's body, such as a planned CT scan.

[0057] System 100 further includes an intervention path providing unit 104 configured to process an initial image 103 provided by an image providing unit 102. In particular, the intervention path providing unit 104 is configured to provide an intervention path 101 in the initial image 103 along which a medical device is guided to a target position shown in the initial image 103. For example, the intervention path providing unit 104 can have software configured to define the intervention path 101 of the medical device. After processing the initial image 103 using the intervention path providing unit 104, the initial image 103 can be combined with an overlay of the intervention path 101 for inserting the medical device.

[0058] System 100 further includes a dangerous structure providing unit 106 configured to process the initial image 103 to provide a position 105 of a dangerous structure in this initial image 103. The dangerous structure 105 can be an organ or blood vessel that should not be punctured by the medical device on the way towards the target position. To provide the position 105 of the dangerous structure in the initial image 103, the dangerous structure providing unit 106 can have a neural network trained to identify the position of the dangerous structure when the initial image is provided and output an initial image having a label representing the position of the dangerous structure in this initial image. After being processed by the intervention path providing unit 104 and the dangerous structure providing unit 106, an overlay indicating the intervention path may be added to the processed initial image 103, as well as the position of at least one dangerous structure 105.

[0059] Furthermore, system 100 has a position providing unit 108 configured to provide a tracked position 109 of a medical instrument. In particular, the position providing unit 108 can receive the tracked position 109 of the medical instrument during a surgical procedure. Thus, the received tracked position 109 of the medical instrument represents the position of the medical instrument relative to the patient during the surgery. The graphic representation of the tracked position 109 provided by the position providing unit 108 is included in the initial image 103 and is updated when the position providing unit 108 receives a new tracked position 109. In this way, it is possible to follow the tracked position 109 of the medical instrument in the initial image 103 while performing a surgical procedure. The position providing unit 108 can receive the tracked position 109 from a tracking system (not part of system 100) configured to track the position of the medical instrument relative to the patient. In particular, the tracking system tracks the position of a medical instrument, such as a needle, relative to an imaging device, such as a C-arm or CBCT, in the coordinate system of this tracking system. The tracking system may be an optical tracking system or an electromagnetic tracking system. The tracking system may be configured to track the position of the medical instrument in six degrees of freedom (6DoF: six degrees of freedom). The tracking system can further use knowledge of the X-ray geometry of the imaging system, such as, for example, the focus-detector distance (FDD) or the detector active-area size of the detector.

[0060] System 100 has an evaluation unit 110 configured to evaluate whether a spatial relationship between the tracked position 109 provided by the position providing unit 108 and the position of the dangerous structure 106 provided by the dangerous structure providing unit 106 and / or the intervention path 101 provided by the intervention path providing unit 104 meets a predetermined criterion. The predetermined criterion used by the evaluation unit 110 for evaluating the spatial relationship is defined in various ways.

[0061] For example, in the initial image, the dangerous structure providing unit 106 may define a safety distance with respect to the provided dangerous structure. The safety distance is a distance corresponding to this safety distance and can be used to generate a margin or envelope that at least partially surrounds the dangerous structure. The safety distance is used to define a buffer zone around the dangerous structure. One way to define the predetermined criterion can be based on the safety distance such that when the medical device is closer to the dangerous structure than the defined safety distance, the predetermined criterion is considered to be met. In that case, since the medical device is closer to the dangerous structure as defined by the safety distance, the risk of piercing the dangerous structure increases. When tracing the intervention path using the medical device, there is a possibility that the defined safety distance, such that the medical device approaches the dangerous structure closer than the safety distance, actually intersects the intervention path. In this case, the threshold crossing distance can be defined for the intersection, for example, the intersection point, and the predetermined criterion is that the medical device is closer to the intersection as provided by the threshold crossing distance. In this case, the medical device has actually reached the intended intersection of the safety distance and the intervention path.

[0062] Another way to define the predetermined criterion can be based on the deviation of the tracking position from the intervention path. For example, a threshold deviation value indicating the allowable deviation of the tracking position from the intervention path can be defined. The evaluation unit 110 evaluates the deviation of the tracking position with respect to the threshold deviation value, and when the deviation of the tracking position exceeds the threshold deviation value, it can be determined that the spatial relationship meets the predetermined criterion. In this case, the medical device is expected to have moved too far away from the intervention path.

[0063] In addition to the aforementioned predetermined criteria, the reliability of the tracked position may be determined. For example, the position providing unit 108 is configured to provide the tracked position together with a reliability value associated with this tracked position. To evaluate the reliability, a threshold reliability value can be defined. If the provided reliability value deviates too much from the threshold reliability value, the evaluation unit may provide a warning signal indicating that the reliability value indicates a low reliability. This means that the tracking of the position of the medical device is performed with an increased uncertainty regarding the actual position of this medical device.

[0064] The system 100 further has an output unit 112 configured to provide an output signal indicating whether the spatial relationship satisfies a predetermined criterion evaluated by the evaluation unit 110. Optionally, the output unit 112 may further provide a warning signal if it is found that the correct reliability of the tracked position is below the threshold reliability value.

[0065] The system 100 optionally has a control unit 114 configured to receive and process an output signal from the output unit 112. In particular, when the output signal indicates that the spatial relationship evaluated by the evaluation unit 110 satisfies a predetermined criterion, the control unit 114 can control the image providing unit 102 to provide a control image of the target position. This control image may be generated by the same imaging device used to capture the initial image, or may be generated by an imaging device that is not the same imaging device. In particular, the imaging device used to generate the control image may be the image providing unit 102 itself, or may be an external imaging device such as, for example, a C-arm, CBCT or ultrasonic device.

[0066] It is beneficial if the evaluation unit 110 continuously checks whether the spatial relationship meets at least one of the above-described predetermined criteria. However, in a surgical procedure, there are parts in the patient's body that are associated with a low risk of damaging dangerous structures, and there are other parts in the patient's body that are associated with a high risk of damaging dangerous structures.

[0067] For this purpose, the control unit 114 is configured to define in the initial image one or more control regions that are control points with a predetermined diameter indicating the position of the medical device that meets the predetermined criteria. In particular, when the medical device reaches the control region, it is beneficial to provide a control image of the target position. Thereby, the risk of puncturing dangerous structures is reduced, and thus the safety of the surgical procedure is improved.

[0068] The system 100 further has an optional visualization unit 116 configured to visualize in the initial image 103 the tracking position 109 of the medical device provided by the position providing unit 106 and one or more control regions provided by the control unit 114. Further, the visualization unit 116 is configured to visualize in the initial image 103 the intervention path 101 provided by the intervention path providing unit 104 and the position of the dangerous structure 105 provided by the dangerous structure providing unit 108. Further, the visualization unit 116 can visualize a safety distance, a related safety margin 111, or a safety envelope in the initial image 103.

[0069] Therefore, using the system 100, it is possible to warn the clinician if the medical device approaches a dangerous structure, such as an organ, too dangerously during the insertion phase. This makes it possible to replace continuous X-ray fluoroscopy imaging with intermittent one-directional irradiation X-ray exposure at several time points selected by the system 100, for example, a dose reduction regime of controlled exposure. Thereby, the system 100 can safely guide the medical device to the target position while minimizing X-ray exposure.

[0070] FIG. 2 shows a flowchart representing a method for assisting a surgical procedure. The method described below can be executed using the system 100 described with reference to FIG. 1.

[0071] The method starts with a planning phase that includes providing an initial image of a target location within a patient's body (step S1). The initial image can be a planning CT scan and can be used to identify and depict all relevant structures. Considering an exemplary case of kidney stone removal, the initial image shows the kidney and anatomical structures such as the urinary system (renal pelvis and calyces), the pulmonary pleura, and the ureter anatomical structures, as well as the kidney stone. One or more of such anatomical structures can be identified during the planning phase. In the example described above, the kidney stone is the target location, and additional anatomical structures may be identified as critical structures.

[0072] In particular, in the method, one or more positions of the dangerous structure are provided (step S2). The dangerous structure is identified in the initial image using a neural network trained accordingly. For example, using software, the identified dangerous structure is labeled in the initial image. Rather, piercing a dangerous structure such as an organ or anatomical structure using a medical device is harmful to the patient. Therefore, the intervention path of the medical device should not cross the dangerous structure. In addition to labeling the dangerous structure, it is beneficial to calculate a safety margin around the identified dangerous structure. The safety margin around the dangerous structure is defined based on positioning the patient, for example, positioning the patient in the prone position, and / or the overall spatial range of the expected movement caused by the expected respiratory movement for the intervention path. The safety margin is visualized together with the identified dangerous structure as an additional overlay on the initial image. The range of the safety margin is also affected by the expected accuracy of tracking and / or alignment of the medical device. Since the surgery is typically performed under general anesthesia, the anesthesiologist may induce apnea to control the movement when the clinician advances the medical device, which is reflected in a reduction in the range of the safety margin.

[0073] The method further comprises providing an intervention path by which a medical device is guided to a target position (step S3). The intervention path is visualized in the initial image together with the identified dangerous structure and the safety margin around the dangerous structure. For example, the intervention path may be manually input and then provided, for example, by an intervention path providing unit. The intervention path can also be determined by software using, for example, a trained neural network and then provided by the intervention path providing unit. As part of the planning phase, first, in the initial image, the dangerous structure and the safety margin around the identified dangerous structure are provided. For example, the initial image may be visualized with an overlay of the dangerous structure and another overlay with the determined safety margin. Then, the intervention path is selected and provided such that the contact between the intervention path and the safety margin and the identified dangerous structure is minimized. However, there are situations where it is unavoidable to cross at least one of the dangerous structure and the safety margin in order to reach the target position.

[0074] The intersection of the intervention path with the safety margin and / or the dangerous structure is calculated using, for example, software and defined as a control region that is a control point with a predetermined diameter. The control region defines, for example, the position of the medical device for which it is recommended to provide a control image to verify the actual position of the medical device with respect to the dangerous structure.

[0075] During the operation, when CBCT is used, at the start of the intervention, the transformation T between the planned CT and the CBCT CT→CBCTTo determine, a calibration CBCT is required. The accuracy of the alignment depends on various factors, such as the image quality and field of view of the CBCT, the level of contrast of the target position, and the magnitude of the deformation between the CT and the CBCT, e.g., the magnitude of the non-linear deformation. Considering an example of a kidney intervention, the alignment is performed by segmenting the kidney and internal structures, e.g., renal calyces and renal pelvis, and using this information for a coarse rigid alignment of the images. A more accurate, e.g., non-linear alignment can be obtained by subsequent intensity-based refinement of the transformation. Instead of using CT to perform the said planning, it is possible to use CBCT at the start of the intervention. In such a case, the planning CT is not required, and as a result, the components of the alignment are not required either. Consequently, providing the dangerous structures and the intervention path is done directly on the CBCT.

[0076] During the operation, the position of the medical instrument is tracked using, for example, an optical tracking system or an electromagnetic tracking system. This tracked position is then provided, for example, by a position providing unit (step S4). The provided tracked position of the medical instrument can be received from the tracking system.

[0077] In the method, it is further evaluated (step S6) whether the spatial relationship between the tracking position and the position of the dangerous structure and / or the intervention path meets a predetermined criterion (step S5). Further, an output signal indicating whether the spatial relationship meets a predetermined criterion is provided (step S7). As described above, when the tracking position of the medical instrument is closer to the dangerous structure than the safety distance defined for the dangerous structure and converted into a safety margin around the dangerous structure, the predetermined criterion is met. Alternatively, the predetermined criterion is the case where the tracking position is closer to the intersection of the provided intervention path and the defined safety distance than a predetermined threshold at the section distance. Also, alternatively, the predetermined criterion may be met when the deviation of the tracking position from the intervention path exceeds a threshold deviation value indicating the allowable deviation of the tracking position from the intervention path. Furthermore, alternatively, the predetermined criterion may be met when the tracking position is closer to the target position than a predetermined target distance defined for the target position.

[0078] In the method, when the output signal indicates that the evaluated spatial relationship meets a predetermined criterion, it is further possible to provide a control image of the target position (step S8). Thus, during the intervention, in the method, a recommendation can indicate to provide a control image, and this recommendation is based on the evaluation of the spatial relationship between the tracking position and the dangerous structure and / or the intervention path. Using the control area defined in the initial image, for example, when the medical instrument is near the dangerous structure, there may be a plurality of planned control exposures to provide a control image. This is the case when the distance from the tip of the medical instrument to the dangerous structure is smaller than, for example, a predetermined safety margin. Preferably, when the medical instrument is close to the target position, for example, when reaching the kidney before removing a kidney stone, another planned control exposure is performed.

[0079] Furthermore, in the method, for example, when a tracking system or a position providing unit indicates that the uncertainty regarding the tracking position is relatively high, it can further include performing ad-hoc control exposure to provide a control image. Another reason for performing ad-hoc control exposure is when the planned control exposure indicates that it deviates significantly from the original surgical plan or the alignment is inaccurate. It is beneficial when the C-arm is positioned to enable optimal evaluation of the position of the medical instrument relative to the dangerous structure for control exposure. For example, the C-arm is positioned such that a control image in a direction orthogonal to the shortest path between the tip of the medical instrument and the surface of the dangerous structure is captured. In the method, it may be more preferable when the intervention path in the initial image is provided in such a way that the minimum number of planned control exposures is required.

[0080] FIG. 3 shows an initial image 300 as visualized by a visualization unit, for example, the visualization unit 116 of the system 100, which is described with reference to FIG. 1. The visualization unit may be a graphical user interface (GUI) or may have a graphical user interface (GUI). The initial image 300 is a planned CT scan. As an overlay on the initial image, anatomical structures are identified and labeled. One of these anatomical structures is the target location 302, which is a kidney stone in the kidney 304. Another identified anatomical structure is the dangerous structure 306, which is partially surrounded by a safety margin 308 disposed at a safe distance 310 from this dangerous structure 306. Also, an intervention path 312 is provided in the initial image 300. The intervention path 312 represents a path along which a medical instrument is guided to reach the target location 302. Thus, the intervention path 312 connects the incision point and the target location 302. The actual path of the medical instrument is determined using a tracking system, and the tracking position 314 of this medical instrument is visualized in the initial image 300 during a surgical procedure. This requires converting the coordinates of the position of the medical instrument in the coordinate system of the tracking system to the coordinate system of the C-arm used to record the initial image 300. During surgery, when CBCT is used to record a control image or a new initial image, to visualize the tracking position of the medical device, the inverse T of the said conversion CT→CBCT needs to be determined. As can be seen in the initial image 300, the tracking position 314 of the medical instrument does not exactly follow the planned intervention path 312 but is slightly displaced from this path. If the displacement of the tracking position 314 from the intervention path 312 exceeds a predetermined threshold displacement value, an ad hoc control exposure for providing a control image is triggered.

[0081] The intersection of the access path 312 with the safety margin 308 provides a first control region 316 in the initial image 300. The first control region 316 defines the position of the medical device at which a planned control image is provided. Thus, if it is expected that the medical device will reach the first control region 316, the control unit can control the image providing unit to provide a control image of the target position 302. Before reaching the target position 302, a second control region 318 is provided at a position where the medical device is expected to puncture the kidney 304. In this way, a planned control image is provided immediately before reaching the target position 302 to verify the actual position of the medical device relative to the target position 302.

[0082] In FIG. 4, the spatial registration between the initial image 400 and the preoperative cone beam CT 402 is shown. Further, an overlay of the access path 404 is visualized. For this purpose, a transformation T CT→CARM , or when CBCT imaging is used, T CT→CBCT is calculated using a registration algorithm. Optionally, when non-CBCT imaging is used, the application of a tracking system for establishing a transformation T C-arm→medical instrument between the C-arm coordinate system and the tracking system coordinate system may be used.

[0083] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, by considering the drawings, the disclosure, and the appended claims.

[0084] In the claims, the term "comprising" does not exclude other elements or steps, and does not exclude a plurality even if it does not state that there are a plurality.

[0085] A single unit or device may fulfill the functions of several items recited in the claims. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used advantageously.

[0086] Procedures performed by one or more units or devices, such as providing an initial image, providing an access path in the initial image, providing the location of a dangerous structure in the initial image, providing the tracked position of a medical instrument, etc., can be performed by any other number of units or devices. These procedures can be implemented as program code means of a computer program and / or as dedicated hardware.

[0087] The computer program may be stored / distributed on a suitable medium supplied together with or as part of other hardware, for example, an optical storage medium or a solid-state medium, but may also be distributed in other forms, for example, via the Internet or other wired or wireless telecommunications systems.

[0088] Any reference signs in the claims should not be construed as limiting their scope.

[0089] The present invention relates to a system and method for assisting in image-guided surgical procedures, in particular image-guided percutaneous needle interventions such as, for example, the removal of kidney stones. Using the system and the method, it is possible to provide automatic recommendations for control scans in critical situations during the intervention. For this purpose, in the initial image, dangerous structures are identified and an access path to the target position is provided. Using an evaluation unit, it is possible to evaluate whether the spatial relationship between a) the tracked position of a medical instrument and b) the position of the dangerous structure and / or the access path meets a predetermined criterion. When it is found that the predetermined criterion is met, the acquisition of a control scan is automatically recommended.

Claims

1. In a system for assisting surgical procedures, the system is: An image-providing unit configured to provide an initial image of a target location within the patient's body. An intervention path providing unit configured to provide an intervention path in which a medical device is guided along the initial image to the target position, A hazardous structure providing unit configured to provide the location of the hazardous structure in the initial image, A position providing unit configured to provide the tracking position of the medical device, an evaluation unit configured to evaluate whether the spatial relationship between a) the tracking position and b) the location of the hazardous structure and / or the intervention route meets predetermined criteria, An output unit configured to provide an output signal indicating whether the spatial relationship satisfies the predetermined criteria, and A control unit configured to control the image providing unit to provide a control image of the target position when the output signal indicates that the evaluated spatial relationship satisfies the predetermined criteria. A system that has

2. The system according to claim 1, wherein the control unit defines a control region where the predetermined criteria are met, and controls the image providing unit to provide a control image when the medical device reaches the control region.

3. The initial image further includes a visualization unit configured to visualize the tracking position and the control region. The system according to claim 2, wherein the control region indicates the position of the medical device to which the control image has been or is to be provided.

4. The system according to claim 3, wherein the visualization unit is further configured to visualize the intervention path, and the control area is located on the intervention path.

5. The hazardous structure providing unit is configured to define a safe distance to the hazardous structure, and i) The evaluation unit is configured to evaluate that if the tracking position of the medical device is closer to the hazardous structure than the defined safety distance, a) the spatial relationship between the tracking position and b) the position of the hazardous structure satisfies the predetermined criteria, and / or ii) The hazardous structure providing unit is configured to determine the intersection of the provided intervention path and the defined safety distance, and the evaluation unit is configured to evaluate that if the tracking position is closer to the intersection than a predetermined threshold intersection distance, a) the spatial relationship between the tracking position and b) the location of the hazardous structure and the intervention path meets the predetermined criteria. The system according to any one of claims 1 to 4.

6. The system according to claim 1, wherein the evaluation unit is configured to evaluate that a) the spatial relationship between the tracking position and b) the intervention path satisfies the predetermined criteria if the deviation of the tracking position from the intervention path exceeds a threshold deviation value indicating an acceptable deviation of the tracking position from the intervention path.

7. The evaluation unit is configured to evaluate whether the confidence value associated with the tracking location is below the threshold confidence value of the tracking location. The output unit is further configured to provide a warning signal if the confidence value falls below the threshold confidence value. The system according to claim 1.

8. The system according to claim 1, wherein the evaluation unit is configured to evaluate that a) the spatial relationship between the tracking position and b) the intervention path satisfies the predetermined criteria if the tracking position is closer to the target position than the threshold target distance to the target position.

9. The system according to claim 1, wherein the hazardous structure providing unit has a neural network trained to receive the initial image as input and to provide the location of the hazardous structure in the initial image as output.

10. In a method for assisting surgical procedures, the method is: A step of providing an initial image of the target location within the patient's body, A step of providing an intervention path for guiding a medical device to the target position in the initial image, The step of providing the location of the hazardous structure in the initial image, A step of providing the tracking location of the medical device, A step of determining the spatial relationship between the tracking location and the planned intervention route, a) A step of evaluating whether the spatial relationship between the tracking location and b) the location of the hazardous structure and / or the intervention route meets predetermined criteria. The steps include providing an output signal indicating whether the spatial relationship satisfies the predetermined criteria, and If the output signal indicates that the evaluated spatial relationship satisfies the predetermined criteria, the step of providing a control image of the target position. A method of having.

11. A computer program that, when executed on a computer, includes instructions for performing steps of the method according to claim 10.

12. A non-temporary computer-readable data medium for storing the computer program described in claim 11.

Images