System and method for assisting a surgical procedure

Abstract

The present invention relates to a system and method for assisting image-guided surgical procedures, in particular image-guided percutaneous needle interventions such as for kidney stone removal. With the system and method, automatic recommendations of control scans can be provided during critical phases during an intervention. For this purpose, in the initial image, a risk structure is identified and an interventional path to the target location is provided. With an evaluation unit, it is possible to evaluate whether the spatial relationship between a) the tracking position of the medical instrument and b) the location of the risk structure and / or the intervention path satisfies predefined criteria. If predefined criteria are found to be met, acquisition of control scans may be automatically recommended.

Description

Technical Field

[0001] The invention relates to a system for assisting a surgical procedure, a method for assisting a surgical procedure, a computer program and a non-transitory computer-readable data medium. Background Art

[0002] In minimally invasive surgery, it is known to plan a priori the incision points and trajectories to anatomical regions in the patient's body. For this purpose, previously recorded fluoroscopic or tomographic images showing the patient's anatomical region are usually used. During the operation, continuous low-dose X-ray imaging or X-ray imaging at selected times is usually performed to verify the position of the medical instrument with control images while guiding the medical instrument to the patient's anatomical region along the planned path. Therefore, the patient and sometimes even the clinician performing the operation are exposed to some degree of X-ray radiation during the operation.

[0003] EP 3797724 A1 discloses a system for performing image-guided medical procedures, wherein the system uses three-dimensional fluoroscopic construction to plan and navigate a percutaneously inserted device, such as a biopsy tool, from an entry point into the patient to a target within the patient.

[0004] WO 2018 / 005842 A1 discloses performing a first fluoroscopic sweep including a target area to obtain a first fluoroscopic image. The position of the target is determined based on the first fluoroscopic image, and the position of the entry point is determined based on the position of the target and the first fluoroscopic image. Then, the needle is inserted or advanced, wherein an additional fluoroscopic sweep is performed thereafter to obtain an additional fluoroscopic image. The position and direction of the inserted needle is determined based on the additional fluoroscopic image, and the distance between the tip of the needle and the target is determined. It is then checked whether the tip of the needle has reached the target, wherein, if this is not the case, the needle is further advanced and an additional additional fluoroscopic sweep is performed to obtain an additional additional fluoroscopic image, and if the tip of the needle has reached the target, the method ends.

[0005] EP 3932357 A1 discloses a system for assisting a user in placing a penetrating device in tissue, such as a pedicle screw in a pedicle of a vertebra. The system generates a virtual view from the perspective of the penetrating device tip in the tissue in the direction of a path through a tissue model. The virtual view is generated based on tracking information indicating the posture, model and path of the penetrating device, wherein the virtual view is configured so that it indicates the direction in which the user should move the penetrating device when placing the penetrating device in the tissue. For example, it can show a virtual tunnel arranged along the path.

[0006] WO 2012 / 056386 A1 discloses a system comprising a shape sensing or positioning system and an imaging module, wherein the shape sensing or positioning system is coupled to an interventional device to measure spatial characteristics of the interventional device in an object, and the imaging module is configured to receive the spatial characteristics and generate one or more control signals according to the spatial characteristics. The system also includes at least one imaging device configured to image the object according to the control signal.

[0007] In order to reduce X-ray exposure, it is proposed to use a robot to perform surgical procedures based on an anatomical roadmap. A corresponding arrangement using an interventional robot is disclosed in WO2019 / 092225 A1. Therein, an interventional system is disclosed, which should facilitate the reduction of radiation exposure in an interventional environment by providing an interventional controller for robotic navigation of interventional instruments based on a single static anatomical roadmap, which is automatically updated by the interventional controller when the robot reaches a critical navigation position within the anatomical roadmap and / or when the patient's physiological state is critical.

[0008] However, it remains desirable to further improve patient safety during surgical procedures while reducing X-ray exposure of the patient and clinician (if any) during surgery. Summary of the invention

[0009] The object of the present invention is to provide a system, a method, a computer program and a non-transitory computer-readable data medium for assisting a surgical procedure with increased reliability. Preferably, with the system, the method, the computer program and the non-transitory computer-readable data medium, a surgical procedure can also be performed with a relatively reduced X-ray exposure of the patient.

[0010] Regarding the system, a system for assisting a surgical procedure is proposed, which comprises an image providing unit, an intervention path providing unit, a risk structure providing unit, a position providing unit, an evaluation unit and an output unit. The image providing unit is configured to provide an initial image of a target location inside a patient's body. The intervention path providing unit is configured to provide an intervention path in the initial image, along which a medical instrument should be guided to the target location. The risk structure providing unit is configured to provide the location of the risk structure in the initial image. The position providing unit is configured to provide a tracking position of the medical instrument. The evaluation unit is configured to evaluate whether a) the spatial relationship between the tracking position and b) the location of the risk structure and / or the intervention path meets a predefined criterion. The output unit is configured to provide an output signal indicating whether the spatial relationship meets the predefined criterion.

[0011] The invention includes the recognition that, in order to increase the reliability of the surgical procedure, it is advantageous to plan the surgical procedure a priori using an initial image of a target location inside the patient's body. Based on the initial image of the target location, an intervention path can be planned along which the medical instrument should be guided to the target location. Furthermore, it is advantageous if the position of the medical instrument inside the patient can be tracked and the tracked position of the medical instrument can be evaluated relative to the planned intervention path.

[0012] The present invention now includes the further recognition that on its way to the target location inside the patient's body, the medical instrument will usually pass through less critical areas inside the patient, so that a slight deviation from the intervention path will most likely not cause any serious damage to the patient. However, there may also be one or more risk structures that may be very close to the intervention path in the initial image. The risk structure may be a structure that should not be damaged (e.g., due to puncture) by the medical instrument to avoid causing more serious damage to the patient. Using the system, the tracked position of the medical instrument can be used to evaluate the spatial relationship between the tracked medical instrument and the location of the risk structure and / or the intervention path. In particular, using the evaluation of the system, it can be evaluated whether the spatial relationship meets predefined criteria. Due to the evaluation of the tracked position of the medical instrument and the location of the risk structure and / or the intervention path, it can be determined whether the predefined criteria are met. Based on whether the predefined criteria are met, some correction of the surgical procedure can be performed or the surgical procedure can be continued with increased confidence. Therefore, the evaluation of the spatial relationship between the tracking position and the location of the risk structure and / or the intervention path relative to the predefined criteria improves the reliability of the surgical procedure and reduces the risk of causing damage to the patient's risk structure. Thus, the system helps to safely guide the medical instrument to the target location inside the patient without causing damage to nearby risk structures. Thus, the system allows to increase the success rate of surgical procedures and helps to improve the health status of the patient.

[0013] To provide the initial image, the image providing unit may be configured to receive the initial image from another external device (eg an imaging device).The image providing unit may also be configured to generate the initial image, for example, the image providing unit itself may be an imaging device.

[0014] The initial image may be a fluoroscopic image, a tomographic image or an ultrasound image. Thus, the initial image may be generated by a fluoroscopic imaging device such as a C-arm having an X-ray source and an X-ray detector. The initial image may also be generated by an X-ray computed tomography (CT) device such as a cone beam computed tomography (CBCT) device. The initial image may also be generated by an ultrasound device. Preferably, the initial image is captured before the surgical procedure, preferably as a planning image. The initial image may also be captured directly at the start of the surgical procedure. The initial image may also be recorded during the surgical procedure, for example, to adjust the surgical plan.

[0015] The tracking position of the medical instrument as provided by the position providing unit of the system can be generated by the tracking system and provided to the position providing unit. In particular, the position providing unit can be configured to receive the tracking position from the tracking system. The tracking system can be an optical tracking system. The optical tracking system generally uses infrared light to determine the position of a medical instrument equipped with an active marker (such as a light emitting diode (LED)) or a passive shield (such as a sphere or a disk coated with a highly reflective material), which is attached to the medical device and detected using a camera. The tracking system can be an electromagnetic tracking system. The electromagnetic tracking system generally includes a field generator for generating an electromagnetic field and an electromagnetic sensor attached to the medical instrument for tracking. In order to use the tracking position of the medical instrument as tracked by the tracking system, the tracking position defined in the coordinate system of the tracking system must be transformed into the coordinate system of the initial image. This includes patient registration and calibration of the medical instrument. More generally, the position of a marker set on the proximal end of the medical instrument using a non-imaging tracking system, for example, based on an optical marker or an electromagnetic marker, can be achieved by registering the tracking position to a reference X-ray image of the patient. The position of the distal end of the interventional device relative to critical structures in the subject can then be estimated based on the tracked positions of the markers.The position can also be tracked with the aid of a fixed physical guide (eg, a clamp) between the imaging device and the medical instrument.

[0016] The position of the medical instrument may also be tracked, and thus provided using a mobile application on a standard mobile consumer device (e.g., a smartphone or tablet computer), which relies on a database of one or more digital computer-aided design (CAD) geometries of the medical instrument and the housing of the employed medical imaging device. Using the mobile application, a geometric interrelationship, such as the 3D position of the imaging device and the medical instrument, may be established by matching point clouds, such as produced by a light detection and ranging (LIDAR) or similar depth ranging camera system. As an alternative to using LIDAR or the like, the 3D position of the imaging device and the medical instrument may be extracted from a 2D image feed (i.e., a 2D image feed of a webcam or the like). The method is based on the recognition that if the shape of the imaging device (e.g., CBCT) and the shape of the medical instrument can be identified in the image feed of the detector, the spatial relationship between the imaging device and the medical instrument can be extracted. Since the position of the imaging device may be known, for example, the CBCT gantry position may be extracted from the CBCT control computer, and the spatial relationship of the gantry to the captured image may also be known, the position and orientation of the medical instrument may also be determined.

[0017] In an example, the position providing unit is configured to provide a position that has not been determined by detecting the medical instrument in the initial image or the control image as the tracked position of the medical instrument. In an example, the tracked position is used to define when the control image should be acquired, but the tracked position for this purpose is not determined by detecting the medical instrument in the control image or the initial image.

[0018] The medical instrument whose tracked position is provided by the position providing unit may be a needle or a catheter or a guidewire or some other access channel such as for an endoscope or the like. In particular, the medical instrument may be an instrument for insertion into a patient as part of a surgical procedure. The medical instrument may be a handheld device to be guided by a human, or it may be a medical instrument to be guided by a robot.

[0019] The risk structure may be a human organ (such as a kidney, heart, lung, etc.) or a vessel (such as a blood vessel, eg, a vein, etc.).

[0020] Typically, the components of the system (i.e., in particular, the image providing unit, the intervention path providing unit, the risk structure providing unit, the evaluation unit and the output unit) can be configured by hardware and software suitable for performing the corresponding functions. For example, the above-mentioned components of the system can be implemented as parts of a personal computer or other computing device, etc.

[0021] The system further comprises a control unit configured to control the image providing unit to provide a control image of the target positioning when the output signal indicates that the evaluated spatial relationship satisfies the predefined criterion.

[0022] Thus, based on the evaluated spatial relationship, the control unit may provide an automatic recommendation when to perform a control scan. The control unit may trigger the image providing unit to generate a control image, i.e. to capture the control image by itself or by requesting an external imaging device to provide the control image. Thus, the control image is only provided in certain (e.g. critical) cases where it may be necessary to evaluate the position of the tracked medical instrument relative to certain body parts (e.g. risk structures). Thus, the number of control scans may be limited to the minimum number required to ensure safe guidance of the medical instrument to the target positioning. Advantageously, with the aid of the control unit, it may therefore be avoided to provide a control scan in cases where it would not actually be needed. For example, a recommendation to acquire an updated X-ray control image as provided by the control unit may be based on the estimated proximity of the tracking position of the distal end of the medical instrument relative to the positioning of the risk structure.

[0023] Advantageously, use of a control unit allows for a dose-saving scheme of intermittent single-view X-ray exposures at several time points (i.e., controlled exposures selected by the control unit based on an evaluation of the spatial relationship between the tracking position and the positioning of the interventional path and / or risk structures) to be used instead of continuous X-ray fluoroscopic imaging.

[0024] Preferably, the control unit may control the image providing unit based on an output signal provided by the output unit and indicating whether the spatial relationship meets the predefined criteria. Based on whether the spatial relationship meets the predefined criteria, the control unit may or may not request the image providing unit to provide a control image.

[0025] As described above, in an example, the control unit may provide an automatic recommendation of when to perform a control scan based on the evaluated spatial relationship. Therefore, the control unit may be configured to control the image providing unit to provide a control image of target positioning by providing an automatic recommendation, and if a user such as a surgeon has accepted the automatic recommendation via an input device of the system, the control unit controls the image providing unit to provide a control image of target positioning. As an alternative example, if the output signal indicates that the evaluated spatial relationship meets the predefined criteria, the control unit may also directly provide a control image of target positioning without waiting for the user to accept the corresponding recommendation. In other words, in an example, the control unit is configured to control the image providing unit to provide a control image of target positioning when the output signal indicates that the evaluated spatial relationship meets the predefined criteria and when another user (e.g., by accepting the corresponding recommendation) has indicated that the control image should really be provided (e.g., should really be collected). In another example, the control unit is configured to control the image providing unit to provide a control image of target positioning when the output signal indicates that the evaluated spatial relationship meets the predefined criteria, without requiring another user to indicate that the control image of target positioning should really be provided via an input device of the system.

[0026] The control image can be used by the system as a new initial image, i.e. the intervention path providing unit can provide in the new initial image a new intervention path along which the medical instrument should be guided to the target location, and the risk structure providing unit can provide in the new initial image a new location of the risk structure. The evaluation unit can then use the new initial image for evaluation purposes. It is possible that the initial image is, for example, a fluoroscopic image and the control image that can be used as the new initial image is a tomographic or ultrasound image, or vice versa.

[0027] In particular, the control image may show changes to the scene captured in the initial image, for example because the patient may have moved or interaction between the medical instrument and the patient's anatomy may have caused a shift of the anatomy, for example the medical instrument may have pushed an organ or another structure at risk to the side.

[0028] Alternatively or additionally, the control unit may be configured for defining a control region satisfying the predefined criterion, and for controlling the image providing unit to provide a control image when the medical instrument reaches the control region.

[0029] Preferably, the control region is defined in the initial image and indicates the position of the medical instrument that meets the predefined criteria. Thus, the control region indicates the location inside the patient for which the spatial relationship meets the predefined criteria such that the control image should be captured. Thus, the control region indicates the position of the medical instrument that has or will capture the control image. Preferably, the control region is defined in the initial image before the operation. However, it may also be beneficial if the control region is defined during the operation, for example based on the control image.

[0030] For example, a control region may be defined relative to a risk structure. In particular, the control region may be positioned on the intervention path such that if the tracked medical instrument is expected to reach the control region based on the tracking position, the control unit may trigger the image providing unit to provide a control image. The control region may have a geometric structure of a point, a sphere or a 3D volume. Its size may be selected depending on its position relative to, for example, the risk structure. For example, if the control region is located near a risk structure, it may be defined to have a large size compared to a case where the risk structure is further away from the intervention path. If a critical spatial relationship between the medical instrument and the risk structure is expected, the control unit may then trigger the image providing unit to provide a control image relatively earlier (i.e. further away from the risk structure).

[0031] The system may further comprise a visualization unit configured to visualize the tracking position and the control region in the initial image. The control region preferably indicates the position of the medical instrument that has or will provide the control image. The visualization unit may further be configured to visualize the intervention path, and wherein the control region is located on the intervention path. For example, the visualization unit may be or may comprise a graphical user interface. The visualization unit may further be configured to visualize the provided risk structure. Thus, the visualization unit may visualize the planned intervention path, such as provided by the intervention path providing unit, together with the risk structure. During surgery, the visualization unit may further visualize the tracking position of the medical instrument, for example relative to the intervention path.

[0032] Preferably, the risk structure providing unit is configured for defining a safety distance relative to the risk structure. Preferably, the evaluating unit is configured for evaluating that the spatial relationship between a) the tracking position and b) the positioning of the risk structure meets the predefined criterion when the tracking position of the medical instrument is closer to the risk structure than the defined safety distance.

[0033] A specific purpose of the safety distance is to define a buffer zone relative to the risk structure, so that a control image can already be provided when the medical instrument reaches or is expected to reach the safety distance. In this case, the control image is provided before the risk structure is actually reached. Thus, the risk of piercing the risk structure can be reduced. For example, the control area can be located at the safety distance, so that the control unit can control the image providing unit to provide a control image when the medical instrument is expected to have reached the control area based on the provided tracking position.

[0034] The safety distance can be converted into a safety margin around the risk structure or, in 3D, into a safety envelope surrounding the risk structure.

[0035] For defining the safety margin, expected relative movements caused by positioning the patient before the intervention and / or expected movements caused by the patient's breathing during the intervention may be taken into account. The risk structure providing unit may be configured for defining a safety margin around the risk structure, and wherein the intervention path providing unit may be configured for providing the intervention path such that the intervention path does not intersect the risk structure and the defined safety margin.

[0036] Alternatively or additionally, the risk structure providing unit may be configured for defining a safety distance relative to the risk structure and for determining an intersection point between the provided intervention path and the defined safety distance. Preferably, the evaluation unit is configured for evaluating that the spatial relationship between a) the tracking position and b) the location of the risk structure and the intervention path meets the predefined criterion when the tracking position is closer to the intersection point than a predefined threshold intersection distance.

[0037] In this case, the intervention path intersects the safety distance so that the intervention part is provided closer to the risk structure than an extension of the safety distance. With respect to the intersection between the intervention path and the safety distance, an intersection distance may be defined. If the tracking position is closer to the intersection than a predefined threshold intersection distance, the medical instrument is expected to move relatively close to (i.e., closer than an extension of the safety distance) the risk structure. However, since the evaluation unit is configured to use a predefined criterion when the tracking position is closer to the intersection than a predefined threshold intersection distance, an output signal may be provided that may be used by the control unit to trigger the image providing unit to provide a control image. Thus, the actual spatial relationship between the medical instrument and the risk structure may be evaluated to guide the medical instrument safely without puncturing the risk structure. In particular, the intersection is an intersection having a predetermined diameter. The intersection itself may be used as a predefined or planned control area.

[0038] Preferably, the evaluation unit is configured to evaluate that a) the spatial relationship between the tracking position and b) the intervention path satisfies the predefined criterion when the deviation of the tracking position from the intervention path exceeds a threshold deviation value indicating an allowable deviation of the tracking position from the intervention path. Thus, the threshold deviation value defines how far the tracking position is allowed to deviate before it is deemed necessary to interrupt the surgical procedure, e.g. in order to provide a control image.

[0039] Alternatively or additionally, the evaluation unit may be configured to evaluate whether a reliability value associated with the tracking position is below a threshold reliability value for the tracking position, and the output unit may be further configured to provide a warning signal when the reliability value is below the threshold reliability value.

[0040] Thus, the threshold reliability value defines how well the expected tracked position represents the true position of the medical instrument relative to the patient. It is possible that a certain degree of deviation of the tracked position from the true position is permissible, however, if the deviation of the tracked position from the true position becomes too large, the tracked position can no longer be used with sufficient reliability during the surgical procedure. The transition from permissible deviation to inadmissible deviation is defined by the threshold reliability value and can be selected, for example, depending on the type of surgical procedure to be performed or, for example, depending on the specific anatomy of the patient.

[0041] In order to evaluate the reliability of the tracking position, the tracking position as provided by the position providing unit can be additionally provided with an associated reliability value that can be compared with a threshold reliability value. The reliability value for the tracking position can be generated by the position providing unit or by the tracking system. For example, in order to generate the reliability value, the previous tracking position can be compared with the current tracking position to determine whether the current tracking position is located on the same path as the previous tracking position. If the tracking position is determined by an electromagnetic tracking system, the distortion detected in the generated electromagnetic field can also be taken into account for determining the reliability value for the tracking position. Therefore, based on the provided reliability value, it can be detected whether the tracking position is reliable. If the reliability value indicates that the tracking position is relatively reliable, that is, it is above the threshold reliability value, it can be assumed that the medical instrument is on the intervention path. A sufficiently reliable tracking position also means that a planned control area can be used, for example, to avoid puncture of risk structures. Otherwise, if the comparison of the reliability value of the tracking position with the threshold reliability value results in a high uncertainty about the tracking position, an additional control image that is not triggered by a previously set control area can be provided. Based on the control image, tracking can be triggered to recalibrate the medical instrument. The threshold for triggering recalibration can depend on the distance to the risk structure such as an organ. Thus, if the reliability value is below a threshold reliability value, a control image may be captured to verify the position of the medical instrument.

[0042] In particular, the control unit may also be configured to control the image providing unit to provide a control image based on the provided warning signal. The warning signal may also trigger a visual signal or an audio signal indicating that the reliability value is below a threshold reliability value. The warning signal may be used to warn a clinician that the reliability of the tracking position is low.

[0043] Alternatively or additionally, the evaluation unit may be configured for evaluating that the spatial relationship between a) the tracking position and b) the intervention path satisfies the predefined criterion when the tracking position is closer to the target location than a threshold target distance to the target location.

[0044] The threshold target distance may also be zero, so that if the tracking position is at the target location, the spatial relationship between a) the tracking position and b) the intervention path satisfies the predefined criteria. However, it is preferred that the threshold target distance is greater than zero. If the tracking position is closer to the target location than the threshold target distance to the target location, a control image may be captured before the target location is actually reached. Preferably, the intervention path ends at the target location, i.e., the intervention path may connect the incision point with the target location. Thus, if the medical instrument reaches the end of the intervention path, it reaches the target location at the same time. The threshold target distance is preferably defined relative to the intervention path and, for example, includes the end portion of the intervention path. Thus, before reaching the end of the intervention path, the tracking position may be closer to the target location than the threshold target distance. Thus, before actually reaching the target location, the predefined criteria are met and a control image may be provided. For example, the target location may be within a risk structure (e.g., an organ), and the threshold target distance may be defined such that it represents the distance between the target location and the outer surface of the risk structure. In this case, the predefined criteria are met when the medical instrument enters the risk structure before reaching the target location inside. In one example, the target location may be a kidney stone that is removed from the kidney as a risk structure via minimally invasive surgery. The threshold target distance may be selected to represent the distance from the kidney stone to the outer surface of the kidney such that the predefined criteria are met when the medical instrument enters the kidney before reaching the kidney stone to be removed.

[0045] Preferably, the risk structure providing unit comprises a neural network, which is trained for receiving the initial image as input and for providing the localization of the risk structure in the initial image as output. The trained neural network may be a multiscale neural network or a recurrent neural network (RNN), such as but not limited to a gated recurrent unit (GRU) recurrent neural network or a long short-term memory (LSTM) recurrent neural network. Alternatively, the neural network may be a convolutional neural network (CNN). The training data used to train the neural network may represent images of risk structures that have been previously identified. Thereby, the neural network can be adapted to identify risk structures in the provided initial image.

[0046] Furthermore, according to the present invention, a method for assisting a surgical procedure is proposed, the method comprising the following steps:

[0047] Provides initial images of target location inside the patient's body,

[0048] providing in the initial image an intervention path for guiding a medical instrument to the target location,

[0049] providing a location of the risk structure in said initial image,

[0050] providing a tracked location of the medical instrument,

[0051] determining a spatial relationship between the tracking position and the planned intervention path using the determination unit,

[0052] evaluating whether a) the tracking position and b) the location of the risk structure and / or the intervention path satisfies predefined criteria, and

[0053] An output signal is provided indicating whether the spatial relationship satisfies the predefined criteria.

[0054] The order of steps:

[0055] providing an intervention path in the initial image for guiding a medical instrument to the target location, and

[0056] providing a location of the risk structure in said initial image,

[0057] This can be reversed in different variations of the method.The method according to the invention can be performed using the above-described system for assisting a surgical procedure.

[0058] The method further comprises the following steps:

[0059] A control image for positioning the object is provided when the output signal indicates that the evaluated spatial relationship satisfies the predefined criterion.

[0060] Furthermore, according to the present invention, a computer program is proposed, comprising instructions for performing the steps of the above method when run on a computer.The computer program may be stored on a non-transitory computer-readable data medium.

[0061] In summary, the invention relates inter alia to using the tracked position of a medical instrument to estimate the position of a distal end of the medical instrument relative to risk structures and - if the tracked position of the distal end is too close to the risk structure - recommending acquisition of updated control images of target localization.

[0062] In particular, the above-mentioned system and method for assisting surgical procedures can be used in combination with multiple surgical procedures, where a 3D planning imaging examination is performed before the operation, or where it may be helpful to do so, and in particular for minimally invasive surgery, where the surgical pathway, i.e., the incision point and the trajectory of the surgical instrument inside the body, is determined on the initial image acquired during the 3D planning imaging examination. For example, the above-mentioned system and method for assisting surgical procedures can be used for surgical kidney stone removal percutaneous nephrolithotomy (PCNL), where the access channel to the best calyx to reach the stone is planned on a 3D-CT scan and then expanded during the operation through the skin and renal parenchyma to allow insertion of endoscopic instruments for stone fragmentation and removal. Typically, obtaining renal access is technically challenging because the selected calyx will be entered at the correct angle to avoid bleeding and the kidney moves due to breathing. In addition, PCNL procedures are often performed with the patient in a prone position, while diagnostic CT is performed with the patient in a supine position. In such operations, the above-mentioned system and method for assisting surgical procedures can be used to improve patient safety while minimizing the required X-ray exposure.

[0063] It shall be understood that the system of claim 1, the method of claim 10 and the computer program of claim 11 have similar and / or identical preferred embodiments, in particular as defined in the dependent claims.

[0064] It shall be understood that a preferred embodiment of the invention can also be any combination of the independent claims or the above embodiments with the respective dependent claims.

[0065] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In the following figures:

[0067] Figure 1 A system for assisting a surgical procedure is shown.

[0068] Figure 2 A flow chart representing a method for assisting a surgical procedure is shown,

[0069] Figure 3 shows an initial image visualized by a visualization unit, and

[0070] Figure 4 Spatial registration of the initial image and preoperative cone beam CT with superposition of the interventional pathway is shown. DETAILED DESCRIPTION

[0071] Figure 1A system 100 for assisting a surgical procedure is shown. In particular, the system 100 can support a clinician in safely guiding a medical instrument (not part of the system 100) such as a needle towards a target location (eg, a calyx of a kidney).

[0072] The system 100 includes an image providing unit 102 for providing an initial image of a target location inside a patient's body. The image providing unit 102 can be, for example, a fixed or mobile C-arm, a CBCT device, or an ultrasound device having an X-ray source 113 and an X-ray detector 115 (as exemplarily shown in the present case) for detecting X-rays 117 that have passed through the patient's body 119. The C-arm can also be part of a CBCT device. It can also be that the image providing unit 102 is configured to receive image data representing the initial image from an external imaging device. Therefore, the imaging device can be the image providing unit 102 or an external imaging device for recording the initial image, and can be configured to record a projection X-ray image with a single exposure or in a fluoroscopic mode (e.g., in a continuous fluoroscopic mode). In particular, the initial image is a planning image showing the target location inside the patient's body, for example, a planning CT scan.

[0073] The system 100 further comprises an intervention path providing unit 104 configured to process the initial image 103 provided by the image providing unit 102. In particular, the intervention path providing unit 104 is configured to provide an intervention path 101 in the initial image 103, along which a medical instrument should be guided to a target location shown in the initial image 103. For example, the intervention path providing unit 104 may comprise software configured to define the intervention path 101 for the medical instrument. After processing the initial image 103 with the intervention path providing unit 104, the initial image 103 may be combined with a superposition of the intervention path 101 for inserting the medical instrument.

[0074] The system 100 further comprises a risk structure providing unit 106 configured to process the initial image 103 to provide a location of a risk structure 105 in the initial image 103. The risk structure 105 may be an organ or a blood vessel that should not be punctured by a medical instrument on its way to the target location. In order to provide the location of the risk structure 105 in the initial image 103, the risk structure providing unit 106 may comprise a neural network that is trained to recognize the location of the risk structure when provided with the initial image and outputs the initial image comprising a label representing the location of the risk structure in the initial image. After being processed by the intervention path providing unit 104 and the risk structure providing unit 106, an overlay showing the intervention path and the location of at least one risk structure 105 may be added to the processed initial image 103.

[0075] Furthermore, the system 100 includes a position providing unit 108 configured to provide a tracked position 109 of a medical instrument. In particular, the position providing unit 108 may receive the tracked position 109 of the medical instrument during a surgical procedure. Thus, the received tracked position 109 of the medical instrument represents the position of the medical instrument relative to the patient when the surgery is performed. A graphical representation of the tracked position 109 as provided by the position providing unit 108 may be included in the initial image 103 and updated when a new tracked position 109 is received using the position providing unit 108. In this way, the tracked position 109 of the medical instrument in the initial image 103 may be followed when the surgical procedure is performed. The position providing unit 108 may receive the tracked position 109 from a tracking system (not part of the system 100) configured to track the position of the medical instrument relative to the patient. In particular, the tracking system tracks the position of the medical instrument (e.g., a needle) relative to the imaging device (e.g., a C-arm or CBCT) in the coordinate system of the tracking system. The tracking system may be an optical tracking system or an electromagnetic tracking system. The tracking system may be configured to track the position of the medical instrument in six degrees of freedom. The tracking system may furthermore use knowledge of the X-ray geometry of the imaging system, such as the focus-detector distance or the detector active area size.

[0076] The system 100 comprises an evaluation unit 110 configured for evaluating whether a spatial relationship between a tracking position 109 as provided by a position providing unit 108 and a location of a risk structure 105 as provided by a risk structure providing unit 106 and / or an intervention path 101 as provided by an intervention path providing unit 104 satisfies a predefined criterion. The predefined criterion used by the evaluation unit 110 for evaluating the spatial relationship may be defined in various ways.

[0077] For example, the risk structure providing unit 106 may define a safety distance relative to the risk structure provided in the initial image. The safety distance may be used to generate a margin or envelope that at least partially encloses the risk structure at a distance corresponding to the safety distance. The safety distance may be used to define a buffer zone around the risk structure. One way to define the predefined criterion may be based on the safety distance, wherein the predefined criterion is met if the medical instrument is closer to the risk structure than the defined safety distance. In this case, the medical instrument will be closer to the risk structure according to the requirements of the safety distance, so that there will be an increased risk of piercing the risk structure. It is possible that the defined safety distance actually intersects the intervention path, so that when the intervention path is followed with the medical instrument, the medical instrument will be closer to the risk structure than the safety distance. In this case, a threshold intersection distance may be defined relative to the intersection point (e.g., the intersection point), and the predefined criterion may be that the medical instrument is closer to the intersection point as provided by the threshold intersection distance. In this case, the medical instrument is actually reaching the expected intersection point of the safety distance and the intervention path.

[0078] Another way to define the predefined criterion may be based on the deviation of the tracking position from the intervention path. For example, a threshold deviation value may be defined that indicates the allowed deviation of the tracking position from the intervention path. The evaluation unit 110 may now evaluate the deviation of the tracking position relative to the threshold deviation value and may determine that the spatial relationship meets the predefined criterion if the deviation of the tracking position exceeds the threshold deviation value. In this case, it is expected that the medical instrument has moved too far away from the intervention path.

[0079] In addition to the predefined criteria just described, the reliability of the tracked position may also be determined. For example, the position providing unit 104 may be configured to provide the tracked position and a reliability value associated with the tracked position. In order to assess the reliability, a threshold reliability value may be defined. If the provided reliability value deviates too much from the threshold reliability value, the evaluation unit may provide a warning signal indicating that the reliability value indicates a low reliability. This would mean that the tracking of the position of the medical instrument is performed with an increased uncertainty relative to the actual position of the medical instrument.

[0080] The system 100 further comprises an output unit 112 configured for providing an output signal indicating whether the spatial relationship satisfies a predefined criterion as evaluated by the evaluation unit 110. Optionally, the output unit 112 may also provide a warning signal if the correctness reliability of the tracked position is found to be below a threshold reliability value.

[0081] The system 100 may optionally include a control unit 114 that may be configured to receive and process output signals from the output unit 112. In particular, if the output signal indicates that the spatial relationship evaluated by the evaluation unit 110 satisfies a predefined criterion, the control unit 114 may control the image providing unit 102 to provide a control image of the target positioning. The control image may be generated by an imaging device, which may or may not be the same imaging device used to capture the initial image. In particular, the imaging device used to generate the control image may be the image providing unit 102 itself or an external imaging device, such as a C-arm or CBCT or ultrasound device.

[0082] It may be beneficial if the evaluation unit 110 continuously checks whether the spatial relationship meets at least one of the predefined criteria described previously. However, during a surgical procedure, there may be parts inside the patient's body that are associated with a lower risk of causing damage to the risk structure, and there may be other parts inside the patient's body that are associated with a higher risk of causing damage to the risk structure.

[0083] For this purpose, the control unit 114 may be configured to define one or more control regions, which may be control points with a predefined diameter in the initial image, indicating the position of the medical instrument that meets the predefined criteria. In particular, if the medical instrument reaches the control region, it may be beneficial to provide a control image of the target positioning. Thereby, the risk of puncturing risk structures may be reduced, and thus the safety of the surgical procedure may be improved.

[0084] The system 100 may further comprise an optional visualization unit 116 configured for visualizing in the initial image 103 a tracked position 109 of the medical instrument as provided by the position providing unit 106 and one or more control regions as provided by the control unit 114. Additionally, the visualization unit 116 may be configured for visualizing in the initial image 103 the positioning of the intervention path 101 as provided by the intervention path providing unit 104 and the risk structure 105 as provided by the risk structure providing unit 108. Furthermore, additionally, the visualization unit 116 may visualize in the initial image 103 a safety distance or an associated safety margin 111 or a safety envelope.

[0085] Thus, using the system 100, a clinician can be alerted if a medical instrument is dangerously close to a risk structure such as an organ during the insertion phase. This allows replacing continuous X-ray fluoroscopic imaging with a dose-saving scheme of intermittent single-view X-ray exposures (e.g., controlled exposures) at several time points selected by the system 100. Thus, the system 100 allows the medical instrument to be safely guided to the target location while reducing X-ray exposure to a minimum.

[0086] Figure 2 A flow chart showing a method for assisting a surgical procedure is shown. Figure 1 The system 100 is described to perform the methods as described below.

[0087] The method begins with a planning phase, which includes providing an initial image of the target location inside the patient's body (step S1). The initial image can be a planning CT scan and can be used to identify and depict all relevant structures. Considering the exemplary case of kidney stone removal, the initial image can show the kidneys and anatomical structures such as the urine collecting system (renal pelvis and calyces), lung pleura and ureteral anatomical structures and kidney stones. During the planning phase, one or more of such anatomical structures can be identified. In the example just mentioned, kidney stones will be the target location, and other anatomical structures can be identified as risk structures.

[0088] In particular, in the method, one or more locations of risk structures may be provided (step S2). Risk structures may be identified in the initial image using a correspondingly trained neural network. For example, using software, the identified risk structures may be marked in the initial image. Preferably, risk structures that are organs or anatomical structures pierced by a medical instrument will be harmful to the patient. Therefore, the intervention path for the medical instrument should not pass through the risk structure. In addition to marking the risk structure, it may be beneficial to calculate the safety margin around the identified risk structure. The safety margin around the risk structure may be defined based on the total spatial range of expected movement caused by patient positioning (e.g., prone patient positioning) and / or expected respiratory movement relative to the intervention path. The safety margin may be visualized as an additional overlay to the initial image together with the identified risk structure. The extent of the safety margin may also be affected by the expected accuracy of medical instrument tracking and / or registration. Since surgery is usually performed under general anesthesia, the anesthesiologist may induce apnea to control the movement of the clinician when advancing the medical instrument, which may be reflected by a reduction in the extension of the safety margin.

[0089] The method further comprises providing an intervention path (step S3), along which the medical instrument should be guided to the target location. The intervention path can be visualized together with the identified risk structure in the initial image and the safety margin around the risk structure. For example, the intervention path can be manually input, and then the intervention path can be provided, for example, by an intervention path providing unit. The intervention path can also be determined by software (for example using a trained neural network) and then provided by the intervention path providing unit. As part of the planning phase, initially, the risk structure and the safety margin around the identified risk structure can be provided in the initial image. For example, the initial image can be visualized together with an overlay of the risk structure and another overlay with a determined safety margin. The intervention path can then be selected and provided so that it has minimal contact with the safety margin and the identified risk structure. However, there may be situations where it may not be possible to avoid crossing at least one of the safety margin and the risk structure to reach the target location.

[0090] The intersection of the intervention path and the safety margin and / or risk structures may be calculated, for example, using software, and defined as a control region, which may be a control point with a predefined diameter. The control region defines the position of the medical instrument at which it is recommended to provide a control image to verify the actual position of the medical instrument relative to, for example, the risk structure.

[0091] If CBCT is used during surgery, at the beginning of the intervention, it may be necessary to calibrate the CBCT to determine the transformation T between the planning CT and the CBCT. CT→CBCT. The accuracy of the registration may depend on various factors, such as the image quality and field of view of the CBCT, the contrast level of the target positioning, and the amplitude of the deformation (e.g., non-linear deformation) between the CT and the CBCT. Considering the example of a renal intervention, the registration can be performed by segmenting the kidney and internal structures (such as the calyces and renal pelvis) and using this information for a rough rigid alignment of the images. A more precise and, for example, non-linear alignment can be obtained by subsequent intensity-based transformation refinement. Instead of using CT to perform planning, CBCT can be used at the beginning of the intervention. In this case, there is no planning CT as required, and therefore no registration component. Therefore, providing the risk structures and intervention path can be done directly on the CBCT.

[0092] During the operation, the position of the medical instrument is tracked, for example using an optical tracking system or an electromagnetic tracking system. The tracked position is then provided, for example by a position providing unit (step S4). The provided tracked position of the medical instrument may be received from the tracking system.

[0093] In the method, it is also evaluated whether the determined spatial relationship between the tracking position and the location of the risk structure and / or the intervention path (step S5) meets the predefined criteria (step S6). In addition, an output signal indicating whether the spatial relationship meets the predefined criteria is provided (step S7). As previously mentioned, the predefined criteria may be met if the tracking position of the medical instrument is closer to the risk structure than a safety distance, which is defined relative to the risk structure and can be converted into a safety margin around the risk structure. Alternatively, the predefined criteria may be whether the tracking position is closer to the intersection between the provided intervention path and the defined safety distance than a predefined threshold in the segment distance. Again alternatively, the predefined criteria may be met if the deviation of the tracking position from the intervention path exceeds a threshold deviation value indicating the allowable deviation of the tracking position from the intervention path. However, again alternatively, the predefined criteria may be met if the tracking position is closer to the target location than a threshold target distance defined relative to the target location.

[0094] In the method, if the output signal indicates that the evaluated spatial relationship meets a predefined criterion, a control image of the target positioning may also be provided (step S8). Therefore, during the intervention, in the method, a recommendation may indicate the provision of a control image, wherein the recommendation is based on the evaluation of the spatial relationship between the tracking position and the risk structure and / or the intervention path. With the help of the control area defined in the initial image, there may be multiple planned control exposures for providing control images, for example when the medical instrument is close to the risk structure. This may be the case when the distance from the tip of the medical instrument to the risk structure is less than, for example, a predefined safety margin. Preferably, another planned control exposure is performed when the medical instrument is close to the target positioning, for example when reaching the kidney before removing a kidney stone.

[0095] In addition, in the method, it may also be included that, for example, if the tracking system or the position providing unit indicates a relatively high uncertainty about the tracked position, a specific control exposure is performed to provide a control image. Another reason for performing a specific control exposure may be if the planned control exposure shows a large deviation from the original surgical plan or an inaccuracy in the alignment. For the control exposure, it may be beneficial if the C-arm is positioned to allow an optimal assessment of the position of the medical instrument relative to the risk structure. For example, the C-arm may be positioned so that the control image is captured in a direction orthogonal to the shortest path between the tip of the medical instrument and the surface of the risk structure. In the method, it may also be preferred if the intervention path is provided in the initial image in such a way that a minimum number of planned control exposures is required.

[0096] Figure 3 As shown by the visualization unit (e.g., reference Figure 1 1 . The initial image 300 is visualized by a visualization unit 116 of the system 100. The visualization unit may be or may include a graphical user interface (GUI). The initial image 300 is a planning CT scan. As an overlay of the initial image, anatomical structures are identified and marked. One of these structures is a target location 302 for a kidney stone of a kidney 304. Another identified anatomical structure is a risk structure 306 partially surrounded by a safety margin 308, which is arranged at a safe distance 310 from the risk structure 306. In addition, an intervention path 312 is provided in the initial image 300. The intervention path 312 represents the route along which the medical instrument should be guided to reach the target location 302. Thus, the intervention path 312 connects the incision point with the target location 302. The actual path of the medical instrument is determined using a tracking system, and the tracked position 314 of the medical instrument can be visualized in the initial image 300 during the surgical procedure. This requires transforming the coordinates of the position of the medical instrument in the coordinate system of the tracking system into the coordinate system of the C-arm used to record the initial image 300. If CBCT is used to record control images or new initial images during surgery, it may be necessary to determine the transformation T CT→CBCT The inverse of is used to visualize the tracked position of the medical instrument. As can be seen in the initial image 300, the tracked position 314 of the medical instrument does not exactly follow the planned intervention path 312, but slightly deviates from it. If the deviation between the tracked position 314 and the intervention path 312 exceeds a predefined threshold deviation value, a specific control exposure can be triggered for providing a control image.

[0097] Since the intervention path 312 intersects the safety margin 308, a first control region 316 is provided in the initial image 300. The first control region 316 defines the position of the medical instrument providing the planning control image. Therefore, if the medical instrument is expected to reach the first control region 316, the control unit can control the image providing unit to provide a control image of the target location 302. Before reaching the target location 302, a second control region 318 is provided at the position where the medical instrument is expected to pierce the kidney 304. Therefore, the planning control image is provided just before reaching the target location 302 to verify the actual position of the medical instrument relative to the target location 302.

[0098] exist Figure 4 In FIG. 4 , the spatial registration of the initial image 400 and the preoperative cone beam CT 402 is shown. In addition, the superposition of the intervention path 404 is visualized. For this purpose, in case of CBCT imaging, a registration algorithm can be employed to calculate the transformation T between the CT and C-arm coordinate systems CT→CARM or T CT→CBCT Alternatively, if non-CBCT imaging is used, the application of a tracking system may be employed to establish a transformation T between the C-arm and tracking system coordinate systems. C型臂→医学仪器 .

[0099] Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.

[0100] In the claims, the word "comprising" does not exclude other elements or steps, and the word "a" or "an" does not exclude a plurality.

[0101] A single unit or device may fulfill the functions of several items recited in the claims.The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0102] The processes performed by one or several units or devices, such as providing an initial image, providing an intervention path in the initial image, providing a location of a risk structure in the initial image, providing a tracking position of a medical instrument, etc., can be performed by any other number of units or devices. These processes can be implemented as program code units of a computer program and / or as dedicated hardware.

[0103] The computer program may be stored / distributed on suitable media, such as optical storage media or solid-state media provided together with or as part of other hardware, but the computer program may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

[0104] Any reference signs in the claims should not be construed as limiting the scope.

[0105] The present invention relates to a system and method for assisting image-guided surgical procedures, in particular assisting image-guided percutaneous needle interventions such as for kidney stone removal. With the system and method, automatic recommendations for control scans can be provided at key stages during an intervention. For this purpose, in an initial image, risk structures are identified and an intervention path to target positioning is provided. With an evaluation unit, it can be evaluated whether the spatial relationship between a) the tracking position of a medical instrument and b) the positioning of the risk structure and / or the intervention path meets predefined criteria. If the predefined criteria are found to be met, the acquisition of a control scan can be automatically recommended.

Claims

1. A system (100) for assisting a surgical procedure, the system (100) comprising: an image providing unit (102) configured to provide an initial image (300) of a target location (302) inside a patient's body, an intervention path providing unit (104) configured to provide an intervention path (312) in the initial image (300), along which a medical instrument should be guided to the target location (302), a risk structure providing unit (106) configured to provide a location of a risk structure (306) in the initial image (300), a position providing unit (108) configured to provide a tracked position (314) of the medical instrument, an evaluation unit (110) configured to evaluate whether a) a spatial relationship between the tracking position (314) and b) the location of the risk structure (306) and / or the intervention path (312) satisfies a predefined criterion, an output unit (112) configured to provide an output signal indicating whether the spatial relationship satisfies the predefined criterion, and A control unit (114) is configured to control the image providing unit (102) to provide a control image of the target location (302) when the output signal indicates that the evaluated spatial relationship meets the predefined criterion.

2. The system according to claim 1, wherein: The control unit (114) is configured to: define a control area (316, 318) that meets the predefined criteria; and control the image providing unit (102) to provide a control image when the medical instrument reaches the control area (316, 318).

3. The system according to claim 2 further comprises a visualization unit (116) configured to visualize the tracking position (314) and the control area (316, 318) in the initial image (300), wherein the control area (316, 318) indicates the position of the medical instrument that has or will provide the control image.

4. The system according to claim 3, wherein: The visualization unit (116) is further configured to visualize the intervention path (312), and wherein the control area (316, 318) is located on the intervention path (312).

5. System according to at least one of the preceding claims, wherein: The risk structure providing unit (106) is configured to define a safety distance (310) relative to the risk structure (306), and wherein: i) the evaluation unit (110) is configured to evaluate whether a) the spatial relationship between the tracking position (314) and b) the location of the risk structure (306) satisfies the predefined criterion when the tracking position (314) of the medical instrument is closer to the risk structure (306) than a defined safety distance (310), and / or ii) the risk structure providing unit (106) is configured to determine an intersection (316) between the provided intervention path (312) and the defined safety distance (310), and the evaluation unit (110) is configured to evaluate whether a) the spatial relationship between the tracking position (314) and b) the positioning of the risk structure (306) and the intervention path (312) meets the predefined criteria when the tracking position (314) is closer to the intersection (316) than a predefined threshold intersection distance.

6. System according to at least one of the preceding claims, wherein: The evaluation unit (110) is configured to evaluate whether a) the spatial relationship between the tracking position (314) and b) the intervention path (312) satisfies the predefined criteria when the deviation between the tracking position (314) and the intervention path (312) exceeds a threshold deviation value, wherein the threshold deviation value indicates an allowable deviation between the tracking position (314) and the intervention path (312).

7. System according to at least one of the preceding claims, wherein: The evaluation unit (110) is configured to evaluate whether a reliability value associated with the tracking position (314) is lower than a threshold reliability value for the tracking position (314), and the output unit (112) is further configured to provide a warning signal when the reliability value is lower than the threshold reliability value.

8. System according to at least one of the preceding claims, wherein: The evaluation unit (110) is configured to evaluate that a) the spatial relationship between the tracking position (314) and the intervention path (312) satisfies the predefined criterion when the tracking position (314) is closer to the target location (302) than a threshold target distance to the target location (302).

9. System according to at least one of the preceding claims, wherein: The risk structure providing unit (106) comprises a neural network, which is trained to receive the initial image (300) as input and to provide the location of the risk structure (306) in the initial image (300) as output.

10. A method for assisting a surgical procedure, the method comprising the following steps: Providing an initial image (S1) of the target location inside the patient's body, providing an intervention path in the initial image for guiding a medical instrument to the target location (S3), providing a location of the risk structure in said initial image (S2), providing a tracked position of the medical instrument (S4), determining a spatial relationship between the tracking position and the planned intervention path (S5), evaluating whether a) the tracking position and b) the location of the risk structure and / or the intervention path has a spatial relationship that satisfies a predefined criterion (S6), providing an output signal (S7) indicating whether the spatial relationship satisfies the predefined criterion, and A control image for positioning the object is provided when the output signal indicates that the evaluated spatial relationship satisfies the predefined criterion (S8).

11. A computer program comprising instructions for performing the steps of the method according to claim 10 when the instructions are run on a computer.

12. A non-transitory computer-readable data medium storing a computer program according to claim 11.

Images