Method, computing device, system and computer program product for assisting in positioning tool relative to specific body part of patient

Through a computer method based on intraoperative imaging data, using artificial intelligence algorithms to reconstruct anatomical 3D shape and tool positioning, the problem of line of sight and registration assisted by surgical positioning in the prior art is solved, and efficient and precise positioning is achieved without the need for external navigation hardware. Assisted, simplifies the surgical procedure and reduces radiation exposure.

CN119053992APending Publication Date: 2024-11-29UNIVERSITY OF ZURICH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380035125.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-02-21
Filing Date
2023-02-17
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

Existing computer-assisted surgical guidance systems have line of sight problems when positioning tools relative to specific body parts of the patient, require extensive registration processes and external navigation hardware installation, interfere with surgical workflow and may lead to radiation exposure and increased surgical time.

Method used

Using computer implementation methods based on intraoperative imaging data, the current positioning of anatomical 3D shapes and tools are reconstructed through artificial intelligence algorithms to generate visual representations of tools relative to specific body parts, realizing positioning assistance without external navigation hardware and preoperative planning registration .

Benefits of technology

Improves the accuracy and efficiency of surgical procedures, reduces surgical time and radiation exposure, simplifies the surgical process, and avoids the complexity of navigation hardware installation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119053992A_ABST
    Figure CN119053992A_ABST
Patent Text Reader

Abstract

A computer-implemented method, computing device, system and computer program product for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200), comprising: receiving intraoperative imaging data (ID), it comprises capturing 2D images of the particular body part (202) from a plurality of viewing angles and capturing 2D images of at least a portion of the tool (5) from at least one viewing angle; reconstructing an anatomical 3D shape (AS) of the particular body part (202) using an artificial intelligence-based algorithm corresponding to the particular body part (202) based on the intraoperative imaging data (ID) and the data indicative of the perspective of the 2D image; estimating a current position (5c) of the tool (5) relative to the anatomical 3D shape (AS) based on the intraoperative imaging data (ID); and generating positioning guidance data (GD) comprising an estimated current positioning (5c) of the tool (5) relative to the anatomical 3D shape (AS) of the particular body part (202).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a computer-implemented method of assisting in positioning a tool, such as a surgical tool, relative to a specific body part of a patient. The present invention also relates to a computing device configured to assist in positioning a tool, such as a surgical tool, relative to a specific body part of a patient. The present invention even relates to a system for assisting in positioning a tool relative to a specific body part of a patient. The present invention even relates to a computer program product comprising instructions that, when executed by a processing unit of a computing device, cause the computing device to assist in positioning a tool, such as a surgical tool, relative to a specific body part of a patient. Background Art

[0002] Conventional, state-of-the-art computer-assisted surgery (surgery) and surgical navigation techniques have enabled surgeons to work with "glass" patients, where important information from preoperative imaging and preoperative planning is directly available to the surgeon's perception. Conventional computer-assisted surgical guidance systems rely on three main aspects: 1) preoperative planning based on anatomical 3D models derived from preoperative images (such as CT or MRI scans); 2) registration of preoperative data with intraoperative anatomy; and 3) real-time tracking of surgical instruments.

[0003] Preoperative planning used in state-of-the-art surgical guidance systems depicts the surgical procedure in a step-by-step manner on a generated 3D anatomical model. However, these preoperative plans are usually idealized descriptions (sketches) of intraoperative reality, which may be affected by preoperative events and / or intraoperative conditions (such as bleeding, complications, and surgical inaccuracies) that affect the corresponding body parts of the patient. Therefore, surgeons are often forced to fall back on conventional (non-navigation) techniques.

[0004] Another shortcoming of conventional computer-assisted surgical guidance systems is often regarded as the registration of preoperative planning with intraoperative anatomical structures. In the context of the present application, the term registration refers to the process of calibrating / aligning the preoperative planning with the actual, true anatomical structure and body position and orientation of the patient during surgery. Conventional computer-assisted surgical guidance systems solve this problem by using techniques such as the following: using an optical tracking pointer or by using an image-based registration method to match common landmarks and / or features in the preoperative planning with a 3D anatomical model, the method automatically matches preoperative images (such as CT or MRI scans) with images obtained during surgery (i.e., 2D-3D registration). However, such common techniques in computer-assisted surgery and surgical guidance systems are susceptible to different sources of error or technical complexity, which include from marker movement and a small capture range to slow calculations.

[0005] Documents US2022 / 044440A1 and WO2020 / 108806A1 describe methods for performing artificial intelligence-assisted surgery using statistical shape modeling, in which objects identified from intraoperative imaging data (using an artificial intelligence algorithm) are classified in an X-ray projection image, and a 3D representation and positioning of the classified objects are determined by deforming the statistical shape model of the classified objects to fit the imaging of the classified objects in the X-ray image. However, statistical shape modeling has the disadvantage that in the process of deforming the statistical shape model to fit the intraoperative image, individual features captured in the intraoperative image that were not captured by the statistical shape model are lost. Therefore, statistical shape modeling can at best provide a general statistical approximation of the 3D dimensional shape based on intraoperative imaging, but is not suitable for 3D reconstruction of anatomical shapes.

[0006] In the third aspect of conventional computer-assisted surgical navigation, the three-dimensional pose of the tracking tool (especially the desired surgical hardware) should be estimated within the same reference frame of the anatomy of the patient's specific body part. To this end, known computer-assisted surgery and surgical guidance systems rely on reference markers attached to the surgical tools. However, line of sight issues are considered to be a significant burden for the clinical use of such systems. There are methods for 2D X-ray-based surgical tool pose estimation, which use traditional (usually intensity-based) registration techniques. However, such techniques are susceptible to the same limitations as image-based registration methods.

[0007] Recently, with the introduction of fluoroscopy machines capable of intraoperative cone-beam computed tomography (CBCT), 3D volumetric images of patients can be acquired during surgery and this data can be used in conjunction with an optical tracking system to provide registration-free surgical navigation. In such an approach, the patient is registered to the preoperative plan by monitoring patient-mounted optical tracking reference markers that are visible both in intraoperative cone-beam computed tomography and through the optical tracking system. Since full-range cone-beam computed tomography images are acquired, such an approach results in increased ionizing radiation and may also suffer from metal artifact problems. In addition, the main technical limitation of cone-beam computed tomography-based intraoperative navigation is the assumption of fixed reference markers attached to the patient's anatomy, which has proven to be insufficient in operating room conditions.

[0008] Although these methods have been shown to result in higher implant accuracy compared to standard freehand surgical methods, they have not yet been widely adopted in the most advanced operating rooms around the world. According to reports in a worldwide survey, only 11% of spinal surgeries are performed using a computer-assisted navigation system, and most operations are performed using conventional freehand open techniques, in which surgeons rely on their visual and tactile feedback to place spinal implants in the pedicle region. This is due to the fact that the above-mentioned computer-assisted surgical navigation methods require a large amount of registration processes to transfer preoperative planning to anatomical structures and / or generally require the installation of external navigation hardware in the operating room. This may interfere with existing surgical workflows and may result in increased operating time, radiation exposure and cost. For example, a navigation system that requires cone beam computed tomography to be obtained during surgery may result in an increase in surgical time of up to 8.2 minutes and result in 2.09 to 4.81 mSV of ionizing radiation.

[0009] In summary, known computer-assisted surgery and surgical guidance systems are prone to line-of-sight and / or marker movement issues; require extensive registration for preoperative planning; require installation of external navigation hardware in the operating room and / or significantly interfere with the surgical workflow. Summary of the invention

[0010] It is an object of the present invention to provide a method, computing device, system and computer program product for assisting in positioning a tool relative to a specific body part of a patient that overcomes one or more of the disadvantages of the prior art.

[0011] Specifically, an object of the present invention is to provide a registration-free method for assisting in positioning a tool relative to a specific body part of a patient, which method can reconstruct the anatomical 3D shape and generate a visual representation of the positioning of the tool relative to the specific body part of the patient using only intraoperative imaging data (i.e., without the need to install navigation hardware in the operating room).

[0012] According to the present disclosure, this object is solved by the features of independent claim 1. Further advantageous embodiments emerge, among others, from the dependent claims and the description.

[0013] In particular, this object is achieved by a computer-implemented method for assisting in positioning a tool, such as a surgical tool (e.g., a surgical drill, a knife, or a surgical laser device) or a medical diagnostic tool relative to a specific body part of a patient, the method comprising:

[0014] - Receive intraoperative imaging data;

[0015] -Use intraoperative imaging data and reconstruct anatomical 3D shapes using artificial intelligence-based algorithms corresponding to specific body parts;

[0016] - estimating the current positioning of the tool based on intraoperative imaging data; and

[0017] - Generating positioning guidance data comprising a visual representation of the estimated current positioning of the tool relative to the anatomical 3D shape of a specific body part.

[0018] In a specific embodiment, in preparation for / before surgical treatment of the patient, the following steps are repeatedly or continuously performed over a period of time: receiving intraoperative imaging data; estimating the current positioning of the tool; and generating guidance data.

[0019] Receiving intraoperative imaging data

[0020] A computing device receives intraoperative imaging data from an imaging device arranged in the vicinity of a patient. Here, the imaging device being arranged in the vicinity of the patient refers to a positioning that allows the imaging device to capture intraoperative imaging data of the patient. The imaging data includes multiple 2D images. Two or more of the 2D images capture a specific body part of the patient from two or more different perspectives relative to the specific body part of the patient. One or more of the same multiple 2D images that capture a specific body part also capture at least a portion of a tool from at least one perspective. In the context of the present invention, the term perspective (perspective relative to the imaging data) refers to the position and / or orientation (e.g., roll, pitch, yaw) of the imaging device relative to the specific body part (or relative to the tool) (e.g., in an x, y, and z Cartesian coordinate system).

[0021] According to an embodiment of the present invention, the intraoperative imaging data includes one or more of the following: a) radiation-based images, in particular X-ray images; b) ultrasound images; c) arthroscopic images; d) optical images; and / or e) any other cross-sectional images.

[0022] An imaging device communicatively connected to a computing device is used to capture an image of a specific body part of a patient or an image of a part of a tool. In the case of radiation-based images, the imaging device may include a C-arm imaging device based on X-ray technology. The C-arm imaging device includes a generator (X-ray source) and an image intensifier or a flat panel detector. The C-shaped connecting element allows movement horizontally, vertically and / or around a rotation axis so that a 2D X-ray image of the patient can be generated from various viewing angles around the patient. The generator emits X-rays that penetrate the patient's body. The image intensifier or detector converts the X-rays into visible images that are transmitted to the computing device.

[0023] The intraoperative imaging data includes data indicating the viewing angles corresponding to the plurality of 2D images, data identifying the position and / or orientation of an imaging device that captured the plurality of 2D images (such as the position of the imaging device in an x, y, and z Cartesian coordinate system relative to a particular body part and / or orientations such as roll, pitch, and yaw). According to embodiments disclosed herein, the data indicating the viewing angles corresponding to the plurality of 2D images is stored in a data storage device included by a computing device or communicatively connected to the computing device. Alternatively or additionally, the viewing angle corresponding to the intraoperative imaging data is estimated by the computing device based on the intraoperative imaging data.

[0024] In a specific embodiment, a tool geometry model indicating the geometry of the tool is used to estimate the viewing angle corresponding to the intraoperative imaging data. First, multiple projections of the tool geometry model are calculated based on multiple candidate viewing angles. The candidate viewing angles are selected as discrete viewing angles within a predefined space of possible viewing angles of the imaging device. In other words, the unrealistic position and orientation of the imaging device relative to the patient are not taken into account to save computing power. Thereafter, by comparing at least a portion of the tool as captured by the corresponding 2D image of the intraoperative imaging data with the multiple projections calculated based on the multiple candidate viewing angles, the viewing angle corresponding to the 2D image of the intraoperative imaging data is identified. Specifically, the comparison includes applying a matching function to identify the best match between one of the projections of a "virtual" tool (based on the tool geometry model) calculated based on multiple candidate viewing angles and a portion of the "physical" tool as captured by the imaging device. The candidate viewing angle that produces the best match is selected as the estimated viewing angle.

[0025] According to the embodiments disclosed herein, an artificial intelligence-based algorithm is used to estimate the viewing angle corresponding to intraoperative imaging data, and the algorithm is trained using a large number of imaging data sets with known viewing angles. In order to overcome the limitations of the availability and / or accuracy of imaging data sets with known viewing angles, a large number of imaging data sets including 2D images from known viewing angles are generated based on 3D imaging data (especially computed tomography CT scans). Using this artificial intelligence-based algorithm trained before surgery, the intraoperative positioning of the imaging device can be estimated based solely on the intraoperative images without the need for external tracking devices or calibration phantoms.

[0026] Generate anatomical 3D shapes

[0027] The anatomical 3D shape of the specific body part is reconstructed by a computing device based on the intraoperative imaging data and data indicating the viewing angles corresponding to the multiple 2D images using an artificial intelligence-based algorithm corresponding to the specific body part.

[0028] According to the embodiments disclosed herein, the anatomical 3D shape is reconstructed as a voxelized volume and / or mesh. It is important to emphasize that the artificial intelligence-based algorithm must be a model corresponding to a specific body part, so that the 3D anatomical shape can be reconstructed from multiple 2D images of the specific body part.

[0029] According to specific embodiments disclosed herein, an artificial intelligence-based algorithm is trained using a large number of annotated imaging data sets that capture body parts (of a person other than the patient) corresponding to specific body parts of the patient. The annotations of the imaging data sets include data that identify and / or describe attributes of the body parts, such as data that identifies pixels, vectors, contours, surfaces within a 2D image, and / or voxels within a 3D image that captures the specific body parts.

[0030] In order to overcome the limitations of the availability and / or accuracy of annotated imaging data sets that capture body parts corresponding to specific body parts of a patient, according to the embodiments disclosed herein, a large number of annotated imaging data sets are generated based on annotated 3D imaging data (particularly computed tomography CT scans), which capture body parts corresponding to specific body parts of a patient. Specifically, given an input preoperative CT scan, a synthetic 2D image, such as a fluoroscopic shot (i.e., DRR), is generated based on different viewpoints around the patient. For example, using this method, up to hundreds of annotated "synthetic" 2D images (capturing specific body parts) can be generated based on a single annotated CT scan, and the annotated "synthetic" 2D images can be used by artificial intelligence-based algorithms to improve their ability to reconstruct accurate anatomical 3D shapes based on as few 2D intraoperative images as possible.

[0031] According to an embodiment, reconstructing the anatomical 3D shape is performed in two stages: segmenting the intraoperative imaging data to identify specific body parts of the patient; and further using the segmented intraoperative imaging data to reconstruct the anatomical 3D shape.

[0032] In order to segment the intraoperative imaging data, an artificial intelligence-based detection and segmentation model (such as a convolutional neural network-based detection and segmentation model) is used to first identify a region of interest within the intraoperative imaging data, where the region of interest contains a specific body part of the patient. The region of interest is then semantically segmented using the artificial intelligence-based detection and segmentation model to generate segmented intraoperative imaging data.

[0033] Estimation tool current positioning

[0034] The anatomical 3D shape has been reconstructed, and the current positioning of the tool relative to the anatomical 3D shape of a specific body part is estimated based on intraoperative imaging data (especially a 2D image of the imaging data of the tool). According to the embodiments disclosed herein, the current positioning of the tool is estimated using a tool geometric model indicating the geometric shape of the tool. First, the projection of the tool geometric model is compared with at least a portion of the tool captured by the corresponding 2D image of the intraoperative imaging data. The tool geometric model is projected onto the plane of one or more 2D images in the 2D image of the intraoperative imaging data that captures at least a portion of the tool. The plane of the 2D image of the intraoperative imaging data is determined based on the perspective of each 2D image. Thereafter, the positioning of the tool geometric model that generates the projection on the plane of the 2D image of the intraoperative imaging data is determined, and the positioning is (best) matched with at least a portion of the tool captured by the corresponding 2D image of the intraoperative imaging data. In other words, an inverse process is applied compared to the (initial) determination of the perspective of the intraoperative imaging data. However, this inverse process is not necessarily applicable to the 2D image (of the intraoperative imaging data) that is the same as the following 2D image: the 2D image is used to determine the perspective of the image used to reconstruct the anatomical 3D shape.

[0035] According to the embodiments disclosed herein, in the initial stage of the method of assisting in positioning a tool, although the anatomical 3D shape is reconstructed once, the estimation of the current positioning of the tool is repeatedly performed at set intervals and / or triggered by certain events and / or manually triggered.

[0036] In order to improve the accuracy of estimating the positioning of the tool and / or improve the accuracy of estimating the viewing angle corresponding to the intraoperative imaging data, according to other embodiments, the method of the present invention also includes providing the tool according to a tool geometric model. The tool geometric model is specially designed to optimize the estimation of its positioning based on as few intraoperative images as possible. Specifically, the tool is designed so that at least a portion of it is not completely rotationally symmetric around any axis of the Cartesian coordinate system, so as to allow the orientation of the tool to be estimated based on a 2D image. Alternatively or additionally, the tool is designed to include special markings to facilitate its identification based on a 2D intraoperative image.

[0037] Generate positioning guidance data

[0038] Having reconstructed the anatomical 3D shape of a particular body part and having estimated the current position of a tool, positioning guidance data is reconstructed by a computing device, including a visual representation of the estimated current position of the tool relative to the anatomical 3D shape of the particular body part. According to the embodiments disclosed herein, the guidance data is reconstructed as a 2D image to be displayed on a computer display. Alternatively or additionally, the guidance data is reconstructed as an augmented reality overlay, including overlay metadata that allows an augmented reality device, such as a headset, to project the overlay onto the user's field of view so that the overlay is aligned with the user's view of the patient's particular body part and / or aligned with the user's view of the tool. According to the embodiments disclosed herein, a visual representation of the estimated current position of the tool is superimposed on the visual representation of the reconstructed anatomical 3D shape.

[0039] According to the embodiments disclosed herein, the computing device controls a display device to display at least a portion of the guidance data, the display device being a computer screen, an augmented reality headset, or any device configured to display the guidance data.

[0040] In order to guide the surgeon to correctly position the tool, according to other embodiments disclosed herein, a prescribed position of the tool relative to the anatomical 3D shape of a particular body part is identified by a computing device, and a visual representation of the prescribed position of the tool is superimposed on a visual representation of the estimated current position of the tool. The prescribed position of the tool is retrieved or received by the computing device from a data storage device, which is included by the computing device or is communicatively connected to the computing device. Alternatively or additionally, the prescribed position of the tool is calculated by the computing device, and the prescribed position of the tool is determined by an optimization function based on the anatomical 3D shape of the body part and data indicating the surgical procedure.

[0041] The embodiments disclosed herein are advantageous because they can automatically perform pre-surgical planning and guide the surgeon in the placement of surgical tools based on the reconstructed anatomical 3D shape. Considering that intraoperative imaging data is used to reconstruct the anatomical 3D shape of a body part (e.g., the spine), there is no need for a pre-operative planning stage to define a safe implantation trajectory, nor is there a need to align the pre-operative data to the intra-operative patient's positioning, and the prescribed positioning / trajectory of the tool can be identified based on the anatomical 3D shape.

[0042] Another object of the present invention is to provide a computing device for positioning a tool relative to a specific body part of a patient, which can reconstruct the anatomical 3D shape and generate a visual representation of the positioning of the tool relative to the specific body part of the patient using only intraoperative imaging data (i.e., without the need to install navigation hardware in the operating room and without the need for a preoperatively planned alignment process).

[0043] According to the present disclosure, this object is solved by the features of independent claim 15. Further advantageous embodiments emerge, among others, from the dependent claims and the description.

[0044] Specifically, the above purpose is also achieved by a computing device, which includes: a data input interface; a data output interface; a processing unit; and a storage unit. The data input interface, such as a wired (e.g., Ethernet, DVI, HDMI, VGA) and / or wireless data communication interface (e.g., 4G, 5G, Wifi, Bluetooth, UWB) can be connected to the imaging device for communication and is configured to receive intraoperative imaging data therefrom. The data output interface, such as a wired (e.g., Ethernet, DVI, HDMI, VGA) and / or wireless data communication interface (e.g., 4G, 5G, Wifi, Bluetooth, UWB) is configured to transmit at least a portion of the guidance data to a display device that is communicatively connected to the data output interface. The storage unit includes instructions that, when executed by the processing unit, cause the computing device to perform a method for assisting in positioning a tool according to any one of the embodiments disclosed herein.

[0045] According to an embodiment, the computing device is a stand-alone computer communicatively connected to the imaging device. Alternatively or additionally, the computing device is a remote computer (e.g., a cloud-based computer) communicatively connected to the imaging device using a communication network (especially at least partially using a mobile communication network). Alternatively or additionally, the computing device is integrated into the imaging device or the display device.

[0046] Another object of the present invention is to provide a system for positioning a tool relative to a specific body part of a patient, which system can reconstruct the anatomical 3D shape and generate a visual representation of the positioning of the tool relative to the specific body part of the patient using only intraoperative imaging data (i.e., without the need to install navigation hardware in the operating room and without the need for a preoperatively planned alignment process).

[0047] According to the present disclosure, this object is solved by the features of independent claim 16. Further advantageous embodiments ensue, among other things, from the dependent claims and the description.

[0048] Specifically, the above purpose is also achieved by a system, which includes: a computing device according to any one of the embodiments disclosed herein; an imaging device; and a display device, which is configured to perform the method according to any one of the embodiments disclosed herein. The imaging device is communicatively connected to the computing device and arranged in the vicinity of the patient, so that the imaging device is allowed to capture the patient's intraoperative imaging data, so that two or more of the 2D images capture the patient's specific body part from two or more different perspectives relative to the patient's specific body part. One or more of the same multiple 2D images capturing a specific body part also captures at least a portion of the tool from at least one perspective. In the case of radiation-based images as intraoperative images, the imaging device includes a C-arm imaging device based on X-ray technology. The C-arm imaging device includes a generator (X-ray source) and an image intensifier or a flat panel detector. The C-shaped connecting element allows horizontal, vertical and / or rotational movement around the axis of rotation, so that a 2D X-ray image of the patient can be generated from various perspectives around the patient. The generator emits X-rays that penetrate the patient's body. The image intensifier or detector converts the X-rays into visible images that are transmitted to the computing device. The display device is a computer screen, an augmented reality headset, or any device configured to display guidance data.

[0049] Another object of the present invention is to provide a computer program product for positioning a tool relative to a specific body part of a patient, which computer program product can reconstruct the anatomical 3D shape and generate a visual representation of the positioning of the tool relative to a specific body part of the patient using only intraoperative imaging data (i.e., without the need to install navigation hardware in the operating room and without the need for a preoperatively planned alignment process).

[0050] According to the present disclosure, this object is solved by the features of independent claim 18. Further advantageous embodiments emerge, among others, from the dependent claims and the description.

[0051] Specifically, the above-mentioned object is solved by a computer program product, which includes instructions, which, when executed by a processing unit of a computing device, cause the computing device to perform the method according to any one of the embodiments disclosed herein.

[0052] According to an embodiment, the instructions (included by the computer program product) include an artificial intelligence-based algorithm corresponding to a specific body part of the patient, which artificial intelligence-based algorithm has been trained using a large amount of annotated imaging data sets that capture body parts corresponding to the specific body parts of the patient, wherein the annotations include data identifying and / or describing attributes of the body parts.

[0053] According to an embodiment, the instructions (included by the computer program product) include instructions to control an imaging device to capture intraoperative imaging data including 2D images, wherein the plurality of 2D images capture a specific body part of the patient from a plurality of different perspectives relative to the specific body part of the patient, and one or more of the plurality of 2D images capture at least a portion of a tool from at least one perspective.

[0054] According to an embodiment, the instructions (included by the computer program product) include instructions for controlling a display device such as displaying at least a portion of the guidance data, including a visual representation of an estimated current position of the tool, a visual representation of a reconstructed anatomical 3D shape, and / or a visual representation of a prescribed position of the tool.

[0055] It should be understood that both the foregoing general description and the following detailed description present embodiments and are intended to provide an overview or framework for understanding the nature and features of the present disclosure. The accompanying drawings are included to provide further understanding and are incorporated into and constitute a part of this specification. The drawings illustrate various embodiments and, together with the description, are used to explain the principles and operation of the disclosed concepts.

[0056] As used in this specification, the term "particular" refers to embodiments of the invention and does not indicate any preference or that a feature introduced as a particular will be essential to all embodiments of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The invention described herein will be more fully understood from the detailed description given below and the accompanying drawings, which should not be considered as limiting the invention described in the appended claims. The drawings show:

[0058] Figure 1 is a highly schematic perspective view of a system for assisting in positioning a tool as installed in an operating room, according to an embodiment of the present invention;

[0059] Figure 2 is a flow chart illustrating the steps of a method of assisting in positioning a tool according to an embodiment of the present invention;

[0060] Figure 3 is a flow chart illustrating steps for reconstructing an anatomical 3D shape based on intraoperative imaging data and data indicating viewing angles corresponding to a plurality of 2D images according to an embodiment of the present invention;

[0061] Figure 4 is a schematic illustration of segmenting intraoperative imaging data to identify specific body parts of a patient;

[0062] Figure 5 is a schematic illustration of another embodiment of segmenting intraoperative imaging data, including identifying a region of interest and then performing semantic segmentation on the region of interest to identify a specific body part of a patient within the region of interest;

[0063] Figure 6 is a schematic illustration of reconstructing an anatomical 3D shape based on segmented intraoperative imaging data using an artificial intelligence-based algorithm corresponding to a specific body part;

[0064] Figure 7 is a flow chart illustrating the steps of another embodiment of reconstructing an anatomical 3D shape in multiple stages;

[0065] Figure 8 It is a schematic illustration of the prescribed positioning of the determination tool;

[0066] Fig. 9A is an illustrative embodiment of a visual representation of an estimated current position of a tool and a visual representation of a prescribed position of the tool superimposed onto a visual representation of a reconstructed anatomical 3D shape;

[0067] Fig. 9B is an illustrative embodiment of a visual representation of an estimated current position of a tool superimposed onto a 2D image of intraoperative imaging data ID; and

[0068] Fig. 9C An illustrative embodiment of a visual representation of an estimated current position of a tool, a visual representation of a prescribed position of the tool, and a visual representation of an ideal screw trajectory for a surgical implant on a visual representation of a reconstructed anatomical 3D shape. DETAILED DESCRIPTION

[0069] Reference will now be made in detail to certain embodiments, examples of which are illustrated in the accompanying drawings, in which some, but not all, of the features are shown. Indeed, the embodiments disclosed herein may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Whenever possible, the same reference numerals will be used to refer to the same components or parts.

[0070] Figure 1 A highly schematic perspective view of a system 1 for assisting in positioning a tool 5 installed in an operating room is shown, with a patient 200 lying on an operating table 2. As illustrated, the system 1 includes a computing device 10; an imaging device 20; and a display device 30. The system 1 is illustrated based on an embodiment that utilizes radiation-based images as intraoperative images. Therefore, the imaging device 20 includes a C-arm imaging device 20 that captures intraoperative images using X-ray technology. The C-arm imaging device 20 includes a generator (X-ray source) 22. The C-shaped connecting element (C-arm) 24 allows movement horizontally, vertically and / or around a rotation axis, so that a 2D X-ray image of the patient 200 can be generated from various viewing angles around the patient. The generator 22 emits X-rays that penetrate the patient's body 200. The detector 26 converts the X-rays into imaging data ID that is transmitted to the computing device 10.

[0071] The imaging device 20 is communicatively connected to the computing device 10 and is arranged in the vicinity of the patient 200, thereby allowing the imaging device 20 to capture intraoperative imaging data ID of the patient 200, so that two or more of the 2D images capture a specific body part 202 of the patient 200 from two or more different perspectives relative to the specific body part 202 of the patient 200. One or more of the same plurality of 2D images capturing the specific body part 202 also captures at least a portion of the tool 5 from at least one perspective.

[0072] In the illustrated embodiment, the display device 30 includes a series of computer screens 32 that are communicatively connected to the computing device 10 and configured to display the guidance data GD.

[0073] Now go to Figure 2 , will describe the steps of a computer-implemented method for assisting in positioning a tool 5 relative to a specific body part 202 of a patient 200.

[0074] As in Figure 2 As shown in , the method includes the following main steps:

[0075] Step S10: capturing intraoperative imaging data;

[0076] Step S20: receiving intraoperative imaging data;

[0077] Step S30: using the intraoperative imaging data and using an artificial intelligence based algorithm corresponding to the specific body part 202 to reconstruct the anatomical 3D shape;

[0078] Step S40: estimating the current position of the tool 5c based on the intraoperative imaging data ID;

[0079] Step S50: identifying the specified position 5p of the tool;

[0080] Step S60: generating positioning guidance data GD comprising a visual representation of the estimated current position 5c of the tool 5 relative to the anatomical 3D shape AS of the specific body part 202; and

[0081] Step S70: Use the display device 30 to output the positioning guide data GD.

[0082] Steps specific to particular embodiments are illustrated in the figures using dashed lines.

[0083] In step S10, intraoperative imaging data ID is captured by an imaging device 20 arranged in the vicinity of the patient 200. The intraoperative imaging data ID includes data indicating viewing angles corresponding to the plurality of 2D images, data identifying the position and / or orientation of the imaging device 20 that captured the plurality of 2D images (such as the position of the imaging device 20 in an x, y, and z Cartesian coordinate system and / or orientations such as roll, pitch, and yaw relative to a specific body part 202 of the patient 200).

[0084] According to a first embodiment, data indicating viewing angles corresponding to a plurality of 2D images are stored in a data storage device included by or communicatively connected to the computing device 10. The viewing angles stored in the data storage device are determined by tracking the C-arm 24 to estimate imaging parameters at exposure, by which the 2D images acquired intraoperatively can be assigned to their corresponding intrinsic and extrinsic imaging parameters, which effectively define the viewing angles according to which the 2D images have been generated. Optionally, tracking the C-arm 24 is preceded by a calibration process, i.e., a preoperative calibration process (i.e., precalibration), in which the C-arm 24 is manipulated in a specific manner to cover a jagged range of motion. During this precalibration phase, a mathematical relationship between tracking observations and imaging parameters is established at specific posture intervals, which will later be used to derive an interpolation function that can generate intraoperative imaging parameters based on the tracking data.

[0085] Alternatively or additionally, the computing device 10 estimates the viewing angle corresponding to the intraoperative imaging data ID based on the intraoperative imaging data ID. In one embodiment, the calibration algorithm extracts the viewing angle of the 2D image by placing a precisely manufactured calibration object (i.e., a phantom) (the calibration object includes different features (e.g., radiopaque features) with known geometry) in the imaging field and estimating imaging parameters based on the projections of those features.

[0086] Alternatively or additionally, an artificial intelligence-based algorithm is used to estimate the viewing angle corresponding to the intraoperative imaging data ID, and the algorithm is trained using a large number of imaging data sets with known viewing angles. In order to overcome the limitations of the availability and / or accuracy of imaging data sets with known viewing angles, a large number of imaging data sets including 2D images from known viewing angles are generated based on 3D imaging data (especially computed tomography CT scans). For example, simulated intraoperative fluoroscopic photographs (i.e., digitally reconstructed radiographs (DRRs) generated based on preoperative CT scans and their corresponding posture parameters are used to train a convolutional neural network (CNN) for regression tasks. Using this artificial intelligence-based algorithm trained before surgery, the intraoperative positioning of the imaging device 20 can be estimated based solely on intraoperative images without the need for external tracking devices or calibration phantoms.

[0087] In a subsequent step S20 , the computing device 10 receives the intraoperative imaging data ID from the imaging device 20 via its data input interface 14 .

[0088] In the subsequent step S30, the computing device 10 reconstructs the anatomical 3D shape AS of the specific body part 202 using an artificial intelligence-based algorithm corresponding to the specific body part 202 based on the intraoperative imaging data ID and the data indicating the viewing angles corresponding to the plurality of 2D images. Figure 3 , Figure 4 , Figure 5 , Figure 6 and Figure 7 A detailed description of step S30 of reconstructing the anatomical 3D shape AS is provided.

[0089] In step S40, the current position 5c of the tool 5 relative to the anatomical 3D shape AS of the specific body part 202 is estimated based on the 2D image of the imaging data ID that captures the tool 5. The current position 5c of the tool 5 is performed based on the prior knowledge of the geometry of the tool 5 described by the tool geometry model. First, the projection of the tool geometry model is compared with at least a portion of the tool 5 captured by the corresponding 2D image of the intraoperative imaging data ID. The tool geometry model is projected onto the plane of one or more 2D images in the 2D image of the intraoperative imaging data ID that captures at least a portion of the tool 5. The plane of the 2D image of the intraoperative imaging data ID is determined based on the viewing angle of each 2D image. Thereafter, the positioning of the tool geometry model projected onto the plane of the 2D image of the intraoperative imaging data ID is determined, which matches (best) at least a portion of the tool 5 as captured by the corresponding 2D image of the intraoperative imaging data ID.

[0090] According to the embodiments disclosed herein, in the initial stage of the method of assisting in positioning a tool, although the anatomical 3D shape is reconstructed once, the estimation of the current position 5c of the tool 5 is repeatedly performed at set intervals and / or triggered by certain events and / or manually triggered.

[0091] In order to improve the accuracy of the estimation of the positioning of the tool 5, the tool 5 is verified according to a tool geometric model. The tool geometric model is specifically designed to optimize the estimation of its positioning based on as few intraoperative 2D images as possible. Specifically, the tool is designed so that at least a part of it is not completely rotationally symmetric around any axis of the Cartesian coordinate system, so as to allow the orientation of the tool to be estimated based on the 2D image. Alternatively or additionally, the tool is designed to include special markings to facilitate its identification based on the 2D intraoperative image.

[0092] In step S50, the computing device 10 identifies the prescribed position 5p of the tool 5 relative to the anatomical 3D shape AS of the specific body part 202, and the visual representation of the prescribed position 5p of the tool 5 is superimposed on the visual representation of the estimated current position 5c of the tool 5 to assist the surgeon in correctly positioning the tool 5.

[0093] refer to Figure 8 An embodiment of determining a prescribed position 5p of the tool 5 is described.

[0094] Figure 3A flowchart illustrating the steps of reconstructing an anatomical 3D shape AS based on an intraoperative imaging data ID and data indicating viewing angles corresponding to a plurality of 2D images is shown. As illustrated, the reconstruction of the anatomical 3D shape AS is performed in two stages: step S32 - segmenting the intraoperative imaging data ID so as to identify a specific body part 202 of the patient 200; and step S34 - further reconstructing the anatomical 3D shape AS using the segmented intraoperative imaging data ID. Step S32 - segmenting the intraoperative imaging data ID using an artificial intelligence-based detection and segmentation model so as to identify a specific body part 202 of the patient 200 is illustrated in Figure 4 In the present invention, a method is used to segment intraoperative 2D images of the spine to identify individual vertebrae, such as for segmenting intraoperative 2D images of the spine to identify individual vertebrae. To train an AI-based detection and segmentation model, synthetic X-rays (i.e., DRRs) are generated based on different viewpoints around the patient 200 given an input viewing angle CT scan. CT scans for this purpose can be collected using a public dataset that includes CT scans and corresponding vertebral level annotations. For example, using this method, a training database with over 40,000 annotated 2D images can be created based on only 200 preoperative CT scans.

[0095] Figure 5 A schematic illustration of another embodiment of step S32 of segmenting intraoperative imaging data ID according to a two-stage method is shown, including identifying a region of interest and then semantically segmenting the region of interest so as to identify a specific body part 202 of the patient 200 within the region of interest. In order to segment the intraoperative imaging data ID, an artificial intelligence-based detection and segmentation model (such as a convolutional neural network-based detection and segmentation model) is used to first identify a region of interest within the intraoperative imaging data ID, the region of interest containing the specific body part 202 of the patient 200. The region of interest is then semantically segmented using the artificial intelligence-based detection and segmentation model, thereby generating segmented intraoperative imaging data ID. The artificial intelligence-based detection and segmentation model for the segmentation of step S32 is trained using supervised learning. First, a convolutional neural network (CNN)-based detection model is trained to identify individual body parts (vertebral levels in the illustrated embodiment) on a 2D image by detecting the coordinates of each bounding box including a single body part (a single vertebra). The identified bounding box is then used as the region of interest to crop the intraoperative 2D image. Furthermore, an end-to-end segmentation model is trained to semantically segment the projections of the vertebrae within the region of interest.During the inference phase, intraoperative X-ray images are fed into the segmentation model, which produces semantic segmentations for each vertebral level (to be used for 3D reconstruction purposes).

[0096] Figure 6 shows the intraoperative imaging data ID based on the segmentation and the view angle P corresponding to the plurality of 2D images.1-n 202, using an artificial intelligence-based algorithm corresponding to a specific body part 202 to reconstruct an anatomical 3D shape AS of the specific body part 202. As illustrated, the segmented intraoperative imaging data ID is back-projected into a 3D coordinate system to create an anatomical 3D shape for each body part 202 (vertebra). Back-projection is performed at a view angle P of the 2D image. 1-n The anatomical 3D shape AS is constructed step by step, and the more 2D images the intraoperative data ID includes, the more accurate the reconstructed anatomical 3D shape AS becomes - as shown in Figure 6 The lower part of the anatomy is illustrated in the sequence of 3D shapes.

[0097] In addition to the reference Figure 3 In addition to steps S32 and S34 described above, according to Figure 7 In another embodiment illustrated in the flowchart of FIG. 1 , in another step S36, the anatomical 3D shape AS is further enhanced using the non-segmented imaging data. init Taking into account the potential errors in calibration and segmentation, a 3D shape enhancement model is used to enhance the reconstructed initial anatomical 3D shape AS init The 3D shape enhancement model (particularly the convolutional neural network CNN architecture) is provided with two input streams. The first input stream includes the anatomical 3D shape AS init The second input stream includes a 2D segmentation of a specific body part 202 on a 2D image of the intraoperative imaging data ID. In this way, the 3D shape enhancement model is trained to complete the missing parts of the initial reconstruction (taking into account the possibility of data loss during the initial reconstruction due to missing projection views) by injecting the patient-specific shape information retained in the original intraoperative 2D image, thereby reconstructing an enhanced anatomical 3D shape AS enh .

[0098] Now go to Figure 8 , an embodiment of determining a prescribed position 5p of a tool 5 is described with reference to a use case of a surgical procedure of implanting a pedicle screw into a vertebra of a patient 200. The prescribed position 5p of the tool 5 is determined by an optimization function based on the anatomical 3D shape AS of a specific body part 202 and data indicative of the surgical procedure.

[0099] Determining the prescribed position 5p of the tool 5 based on the anatomical 3D shape AS is advantageous because it has the potential to improve surgical workflow by eliminating the need for preoperative scanning and the corresponding manual process, which can be expensive and time-consuming. Supervised learning and reinforcement learning RL are used to train an artificial intelligence-based optimization function based on a clinical dataset consisting of ideal screw trajectories identified by experts. The prescribed position 5p of the tool 5 is then determined based on the ideal screw trajectory IST, which is further based on the prior knowledge of the geometry of the tool 5.

[0100] Fig. 9A , Fig. 9B and Fig. 9C An implementation scheme for positioning the guidance data GD is shown. Fig. 9A Positioning guidance data GD is shown comprising a visual representation of an estimated current position 5c of the tool 5 and a visual representation of a prescribed position 5p of the tool 5 superimposed onto a visual representation of the reconstructed anatomical 3D shape AS.

[0101] Fig. 9B Positioning guidance data GD is shown, comprising a visual representation of the estimated current position 5c of the tool 5 superimposed onto the 2D image of the intraoperative imaging data ID.

[0102] Fig. 9C Positioning guidance data GD is shown comprising a visual representation of an estimated current position 5c of the tool 5 on a visual representation of the reconstructed anatomical 3D shape AS, a visual representation of a prescribed position 5p of the tool 5 and a visual representation of an ideal screw trajectory for the surgical implant.

[0103] Name List

[0104] 1 System

[0105] 2 Operating table

[0106] 5 Tools

[0107] Current Positioning of 5c Tools

[0108] 5p tool’s prescribed positioning

[0109] 10 Computing Devices

[0110] 12 (Computing device) data output interface 14Data input interface (of computing equipment) 16 (processing unit of a computing device)

[0111] 18 (computing device) storage unit 20 Imaging equipment

[0112] 22 Generator (X-ray source)

[0113] 24C-shaped connecting element (C-arm)

[0114] 26 Detector

[0115] 30 Display Devices

[0116] 32 Computer Screen Arrangement 200 patients

[0117] 202 Specific body part ID Intraoperative Imaging Data GD Positioning Guide Data AS Anatomical 3D Shapes

[0118] 1ST Ideal pedicle screw trajectory P 1-n Viewing angle (of 2D intraoperative images)

Claims

1. A computer-implemented method for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200), the method comprising: a) receiving, by a computing device (10), intraoperative imaging data (ID) from an imaging device (20) arranged in the vicinity of the patient (200), the intraoperative imaging data (ID) comprising 2D images, a plurality of the 2D images capturing the specific body part (202) of the patient (200) from a plurality of different viewing angles relative to the specific body part (202) of the patient (200), and one or more of the plurality of the 2D images capturing at least a portion of the tool (5) from at least one viewing angle; b) reconstructing, by the computing device (10), an anatomical 3D shape (AS) of the specific body part (202) using an artificial intelligence-based algorithm corresponding to the specific body part (202) based on the intraoperative imaging data (ID) and data indicating viewing angles corresponding to the plurality of the 2D images; c) estimating, by the computing device (10), a current positioning (5c) of the tool (5) relative to the anatomical 3D shape (AS) of the specific body part (202) based on the intraoperative imaging data (ID); as well as d) generating, by the computing device (10), positioning guidance data (GD), the positioning guidance data comprising a visual representation of an estimated current positioning (5c) of the tool (5) relative to the anatomical 3D shape (AS) of the specific body part (202).

2. The computer-implemented method of assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to claim 1, wherein the step of reconstructing the anatomical 3D shape (AS) of the specific body part (202) by the computing device (10) comprises: The artificial intelligence-based algorithm is trained using a large number of annotated imaging data sets that capture body parts corresponding to the specific body parts (202) of the patient (200), wherein the annotations include data identifying and / or describing attributes of the body parts.

3. The computer-implemented method for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200) according to claim 2 comprises generating a large number of annotated imaging data sets based on annotated 3D imaging data, in particular based on a computed tomography (CT) scan, wherein the annotated imaging data sets comprise 2D images of body parts corresponding to the specific body part (202) of the patient (200) captured from a plurality of different perspectives, the annotated 3D imaging data capturing the body parts corresponding to the specific body part (202) of the patient (200).

4. The computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 1 to 3, wherein the step of reconstructing the anatomical 3D shape (AS) comprises: a) segmenting the intraoperative imaging data (ID) so as to identify the specific body part (202) of the patient (200); as well as b) The segmented intraoperative imaging data (ID) is further used to reconstruct the anatomical 3D shape (AS).

5. The computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to claim 4, wherein the step of segmenting the intraoperative imaging data (ID) comprises: a) identifying a region of interest within the intraoperative imaging data (ID) containing the specific body part (202) of the patient (200) using an artificial intelligence based detection and segmentation model, such as a convolutional neural network based detection and segmentation model; as well as b) performing semantic segmentation on the region of interest using the artificial intelligence-based detection and segmentation model, thereby generating the segmented intraoperative imaging data (ID).

6. A computer-implemented method for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 1 to 5, further comprising estimating by the computing device (10) a viewing angle corresponding to the intraoperative imaging data (ID).

7. A computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to claim 6, wherein estimating the viewing angle corresponding to the intraoperative imaging data (ID) is performed using a tool geometry model indicating the geometry of the tool (5) and comprises: a) calculating a plurality of projections of the tool geometric model according to a plurality of candidate viewing angles; as well as b) identifying a viewing angle corresponding to the intraoperative imaging data (ID) by comparing at least a portion of the tool (5) captured by a corresponding 2D image of the intraoperative imaging data (ID) with the plurality of projections from the plurality of candidate viewing angles.

8. A computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 1 to 7, wherein estimating the current positioning (5c) of the tool (5) is performed using a tool geometry model indicating the geometry of the tool (5), the step of estimating the current positioning (5c) of the tool (5) comprising: a) comparing a projection of the tool geometry model onto one or more planes of the 2D images of the intraoperative imaging data (ID) with at least a portion of the tool (5) captured by a corresponding 2D image of the intraoperative imaging data (ID), and / or b) determining a positioning of the tool geometric model, said positioning producing a projection onto the plane of the 2D image of the intraoperative imaging data (ID), said projection matching at least a portion of the tool (5) captured by the corresponding 2D image of the intraoperative imaging data (ID).

9. A computer-implemented method for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200) according to one of claims 1 to 8, wherein generating positioning guidance data (GD) comprises superimposing a visual representation of an estimated current positioning (5c) of the tool (5) onto a visual representation of a reconstructed anatomical 3D shape (AS).

10. The computer-implemented method of assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 1 to 9, further comprising: a) identifying, by the computing device (10), a prescribed positioning (5p) of the tool (5) relative to the anatomical 3D shape (AS) of the specific body part (202); and b) superimposing a visual representation of the prescribed position (5p) of the tool (5) onto a visual representation of the estimated current position (5c) of the tool (5).

11. A computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 10 or 11, wherein the step of identifying the prescribed position (5p) of the tool (5) comprises: a) retrieving, by the computing device (10), the specified position (5p) of the tool (5) from a data storage device comprised by the computing device (10) or communicatively connected to the computing device (10); and / or b) calculating, by the computing device (10), the prescribed position (5p) of the tool (5), the prescribed position (5p) of the tool (5) being determined by an optimization function based on the anatomical 3D shape (AS) of the body part and data indicative of a surgical procedure.

12. The computer-implemented method of assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 1 to 11, further comprising: a) providing a tool (5) according to a tool geometry model; and / or b) capturing intraoperative imaging data (ID) using an imaging device (20), the intraoperative imaging data capturing at least one body part of the patient (200) and at least a portion of the tool (5); and / or c) controlling a display device (30) by the computing device (10) to display at least a part of the guidance data (GD).

13. A computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to any one of claims 1 to 12, wherein the intraoperative imaging data (ID) comprises one or more of the following: a) a radiation-based image, in particular an X-ray image of the specific body part (202) of the patient (200) or a part of the tool (5); b) an ultrasound image of the specific body part (202) of the patient (200) or a portion of the tool (5); c) an arthroscopic image of the specific body part (202) of the patient (200) or a portion of the tool (5); d) an optical image of the specific body part (202) of the patient (200) or a portion of the tool (5); e) Cross-sectional imaging of a portion of the patient (200) or the tool (5).

14. A computer-implemented method for assisting positioning of a tool (5) relative to a specific body part (202) of a patient (200) according to one of claims 1 to 13, wherein in preparation / performing surgical treatment of the patient (200), the following steps are repeatedly or continuously performed over a period of time: receiving intraoperative imaging data (ID); estimating the current positioning (5c) of the tool (5); and generating guidance data (GD).

15. A computing device (10), comprising: a) a data input interface (14) communicatively connected to the imaging device (20) and configured to receive intraoperative imaging data (ID); b) a data output interface (12) configured to transmit at least a portion of the guidance data (GD) to a display device (30) communicatively connected to the data output interface (12); b) a processing unit (16); and c) A storage unit (18) comprising instructions which, when executed by the processing unit (16), cause the computing device (10) to perform a method according to any one of claims 1 to 14.

16. A system (1) for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200), the system (1) comprising: a) The computing device (10) according to claim 15; b) an imaging device (20) arranged and configured to capture intraoperative imaging data (ID) comprising 2D images, wherein a plurality of the 2D images capture the specific body part (202) of the patient (200) from a plurality of different viewing angles relative to the specific body part (202) of the patient (200), and one or more of the plurality of the 2D images capture at least a portion of the tool (5) from at least one viewing angle; and c) a display device (30) configured to display a human interpretable representation of at least a portion of the guidance data (GD), the display device (30) being communicatively connected to the data output interface (12) of the computing device (10), The system (1) is configured to implement the method according to any one of claims 1 to 14.

17. A system (1) for assisting in positioning a tool (5) relative to a specific body part (202) of a patient (200) according to claim 16, wherein the system (1) further comprises a tool (5) having a geometric shape corresponding to a tool geometric model.

18. A computer program product comprising instructions which, when executed by a processing unit (16) of a computing device (10), cause the computing device (10) to perform the method according to any one of claims 1 to 14.

19. A computer program product according to claim 18, wherein the instructions include an artificial intelligence-based algorithm corresponding to a specific body part (202) of the patient (200), the artificial intelligence-based algorithm having been trained using a large number of annotated imaging data sets, the annotated imaging data sets capturing the body part corresponding to the specific body part (202) of the patient (200), wherein the annotation includes data identifying and / or describing attributes of the body part.