Navigation system and method having a 3D surface scanner - Patents.com

Abstract

The present disclosure relates to a surgical navigation system (1) for use during a surgical intervention on a patient (P), comprising a tracking system (2) adapted to track at least the patient (P) and preferably at least one medical instrument (4), a 3D surface scanner (6) adapted to generate a 3D surface image (8) of an intracorporeal anatomical structure of the patient (P) of interest, the 3D surface scanner (6) being tracked by the tracking system (2), in particular via an attached tracking device (10), and a control unit (12), which controls the 3D surface recording (8) and the detection of the detected 3D surface scanner (6) and the patient (P). The present disclosure further relates to a surgical navigation system (1) adapted to process the detected and tracked poses of the 3D surface scanner (6), determine a pose of the 3D surface record (8) relative to the patient (P) using the detected and tracked poses of the 3D surface scanner (6), supplement a virtual digital 3D anatomical model (14) of the patient (P) in the determined pose with the 3D surface record (8), and visually output a view of the supplemented digital 3D anatomical model (14) of the patient (P) for surgical navigation via an output device (16), in particular an operating room monitor, in order to provide a current digital 3D anatomical model (14) supplemented with intracorporeal anatomical structures of the target patient (P) for navigation.Furthermore, the present disclosure relates to a surgical assistance system (100) with a surgical robot (102), a navigation method, and a computer readable storage medium according to alternative independent claims.

Description

[Technical field]

[0001] The present disclosure relates to a surgical navigation system for navigation during a surgical intervention on a patient, in particular during a knee joint operation. Furthermore, the present disclosure relates to a surgical assistance system, a navigation method and a computer-readable storage medium according to the preambles of the independent claims. [Background technology]

[0002] Current surgical navigation systems can be essentially divided into two groups / types / classes or varieties of surgical navigation systems.

[0003] The first group of navigation systems is image-assisted or picture-assisted navigation, which uses images such as computed tomography images (CT images), magnetic resonance images (MR images), X-ray images, or ultrasound images that are created preoperatively and stored before the intervention is performed. These images must then be compared with the patient's position before and during the intervention to allow navigation of the instruments or implants.

[0004] The second group of navigation systems is imageless or pictureless navigation, which works without (pre-operative) images. This second group of navigation systems creates a basic (geometric) virtual or digital model of the patient during the intervention, usually performed by sweeping anatomical landmark or orientation points with navigation probes and pointers, or by kinematic analysis, for example. Such a 3D anatomical model digitally reproduces important anatomical points in computer-aided systems, allowing the surgeon to navigate.

[0005] However, a drawback of image-based navigation systems is that they require complex and time-consuming registration, which often does not provide the necessary precision, especially in knee or cranial surgery.

[0006] In contrast, the second group of imageless navigation systems does not require preoperative imaging or intraoperative registration, but has the disadvantage of using only a very rough and basic digital 3D model of the patient's anatomy as reference, e.g. an axial model of the leg during knee surgery. To obtain at least a rough 3D (overview) model of the leg, the actual three-dimensional (3D) structure of the leg is simplified and modeled using straight lines (2D structures) in the (3D) space. This geometric schematic 3D model can then be viewed from different angles with a computer-aided system. Summary of the Invention

[0007] It is therefore an object and goal of the present disclosure to avoid or at least reduce the drawbacks of the prior art, in particular to provide a surgical navigation system, a surgical assistance system, a navigation method and a computer readable storage medium that allow improved and more flexible navigation, in particular without the need for pre-operative imaging / images with subsequent intra-operative registration, and that provide a sufficiently accurate intra-body cavity 3D image of the patient's targeted interventional area as a reference for navigation, e.g. to allow alignment of instruments or implants. In particular, it is an object to provide better, faster, more flexible and more accurate navigation with a navigation system that is as easy to use as possible, and allows three-dimensional detection and visualization of targeted anatomical structures of the interventional area during surgery.

[0008] These objects are solved by the features of claim 1 for a generic surgical navigation system according to the invention, by the features of claim 12 for a generic surgical support system according to the invention, by the steps of claim 13 for a navigation method according to the invention and by the features of claim 15 for a computer-readable storage medium according to the invention.

[0009] The basic idea of ​​the present disclosure is therefore that a surgical navigation system comprises a following / tracking system (especially with a navigation camera) for locating or identifying (the pose of) at least one body part of a patient on which an intervention is to be performed, and a 3D surface scanner, in particular consisting of a following / tracking system and a 3D surface scanner. Furthermore, the navigation system may preferably comprise a medical instrument for the intervention. The 3D surface scanner, in particular a 3D surface scanner camera, is used during the intervention to create a 3D surface image intraoperatively after opening the patient, which is used to create a digital 3D surface model of the patient's internal anatomical structures (digital 3D anatomical model of the patient) or to supplement an existing 3D anatomical model of the patient, which model may ultimately serve as a reference for navigation, in particular for guiding instruments or implants to a target position.

[0010] In particular, the surgical navigation system according to the present disclosure comprises a tracking system, in particular with a navigation camera, preferably one or more tracking devices for following the patient, and preferably one or more tracking devices for following / tracking an instrument or implant. Furthermore, the navigation system comprises a 3D surface scanner that is trackable (or whose pose is detectable in space) by the tracking system / tracking system, in particular with tracking devices / markers for (indirectly) detecting and identifying (via the tracking device) the position, in particular the position and orientation (i.e. the pose) of the scanned intra-body cavity anatomical 3D surface by the tracking system (navigation camera). As the tracking system detects both the pose of the patient's body part and the pose of the 3D surface image, a relative transformation of the patient (or the body part of the target) to the 3D surface image can be established and the pose of the 3D surface image can be related to the patient. The 3D surface scanner targets or is aligned to the anatomical region of interest and detects the 3D surface image of this region. This results in a digital 3D surface model that is integrated into the 3D anatomical model of the patient. This 3D surface image can also be extended to include other anatomical regions by moving the 3D surface scanner, for example manually or under the control of a robotic arm. During (i.e. simultaneously) detecting the real anatomical 3D surface as a (digital) 3D surface image, the 3D surface scanner, in particular the 3D scanning camera, is localized in space (position and / or orientation) and the 3D surface image is associated with the patient on this basis, in particular with the help of a patient tracking device provided on the patient. In particular, the position and / or orientation of the 3D surface image relative to the patient is determined. In other words, the navigation system thus comprises in particular a tracking system (localization system), at least one tracking device attached to the patient and a 3D surface scanner equipped with a tracking device, the 3D surface scanner, in particular as a 3D surface scanning camera, creating a 3D surface image of the patient's targeted internal anatomical structures and the 3D surface scanner being localized relative to the patient. The 3D surface image is transmitted to a control unit or computer system.The navigation system is equipped with an output device, in particular a display, and the 3D anatomical model supplemented by the 3D surface image can be used as a reference for positioning the navigated instrument or implant without the need to use pre-operative images of the patient.

[0011] This 3D surface image is provided to the control unit (as the generated 3D surface) or transmitted to an adapted computer system (as the control unit). The navigation system further comprises an output device, in particular a display (such as an operating room monitor), for visual output. The control unit or computer system creates and supplements a digital three-dimensional (3D) anatomical model of the patient (patient's 3D anatomical model). This 3D anatomical model can optionally be supplemented by palpated / touched anatomical landmark points or with landmark points already present. The computer system or control unit allows planning and navigation of the intervention, for example by visually positioning the orthopedic prosthesis in a target pose. During the intervention, the navigation system of the present disclosure then displays the current position and / or orientation, in particular the pose, of the navigated instruments or implants relative to the patient's 3D anatomical model, together with an optionally included and viewable intervention plan, allowing the surgeon to optimally guide and position the instruments or implants to provide the best patient outcome.

[0012] The present surgical navigation system therefore combines the advantages of image-based and image-less navigation techniques in the best possible way, since it does not require pre-operative images / imaging and subsequent intra-operative registration, but still has an accurate 3D model of the patient (3D anatomical model) as a reference, in particular to position the instruments or implants in the best possible way to obtain the best possible interventional result. Surgical navigation systems show their advantages especially in orthopedic navigation, where the exact shape and arrangement of the bones must be recorded and made available for navigation.

[0013] In further other words, a surgical navigation system for use in a surgical intervention on a patient is disclosed, the surgical navigation system comprising a tracking / following / localization system adapted to detect and track, with respect to its pose, at least a body part of a patient on which an intervention is to be performed, in particular the entire patient, and preferably at least one medical instrument, a 3D surface scanner adapted to create a 3D surface image (in a sense a 2D surface structure in a 3D space) of an intracavitary anatomical structure of the patient on which an intervention is to be performed, the 3D surface scanner being tracked by the tracking system, in particular via an attached tracking device, and a control unit, the control unit being adapted to perform a process for detecting the 3D surface image, the tracked 3D surface scanner and the tracked anatomical structure on which the anatomical structure is to be created, the 3D surface scanner being tracked by the tracking system, in particular via an attached tracking device, and a control unit. the 3D surface scanner is adapted / configured to process the detected and tracked pose of the detected patient (or body part thereof), determine via the detected and tracked pose of the 3D surface scanner a position and orientation (i.e. pose) of the 3D surface image relative to the patient, supplement (in a correlative manner) a virtual digital 3D anatomical model of the patient (or at least a digital 3D anatomical model of the body part of interest) stored in a storage device with the (local) 3D surface image in the specific position and orientation (i.e. pose), and visually output via an output device, in particular an operating room monitor, a view of the supplemented digital 3D anatomical model of the patient for surgical navigation in order to provide an up-to-date digital 3D anatomical model supplemented with intra-body cavity anatomical structures of the patient of interest for navigation.

[0014] 3D surface scanners for navigation systems operate with high speed and precision, are cost-effective and easy to handle, making them particularly suitable for use in the field of medical technology. These advantages therefore extend to navigation systems as well. In particular, 3D surface scanners with structured light can be used for spinal and neuronavigation, creating 3D surface images of the targeted patient anatomical structures during the intervention. This can also be used for the registration of pre-operative data, such as CT or MR images acquired before the intervention. In particular, 3D surface scanners can be used to create 3D point clouds, preferably with overlaying color maps.

[0015] The surgical navigation system can be used for various indications. For example, in knee surgery, after incising and exposing the knee, it can be used to create a 3D surface image of the articular surface of the knee joint. In particular, the 3D surface image of the distal femur can be detected relative to the fiducials of the femoral tracker, while the 3D surface image of the proximal tibia can be detected relative to the fiducials of the tibial tracker. The 3D surface images of the femur and tibia can be optionally supplemented with anatomical orientation points such as the hip joint center, the knee joint center, the patella center, the tibia center, the ankle center, or a 3D anatomical model of the patient with existing anatomical orientation points can be supplemented with the 3D surface images. Upon detection of at least one 3D surface image and preferably anatomical landmarks, in particular the knee prosthesis can be positioned in the intended alignment with the biomechanical axes. The 3D surface images of the femur and tibia allow particularly precise positioning and sizing of the prosthesis.

[0016] For example, in hip surgery, the surgical navigation system can create a 3D surface image of the acetabulum and complement the image-less, landmark-based digital 3D anatomical model of the patient with the 3D surface image of the acetabulum, allowing visualization of anatomical angles and distances during socket alignment as well as visual confirmation of the fit and alignment of the socket implant relative to the bone structure.

[0017] The term "location" means a geometric position in three-dimensional space, in particular specified by coordinates in the Cartesian coordinate system. In particular, a location can be specified by three coordinates X, Y, Z.

[0018] The term "orientation" refers to a direction (such as a position) in space. Orientation can also be said to refer to a direction or rotation in three-dimensional space. In particular, orientation can be specified using three angles.

[0019] The term "pose" includes both position and orientation. In particular, a pose can be specified using six coordinates: three position coordinates X, Y, Z, and three angular coordinates for orientation.

[0020] The term 3D defines that the image data is spatial, i.e., three-dimensional: the patient's body or at least a portion of the body having a spatial extent may be digitally available as image data in a three-dimensional space, for example having a Cartesian coordinate system (X,Y,Z).

[0021] Advantageous embodiments are set forth in the dependent claims and are particularly described below.

[0022] According to an embodiment, the 3D surface scanner of the navigation system may comprise a handle for manually guiding the 3D surface scanner and creating 3D surface images in various poses, and preferably a wireless communication module (WIFI, Bluetooth, IR, etc.). In particular, the 3D surface scanner with the handle may have a battery acting as a self-sufficient module. Alternatively or additionally, the 3D surface scanner may have a mounting, in particular a clamping module, in particular for attachment to a holding arm of the navigation system. Alternatively or additionally, the surgical navigation system may comprise a robot having a robot arm to which the 3D surface scanner is connected as an end effector, in order to automatically create a 3D surface image in at least one predefined pose of the 3D surface scanner, in particular for creating multiple 3D surface images in multiple poses of the 3D surface scanner, preferably according to a sweep algorithm or sweep pattern. In other words, in particular, the 3D surface scanner may be configured to be held by hand, to be attached to a static mechanical holding arm of the navigation system, or to be held and actively moved by a robot arm. In particular, the 3D surface scanner may be attached to a robotic arm of a navigation or assistance system, which autonomously moves (is controlled by) the 3D surface scanner to detect anatomical regions of interest.

[0023] According to further embodiments, the 3D surface scanner may have an emitter for structured light and a sensor for detecting light, in particular an RGB sensor (e.g. a camera), and / or a stereo camera, and / or a ToF (time-of-flight) camera, and / or a LIDAR scanner, in particular to create a 3D point cloud of the anatomical region of the subject as a 3D surface image. In other words, the 3D surface scanner can therefore create a 3D surface image using various techniques, in particular structured light and corresponding detection, stereo camera (stereo vision), ToF camera, or LIDAR sensor / LIDAR camera, in particular to create a 3D point cloud of the anatomical region of the subject.

[0024] Preferably, the tracking system and / or the control unit and / or the 3D surface scanner of the navigation system are adapted to determine the pose using machine vision. In particular, a 2D camera is used to create images from different viewing directions of the object or surface of interest and to calculate spatial information from these images.

[0025] Preferably, the tracking system may comprise an optical tracking system, and / or a machine vision based tracking / localization system, and / or an EM tracking system. In particular, the navigation system may be configured to perform electromagnetic tracking (EM navigation system) in addition to the optical tracking system.

[0026] Preferably, a virtual digital 3D anatomical model of the patient's leg, including the knee, may be stored in a navigation system storage device, and the navigation system with a 3D surface scanner may be adapted to create a 3D surface image of the knee joint, the navigation system for this purpose comprising a tibia tracker and a separate femur tracker adapted to be rigidly placed on the tibia and on the femur for bone tracking. The control unit is adapted to associate the 3D surface image with the tracked leg based on the tracked tibia tracker and femur tracker. In this way, two bone fiducials (the bone parts of the tibia and femur in the area of ​​the knee joint) are tracked separately by navigation, and two or more 3D surfaces of the anatomical structure are detected relative to the corresponding fiducial bones. In particular, the navigation system may thus be adapted to track two fiducial bones with respective trackers separately attached to the tracking system, and to create at least two intracorporeal anatomical 3D surface images relative to the fiducial bones.

[0027] According to an embodiment, the control unit may be adapted to supplement the digital 3D anatomical model of the patient with landmark points, in particular the hip joint center, the knee joint center, and the patella center, the tibia center, and / or the ankle center, in particular via the pointer of the navigation system, in order to obtain a composite digital 3D anatomical model of points, (biomechanical) axes, and surfaces. In other words, the detected 3D surface may be supplemented with additional anatomical landmark points / landmarks detected on the patient by sweeping or kinematic movements, resulting in a composite model of points, (biomechanical) axes, and surfaces.

[0028] According to a further embodiment, the control unit may be adapted to process at least two 3D surface images in at least two different (detected) poses of the 3D surface scanner in order to obtain an extended and / or more accurate 3D surface image as a whole image based on the individual 3D surface images. This technique is also known as stitching technique, combining multiple individual 3D surface images to form a whole image. Similar to the panorama function of photography, multiple images are linked together, especially in overlapping areas. However, a pose detected by the 3D surface scanner does not necessarily mean that there is an overlapping area, separate areas may be added. Thus, the navigation system allows absolute addition, no relative reference is required. Preferably, the control unit may be adapted to calculate a resultant 3D surface image for this area in the overlapping area of ​​the 3D surface images in order to increase the accuracy. In particular, the 3D surface image may be detected over a single rest position, a specific pose, or multiple positions, specific poses, thereby increasing the scanned surface area and thus the 3D anatomical model of the patient.

[0029] Preferably, the 3D surface scanner may be adapted to be inserted into a body cavity of a patient, in particular with a maximum width and height, particularly preferably with a maximum dimension of 10 cm. The 3D surface scanner, in particular the 3D surface scanner camera, may be small enough to be inserted into the patient's anatomy in order to detect the surface of the area deep within as a 3D surface image. The detected 3D surface image may be used to plan the position of an implant or instrument during surgery.

[0030] In particular, the surgical navigation system may comprise a surgical endoscope, in particular the front / distal endoscope head thereof, with an integrated 3D surface scanner. The 3D surface scanner may therefore be a (component) part of the surgical endoscope.

[0031] According to one embodiment, the 3D anatomical model of the patient with the implant according to the intervention plan may be stored in the storage device of the navigation system, and the control unit may be adapted to verify the actual implant orientation intraoperatively with respect to the implant orientation according to the intervention plan via the 3D surface images by the 3D surface scanner and to issue a warning via the display device in case of deviations. Thus, the 3D surface scanner may be used to check the position and compare with the intervention plan after the implant has been inserted. In particular, the control unit compares the target orientation of the implant in the 3D anatomical model of the patient with the actual orientation according to the executed 3D surface images and deviations are displayed via the output device in order to provide the surgeon with information about the alignment.

[0032] In particular, the control unit may be adapted to supplement the 3D surface image in the digital 3D anatomical model of the patient as a featureless surface or as a surface with a color texture or as a surface with a grayscale texture, in other words, the 3D surface image may be preferably converted and stored in the form of a featureless surface or a surface with a color or grayscale texture.

[0033] According to one embodiment, the control unit may be adapted to recognize predefined structures of the object in the 3D surface image, in particular bones, and to supplement the digital 3D anatomical model with only these predefined structures of the object. Thus, in particular, the detected 3D surface image may be manipulated such that only structures of the object remain in the 3D surface image. Structures that do not belong to bones, such as skin, are automatically excluded from the 3D surface image.

[0034] With regard to the surgical assistance system, the object is solved by including a navigation system according to the present disclosure and a medical robot having a medical end effector. In particular, the navigation system can be integrated into the surgical robot without the need for pre-operative image capacity. Thus, the surgical assistance system can be used to provide data for navigation via the navigation system, and automated or semi-automated interventions can be performed via the robot with the end effector. In particular, in knee surgery, the robot can be controlled based on the created and supplemented 3D anatomical model of the patient so that the end effector performs the resection of the tibia and femur and automatically inserts, preferably in a correct orientation, a knee prosthesis.

[0035] With regard to a navigation method for use in a surgical intervention on a patient, in particular in knee arthroplasty, preferably in the surgical navigation system of the present disclosure, the object is solved in that the navigation method comprises the following steps: Tracking the patient and preferably at least one medical instrument, in particular the pointer, via a tracking / following system, creating a 3D surface image of the intracavitary anatomical structure of the subject patient with the 3D surface scanner; tracking the 3D surface scanner with the tracking system, in particular via an attached tracking device; determining the orientation of the 3D surface image relative to the patient via the detected orientation of the 3D surface scanner; A digital 3D anatomical model of the patient in a specific position is supplemented with (associated) 3D surface images, Outputting a visual view of the supplemented digital 3D anatomical model of the patient for surgical navigation via an output device, in particular an operating room monitor, in order to provide an up-to-date digital 3D anatomical model supplemented with intracorporeal anatomical structures of the target patient for navigation. Using this navigation method, in particular in knee joint surgery with knee prostheses, allows for quick, easy and flexible interventions and further improves patient safety.

[0036] In particular, the navigation method may further comprise creating a 3D surface image of the front side of the femur in a first position of the 3D surface scanner, in particular via a robot-guided 3D surface scanner, detecting the position of the 3D surface scanner and the femur via a femur tracker (on the femur) by a tracking system, creating a 3D surface image (in particular of the front side) of the tibia in a second position of the 3D surface scanner, in particular in a robot-guided manner, detecting the position of the 3D surface scanner and the tibia via a tibia tracker on the tibia by the tracking system, and supplementing a digital 3D analytical model of the patient with the 3D surface image in the specific position by the control unit (posture-corrected integration of digital image data).

[0037] The object is solved by a computer readable storage medium containing instructions which, when executed by a computer, cause the computer to perform the method steps of the navigation method according to the present disclosure.

[0038] Any disclosure relating to the navigation system according to the present disclosure also applies to the navigation method according to the present disclosure and vice versa. In particular, a control unit of the navigation system may be adapted (e.g., to comprise a processor and a storage device having instructions) to perform method steps according to the navigation method.

[0039] The present invention will now be described in more detail with reference to preferred embodiments using the drawings. [Brief description of the drawings]

[0040] [Figure 1] FIG. 1 is a schematic perspective view of an assistance system having a surgical navigation system according to a preferred embodiment of the present disclosure adapted for knee surgery. [Diagram 2] Figure 2 shows a schematic 3D anatomical model of the patient with digitized anatomical landmarks and the scanned 3D surface image of the knee surgery from Figure 1. [Diagram 3]FIG. 3 is a schematic perspective view of a 3D surface scanner of a further embodiment navigation system in the form of a 3D scanning camera that detects the articular surfaces of the knee joint and outputs a view of a 3D anatomical model via a screen. [Figure 4] FIG. 4 is a flowchart of a navigation method according to a preferred variant.

[0041] The figures are schematic in nature and are intended only to aid in the understanding of the invention. Identical elements are provided with the same reference numbers. Features of the various embodiments are interchangeable. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0042] FIG. 1 is a schematic perspective view of a preferred embodiment of a surgical assistance system 100 including a robot 102 and a preferred embodiment of a surgical navigation system 1. As shown in FIG.

[0043] The surgical navigation system 1 is used for a knee operation on a patient P, during which a resection of the tibia is performed and an implant is inserted into the knee joint. Accurate and up-to-date navigation is crucial for this surgical intervention on the patient.

[0044] The navigation system 1 comprises an optical tracking system 2 for detecting the pose of a solid object, the optical tracking system 2 being adapted to detect and track a patient P and a medical instrument to which the solid object is attached. In this embodiment, the tracking system 2 comprises a stereo camera for three-dimensional detection and a downstream processing unit for determining the pose in space based on the stereo images. The solid object detected by the tracking system 2 is in the form of a tracking device 10 having three spaced spheres, and via detection of the pose of the tracking device 10, the tracking system can indirectly infer the pose of the relevant part to which the tracking device 10 is attached.

[0045] The navigation system 6 further comprises a mobile handheld 3D surface scanner 6, which is specially adapted to create a 3D surface image 8 of the intracorporeal anatomical structure of the subject patient P (in this case the knee joint with the corresponding bones). The 3D surface scanner 6 itself is tracked by the tracking system 2 via an attached tracking device 10, so that the pose of the 3D surface scanner 6 in space can be determined by the tracking system 2. That is, the 3D surface scanner 6 records a three-dimensional surface of the knee joint of the patient P (with color information like a three-dimensional map and to some extent height information), and the pose of the 3D surface scanner 6 is detected in the 3D surface image 8. Since the pose of the 3D surface image 8 relative to the pose of the 3D surface scanner 6 can be determined, and furthermore the pose of the 3D surface scanner relative to the stereo camera of the tracking system 2 can also be determined, the transformation between the 3D surface image 8 and the tracking system can also be determined.

[0046] Since the knee joint (here the femur and tibia) of the patient P is further detected via the tracking device 10, the pose of the legs as part of the patient's torso can also be determined with respect to the tracking system 2 and a transformation from the poses of the individual bones into the local coordinate system (KOS) of the tracking system 2 can be determined. That is, the pose of the femur (in the local KOS) and the pose of the tibia (in the local KOS) and thus the pose of the knee joint with respect to the 3D surface image 8 can be determined by linking the two transformations.

[0047] In particular, a control unit 12 with a processor, a RAM module and a storage is adapted to process the 3D surface image 8 and the detected and tracked poses of the 3D surface scanner 6 and the patient P, to determine the pose of the 3D surface image 8 relative to the patient P via the pose of the 3D surface scanner 6 and to supplement a virtual digital 3D anatomical model 14 of the patient P in a particular pose with the 3D surface image 8. By detecting the real leg of the patient P by the tracking system 2, a correlation can be made between the virtual model of the leg of the patient P and the real leg, and the pose of the 3D surface image 8 relative to the pose of the virtual leg can also be determined. It can also be said that the 3D anatomical model 14 of the patient P is stored in a virtual or digital space, for example as a schematic biomechanical axis model (axes with start and end points as bone representations) with further landmark points, and that this 3D anatomical model 14 of the patient P is supplemented by the surfaces of the scanned object, i.e. the intervention area, in order to provide more detailed digital information on the intervention area and to assist in navigation during the operation.

[0048] The view of the supplemented digital 3D anatomical model 14 of the patient P can then be visually output with the support of a computer via a display, e.g., an operating room monitor, as an output device 16 for surgical navigation, in order to provide for navigation an up-to-date digital 3D anatomical model 14 supplemented with the intracavitary anatomical structures of the target patient P. This makes it possible to avoid the need for time-consuming registration, yet still provide a 3D anatomical model 14 of the patient P, which provides particularly accurate anatomical structure information for navigation in the intervention area.

[0049] The surgical navigation system 1 offers the special possibility of being able to create 3D surface images 8 in different poses and therefore with different viewing angles and views in order to extend the 3D anatomical model 14 of the patient P with regard to the relevant recording areas and to further improve the accuracy by means of redundant information and optimization algorithms.

[0050] The 3D surface scanner 6 is configured so that it can be easily handheld and guided by the surgeon. For this purpose, the 3D surface scanner 6 is provided with a handle 18 for manual gripping and guidance, a rechargeable battery for providing a self-sufficient energy supply, and a wireless communication module 20 in the form of a WLAN or Bluetooth module for transmitting data to the control unit 12 wirelessly.

[0051] In another embodiment not shown, the surgical navigation system may further comprise a robot having a robot arm to which the 3D surface scanner 6 is connected as an end effector in order to automatically create a 3D surface image 8 in at least one predefined pose of the 3D surface scanner 6, in particular to create multiple 3D surface images 8 in multiple poses of the 3D surface scanner 6 according to a sweep algorithm or a sweep pattern. If the 3D surface scanner 6 is robot-guided, an algorithm or predefined poses can be used to automatically move the 3D surface scanner 6 to these poses and create images. This configuration also has the advantage that the pose of the 3D surface scanner 6 is not only detectable via the tracking device, but also by its kinematics in relation to the robot base, where the transformation between the robot base and the tracking system can be determined. In this way, the pose of the 3D surface scanner 6 can be determined even if the view of the tracking device is obstructed.

[0052] A virtual digital 3D anatomical model 14 of the leg of the patient P, including the knee, is stored in a storage device 22 of the navigation system 1. A 3D surface scanner 6 creates a 3D surface image 8 of the knee joint, the corresponding bones of which can be detected by the tracking system 2 via separate tibia tracker 24 and femur tracker 26 rigidly screwed to the bones. The control device 12 is adapted to correlate the 3D surface image 8 to the tracked leg parts based on the tracked tibia tracker 24 and femur tracker 24.

[0053] FIG. 2 shows an example of creating a patient model or 3D anatomical model 14 based on scanned 3D surface images and digitized anatomical landmarks for navigation during an intervention on the knee of the patient shown in FIG. 1. The patient's real bones are shown diagrammatically on the left side of FIG. 2, while on the right side the 3D anatomical model 14 of the patient P is shown with linear biomechanical axes 28 (approximately) along the longitudinal axes of the two bones and with anatomical landmark points relevant for the intervention (supplemented in FIG. 2 by line references for explanatory purposes) especially around the hip joint, the knee joint and the center of the patella, the center of the tibia and / or the center of the ankle. On the real knee joint of the patient P, the 3D surface scanner 6 locally creates a 3D surface image 8 as a partial cross-section of the knee joint, which is added to the 3D anatomical model 14 of the patient P as described above, here shown by the inserted 3D surface on the right side of FIG. 2.

[0054] The surgeon not only performs surgery based on this 3D anatomical model 14 of patient P and navigates accordingly, but when inserting the implant, he can compare the position of the target implant posture according to the intervention plan, in which the target posture is reflected in the 3D anatomical model 14 of patient P, with the actual implant posture and, if necessary, make corrections during surgery to align the actual posture with the target posture.

[0055] Fig. 3 shows a surgical navigation system 1 according to another preferred embodiment. It uses as a 3D surface scanner 6 a 3D camera with an emitter for structured light and several RGB sensors / RGB cameras. A solid body is again placed on this as a tracking device 10 in order to detect the pose via the tracking system 2. In its field of view 32 the 3D camera creates a surface image in the form of a point cloud in three-dimensional space in order to obtain geometric information, and furthermore, color information is also added according to the recorded colors in order to make the navigation more intuitive. The knee joint is scanned by the 3D surface scanner 6 and the control unit 12 is adapted to recognize the bones from the 3D surface image and to determine only these as the relevant parts of the 3D surface image and to add them to the 3D anatomical model 14 of the patient.

[0056] In this embodiment, the control unit 12 creates two views of the 3D anatomical model 14, one of the tibia and one of the femur, and displays them on the navigation screen, thus providing the surgeon with immediate and intra-operative information relevant to his intervention, allowing him to perform the surgery and to better insert the implant.

[0057] FIG. 4 is a flow chart of a navigation method for use in a surgical intervention on a patient P, in particular a tibial resection.

[0058] In a first step S1, the pose of the leg of a patient P with the knee joint where an intervention will be performed and the pose of the medical instruments used during the intervention are detected and tracked via the tracking system 2 of the navigation system 1. In particular, the poses are stored in the form of three position coordinates and three azimuth angles.

[0059] In step S2, a 3D surface image 8 of the intracavitary anatomical structure of the target patient P is created by the 3D surface scanner 6, and the orientation of the 3D surface scanner 6 at the time of the 3D surface image 8 is determined by the tracking system 2 via a tracking device 10 attached to the 3D surface scanner 6 and tracked in space.

[0060] In a following step S3, the pose of the 3D surface image 8 relative to the patient P is determined or calculated using the detected pose of the 3D surface scanner 6. In particular, the pose of the 3D surface image 8 relative to the 3D surface scanner 8 and then relative to the tracking system 2 and then relative to the patient P can first be determined. This transformation between the patient P and the 3D surface image is used by the control unit 12 in step S4 to supplement the 3D surface image 8 with a digital 3D anatomical model (14) of the patient (P) exactly in this specific pose. In other words, the virtual digital 3D model of the patient's body part is supplemented with a pose-corrected 3D surface image 8.

[0061] Finally, views of the virtually supplemented digital 3D anatomical model 14 of the patient P, e.g., front, side, oblique views, are created by the control unit 12, and this visual view of the supplemented digital 3D anatomical model 14 is then output or displayed for surgical navigation via an output device 16 in the form of an operating room monitor 34, providing an intra-operative up-to-date digital 3D anatomical model 14 augmented with the intracavitary anatomical structures of the target patient (P) for navigation. [Explanation of symbols]

[0062] 1. Surgical navigation system 2. Tracking System 4 Medical equipment 6 3D surface scanner 8. 3D Surface Imaging 10 Tracking Devices / Markers 12 Control unit 14 3D anatomical models 16 Output Devices 18 Handle 20 Wireless communication module 22 Storage device 24 Tibia Tracker 26 Femur Tracking Device 28 Axis 30 Landmark Points 32 field of view 34 Monitor 36 Light sources for structured light 38 RGB Sensor 1',2',3',4',5',6',7',8',9' Landmark points 100 Surgical Support System 102 Robot 104 Medical End Effector P patient S1 Step of determining the position of the patient's legs and the instrument S2 Creating a 3D surface image using a 3D surface scanner and detecting the orientation of the 3D surface scanner S3. Determining the transformation between the 3D surface image and the patient S4 Steps for supplementing 3D anatomical models with 3D surface images and correcting posture S5. Outputting a view of the augmented 3D anatomical model of the patient on an operating room monitor.

Claims

1. A surgical navigation system (1) for use in a surgical intervention on a patient (P), comprising: a tracking system (2) adapted to track at least a patient (P) and preferably at least one medical instrument (4); - a 3D surface scanner (6) adapted to generate a 3D surface image (8) of an intracorporeal anatomical structure of a subject patient (P), said 3D surface scanner (6) being tracked by said tracking system (2), in particular via an attached tracking device (10); A control unit (12) Equipped with The control unit (12) processing said 3D surface image (8) and the detected and tracked poses of said 3D surface scanner (6) and of the patient (P); determining a pose of said 3D surface image (8) relative to a patient (P) via the detected and tracked pose of said 3D surface scanner (6) and supplementing a virtual digital 3D anatomical model (14) of the patient (P) in a specific pose with said 3D surface image (8); Visually outputting a view of the supplemented digital 3D anatomical model (14) of the patient (P) for surgical navigation via an output device (16), in particular an operating room monitor, in order to provide an up-to-date digital 3D anatomical model (14) supplemented with intracorporeal anatomical structures of the target patient (P) for navigation. The surgical navigation system (1) is adapted for:

2. the 3D surface scanner (6) of the navigation system (1) comprises a handle (18) for manually guiding the 3D surface scanner (6) and creating the 3D surface image (8) in different poses, and preferably a wireless communication module (20); or the 3D surface scanner (6) has a mounting part, in particular for mounting on a holding arm of the navigation system (1), or 2. The surgical navigation system (1) according to claim 1, characterized in that the surgical navigation system (1) comprises a robot having a robot arm to which the 3D surface scanner (6) is connected as an end effector for automatically creating the 3D surface image (8) in at least one predefined pose of the 3D surface scanner (6), in particular for creating multiple 3D surface images (8) in multiple poses of the 3D surface scanner (6) according to a sweep algorithm or pattern.

3. 3. The surgical navigation system (1) according to claim 1 or 2, characterized in that the 3D surface scanner (6) has an emitter for structured light and an RGB sensor for detecting light and / or has a stereo camera and / or has a ToF camera and / or has a LIDAR scanner, in particular to create a 3D point cloud of the anatomical region of interest as a 3D surface image (8).

4. The surgical navigation system (1) according to any one of claims 1 to 3, characterized in that the tracking system (2) comprises an optical tracking system and / or an EM tracking system.

5. a virtual digital 3D anatomical model (14) of the leg of a patient (P) including the knee is stored in a storage device of the navigation system (1), the navigation system (1) having the 3D surface scanner (6) being adapted to generate a 3D surface image (8) of the knee joint, Said navigation system (1) for this purpose comprises a tibia tracker (24) and a separate femur tracker (26) adapted to be firmly placed on the tibia and on the femur to track the bones, 5. The surgical navigation system (1) of claim 1, wherein the control unit (12) is adapted to associate the 3D surface image (8) with the tracked leg based on the tracked tibia tracker (24) and the tracked femur tracker (26).

6. 6. The surgical navigation system (1) according to claim 1, characterized in that the control unit (12) is adapted to supplement the digital 3D anatomical model (14) of the patient (P) with landmark points (30), in particular the hip joint center, the knee joint center, the patella center, the tibia center and / or the ankle center, in particular via a pointer of the navigation system (1), in order to obtain a combined digital 3D anatomical model (14) of points (30), axes (28) and planes (8).

7. 7. The surgical navigation system (1) according to any one of claims 1 to 6, characterized in that the control unit (12) is adapted to process at least two 3D surface images (8) in at least two different orientations of the 3D surface scanner (6) in order to obtain, based on the individual 3D surface images (8), an extended and / or more precise 3D surface image (8) as a whole image.

8. 8. The surgical navigation system (1) according to any one of claims 1 to 7, characterized in that the 3D surface scanner (6) is adapted to be inserted into a body cavity of a patient (P) and has a maximum width and height, particularly preferably a maximum dimension of 10 cm.

9. 9. The surgical navigation system (1) according to any one of claims 1 to 8, characterized in that the surgical navigation system (1) comprises a surgical endoscope in which the 3D surface scanner (6) is integrated, in particular in a distal endoscope head of the surgical endoscope in which the 3D surface scanner (6) is integrated.

10. said 3D anatomical model (14) of a patient (P) with an implant according to an intervention plan is stored in a storage device (22) of said navigation system (1); 10. The surgical navigation system (1) according to any one of claims 1 to 9, characterized in that the control unit (12) is adapted to verify during surgery, via the 3D surface images (8) from the 3D surface scanner (6), the actual implant position relative to the implant position according to the intervention plan, and to output warning or deviation information via an output device (16) in case of deviations.

11. 11. The surgical navigation system (1) according to any one of claims 1 to 10, characterized in that the control unit (12) is adapted to recognize predefined structures of an object, in particular bones, in the 3D surface image (8) and to supplement the digital 3D anatomical model (14) with only the predefined structures of the object.

12. A surgical assistance system (100) comprising a navigation system (1) according to any one of claims 1 to 11 and a medical robot (102) having a medical end effector (104).

13. A navigation method for use in a surgical intervention, in particular knee arthroplasty, on a patient (P), preferably in a surgical navigation system (1) according to any one of claims 1 to 11, comprising: tracking a patient (P), and preferably at least one medical instrument, via a tracking system (2) of the navigation system (1); generating a 3D surface image (8) of the intracavitary anatomical structure of the subject patient (P) by means of a 3D surface scanner (6) and tracking said 3D surface scanner (6) by means of said tracking system (2), in particular via an attached tracking device (10); determining, by the control unit (12), a position of the 3D surface image (8) relative to the patient (P) via the detected position of the 3D surface scanner (6); Supplementing a digital 3D anatomical model (14) of a patient (P) in a specific pose with the 3D surface image (8) by the control unit (12), outputting a visual view of the supplemented digital 3D anatomical model (14) of the patient (P) for surgical navigation via an output device (16), in particular via an operating room monitor, in order to provide an intraoperative up-to-date digital 3D anatomical model (14) supplemented with intracavitary anatomical structures of the subject patient (P) for navigation; 16. A method of navigation comprising:

14. creating a 3D surface image (8) of the anterior surface of the femur in a first position of the 3D surface scanner (6), in particular via a robot-guided 3D surface scanner (6); Detecting the orientation of the 3D surface scanner (6) and a femoral tracker (26) on the femur by a tracking system (2); creating said 3D surface image (8), in particular of the anterior surface, of the tibia in a second position of the 3D surface scanner (6), in particular by robotic guidance, Detecting the orientation of the 3D surface scanner (6) and the tibial tracker (24) on the tibia by the tracking system (2); A control unit (12) supplements a digital 3D anatomical model (14) of a patient (P) with a posture-corrected 3D surface image (8).

14. The method of claim 13, further comprising:

15. A computer readable storage medium comprising instructions which, when executed by a computer, cause the computer to carry out the method steps of the navigation method according to claim 13 or 14.

Images