Computer-implemented method for augmented reality spinal rod planning and bending for navigation spine surgery

Abstract

A computer-implemented method for augmented reality spinal rod planning and bending for navigation spine surgery is disclosed. A proposed spinal rod is determined, which is a virtual model of the spinal rod having a desired shape. The proposed spinal rod is determined based on acquired positions of a plurality of spinal screws placed on the patient's spine. The spinal screws are configured to receive a helical rod that interconnects the plurality of spinal screws. Furthermore, the spinal rod itself is calibrated for tracking by a medical navigation device. This allows the proposed spinal rod to be displayed by an augmented reality device, thereby superimposing the proposed spinal rod on the tracked spinal rod. Thus, the proposed method provides the surgeon with, among other things, improved information regarding the bending state of the spinal rod.

Description

[Technical field]

[0001] FIELD OF THEINVENTION The present invention relates to a computer-implemented method, medical navigation device, and corresponding computer program for augmented reality spinal rod planning and bending for navigation spine surgery. [Background technology]

[0002] Technical background In spinal surgery, spinal rods are used as implants for stabilization surgery of the human spine. When spinal screws are inserted into the patient's spine, they are interconnected by spinal rods that run the length of the vertebrae to stabilize each side of the spine. The rods must be shaped / bent to fit over the heads of the spinal screws.

[0003] Currently, bending of the spinal rods is performed intraoperatively based on rough estimates, particularly by the surgeon's eye, and on trial and error, and therefore often time-consuming, trial and error procedures.

[0004] Alternatively, the rod bending device relies on the input of measured screw head position to suggest the rod bending that can be achieved with a physical device.

[0005] Therefore, more guidance from the surgeon is needed when planning and bending the spinal rods. Summary of the Invention [Problem to be solved by the invention]

[0006] The present invention aims to provide an improved method for augmented reality spinal rod planning and bending for navigation spine surgery. [Means for solving the problem]

[0007] The present invention may be used in spinal stabilization procedures, for example in conjunction with a system for image-guided surgery, such as the Spine & Trauma Navigation System, a product of Brainlab AG.

[0008] Aspects, examples and exemplary steps of the invention and embodiments thereof are disclosed below. The various exemplary features of the invention can be combined in accordance with the invention where technically suitable and practicable.

[0009] Illustrative Brief Description of the Invention Below, a brief description of certain features of the invention is given, which should not be understood to limit the invention to only the features or combination of features described in this section.

[0010] A computer-implemented method for augmented reality spinal rod planning and bending for navigation spine surgery is presented.

[0011] In particular, in this method, a proposed spinal rod is determined, which is a virtual model of a spinal rod having a desired shape. The proposed spinal rod is determined based on the acquired positions of a plurality of spinal screws placed on the patient's spine. The spinal screws are configured to receive a helical rod that interconnects the plurality of spinal screws. Furthermore, the spinal rod itself is calibrated for tracking by a medical navigation device. This allows the proposed spinal rod to be displayed by an augmented reality device, thereby superimposing the proposed spinal rod on the tracked spinal rod. Thus, the proposed method, among other things, provides the surgeon with improved information regarding the bending state of the spinal rod.

[0012] General description of the invention In this section, a description of general features of the invention is given, for example, by reference to possible embodiments of the invention.

[0013] This is achieved by the subject matter of the dependent claims and the independent claims, further embodiments of which are incorporated in the following description.

[0014] The described embodiments also relate to a method for augmented reality spinal rod planning and bending for navigation spine surgery, a system for spinal rod planning and bending, and corresponding computer programs. Synergistic effects may result from different combinations of the embodiments, which may not be described in detail below. Furthermore, it should be noted that all embodiments of the present invention relating to the method may be performed in the order of steps explicitly described herein. Nevertheless, this need not be the only and essential order of the steps of the method. The methods presented herein may be performed in another order of the disclosed steps without departing from the respective method embodiments, unless expressly stated below to the contrary.

[0015] Technical terms are used according to their ordinary meaning. Where a specific meaning is conveyed to a particular term, the definition of the term is given below in the context in which the term is used.

[0016] According to one aspect of the disclosure, a computer-implemented method for augmented reality spinal rod planning and bending for navigation spine surgery includes the following steps: in one step, positions of a plurality of spinal screws placed on a spine are obtained, the plurality of spinal screws being configured to receive a spinal rod interconnecting the plurality of spinal screws; in another step, a proposed spinal rod is determined, which is a virtual model of the spinal rod having a desired shape, using the obtained positions of the plurality of spinal screws; in another step, the spinal rod is calibrated for tracking the spinal rod by a medical navigation device; in another step, the proposed spinal rod is superimposed on the tracked spinal rod by displaying the proposed spinal rod by an augmented reality device.

[0017] The term "spinal rod" as used herein relates to implants used for stabilization of the human spine. Spinal rods are generally elongated cylindrical rods that are bent into a shape that allows a surgeon to attach the spinal rod to the patient's spine with the aid of spinal screws.

[0018] The term "proposed spinal rod" as used herein includes a virtual model of a spinal rod that reflects a spinal rod that is ideally already bent for a spinal procedure. In other words, the proposed spinal rod is a digital template of a spinal rod according to a spinal procedure. The proposed spinal rod is determined based on a predefined model of a spinal rod, for example, a standardized non-bent spinal rod. Alternatively, the proposed spinal rod is determined based on a spinal rod model that is a virtual model of a spinal rod that needs to be planned and bent.

[0019] As used herein, the term "spinal screw" includes any type of spinal screw, such as a pedicle screw, an external mass screw, or an SAI screw. A spinal screw is directly connected to the patient's spine, for example, by drilling a hole in the spine.

[0020] Preferably, the augmented reality device comprises augmented reality glasses, in particular comprising at least one 3D scanner.

[0021] Preferably, calibrating the spinal rod, especially by a calibration device of the medical navigation device, includes determining the position and / or shape of the spinal rod in space. Thus, the position of the spinal rod and the proposed spinal rod, especially the shape of the proposed spinal rod, are known. More preferably, displaying the proposed spinal rod by the augmented reality device includes superimposing the proposed spinal rod on the tracked spinal rod using the determined position and / or shape of the spinal rod in space. Thus, to support the surgeon in bending the spinal rod, the proposed spinal rod is displayed to the surgeon by the augmented reality device. Since the position, especially the shape, of the spinal rod in space is known due to the tracking of the spinal rod, the augmented reality device is configured to superimpose the proposed spinal rod on the tracked spinal rod. In other words, the proposed spinal rod is displayed in the field of view of the surgeon such that the surgeon always sees the proposed spinal rod in a specific spatial relationship to the spinal rod. For example, the left end of the spinal rod is always superimposed with the left end of the proposed spinal rod. The type of superimposition of the proposed spinal rod on the spinal rod is preferably dynamically adjustable. Preferably, superimposing the proposed spinal rod on the spinal rod includes displaying the proposed spinal rod in spatial relationship to the spinal rod using the determined position and / or shape of the spinal rod.

[0022] Preferably, the method comprises the step of tracking the spinal rod, in particular by means of a medical navigation device, more in particular by means of a tracking device of the medical navigation device.

[0023] The augmented reality device is preferably included in a medical navigation device. Preferably, calibrating the spinal rod includes determining a spinal rod model, which is a virtual representation of the spinal rod. In other words, during the calibration of the spinal rod, the actual shape of the spinal rod is determined. Therefore, any kind of spinal rod, non-flexed or pre-flexed, can be used for the augmented reality device to display the proposed spinal rod on the actual spinal rod in the surgeon's field of view.

[0024] Preferably, the proposed spinal rod is determined using the spinal rod model and the positions of the multiple spinal screws. In other words, the spinal rod model is digitally adjusted, or in other words, flexion is simulated, to determine the proposed spinal rod using the positions of the multiple spinal screws. Thus, the proposed spinal rod reflects a spinal rod calibrated in a flexed shape that is ideal for the spinal surgery.

[0025] Using the acquired positions of multiple spinal screws to determine the proposed spinal rod allows for user interaction, particularly the work of the surgeon, to be minimized.

[0026] Superimposing the tracked spinal rod with the proposed spinal rod on the augmented reality device allows the surgeon to bend the spinal rod right in front of his or her eyes, in other words, without having to control the progress of the bend on a separate control screen showing the proposed spinal rod. Thus, the surgeon sees the spinal rod that has to be bent through the augmented reality device and also sees the virtual proposed spinal rod displayed in the surgeon's field of view by the augmented reality device.

[0027] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0028] Preferred embodiments are described in more detail below. In a preferred embodiment, the proposed spinal rod is shaped to match the location of multiple spinal screws on the spine.

[0029] The term "shape" as used herein also refers to the size and length of the spinal rod.

[0030] If the shape of the patient's spine is to be reinforced by a spinal rod, the spinal rod must be formed into a shape that fits the patient's actual spine. The location of the spinal screws, in other words the arrangement of the spinal screws on the spine, defines the shape of the proposed spinal rod. The same applies to the length of the proposed spinal rod. The proposed spinal rod therefore represents a spinal rod that is an ideal shape and has an ideal length for the purpose of reinforcing the spine, and in particular the shape of the spine.

[0031] Preferably, the shape of the spinal rod is digitized, for example by a 3D scanner, to determine a spinal rod model. This spinal rod model is preferably used to determine the proposed spinal rod. This allows the use of any type of spinal rod of any shape or size, in particular pre-bent spinal rods.

[0032] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0033] In a preferred embodiment, the method includes determining a proposed spinal rod using the acquired positions of the plurality of spinal screws and the planned shape of the spine.

[0034] If the shape of the patient's spine needs to be adjusted, or in other words corrected, rather than just reinforced, the spinal rods must be shaped to match the shape of the patient's spine that is to be achieved by surgery. The planned shape of the spine is, in other words, the shape of the spine that is to be achieved by surgery.

[0035] In other words, the actual spinal screw positions are combined with additional planned corrections to be applied to the spine. Combining the multiple spinal screw positions with the planned shape of the spine preferably includes superimposing or adding the multiple spinal screw positions and the planned shape of the spine.

[0036] Preferably, the planned shape of the spine is determined by using a predetermined surgical plan. In particular, the planning software automatically determines proposed spinal rods using the locations of the multiple spinal screws and the planned shape of the spine.

[0037] In other words, the positions of the spinal screws are analyzed taking into account the planned shape of the spine. The positions of the spinal screws are provided to the planning software, inter alia, by digitizing the spinal screws. The planned shape of the spine is provided to the planning software, inter alia, by extraction from a predefined surgical plan. Based on this analysis, which includes inter alia a comparison between the positions of the spinal screws and the planned shape of the spine, a proposed spinal rod is automatically determined.

[0038] More preferably, the surgeon manually determines the proposed spinal rods by using the planned shape of the spine and the positions of the multiple spinal screws.

[0039] In other words, the planning software is provided with the positions of the spinal screws. The positions of the spinal screws are visualized for the surgeon by the planning software. In particular, the planning software provides a provisional proposed spinal rod based on the positions of the spinal screws. The surgeon analyzes the positions of the spinal screws taking into account the planned shape of the spine. By using the planning software, the surgeon manually determines the proposed spinal rod, in particular by adjusting the provisional proposed spinal rod provided. For example, the surgeon adds additional lordosis to the provisional proposed spinal rod or the positions of the spinal screws.

[0040] In other words, the shape of the spinal rod can be corrected further than the curvature represented by the digitized positions of the spinal screws or the proposed spinal rod generated by the planning software.

[0041] For example, the surgeon can add a few more degrees of "lordosis" in the planning software, e.g., by manipulating the proposed spinal rods or the displayed vertebrae, or can select in the software interface "Add / Remove More Lordosis to Digitized Screws."

[0042] The surgeon is then preferably guided to bend the spinal rods into this new "virtual" position represented by the proposed spinal rods.

[0043] As a result, in surgery, the desired shape, e.g. with additional lordosis, is then finally introduced into the spine by means of the bent spinal rods, in other words, the spinal rods pull the anatomical structures, in particular the vertebrae of the patient's spine, into the desired position.

[0044] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0045] In a preferred embodiment, calibrating the spinal rod includes determining a spinal rod model that is a virtual representation of the spinal rod. The method includes determining support data using the spinal rod model, the support data including information related to the spine. The method further includes overlaying the support data on the spinal rod with an augmented reality device.

[0046] The term "support data" as used herein refers to information of the spine as applied to the current spine. However, the support data preferably also refers to information of the spine as applied to the spine when spinal rods are connected with a plurality of spinal screws. In other words, the support data includes information for the surgeon or planning software regarding the current shape of the spine or regarding the planned shape of the spine, or in other words, parameters of the spine.

[0047] Preferably, the support data is used by the planning software or by the surgeon via the planning software to adjust the proposed spinal rod, in other words, the support data provides thresholds of different parameters about the patient's spine that need to be taken into account when adjusting the shape of the proposed spinal rod.

[0048] Thus, the surgeon is provided with additional information regarding planning and bending of the spinal rods by the augmented reality device.

[0049] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0050] In a preferred embodiment, determining the spinal rod model includes knowing the shape of the spinal rod relative to the tracked reference array.

[0051] In other words, the spinal rod is calibrated by detecting the shape of the spinal rod, particularly by a 3D camera or 3D laser scanner device, and detecting the tracked reference array. The detected shape of the spinal rod is used to determine a spinal rod model representing the spinal rod. By means of the similarly detected reference array, particularly with the fiducial markers, the position of the spinal rod in space is known.

[0052] Preferably, the augmented reality device is configured to obtain a surface model of the spinal rod, based on which the spinal rod model is determined. In particular, a correlated video image of the optical channel of the augmented reality device allows for a surface reconstruction of the spinal rod. Thus, the augmented reality device is configured to determine the spinal rod model.

[0053] Preferably, the spinal rod is calibrated using a calibration block or pre-calibration data that is adjusted by the detected position of the tracked reference array when the length of the spinal rod is known.

[0054] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0055] In a preferred embodiment, determining the spinal rod model includes acquiring the shape of the spinal rod by a tracking device.

[0056] Preferably, the tracking device comprises a tracked pointer, the position of which in space is known. A plurality of surface points of the spinal rod, in particular the spinal rod, are sampled by the tracking device to calibrate the spinal rod and to determine the spinal rod model. Preferably, the tracking device is slid along at least a portion of the surface of the spinal rod to sample the spinal rod. Preferably, the tracking device comprises a tracking pointer with a tip of a particular shape, for example a ring-shaped tip or a half-pipe-shaped tip.

[0057] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0058] In a preferred embodiment, the method includes dynamically adjusting the spinal rod model using the tracked spinal rods.

[0059] In other words, the shape of the spinal rod is continuously detected, for example, by a 3D camera, and the spinal rod model is adjusted using the continuously detected shape of the spinal rod. Therefore, the spinal rod model is always up-to-date compared to the spinal rod. Therefore, during bending the spinal rod, the change in the shape of the spinal rod due to bending is directly reflected by the spinal rod model. In other words, the shape of the spinal rod model is congruent with the shape of the spinal rod.

[0060] Preferably, the spinal rod model is adjusted in real time. Thus, the support data determined using the spinal rod model also reflects changes in the shape of the spinal rod, for example, when the support data includes different bend indicators that indicate spots where the spinal rod should be bent, any bend indicators that have already been recognized by bending the spinal rod are discarded and no longer displayed.

[0061] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0062] In a preferred embodiment, the supporting data includes at least one bend indicator determined by using the proposed spinal rod and spinal rod model.

[0063] Preferably, the bend indicator indicates a spot on the spinal rod to which the spinal rod should be bent by the surgeon. Thus, the surgeon is guided to bend the spinal rod to arrive at the proposed spinal rod in an improved manner. The bend indicator is preferably superimposed directly on the spinal rod. For example, the bend indicator comprises a marker such as a dot or a vertical line.

[0064] Preferably, the at least one bend indicator includes a bend sequence, e.g., the at least one bend indicator is numbered to indicate to the surgeon in what sequence the spinal rod should be bent to ideally arrive at the proposed spinal rod.

[0065] Thus, the surgeon is provided with improved guidance within the surgeon's field of view while bending the spinal rod.

[0066] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0067] In a preferred embodiment, the method includes the steps of: determining a spine model, which is a virtual representation of the spine, and adjusting the spine model using the spine rod model. The support data includes spine indicators, which are determined by using the spine model and indicate the vertebrae on the spine rods.

[0068] Preferably, the spine model is determined using specifically pre-determined patient-specific spine data. For example, the patient-specific spine data is determined by using image segmentation techniques, such as atlases and / or artificial intelligence methods, to detect vertebrae in available image data of the patient's spine. The image data may be pre-operative or intra-operative image data. The image data preferably comprises a 3D dataset, such as a CT dataset or an MRT dataset. Alternatively, the image data comprises a 2D or 3D X-ray image, allowing an approximate reconstruction of the spine shape. The image data is preferably model-enhanced, in particular comprising a morphing of the model to the detected contours in the X-ray image.

[0069] The image data may, for example, include only one X-ray image along with a segmentation technique as long as the alignment of the spine model is visible from the angle at which the X-ray shows the spine.

[0070] Thus, the spinal indicator shows the surgeon in this field of view how adjustment of the spinal rods will affect the spinal deformity.

[0071] Additionally, the spine model is displayed to the surgeon at a different angle than the proposed spinal rod model. Often, the spine model is only displayed to the surgeon from the top of the spine, and adjustments to the spine model are largely invisible to the surgeon. Therefore, the spine model is also displayed from a lateral angle of the spine that does not specifically overlap the spinal rods, but is still within the surgeon's field of view. For example, the spine model is displayed in the corner of the surgeon's field of view.

[0072] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0073] In a preferred embodiment, the method includes determining a spinal screw model that is a virtual representation of a plurality of spinal screws disposed on a spine. The support data is determined using positions of the plurality of spinal screws and includes at least one screw indicator indicative of the plurality of spinal screws on a spinal rod.

[0074] In other words, the at least one screw indicator represents a digitized model of the spinal screw specifically at a determined position of the spinal screw.

[0075] Preferably, at least one screw indicator is displayed on the proposed spinal rod.

[0076] When at least one screw indicator is displayed on a proposed spinal rod, the at least one screw indicator represents a number of spinal screws placed on the spine as actually positioned, or represents a number of spinal screws placed on the spine as planned to be positioned by spinal adjustment.

[0077] In other words, initially, the at least one screw indicator is determined by using the positions of the multiple spinal screws, and thus reflects the reality of the shape and position of the spinal screws on the spine. However, when the surgeon adjusts the proposed spinal rod, particularly by determining the planned shape of the spine, the position of the at least one spinal screw indicator is also adjusted accordingly.

[0078] Thus, the at least one screw indicator dynamically indicates the shape and position of the plurality of spinal screws along the proposed spinal rod. Preferably, the surgeon is constantly provided with information regarding the shape and position of the plurality of spinal screws as they are positioned on the planned shape of the spine.

[0079] Thus, the surgeon is provided with constant feedback as to how the planned adjustment of the spine will affect the placement of the spinal screws, particularly as indicated by the spinal indicators. Additionally, the spinal screw model is preferably used by the planning software when determining the proposed spinal rods.

[0080] Furthermore, when planning spinal rods, and in particular when determining proposed spinal rods, the surgeon or planning software virtually adjusts the position of the spinal screws with respect to the spine, especially independent of the particular planned alignment, which allows, for example, to increase biomechanical strength or minimize skin incision size.

[0081] During spinal rod planning and bending, this allows visualization of the relative spinal screw positions and the effect of the current bending on the patient's anatomy, enhancing the spinal rod planning and bending process.

[0082] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0083] In a preferred embodiment, the method includes determining forces applied to the plurality of spinal screws by using a spinal model and a spinal rod model, and the support data includes force indicators determined by using the determined forces and indicative of forces that would be applied to the plurality of spinal screws if the spinal rods were connected to the spinal screws.

[0084] Preferably, the force indicator comprises a vector that indicates the amount of force applied to the vertebrae and / or spinal screws.

[0085] Preferably, the forces are determined using finite element methods, FEM methods, based on a biomechanical model, taking into account the material properties of the spinal rods and spinal screws.

[0086] Thus, the surgeon is provided with constant feedback on how the planned adjustment of the spine will affect the force applied to the spinal screws when the spinal rods are connected to the spinal screws. In other words, a particular planned adjustment of the spine may appear ideal, but may introduce a relatively large amount of tension or stress to the spine or one or more spinal screws. The force indicator guides the surgeon not to select a spinal adjustment that introduces an unreasonable amount of force to the spine or one or more spinal screws when connecting the spinal rods to the spinal screws. Furthermore, the determined force is used by the planning software to determine the proposed final rod, particularly by comparing the determined force to a predefined threshold.

[0087] This makes it possible to prevent loosening of the spinal screws in the spine due to spinal rod induced forces.

[0088] This also ensures mechanical stability of the rod construct, i.e. the spinal rod connected to the spine by the spinal screw.

[0089] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0090] In a preferred embodiment, the method includes determining a force warning if the determined force exceeds a predetermined threshold. The supporting data includes a force warning indicator determined by using the determined force warning.

[0091] Preferably, the force warning indicator includes a color code. To further guide the surgeon in the process of planning and bending the spinal rods, the determined force is automatically compared to a predetermined threshold and a force warning indicator is displayed to the surgeon within the field of view to alert the surgeon of an excessive amount of force that will be applied to the spine or one or more spinal screws when connecting the spinal rods together with the proposed spinal rods with multiple spinal screws.

[0092] In other words, the surgeon is actively alerted if the planned adjustment of the spine introduced by the planned bending of the spinal rods results in an undesirable amount of tension in the spine or tension between the spine and the multiple spinal screws.

[0093] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0094] In a preferred embodiment, the method includes determining at least one anatomical parameter of the spine by using a spinal model and a spinal rod model, and the supporting data includes at least one anatomical parameter indicator determined by using the at least one determined anatomical parameter.

[0095] Preferably, the anatomical parameters include intervertebral angles, in particular the Cobb angle, lordosis, kyphosis, scoliosis for sagittal and coronal balance, and intervertebral distance or distance of anterior vertebral displacement. Preferably, the availability of the anatomical parameters depends on available information such as the number and position of the imaged vertebrae.

[0096] Thus, the surgeon is provided with additional information which is displayed directly in the surgeon's field of view: in case of spinal adjustments, at least one parameter of the spine is also displayed for the proposed spinal rod.

[0097] This allows prediction of the outcome of anatomical parameters during spinal rod planning, in other words during the determination of the proposed spinal rod.

[0098] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0099] In a preferred embodiment, the method includes determining a mean deviation between the spinal rod model and the proposed spinal rod by using the spinal rod model and the proposed spinal rod, and the supporting data includes a deviation indicator determined by using the determined mean deviation.

[0100] In other words, the deviation indicator allows the surgeon to assess how accurately the bending of the spinal rod was performed and whether the surgeon should continue bending or has finished.

[0101] Thus, the surgeon is further guided in his pursuit to bend the spinal rod.

[0102] Thus, an improved method for augmented reality spinal rod planning and bending for navigation spine surgery is provided.

[0103] In a preferred embodiment, calibrating the spinal rod includes providing a reference device on the spinal rod that defines the origin of the spinal rod coordinate system and determining a spinal rod-to-cam coordinate transformation that describes the transformation between the spinal rod coordinate system and the camera coordinate system.

[0104] Preferably, the reference device is a reference star. In other words, the origin of the spinal rod coordinate system is defined by the location of the reference device on the spinal rod.

[0105] In a preferred embodiment, obtaining the positions of the plurality of spinal screws includes recognizing the plurality of spinal screws with an augmented reality device.

[0106] Preferably, the locations of the multiple spinal screws are scanned by a 3D scanner integrated into the augmented reality device or by image processing from a video recorded by the augmented reality device.

[0107] Further preferably, correlated images of at least one video camera or 3D depth camera of the augmented reality device are used for surface reconstruction of the spinal screw, in particular the screw head, and matched with a generic model or a manufacturer specific model from a database.

[0108] For example, an augmented reality device includes a single RGB stereo camera and a time-of-flight camera that are used to acquire the positions of multiple spinal screws.

[0109] Thus, the positions of a number of spinal screws are obtained by the augmented reality device. In a preferred embodiment, obtaining the positions of the plurality of spinal screws includes extracting the positions of the plurality of spinal screws from a planning application.

[0110] Preferably, the planning application includes information about the shape of the spine and the spinal screws already placed on the spine, in particular as indicated by the preoperative image data. In other words, the spinal screws planned with the preoperative image data are transferred after registering these data to the patient coordinate system. The position and the axial direction of the spinal screws, in particular the spinal screw head, are pre-determined for uniaxial spinal screws. In case of multiaxial spinal screws, the best fit can be modeled.

[0111] This allows the positions of multiple spinal screws to be obtained automatically from the outside. In a preferred embodiment, obtaining the positions of the plurality of spinal screws includes detecting the positions of the plurality of spinal screws within the intraoperative image data.

[0112] Preferably, metal artifacts detected in the paired registered 2D images or the single registered 3D scan are matched to generic or manufacturer specific models from a database of spinal screws. Once the image data is registered, the 3D positions of the identified spinal screws are known in the patient coordinate system.

[0113] In a preferred embodiment, obtaining the positions of the plurality of spinal screws includes calibrating each of the plurality of spinal screws by using a tracked pointer.

[0114] For example, the tip of a tracked pointer touches or pivots to the center of the spinal screw head to obtain the position of the spinal screw.

[0115] According to another aspect of the present invention, a medical navigation device is configured to perform the methods described herein.

[0116] Preferably, the medical navigation device comprises an augmented reality device and a control unit. The augmented reality device is configured to obtain positions of a plurality of spinal screws disposed on a spine, the plurality of spinal screws being configured to receive a spinal rod interconnecting the plurality of spinal screws. Further, the augmented reality device is configured to calibrate the spinal rod for tracking the spinal rod by the medical navigation device. Further, the augmented reality device is configured to display a proposed spinal rod, thereby superimposing the proposed spinal rod on the tracked spinal rod.

[0117] The control unit is configured to use the obtained positions of the plurality of spinal screws to determine a proposed spinal rod, which is a virtual model of the spinal rod.

[0118] According to another aspect of the invention there is provided a computer program which, when executed on or loaded onto a computer, causes the computer to perform the method steps of the methods described herein, and / or a program storage medium on which the program is stored, and / or a computer comprising at least one processor and a memory and / or a program storage medium, the computer being run on the computer or loaded into the memory of the computer, and / or a data stream representing the program.

[0119] For example, it should be noted that the present invention does not involve, or specifically does not include, any invasive steps, which represent substantial physical interference with the body that requires professional medical expertise to be performed and which entail substantial health risks even when performed with the necessary professional care and expertise. For example, the present invention does not include steps of positioning a medical implant for fixing the medical implant to an anatomical structure, or steps of fixing a medical implant to an anatomical structure, or steps of preparing an anatomical structure for fixing a medical implant thereto. More specifically, the present invention does not involve, or specifically does not include, any surgical or therapeutic activity. Instead, the present invention is directed to be applicable to planning and bending spinal rods outside the patient's body. For this reason alone, there is no surgical or therapeutic activity, and in particular, no surgical or therapeutic steps are required or implied by practicing the present invention.

[0120] The method may be performed intraoperatively, however, the steps of the method do not involve surgical or therapeutic activity.

[0121] definition This section provides definitions of certain terms used in this disclosure, and also forms part of this disclosure.

[0122] Computer Implementation Method The method according to the invention is, for example, a computer-implemented method. For example, it includes that all steps of the method according to the invention, or just a part of the steps (i.e., less than the total number of steps), may be performed by a computer (e.g., at least one computer). One embodiment of the computer-implemented method is the use of a computer to perform a data processing method. One embodiment of the computer-implemented method is a method relating to the operation of a computer, such that the computer is operative to perform one, several, or all steps of the method.

[0123] A computer comprises, for example, at least one processor and, for example, at least one memory, for (technically) processing data, for example electronically and / or optically. The processor is, for example, made of a substance or composition that is, for example, a semiconductor, for example at least partially, n-type and / or p-type doped semiconductor, for example at least one of type II, III, IV, V, VI semiconductor materials, for example (doped) silicon and / or gallium arsenide. The described calculation or determination step is, for example, performed by a computer. The determining or calculating step is, for example, a step of determining data within the framework of a technical method, for example within the framework of a program. The computer is, for example, any kind of data processing device, for example an electronic data processing device. The computer can be a device generally considered as such, for example a desktop PC, a notebook, a netbook, etc., but can also be, for example, any programmable device, such as a mobile phone or an embedded processor. The computer can, for example, include a system (network) of "sub-computers", each sub-computer representing a computer in itself. The term "computer" includes cloud computers, for example cloud servers. The term "cloud computer" includes cloud computer systems, including at least one system of cloud computers and a plurality of operatively interconnected cloud computers, such as server farms. Such cloud computers are preferably connected to a wide area network, such as the World Wide Web (WWW), and are located within a so-called cloud of computers, all connected to the World Wide Web. Such an infrastructure is used in "cloud computing" to describe computation, software, data access and storage services without the end user needing to know the physical location and / or configuration of the computers delivering a particular service. For example, the term "cloud" is used in this respect as a metaphor for the Internet (World Wide Web).For example, the cloud provides a computing infrastructure as a service (IaaS). The cloud computer can act as a virtual host for the operating system and / or data processing application used to execute the method of the invention. The cloud computer is for example the elastic compute cloud (EC2) provided by Amazon Web Services™. The computer is for example equipped with an interface for receiving or outputting data and / or performing analog-to-digital conversion. The data is for example data representative of physical properties and / or data generated from technical signals. The technical signals are for example generated by (technical) detection devices (such as for example devices for detecting marker devices) and / or (technical) analysis devices (such as for example devices for performing (medical) imaging methods), the technical signals being for example electrical or optical signals. The technical signals represent for example data received or output by the computer. The computer is preferably operatively coupled to a display device that allows the information output by the computer to be displayed, for example to a user. An example of a display device is a virtual reality device or an augmented reality device (also called virtual reality glasses or augmented reality glasses) that can be used as "goggles" for navigation. An example of such augmented reality glasses is Google Glass (a trademark of Google, Inc.). Augmented reality or virtual reality devices can be used both to input information into a computer by user interaction and to display information output by the computer. Another example of a display device is a standard computer monitor that includes, for example, a liquid crystal display operatively coupled to a computer to receive display control data from the computer to generate signals used to display image information content on the display device. A particular embodiment of such a computer monitor is a digital light box. An example of such a digital light box is Buzz®, a product of Brainlab AG.The monitor may also be the monitor of a portable, e.g. handheld, device such as a smart phone or a personal digital assistant or a digital media player.

[0124] The invention also comprises a program, which when executed on a computer causes the computer to perform one or more or all of the method steps described herein, and / or a program storage medium (particularly in non-transitory form) on which the program is stored, and / or a computer comprising said program storage medium, and / or a (physical, e.g. electrical, e.g. technologically generated) signal wave, e.g. a digital signal wave, carrying information representing a program, e.g. code means adapted to perform any or all of the method steps of the aforementioned program, e.g. as described herein.

[0125] Within the framework of the present invention, computer program elements may be embodied by hardware and / or software (which includes firmware, resident software, microcode, etc.). Within the framework of the present invention, computer program elements may take the form of a computer program product which may be embodied by a computer usable, e.g. computer readable data storage medium, such as a computer readable data storage medium comprising computer readable program instructions, e.g. a "code" or "computer program" embodied in said data storage medium for use on or in connection with an instruction execution system. Such a system may be a computer, which may be a data processing device comprising means for executing computer program elements and / or programs according to the present invention, e.g. a digital processor (Central Processing Unit or CPU) for executing the computer program elements, and optionally a volatile memory (e.g. Random Access Memory or RAM) for storing data used in and / or generated by the execution of the computer program elements. Within the framework of the present invention, a computer usable, e.g., computer readable, data storage medium can be any data storage medium that can contain, store, communicate, propagate, or transport a program for use on or in connection with an instruction execution system, apparatus, or device. A computer usable, e.g., computer readable, data storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a propagation medium, such as the Internet. A computer usable or computer readable data storage medium can be, for example, paper or other suitable medium on which a program is printed, since a program can be electronically captured, for example, by optically scanning the paper or other suitable medium, and then compiled, interpreted, or otherwise processed in a suitable manner.The data storage medium is preferably a non-volatile data storage medium. The computer program product and any software and / or hardware described herein form various means for carrying out the functions of the present invention in the exemplary embodiment. The computer and / or data processing device may include, for example, a guidance information device including a means for outputting guidance information. The guidance information may be output to the user, for example, visually by visual indication means (e.g., a monitor and / or a lamp), and / or acoustically by acoustic indication means (e.g., a speaker and / or a digital audio output device), and / or tactilely by tactile indication means (e.g., a vibration element or vibration elements integrated in the device). For the purposes of this document, a computer is, for example, a technical computer including technical, e.g. tangible components, e.g. mechanical and / or electronic components. Any device so referred to in this specification is a technical, e.g. tangible device.

[0126] Get the data The expression "obtaining data" encompasses, for example, scenarios in which data are determined by a computer-implemented method or program (within the framework of a computer-implemented method). Determining data includes, for example, measuring physical quantities and converting measurements into data, for example digital data, and / or calculating (and, for example, outputting) data by a computer, for example within the framework of a method according to the invention. The meaning of "obtaining data" also encompasses scenarios in which data are received or obtained by a computer-implemented method or program (for example by input to a computer-implemented method or program), for example from another program, a previous method step, or a data storage medium for further processing, for example by a computer-implemented method or program. The generation of the data to be obtained may be, but need not be, part of the method according to the invention. Thus, the expression "obtaining data" can also mean, for example, waiting for the reception of data and / or receiving data. The received data can be, for example, input via an interface. The expression "obtaining data" can also mean that the computer-implemented method or program executes a step to (actively) receive or retrieve data from a data source, such as a data storage medium (e.g., ROM, RAM, database, hard drive, etc.) or via an interface (e.g., from another computer or network). The data obtained by the disclosed method or device, respectively, can be obtained from a database, for example, from a database to a computer, located on a data storage device operable to the computer for data transfer between the database and the computer. The computer obtains the data and uses it as input for a step of determining the data. The determined data can be output again to the same or another database, where it is stored for later use.The database or the database used to implement the disclosed method can be located on a network data storage device or network server (e.g., a cloud data storage device or cloud server) or on a local data storage device (such as a mass storage device operatively connected to at least one computer that executes the disclosed method). The data can be made "ready for use" by performing an additional step prior to the acquisition step. According to this additional step, the data is generated in order to be acquired. The data is, for example, detected or captured (e.g., by an analysis device). Alternatively or additionally, the data is, according to the additional step, entered, for example, via an interface. The generated data can be, for example, entered (e.g., into a computer). According to an additional step (preceding the acquisition step), the data can also be provided by performing an additional step of storing the data in a data storage medium (e.g., ROM, RAM, CD and / or hard drive, etc.), whereby it is ready for use within the framework of the method or program according to the invention. Thus, the step of "acquiring data" can also include instructing the device to acquire and / or provide the data to be acquired. In particular, the acquiring step does not involve an invasive step that represents a substantial physical intervention with the body that requires professional medical expertise to be performed and that entails substantial health risks even if performed with the necessary professional care and expertise. In particular, the acquiring data step, e.g., determining data step, does not involve a surgical step, and in particular does not involve a step of treating the human or animal body using surgery or treatment. In order to distinguish between the different data used by the method, the data are indicated (i.e., referred to) as "XY data" or the like, and are defined in terms of the information they describe, which is thereafter preferably referred to as "XY information" or the like.

[0127] Alignment The n-dimensional image of the body is registered when the spatial location of each point of a real object in space, e.g. a body part in an operating room, is assigned image data points of images (CT, MR, etc.) stored in the navigation system.

[0128] Image Registration Image registration is the process of transforming different data sets into one coordinate system. The data can be multiple photographs and / or data from different sensors, different times or different viewpoints. It is used in computer vision, medical imaging, and in compiling and analyzing images and data from satellites. Registration is necessary to be able to compare or integrate the data acquired from these different measurements.

[0129] Marker This is the function of the marker to be detected by a marker detection device (e.g. a camera or an ultrasound receiver, or an analysis device such as a CT or MRI device) so that the spatial position of the marker (i.e. its spatial location and / or alignment) can be ascertained. The detection device is for example part of a navigation system. The marker may be an active marker. An active marker may emit electromagnetic radiation and / or waves, which may for example be in the infrared, visible and / or ultraviolet spectral range. However, the marker may also be passive, i.e. for example, it may reflect electromagnetic radiation in the infrared, visible and / or ultraviolet spectral range, or it may block X-ray radiation. For this purpose, the marker may be equipped with a surface with corresponding reflection properties, or it may be made of metal to block X-ray radiation. It is also possible for the marker to reflect and / or emit electromagnetic radiation and / or waves in the radio frequency range or in ultrasound wavelengths. The marker preferably has a spherical and / or spheroidal shape and can therefore be called a marker sphere. However, the marker may also exhibit an angular, for example cubic, shape.

[0130] Marker Device The marker device may be, for example, a reference star or a pointer or a single marker or multiple (individual) markers, preferably in a predefined spatial relationship. The marker device may comprise one, two, three or more markers, two or more such markers being in a predefined spatial relationship. This predefined spatial relationship is, for example, known to the navigation system and, for example, stored in a computer of the navigation system.

[0131] In other embodiments, the marker device comprises an optical pattern, for example on a two-dimensional surface. The optical pattern can comprise a number of geometric shapes, such as a circle, a rectangle, and / or a triangle. The optical pattern can be identified in an image captured by a camera, and the position of the marker device relative to the camera can be determined from the size of the pattern in the image, the orientation of the pattern in the image, and the distortion of the pattern in the image. This allows the relative position of up to three rotational dimensions and up to three translational dimensions to be determined from a single two-dimensional image.

[0132] The position of the marker device can be ascertained, for example, by a medical navigation system. If the marker device is attached to an object, such as a bone or a medical instrument, the position of the object can be determined from the position of the marker device and the relative position between the marker device and the object. Determining this relative position is also called registering the marker device with the object. The marker device or object can be tracked, meaning that the position of the marker device or object is ascertained more than once over time.

[0133] Marker Holder A marker holder is understood to mean an attachment device for an individual marker, which serves to attach the marker to an instrument, a part of the body and / or a holding element of a reference star, which can be fixedly attached or, advantageously, detachably attached. The marker holder may, for example, be rod-shaped and / or cylindrical. A fastening device (e.g., a latch mechanism, etc.) for the marker device can be provided at the end of the marker holder facing the marker, which helps to place the marker device on the marker holder with a force fit and / or a positive fit.

[0134] Pointer The pointer is a rod with one or more, advantageously two, markers fixed to it and that can be used to measure individual coordinates, e.g. spatial coordinates (i.e. three-dimensional coordinates), of a part of the body, and the user guides the pointer (e.g. a part of the pointer having a defined and advantageously fixed position relative to at least one marker attached to the pointer) to a position corresponding to the coordinates, such that the position of the pointer can be determined by detecting the markers on the pointer using the surgical navigation system. The relative position between the markers of the pointer and the part of the pointer used to measure the coordinates (e.g. the tip of the pointer) is for example known. The surgical navigation system then allows the position (in three-dimensional coordinates) to be assigned to a given body structure, which can be done automatically or by user intervention.

[0135] Standard Star A "reference star" refers to a device having a number of markers attached to it, advantageously three markers, which are (e.g. removably) attached to the reference star so that they are stationary, thus providing a known (advantageously fixed) position of the markers relative to each other. The positions of the markers relative to each other can be individually different for each reference star used within the framework of the surgical navigation method, in order to enable the surgical navigation system to identify the corresponding reference star based on the positions of the markers relative to each other. It is therefore also possible to identify and / or distinguish the object (e.g. instrument and / or body part) to which the reference star is attached accordingly. In the surgical navigation method, the reference star serves to attach a number of markers to an object (e.g. bone or medical instrument) in order to enable the position of the object (i.e. its spatial position and / or alignment) to be detected. Such a reference star may be characterized, for example, by a method of attachment to the object (e.g., a clamp and / or a screw thread), and / or a retaining element that ensures the distance between the marker and the object (e.g., to aid in the visibility of the marker to a marker detection device), and / or a marker holder that is mechanically connected to the retaining element and to which the marker can be attached.

[0136] Navigation system The present invention also relates to a computer-aided surgical navigation system. The navigation system preferably comprises a computer as described above for processing data provided according to the computer-implemented method according to any one of the embodiments described herein. The navigation system preferably comprises a detection device for detecting the positions of detection points representing the principal and auxiliary points, in order to generate detection signals and provide the generated detection signals to the computer so that the computer can determine absolute principal and auxiliary point data based on the received detection signals. The detection points are points on the surface of the anatomical structure detected, for example, by a pointer. In this way, absolute point data can be provided to the computer. The navigation system also preferably comprises a user interface for receiving the calculation results (e.g., the position of the principal plane, the position of the auxiliary plane, and / or the position of the standard plane) from the computer. The user interface provides the received data as information to the user. The user interface may, for example, be a display device such as a monitor or a speaker. The user interface may use any kind of instruction signal (e.g., a visual signal, an audio signal, and / or a vibration signal). One example of a display device is a virtual reality device or an augmented reality device (also called virtual reality glasses or augmented reality glasses) that can be used as "goggles" for navigation. An example of such augmented reality glasses is Google Glass (a trademark of Google, Inc.) Augmented reality devices can be used both to input information into a navigation system's computer through user interaction and to display information output by the computer.

[0137] The present invention also relates to a navigation system for computer-assisted surgery, a computer for processing the absolute point data and the relative point data; a detection device for detecting the positions of the principal and auxiliary points so as to generate and supply absolute point data to a computer; a data interface for receiving the relative point data and providing the relative point data to a computer; A user interface for receiving data from a computer to provide information to a user, the received data being generated by the computer based on results of processing performed by the computer; Equipped with.

[0138] Surgical Navigation Systems A navigation system, such as a surgical navigation system, is understood to mean a system that may comprise at least one marker device, a transmitter for emitting electromagnetic waves and / or radiation and / or ultrasound, a receiver for receiving electromagnetic waves and / or radiation and / or ultrasound, and an electronic data processing device connected to the receiver and / or the transmitter, the data processing device (e.g. a computer) comprising, for example, a processor (CPU), a working memory, advantageously an indication device for sending indication signals (e.g. a visual display device such as a monitor, and / or an audio display device such as a speaker, and / or a tactile display device such as a vibrator), and a permanent data memory, the data processing device being able to process the navigation data transferred by the receiver and advantageously output guidance information to the user via the indication device. The navigation data may be stored in the permanent data memory and, for example, compared with data previously stored in said memory.

[0139] Shape representation A shape representation represents characteristic aspects of the shape of an anatomical structure. Examples of shape representations include lines, planes, geometric figures, etc. A geometric figure may be one-dimensional, e.g., an axis or an arc, two-dimensional, e.g., a polygon and a circle, or three-dimensional, e.g., a cube, a cylinder, and a sphere. The relative position between shape representations can be described in a reference system, e.g., by coordinates or vectors, or by geometric variables, e.g., length, angle, area, volume, and proportion. A characteristic aspect represented by a shape representation is a symmetry property, e.g., represented by a symmetry plane. Another example of a characteristic aspect is the extension direction of an anatomical structure, e.g., represented by a longitudinal axis. Another example of a characteristic aspect is the cross-sectional shape of an anatomical structure, e.g., represented by an ellipse. Another example of a characteristic aspect is the surface shape of a part of an anatomical structure, e.g., represented by a plane or a hemisphere. For example, a characteristic aspect constitutes an abstraction of a real shape, or an abstraction of a property of a real shape, e.g., its symmetry or longitudinal extension. For example, a shape representation represents this abstraction.

[0140] reference Determining a position is called referencing if it means informing the navigation system of said position in its reference system.

[0141] Atlas / Atlas Segmentation Preferably, atlas data is obtained that describes (e.g. defines, more specifically represents and / or is) a general three-dimensional shape of an anatomical body part. The atlas data thus represents an atlas of an anatomical body part. An atlas typically consists of a number of general models of an object, which together form a complex structure. For example, an atlas constitutes a statistical model of a patient's body (e.g. a part of a body) generated from anatomical information collected from a number of human bodies, e.g. medical imaging data including images of such human bodies. Thus, in principle, the atlas data represents the result of a statistical analysis of such medical imaging data for a number of human bodies. This result can be output as an image, and thus the atlas data includes or is comparable to the medical imaging data. Such a comparison can be performed, for example, by applying an image fusion algorithm that performs image fusion between the atlas data and the medical imaging data. The result of the comparison can be a measure of similarity between the atlas data and the medical imaging data. The atlas data includes image information (e.g., positional image information) that can be matched (e.g., by applying an elastic or rigid image fusion algorithm) to image information (e.g., positional image information) contained in the medical image data, e.g., to compare the atlas data with the medical image data to determine the positions of anatomical structures in the medical image data that correspond to anatomical structures defined by the atlas data.

[0142] The human bodies, whose anatomical structures serve as input for generating the atlas data, advantageously share common characteristics such as at least one of gender, age, ethnicity, body measurements (e.g., size and / or mass), and pathological conditions. The anatomical information describes, for example, the anatomical structures of the human body and is extracted, for example, from medical image information relating to the human body. For example, an atlas of the femur may include as objects the head, neck, body, greater trochanter, lesser trochanter, and lower limb, which together constitute a complete structure. An atlas of the brain may include as objects the telencephalon, cerebellum, diencephalon, pons, midbrain, and medulla oblongata, which together constitute a complex structure. One application of such an atlas is medical image segmentation, where the atlas is matched with medical image data, and the image data is compared to the matched atlas in order to assign points (pixels or voxels) of the image data to objects in the matched atlas, thereby segmenting the image data into objects.

[0143] Imaging methods In the field of medicine, imaging methods (also called imaging modalities and / or medical imaging modalities) are used to generate image data (e.g. two-dimensional or three-dimensional image data) of anatomical structures of the human body (e.g. soft tissues, bones, organs, etc.). The term "medical imaging method" is understood to mean (advantageously device-based) imaging methods (e.g. so-called medical imaging modalities and / or radiological imaging methods), such as, for example, computed tomography (CT) and cone-beam computed tomography (CBCT, e.g. volumetric CBCT), X-ray tomography, magnetic resonance tomography (MRT or MRI), conventional X-ray examination, sonography and / or ultrasound examination, and positron emission tomography. For example, medical imaging methods are performed by an analysis device. Examples of medical imaging modalities applied by medical imaging methods, as mentioned by Wikipedia, are X-ray radiography, magnetic resonance imaging, medical sonography or ultrasound, endoscopy, elastography, tactile imaging, thermography, medical photography, and nuclear medicine functional imaging techniques such as positron emission tomography (PET) and single photon emission computed tomography (SPECT).

[0144] The image data thus generated is also referred to as "medical imaging data". The analysis device is used to generate the image data, for example, in an apparatus-based imaging method. The imaging method is used in medical diagnosis, for example, to analyze the anatomical body to generate an image described by the image data. The imaging method is also used, for example, to detect pathological changes in the human body. However, some changes in the anatomical structure, such as pathological changes in the structure (tissue), may not be detectable, for example, may not be visible in the image generated by the imaging method. A tumor represents an example of a change in the anatomical structure. If the tumor grows, it is said to represent an extended anatomical structure. This extended anatomical structure may not be detectable, for example, only a part of the extended anatomical structure may be detectable. Primary / high-grade brain tumors are usually visible in an MRI scan, for example, if a contrast agent is used to infiltrate the tumor. An MRI scan represents an example of an imaging method. In the case of an MRI scan of such a brain tumor, signal enhancement in the MRI image (due to the contrast agent infiltrating the tumor) is considered to represent a solid tumor mass. Thus, the tumor is detectable, for example, identifiable in the image generated by the imaging method. In addition to these tumors, called "enhancing" tumors, approximately 10% of brain tumors are not identifiable on a scan, i.e., are thought to be invisible to a user looking at the image produced by the imaging method.

[0145] mapping The mapping describes a transformation (e.g., a linear transformation) of elements (e.g., pixels or voxels), e.g., positions of elements, of a first data set in a first coordinate system to elements (e.g., pixels or voxels), e.g., positions of elements, of a second data set in a second coordinate system (which may have a basis different from that of the first coordinate system). In one embodiment, the mapping is determined by comparing (e.g., matching) color values ​​(e.g., gray values) of the respective elements by an elastic or rigid fusion algorithm. The mapping is, for example, embodied by a transformation matrix (such as a matrix defining an affine transformation).

[0146] BRIEF DESCRIPTION OF THE DRAWINGS The present invention will now be described with reference to the accompanying drawings, which provide a background description and depict certain embodiments of the invention, however the scope of the invention is not limited to the specific features disclosed in the context of the drawings. [Brief description of the drawings]

[0147] [Figure 1] FIG. 1 illustrates a medical navigation device used by a surgeon to plan and bend spinal rods. [Figure 2a] FIG. 13 is a schematic diagram through an augmented reality device displaying a non-bent spinal rod superimposed on a proposed spinal rod. [Figure 2b] FIG. 13 is a schematic view through an augmented reality device displaying a partially bent spinal rod superimposed on a proposed spinal rod. [Diagram 3] FIG. 1 is a schematic diagram of tracking a spinal rod by a medical navigation device. [Figure 4] 1 is a schematic diagram of a medical navigation device. [Figure 5a] FIG. 13 is a schematic diagram through an augmented reality device displaying a non-bent spinal rod superimposed on a proposed spinal rod and spinal screw indicator. [Figure 5b]FIG. 13 is a schematic diagram through an augmented reality device displaying a non-bent spinal rod superimposed on the proposed spinal rod and the bend indicator. [Figure 6] FIG. 1 is a schematic diagram of a patient's spine with spinal screws connected by spinal rods. [Figure 7] FIG. 1 is a schematic diagram of a computer-implemented method for augmented reality spinal rod planning and bending for navigation spine surgery. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0148] Description of the embodiments 1 shows a medical navigation device used by a surgeon 60 for planning and bending a spinal rod 10. The spinal rod 10 is to be used in a spinal surgery in which the spine of a patient 70 is adjusted and / or reinforced by the spinal rod 10. For this purpose, a spine 40 is provided with a number of spinal screws. In the spinal surgery, the spinal rod 10 is connected and attached to the spine 40 by the spinal screws 30. Thus, the patient's spine 40 is reinforced or an adjustment to the spine 40 is applied by the spinal rod 10.

[0149] However, before the spinal rod 10 is attached to the patient's spine 40, it must be shaped accordingly, in particular by bending the spinal rod 10 into the desired shape that will achieve the stiffening and / or adjusting effect of the surgery. The bending itself is generally performed by a bending device, which is usually manually operated by the surgeon 60.

[0150] Typically, a proposed spinal rod 20, in other words a virtual model of the spinal rod 10 reflecting the desired shape of the spinal rod, is displayed to the surgeon 60 on a separate screen. The surgeon then attempts to bend the spinal rod 10 into the desired shape according to the representation of the proposed spinal rod 20.

[0151] In the illustrated case, a surgeon 60 uses a medical navigation device 50 that is also typically used in spinal surgery. The surgeon 60 wears an augmented reality device 53, in particular augmented reality glasses, which are part of the medical navigation device 50 and serve as a screen for displaying the proposed spinal rod 20.

[0152] The proposed spinal rod 20 itself is determined based on the positions Ps of the multiple screws 30. The positions Ps of the multiple screws 30 are acquired, for example, by a camera 51 of the medical navigation device 50. The camera 51 comprises, for example, a 3D camera configured to acquire the shape and the position in space of the multiple spinal screws 30.

[0153] Using the acquired positions Ps of the multiple screws 30, the medical navigation device 50 analyzes the placement of the multiple spinal screws 30 on the spine and determines the proposed spinal rod 20. In other words, the proposed spinal rod 20 is a virtual model of the spinal rod 10 as it must be shaped to fulfill its task in the spinal surgery. In a first step, the shape of the proposed spinal rod 20 is directly related to the shape of the spine 40 of the patient 70. However, the shape of the proposed spinal rod 20 can be adjusted automatically by the planning software or manually by the surgeon 60. If the spine 40 of the patient 70 should be adjusted as well as reinforced, the shape of the proposed spinal rod 20 should reflect such adjusted shape of the patient's spine 40 as it will be throughout the spinal surgery. For example, the surgeon adjusts the proposed spinal rod 20 by adding a certain amount of lordosis to the proposed spinal rod 20 via the user interface of the planning software.

[0154] The planning software is preferably provided with supporting data Ds which are displayed to the surgeon 60. The surgeon thus virtually adjusts the proposed spinal rods 20, and in particular the spinal alignment of the patient 70, until the desired medical outcome is reached. The medical outcome is preferably indicated by the supporting data Ds which include surgery-related parameters.

[0155] As a result, the proposed spinal rod 20 is provided to the augmented reality device 53 to be displayed within the surgeon's field of view. Thus, the surgeon always sees the proposed spinal rod 20 within this field of view when holding the spinal rod 10 in his / her hand to bend the spinal rod 10 according to the proposed spinal rod 20.

[0156] Also, the spinal rod 10 itself is calibrated, for example by a calibration device of the medical navigation device 50. In this case, the spinal rod 10 comprises a reference device 52 attached to the spinal rod 10. Thus, when performing the calibration, the acquired position of the spinal rod 10 is converted into the coordinates of the medical navigation device 50. In other words, the medical navigation device 50 learns about the position of the spinal rod 10 in its own coordinates. Thus, when displaying the proposed spinal rod 20 to the surgeon 60 via the augmented reality device 53, the augmented reality device 53 positions the proposed spinal rod 20 so that it overlaps the spinal rod 10 that the surgeon observes through the augmented reality device 53. By calibrating and continuously tracking the spinal rod 10, the augmented reality device 53 can always superimpose the proposed spinal rod 20 on the spinal rod 10 in the field of view of the surgeon 60. This allows for an improved display of the proposed spinal rod 20 for the surgeon 60 when bending the spinal rod 10.

[0157] FIG. 2a shows a schematic view through the augmented reality device 53 displaying the non-bent spinal rod 10 superimposed on the proposed spinal rod 20. In other words, the surgeon 60 has the spinal rod 10 in his field of view in order to bend the spinal rod 10 to the shape required for the spinal surgery. The surgeon 60 wants to bend the shape of the spinal rod 10 based on the proposed spinal rod 20, in particular with the help of a bending tool. The proposed spinal rod 20 is therefore displayed in the field of view of the surgeon 60 by the augmented reality device 53. The augmented reality device 53 not only randomly displays the proposed spinal rod 20 in the surgeon's field of view, but also displays the proposed spinal rod 20 superimposed on the spinal rod 10 from the surgeon's 60 perspective. In this case, the augmented reality device 53 positions the proposed spinal rod 20 so that the left end of the proposed spinal rod 20 coincides with the left end of the spinal rod 10. This allows for an improved display of information for the surgeon to bend the spinal rod 10.

[0158] FIG. 2b shows a schematic view through an augmented reality device 53 displaying a non-bent spinal rod 10 superimposed on the proposed spinal rod 20. In comparison with the spinal rod 10 in FIG. 2a, the spinal rod 10 is already bent. The spinal rod 10 is pre-bent by the surgeon 60 from experience or pre-bent by the surgeon 60 with the aid of the augmented reality device 53. The spinal rod 10 does not have to be a non-bent spinal rod 10 used by the medical navigation device 50, since the spinal rod 10 is tracked by the medical navigation device 50. Any pre-bent spinal rod 10 can be calibrated and tracked by the medical navigation device 50 and superimposed on the proposed spinal rod 20. In other words, the surgeon 60 has a partially bent spinal rod 10 in his field of view to finish bending the spinal rod 10 to the shape required for the spinal surgery. Similar to FIG. 2a, the proposed spinal rod 20 is displayed in the field of view of the surgeon 60 by the augmented reality device 53. Augmented reality device 53 not only randomly displays proposed spinal rod 20 within the surgeon's field of view, but also displays proposed spinal rod 20 to overlap spinal rod 10 from the perspective of surgeon 60. In this case, augmented reality device 53 positions proposed spinal rod 20 such that the already bent portion of spinal rod 10 matches the corresponding portion of spinal rod 10. This allows surgeon 60 to ensure that the already bent portion of spinal rod 10 will meet proposed spinal rod 20.

[0159] FIG. 3 shows a schematic diagram of tracking a spinal rod 10 by a medical navigation device 50. The spinal rod 10 is provided with a reference device 52, in this case a reference array of three markers. The reference device 52 marks the origin of a rod coordinate system Rod. The medical navigation device 50 comprises a camera 51 which marks the origin of a cam coordinate system Cam. The cam coordinate system Cam is known to the medical navigation device 50. When calibrating the spinal rod 10, for example by using a calibration device such as a calibration block, the relationship between the rod coordinate system and the cam coordinate system is determined. This relationship is described by the spinal rod-camera coordinate transformation RodToCam.

[0160] The term "transformation" as used herein specifically describes a translation and / or rotation between two objects, such as the tracking system of the medical navigation device 50 and the calibration device of the medical navigation device 50. When each object is represented by a position and orientation in space, preferably a coordinate system is defined for each object, so that the transformation allows the coordinates of a point in one system to be described in terms of coordinates in another system. For example, the calibration points of the calibration device are given in the local coordinates of the calibration device. The transformation from the calibration device to the spinal rod 10 can be used to represent the spinal rod 10 in the calibration device coordinates. Since every transformation has a unique inverse transformation, the spinal rod coordinates can also be represented in the calibration device coordinates. To optimize the meaning of the coordinate systems, their origins are usually located at the points of interest in those objects. A preferred implementation of such a transformation is the use of a 4x4 matrix, which is widely used in the field of computer graphics for exactly this purpose. Thus, one transformation matrix can include translations and rotations, theoretically any affine transformation in 3D space, which makes the matrix invertible. The composition of transformations such as from the calibration device to the camera and then from the camera to the spinal rod 10 is represented by the multiplication (in reverse order) of the corresponding matrices. The transformation between two coordinate systems can be set by knowing the origin and the three perpendicular axes of one coordinate system in the coordinates of the other coordinate system. In the case of a 4x4 matrix, a commonly used technique is a change of basis where the axes are normalized and written into the top left 3x3 part of the 4x4 matrix, and the translation between the coordinate systems is taken into account in the fourth column.

[0161] In the tracking setup for the calibration of the spinal rod 10, the different coordinate systems of the objects involved in the tracking system need to be related to each other: in other words, the tracking system, in particular the camera 51, comprises a camera coordinate system Cam, the calibration device comprises a calibration device coordinate system, and the spinal rod 10 comprises a spinal rod coordinate system Rod in its marker array.

[0162] To calibrate the spinal rod 10, it is necessary to find the relationship between the spinal rod 10 and the calibration device. This relationship can be determined by holding the spinal rod 10 over a known spot on the calibration device. Given that the relationship between the camera coordinate system Cam and the calibration device coordinate system is known, the relationship between the camera coordinate system Cam and the spinal rod coordinate system Rod can be calculated.

[0163] The orientation of the instrument tip coordinate system is preferably predefined with respect to the instrument marker coordinate system, however, a plane or other feature of the calibration device can be used to specifically calibrate an axis of the instrument, which is not the primary focus of the present invention.

[0164] Due to the spinal rod-camera coordinate transformation, the position, and especially the shape, of the spinal rod 10 is always known to the medical navigation device 50 .

[0165] FIG. 4 shows a schematic diagram of a medical navigation device 50. The medical navigation device 50 comprises a camera 51 configured, inter alia, to digitize the plurality of spinal screws 30 on the spine 40 of the patient 70, an augmented reality device 53 serving as a display of the medical navigation device 50, and a control unit 54. The camera 51 determines the positions Ps of the plurality of spinal screws 30, inter alia, by using the tracked instruments, and provides the positions Ps to the control unit 54. The control unit 54 uses the positions Ps of the plurality of spinal screws 30 to determine a proposed spinal rod 20, which is a virtual model of the spinal rod 10 as it must be shaped to fit the plurality of spinal screws 30. The proposed spinal rod 20 is provided to the augmented reality device 53, where the proposed spinal rod 20 is displayed to the surgeon as a template for bending the spinal rod 10 to a shape. In addition to the proposed spinal rod 10, additional information can be provided to the augmented reality device 53 by the control unit 54. For example, the control unit 54 is provided with a spinal model Ms representing the spine 40 of the patient 70. The spine model Ms is used by the control unit 54 to determine the support data Ds, which is provided, for example, to the augmented reality device 53. The spine model Ms can be used to determine how the proposed shape of the spinal rod 20 affects the forces applied to the vertebrae 40 or the spinal screws 30. This information is then included in the support data Ds and used by the augmented reality device 53 to display the forces applied to different objects. Thus, the surgeon wearing the augmented reality device 53 is provided with additional information regarding the case.

[0166] The control unit 54 preferably comprises planning software that allows the already determined proposed spinal rods 20 to be adjusted either automatically by the planning software itself or manually by the surgeon 60 via an input interface.

[0167] 5a shows a schematic diagram through an augmented reality device displaying the non-bent spinal rod 10 of FIG. 2a superimposed on the proposed spinal rod 20 and spinal screw indicators Is. The spinal screw indicators Is are based in particular on the support data Ds provided by the control device 54. The spinal screw indicators Is indicate where the plurality of spinal screws 30 would be placed on the spinal rod 10 when the spinal rod 10 has the shape of the proposed spinal rod 20.

[0168] 5b shows a schematic view through an augmented reality device displaying the unbent spinal rod 10 of FIG. 2a superimposed on the proposed spinal rod 20 and the bend indicator Ib. The bend indicator Ib is based in particular on the support data Ds provided by the control device 54. The bend indicator Ib is superimposed on the spinal rod 10 and indicates the spot where the spinal rod 10 should ideally be bent to reach the shape of the proposed spinal rod 20.

[0169] 6 shows a schematic diagram of a spine 40 of a patient 70 with spinal screws 30 connected by spinal rods 10. The spinal screws 30 are inserted into the vertebral bodies, specifically the pedicles or lateral masses, and thus directly connected to the patient's spine 40. FIG. 6 shows that, generally, two parallel rows of spinal screws 30 are inserted into the spine 40, with each row of spinal screws 30 connected to one spinal rod 10.

[0170] FIG. 7 shows a schematic diagram of a computer-implemented method for augmented reality spinal rod planning and bending for navigation spine surgery. In a first step S10, the positions Ps of the spinal screws 30 placed on the spine 40 are acquired, and the spinal screws 30 are configured to receive the spinal rod 10 that interconnects the spinal screws 30. In another step S20, the acquired positions Ps of the spinal screws 30 are used to determine a proposed spinal rod 20, which is a virtual model of the spinal rod 10 having a desired shape. In another step S30, the spinal rod 10 is calibrated to track the spinal rod 10 by the medical navigation device 50. In another step S40, the proposed spinal rod 20 is superimposed on the tracked spinal rod 10 by displaying the proposed spinal rod 20 by the augmented reality device 53.

Claims

1. 1. A computer implemented method for augmented reality spinal rod planning and bending for navigation spine surgery, comprising: obtaining (S10) positions (Ps) of a plurality of spinal screws (30) placed on a spine (40), the plurality of spinal screws (30) being configured to receive a spinal rod (10) interconnecting the plurality of spinal screws (30); determining (S20) a proposed spinal rod (20), which is a virtual model of a spinal rod having a desired shape, using the acquired positions (Ps) of the plurality of spinal screws (30); calibrating (S30) the spinal rod (10) for tracking the spinal rod (10) by a medical navigation device (50); displaying (S40) said proposed spinal rod (20) by means of an augmented reality device (53), thereby superimposing said proposed spinal rod (20) on said tracked spinal rod (10); 4. A computer-implemented method comprising:

2. The proposed spinal rod (20) has a shape that matches the location of the plurality of spinal screws (30) on the spine (40). The method of claim 1.

3. determining the proposed spinal rod (20) using the acquired positions (Ps) of the plurality of spinal screws (30) and a planned shape of the spine (40); The method of claim 1.

4. Calibrating (S30) the spinal rod (10) determining a spinal rod model, which is a virtual representation of the spinal rod (10); The method comprises: determining support data (Ds) using said spinal rod model, said support data (Ds) including information related to said spine (40); superimposing said support data (Ds) on said spinal rod (10) by means of said augmented reality device (53); The method according to any one of claims 1 to 3, comprising:

5. Determining the spinal rod model comprises: The method of claim 4, comprising recognizing the shape of the spinal rod (10) relative to a tracked reference array.

6. Determining the spinal rod model comprises: The method of claim 4, comprising obtaining the shape of the spinal rod (10) by a tracking device.

7. dynamically adjusting the spinal rod model using the tracked spinal rod (10). The method according to any one of claims 4 to 6.

8. The supporting data (Ds) includes at least one bending indicator (Id) determined by using the proposed spinal rod (20) and the spinal rod model. The method according to any one of claims 4 to 7. determining a spinal screw model that is a virtual representation of the plurality of spinal screws (30) disposed on the spine (40); The support data (Ds) is determined by using the positions (Ps) of the plurality of spinal screws (30) and includes at least one screw indicator (Is) indicative of the plurality of spinal screws (30) on the spinal rod (10). The method according to any one of claims 4 to 8. determining a mean deviation between the spinal rod (10) and the proposed spinal rod (20) by using the spinal rod model and the proposed spinal rod (20); The supporting data (Ds) includes a deviation indicator determined by using the determined mean deviation. The method according to any one of claims 4 to 9. determining a spine model, the spine being a virtual representation of the spine (40); adjusting the spinal model using the spinal rod model; Including, the support data (Ds) is determined by using the spinal model and includes a spinal indicator indicating the vertebrae (40) on the spinal rod (10); The method according to any one of claims 4 to 8.

12. The method of claim 1, further comprising: determining at least one anatomical parameter of the spine (40) by using the spinal model and the spinal rod model; The supporting data (Ds) includes at least one anatomical parameter indicator determined by using the at least one determined anatomical parameter. The method of claim 11.

13. The method of claim 12, further comprising: determining forces applied to the plurality of spinal screws (30) by using the spinal model and the spinal rod model; the support data (Ds) is determined by using the determined forces and includes a force indicator indicative of a force that would be applied to the plurality of spinal screws (30) if the spinal rod (10) were connected to the spinal screws (30); The method according to any one of claims 11 to 12.

14. The method of claim 13, further comprising: determining a force alert if the determined force exceeds a predetermined threshold; The support data (Ds) includes a force warning indicator determined by using the determined force warning. The method of claim 13.

15. Calibrating the spinal rod (10) comprises: Providing a reference device (52) on said spinal rod (10) that defines the origin of a spinal rod coordinate system (Rod); determining a spinal rod-to-cam coordinate transformation (RodToCam) that describes the transformation between the spinal rod coordinate system (Rod) and a camera coordinate system (Cam); Including, The method according to any one of claims 1 to 14.

16. Obtaining the positions (Ps) of the plurality of spinal screws (30) includes recognizing the plurality of spinal screws (30) by the augmented reality device (53). The method according to any one of claims 1 to 15.

17. Obtaining the positions (Ps) of the plurality of spinal screws (30) includes extracting the positions (Ps) of the plurality of spinal screws (30) from a planning application. The method according to any one of claims 1 to 16.

18. Obtaining the positions (Ps) of the plurality of spinal screws (30) includes detecting the positions (Ps) of the plurality of spinal screws (30) in intraoperative paired image data. The method according to any one of claims 1 to 17.

19. Obtaining the positions (Ps) of the plurality of spinal screws (30) includes calibrating each of the plurality of spinal screws (30) by using a tracked pointer. The method according to any one of claims 1 to 18.

20. A medical navigation device (50) configured to perform the method according to any of the claims 1 to 19.

21. A computer program which, when executed or loaded onto a computer, causes said computer to carry out the method steps of the method according to any of claims 1 to 19.

22. A program storage medium on which the computer program described in claim 21 is stored.

23. A computer having at least one processor and a memory and / or a program storage medium as described in claim 22, wherein the computer program as described in claim 21 is running on the computer or loaded into the memory of the computer.

Images