A composite surgical navigation device, method, and system

Abstract

The application belongs to the technical field of surgical navigation, and specifically discloses a compound surgical navigation device, method and system, which comprises a coordinate system registration module, a positioning module and an error calculation module. The coordinate system registration module is used for acquiring an image of a target to be operated and registering the image with a preset first coordinate system, a second coordinate system and a third coordinate system. The positioning module is used for acquiring the positioning positions of an execution mechanism in the first coordinate system, the second coordinate system and the third coordinate system. The error calculation module is used for obtaining corresponding reference positions according to the positioning positions and a preset corresponding conversion relationship, acquiring corresponding positioning errors according to the positioning positions and the obtained reference positions in the same coordinate system, and determining the current position of the execution mechanism according to the acquired positioning errors. The application realizes high stability and high precision navigation by complementing the advantages and disadvantages of several navigation positioning modes and verifying errors, and provides stable, reliable and precise position positioning information for doctors, thereby greatly improving the operation quality and efficiency of doctors.

Description

Technical Field

[0001] This invention belongs to the field of surgical navigation technology, and particularly relates to a composite surgical navigation device, method and system. Background Technology

[0002] With the development of medical image processing technology, surgical navigation technology has emerged. Surgical navigation technology accurately correlates the patient's image data with the patient's anatomical structure, tracks surgical instruments during surgery, and updates and displays the position of the surgical instruments on the patient's image in real time in the form of virtual probes, so that doctors can clearly see the position of the surgical instruments relative to the patient's anatomical structure.

[0003] Currently, surgical navigation technologies mainly include optical, electromagnetic, and ultrasound navigation. Traditional optical and electromagnetic navigation systems utilize preoperative computed tomography (CT) or magnetic resonance imaging (MRI) images. However, these systems struggle to track and reflect elastic human tissues or organs in real time. Because these tissues or organs undergo elastic deformation upon contact with surgical instruments, preoperative medical imaging data cannot perfectly match the actual intraoperative situation. Furthermore, the lack of real-time monitoring methods for this deformation leads to significant positioning errors during surgical navigation. Additionally, these systems can only track bony tissues and instruments, failing to track soft tissues such as blood vessels, nerves, and muscles. Optical navigation is also susceptible to obstructions, while electromagnetic navigation is prone to interference from ferromagnetic objects, resulting in inaccurate positioning. Ultrasound navigation, on the other hand, has a narrow positioning range, only able to see the area illuminated by ultrasound, resulting in a limited field of view. Moreover, the ultrasound probe can only detect what is within its range, unable to determine the probe's own position within the body, and therefore cannot accurately locate surgical instruments or the patient.

[0004] In summary, existing navigation systems are susceptible to external interference and limitations in their own positioning, resulting in an inability to accurately obtain the positioning information of surgical tools and surgical sites, and an inability to perform stable surgical navigation for extended periods. Summary of the Invention

[0005] The purpose of this invention is to provide a composite surgical navigation device, method, and system that complements the advantages and disadvantages of several positioning methods and verifies the positioning positions of various methods to achieve highly stable and accurate navigation. This provides doctors with stable, reliable, and accurate positioning information, greatly improving the quality and efficiency of their surgeries.

[0006] To achieve the above objectives, the technical solution of the present invention is as follows:

[0007] A composite surgical navigation device, comprising:

[0008] The coordinate system registration module is used to acquire image images of the target and register the coordinate system of the image images with preset first coordinate system, second coordinate system and third coordinate system;

[0009] The positioning module is used to obtain the positioning positions of the actuator in the first coordinate system, the second coordinate system and the third coordinate system respectively;

[0010] The error calculation module is used to convert each acquired positioning position into a corresponding reference position based on a preset conversion relationship; and to obtain the corresponding positioning error based on the positioning position in the same coordinate system and the converted reference position; and to determine the current position of the actuator based on the positioning error obtained in different coordinate systems.

[0011] Optionally, the image includes a preset image and a real-time image of the target, and the coordinate system registration module includes:

[0012] The image data acquisition unit is used to acquire preset image data and real-time image data of the target, and to reconstruct a three-dimensional model based on the preset image data.

[0013] The first transformation relationship calculation unit is used to obtain the first image coordinates of the image marker points in the real-time image, obtain the feature coordinates of the feature marker points in the feature coordinate system, and calculate the transformation relationship between the real-time image and the feature coordinate system based on the first image coordinates and the feature coordinates.

[0014] The second conversion relationship calculation unit is used to obtain the second image coordinates of the image marker points in the preset image, and calculate the conversion relationship from the preset image to the real-time image based on the first image coordinates and the second image coordinates;

[0015] The registration unit calculates the transformation relationship between the preset image and the feature coordinate system based on the transformation relationship between the real-time image and the feature coordinate system, and the transformation relationship between the preset image and the real-time image.

[0016] Optionally, the positioning position includes a first position of the actuator in the first coordinate system, a second position in the second coordinate system, and a third position in the third coordinate system; the error calculation module includes:

[0017] The first calculation unit is used to calculate the coordinates of the corresponding second reference position, third reference position and first reference position based on the coordinates of the first position, second position and third position obtained by the positioning module and the preset transformation relationship;

[0018] The second calculation unit is used to calculate the first error, the second error, and the third error based on the coordinates of the first position and the first reference position, the second position and the second reference position, and the third position and the third reference position.

[0019] The position determination unit is used to determine the current position of the actuator based on a comparison of the first error, the second error, and the third error with a preset threshold.

[0020] Optionally, the positioning module includes:

[0021] An optical positioning unit is used to acquire the optical coordinates of an optical marker point in a first coordinate system.

[0022] An electromagnetic positioning unit is used to obtain the electromagnetic coordinates of an electromagnetic marker point in the second coordinate system.

[0023] The structured light positioning unit is used to obtain the structured light coordinates of the structured light marker points in the third coordinate system.

[0024] Optionally, the positioning module further includes:

[0025] An ultrasonic positioning unit is used to acquire ultrasonic images scanned by an ultrasonic probe disposed at the front end of the actuator, and to fuse the ultrasonic images with the three-dimensional model.

[0026] On the other hand, the present invention also discloses a composite surgical navigation method, comprising:

[0027] Acquire an image of the target and register the image with a preset feature coordinate system, wherein the feature coordinate system includes at least a first coordinate system, a second coordinate system and a third coordinate system;

[0028] The positioning positions of an actuator in the first coordinate system, the second coordinate system, and the third coordinate system are collected respectively;

[0029] The corresponding reference position is obtained by converting each of the positioning positions and the preset corresponding conversion relationship;

[0030] The positioning error is obtained based on the positioning position in the same coordinate system and the reference position obtained by conversion, and the current position of the actuator is determined based on the positioning error obtained in different coordinate systems.

[0031] Optionally, the image includes a preset image and a real-time image of the target, and the registration of the image with a preset feature coordinate system includes:

[0032] Image markers and feature markers are configured on the implementation target, and a real-time image of the implementation target is acquired, as well as the image coordinates of the image markers in the real-time image and the feature coordinates of the feature markers in the feature coordinate system are acquired.

[0033] The transformation relationship between the real-time image and the feature coordinate system is obtained based on the image coordinates and the feature coordinates;

[0034] Based on the positions of the image markers in the preset image and the real-time image, respectively, the transformation relationship from the preset image to the real-time image is obtained;

[0035] Based on the conversion relationship between the preset image and the real-time image, and the conversion relationship between the real-time image and the feature coordinate system, the conversion relationship between the preset image and the intraoperative feature coordinate system is obtained.

[0036] Optionally, the positioning positions include a first position of the actuator in the first coordinate system, a second position in the second coordinate system, and a third position in the third coordinate system; the step of converting each positioning position and a preset corresponding transformation relationship to obtain the corresponding reference position includes:

[0037] The second reference position of the first position in the second coordinate system is obtained based on the transformation relationship between the first and second coordinate systems; and

[0038] The third reference position of the second position in the third coordinate system is obtained according to the transformation relationship between the second coordinate system and the third coordinate system; and

[0039] The first reference position of the third position in the first coordinate system is obtained according to the transformation relationship between the third coordinate system and the first coordinate system.

[0040] Optionally, the positioning error includes a first error calculated based on the first position and the first reference position, a second error calculated based on the second position and the second reference position, and a third error calculated based on the third position and the third reference position;

[0041] The first coordinate system is a coordinate system established based on an optical positioning system, the second coordinate system is a coordinate system established based on an electromagnetic positioning system, and the third coordinate system is a coordinate system established based on a structured light positioning system.

[0042] Determining the current position of the actuator based on the positioning error obtained in different coordinate systems includes:

[0043] If the first error is less than the first threshold and / or the second error is less than the second threshold, then the first position is determined as the current position of the actuator.

[0044] If the first error is greater than the first threshold, the second error is greater than the second threshold, and the third error is less than the third threshold, then the second position or the third position is determined as the current position of the actuator.

[0045] On the other hand, the present invention also discloses a composite surgical navigation system, including a positioning system, an actuator, and a computer device;

[0046] The positioning system includes a tracker configured on the actuator and the target, and a tracking device matched with the tracker. The tracker is used to assist the tracking device in determining the position coordinates of the actuator and the target in a feature coordinate system.

[0047] The computer device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described composite surgical navigation method.

[0048] The beneficial effects of this solution are: it complements the advantages and disadvantages of several positioning methods, including visible light, electromagnetic, structured light, and ultrasound, and verifies the positioning positions of various methods to achieve highly stable and accurate navigation. This provides doctors with stable, reliable, and accurate positioning information, greatly improving the quality and efficiency of their surgeries. Attached Figure Description

[0049] Figure 1 This is a flowchart of an embodiment of the composite surgical navigation method of the present invention;

[0050] Figure 2 This is a structural block diagram of an embodiment of the composite surgical navigation device of the present invention;

[0051] Figure 3 This is a structural block diagram of an embodiment of the composite surgical navigation system of the present invention;

[0052] Figure 4 This is an internal structural diagram of a computer device according to an embodiment of the composite surgical navigation system of the present invention. Detailed Implementation

[0053] The various embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0054] See Figure 1 This embodiment provides a composite surgical navigation method, including:

[0055] S100. Acquire an image of the target and register the image with a preset feature coordinate system, wherein the feature coordinate system includes at least a first coordinate system, a second coordinate system and a third coordinate system.

[0056] It should be noted that, in this embodiment, the target of implementation can be the lesion site of the patient, etc.

[0057] In this embodiment, the images include preset images and real-time images. Both preset images and real-time images are medical images of the patient that include the target portion of the procedure. The preset images refer to images taken in advance before the operation (e.g., preoperative examination), while the real-time images refer to images taken in real time during the operation.

[0058] In this embodiment, the images captured are specifically one or more combinations of medical images such as MRI tomography images, CT scan images, and X-ray images. Furthermore, the images obtained from scanning using the above methods are all images of the bone tissue of the target.

[0059] Before surgery, a three-dimensional reconstruction is performed based on preset images to obtain a three-dimensional model of the target. In this embodiment, a three-dimensional model of the target is established by processing CT images, X-ray images, or MRI images. The three-dimensional model allows doctors to observe the target more intuitively. After the three-dimensional model is established, it can also be visualized and displayed on a screen.

[0060] The reconstruction of the preset image adopts the existing three-dimensional reconstruction method in surgical navigation. For example, a three-dimensional model is obtained by segmenting, reconstructing and rendering the preset image; or, for example, the image feature information of the preset image is extracted and the three-dimensional model of the target is reconstructed based on the image feature information; the geometric information of the three-dimensional model is read and the three-dimensional model is rendered based on the geometric information to obtain the rendered three-dimensional model; wherein, the image feature information is grayscale or boundary information, and the geometric information includes vertices, mesh surfaces and the normal direction of the mesh surfaces.

[0061] It should be noted that the aforementioned characteristic coordinate system refers to the coordinate system built into the positioning device or system, such as the optical coordinate system in an optical positioning system, the electromagnetic coordinate system in an electromagnetic positioning system, etc.

[0062] In this embodiment, the feature coordinate system includes a first coordinate system, a second coordinate system, and a third coordinate system.

[0063] In one embodiment, the first coordinate system is a coordinate system established based on an optical positioning system, the second coordinate system is a coordinate system established based on an electromagnetic positioning system, and the third coordinate system is a coordinate system established based on a structured light positioning system.

[0064] In this embodiment, registering the image with a preset feature coordinate system specifically involves first registering the image with an optical positioning system. The optical positioning system registration with the image specifically includes the following steps:

[0065] Before and during the procedure, image markers and optical markers are placed (implanted or selected) at the target site. Image markers are markers that can be automatically identified on images, such as surface features selected at the patient's lesion (spinal protrusions, etc.) or surface feature points (protrusion points, etc.) on a tracker. Optical markers are optical markers that can be tracked by the optical tracking device in the optical positioning system, such as optical spheres implanted at the target site or optical spheres set on the tracker. At least three image markers are included, and these three non-collinear image markers can form a local coordinate system in the surgical space. At least three non-collinear feature markers are also included, and these three non-collinear feature markers can form a local coordinate system, thus forming an optical coordinate system.

[0066] Then, real-time images containing image markers and feature markers are captured using CT scanning equipment, X-ray scanning equipment, or MRI scanning equipment. The image coordinates of the image markers in the real-time images are automatically identified by computer equipment, and the optical coordinates of the optical markers in the optical coordinate system are obtained through an optical positioning system.

[0067] The transformation relationship between the real-time image space and the optical coordinate system is then calculated using the coordinate system constructed from the image coordinates and the coordinate system constructed from the optical coordinates, as well as the matrix transformation algorithm built into the computer device. The matrix transformation algorithm is an existing transformation algorithm between two matrices, which will not be described in detail in this embodiment.

[0068] Then, corresponding marker points are selected on the preset image and the real-time image, such as the same set of vertebral protrusion points, or the above-mentioned image marker points. The coordinates of the marker points on the preset image and the real-time image are obtained respectively. Based on the relationship between the coordinates, the transformation relationship between the preset image space and the real-time image space is calculated by a matrix transformation algorithm.

[0069] Then, based on the conversion relationship between the preset image space and the real-time image space, and the conversion relationship between the real-time image space and the optical coordinate system, the conversion relationship between the preset image space and the optical coordinate system is obtained, and the registration of the image and the optical coordinate system is completed.

[0070] Finally, the electromagnetic positioning system and the structured light positioning system are registered with the image based on the conversion relationship between the optical positioning system, the electromagnetic positioning system and the structured light positioning system.

[0071] Specifically, in one embodiment, a reference tool is provided, which includes at least three optical feature points that can be tracked by an optical positioning system, at least one electromagnetic feature point that can be tracked by an electromagnetic positioning system, and a set of structured light feature points that can be identified by a structured light positioning system. The relative relationships between these three sets of feature points are known during the design process, wherein the conversion relationship between the optical feature points and the electromagnetic feature points is as follows: The conversion relationship between optical feature points and structured light feature points is as follows: The conversion relationship between electromagnetic feature points and structured light feature points can be determined based on... and The results were obtained through calculation. Optical feature points can be selected from optical spheres, electromagnetic feature points can be selected from electromagnetic sensors, and structured light feature points can be selected from surface features on a reference tool.

[0072] During registration, the positions of the three tracking devices are adjusted so that they can simultaneously observe the reference tool, and the coordinate representation of the reference tool under the optical tracking devices is obtained respectively. Coordinate representation under electromagnetic tracking equipment and coordinate representation under structured light tracing equipment The coordinate representations above are all 4x4 pose matrices of the local coordinate system constructed from the corresponding feature points under the corresponding tracking device.

[0073] With the above relationships established, the three coordinate systems can be unified. Taking the unification into an optical system as an example, the local coordinate system of an optical feature point in the electromagnetic coordinate system is described as follows: * The transformation relationship from the electromagnetic coordinate system to the optical coordinate system is as follows. for Similarly, the transformation relationship from the structured light coordinate system to the optical coordinate system... for .

[0074] At this point, the three coordinate systems have been standardized and unified, enabling the electromagnetic positioning system and structured light positioning system to be registered with the image.

[0075] After completing the registration of the image with the feature coordinate system according to the above steps, the preset image is converted to the real-time image space for positioning during surgery, so that the clear preset image can be used for surgical navigation.

[0076] S200: Collect the positioning positions of an actuator in the first coordinate system, the second coordinate system, and the third coordinate system.

[0077] Specifically, the actuator is equipped with feature markers that match the corresponding coordinate system, and the relative positional relationships of each feature marker are known. These feature markers are optical markers, electromagnetic markers, and structured light markers.

[0078] By using feature markers, the first position of the actuator in the optical coordinate system can be acquired using an optical positioning system, the second position of the actuator in the electromagnetic coordinate system can be acquired using an electromagnetic positioning system, and the third position of the actuator in the structured light coordinate system can be acquired using a structured light positioning system.

[0079] All of the above-mentioned positioning information can be converted into the image coordinate system through the registration transformation relationship. Based on the positioning information, the spatial position and attitude of the actuator can be determined, so that doctors can observe the operation of the actuator at the target location through the device and provide doctors with positioning information.

[0080] It should be noted that the actuator in this embodiment refers to surgical tools such as scalpels, surgical robots, and surgical forceps.

[0081] In traditional methods, the position of the actuator is usually determined by one of the aforementioned optical positioning system, electromagnetic positioning system, and structured light positioning system. However, optical positioning systems are easily blocked, electromagnetic positioning systems are easily interfered with by ferromagnetic objects, and structured light positioning systems are easily blocked and can only identify surface features, which makes it impossible to guarantee the positioning accuracy and stability of a single positioning system.

[0082] This method employs a composite approach for positioning and navigation, and verifies the error at each positioning location to ensure positioning accuracy. The specific error verification method is as follows:

[0083] S300: The corresponding reference position is obtained by converting each positioning position and the corresponding preset conversion relationship.

[0084] The transformation relationships include the transformation relationship M1 between the first coordinate system and the second coordinate system, the transformation relationship M2 between the second coordinate system and the third coordinate system, and the transformation relationship M3 between the third coordinate system and the first coordinate system.

[0085] Specifically, the acquisition of each conversion relationship is described in the registration of the optical positioning system, electromagnetic positioning system and structured light positioning system mentioned above, and will not be repeated here.

[0086] In this step, the reference positions are obtained as follows:

[0087] The first position is calculated using the transformation relationship M1 between the first and second coordinate systems, resulting in a second reference position in the second coordinate system. Here, the second reference position is the theoretical position calculated using transformation relationship M1, while the second position is the actual detection position. By determining the deviation between the second position and the second reference position, the accuracy of the first and second positions can be verified.

[0088] Similarly, the third reference position of the second position in the third coordinate system is calculated based on the second position and the transformation relationship M2 between the second and third coordinate systems. Here, the third reference position is the theoretical position calculated using transformation relationship M2, and the third position is the actual detection position. By judging the deviation between the third position and the third reference position, the accuracy of the second and third positions can be determined.

[0089] Similarly, the first reference position of the third position in the first coordinate system is obtained based on the third position and the transformation relationship M3 between the third coordinate system and the first coordinate system. Here, the first reference position is the theoretical position calculated using transformation relationship M3, and the first position is the actual detection position. By judging the deviation between the first position and the first reference position, the accuracy of the first and third positions can be determined.

[0090] S400. Obtain the corresponding positioning error based on the positioning position in the same coordinate system and the converted reference position, and determine the current position of the actuator based on the positioning error obtained in different coordinate systems.

[0091] The positioning position and reference position in the same coordinate system refer to the first position and the first reference position in the optical coordinate system, the second position and the second reference position in the electromagnetic coordinate system, and the third position and the third reference position in the structured light coordinate system.

[0092] Accordingly, the positioning error includes a first error calculated based on the first position and the first reference position, a second error calculated based on the second position and the second reference position, and a third error calculated based on the third position and the third reference position.

[0093] In this embodiment, the aforementioned errors refer to the differences between the coordinates of the positioning position and the coordinates of the reference position in the x, y, and z directions. For example, if the first position coordinates are (x1, y1, z1) and the first reference coordinates are (x2, y2, z2), then the first errors are a1=|x1-x2|, b1=|y1-y2|, and c1=|z1-z2|.

[0094] In this step, determining the current position of the actuator based on the positioning error includes:

[0095] If the first error is less than the first threshold and / or the second error is less than the second threshold, then the first position is determined as the current position of the actuator;

[0096] If the first error is greater than the first threshold, the second error is greater than the second threshold, and the third error is less than the third threshold, then the second position or the third position is determined as the current position of the actuator.

[0097] For example, if the first threshold is x=a, y=b, z=c, and if |x1-x2|≤a, |y1-y2|≤b, |z1-z2|≤c are all satisfied, it means that the first error is less than the first threshold, the first position and the third position are accurately located, and the first position is determined as the current position of the actuator; otherwise, it means that the first error is greater than the first threshold, and at least one of the first position and the third position is not accurately located.

[0098] The calculation methods for the second and third errors are the same as those for the first error, and will not be repeated here.

[0099] Specifically, at a certain moment, if the first error, the second error, and the third error are all less than a preset threshold, it indicates that the first position, the second position, and the third position are all accurately located. Since the positioning accuracy of the optical positioning system is higher, in this case, the first position under the optical positioning system is preferred as the current position of the actuator.

[0100] When the error value between the first position and the position located by any other positioning system is less than a preset threshold, it indicates that the first position is accurately located at least as the second or third position. In this case, the first position is preferred as the current position of the actuator.

[0101] When the error values ​​between the first position and the second and third positions both exceed preset thresholds, and the error value between the second and third positions is less than the preset threshold, it indicates that the optical positioning system is not accurate. In this case, the current position of the actuator is determined to be either the second or third position. Furthermore, if the actuator is located on soft tissue, the third position is used; if the actuator is located within bone tissue, the second position is used.

[0102] This embodiment avoids the situation where one of the positioning methods is inaccurate at a certain moment by verifying the mutual error of the first position, the second position and the third position, thus ensuring the positioning accuracy at every moment, so as to accurately determine the current position of the actuator and provide precise navigation for doctors.

[0103] In some embodiments, an ultrasonic probe is also provided at the front end of the actuator. The relative positional relationship between the target and the actuator can be determined based on the ultrasonic waves emitted and received by the ultrasonic probe, and the tissue structure directly in front of the ultrasonic probe can be accurately detected.

[0104] In this method, the ultrasound probe moves synchronously along the trajectory of the actuator and acquires ultrasound images of the target during the synchronous movement. The ultrasound images are then fused in real time with a model based on three-dimensional reconstruction of preoperative images using computer equipment to form a first image model.

[0105] Specifically, in this embodiment, the ultrasound image is a soft tissue image of the target, and the first image model formed by fusing it with the bone tissue model is the complete reconstructed image of the target.

[0106] The fusion principle is as follows: the ultrasound probe is rigidly connected to the actuator to ensure that the transmission and reception angles of the ultrasound probe are fixed, and the relative position information between the ultrasound probe and the actuator is known. Furthermore, since the actuator is precisely positioned by the optical positioning system, electromagnetic positioning system, and structured light positioning system, the spatial position and orientation of the actuator are known, and the position of the actuator can be converted into the image in real time. The relative position information (distance, angle, etc.) between the rigidly connected ultrasound probe and the actuator is also fixed and known. Based on the positioning information of the actuator, the positioning information of the ultrasound probe can be obtained. Therefore, the position and range detected by the ultrasound probe are known in real time, and the ultrasound image detected by the ultrasound probe can be accurately converted into the image coordinate system, fusing the soft tissue image data and bone tissue image data into a first image model.

[0107] Furthermore, during the positioning process, when the ultrasound probe scans the same location repeatedly, the ultrasound image at that location is updated in real time, enabling real-time image fusion of bone and soft tissues, and achieving real-time tracking and feedback of elastic soft tissues.

[0108] By fusing real-time ultrasound images into the reconstruction model, doctors can not only clearly see the bony tissue of the target, but also see images of soft tissues such as blood vessels and nerves in real time. This provides doctors with a full view of the target, greatly facilitating the positioning of the actuator. Doctors can make local fine adjustments to the position of the actuator based on the soft tissue structure, allowing the actuator to avoid arteries, nerves, and other areas, effectively improving surgical efficiency and quality.

[0109] This embodiment combines the advantages and disadvantages of several positioning methods, including optical positioning, electromagnetic positioning, structured light positioning, and ultrasonic positioning, to achieve highly stable and accurate positioning, providing stable, reliable, and accurate location information, which greatly improves the quality and efficiency of surgery.

[0110] This embodiment verifies the positioning information of various positioning methods to determine the accurate position of the actuator. Compared with a single positioning method, this method can avoid interference and achieve accurate and stable positioning.

[0111] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but may be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0112] See Figure 2 Based on the same inventive concept, this embodiment also provides a composite surgical navigation device, including:

[0113] The coordinate system registration module 501 is used to acquire image images of the target and register the coordinate system of the image images with the preset first coordinate system, second coordinate system and third coordinate system;

[0114] Positioning module 502 is used to obtain the positioning positions of the actuator in the first coordinate system, the second coordinate system and the third coordinate system respectively;

[0115] The error calculation module 503 is used to calculate the corresponding reference position based on each acquired positioning position and the preset corresponding conversion relationship; and to obtain the corresponding positioning error based on the positioning position in the same coordinate system and the converted reference position; and to determine the current position of the actuator based on the positioning error obtained in different coordinate systems.

[0116] The positioning module 502 determines the spatial coordinates of the actuator and the target in the first coordinate system, the second coordinate system, and the third coordinate system based on the feature signals received from the actuator and the target.

[0117] In this embodiment, the positioning module includes:

[0118] An optical positioning unit is used to acquire the optical coordinates of an optical marker point in a first coordinate system.

[0119] An electromagnetic positioning unit is used to obtain the electromagnetic coordinates of an electromagnetic marker point in the second coordinate system.

[0120] The structured light positioning unit is used to obtain the structured light coordinates of the structured light marker points in the third coordinate system.

[0121] In an optical positioning system, the characteristic signal (optical marker) refers to the infrared light signal reflected or actively emitted by an optical sphere configured on the actuator and the target. In an electromagnetic positioning system, the characteristic signal (electromagnetic marker) refers to the magnetic field signal fed back by an electromagnetic sensor configured on the actuator and the target in an electromagnetic field. In a structured light positioning system, the characteristic signal (structured light marker) refers to the information of the selected point on the surface of the actuator and the target fed back by the camera.

[0122] The image includes a preset image and a real-time image of the target, and the coordinate system registration module 501 includes:

[0123] The image data acquisition unit is used to acquire preset image data and real-time image data of the target. Specifically, both preset and real-time image data are obtained by scanning the target using an MRI scanner, CT scanner, or X-ray scanner. These can be anteroposterior or lateral radiographs. The scanning equipment transmits the scanned image data to this unit. This unit is also used to process the real-time and preset images. Specifically, the processing includes three-dimensional reconstruction of a three-dimensional model based on the preset image data. The reconstruction of the three-dimensional model can be found in the content described in the aforementioned composite surgical method.

[0124] The first transformation relationship calculation unit is used to obtain the first image coordinates of the image marker points in the real-time image, obtain the feature coordinates of the feature marker points in the feature coordinate system, and calculate the transformation relationship between the real-time image and the feature coordinate system based on the first image coordinates and the feature coordinates.

[0125] Image markers are surface features of the target or markers placed on the target, selected by the user. Each image marker must include at least three non-collinear points, which together form a local coordinate system in the surgical space. Similarly, each feature marker must include at least three non-collinear points, which together form a local coordinate system, thus creating a feature coordinate system.

[0126] The transformation relationship between the real-time image and the feature coordinate system refers to the transformation relationship between the real-time image space and the first, second, and third coordinate systems calculated using a matrix transformation algorithm.

[0127] The second conversion relationship calculation unit is used to obtain the second image coordinates of the image marker points in the preset image, and calculate the conversion relationship from the preset image to the real-time image based on the first image coordinates and the second image coordinates. The corresponding conversion relationship is calculated using the same group of marker points in the preset image and the real-time image.

[0128] The registration unit calculates the transformation relationship between the preset image and the feature coordinate system based on the transformation relationship between the real-time image and the feature coordinate system, as well as the transformation relationship between the preset image and the real-time image, to complete the registration of the image and the feature coordinate system and establish the coordinate relationship between the image and the positioning system.

[0129] Based on the coordinate system registration module 501 described above, the registration process between the image and the feature coordinate system is as follows:

[0130] First, the optical positioning unit is used to register with the image. Specifically, the registration of the optical positioning unit with the image includes the following steps:

[0131] Before and during the procedure, image markers and optical markers are placed (implanted or selected) at the target site. Image markers are markers that can be automatically identified on images, such as surface features selected at the patient's lesion (spinal protrusions, etc.) or surface feature points (protrusion points, etc.) on a tracker. Optical markers are optical markers that can be tracked by the optical tracking device in the optical positioning system, such as optical spheres implanted at the target site or optical spheres set on the tracker. At least three image markers are included, and these three non-collinear image markers can form a local coordinate system in the surgical space. At least three non-collinear feature markers are also included, and these three non-collinear feature markers can form a local coordinate system, thus forming an optical coordinate system.

[0132] Then, real-time image data containing image markers and feature markers is captured by CT scanning equipment, X-ray scanning equipment or MRI scanning equipment, and the real-time image data is acquired by the image data acquisition unit.

[0133] Then, the first image coordinates of the image marker points in the real-time image are automatically identified and obtained by the first transformation relationship calculation unit, and the optical coordinates of the optical markers in the optical coordinate system are obtained by the optical positioning unit.

[0134] The transformation relationship between the real-time image space and the optical coordinate system is then calculated using the coordinate system constructed from the first image coordinates and the coordinate system constructed from the optical coordinates, as well as the matrix transformation algorithm built into the first transformation relationship calculation unit. The matrix transformation algorithm is an existing transformation algorithm between two matrices, which will not be described in detail in this embodiment.

[0135] Then, corresponding marker points are selected on the preset image and the real-time image, namely the image marker points mentioned above. The second image coordinates of the marker points on the preset image are obtained through the second transformation relationship calculation unit. Based on the first image coordinates and the second image coordinates, the transformation relationship between the preset image space and the real-time image space is calculated by the matrix transformation algorithm.

[0136] Then, the registration unit obtains the conversion relationship between the preset image space and the optical coordinate system based on the conversion relationship between the preset image space and the real-time image space, as well as the conversion relationship between the real-time image space and the optical coordinate system, and completes the registration of the image and the optical coordinate system.

[0137] Finally, the electromagnetic positioning system and the structured light positioning system are registered with the image based on the conversion relationship between the optical positioning system, the electromagnetic positioning system and the structured light positioning system. For details, please refer to the registration method in the above-mentioned composite surgical navigation method.

[0138] The error calculation module 503 includes a first calculation unit, a second calculation unit, and a position determination unit.

[0139] The first calculation unit calculates the coordinates of the corresponding second reference position, third reference position, and first reference position based on the coordinates of the first, second, and third positions obtained by the positioning module and a preset transformation relationship. Specifically, the actuator is equipped with feature markers that match the corresponding coordinate system, and the relative positional relationships of each feature marker are known. The feature markers are optical markers, electromagnetic markers, and structured light markers.

[0140] By using feature markers, the first position of the actuator in the optical coordinate system can be acquired using an optical positioning system, the second position of the actuator in the electromagnetic coordinate system can be acquired using an electromagnetic positioning system, and the third position of the actuator in the structured light coordinate system can be acquired using a structured light positioning system.

[0141] The reference positions for this unit are obtained as follows:

[0142] The first position is calculated using the transformation relationship M1 between the first and second coordinate systems, resulting in a second reference position in the second coordinate system. Here, the second reference position is the theoretical position calculated using transformation relationship M1, while the second position is the actual detection position. By determining the deviation between the second position and the second reference position, the accuracy of the first and second positions can be verified.

[0143] Similarly, the third reference position of the second position in the third coordinate system is calculated based on the second position and the transformation relationship M2 between the second and third coordinate systems. Here, the third reference position is the theoretical position calculated using transformation relationship M2, and the third position is the actual detection position. By judging the deviation between the third position and the third reference position, the accuracy of the second and third positions can be determined.

[0144] Similarly, the first reference position of the third position in the first coordinate system is obtained based on the third position and the transformation relationship M3 between the third coordinate system and the first coordinate system. Here, the first reference position is the theoretical position calculated using transformation relationship M3, and the first position is the actual detection position. By judging the deviation between the first position and the first reference position, the accuracy of the first and third positions can be determined.

[0145] The second calculation unit calculates the first error, the second error, and the third error based on the coordinates of the first position and the first reference position, the second position and the second reference position, and the third position and the third reference position. In this embodiment, each of the above errors refers to the difference between the coordinates of the positioning position and the coordinates of the reference position in the x, y, and z directions. For example, if the coordinates of the first position are (x1, y1, z1) and the coordinates of the first reference position are (x2, y2, z2), then the first error is a1=|x1-x2|, b1=|y1-y2|, and c1=|z1-z2|.

[0146] The position determination unit is used to determine the current position of the actuator based on a comparison of the first error, the second error, and the third error with a preset threshold.

[0147] Specifically, the determination of the current location of the executing agency can be found in the description of the composite surgical navigation method above, and will not be repeated here.

[0148] In some embodiments, the device further includes an ultrasonic positioning module 504, which includes an ultrasonic probe disposed at the front end of the actuator. The frequency of the ultrasonic waves emitted by the ultrasonic probe is not limited; it can be high-frequency ultrasound or ordinary ultrasound. The ultrasonic probe moves synchronously with the operating trajectory of the actuator and acquires an ultrasonic image of the target during the synchronous movement. The ultrasonic image and the imaging image are then fused in real time by a computer device to form a first image model. The principle of ultrasonic image and imaging image fusion can be found in the description of the composite surgical navigation method above.

[0149] It should be understood that the understanding of each unit and module in this composite surgical navigation device can refer to the explanation in the above-mentioned composite surgical navigation method, and will not be repeated here.

[0150] Each module in the aforementioned composite surgical navigation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0151] See Figure 3 Based on the same inventive concept, this embodiment also provides a composite surgical navigation system, including a positioning system, an actuator, and a computer device;

[0152] The positioning system includes a tracker configured on the actuator and the target, and a tracking device matched with the tracker. The tracker is used to assist the tracking device in determining the position coordinates of the actuator and the target in a characteristic coordinate system.

[0153] The computer device includes a memory and a processor. The memory stores the computer program, and the processor executes the computer program to implement the steps S100-S400.

[0154] In this embodiment, the positioning system specifically includes an optical positioning system, an electromagnetic positioning system, and a structured light positioning system.

[0155] In some embodiments, the tracker in the optical positioning system is an optical tracker, specifically a plurality of physical optical spheres. The tracking device is a binocular infrared camera, which emits infrared light to the physical optical spheres and receives the infrared light reflected by the physical optical spheres, thereby accurately knowing the position of the physical optical spheres and determining the spatial position and attitude of the actuator.

[0156] It should be understood that existing optical positioning systems can also be used, such as the Polaris optical positioning and tracking system from NDI.

[0157] In some embodiments, the tracker in the electromagnetic positioning system is an electromagnetic tracker, specifically multiple electromagnetic sensors. The tracking device is an electromagnetic generator, which also includes an amplifier and a controller. The electromagnetic generator can produce a magnetic field with a known magnetic field strength in the surgical space. When the electromagnetic sensor enters the controllable variable magnetic field generated by the electromagnetic generator, the coil of the electromagnetic sensor will generate a potential difference. The potential difference is collected and amplified by the amplifier and then input into the controller to calculate the position and attitude of the electromagnetic sensor in the magnetic field, thereby determining the spatial position and attitude of the actuator.

[0158] It should be understood that existing electromagnetic positioning systems can also be used, such as NDI's Aurora positioning and tracking system.

[0159] In some embodiments, the tracker in a structured light positioning system may also use markers. However, in this embodiment, no tracker is used; instead, the surface feature points of the target and actuator are directly used as the tracking targets. The tracking device includes a 3D structured light camera, which projects infrared light onto the target and actuator, and then collects the data to obtain the surface features, depth, and height information of the target and actuator, thereby determining their positions. It should be understood that an existing 3D structured light system can also be used for the structured light positioning system.

[0160] In some embodiments, the positioning system further includes an ultrasonic positioning system, which includes an ultrasonic probe disposed at the front end of the actuator. The ultrasonic probe emits ultrasonic waves at an unrestricted frequency, which can be high-frequency ultrasound or ordinary ultrasound. The ultrasonic probe moves synchronously with the operating trajectory of the actuator and acquires ultrasonic images of the target during the synchronous movement. The ultrasonic images are then fused with the reconstructed model in real time using computer equipment to form a first image model.

[0161] The computer device can be a terminal. See also Figure 4 The computer device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a surgical navigation spatial registration method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0162] The computer device executes computer programs stored in the memory through a processor to implement steps S100-S400.

[0163] The steps for S100-S400 are described in the above description of the composite navigation method, and will not be repeated here.

[0164] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0165] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A composite surgical navigation device, characterized in that, include: The coordinate system registration module is used to acquire an image of the target and register the coordinate system of the image with a preset feature coordinate system. The feature coordinate system includes a first coordinate system, a second coordinate system, and a third coordinate system. The first coordinate system is a coordinate system established based on an optical positioning system, the second coordinate system is a coordinate system established based on an electromagnetic positioning system, and the third coordinate system is a coordinate system established based on a structured light positioning system. The positioning module is used to obtain the positioning positions of the actuator in the first coordinate system, the second coordinate system and the third coordinate system respectively. The positioning positions include the first position of the actuator in the first coordinate system, the second position in the second coordinate system and the third position in the third coordinate system. The error calculation module is used to calculate the second reference position of the first position in the second coordinate system based on the first position and the first transformation relationship between the first coordinate system and the second coordinate system; Based on the second position and the second transformation relationship between the second coordinate system and the third coordinate system, the third reference position of the second position in the third coordinate system is calculated; Based on the third position and the third transformation relationship between the third coordinate system and the first coordinate system, the first reference position of the third position in the first coordinate system is calculated; The error calculation module is further configured to: calculate a first error based on the first position and the first reference position, calculate a second error based on the second position and the second reference position, and calculate a third error based on the third position and the third reference position; if the first error is less than a first threshold and / or the second error is less than a second threshold, then determine the first position as the current position of the actuator; If the first error is greater than the first threshold, the second error is greater than the second threshold, and the third error is less than the third threshold, then the second position or the third position is determined to be the current position of the actuator.

2. The composite surgical navigation device according to claim 1, characterized in that, The imagery includes a preset image and a real-time image of the target, and the coordinate system registration module includes: The image data acquisition unit is used to acquire preset image data and real-time image data of the target, and to reconstruct a three-dimensional model based on the preset image data. The first transformation relationship calculation unit is used to obtain the first image coordinates of the image marker points in the real-time image, obtain the feature coordinates of the feature marker points in the feature coordinate system, and calculate the transformation relationship between the real-time image and the feature coordinate system based on the first image coordinates and the feature coordinates. The second conversion relationship calculation unit is used to obtain the second image coordinates of the image marker points in the preset image, and calculate the conversion relationship from the preset image to the real-time image based on the first image coordinates and the second image coordinates; The registration unit calculates the transformation relationship between the preset image and the feature coordinate system based on the transformation relationship between the real-time image and the feature coordinate system, and the transformation relationship between the preset image and the real-time image.

3. The composite surgical navigation device according to claim 1, characterized in that, The error calculation module includes: The first calculation unit is used to calculate the coordinates of the corresponding second reference position, third reference position and first reference position based on the coordinates of the first position, second position and third position obtained by the positioning module and the preset transformation relationship; The second calculation unit is used to calculate the first error, the second error, and the third error based on the coordinates of the first position and the first reference position, the second position and the second reference position, and the third position and the third reference position. The position determination unit is used to determine the current position of the actuator based on a comparison of the first error, the second error, and the third error with a preset threshold.

4. A composite surgical navigation device according to claim 2, characterized in that, The positioning module includes: An optical positioning unit is used to acquire the optical coordinates of an optical marker point in a first coordinate system. An electromagnetic positioning unit is used to obtain the electromagnetic coordinates of an electromagnetic marker point in the second coordinate system. The structured light positioning unit is used to obtain the structured light coordinates of the structured light marker points in the third coordinate system.

5. A composite surgical navigation device according to claim 4, characterized in that, The positioning module further includes: An ultrasonic positioning unit is used to acquire ultrasonic images scanned by an ultrasonic probe disposed at the front end of the actuator, and to fuse the ultrasonic images with the three-dimensional model.

6. A composite surgical navigation method, characterized in that, include: Acquire an image of the target and register the image with a preset feature coordinate system, wherein the feature coordinate system includes at least a first coordinate system, a second coordinate system, and a third coordinate system; the first coordinate system is a coordinate system established based on an optical positioning system, the second coordinate system is a coordinate system established based on an electromagnetic positioning system, and the third coordinate system is a coordinate system established based on a structured light positioning system. The positioning positions of an actuator are collected in the first coordinate system, the second coordinate system, and the third coordinate system, respectively; the positioning positions include the first position of the actuator in the first coordinate system, the second position in the second coordinate system, and the third position in the third coordinate system; Based on the first position and the first transformation relationship between the first coordinate system and the second coordinate system, the second reference position of the first position in the second coordinate system is calculated; based on the second position and the second transformation relationship between the second coordinate system and the third coordinate system, the third reference position of the second position in the third coordinate system is calculated; based on the third position and the third transformation relationship between the third coordinate system and the first coordinate system, the first reference position of the third position in the first coordinate system is calculated. A first error is calculated based on the first position and the first reference position; a second error is calculated based on the second position and the second reference position; and a third error is calculated based on the third position and the third reference position. If the first error is less than a first threshold and / or the second error is less than a second threshold, then the first position is determined to be the current position of the actuator. If the first error is greater than the first threshold, the second error is greater than the second threshold, and the third error is less than the third threshold, then the second position or the third position is determined to be the current position of the actuator.

7. The composite surgical navigation method according to claim 6, characterized in that, The image includes a preset image and a real-time image of the target, and the registration of the image with a preset feature coordinate system includes: Image markers and feature markers are configured on the implementation target, and a real-time image of the implementation target is acquired, as well as the image coordinates of the image markers in the real-time image and the feature coordinates of the feature markers in the feature coordinate system are acquired. The transformation relationship between the real-time image and the feature coordinate system is obtained based on the image coordinates and the feature coordinates; Based on the positions of the image markers in the preset image and the real-time image, respectively, the transformation relationship from the preset image to the real-time image is obtained; Based on the conversion relationship between the preset image and the real-time image, and the conversion relationship between the real-time image and the feature coordinate system, the conversion relationship between the preset image and the intraoperative feature coordinate system is obtained.

8. A composite surgical navigation system, characterized in that, This includes positioning systems, actuators, and computer equipment; The positioning system includes a tracker configured on the actuator and the target, and a tracking device matched with the tracker. The tracker is used to assist the tracking device in determining the position coordinates of the actuator and the target in a feature coordinate system. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method according to any one of claims 6 to 7.

Images