Image registration method based on magnetic navigation

Abstract

The application relates to the technical field of medical devices, and provides an image registration method based on magnetic navigation, which is applied to an image registration device, the image registration device can determine at least two projection images based on different angles, and comprises the following steps: calibrating a tool; establishing a plurality of cone beam models by using the calibration tool; constructing a positioning model according to the at least two cone beam models and the at least two projection images during a surgery process; and rendering an intraoperative instrument to the projection image based on the positioning model to realize intraoperative navigation. According to the method provided in the scheme, the calibration tool is calibrated before the surgery, the corresponding cone beam model is established by using the calibrated calibration tool, the real-time positioning and image registration of the intraoperative instrument can be realized without installing and dismounting the registration tool during the surgery process, the magnetic navigation technology is adopted to replace the optical navigation, the dependence on the line-of-sight penetration is eliminated, the intraoperative shielding and environmental light interference problems are effectively avoided, the stability and reliability of the intraoperative navigation are enhanced, and the surgery efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of medical device technology, and in particular to an image registration method based on magnetic navigation. Background Technology

[0002] Surgical robots are intelligent devices that assist surgeons during operations and are widely used in minimally invasive surgery and high-precision treatments. By combining image registration technology, surgical robots spatially align two or more medical images, matching preoperative or intraoperative medical images with the patient's actual anatomical location, thus providing surgeons with precise intraoperative navigation.

[0003] To assist in surgical procedures, image registration technology requires the installation and removal of registration fixtures during surgery. These fixtures are often based on a dual-plane structure and use an optical navigation array to achieve surgical navigation.

[0004] However, the installation and removal of the registration tool during surgery takes a long time, which increases the operation time and affects the efficiency of the operation. At the same time, optical navigation depends on clear line of sight, and in complex surgical environments, it is easily interfered with by factors such as obstructed line of sight and poor lighting, which can lead to a decrease in positioning accuracy or even failure, affecting the safety and stability of surgical navigation. Summary of the Invention

[0005] This application provides an image registration method based on magnetic navigation to avoid the problems of excessively long operation time for installing and removing registration tools during surgery, and low surgical navigation efficiency caused by easy interference with optical navigation.

[0006] This application provides an image registration method based on magnetic navigation, applied to an image registration device. The image registration device can determine at least two projected images based on different angles, including: calibrating a calibration fixture; establishing multiple cone-beam models using the calibration fixture; constructing a positioning model based on at least two cone-beam models and at least two projected images during surgery; and rendering intraoperative instruments onto the projected images based on the positioning model to achieve intraoperative navigation.

[0007] By implementing the above scheme, the calibration fixture can be calibrated before surgery, and a corresponding cone-beam model can be established using the calibrated fixture. This allows for real-time instrument positioning and image registration during surgery without the need to install or remove the registration fixture, thus improving surgical efficiency and navigation accuracy. At the same time, magnetic navigation technology is used to replace optical navigation, eliminating the dependence on visual transparency and effectively avoiding problems such as intraoperative obstruction and ambient light interference, thereby enhancing the stability and reliability of intraoperative navigation.

[0008] In one feasible implementation, the calibration fixture includes multiple markers. Calibrating the calibration fixture involves: setting the coordinate deviation between the designed and actual coordinate values ​​of the multiple markers; establishing a mapping relationship between the designed and actual coordinate systems based on the set coordinate deviations; acquiring the first and second measured coordinate values ​​corresponding to each marker using a calibration device; setting a residual term based on the distance difference between the first and second measured coordinate values; determining an objective function based on the residual term; and determining the actual coordinate value of each marker based on the objective function and the designed coordinate value of each marker. This enables high-precision calibration of the calibration fixture, improves the accuracy of modeling the spatial positional relationships of the markers, thereby enhancing the construction accuracy of the cone-beam model, reducing systematic errors in subsequent registration processes, and providing a reliable spatial mapping basis for intraoperative real-time navigation.

[0009] In one feasible implementation, the calibration fixture includes multiple markers. Multiple cone-beam models are established using the calibration fixture, which is positioned within the imaging range of the image registration device. The process includes: acquiring the actual coordinate values ​​of the markers; aligning the projected image with a physical plane, where the physical plane includes any one of the planes on the standard fixture with the markers; constructing a virtual projection model based on the alignment result of the projected image and the physical plane; the virtual projection model includes an optical center and a virtual imaging plane; determining the parameters corresponding to the cone-beam model based on the optical center and the virtual imaging plane; and transforming the coordinate system corresponding to the cone-beam model to the receiving coordinate system, where the receiving coordinate system is the coordinate system determined by the image registration device based on the receiving end. In this way, through model establishment and coordinate system transformation, a unified expression of the cone-beam model in the receiving coordinate system is achieved, fixing the cone-beam model in the coordinate system of the image registration device, ensuring the consistency and fusionability of intraoperative multi-view images and spatial positioning data, and reducing preoperative preparation time.

[0010] In one feasible implementation, the calibration fixture includes at least two physical planes. Aligning the projected image with the physical planes includes: acquiring the image coordinates corresponding to markers in the at least two physical planes; and determining the homography matrix between the physical coordinates corresponding to the markers and the image coordinates corresponding to the markers in the at least two physical planes. This enables precise alignment between the projected image and the physical planes, improving the geometric accuracy of the cone-beam model construction.

[0011] In one feasible implementation, a virtual projection model is constructed based on the alignment results between the projected image and the physical plane, including mapping the image coordinates corresponding to the markers to the virtual imaging plane according to the homography matrix. This ensures that the imaging plane in the virtual projection model maintains a strict geometric correspondence with the actual physical structure, improving the accuracy of subsequent determination of various parameters of the cone-beam model.

[0012] In one feasible implementation, a virtual projection model is constructed based on the alignment results of the projected image and the physical plane. This further includes: uniformly sampling the plane corresponding to the projected image to obtain a set of sampling points, the set of sampling points comprising multiple sampling points; mapping each sampling point to at least two physical planes; constructing a physical space line based on the mapped coordinates of each sampling point on the at least two physical planes; and determining the intersection of the physical space lines corresponding to each sampling point as the optical center. This effectively improves the accuracy of optical center positioning, reduces model deviations caused by projection distortion or calibration errors, and thus enhances the reliability and stability of intraoperative navigation and positioning.

[0013] In one feasible implementation, multiple cone-beam models are established using calibration fixtures, and the method further includes: establishing multiple cone-beam models corresponding to different projection angles based on the projection angles of the projected images. This enables the construction of cone-beam models under multiple projection angles, meeting the needs of intraoperative multi-view imaging and improving the integrity and spatial resolution of 3D reconstruction.

[0014] In one feasible implementation, during the surgery, a localization model is constructed based on at least two cone-beam models and at least two projection images. This includes: selecting at least two cone-beam models from a plurality of cone-beam models; collecting patient-related data during the surgery, including a reference coordinate system; transforming the coordinate systems corresponding to the at least two cone-beam models to the reference coordinate system; and reconstructing the projection images corresponding to the at least two cone-beam models in three dimensions to construct the localization model. This allows for patient data acquisition and transformation during surgery, enabling precise fusion of cone-beam images from multiple projection angles within a unified coordinate system, thus improving the coordinated accuracy of lesion localization and surgical instrument navigation.

[0015] In one feasible implementation, during the surgery, a positioning model is constructed based on at least two cone-beam models and at least two projected images. The method further includes: in response to the patient entering the imaging area, sending a magnetic sensor installation prompt, which guides the user to install a fourth magnetic sensor on the patient; and in response to the completion of the fourth magnetic sensor installation, determining a reference coordinate system based on the coordinate system corresponding to the fourth magnetic sensor. This enables the real-time establishment of a patient-specific reference coordinate system, achieving intraoperative dynamic tracking and precise registration of multimodal images.

[0016] In one feasible implementation, the image registration device further includes a third magnetic sensor and a second magnetic base station. During the surgery, it collects patient-related data, including: determining at least two projected images of the patient based on the projection angles corresponding to at least two cone-beam models; obtaining the pose of the third magnetic sensor in the magnetic base station coordinate system corresponding to the second magnetic base station based on the projection angles corresponding to at least two cone-beam models; and obtaining the pose of the fourth magnetic sensor in the magnetic base station coordinate system. This allows for real-time acquisition of patient-related data during surgery, facilitating the image registration device's performance of image registration and localization model construction tasks.

[0017] In one feasible implementation, three-dimensional reconstruction of the projection images corresponding to at least two cone-beam models to construct a positioning model includes: in response to a selection operation, acquiring feature points corresponding to the same feature in the projection images corresponding to at least two cone-beam models, wherein the number of feature points corresponding to the same feature is the same as the number of selected cone-beam models; determining the three-dimensional coordinates of the feature points in a reference coordinate system; and constructing a positioning model based on the three-dimensional coordinates of the feature points. This allows for spatial reconstruction of the same anatomical feature using projection rays from different projection angles, obtaining the three-dimensional position of the feature point in the patient coordinate system, achieving the three-dimensional reconstruction task of two-dimensional projection images, and obtaining a positioning model for intraoperative navigation, providing high-precision spatial positioning support for subsequent surgical path planning and real-time guidance.

[0018] In one feasible implementation, based on a positioning model, the intraoperative instruments are rendered onto the projected image, including: acquiring the poses of the intraoperative instruments and the fourth sensor in the magnetic base station coordinate system; transforming the poses of the intraoperative instruments to a reference coordinate system based on the poses of the fourth sensor in the magnetic base station coordinate system; determining the projected coordinates of the intraoperative instruments corresponding to the projected image based on the reference coordinate values ​​of the intraoperative instruments and at least two cone-beam models; and rendering the intraoperative instruments onto the projected coordinate region. This allows the real-time position of the intraoperative instruments to be displayed on the two-dimensional projected image, enabling dynamic tracking of the intraoperative instrument poses and synchronous updates of the projected image. This ensures consistency between instrument movement and image feedback during surgery, facilitating a more intuitive understanding of the spatial relationship between the surgical instruments and the lesion, and improving the accuracy of surgical navigation. Attached Figure Description

[0019] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic flowchart illustrating an image registration method based on magnetic navigation provided in an embodiment of this application; Figure 2This is a schematic diagram of the structure of a calibration fixture provided in an embodiment of this application; Figure 3 A schematic diagram of a calibration device and calibration fixture provided for an embodiment of this application; Figure 4 A schematic flowchart of a calibration fixture provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an image registration device provided in an embodiment of this application; Figure 6 A schematic flowchart illustrating the calibration of a cone-beam model provided in this application embodiment; Figure 7 A cone-beam diagram of a calibration cone-beam model provided in this application embodiment; Figure 8 This is a schematic diagram of a uniform sampling method provided in an embodiment of this application; Figure 9 A schematic diagram illustrating intraoperative image registration and positioning navigation provided for an embodiment of this application; Figure 10 This is a schematic diagram of an intraoperative positioning model construction process provided in an embodiment of this application. Detailed Implementation

[0021] The embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but should not be used to limit the scope of this application.

[0022] In the description of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. The "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences.

[0023] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0024] A medical robot is an intelligent device used to assist medical staff in performing surgical procedures. By combining image registration technology and robotic arm control technology, it achieves the positioning and navigation of surgical instruments during surgery. The medical robot can acquire the real-time position information of surgical instruments during surgery and provide visual feedback to medical staff, thereby improving the accuracy and safety of the surgery.

[0025] Image registration technology is a core component for medical robots to achieve intraoperative positioning and navigation. This technology can align at least two medical images in space to unify medical image data under the same coordinate system, providing a reference for the navigation of surgical instruments during surgery.

[0026] Image registration methods used in medical robots acquire patients' medical images before or during surgery, and spatially align and verify medical images of different modalities or temporal phases based on artificial markers or natural anatomical features, ultimately achieving surgical navigation. Based on the execution process of image registration methods, medical robots are equipped with modules for locating spatial coordinates to achieve spatial alignment of medical images and subsequent surgical navigation.

[0027] Traditionally, optical navigation modules are used to locate spatial coordinates. They can detect or receive signals from markers fixed in surgical instruments and patients using devices such as infrared optical cameras or visible light cameras, and determine the spatial coordinates of each marker based on the principle of triangulation, thereby determining the position and posture of the surgical instruments and patients.

[0028] However, optical navigation modules rely on unobstructed vision during surgery, making them susceptible to interference from factors such as obstructed vision and poor lighting in complex surgical environments. This leads to decreased positioning accuracy and affects the navigation performance of the medical robot during surgery. Furthermore, image registration methods based on optical navigation typically require the installation and removal of registration fixtures during surgery, impacting surgical efficiency, increasing operational complexity, extending surgical time, and raising the risks associated with complex surgeries.

[0029] To address the aforementioned issues, this application provides an image registration method and system based on magnetic navigation. By calibrating the tooling preoperatively and determining its position in the magnetic navigation space, the installation and disassembly of the tooling during surgery is reduced. Simultaneously, based on magnetic navigation technology, the relative positions of surgical instruments and the patient's anatomical structures are captured in real time, achieving high-precision spatial registration and surgical navigation. This reduces intraoperative procedures, shortens surgical preparation time, and effectively avoids the line-of-sight obstruction problem inherent in optical navigation.

[0030] The technical solutions provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0031] Figure 1This is a flowchart illustrating an image registration method based on magnetic navigation provided in an embodiment of this application.

[0032] like Figure 1 As shown, the image registration method provided in this application embodiment includes the following steps S100 to S400.

[0033] S100: Calibration fixture.

[0034] The calibration fixture refers to a physical device containing specific marker points. Based on the calibration fixture, a known and stable geometric reference can be provided for subsequent image registration and surgical navigation, achieving a precise correspondence between image coordinates and real space. In the embodiments of this application, the specific marker points in the calibration fixture are magnetic induction marker points, which can be detected by magnetic sensors and identified in their position and orientation in three-dimensional space.

[0035] The calibration fixture, based on the principle of magnetic induction, can determine the precise coordinates of each magnetic induction marker point in the magnetic navigation space, thereby unifying the image coordinate system with the real-world coordinate system. It should be understood that the calibration fixture process is completed preoperatively. By placing the calibration fixture within the magnetic navigation working area of ​​the medical robot system, the magnetic field signals of each magnetic induction marker point are collected using a magnetic positioning sensor array. Combined with a pre-set magnetic field model, the three-dimensional coordinates and attitude information of these marker points in the magnetic navigation space are calculated, and this coordinate information is used as a reference for subsequent image registration.

[0036] Figure 2 This is a schematic diagram of a calibration fixture provided in an embodiment of this application. The following is in conjunction with… Figure 2 The specific structure of the calibration fixture in the embodiments of this application will be described.

[0037] like Figure 2 As shown, the calibration fixture 20 provided in this application may include a base 21, markers 22, and a first magnetic sensor 23. The base 21 can support multiple markers 22 and the first magnetic sensor 23, enabling operators to calibrate the medical robot system using the calibration fixture 20. The base 21 is made of non-metallic and non-magnetic material to avoid interference with the magnetic field signal used during calibration, ensuring that the calibration device can accurately capture the relative position and attitude information between the markers 22 and the first magnetic sensor 23 during the calibration process.

[0038] In this embodiment, the base 21 may be a polyhedral structure having at least two sets of opposing surfaces. Each of these opposing surfaces may have at least three markers 22, and the markers on each surface are non-collinearly distributed to ensure the uniqueness and stability of spatial positioning. For example, at least three markers 22 on the same surface may form a non-collinear triangular structure, thereby constituting a stable geometric reference in space.

[0039] Furthermore, the polyhedral structure also has a plane enclosed by an edge of multiple surfaces provided with multiple markers 22. The plane may be provided with a first magnetic sensor 23. The calibration device can achieve the calibration effect based on the markers 22 and the first magnetic sensor 23 provided on each surface of the base 21.

[0040] For example, the base 21 may be a hexahedral structure, such as a cubic structure, a square prism structure, etc., having three sets of opposing surfaces. Two sets of opposing surfaces are used to support multiple markers 22, and one of the remaining sets of surfaces is used to support the first magnetic sensor 23. It should be understood that the embodiments of this application and... Figure 2 The base 21 shown is a hexahedral structure, which is only one example provided by this application. In actual applications, the base 21 can also be an octahedral structure, a decahedral structure, etc., and this application does not impose any restrictions here.

[0041] The marker 22 can be made of a non-magnetic metallic material, enabling it to be accurately detected and its spatial position identified by the first magnetic sensor 23 and the image registration device. For example, the marker 22 can be a small ball or cylinder made of a cermet material or a non-magnetic metal such as titanium, which can enhance the identification accuracy and avoid interference from magnetic materials with the magnetic field signal used in the calibration process. In this embodiment, the marker 22 can be a titanium ball.

[0042] One of the surfaces of the base 20 that does not have a marker 22 is provided with a slot, and the first magnetic sensor 23 is provided in the slot. The first magnetic sensor 23 can be used to receive and feedback magnetic field signals.

[0043] For example, all four surfaces with markers 22 can be in contact with the surface with the first magnetic sensor 23. Taking the base 21 as a cubic structure as an example, if the surface with the first magnetic sensor 23 is taken as the front of the base 21, then the four surfaces with markers 22 are the top surface, bottom surface, left side surface and right side surface of the base 21.

[0044] In some embodiments of this application, the markers 22 disposed on the same surface of the base 20 can also be enclosed to form a circular structure, thereby making the spatial distribution of the markers 22 more symmetrical and stable in recognition, and facilitating higher positioning accuracy and anti-interference capability.

[0045] It should be understood that, Figure 2 The structure of the calibration fixture 20 and the materials of its components shown in the foregoing embodiments are merely illustrative. In practical applications, the structural parameters of the calibration fixture 20 can be adaptively adjusted according to the accuracy requirements of the magnetic navigation system and the surgical scenario. Similarly, the number and arrangement of the markers 22 in the calibration fixture 20 can be flexibly configured according to actual needs. For example, the density of the markers 22 can be increased to improve local resolution, or their geometric arrangement can be adjusted to adapt to different magnetic field distribution characteristics.

[0046] In this embodiment, since the calibration fixture 20 is a spatial reference body with a defined spatial geometry and a corresponding design coordinate system, the spatial position of each marker 22 can be accurately modeled through the design coordinate system.

[0047] However, there may be deviations between the designed coordinate system and the actual spatial position of the marker 22 in real space due to installation errors or material deformation. If the designed coordinate system is applied directly, the idealized position in the designed coordinate system will be used as the real reference, introducing the deviation between the designed and actual coordinate systems into all subsequent stages, ultimately leading to a decrease in the accuracy of surgical navigation. Therefore, the calibration fixture 20 needs to be calibrated using a high-precision calibration method to maintain its spatial positioning accuracy in practical applications.

[0048] For example, the calibration process of the calibration fixture 20 can use high-precision equipment such as a coordinate measuring machine to accurately collect the actual spatial coordinates of each marker 22 and the first magnetic sensor 23 on the calibration fixture 20, and compare them with the theoretical positions in the design coordinate system, thereby establishing an error compensation model.

[0049] Figure 3 This is a schematic diagram of a calibration device and calibration fixture provided in an embodiment of this application. Figure 4 This is a schematic flowchart illustrating a calibration fixture provided in an embodiment of this application. The following is in conjunction with... Figure 3 and Figure 4 The calibration procedure for calibration fixture 20 is explained.

[0050] like Figure 4 As shown, the process of calibrating the calibration fixture 20 using the calibration device 30 may include the following steps S110 to S160.

[0051] S110: Set the coordinate deviation between the design coordinate values ​​and the actual coordinate values ​​of the markers in the calibration fixture.

[0052] In this embodiment of the application, since each coordinate system is used to identify the position and orientation of the marker 22 in physical space, the origin and coordinate axis direction of each coordinate system must strictly correspond to the actual physical layout, and have three mutually perpendicular coordinate axes, namely x, y, and z axes.

[0053] Therefore, the coordinates of each marker 22 of the calibration fixture 20 in the design coordinate system provided in this application embodiment can be set as follows: The actual coordinates of each marker 22 can be represented as: Where i is the index of the marker, indicating the i-th marker, and the actual coordinates refer to the real spatial coordinates of marker 22 in real space.

[0054] Based on the coordinate representation of each marker 22 in the design coordinate system and the actual coordinate system, the coordinate deviation corresponding to each marker 22 can be expressed as: .in, Can be and The difference between them, correspondingly, Can be and The difference between them Can be and The difference between them.

[0055] In this embodiment, the total change in coordinate deviations corresponding to all markers 22 can be used as the parameter to be optimized for the calibration fixture 20 for parametric modeling. Specifically, the parameter to be optimized... It can be expressed by the following formula (1): (1); Where n is the total number of markers 22 on the calibration fixture 20, These are the parameters to be optimized, used to describe the overall spatial deviation between the design coordinate system and the actual coordinate system. The superscript T indicates the transpose of the vector. That is, the parameters to be optimized are a column vector containing the coordinate deviations of all markers in the x, y, and z directions. The transposed parameters are... It is in column vector form, which facilitates subsequent matrix operations and optimization solutions.

[0056] It should be noted that the determination of the above-mentioned parameters to be optimized is a preprocessing step before the calibration equipment 30 calibrates the calibration fixture 20, so as to provide a mathematical basis for the subsequent optimization steps. The introduction of this parameter transforms the calibration process from a simple coordinate comparison into a quantifiable error optimization problem, thereby achieving accurate alignment of the coordinate system by minimizing the overall deviation.

[0057] S120: Based on the setting of coordinate deviation, establish the mapping relationship between the design coordinate system and the actual coordinate system.

[0058] After determining the parameters to be optimized, since each marker 22 in the design coordinate system and the actual coordinate system has two coordinate values, namely the design coordinate value and the actual coordinate value, there is a coordinate deviation between the two and no actual transformation relationship. Therefore, it is not possible to directly obtain the actual coordinate value from the design coordinate value. Thus, it is necessary to establish a spatial transformation relationship between the two.

[0059] In this embodiment, the calibration device 30 can define a rigid body transformation matrix to describe the mapping relationship between the design coordinate system and the actual coordinate system. The rigid body transformation matrix includes two parts: rotation and translation. That is, the design coordinate values ​​in the design coordinate system can be transformed by rotation and translation to obtain the corresponding actual coordinate values ​​in the actual coordinate system.

[0060] Specifically, based on the rigid body transformation matrix, the mapping relationship between the design coordinate system C0 and the actual coordinate system C1 can be expressed as the following formula (2).

[0061] (2); in, This refers to the design coordinate value of the i-th marker 22 in the design coordinate system C0, while Let T represent the actual coordinates of the i-th marker 22 in the actual coordinate system C1, and let T represent the rigid body transformation matrix.

[0062] In some embodiments, the rigid body transformation matrix T can be composed of a rotation matrix R and a translation vector t, and can be expressed as T = [R | t]. Its function is to map points in the design coordinate system to the actual coordinate system after rotation and translation. By introducing the rigid body transformation matrix T, a systematic association between the design coordinate system and the actual coordinate system can be achieved.

[0063] It should be understood that the rigid body transformation matrix T is only one implementation method in the embodiments of this application. In actual applications, the rigid body transformation matrix can also be represented in other forms, which will not be elaborated here.

[0064] S130: Using calibration equipment, obtain the first and second measurement coordinate values ​​corresponding to each marker.

[0065] Because subsequent surgical navigation requires determining coordinate data and navigation based on magnetic navigation, if only magnetic sensors are used for calibration during the calibration fixture 20, the errors generated by the sensors and / or magnetic fields in the magnetic navigation system cannot be eliminated due to inherent errors in the magnetic sensors and magnetic field distortion. Therefore, in this embodiment, a higher-precision coordinate measurement method is needed to measure and calibrate the coordinates of each marker 22 and the first magnetic sensor 23 in the calibration fixture 20.

[0066] Specifically, the calibration device 30 can measure the coordinate position of the same marker 22 on the calibration fixture 20 in two different ways, which are recorded as the first measurement coordinate value and the second measurement coordinate value, respectively. The first measurement coordinate value can be high-precision coordinate data obtained based on a precision coordinate measuring instrument, while the second measurement coordinate value is coordinate data obtained based on magnetic field sensing measurement.

[0067] like Figure 3 As shown, the calibration device 30 may include a coordinate measuring instrument 31, a probe 32, and a first magnetic base station 33. The coordinate measuring instrument 31 is used to acquire the high-precision spatial coordinates of the marker 22, i.e., the first measured coordinate value. The probe 32 is used to collect the magnetic induction coordinate data of each marker 22 in the calibration fixture 20, i.e., the second measured coordinate value. The first magnetic base station 33 is used to provide a stable magnetic field, enabling the magnetic sensor to sense the position of the marker 22 in the magnetic field and determine the corresponding coordinate data.

[0068] The probe 32 may include a second magnetic sensor 321, which enables the acquisition of magnetic induction coordinate data of each marker 22 in the calibration fixture 20. Based on the stable magnetic field generated by the first magnetic base station 33, the second magnetic sensor 321 in the probe 32 can measure the response signals of the first magnetic sensor 23 and each marker 22 to the magnetic field, thereby obtaining their second measurement coordinate values ​​in the magnetic field.

[0069] It should be noted that before the probe 32 collects the coordinates of the marker 22 on the calibration fixture 20 based on the magnetic field, it also needs to be calibrated to eliminate errors and interference and ensure measurement accuracy.

[0070] During the calibration process of the calibration fixture 20, the standard fixture 20 can first be set within the measurement range of the calibration device 30, and the relative positional relationship between the calibration device 30 and the calibration fixture 20 can be determined so that the marker 22 and the first magnetic sensor 23 in the calibration fixture 20 can be accurately identified and measured by the calibration device 30.

[0071] Then, the calibration device 30 can calibrate the first magnetic sensor 23 and each marker 22 using the coordinate measuring instrument 31 and the probe 32 respectively, and determine the first and second measurement coordinate values ​​corresponding to the first magnetic sensor 23 and each marker 22. Both the coordinate measuring instrument 31 and the probe 32 can determine the measured coordinate data of the first magnetic sensor 23 as the origin of the coordinate system, and thus use this as a reference to unify the positional relationship of all markers 22 in the coordinate system.

[0072] After the calibration device 30 obtains the first and second measurement coordinate values ​​of the calibration fixture 20 at the position, the calibration fixture 20 can be moved along a preset trajectory to the next calibration position, and the above measurement process can be repeated to obtain another set of first and second measurement coordinate values.

[0073] It should be understood that the coordinate data of each component in the calibration fixture 20 obtained by the coordinate measuring instrument 31 and the probe 32 in the calibration device 30 are coordinate values ​​used to optimize the actual coordinate system. Through subsequent optimization processes, the actual coordinate system C1 of the calibration fixture 20 can be determined based on the coordinate system obtained by the coordinate measuring instrument 31 measuring the calibration fixture 20 and the coordinate system obtained by the probe 32 measuring the calibration fixture 20.

[0074] S140: Set the residual term based on the distance difference between the first and second measured coordinate values.

[0075] After obtaining two sets of measured coordinate values, the residual term can be determined by calculating the distance difference between the first and second measured coordinate values, and the deviation between the coordinate values ​​measured by the second magnetic sensor 321 and the coordinate measuring instrument 31 can be obtained.

[0076] Taking two sets of measurement data from the calibration fixture 20 at positions m and n as an example, the calibration device 30 can determine the true value based on the first measurement coordinate values ​​of the calibration fixture 20 at positions m and n. The specific calculation method is as follows (3).

[0077] (3); in, This refers to the first measured coordinate value of the i-th marker 22 when the calibration fixture 20 is at position m. This refers to the first measured coordinate value of the i-th marker 22 when the calibration fixture 20 is at position n. (True value) The distance between the i-th marker 22 of the calibration fixture 20 at positions m and n can be determined by calculating the Euclidean distance, which can be determined by calculating the square root of the sum of squares of the coordinate differences.

[0078] Based on truth value The calibration device 30 can determine the coordinate changes of each marker 22 on the calibration fixture 20 after it has moved from position m to position n. After obtaining the true value... Afterwards, the calibration device 30 can be based on the true value. The deviation value of the measurement result of the second magnetic sensor 321 is calculated by the coordinate change corresponding to the second measured coordinate value, and the residual term is obtained.

[0079] In the embodiments of this application, the residual term It can be determined by the following formula (4): (4); in, This refers to the second measured coordinate value of the i-th marker 22 when the calibration fixture 20 is at position m. This refers to the second measurement coordinate value of the i-th marker 22 when the calibration fixture 20 is at position n.

[0080] It should be understood that the residual term It is the difference between the Euclidean distance between the second measured coordinate values ​​of each marker 22 at positions m and n, and the Euclidean distance between the first measured coordinate values ​​of each marker 22 at positions m and n. According to formulas (3) and (4), when the second magnetic sensor 321 is working normally and without error, the first and second measured coordinate values ​​of the same marker 22 at position m are the same, and the first and second measured coordinate values ​​of the same marker 22 at position n are also the same. At this time, the residual term... The value is 0.

[0081] Therefore, the calibration device 30 can be achieved through the residual term. The value determines the deviation between the result measured by probe 32 and the result measured by coordinate measuring instrument 31. When the residual term... When the value is 0, it indicates that there is no deviation between the two, and the measurement result of probe 32 has the highest accuracy; when the residual term is 0, it means that there is no deviation between the two, and the measurement result of probe 32 has the highest accuracy. When it is not 0, the residual term The smaller the absolute value, the smaller the deviation between the result measured by probe 32 and the result measured by coordinate measuring instrument 31, and the higher the accuracy of the measurement result of probe 32.

[0082] S150: Determine the objective function based on the residual term.

[0083] After obtaining the residual terms for each marker 22, the calibration device 30 can determine the objective function using the values ​​corresponding to the residual terms. Specifically, the expression for the objective function can be the following formula (5): (5); In this context, the first of the three summation symbols refers to summing the coordinates of the markers 22 on the calibration fixture 20, where M represents the number of markers 22 on the calibration fixture 20; the latter two summation symbols sum over all possible pose combinations of the calibration fixture 20, where P represents the number of possible placement positions of the calibration fixture 20. This is to avoid summing when the positions are the same.

[0084] Taking the coordinates of the calibration fixture 20, which determines the m and n positions respectively, provided in the aforementioned embodiment as an example, when summing in formula (5), the square values ​​of the residual terms corresponding to the two positions of each marker 22 can be summed to obtain the objective function.

[0085] It should be understood that the objective function is constructed and the subsequent optimization process is performed to minimize the sum of the residual terms corresponding to each marker 22, so that the measured coordinate values ​​detected by the probe 32 are close to the true values.

[0086] S160: Based on the objective function and the design coordinates of each marker, determine the actual coordinates of each marker.

[0087] In this embodiment, the design coordinates of each marker 22 can be used as initial values, and the objective function can be iteratively optimized using the Levenberg-Marquardt (LM) algorithm. Because the LM algorithm is suitable for stably solving nonlinear least squares optimization problems, in this embodiment, the parameters to be optimized can be optimized through iterative calculation of the objective function.

[0088] After the objective function meets the convergence condition, the final result is the high-precision coordinate values ​​of all markers 22 in the actual coordinate system C1. In this embodiment, the convergence condition may include any one of the following: the objective function parameter update step size is less than a threshold, the residual term decrease is less than a threshold, and the maximum number of iterations is reached. This embodiment does not limit the method for determining whether the objective function meets the convergence condition.

[0089] S200: Use calibration fixtures to determine multiple cone-beam models.

[0090] After calibrating the calibration fixture 20, the image registration equipment needed during the operation can be calibrated using the calibrated fixture 20, enabling the image registration equipment to establish a high-precision projection geometry model. After establishing the corresponding model based on the calibration fixture 20, the image registration equipment can convert and store the model for direct application in subsequent surgical procedures, avoiding intraoperative registration and calibration.

[0091] In this embodiment, the model for establishing and calibrating the image registration device can be a cone-beam model, which is a set of mathematical parameters used to describe the imaging process of the image registration device. By establishing and calibrating the cone-beam model, a precise mapping relationship between the detector image plane and the physical space corresponding to the image registration device can be established, providing a coordinate transformation method for subsequent surgical navigation.

[0092] Figure 5 This is a schematic diagram of an image registration device provided in an embodiment of this application. The following is based on... Figure 5 The structure shown illustrates the image registration device and its corresponding cone-beam model.

[0093] like Figure 5 As shown, the image registration device 50 can be a C-arm X-ray machine or other medical imaging device with a C-arm structure. Therefore, the image registration device 50 includes a C-arm 51, which is used to provide the function of acquiring projected images from different angles, so as to facilitate the construction of a spatial model during the operation to realize the navigation function.

[0094] The C-arm 51 includes an imaging element and a physical carrier. The imaging element includes a transmitting end and a receiving end. The physical carrier is a C-shaped rigid structure, and there are bearing end faces at both ends of the C-shaped structure, which are used to bear the transmitting end and the receiving end of the imaging element, respectively.

[0095] Meanwhile, the image registration device 50 is also equipped with a second magnetic base station 52. The function of the second magnetic base station 52 is the same as that of the first magnetic base station 33, which is to provide a stable magnetic field so that the various magnetic sensors can acquire and switch coordinate values ​​through magnetic field induction.

[0096] Based on the cone-beam imaging principle of the C-arm 51, it is known that the rays of the cone-beam projection can be emitted from the transmitting end and captured by the receiving end. The image registration device 50 can obtain the cone-beam model by determining the imaging process of the C-arm 51.

[0097] Figure 6 This is a schematic flowchart illustrating a calibration cone-beam model provided in an embodiment of this application. The following is based on... Figure 5 The image registration device 50 shown in the figure and Figure 6 The process shown in the document provides an exemplary illustration of how to calibrate a cone-beam model.

[0098] like Figure 6 As shown, the image registration device 50 performs a cone-beam model calibration process based on the calibration fixture 20, which may include the following steps S210 to S270.

[0099] S210: Set the calibration fixture within the imaging range of the image registration device.

[0100] Before establishing the cone-beam model, the calibrated calibration fixture 20 can be placed within the imaging range of the image registration device 50, so that the cone-beam model obtained and calibrated in this embodiment includes information related to the calibration fixture 20.

[0101] Figure 7 This is a schematic diagram of a cone-beam calibration model provided in an embodiment of this application. The following is in conjunction with… Figure 5 , Figure 6 and Figure 7 The process of calibrating the cone-beam model of the image registration device 50 is explained.

[0102] like Figure 7 As shown, based on the cone-beam imaging principle of the C-arm 51, the rays of the cone-beam projection are emitted from the transmitting end, pass through the calibration fixture 20 in a radial pattern, and are captured by the receiving end, forming a two-dimensional image 70 containing spatial geometric information. By using the shape corresponding to the marker 22 in the formed two-dimensional image 70, the projection coordinates of each marker 22 in the two planes in the two-dimensional projection are determined. Combined with the known physical coordinates, the mapping relationship between pixel coordinates and three-dimensional physical coordinates is established using the projection geometry model.

[0103] In the embodiments of this application, see Figure 5 and Figure 7 The calibrated fixture 20 can be placed within the imaging area of ​​the C-arm 51, specifically between the transmitting and receiving ends of the imaging element. Then, using the calibration results from the aforementioned steps, the magnetic coordinates of the markers 22 on the two planes of the calibration fixture 20 are obtained. For example, as shown... Figure 5 As shown, based on the imaging area of ​​the C-arm 51, two planes parallel to the bearing end face in the calibration fixture 20 can be identified as the two planes in the calibration fixture 20 whose physical coordinates need to be acquired, and the three-dimensional physical coordinates of each marker 22 on the two planes in the magnetic sensor coordinate system can be determined by the first magnetic sensor 23.

[0104] It should be noted that the aforementioned process of determining the two planes for obtaining physical coordinates in the calibration fixture 20 is only a feasible implementation method provided by the embodiments of this application, and the method of determining the planes is not limited in this application.

[0105] S220: Obtain the actual coordinates of the marker.

[0106] like Figure 2 As shown, the calibration fixture 20 calibrated in step S100 includes two known, non-coplanar planes, each bearing multiple markers 22. In this embodiment, the image registration device 50 can obtain the physical coordinates of each marker 22 in the two corresponding planes of the calibration fixture 20 through the calibration results of the aforementioned steps.

[0107] Based on the calibration process of model 20 in step S100, the physical coordinate values ​​of the markers 22 on the two planes of the calibration fixture 20 in the magnetic sensor coordinate system corresponding to the first magnetic sensor 23 can be obtained. Specifically, the physical coordinate values ​​of the markers 22 on different planes can be expressed as follows: and , where i and j both refer to the number of markers 22 in a plane. When there are N markers 22 in a plane, the values ​​of i and j range from 1 to N.

[0108] S230: Align the projected image with the physical plane.

[0109] After obtaining the physical coordinates of the two planes, the C-arm 51 can be controlled to fluoroscopically calibrate the fixture 20 at an α angle, acquiring its two-dimensional projection image at that angle, and extracting the pixel coordinates of the markers 22 in the image. Then, by analyzing the distribution characteristics of the markers 22 in the plane, the correspondence between each marker 22 in the two-dimensional projection image and its physical coordinates is determined, aligning the medical image with the physical plane.

[0110] For example, the image coordinates of the marker projection 71 obtained based on perspective can be represented as follows: and ,in, and This refers to the image coordinate vector of the marker projection 71 corresponding to the marker 22 in the upper and lower planes of the calibration fixture 20. and This refers to the pixel position of the marker 22 in the corresponding plane in the projected image. i and j both refer to the number of markers 22 in a plane. When there are N markers 22 in a plane, the values ​​of i and j range from 1 to N.

[0111] In the embodiments of this application, using Figure 7 The cone beam direction shown in the figure and Figure 2 Taking the arrangement of markers 22 shown in the figure as an example, with the transmitting end of the cone beam as the bottom, the marker projection 71 in the image can be the image formed by the markers 22 on the upper and lower planes of the base 21 of the calibration fixture 20 based on the cone beam projection.

[0112] Because the marker 22 shown in this embodiment can be a titanium ball, in the projection formed by the rays sent by the image registration device 50 based on the calibration fixture 20, each marker projection 71 located on the upper and lower planes of the base 21 presents an approximately circular or elliptical shape in the image. Based on the circular or elliptical projection, the center coordinates of each marker projection 71 are obtained, and its coordinates in the projection plane are determined accordingly. Therefore, by means of edge detection algorithms, approximately circular or elliptical marker projections 71 in the projected image can be identified, and the center coordinates of the marker projections 71 can be determined, thereby matching the correspondence between the marker projections 71 on the upper and lower planes and their physical coordinates.

[0113] Taking a circular array arrangement of markers 22 in the same plane as an example, after detecting marker projections 71, a circular outline can be fitted based on the principle of determining a circle by three points. The image coordinate data is then iteratively optimized using the Random Sample Consensus (RANSAC) algorithm to calculate and determine the center position and radius of the circular projection area formed by the marker projections 71 of markers 22 on the same surface, thus achieving geometric fitting of the marker projection array 71. The center position and radius of the projection area can be determined based on the center coordinates of each marker projection 71, which will not be elaborated upon here.

[0114] Because the markers 22 on the upper and lower planes are arranged in the same way, the actual physical radius of the marker projection array 71 formed in the projected image is the same or has a fixed ratio. Therefore, the fitting result can be checked based on the center position and radius of the marker projection array 71 on the upper and lower planes obtained by fitting. It can be determined whether the difference between the center position and radius is too large. If the difference is too large, the fitting can be re-performed, and then the image coordinates of each marker projection 71 in the projected image can be output.

[0115] It should be noted that the arrangement of the markers 22 on the base 21 is only one example. In actual use, the markers 22 can also be arranged in other array forms, such as rectangular frames, elliptical frames, star-shaped frames, etc., and the method for calculating the center of the projection area is determined according to the arrangement method of the actual application. In this embodiment, the arrangement of the markers 22 on the base 21 is not limited. Furthermore, Figure 7 The plane on which the two-dimensional image 70 is located can be any plane defined by the image registration device 50 between the transmitting end surface of the base 21 away from the C-arm 51 and the receiving end of the C-arm 51. In this embodiment, no other restrictions are placed on the position of the projection plane corresponding to the two-dimensional image 70.

[0116] Furthermore, after obtaining the image coordinates of each marker projection 71, a homography matrix H can be determined for the upper and lower planes respectively. 1 and H 2 Furthermore, the homography matrices corresponding to the two planes are both 3×3 matrices, used to describe the linear mapping relationship between the actual coordinates of the marker 22 on one plane of the standard tooling 20 and the projected coordinates of its projection 71.

[0117] Specifically, the relationship between the actual coordinates of the marker 22 in the plane and the projected coordinates of the marker projection 71 can be described by the following formula (6).

[0118] (6); Where k = 1 or 2, representing two different planes, upper and lower. Let 22 be the 3×1 homogeneous coordinates of the i-th marker 22 on the k-th plane in physical space. The 3×1 homogeneous coordinates of the corresponding marker projection 71 This is the 3×3 homography matrix corresponding to this plane.

[0119] Based on formula (6), the mapping relationship between the actual coordinates of the marker 22 in the calibration fixture 20 and the coordinates of the corresponding marker projection 71 in the projection image can be realized, thus providing a geometric basis for the subsequent construction of the virtual projection model.

[0120] S240: Construct a virtual projection model based on the alignment results between the projected image and the physical plane.

[0121] The virtual projection model may include an optical center and a virtual imaging plane. After aligning the physical plane and the projected image, the image registration device 50 can define the optical center and the virtual imaging plane based on the alignment result of the projected image and the physical plane to construct the corresponding virtual projection model. It should be understood that the virtual projection model is a model defined by the image registration device 50 based on preset content and its definition. After combining the corresponding coordinate parameters, the cone-beam model in this embodiment can be obtained.

[0122] Specifically, the optical center represents the origin of the projection rays in the model, while the virtual imaging plane represents the mapping relationship between physical points in the model and projection pixels. The image registration device 50 can obtain a virtual projection model by determining the optical center and the virtual imaging plane.

[0123] In this embodiment, the virtual imaging plane can be defined by the image registration device 50 based on the setting position of the calibration fixture 20. For example, the virtual imaging plane can be located on the plane of the transmitting end side of the base 21 away from the C-arm 51, or it can be any plane between the surface and the receiving end of the C-arm 51.

[0124] For example, the virtual imaging plane defined by the image registration device 50 can be Figure 7 The two-dimensional image 70 is shown in the figure. Thus, the image registration device 50 can use the homography matrix H... 1 and H 2 It can achieve precise mapping between the projected coordinates of markers 71 on the upper and lower planes and their actual coordinates, thereby establishing a geometric relationship between the projected image and the physical space. Based on this mapping relationship, the marker array on the calibration fixture 20 can be projected onto the virtual imaging plane.

[0125] Specifically, this can be achieved through the homography matrix H 1 and H 2 The inverse matrix, i.e. and This allows us to obtain the mapping coordinates of the marker projection 71 on the virtual imaging plane, thereby constructing a virtual projection model, simplifying the perspective projection relationship of the image registration device 50, and thus improving registration efficiency and accuracy.

[0126] After determining the virtual imaging plane, the position of the optical center can be determined using the principles of perspective projection, the actual coordinates of marker 22, the projected coordinates of marker 22, and the corresponding mapping relationship. In this embodiment, a set of sampling points can be determined by uniformly sampling the projected image. This facilitates the estimation of the optical center positions of the 50 pairs of images in the image registration equipment.

[0127] Figure 8 This is a schematic diagram of a uniform sampling method provided in an embodiment of this application.

[0128] like Figure 8 As shown, using a solid-line frame as the boundary of the projected image, points on the ever-expanding rectangular border are extracted as sampling points around the center of the plane corresponding to the projected image, according to a preset step size. Figure 8 Taking the dashed box shown as an example, the coordinates of each rectangle in the circumference direction can be obtained sequentially at a preset step size. Each rectangle used for sampling can be determined proportionally based on the boundary of the projected image, so that the side length of each rectangle is extended at equal intervals until it covers the entire projected image plane area.

[0129] This method of generating a sampling point set avoids optical center calculation errors caused by overly concentrated sampling point distribution, while ensuring that the sampling points cover the image edge region, thus improving the accuracy and precision of the acquired optical centers. Furthermore, it enhances the spatial generalization ability of the subsequently generated cone-beam model, resulting in higher accuracy within the imaging region and avoiding model distortion caused by sparse local sampling or the model only being effective in areas marked by marker 22, thus failing to accurately reflect the overall projection characteristics.

[0130] After determining the set of sampling points, the homography matrix H can be used as a reference. 1 and H 2 The sampling points in the sampling point set are mapped to two physical planes to obtain the physical plane coordinates corresponding to the sampling points. and These correspond to the physical planes defined by the upper and lower planes in the calibration fixture 20, respectively.

[0131] Among them, the physical plane coordinates of the sampling point and Let represent the corresponding coordinates of the m-th sampling point on the upper and lower physical planes, respectively. Then, based on the mapping relationship between the physical plane coordinates and the corresponding sampling points in the projected image, a physical space line is constructed. For example, the physical space line corresponding to each sampling point in the projected image can be determined by the following formula (7).

[0132] (7); in, This refers to a straight line in physical space, that is, a line formed by two points located on different physical planes. and The direction of the determined straight line is uniquely determined by the vector between the two points, and the straight line passes through the optical center and points to the corresponding sampling point in the image.

[0133] Under ideal conditions, all physical space lines should precisely intersect at the optical center position of the transmitting end of the C-arm 51. However, due to factors such as manufacturing tolerances, installation errors, and projection distortion in actual imaging systems, the physical space lines do not completely intersect at a single point in three-dimensional space, but rather form an approximate intersection region. To accurately estimate the optical center position, the least squares method can be used to fit the common intersection point between all lines, so that the obtained optical center coordinates are statistically closest to the ideal intersection point.

[0134] In this embodiment, after obtaining the physical space lines corresponding to all sampling points, the position of the optical center can be determined by solving for the common intersection point or the approximate intersection point in the least squares sense of these lines. Specifically, the optimal optical center position can be determined by the following formula (8).

[0135] (8); in, Indicates the optimal optical center position. This represents the line from the candidate optical center point O to the m-th physical space line. The Euclidean distance is given by M, where M is the number of physical lines determined based on the sampling points. The optimal solution is obtained by minimizing the sum of the squares of all distances. That is, the precise location of the optical center in physical space.

[0136] It should be understood that the parameter minimization in formula (8) refers to minimizing the summation term, i.e. When the minimum value is obtained, the value of the corresponding optical center position O is determined as follows: This optimization process can be solved using nonlinear least squares methods, such as iterative optimization using the LM algorithm, to ensure convergence to the global optimum, thereby improving the optical center positioning accuracy.

[0137] S250: Determine the parameters corresponding to the cone-beam model.

[0138] After obtaining the optimal optical center position, it can be used as the coordinates of the transmitting end position corresponding to the C-arm 51 in physical space. The image registration device 50 can further determine the parameters of the cone-beam model corresponding to the C-arm 51 based on the optical center position, and then add the corresponding parameters to the virtual projection model to obtain a cone-beam model corresponding to the C-arm 51.

[0139] Specifically, the image registration device 50 can first determine the image boundary based on the flat panel detector at the receiving end, and obtain the four corner points of the projected image. Taking a projected image with width w and height h, i.e., a resolution of w×h, as an example, the coordinates of its four corner points can be (0, 0), (w, 0), (0, h), and (w, h). By drawing rays from the optical center position O determined in the previous steps to the physical space lines corresponding to these four corner points, the four boundary rays of the cone-beam projection can be determined.

[0140] Then the four corners of the projected image can be... The coordinates of the corner point in physical space are determined by projecting the coordinates onto the physical plane corresponding to the upper plane of the calibration fixture 20. The coordinates of the corner point in physical space can be determined by the following formula (9): (9); in, This refers to the physical coordinates of the i-th corner point in the physical plane corresponding to the upper plane of the calibration fixture 20. This refers to the coordinates of the i-th corner point in the projected image. The value of i corresponds to the 3×3 homography matrix of the physical plane, and can be any integer from 1 to 4.

[0141] In this way, the boundary of the flat panel detector at the receiving end of the C-arm 51 can be mapped from the projected image to the upper plane of the calibration fixture 20, thus obtaining its actual boundary in physical space.

[0142] After obtaining the physical coordinates of the four corner points, the direction vector of the image can be constructed based on the coordinates of the four corner points in the physical space, thereby determining the orientation and direction of the projected image in the physical space. Specifically, the four physical corner points can be combined in pairs to determine the u and v directions. The vectors corresponding to the u direction and the v direction can be expressed as the following formula (10): , (10); in, and This represents a direction vector of the projected image in physical space, and and The two vectors are perpendicular to each other. This refers to the physical space coordinates obtained by mapping the corner point with coordinates (w, 0) in the projected image using formula (9). This refers to the physical space coordinates of the corner point with coordinates (0, h) in the projected image, obtained after mapping using formula (9). The physical space coordinates of the corner point with coordinates (0, 0) in the projected image are obtained by mapping using formula (9).

[0143] Furthermore, it is also possible to... and These two direction vectors are normalized to thus and The unit vectors corresponding to the u and v directions in the projected image are determined, and then the direction vectors in the projected image are measured in subsequent processes.

[0144] In this way, the orientation of the detector plane at the receiving end corresponding to the C-arm 51 in physical space can be determined, and based on... and This allows us to describe the coordinate system axes of the receiver detector itself, and obtain the coordinate axis direction vectors of the imaging plane corresponding to the projected image, thus obtaining two direction vectors. and ,in It can be used as the x-axis direction vector of the imaging plane. It can be used as the y-axis direction vector of the imaging plane.

[0145] After determining the direction vector, the image registration device 50 can calculate the pixel size and establish the conversion relationship between pixel units and physical length. Specifically, as shown in the following formula (11), the pixel size can be calculated by the ratio of the vector magnitudes in the u and v directions in physical space to the image resolutions w and h: , (11); in, This represents the magnitude of the vector in the u direction. Let v represent the magnitude of the direction vector, and w and h represent the width and height of the projected image in pixel coordinates, respectively. This allows us to determine the pixel size based on the direction vector and the dimensions of the projected image, establishing a conversion relationship between physical length and pixel units, and providing precise scale information for subsequent 3D reconstruction and surgical navigation.

[0146] Furthermore, the image registration device 50 can also determine the center of the projected image. In this embodiment, the image center can be determined by averaging the coordinates of the four corner points, as shown in the following formula (12). The image coordinates of the image center can be determined by the following formula: (12); in, This refers to the pixel coordinates of the center of the projected image. The pixel coordinates of the four corner points of the projected image are represented by the value of i, which ranges from 1 to 4, corresponding to the four corner points respectively. The image center coordinates obtained in this way can effectively eliminate the deviation caused by projection distortion, are not affected by the content of the projected image, and facilitate the accurate positioning of the image center.

[0147] In some embodiments of this application, when the resolution of the projected image is w×h, the image registration device 50 can also directly determine the point with coordinates (w / 2, h / 2) as the image center. The methods for determining the center of the projected image in this application are not limited to the two mentioned above, and the applicable method can be selected according to the actual calibration requirements.

[0148] Based on the above process, the imaging area of ​​the C-arm 51 at angle α can be determined by the optical center O and the pixel size. and Projection Center Coordinate direction vector and The specific parameters of the cone-beam model at angle α are obtained by characterizing the model.

[0149] It should be understood that the coordinate data in the specific parameters of the cone-beam model determined by steps S250 and S240 are all spatial data based on the magnetic sensor coordinate system corresponding to the calibration fixture 20, and the magnetic sensor coordinate system can be obtained from the actual coordinate system determined by the aforementioned step S100.

[0150] S260: Transform the coordinate system corresponding to the cone-beam model to the receiving coordinate system.

[0151] Because the purpose of setting up the calibration fixture 200 to determine the cone-beam model is to provide a unified and accurate spatial reference for the subsequent image registration and surgical navigation processes in steps S300 and S400, the calibration fixture 20 itself will not participate in the surgical process. Therefore, after obtaining the cone-beam model parameters, the parameters need to be uniformly converted to the coordinate system corresponding to the image registration device 50 that is still used during the operation, so that the cone-beam model parameters are consistent with the coordinate system of the image registration device 50, thereby ensuring that the projected image acquired by the C-arm 51 during the operation is aligned with the preoperative or real-time navigation data under the same spatial reference.

[0152] In the embodiments of this application, such as Figure 5 As shown, the receiving end of the C-arm 51 may be equipped with a flat panel detector and a third magnetic sensor 53. The flat panel detector is used to receive the projection rays sent by the transmitting end to form a projection image based on the object in the imaging area. The third magnetic sensor 53 is used to obtain the pose of the receiving end of the C-arm 51 and can cooperate with the first sensor 23 to determine the relative pose information between the receiving end of the C-arm 51 and the calibration fixture 20.

[0153] It should be understood that, Figure 5 The arrangement of the flat panel detector and the third magnetic sensor 53 in the receiving end of the C-arm 51 shown is only one example in the embodiments of this application. In another embodiment, the flat panel detector may cover the receiving end of the C-arm 51, and the third magnetic sensor 53 may be set at a position closer to the inside of the receiving end of the C-arm 51 than the flat panel detector. The embodiments of this application do not limit the specific arrangement of the flat panel detector and the third magnetic sensor 53.

[0154] Furthermore, the image registration device 50 is equipped with a second magnetic base station 52. The coordinate system transformation between the first magnetic sensor 23 and the third magnetic sensor 53 can be performed through the second magnetic base station 52. Specifically, after determining the cone-beam model parameters at angle α, the image registration device 50 can transform the actual coordinate system in the calibration fixture 20, based on the first magnetic sensor 23, to the magnetic base station coordinate system determined by the second magnetic base station 52. Then, it can transform from the magnetic base station coordinate system back to the coordinate system determined by the third magnetic sensor 53 in the image registration device 50, completing a unified coordinate system mapping. This allows the calibration results of the calibration fixture 20 to be transmitted to the coordinate system corresponding to the magnetic sensor at the receiving end of the C-arm 51, thereby ensuring spatial consistency between the projected images acquired by the C-arm 51 at different angles and the real-time intraoperative navigation data.

[0155] In this embodiment, the image registration device 50 can acquire the spatial coordinates of the first magnetic sensor 23 and the third magnetic sensor 53 respectively through the second magnetic base station 52, and determine the position and attitude information of the two magnetic sensors in the magnetic base station coordinate system.

[0156] For example, the image registration device 50 can use the magnetic sensor coordinate system corresponding to the first magnetic sensor 23. The coordinate system of the magnetic base station corresponding to the second magnetic base station 52 The transformation relationship between the two is used to determine the pose of the first magnetic sensor 23 in the magnetic base station coordinate system. .

[0157] Furthermore, the image registration device 50 can also use the receiving coordinate system corresponding to the third magnetic sensor 53. The coordinate system of the magnetic base station corresponding to the second magnetic base station 52 The transformation relationship between the two is used to determine the pose of the third magnetic sensor 53 in the magnetic base station coordinate system. .

[0158] After obtaining the pose matrices of the first magnetic sensor 23 and the third magnetic sensor 53 in the magnetic base station coordinate system, the precise position of the first magnetic sensor 23 in the receiving coordinate system can be determined based on the relative transformation relationship between the two pose matrices. As shown in the following formula (13), the pose matrix of the first magnetic sensor 23 in the receiving coordinate system is determined as follows: (13); in, This refers to the pose of the first magnetic sensor 23 in the receiving coordinate system. This refers to the inverse matrix of the pose matrix of the third magnetic sensor 53 in the magnetic base station coordinate system. In this way, the coordinate data in the magnetic sensor coordinate system can be transformed to the receiving end coordinate system based on the magnetic base station coordinate system, realizing the coordinate system transformation of the cone-beam model parameters.

[0159] It should be understood that the image registration device 50 can use the above conversion method to convert the optical center O and pixel size... and Coordinate direction vector and The relevant content of the iso-coordinate data is converted to the coordinate system of the receiving end.

[0160] In some embodiments of this application, the image registration device 50 can further determine the corresponding rigid body transformation matrix by transforming the coordinate systems. Then, based on the rigid body transformation matrix between the magnetic sensor coordinate system and the magnetic base station coordinate system, and the rigid body transformation matrix between the receiving end coordinate system and the magnetic base station coordinate system, it determines the rigid body transformation matrix between the magnetic sensor coordinate system and the receiving end coordinate system. Furthermore, through the rigid body transformation matrix between the magnetic sensor coordinate system and the receiving end coordinate system, the optical center O and pixel size are... and Coordinate direction vector and The relevant content of the iso-coordinate data is converted to the coordinate system of the receiving end.

[0161] It should be understood that the coordinate transformation methods of the model parameters provided in the embodiments of this application are only a few examples. In actual applications, model parameters can also be transformed in other ways. This application does not limit the specific method of model parameter transformation.

[0162] After the conversion is completed, the cone-beam model can be fixed from the magnetic sensor coordinate system bound to the calibration fixture 20 to the receiving end coordinate system corresponding to the third magnetic sensor 53 rigidly connected to the C-arm 51, which facilitates subsequent operations such as model construction and intraoperative navigation using the image registration device 50.

[0163] It should be noted that, Figure 3 The first magnetic base station 33 shown in the figure Figure 6 The second magnetic base station 52 shown in the figure and the magnetic base stations in subsequent embodiments can all be the same device, and can all serve as the reference point for the magnetic navigation system, used to unify the spatial coordinate system between different magnetic sensors. In another embodiment, each magnetic base station can also be set up independently, but calibration is required to ensure the consistency of the coordinate system between each magnetic base station, so as to ensure that the spatial data obtained by the magnetic sensors at different locations can be accurately converted to the same coordinate system.

[0164] S270: Based on the projection angle corresponding to the projected image, establish multiple cone-beam models corresponding to different projection angles.

[0165] After obtaining the cone-beam model parameters at angle α and converting them to the coordinate system corresponding to the third magnetic sensor 53, the C-arm 51 can be rotated to angle β, and the operation process of steps S210 to S260 mentioned above can be executed to obtain the cone-beam model parameters at angle β in the coordinate system corresponding to the third magnetic sensor 53. Then, based on the projection angle corresponding to the projection image, multiple cone-beam models corresponding to different projection angles can be established.

[0166] In this embodiment of the application, angle α can be perpendicular to angle β, so that... Figure 2 The structure of calibration fixture 20 and Figure 6 The arrangement of the calibration fixture 20 shown in the image serves as an example. When the C-arm 51 is at angle α, the marker projection 71 on the projected image is the projection of the markers 22 on the upper and lower planes of the calibration fixture 20. When the C-arm 51 is at angle β, the marker projection 71 on the projected image can be the projection of the markers 22 on the left and right planes of the calibration fixture 20. Correspondingly, when determining the cone-beam model parameters at angle β, parameters such as the optical center, pixel size, projection center, and coordinate direction vector are determined by the marker projections 71 corresponding to the left and right planes of the calibration fixture 20 in the projected image.

[0167] After generating multi-angle cone-beam models, the image registration device 50 can store the cone-beam model parameters according to the angle of the C-arm 51 corresponding to each cone-beam model, which is convenient for subsequent intraoperative retrieval and dynamic registration.

[0168] It should be noted that setting the C-arm 51 to angles α and β ensures that the projection ray with the shortest transmission distance between the transmitting and receiving ends of the C-arm 51 is perpendicular to the two opposing surfaces on the base 21. This allows for the projection of each marker 22 on the base 21, and the establishment of multiple cone-beam models. Therefore, the α and β angles are determined based on the shape of the base 21 in the calibration fixture 20. When the base 21 has a hexahedral structure, the α and β angles are perpendicular to each other. In practical applications, the C-arm 51 can be used to construct two or more cone-beam models based on the specific structure of the calibration fixture 20, and the α and β angles can be determined by the specific structure of the calibration fixture 20.

[0169] S300: During the operation, a positioning model is constructed based on at least two cone-beam models and at least two projection images.

[0170] After at least two cone-beam models are solidified into the image registration device 50, if the image registration device 50 is used in surgery, medical staff can use the image registration device 50 to construct a positioning model between two-dimensional images and three-dimensional physical space using the cone-beam models and the projection images and magnetic navigation data acquired during surgery, thereby achieving intraoperative image registration.

[0171] Figure 9 This is a schematic diagram illustrating intraoperative image registration and positioning navigation as provided in an embodiment of this application. Figure 10 This is a schematic diagram illustrating an intraoperative localization model construction process provided in an embodiment of this application. The following is based on... Figure 9 and Figure 10 The process of image registration device 50 performing image registration and positioning model construction tasks during surgery is described.

[0172] like Figure 9 As shown, the image registration device 50 can be used to project the patient's lesion location during the operation to obtain the projected image. At this time, the patient's lesion location can be set within the imaging area of ​​the image registration device 50, so that the image registration device 50 can obtain projected images of the patient's lesion at different angles through the C-arm 51.

[0173] like Figure 10 As shown, the intraoperative image registration and localization model construction process provided in this application embodiment may include the following steps S310 to S340.

[0174] S310: Select at least two cone-beam models.

[0175] In this embodiment of the application, after the patient enters the surgical area and completes the initial positioning, the image registration device 50 can select at least two cone-beam models from the cone-beam models stored in the aforementioned steps as registration references, providing high-precision projection set parameters for intraoperative image registration, reducing the amount of model calculation during the operation, and improving registration efficiency.

[0176] Taking the selection of cone-beam models at angles α and β as an example, the image registration device 50 calls the stored cone-beam model parameters to obtain the cone-beam models corresponding to angles α and β. Specifically, the cone-beam model at angle α can be represented as... The cone-beam model at angle β can be expressed as: .

[0177] in, and These refer to the optical centers of the cone-beam model at the corresponding angles. and This refers to the homography matrix corresponding to the cone-beam model at the corresponding angle. and These refer to the transformation between the receiving end coordinate system and the magnetic base station coordinate system where the cone-beam model at the corresponding angle is located.

[0178] It should be understood that the cone-beam model parameters obtained in this step are all values ​​determined in step S200 of the aforementioned embodiments. The image registration device 50 can select at least two cone-beam models and obtain their corresponding parameters by reading the content from the pre-stored data.

[0179] S320: Collect patient data during the surgery.

[0180] Before performing surgery on the patient, such as Figure 9 As shown, medical staff can install a fourth magnetic sensor 91 near the patient's lesion, and at the same time, a fifth magnetic sensor 93 is installed in the surgical instrument 92 used by the medical staff, so that the image registration device 50 can convert the coordinate parameters between different coordinate systems based on the magnetic base station coordinate system corresponding to the second magnetic base station 52, and realize the real-time spatial mapping between the lesion and the surgical instrument.

[0181] In this embodiment of the application, the patient-related data may include a patient coordinate system. Projected images at angles α and β, acquired by the transmitter and receiver of the C-arm 51. and The pose of the third magnetic sensor 53 in the magnetic base station coordinate system when the C-arm 51 is at angles α and β. and And, the pose of the fourth magnetic sensor 91 in the magnetic base station coordinate system. .

[0182] Specifically, the image registration device 50 can send a magnetic sensor installation prompt to medical staff after the patient enters the surgical area or the patient's lesion enters the imaging area. This installation prompt can be used to guide the user to install the fourth magnetic sensor 91 for the patient. For example, the installation prompt can be displayed on the visualization interface of the image registration device 50, such as "Please install the magnetic sensor near the patient's lesion area" to guide the user to complete the installation of the magnetic sensor.

[0183] After the fourth magnetic sensor 91 is fixed, the C-arm 51 is activated to acquire projected images at angles α and β, respectively, to obtain the projected images. and During the acquisition of projected images, the coordinate system corresponding to the fourth magnetic sensor 91 is determined as the patient's coordinate system. The real-time poses of the third magnetic sensor 53 and the fourth magnetic sensor 91 in the magnetic base station coordinate system are determined by the second magnetic base station 52, thus obtaining... , and .

[0184] This allows for real-time acquisition of patient data during surgery, facilitating the image registration device 50 to perform image registration and localization model construction tasks.

[0185] S330: Convert the coordinate system corresponding to the cone-beam model to the reference coordinate system.

[0186] After the image registration device 50 acquires the patient's corresponding data, the patient coordinate system can be established. As a reference coordinate system, the coordinate system of the patient data obtained in the aforementioned step S320 is aligned with this reference coordinate system, thereby unifying the coordinate system.

[0187] Specifically, 50 image registration devices During the transformation to the reference coordinate system, the patient's coordinate system can be obtained first. As a transformation matrix of the reference coordinate system, the coordinates of the cone-beam model can be transformed to the reference coordinate system. The method for determining the cone-beam model corresponding to different angles can be shown in the following equation (14): , (14); in, and These refer to the pose matrix of the third magnetic sensor 53 in the reference coordinate system when the C-arm 51 is at different angles. Then it refers to The inverse matrix is ​​used to realize the transformation from the magnetic base station coordinate system to the patient coordinate system. Through formula (14), the coordinate transformation matrix for switching the cone-beam model from the receiver coordinate system to the reference coordinate system when the C-arm 51 is at angle α or β can be determined.

[0188] In obtaining and Subsequently, the image registration device 50 can be combined with the cone-beam model. and The spatial position and attitude of the cone-beam model are calculated in the reference coordinate system. and The method for switching from the coordinate system to the reference coordinate system can be shown in the following equation (15): , (15); in, and These represent the spatial representations of the cone-beam model at angles α and β in the reference coordinate system after coordinate transformation. This allows for the representation in the patient coordinate system. As a reference coordinate system, the cone-beam model selected by the image registration device 50 is unified under this coordinate system, and the cone-beam model obtained by calibration is aligned with the patient during surgery to achieve spatial alignment of multi-angle image data.

[0189] S340: Reconstruct the projection images corresponding to at least two cone-beam models in three dimensions to build a localization model.

[0190] After determining the spatial representation of at least two cone-beam models in the reference coordinate system, the image registration device 50 can, in response to a selection operation, project the images at angles α and β. and Select the feature points corresponding to the same anatomical feature or the same marker point. and The number of feature points corresponding to the same anatomical feature or the same marker point is the same as the number of projection images corresponding to the cone-beam model. and The corresponding coordinate system is the same as the coordinate system of the projected image.

[0191] Then, the image registration device 50 can be based on the homography matrix corresponding to the cone-beam model at different angles. and The feature points are mapped onto the virtual imaging plane, and the specific mapping method is shown in equation (16) below: , (16); in, and It refers to and The three-dimensional coordinates in the virtual projection model. (The rest of the text appears to be incomplete and requires further context.) and Afterwards, it can be based on and And the optical center after the cone-beam model transformation, construct the projection ray corresponding to each cone-beam model and calculate their intersection point in three-dimensional space, thereby determining and The corresponding anatomical features in the patient coordinate system The three-dimensional coordinates below.

[0192] For example, Corresponding projection ray any point on and Corresponding projection ray any point on This can be expressed as the following formula (17): (17); in, and Refers to projection rays respectively and The three-dimensional coordinates of any point, and The optical center of the cone-beam model at different angles in the patient coordinate system Coordinates in; Let be a non-zero constant used to describe the extension distance of the projected ray, and The value range of can be (0, 1], in When the value is 0, it corresponds to the ray origin, i.e., the optical center position of the cone-beam model (λ is not taken as 0 in actual calculations). A value of 1 corresponds to the endpoint of the ray. In this embodiment, the endpoint of the ray is a mapping point on the virtual imaging plane. or .

[0193] Image registration device 50 can solve for two projection rays and The closest point is used to obtain the feature point. and In the patient coordinate system 3D intersection The three-dimensional intersection point is and The precise location of the corresponding anatomical feature points in the patient coordinate system.

[0194] It should be understood that, under ideal conditions, there are two projected rays. and They should intersect at a single point, but due to errors in the selection of projection coordinates during the feature point selection process and errors in the actual imaging process of the C-arm 51, the two projected rays may be skew lines. In this case, the midpoint of their shortest distance or the optimal intersection point in the least squares sense can be used as the three-dimensional intersection point. This is used to determine the estimated value, thereby improving positioning accuracy.

[0195] In this way, the image registration device 50 can spatially reconstruct the projection rays of the same anatomical feature under different projection angles, and combine the least squares method to optimize and solve for the midpoint of its shortest distance, thereby accurately obtaining the three-dimensional position of the feature point in the patient coordinate system. The image registration device 50 can perform the above process multiple times during the operation to determine the three-dimensional intersection points corresponding to multiple anatomical features and / or marker points. Then, through multiple three-dimensional intersection points, the three-dimensional reconstruction task of the two-dimensional projection image is realized, and a positioning model for intraoperative navigation is obtained, providing high-precision spatial positioning support for subsequent surgical path planning and real-time guidance.

[0196] S400: Based on the positioning model, intraoperative instruments are rendered onto the projection image to achieve intraoperative navigation.

[0197] After obtaining the positioning model, the image registration device 50 can render the surgical instruments 92 used in the operation onto the projection image formed by the C-arm 51 based on the constructed positioning model, thereby realizing the spatial coordinate fusion of the surgical instruments 92 and the patient's anatomical structure, providing visual feedback to medical staff, and achieving intraoperative navigation.

[0198] like Figure 9 As shown, the image registration device 50 can determine the real-time pose data of the fourth magnetic sensor 91 and the fifth magnetic sensor 93 based on the second magnetic base station 52, thereby obtaining the pose information of the fourth magnetic sensor 91 near the patient's lesion in the magnetic base station coordinate system. And the pose information of the fifth magnetic sensor 93 on each surgical instrument 92 in the magnetic base station coordinate system. .

[0199] Then, and Transform to patient coordinate system Specifically, the transformation process can be referred to the form in the aforementioned formula (13), and the transformation matrix can be determined by the following formula (18): (18); in, This refers to the position of the i-th surgical instrument in the patient's coordinate system. The pose matrix in the system. Based on the above method, the pose information of the fifth magnetic sensor 93 in each surgical instrument 92 can be uniformly converted to the patient coordinate system. Then, the three-dimensional coordinates corresponding to the surgical instrument 92 are rendered into the projection image through the aforementioned positioning model, thereby realizing the real-time spatial fusion visualization of the surgical instrument 92 and the patient's anatomical structure.

[0200] In this embodiment, the method of rendering the three-dimensional coordinates corresponding to the surgical instrument 92 into the projected image can be the reverse process of the aforementioned step S340. The image registration device 50 can determine the reference coordinate value of the surgical instrument 92 in the reference coordinate system as a three-dimensional point. Using two cone-beam models and Determine three-dimensional points In the projected image and Feature points in and Thus, three-dimensional points are obtained. Based on the projection coordinates of the projected image, the specific position of the surgical instrument 92 is then superimposed on the dual-view real-time projection image as a virtual identifier, thereby providing the surgeon with precise real-time navigation guidance.

[0201] It should be understood that, before step S300 is executed, the surgical instrument 92 used in this embodiment can be calibrated through the process in step S100 described above, so as to realize the transformation between the coordinate system corresponding to the fifth magnetic sensor 93 and the magnetic sensor coordinate system, ensuring that the pose data collected by the fifth magnetic sensor 93 can be mapped to the patient coordinate system through the magnetic base station coordinate system during the operation. This ensures the precise positioning of the surgical instrument 92 in three-dimensional space.

[0202] Based on the above embodiments, dynamic tracking of intraoperative instrument pose and synchronous updating of projected images can be achieved, ensuring consistency between instrument movement and image feedback during surgery, realizing the technical effect of intraoperative navigation, and improving the efficiency and accuracy of surgery. Furthermore, the cone-beam model determined by the calibration fixture 20 can be stored permanently in the image registration device 50, and can be directly retrieved in subsequent surgeries without repeated calibration, effectively shortening preoperative preparation time. Simultaneously, when the surgical environment changes, the cone-beam model parameters can be updated through a rapid recalibration process, enabling the image registration device 50 to continuously maintain high-precision intraoperative navigation and positioning capabilities.

[0203] Finally, it should be noted that the above embodiments are only used to illustrate this application and are not intended to limit this application. Although this application has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application and should be covered within the scope of the claims of this application.

Claims

1. An image registration method based on magnetic navigation, characterized in that, An image registration device is used to determine at least two projected images based on different angles, including: Calibration and calibration fixtures; Multiple cone-beam models were established using the aforementioned calibration fixture; During the surgery, a positioning model is constructed based on at least two of the cone-beam models and at least two projection images; Based on the positioning model, the intraoperative instruments are rendered onto the projected image to achieve intraoperative navigation; The calibration fixture includes multiple markers, and the calibration fixture includes: Set the coordinate deviation between the designed coordinate values ​​and the actual coordinate values ​​of the multiple markers; Based on the setting of the coordinate deviation, a mapping relationship between the design coordinate system and the actual coordinate system is established; Using calibration equipment, the first and second measurement coordinate values ​​corresponding to each marker are obtained respectively; the first measurement coordinate value is high-precision coordinate data obtained based on a precision coordinate measuring instrument, and the second measurement coordinate value is coordinate data obtained based on magnetic field sensing measurement. Based on the distance difference between the first and second measured coordinate values, a residual term is set; Determine the objective function based on the residual term; Based on the objective function and the designed coordinate values ​​of each marker, the actual coordinate values ​​of each marker are determined.

2. The image registration method according to claim 1, characterized in that, The calibration fixture includes multiple markers, and multiple cone-beam models are established using the calibration fixture. The calibration fixture is positioned within the imaging range of the image registration device, including: Obtain the actual coordinates of the marker; Align the projected image with a physical plane, the physical plane including any one of the planes on the calibration fixture on which the markers are provided; A virtual projection model is constructed based on the alignment result between the projected image and the physical plane; the virtual projection model includes an optical center and a virtual imaging plane; Based on the optical center and the virtual imaging plane, determine the parameters corresponding to the cone-beam model; The coordinate system corresponding to the cone-beam model is transformed to the receiving end coordinate system, which is the coordinate system determined by the image registration device based on the receiving end.

3. The image registration method according to claim 2, characterized in that, The calibration fixture includes at least two physical planes, and aligning the projected image with the physical planes includes: Obtain the image coordinates corresponding to the markers in the at least two physical planes; Determine the homography matrix between the physical coordinates of the marker and the image coordinates of the marker in the at least two physical planes.

4. The image registration method according to claim 3, characterized in that, The step of constructing a virtual projection model based on the alignment result of the projected image and the physical plane includes: Based on the homography matrix, the image coordinates corresponding to the marker are mapped to the virtual imaging plane.

5. The image registration method according to claim 3, characterized in that, The step of constructing a virtual projection model based on the alignment result of the projected image and the physical plane further includes: Uniform sampling is performed on the plane corresponding to the projected image to obtain a set of sampling points, the set of sampling points including multiple sampling points; Each of the sampling points is mapped to the at least two physical planes; A straight line in physical space is constructed based on the mapped coordinates of each of the sampling points in at least two physical planes; The intersection of the physical space lines corresponding to each sampling point is determined as the optical center.

6. The image registration method according to claim 2, characterized in that, The process of establishing multiple cone-beam models using the calibration fixture also includes: Based on the projection angle corresponding to the projected image, multiple cone-beam models corresponding to different projection angles are established.

7. The image registration method according to claim 1, characterized in that, During the surgery, constructing a positioning model based on at least two cone-beam models and at least two projection images includes: At least two of the cone-beam models are selected from the plurality of cone-beam models; During the surgery, data corresponding to the patient is collected, including a reference coordinate system. Transform the coordinate systems corresponding to the at least two cone-beam models to the reference coordinate system; The projection images corresponding to the at least two cone-beam models are reconstructed in three dimensions to construct the positioning model.

8. The image registration method according to claim 7, characterized in that, The method of constructing a positioning model based on at least two cone-beam models and at least two projection images during the surgical procedure further includes: In response to the patient entering the imaging area, a magnetic sensor installation prompt is sent, the installation prompt being used to instruct the user to install a fourth magnetic sensor on the patient; In response to the completion of the installation of the fourth magnetic sensor, the reference coordinate system is determined based on the coordinate system corresponding to the fourth magnetic sensor.

9. The image registration method according to claim 8, characterized in that, The image registration device also includes a third magnetic sensor and a second magnetic base station. During the surgery, the data collected from the patient includes: Based on the projection angles corresponding to the at least two cone-beam models, at least two projection images of the patient are determined; Based on the projection angles corresponding to the at least two cone-beam models, the pose of the third magnetic sensor in the magnetic base station coordinate system corresponding to the second magnetic base station is obtained; The pose of the fourth magnetic sensor in the coordinate system of the magnetic base station is obtained.

10. The image registration method according to claim 8, characterized in that, The three-dimensional reconstruction of the projected images corresponding to the at least two cone-beam models to construct the positioning model includes: In response to the selection operation, feature points corresponding to the same feature are obtained in the projection images corresponding to the at least two cone-beam models respectively, and the number of feature points corresponding to the same feature is the same as the number of selected cone-beam models; Determine the three-dimensional coordinates of the feature point in the reference coordinate system; The positioning model is constructed based on the three-dimensional coordinates of the feature points.

11. The image registration method according to claim 9, characterized in that, The step of rendering intraoperative instruments onto the projection image based on the positioning model includes: The positions and orientations of the intraoperative instruments and the fourth magnetic sensor in the magnetic base station coordinate system are obtained; Based on the pose of the fourth magnetic sensor in the magnetic base station coordinate system, the pose of the intraoperative instrument is transformed to the reference coordinate system; Based on the reference coordinate values ​​corresponding to the intraoperative instruments and the at least two cone-beam models, determine the projection coordinates of the intraoperative instruments corresponding to the projection image; The intraoperative instruments are rendered to the projection coordinates.

Images