A medical navigation method

Abstract

The application discloses a kind of medical navigation methods, it is related to medical navigation positioning technical field, including establishing the conversion matrix of binocular camera coordinate system and model coordinate system;Obtain MRI data, and convert MRI data into MRI point cloud data;Face point cloud data is obtained by environment point cloud data and environment RGB data;MRI point cloud data and face point cloud data are registered, and point cloud matching matrix is obtained;Calibration matrix between binocular camera coordinate system and point cloud camera coordinate system is established;The position relationship of model coil and MRI data is displayed on computer by double-view navigation algorithm, and target position is determined according to position relationship.The application adopts binocular camera and point cloud camera, and determines target position according to model coil and MRI data position relationship, so as to be able to more effectively track target position, be favorable to provide more accurate, more convenient navigation guidance, improve treatment efficiency and treatment effect, improve the operation difficulty of navigation.

Description

Technical Field

[0001] This invention relates to the field of medical navigation and positioning technology, and more specifically, to a medical navigation method. Background Technology

[0002] With the rapid development of society and the accelerated pace of life and work, modern people face increasing life pressures. Terms such as "depression" and "anxiety" frequently appear in daily life, and the incidence of mental illnesses is also increasing.

[0003] In 1985, Barker et al. placed an electrified stimulation coil above the head of the subject to apply magnetic field stimulation to the motor cortex of the brain. They observed that the hand muscles of the subject twitched and recorded the motor evoked potentials (MEPs) of the abductor digiti minimi muscle through surface electrodes. This form is known as transcranial magnetic stimulation (TMS). In 1987, Amassian et al. demonstrated through experiments that the placement direction of the stimulation coil affects the effect of TMS on the cerebral cortex. In 1993, Hoflide et al. applied transcranial magnetic stimulation to the treatment of depression, demonstrating that TMS has a certain therapeutic effect on depression. In 2005, Edwards et al. demonstrated through experiments that low-intensity repetitive transcranial magnetic stimulation can induce excitability of neurons in the cerebral cortex. In 2007, Joo EY et al. found that long-term, low-frequency repetitive transcranial magnetic stimulation can alleviate the symptoms of epilepsy. In 2015, Ku Y et al. used single-pulse transcranial magnetic stimulation to stimulate the sensory cortex and lateral posterior parietal cortex, finding that this form of stimulation has an intervention effect on the brain's cognitive function.

[0004] However, numerous difficulties exist in the clinical application of transcranial magnetic stimulation (TMS) therapy, significantly limiting its application and promotion in the treatment of mental and neurological diseases. The main difficulties include: First, the positioning of the TMS stimulation coil relies heavily on the physician's experience and skill, making it highly subjective; inaccurate coil placement will affect the treatment effect. Second, the patient's brain structure is not visible during coil placement, and since each person's head structure is different, the positioning cap lacks universality and accuracy. Third, each TMS treatment session lasts 15 to 30 minutes; even slight head movement during this process will cause changes in the coil's placement. If the patient's head is fixed, muscle contraction and tension as stimulation time increases will cause discomfort. These issues are the main reasons currently affecting the accuracy of TMS therapy and represent technical challenges that need to be addressed for its widespread application.

[0005] With the rapid development of medical imaging and image processing technologies, image-guided surgical systems have emerged. Utilizing three-dimensional reconstruction models of medical images, surgeons can intuitively and accurately analyze the structure of organs or tissues and their surrounding tissues. Image-guided surgical systems use intraoperative images of the patient and three-dimensional models of relevant lesions and surrounding tissues to guide clinical procedures in real time. During the surgical operation, image-guided software can accurately display the patient's tissue anatomy and the details of the three-dimensional space surrounding the lesion. Image guidance utilizes medical imaging technology to scan head images, segmenting and reconstructing the acquired head images to create a three-dimensional head model including brain tissue. Stimulation targets are planned on the reconstructed three-dimensional brain model, and image registration technology is used to map the targets from the three-dimensional brain model onto the patient's head during surgery, thereby guiding the surgeon to locate the targets.

[0006] Optical-assisted navigation is currently the most representative commercial TMS navigation system. In 2008, Lars Matthau... ·· S et al. used a Polaris Spectra optical tracking device and an Adept Viper s850 six-axis robot to create a TMS robotic therapy system. The system uses the robot to hold the stimulation coil, and the optical tracking device is fixed by a bracket. A marker is fixed to the subject's head to help the optical tracking device locate the head coordinates. Using an optical navigation and positioning system allows for visualization of the transcranial magnetic stimulation (TMS) treatment process, improving the accuracy of stimulation coil positioning to some extent. However, current navigation systems are relatively cumbersome to operate; accidental camera touches may lead to re-registration issues. Furthermore, the visualization perspective is limited and inconvenient to operate. Additionally, using an infrared binocular camera alone requires the patient to wear a reflective mannequin, which may be unsuitable for patients with autism or other conditions. Summary of the Invention

[0007] In view of this, the present invention provides a medical navigation method that uses a binocular camera and a point cloud camera. Through a point cloud matching algorithm, the target position is determined based on the positional relationship between the model coil and MRI (Magnetic Resonance Imaging) data. This enables more effective tracking of the target position, improves tracking accuracy, provides more precise and convenient navigation guidance, enhances treatment efficiency and effectiveness, and reduces the operational difficulty of navigation.

[0008] This application has the following technical solution:

[0009] This application provides a medical navigation method, including:

[0010] Establish the transformation matrix between the binocular camera coordinate system and the model coordinate system;

[0011] Acquire MRI data and convert the MRI data into MRI point cloud data;

[0012] Use a point cloud camera to acquire environmental point cloud data and environmental RGB data, and then use the environmental point cloud data and environmental RGB data to obtain face point cloud data;

[0013] The MRI point cloud data and the face point cloud data are registered to obtain a point cloud matching matrix.

[0014] Establish a calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system;

[0015] Using the transformation matrix between the binocular camera coordinate system and the model coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, and the point cloud matching matrix, a dual-view navigation algorithm is used to display the positional relationship between the model coil and the MRI data on a computer, and the target position is determined based on the positional relationship.

[0016] Optionally, where:

[0017] The transformation matrix between the binocular camera coordinate system and the model coordinate system is specifically established as follows:

[0018] A model coil is created using a computer, and at least four first calibration points are set in the model coil;

[0019] A first marker is placed on the stimulation coil;

[0020] A second calibration point is provided on the stimulation coil, corresponding one-to-one with at least four first calibration points in the model coil;

[0021] Based on the coordinates of the first calibration point in the model coordinate system and the coordinates of the second calibration point in the first marker coordinate system, the transformation matrix between the first marker coordinate system and the model coordinate system is obtained;

[0022] Based on the binocular camera, the transformation matrix between the first marker coordinate system and the binocular camera coordinate system is obtained;

[0023] Based on the transformation matrices between the first marker coordinate system and the model coordinate system, and between the first marker coordinate system and the stereo camera coordinate system, the transformation matrix between the stereo camera coordinate system and the model coordinate system is obtained.

[0024] Optionally, where:

[0025] The process of obtaining face point cloud data using environmental point cloud data and environmental RGB data is as follows:

[0026] Face data is extracted from environmental RGB data using a face detection algorithm;

[0027] The environmental point cloud data is registered to the RGB coordinate system, and non-face point cloud data is removed based on the face data to obtain the face point cloud data.

[0028] Optionally, where:

[0029] The process of registering the MRI point cloud data and the face point cloud data to obtain a point cloud matching matrix is ​​as follows:

[0030] Using the RANSAC algorithm, coarse matching is performed between the MRI point cloud data and the face point cloud data to obtain coarse matching data;

[0031] The ICP algorithm is used to perform precise matching on the coarse matching data, establish the relationship between MRI point cloud data and face point cloud data from point cloud cameras, and obtain the point cloud matching matrix.

[0032] Optionally, where:

[0033] The establishment of the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system is specifically as follows:

[0034] Select at least four first coordinate points in the point cloud camera coordinate system;

[0035] Select at least four second coordinate points in the binocular camera coordinate system;

[0036] Based on the first coordinate point and the second coordinate point, calculate the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system.

[0037] Optionally, where:

[0038] The establishment of the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system is specifically as follows:

[0039] A third marker is placed on the point cloud camera;

[0040] At least four third calibration points are set on the point cloud camera;

[0041] Based on the coordinates of the third calibration point in the third marker coordinate system and the coordinates of the third calibration point in the point cloud camera coordinate system, the transformation matrix between the third marker coordinate system and the point cloud camera coordinate system is obtained.

[0042] Based on the binocular camera, the transformation matrix between the third marker coordinate system and the binocular camera coordinate system is obtained;

[0043] Based on the transformation matrix between the third marker coordinate system and the point cloud camera coordinate system, and the transformation matrix between the third marker coordinate system and the stereo camera coordinate system, the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system is obtained.

[0044] Optionally, where:

[0045] The dual-view navigation algorithm includes:

[0046] Based on the transformation matrix between the model coordinate system and the binocular camera coordinate system, the coordinates of the model coil in the binocular camera coordinate system are obtained.

[0047] Based on the coordinates of the model coil in the binocular camera coordinate system and the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, the coordinates of the model coil in the point cloud camera coordinate system are obtained.

[0048] Based on the coordinates of the model coil in the point cloud camera coordinate system and the point cloud matching matrix, the coordinates of the model coil in the MRI coordinate system are obtained, and the viewing angle of the model coil moving while the MRI data is stationary is obtained.

[0049] Optionally, where:

[0050] The dual-view navigation algorithm also includes:

[0051] The coordinates of the MRI data in the MRI coordinate system are obtained based on the coordinates of the MRI data in the MRI coordinate system and the point cloud matching matrix;

[0052] Based on the coordinates of the MRI data in the point cloud camera coordinate system and the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, the coordinates of the MRI data in the binocular camera coordinate system are obtained.

[0053] Based on the coordinates of the MRI data in the binocular camera coordinate system and the transformation matrix between the binocular camera coordinate system and the model coordinate system, the coordinates of the MRI data in the model coordinate system are obtained, and the viewing angle of the MRI data moving while the model coil remains stationary is obtained.

[0054] Compared with existing technologies, the medical navigation method provided by this invention achieves at least the following beneficial effects:

[0055] (1) The medical navigation method provided in this application uses a binocular camera and a point cloud camera. Through a point cloud matching algorithm, the method displays the positional relationship between the patient's MRI data and the model coil via a computer, and determines the target position based on the positional relationship between the model coil and the MRI data, thereby guiding the doctor to find the optimal stimulation position. In this way, the patient does not need to wear a reflective model, which not only expands the range of applicable people, but also helps to reduce the difficulty of operation and reduce human error.

[0056] (2) The medical navigation method provided in this application adopts a dual-view algorithm, which allows users to track the target position from two different perspectives, thereby enabling more effective tracking of the target position. This not only avoids the problem of inconvenient operation due to a single perspective, but also improves the accuracy of tracking, provides more precise and convenient navigation guidance, improves treatment efficiency and treatment effect, and reduces the difficulty of navigation operation.

[0057] Of course, any product implementing this invention does not necessarily need to achieve all of the technical effects described above at the same time.

[0058] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0059] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0060] Figure 1 The diagram shown is a flowchart of a medical navigation method provided in an embodiment of this application;

[0061] Figure 2 The diagram shown is a flowchart of a transformation matrix for establishing the binocular camera coordinate system and the model coordinate system provided in an embodiment of this application.

[0062] Figure 3 The diagram shown is a structural schematic of a model coil provided in an embodiment of this application;

[0063] Figure 4 The diagram shown is a structural schematic of a stimulation coil provided in an embodiment of this application;

[0064] Figure 5 The diagram shown is a flowchart of a method for acquiring face point cloud data according to an embodiment of this application.

[0065] Figure 6 The diagram shown is a flowchart of obtaining a point cloud matching matrix according to an embodiment of this application;

[0066] Figure 7 The diagram shown is a flowchart of an embodiment of this application for establishing a calibration matrix between a binocular camera coordinate system and a point cloud camera coordinate system;

[0067] Figure 8 The diagram shown is another flowchart for establishing the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system according to an embodiment of this application.

[0068] Figure 9 The diagram shown is a flowchart of a dual-view navigation method provided in an embodiment of this application.

[0069] Figure 10 The diagram shown is another flowchart of dual-view navigation provided in the embodiments of this application. Detailed Implementation

[0070] Certain terms are used in the specification and claims to refer to specific components. Those skilled in the art will understand that hardware manufacturers may use different names to refer to the same component. This specification and claims do not distinguish components by differences in name, but by differences in function. The term "comprising" throughout the specification and claims is an open-ended term and should be interpreted as "comprising but not limited to." "Approximately" means that within an acceptable margin of error, those skilled in the art can solve the technical problem and substantially achieve the technical effect within a certain margin of error. Furthermore, the term "coupled" here includes any direct and indirect electrical coupling means. Therefore, if a first device is described as coupled to a second device, it means that the first device can be directly electrically coupled to the second device, or indirectly electrically coupled to the second device through other devices or coupling means. The following descriptions in the specification are preferred embodiments for carrying out this application; however, these descriptions are for the purpose of illustrating the general principles of this application and are not intended to limit the scope of this application. The scope of protection of this application shall be determined by the appended claims.

[0071] Optical-assisted navigation is currently the most representative commercial TMS navigation system. In 2008, Lars Matthau... ··S et al. used a Polaris Spectra optical tracking device and an Adept Viper s850 six-axis robot to create a TMS robotic therapy system. The system uses the robot to hold the stimulation coil, and the optical tracking device is fixed by a bracket. A marker is fixed to the subject's head to help the optical tracking device locate the head coordinates. Using an optical navigation and positioning system allows for visualization of the transcranial magnetic stimulation (TMS) treatment process, improving the accuracy of stimulation coil positioning to some extent. However, current navigation systems are relatively cumbersome to operate; accidental camera touches may lead to re-registration issues. Furthermore, the visualization perspective is limited and inconvenient to operate. Additionally, using an infrared binocular camera alone requires the patient to wear a reflective mannequin, which may be unsuitable for patients with autism or other conditions.

[0072] In view of this, the present invention provides a medical navigation method that uses a binocular camera and a point cloud camera. Through a point cloud matching algorithm, the target position is determined based on the positional relationship between the model coil and MRI data, thereby enabling more effective tracking of the target position, improving tracking accuracy, providing more precise and convenient navigation guidance, improving treatment efficiency and efficacy, and reducing the operational difficulty of navigation.

[0073] The following detailed description is provided in conjunction with the accompanying drawings and specific embodiments.

[0074] Figure 1 The diagram shown is a flowchart of a medical navigation method provided in an embodiment of this application. Please refer to it. Figure 1 The medical navigation method provided in this application includes:

[0075] Step 1: Establish the transformation matrix between the stereo camera coordinate system and the model coordinate system;

[0076] Step 2: Acquire MRI data and convert the MRI data into MRI point cloud data;

[0077] Step 3: Use a point cloud camera to acquire environmental point cloud data and environmental RGB data, and obtain face point cloud data through the environmental point cloud data and environmental RGB data;

[0078] Step 4: Register the MRI point cloud data and the face point cloud data to obtain the point cloud matching matrix;

[0079] Step 5: Establish the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system;

[0080] Step 6: Using the transformation matrix between the binocular camera coordinate system and the model coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, and the point cloud matching matrix, the positional relationship between the model coil and the MRI data is displayed on the computer through a dual-view navigation algorithm, and the target position is determined based on the positional relationship.

[0081] For details, please refer to Figure 1 The medical navigation method provided in this application embodiment has a binocular camera and a point cloud camera. In step 1, a transformation matrix between the binocular camera coordinate system and the model coordinate system is established. When establishing the transformation matrix between the binocular camera coordinate system and the model coordinate system, a model coil with the same shape as the stimulation coil needs to be established first. Then, the transformation matrix between the binocular camera coordinate system and the model coordinate system is established with the help of the stimulation coil and the model coil.

[0082] It should be noted that, in order for the binocular camera to recognize the stimulation coil, a marker needs to be placed on the stimulation coil. For example, a first marker is fixed behind the stimulation coil. The first marker can consist of four first reflective balls and a first support. The stimulation coil with the marker is then placed within the field of view of the binocular camera. The number of first reflective balls can also be other, such as five or six, etc., and this application does not limit this.

[0083] In step 2, MRI data is acquired and converted into MRI point cloud data. MRI (Magnetic Resonance Imaging) data refers to magnetic resonance imaging, and point cloud data refers to the set of points obtained after acquiring the spatial coordinates of various sampling points on the surface of an object. In step 3, environmental point cloud data and environmental RGB data are acquired using a point cloud camera, and face point cloud data is obtained from these two datasets. The environmental point cloud data includes face point cloud data and non-face point cloud data of other objects surrounding the face. The environmental RGB data includes face data and non-face data within the RGB coordinate system. A face detection algorithm can extract facial feature information from the environmental RGB data. Registration of the extracted facial feature information within the environmental point cloud data yields the final face point cloud data.

[0084] It should be noted that in practical applications, it is not necessary to execute steps 1, 2, and 3 in that order. Steps 2 and 3 can be executed first, followed by step 1, or all three steps can be executed simultaneously. The specific steps can be set according to actual needs, and this application does not impose any specific restrictions on them.

[0085] After obtaining the MRI point cloud data and face point cloud data, step 4 is used to register the MRI point cloud data and face point cloud data, obtaining the transformation relationship between the MRI point cloud data and face point cloud data, i.e., the point cloud matching matrix. When registering the MRI point cloud data and face point cloud data, the RANSAC (Random Sample Consensus) algorithm can be used for coarse matching first, and then the ICP (Iterative Closest Point) algorithm can be used for precise matching. Here, coarse matching refers to a relatively rough registration when the transformation between the two point clouds is completely unknown, mainly to provide a better initial transformation value for fine registration. Precise matching refers to further optimization to obtain a more accurate transformation given an initial transformation.

[0086] Step 5 establishes the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system, thus obtaining the transformation relationship between the stereo camera and the point cloud camera. When establishing the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system, different calibration matrices can be obtained depending on whether the stereo camera and the point cloud camera remain fixed or move relative to each other.

[0087] It should be noted that, Figure 1 This is merely an illustrative representation of the steps included in the medical navigation method provided in this application, and is not intended to limit the execution order of each step. In other words, in practical applications, the execution order of step 5 and the aforementioned steps can be interchanged. For example, step 5 can be executed first and then step 1, or they can be executed simultaneously. This application does not make any specific limitations in this regard.

[0088] After obtaining the transformation matrix between the binocular camera coordinate system and the model coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, and the point cloud matching matrix, in step 6, the transformation matrix between the binocular camera coordinate system and the model coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, and the point cloud matching matrix are used to display the positional relationship between the model coil and the MRI data on the computer through the binocular camera and point cloud camera dual-view navigation algorithm. The perspectives of the model coil being stationary and the MRI data being relatively moving, or the perspectives of the MRI data being stationary and the model coil being relatively moving, are obtained respectively. The target position is determined according to the positional relationship between the model coil and the MRI data, thereby guiding the doctor to find the optimal stimulation position.

[0089] The medical navigation method provided in this application employs a binocular camera and a point cloud camera. Through a point cloud matching algorithm, it displays the patient's MRI data and the positional relationship between the model coils on a computer, and determines the target location based on this relationship, thereby guiding the doctor to find the optimal stimulation location. This eliminates the need for the patient to wear a reflective model, expanding the applicable population and reducing operational difficulty and human error.

[0090] Furthermore, this application employs a dual-view algorithm, enabling users to track the target position from two different perspectives. This allows for more effective target tracking, avoiding the inconvenience of a single perspective and improving tracking accuracy. It also provides more precise and convenient navigation guidance, enhancing treatment efficiency and effectiveness while reducing the difficulty of navigation operation.

[0091] Optionally, Figure 2 The diagram shown is a flowchart illustrating a transformation matrix for establishing the binocular camera coordinate system and the model coordinate system, as provided in an embodiment of this application. Figure 3 The diagram shown is a structural schematic of a model coil 100 provided in an embodiment of this application. Figure 4 The diagram shown is a structural schematic of the stimulation coil 200 provided in an embodiment of this application. Please refer to it. Figures 1-4 In step 1, the transformation matrix between the stereo camera coordinate system and the model coordinate system is established, specifically as follows:

[0092] Step 11: Create a model coil 100 using a computer, and set at least 4 first calibration points 101 in the model coil 100;

[0093] Step 12: Place a first marker (not shown in the figure) on the stimulation coil 200;

[0094] Step 13: Set second calibration points 201 on the stimulation coil 200 that correspond one-to-one with at least four first calibration points 101 in the model coil 100;

[0095] Step 14: Based on the coordinates of the first calibration point 101 in the model coordinate system and the coordinates of the second calibration point 201 in the first marker coordinate system, obtain the transformation matrix between the first marker coordinate system and the model coordinate system;

[0096] Step 15: Based on the stereo camera, obtain the transformation matrix between the first marker coordinate system and the stereo camera coordinate system;

[0097] Step 16: Based on the transformation matrices of the first marker coordinate system and the model coordinate system, and the transformation matrix of the first marker coordinate system and the stereo camera coordinate system, obtain the transformation matrix between the stereo camera coordinate system and the model coordinate system.

[0098] For details, please refer to Figures 1-4 In step 1, when establishing the transformation matrix between the binocular camera coordinate system and the model coordinate system, it is first necessary to create a model coil 100 with the same shape as the stimulation coil 200 using a computer, and set at least four first calibration points 101 in the model coil 100. Simultaneously, at least four second calibration points 201 are set on the stimulation coil 200, and each second calibration point 201 corresponds one-to-one with a first calibration point 101. For example, when setting the first calibration point 101, a two-dimensional Cartesian coordinate system can be calibrated on the surface of the model coil 100, taking two points symmetrically centered on the origin O on the X-axis and two points symmetrically centered on the origin O on the Y-axis. Thus, setting the second calibration point 201 on the stimulation coil 200 can also be done in the same way as setting the first calibration point 101. Of course, taking symmetrical points on the X and Y axes is only an illustrative example and is not intended to limit this application; the first calibration point 101 and the second calibration point 201 can be set according to the specific application.

[0099] To enable the binocular camera to recognize the stimulation coil 200, a marker needs to be placed on the stimulation coil 200. In step 12 of this embodiment, a first marker is placed on the stimulation coil 200. For example, the first marker is fixed behind the stimulation coil 200. The first marker can consist of four first reflective balls and a first support, with the four first reflective balls fixed to the first support. A second calibration point 201 is set on the stimulation coil 200. Since the first marker is bound to the stimulation coil 200, the second calibration point 201 exists at coordinates below the first marker.

[0100] When obtaining the coordinates of the second calibration point 201 under the first marker, a probe can be used. To enable the stereo camera to recognize the probe, markers need to be placed on it. For example, a second marker is attached to the probe, consisting of four second reflective balls and a second support. The four second reflective balls are fixed to the second support. The distance between the probe tip and each of the second reflective balls can be measured. For easy probe clicking, grooved stickers are affixed to the second marker. The probe tip is placed sequentially in the grooves. The stereo camera is used to obtain the probe's attitude matrix at each second marker and the attitude matrix of the first marker at that moment. Using the distance between the probe tip and each second reflective ball, the coordinates of the probe tip at each second marker under the stereo camera can be obtained. Multiplying the coordinates of the probe tip at each second marker under the stereo camera by the inverse of the attitude matrix of the first marker at the corresponding moment yields the coordinates of the probe tip at each second marker under the first marker, which is also the coordinates of the second marker under the first marker.

[0101] Since the coordinates of the first calibration point 101 and the second calibration point 201 are corresponding, the transformation matrix between the first calibration point 101 in the model coordinate system and the second calibration point 201 in the first marker coordinate system can be obtained. By placing the stimulation coil 200 within the field of view of the binocular camera, the transformation matrix between the first marker coordinate system and the binocular camera coordinate system can be obtained. Then, based on the transformation matrices of the first marker coordinate system and the model coordinate system, and the transformation matrix between the first marker coordinate system and the binocular camera coordinate system, the transformation matrix between the binocular camera coordinate system and the model coordinate system can be obtained.

[0102] It should be noted that, Figure 2 This illustration is intended only to demonstrate the steps involved in establishing the transformation matrix between the binocular camera coordinate system and the model coordinate system, and does not represent the exact execution order of each step. For example, the execution order of steps 12 and 13 can be reversed, as can the execution order of steps 15 and 14. In practical use, the order can be set according to specific needs, and this application does not impose any specific limitations on this.

[0103] Alternatively, please refer to Figure 1 and Figure 5 , Figure 5 The diagram shown is a flowchart of obtaining face point cloud data according to an embodiment of this application. In step 3, face point cloud data is obtained through environmental point cloud data and environmental RGB data. Specifically, step 31: extract face data from environmental RGB data using a face detection algorithm; step 32: register the environmental point cloud data to the RGB coordinate system and remove non-face point cloud data based on the face data to obtain face point cloud data.

[0104] For details, please refer to Figure 1 and Figure 5 The environmental RGB data contains facial feature information and non-facial feature information surrounding the face. When acquiring face point cloud data, face data is first extracted from the environmental RGB data using a face detection algorithm in step 31. Similarly, the environmental point cloud data contains both face and non-face point cloud data. The environmental point cloud data is registered to the RGB coordinate system, and information outside the face region is removed from the environmental point cloud data using the face data obtained in step 31. What remains is the face point cloud data. Because information irrelevant to the face is removed, the efficiency and robustness of subsequent point cloud matching algorithms can be improved.

[0105] It should be noted that after obtaining the face point cloud data, it needs to be registered to the point cloud camera coordinate system. During registration, if the point cloud camera coordinate system and the RGB coordinate system are inconsistent, the RGB data should be converted to the point cloud camera coordinate system first, and then the point cloud data should be processed. Alternatively, the point cloud data can be converted to the RGB coordinate system, processed, and then converted back to the point cloud camera coordinate system.

[0106] Alternatively, please refer to Figure 1 and Figure 6 , Figure 6 The diagram shown is a flowchart of obtaining a point cloud matching matrix according to an embodiment of this application. In step 4, the MRI point cloud data and the face point cloud data are registered to obtain the point cloud matching matrix. Specifically, step 41: using the RANSAC algorithm, the MRI point cloud data and the face point cloud data are coarsely matched to obtain coarse matching data; step 42: using the ICP algorithm, the coarse matching data is precisely matched to establish the relationship between the MRI point cloud data and the face point cloud data under the point cloud camera to obtain the point cloud matching matrix.

[0107] For details, please refer to Figure 1 and Figure 6 When registering MRI point cloud data and facial point cloud data, the RANSAC algorithm is first used for coarse registration. RANSAC is an iterative algorithm that correctly estimates mathematical model parameters from a set of data containing "outliers." "Outliers" generally refer to noise in the data, such as mismatches in matching and outliers in the estimated curve. RANSAC is an uncertain algorithm; it can only produce results with a certain probability, and this probability increases with the number of iterations. After obtaining the registration result from the RANSAC algorithm, the ICP algorithm is used for fine registration. ICP is a data registration algorithm that uses the nearest distance search method (point-to-point, point-to-line, or point-to-surface) to solve problems based on freeform surfaces. The specific algorithm flow of RANSAC and ICP can be found in existing algorithms and will not be elaborated here.

[0108] Alternatively, please refer to Figure 1 and Figure 7 , Figure 7 The diagram shows a flowchart of an embodiment of this application for establishing a calibration matrix between a stereo camera coordinate system and a point cloud camera coordinate system. In step 5, establishing the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system specifically involves: Step 51: Selecting at least 4 first coordinate points in the point cloud camera coordinate system; Step 52: Selecting at least 4 second coordinate points in the stereo camera coordinate system; Step 53: Calculating the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system based on the first and second coordinate points.

[0109] For details, please refer to Figure 1 and Figure 7 When establishing the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system, in step 51, at least 4 first coordinate points are selected in the point cloud camera coordinate system; then in step 52, at least 4 second coordinate points are selected in the stereo camera coordinate system; in step 53, the quaternion algorithm is used to calculate the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system using the first and second coordinate points.

[0110] The calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system obtained in this embodiment needs to ensure that the positions of the binocular camera and the point cloud camera are fixed. When there is a displacement between the two, it will affect the navigation process.

[0111] Besides the method described above, other methods can be used to establish the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system. Please refer to [the relevant documentation]. Figure 1 and Figure 8 , Figure 8 The diagram shows another flowchart for establishing the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system according to an embodiment of this application. In step 5, establishing the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system specifically involves: Step 61: Setting a third marker on the point cloud camera; Step 62: Setting at least four third calibration points on the point cloud camera; Step 63: Obtaining the transformation matrix between the third marker coordinate system and the point cloud camera coordinate system based on the coordinates of the third calibration points in the third marker coordinate system and the coordinates of the third calibration points in the point cloud camera coordinate system; Step 64: Obtaining the transformation matrix between the third marker coordinate system and the stereo camera coordinate system based on the stereo camera; Step 65: Obtaining the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system based on the transformation matrix between the third marker coordinate system and the point cloud camera coordinate system, and the transformation matrix between the third marker coordinate system and the stereo camera coordinate system.

[0112] For details, please refer to Figure 1 and Figure 8 In this embodiment, when establishing the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, a third marker is first set on the point cloud camera to facilitate binocular camera recognition. At least four third calibration points are set on the point cloud camera. Based on the coordinates of the third calibration points in the third marker coordinate system and the coordinates of the third calibration points in the point cloud camera coordinate system, the transformation matrix between the third marker coordinate system and the point cloud camera coordinate system is obtained. Obtaining the coordinates of the third calibration points in the third marker coordinate system can be achieved using a probe; specifically, refer to the method for obtaining the coordinates of the second calibration point 201 under the first marker in step 14 above, which will not be elaborated here.

[0113] Using a stereo camera, the transformation matrix between the third marker coordinate system and the stereo camera coordinate system is obtained. Then, based on the transformation matrices between the third marker coordinate system and the point cloud camera coordinate system, and between the two coordinate systems, the calibration matrix between the stereo camera coordinate system and the point cloud camera coordinate system is obtained. This embodiment establishes a connection between the third marker and the point cloud camera by fixing the third marker to the point cloud camera. This allows the stereo camera and the point cloud camera to move freely within the operating range during navigation without affecting the navigation process, simplifying the operation and reducing the difficulty of navigation.

[0114] It should be noted that, Figure 8 This is merely an illustrative representation of the steps included in the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, and is not intended to limit the execution order of the steps. For example, the execution order of steps 61 and 62 can be interchanged.

[0115] Alternatively, please refer to Figure 1 , Figures 3-4 and Figure 9 , Figure 9 The diagram shown is a flowchart of a dual-view navigation algorithm provided in an embodiment of this application. In step 6, the dual-view navigation algorithm includes: Step 71: Obtaining the coordinates of the model coil 100 in the binocular camera coordinate system based on the transformation matrix between the model coordinate system and the binocular camera coordinate system; Step 72: Obtaining the coordinates of the model coil 100 in the point cloud camera coordinate system based on the coordinates of the model coil 100 in the binocular camera coordinate system and the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system; Step 73: Obtaining the coordinates of the model coil 100 in the MRI coordinate system based on the coordinates of the model coil 100 in the point cloud camera coordinate system and the point cloud matching matrix, thus obtaining the viewpoint where the MRI data is stationary and the model coil 100 is moving.

[0116] For details, please refer to Figure 1 , Figures 3-4 and Figure 9In this embodiment, during dual-view navigation, in step 71, the coordinates of the model coil 100 in the binocular camera coordinate system are obtained based on the transformation matrix between the model coordinate system and the binocular camera coordinate system. Then, in step 72, the coordinates of the model coil 100 in the point cloud camera coordinate system are obtained based on the coordinates of the model coil 100 in the binocular camera coordinate system and the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system. In step 73, the coordinates of the model coil 100 in the MRI coordinate system are obtained based on the coordinates of the model coil 100 in the point cloud camera coordinate system and the point cloud matching matrix. Thus, target position tracking can be achieved from the perspective of stationary MRI data and moving model coil 100, guiding doctors to find the optimal stimulation position. This not only avoids the problem of inconvenient operation due to a single perspective but also improves tracking accuracy, providing more precise and convenient navigation guidance, improving treatment efficiency and effectiveness, and reducing the operational difficulty of navigation.

[0117] Alternatively, please refer to Figure 1 , Figures 3-4 and Figure 10 , Figure 10 The diagram shows another flowchart of dual-view navigation provided in this application embodiment. In step 6, the dual-view navigation algorithm further includes: Step 74: Obtaining the coordinates of the MRI data in the point cloud camera coordinate system based on the coordinates of the MRI data in the MRI coordinate system and the point cloud matching matrix; Step 75: Obtaining the coordinates of the MRI data in the binocular camera coordinate system based on the coordinates of the MRI data in the point cloud camera coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system; Step 76: Obtaining the coordinates of the MRI data in the model coordinate system based on the coordinates of the MRI data in the binocular camera coordinate system and the transformation matrix between the binocular camera coordinate system and the model coordinate system, thus obtaining the viewpoint of the moving MRI data while the model coil 100 remains stationary.

[0118] For details, please refer to Figure 1 , Figures 3-4 and Figure 10In this embodiment, during dual-view navigation, after obtaining the viewpoint where the MRI data is stationary and the model coil 100 is moving, step 74 further obtains the coordinates of the MRI data in the point cloud camera coordinate system based on the coordinates of the MRI data in the MRI coordinate system and the point cloud matching matrix. Then, in step 75, the coordinates of the MRI data in the binocular camera coordinate system are obtained based on the coordinates of the MRI data in the point cloud camera coordinate system and the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system. In step 76, the coordinates of the MRI data in the model coordinate system are obtained based on the coordinates of the MRI data in the binocular camera coordinate system and the transformation matrix between the binocular camera coordinate system and the model coordinate system. In this way, target position tracking can be achieved from the viewpoint where the model coil 100 is stationary and the MRI data is moving. This guides the doctor to find the optimal stimulation position, not only avoiding the problem of inconvenient operation due to a single viewpoint, but also improving the accuracy of tracking, providing more precise and convenient navigation guidance, improving treatment efficiency and effect, and reducing the difficulty of navigation operation.

[0119] As can be seen from the above embodiments, the beneficial effects of this application are:

[0120] (1) The medical navigation method provided in this application uses a binocular camera and a point cloud camera. Through a point cloud matching algorithm, the method displays the positional relationship between the patient's MRI data and the model coil via a computer, and determines the target position based on the positional relationship between the model coil and the MRI data, thereby guiding the doctor to find the optimal stimulation position. In this way, the patient does not need to wear a reflective model, which not only expands the range of applicable people, but also helps to reduce the difficulty of operation and reduce human error.

[0121] (2) The medical navigation method provided in this application adopts a dual-view algorithm, which allows users to track the target position from two different perspectives, thereby enabling more effective tracking of the target position. This not only avoids the problem of inconvenient operation due to a single perspective, but also improves the accuracy of tracking, provides more precise and convenient navigation guidance, improves treatment efficiency and treatment effect, and reduces the difficulty of navigation operation.

[0122] The foregoing description illustrates and describes several preferred embodiments of this application. However, as previously stated, it should be understood that this application is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the inventive concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of this application should be within the protection scope of the appended claims.

Claims

1. A medical navigation method characterized by, The method comprises the following steps: establishing a conversion matrix between a binocular camera coordinate system and a model coordinate system; obtaining MRI data and converting the MRI data into MRI point cloud data; obtaining environmental point cloud data and environmental RGB data by using a point cloud camera, and obtaining facial point cloud data through the environmental point cloud data and the environmental RGB data; registering the MRI point cloud data and the facial point cloud data to obtain a point cloud matching matrix; establishing a calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system; displaying the position relationship between a model coil and MRI data on a computer by using the conversion matrix between the binocular camera coordinate system and the model coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, and the point cloud matching matrix, and determining a target position according to the position relationship.

2. The medical navigation method of claim 1, wherein, The conversion matrix between the binocular camera coordinate system and the model coordinate system is established in the following manner: a model coil is established by using a computer, and at least four first calibration points are arranged in the model coil; a first marker is arranged on a stimulating coil; second calibration points corresponding to the at least four first calibration points in the model coil are arranged on the stimulating coil; a conversion matrix between the first marker coordinate system and the model coordinate system is obtained according to the coordinates of the first calibration points in the model coordinate system and the coordinates of the second calibration points in the first marker coordinate system; a conversion matrix between the first marker coordinate system and the binocular camera coordinate system is obtained according to the binocular camera; the conversion matrix between the binocular camera coordinate system and the model coordinate system is obtained according to the conversion matrix between the first marker coordinate system and the model coordinate system and the conversion matrix between the first marker coordinate system and the binocular camera coordinate system.

3. The medical navigation method of claim 1, wherein, The facial point cloud data is obtained through the environmental point cloud data and the environmental RGB data in the following manner: facial data is extracted from the environmental RGB data by using a facial detection algorithm; the environmental point cloud data is registered into an RGB coordinate system, and non-facial point cloud data is removed according to the facial data to obtain facial point cloud data.

4. The medical navigation method of claim 1, wherein, The point cloud matching matrix is obtained by registering the MRI point cloud data and the facial point cloud data in the following manner: the MRI point cloud data and the facial point cloud data are coarsely matched by using a RANSAC algorithm to obtain coarse matching data; the coarse matching data is accurately matched by using an ICP algorithm to establish a relationship between the MRI point cloud data and the facial point cloud data under the point cloud camera, and a point cloud matching matrix is obtained.

5. The medical navigation method of claim 1, wherein, The calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system is established in the following manner: at least four first coordinate points are selected in the point cloud camera coordinate system; at least four second coordinate points are selected in the binocular camera coordinate system; the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system is calculated according to the first coordinate points and the second coordinate points.

6. The medical navigation method of claim 1, wherein, The calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system is established in the following manner: a third marker is arranged on the point cloud camera; at least four third calibration points are arranged on the point cloud camera; According to the coordinates of the third calibration point in the third marker coordinate system and the coordinates of the third calibration point in the point cloud camera coordinate system, a conversion matrix of the third marker coordinate system and the point cloud camera coordinate system is obtained; According to the binocular camera, a conversion matrix of the third marker coordinate system and the binocular camera coordinate system is obtained; According to the conversion matrix of the third marker coordinate system and the point cloud camera coordinate system and the conversion matrix of the third marker coordinate system and the binocular camera coordinate system, a calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system is obtained.

7. The medical navigation method of claim 1, wherein, The dual-view navigation algorithm comprises: According to the model coordinate system and the conversion matrix of the binocular camera coordinate system and the model coordinate system, the coordinates of the model coil in the binocular camera coordinate system are obtained; According to the coordinates of the model coil in the binocular camera coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, the coordinates of the model coil in the point cloud camera coordinate system are obtained; According to the coordinates of the model coil in the point cloud camera coordinate system and the point cloud matching matrix, the coordinates of the model coil in the MRI coordinate system are obtained, and the view of MRI data immobility and model coil movement is obtained.

8. The medical navigation method of claim 7, wherein, The dual-view navigation algorithm further comprises: According to the coordinates of the MRI data in the MRI coordinate system and the point cloud matching matrix, the coordinates of the MRI data in the point cloud camera coordinate system are obtained; According to the coordinates of the MRI data in the point cloud camera coordinate system, the calibration matrix between the binocular camera coordinate system and the point cloud camera coordinate system, the coordinates of the MRI data in the binocular camera coordinate system are obtained; According to the coordinates of the MRI data in the binocular camera coordinate system, the conversion matrix of the binocular camera coordinate system and the model coordinate system, the coordinates of the MRI data in the model coordinate system are obtained, and the view of model coil immobility and MRI data movement is obtained.

Images