Method for performing surgical puncture navigation by applying virtual reality technology

Abstract

The invention relates to the technical field of puncture needle position guidance, in particular to a method for performing surgical puncture navigation by applying a virtual reality technology. The method comprises the following steps: creating a three-dimensional space model conforming to a real space and a three-dimensional model of a puncture needle; a plurality of points are marked on the puncture path of the puncture needle planned in advance, and the nearest line of the three-dimensional model of the puncture needle close to the puncture path of the puncture needle planned in advance is planned. According to the method, the three-dimensional space model and the real space of the operation are combined together, and the real world and the virtual world are combined, so that a doctor obtains the position of a puncture needle in the three-dimensional space model through MR glasses in the process of puncturing a patient through the puncture needle; when the position of the puncture needle deviates from the pre-planned puncture path of the puncture needle, a path for guiding the puncture needle to move is formed between the puncture needle and the pre-planned puncture path of the puncture needle, so that a doctor can quickly find the puncture position of the puncture needle according to the path.

Description

Technical Field

[0001] The invention relates to the technical field of puncture needle position guidance, and in particular to a method for surgical puncture navigation using virtual reality technology. Background Art

[0002] With the rapid development of medical technology, minimally invasive surgery has become an important development direction of modern surgery. Minimally invasive spinal surgery has gradually received widespread attention and application due to its advantages of reducing patient pain, shortening recovery time and reducing surgical risks.

[0003] In recent years, the concept of the metaverse has received extremely high attention. In the medical field, the metaverse can be considered to have the potential to solve business pain points in various aspects such as disease prevention, diagnosis, treatment, rehabilitation, education and innovation, and has demonstrated good results and huge potential in existing explorations.

[0004] MXR, including VR, AR and MR, is the technical support behind the metaverse. MXR combined with medical care has received high attention from countries around the world. MXR can be recognized mainly because of its ability to remotely provide standardized intervention methods and new types of content in a highly immersive and realistic manner, and its ability to be tailored to various clinical environments, thereby reducing costs and increasing efficiency in the medical process, and the mature development of virtual reality and augmented reality technologies.

[0005] MR technology has been used in various aspects of the medical and health field. As a high-risk and high-tech department, the introduction of MR technology into neurosurgery can provide patients with more precise, minimally invasive and individualized treatment plans. To a certain extent, it can reduce intraoperative damage to blood vessels, nerves and important functional brain tissues, reduce the probability of postoperative complications in patients, and improve patients' quality of life.

[0006] At present, the traditional presentation of medical imaging data is to generate a two-dimensional film after the completion of imaging examinations such as CT and MRI. The doctor then identifies the lesion by reading the film and then diagnoses the disease. In traditional minimally invasive surgery, the doctor often relies on experience to locate the position by touch, and then inserts the puncture needle according to the determined position. However, positioning by touch alone is not intuitive. During the operation, the doctor marks and draws lines based on his imagination, but the positioning method using positioning marks has large variability. The doctor needs to interrupt the operation to repeatedly locate the position during the operation. This not only increases the trouble for the doctor to perform minimally invasive surgery, but also prolongs the doctor's operation time and increases the radiation exposure of the doctor and the patient. Therefore, a method is needed to help the doctor guide the position. Summary of the invention

[0007] The purpose of the present invention is to provide a method for surgical puncture navigation using virtual reality technology to solve the problems raised in the above-mentioned background technology.

[0008] To achieve the above object, a method for surgical puncture navigation using virtual reality technology is provided, comprising the following steps:

[0009] S1. Collect content data of the real space, and define the objects in the real space as fixed objects and animal objects according to whether the objects in the real space move, collect specification information of the puncture needle and information of the puncture path of the puncture needle planned in advance, and obtain target information of the puncture needle for puncturing the human body;

[0010] S2, creating a three-dimensional space model that is consistent with the real space, and at the same time creating a three-dimensional model of the puncture needle, and also establishing the puncture path of the puncture needle planned in advance, and importing the three-dimensional model of the puncture needle and the puncture path of the puncture needle planned in advance into the three-dimensional space model;

[0011] S3. Mark several points on the puncture path of the puncture needle planned in advance. The several points are defined as key nodes. At the same time, multiple marking points are set at the tail end of the puncture needle three-dimensional model. At the same time, multiple markers are linearly set at the tail end of the puncture needle in reality. The positions of the markers correspond to the positions of the marking points on the three-dimensional model of the puncture needle. The markers are selected to be reflective stickers. The reflective stickers are wound around the tail end of the puncture needle to form a ring-shaped reflective area on the tail end of the puncture needle. The reflective stickers are set to a ring shape to facilitate multi-angle recognition by MR glasses.

[0012] S4, establishing an initial position point in the three-dimensional space, which is represented by the zero point coordinate P (0, 0, 0) of the three-dimensional space model, defining the coordinate point inside the three-dimensional space model, and setting a space observation point, which is the space position of the MR glasses worn on the human head in the three-dimensional space model;

[0013] S5, the MR glasses scan the real space, obtain the position of the spatial observation point in the three-dimensional space model and the real space, and simultaneously identify the position of the puncture needle in the real space in the three-dimensional space model;

[0014] S6. When the puncture needle and the spatial observation point move, the MR glasses match the three-dimensional model of the puncture needle after movement based on the position of the puncture needle in the real space in the three-dimensional spatial model and the distance from the MR glasses to the initial spatial observation point, adjust the position of the three-dimensional model of the puncture needle at this time, and plan the three-dimensional model of the puncture needle to be close to the nearest route of the puncture path of the puncture needle planned in advance.

[0015] As a further improvement of the technical solution, the specific steps of defining the coordinate points inside the three-dimensional space model and setting the space observation points in S4 are as follows:

[0016] S4.1. First, several distance points are set on the model in the three-dimensional space model. According to the models of fixed objects and animal objects in the real space corresponding to the distance points, fixed distance points and dynamic distance points are defined. At the same time, a standard scale is established. The standard scale is the same as the real scale. The spatial coordinates P of the distance point on the model in the three-dimensional space model to the initial position point are marked according to the distance from the initial position point to the spatial coordinates of the distance point. n (x n ,y n 、z n );

[0017] S4.2. Set a spatial observation point. The spatial observation point is a movable point. At the same time, the spatial observation point when the MR glasses and the three-dimensional space model are just connected is the initial point Y (0, 0, 0) of the MR glasses in the three-dimensional space model. At the same time, match the initial point Y (0, 0, 0) with the initial position point in the three-dimensional space model to determine the position of the initial point Y (0, 0, 0) in the three-dimensional space model.

[0018] As a further improvement of the technical solution, the method steps of the MR glasses in S5 scanning the real space to obtain the position of the spatial observation point in the three-dimensional space model and the real space are as follows:

[0019] S5.1. In the real space, the MR glasses emit light to scan the scene in the real space, obtain a two-dimensional depth map, and obtain the distance value from the MR glasses to the scene in the real space;

[0020] S5.2. Bring the scene in the real space scanned by the MR glasses into the three-dimensional space model, match the model of the scene in the real space in the three-dimensional space model, and after determining the model of the scene, bring the distance value from the MR glasses to the scene in the real space into the three-dimensional space model to determine the coordinate position of the spatial observation point corresponding to the MR glasses in the three-dimensional space model.

[0021] As a further improvement of the technical solution, the method steps for identifying the position of the puncture needle in the real space in the three-dimensional space model in S5 are:

[0022] S5.3, the MR glasses in the real space emit light, capture reflected light, and extract the reflective contour whose brightness exceeds the value V from the intensity of the captured reflected light;

[0023] S5.4, filtering the extracted reflective contours, retaining qualified reflective contours, and calculating the center points of the polygons of the retained reflective contours;

[0024] S5.5, bringing the obtained center point into the two-dimensional depth map, finding the plane coordinate position of the center point of the reflective profile in the two-dimensional depth map, and then using the formula of homogeneous coordinates to calculate the three-dimensional coordinates of the center point in the real space, and calculate the distance value d from the MR glasses to the center point;

[0025] S5.6. Calculate the spatial coordinates of the center point of the reflective contour in the three-dimensional space model through the real three-dimensional coordinates of the center point of the reflective contour and the distance value d to the MR glasses, and integrate the obtained qualified coordinates of the center point of the reflective contour.

[0026] As a further improvement of the technical solution, the step of filtering the extracted reflective contour in S5.4 is:

[0027] S5.4.1. Calculate the area of ​​the reflective contour and extract the reflective contours whose contour area is in the range of 1-100;

[0028] S5.4.2. Calculate the circularity of the reflective profile using the formula: Wherein α is the circularity, and its value range is 0.3-1.0. The reflective contours whose circularity is not within the value range are eliminated. Area refers to the area of ​​the polygon of the reflective contour, and perimeter refers to the perimeter of the reflective contour.

[0029] As a further improvement of the technical solution, the method for calculating the three-dimensional coordinates of the center point of the reflective profile in the three-dimensional space model in S5.6 is:

[0030] S5.6.1. Based on the two-dimensional depth map of the center point, the spatial coordinates of the center point of the reflective contour in the three-dimensional space model are obtained by collecting the initial point Y (0, 0, 0) of the spatial observation point where the MR glasses enter the three-dimensional space model and the distance value d from the center point of the reflective contour to the MR glasses, as well as the coordinate difference between the MR glasses in the real space and the center point of the reflective contour.

[0031] As a further improvement of the present technical solution, in S5.6.2, after integrating the center point coordinates of the acquired qualified reflective contours, multiple reflective contours are connected in the three-dimensional space model according to their coordinate positions in the real space, so that the multiple reflective contours are combined to form the coordinates of a puncture needle in the three-dimensional space model.

[0032] As a further improvement of the technical solution, the method steps of matching the three-dimensional model of the puncture needle after it moves according to the position of the puncture needle in the real space in the three-dimensional space model and the distance from the MR glasses to the initial space observation point in S6 are as follows:

[0033] S6.1. After the MR glasses move, they scan the device in the real space, obtain scene information in the real space and a two-dimensional depth map during movement, and bring the scene in the real space from the MR glasses into the three-dimensional space model, match the model of the scene in the three-dimensional space model corresponding to the scene in the real space, and then find multiple fixed-distance points on a fixed object in the scene model according to the matched scene model;

[0034] S6.2, then using the positions of the two-dimensional depth map of the multiple fixed-distance points when moving, calculate the distance data from the MR glasses after moving to the fixed-distance points, and use the calculated distance data to determine the position of the MR glasses after moving in the three-dimensional space model;

[0035] S6.3, repeat step S5 to obtain the coordinates of the puncture needle after the movement in the three-dimensional space model;

[0036] S6.4. Calculate the movement of the puncture needle in the three-dimensional space model based on the coordinates of the puncture needle in the three-dimensional space model after the movement and the coordinates of the puncture needle in the three-dimensional space model before the movement;

[0037] S6.5. According to the coordinates of the puncture needle after being moved in the three-dimensional space model and the coordinates of several key nodes on the puncture path of the puncture needle planned in advance, obtain the spatial points between the coordinates of several key nodes and the coordinates of the puncture needle after being moved in the three-dimensional space model, and use a matching algorithm to calculate the shortest route from the puncture needle in the three-dimensional space model to the puncture path of the pre-planned puncture needle.

[0038] As a further improvement of this technical solution, the specific steps of S6.5 are:

[0039] S6.5. Count a number of marking points on the puncture needle and a number of key nodes on the puncture path of the puncture needle planned in advance, and record the spatial points of the relevant three-dimensional coordinates involved between the number of marking points and the key nodes;

[0040] S6.5, using the Dijkstra algorithm to obtain the shortest route from each marking point to a key node among several key nodes on the puncture path of the pre-planned puncture needle;

[0041] S6.5. Integrate the nearest routes from the acquired several marking points to the pre-planned puncture path of the puncture needle, select the nearest one as the navigation route for the puncture needle to move toward the pre-planned puncture path of the puncture needle in the three-dimensional space model, and display the route in the three-dimensional space model.

[0042] As a further improvement of this technical solution,.

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] 1. In the method of using virtual reality technology for surgical puncture navigation, a three-dimensional space model and the real space of the operation are combined together, and the puncture path of the puncture needle is planned in advance. During the doctor's operation, through the combination of the real world and the virtual world, the doctor can obtain the position of the puncture needle in the three-dimensional space model through MR glasses during the puncture of the patient. When the position of the puncture needle deviates from the puncture path of the puncture needle planned in advance, a route to guide the movement of the puncture needle will be formed between the puncture needle and the puncture path of the puncture needle planned in advance, so that the doctor can quickly find the puncture position of the puncture needle according to the route, so that the puncture needle can accurately reach the target point that needs to be punctured, eliminating the trouble of repeated positioning for the doctor, speeding up the doctor's puncture speed, and shortening the time of the puncture operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of the overall steps of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Example 1

[0048] See also Figure 1 As shown, the purpose of this embodiment is to provide a method for surgical puncture navigation using virtual reality technology, including the following steps:

[0049] S1. Collect content data of the real space, and define the objects in the real space as fixed objects and animal objects according to whether the objects in the real space move, collect specification information of the puncture needle and information of the puncture path of the puncture needle planned in advance, and obtain target information of the puncture needle for puncturing the human body;

[0050] The real space is the space where the doctor performs puncture surgery on the patient. At the same time, the instruments used for surgery on the patient also belong to the content in the real space. Fixed objects are instruments that will not move during the operation, such as walls, fixed instruments, etc., and animal objects include surgical instruments for surgery on patients and puncture needles for puncturing patients. Among them, animal objects are mainly puncture needles for puncturing patients. The puncture path of the puncture needle planned in advance is the moving path of the puncture needle controlled by the doctor for the upcoming puncture operation.

[0051] S2. Create a three-dimensional space model that is consistent with the real space, that is, the three-dimensional space model is the same as the real space, wherein the three-dimensional space model includes a patient body model, a model of a medical device, and a model of an operating space, and at the same time create a three-dimensional model of a puncture needle. When creating the three-dimensional model of the puncture needle, set the length, diameter, material and other properties of the puncture needle so that the three-dimensional model of the puncture needle matches the puncture needle in actual medical treatment. At this time, the patient's body model is established together with the three-dimensional space model, and the puncture path of the puncture needle planned in advance is also established. One end of the puncture path of the puncture needle planned in advance is connected to the part of the patient's body that needs to be punctured, and the three-dimensional model of the puncture needle and the puncture path of the puncture needle planned in advance are imported into the three-dimensional space model;

[0052] S3. Mark several points on the puncture path of the puncture needle planned in advance. The several points are defined as key nodes. At the same time, multiple marking points are set at the tail end of the puncture needle three-dimensional model. At the same time, multiple markers are linearly set at the tail end of the puncture needle in reality. The positions of the markers correspond to the positions of the marking points on the three-dimensional model of the puncture needle. The markers are selected as reflective stickers. The reflective stickers are wound around the tail end of the puncture needle in a winding manner to form a ring-shaped reflective area on the tail end of the puncture needle. The reflective stickers are set to be ring-shaped to facilitate multi-angle recognition by MR glasses. When light is projected onto the reflective stickers, the reflective stickers reflect the light.

[0053] S4, establishing an initial position point in the three-dimensional space, which is represented by the zero point coordinate P (0, 0, 0) of the three-dimensional space model. After the initial position point is determined, the coordinate point is defined inside the three-dimensional space model so that each position in the three-dimensional space model has a defined point, and a space observation point is set at the same time. The space observation point is the space position of the MR glasses worn on the human head in the three-dimensional space model;

[0054] The specific steps of defining the coordinate points inside the three-dimensional space model and setting the space observation points in S4 are as follows:

[0055] S4.1. First, several distance points are set on the model in the three-dimensional space model. The distance points are coordinate points. According to the models of fixed objects and animal objects in the real space corresponding to the distance points, the fixed distance points and dynamic distance points are defined. The fixed distance points are the coordinate points of the fixed objects in the three-dimensional space model, and the dynamic distance points are the coordinate points of the animal objects in the three-dimensional space model. At the same time, a standard scale is established. The standard scale is the same as the real scale. The spatial coordinates P of the distance point on the model in the three-dimensional space model to the initial position point are marked according to the distance from the initial position point of the three-dimensional space model to the spatial coordinates of the distance point. n (x n ,y n 、z n );

[0056] S4.2. Set a spatial observation point. The spatial observation point is a movable point. At the same time, the spatial observation point when the MR glasses and the three-dimensional space model are just connected is the initial point Y (0, 0, 0) of the MR glasses in the three-dimensional space model. At the same time, match the initial point Y (0, 0, 0) with the initial position point in the three-dimensional space model to determine the position of the initial point Y (0, 0, 0) in the three-dimensional space model. After determining the position of the initial point Y (0, 0, 0), combine the spatial coordinates P from the distance point to the initial position point n (x n ,y n 、z n ) to obtain the coordinate relationship between the initial point Y(0, 0, 0) and the distance point.

[0057] The initial point is usually set at the initial position point to reduce the computational burden of determining the position of the initial point in the three-dimensional space model. At the same time, the position of the MR glasses in the three-dimensional space model is calculated to facilitate the subsequent import of the position of the puncture needle in the real space into the three-dimensional space model.

[0058] When the initial point and initial position of the MR glasses are not on the corresponding points in a 3D space model and real space, please refer to S5:

[0059] S5, the MR glasses scan the real space, obtain the position of the spatial observation point in the three-dimensional space model and the real space, and simultaneously identify the position of the puncture needle in the real space in the three-dimensional space model;

[0060] In S5, the MR glasses scan the real space to obtain the position of the spatial observation point in the three-dimensional space model and the real space. The steps are as follows:

[0061] S5.1. In the real space, the MR glasses emit light to scan the scene in the real space, obtain a two-dimensional depth map, and obtain the distance value from the MR glasses to the scene in the real space. The maximum length and width values ​​of the two-dimensional depth map are both 512 pixels, and the data of each pixel in the two-dimensional depth map is the distance value from the MR glasses to the scene in the real space;

[0062] S5.2. Bring the scene in the real space scanned by the MR glasses into the three-dimensional space model, match the model of the scene in the real space in the three-dimensional space model, and after determining the model of the scene, determine the approximate coordinate position of the spatial observation point corresponding to the MR glasses in the three-dimensional space model based on the fixed distance points in the three-dimensional space model, and then bring the distance value from the MR glasses to the scene in the real space into the three-dimensional space model to determine the coordinate position of the spatial observation point corresponding to the MR glasses in the three-dimensional space model.

[0063] At the same time, this method ensures that when the MR glasses do not enter the three-dimensional space model at the initial position, the real space and the three-dimensional space model can be matched in the MR glasses, ensuring that the image in the three-dimensional space model matches the real image viewed by the MR glasses.

[0064] The method steps for identifying the position of the puncture needle in the real space in the three-dimensional space model in S5 are:

[0065] S5.3. The MR glasses in the real space emit light and capture reflected light, including light reflected by other objects in the space where the patient is undergoing puncture surgery and light reflected by markers on the puncture needle, and extract the reflective contour whose brightness exceeds the value V from the captured reflected light intensity;

[0066] The value of V here is 1024, because the reflection intensity sensor sends a range value (0-65535), and the brightness of the reflective outline is around 1000, which can effectively distinguish the reflective point from the brightness of ordinary materials. Because 2^10 is easy to calculate in the computer, in order to improve the detection efficiency of the reflective outline, the setting value of V is set to 1024.

[0067] S5.4, filtering the extracted reflective contours, retaining qualified reflective contours, and calculating the center points of the polygons of the retained reflective contours;

[0068] The steps for filtering the extracted reflective contours in S5.4 are:

[0069] S5.4.1. Calculate the area of ​​the reflective contour, extract the reflective contours with contour areas within the range of 1-100, and delete the reflective contours with contour areas outside this range. The purpose is to remove the reflective contours with too large an area to prevent the reflective contours with too large an area from affecting the calculation of other reflective contours.

[0070] S5.4.2. Calculate the circularity of the reflective profile using the formula: Wherein α is the circularity, and its value range is 0.3-1.0. The reflective contours whose circularity is not within the value range are eliminated. Area refers to the area of ​​the polygon of the reflective contour, and perimeter refers to the circumference of the reflective contour. If the reflective contour is a perfect circle, then the result of this formula α=1. This method is used to determine the circularity of the polygon. Because the shape of the marker set on the puncture needle is circular, when the circularity of the reflective contour is too low, it means that the reflective object corresponding to the reflective contour is not the marker. By calculating the circularity, non-circular bright spots, such as long strips and irregular reflective contours, are eliminated, the number of light spot contours that are not reflected by the marker on the puncture needle is reduced, and the accuracy of identifying the marker on the puncture needle is improved.

[0071] The calculation method of the area Area of ​​the polygon of the reflective outline is calculated using the shoelace formula. The shoelace formula calculates the area by summing the product of the coordinates of each vertex on the polygon and the next vertex, and then dividing by 2 to obtain the absolute value. After calculating the area of ​​the polygon, the centroid coordinate algorithm is used to calculate the coordinates of all the vertices of the polygon, obtain the length between every two vertex coordinates, and then multiply the midpoint coordinates of each side by the length of the side, and then sum and divide by 6 times the area of ​​the polygon (6 times here is because the polygon is divided into n triangles). The area of ​​each triangle is 1 / n of the total area, and the centroid coordinates of each triangle are the average value of its vertex coordinates divided by 3 (because the centroid of the triangle is at 1 / 3 of the center). At this time, the position of the center point of the reflective outline is calculated.

[0072] S5.5, bringing the obtained center point into the two-dimensional depth map, finding the plane coordinate position of the center point of the reflective profile in the two-dimensional depth map, and then using the formula of homogeneous coordinates to calculate the three-dimensional coordinates of the center point in the real space, and calculate the distance value d from the MR glasses to the center point;

[0073] S5.6. Calculate the spatial coordinates of the center point of the reflective contour in the three-dimensional space model through the real three-dimensional coordinates of the center point of the reflective contour and the distance value d to the MR glasses, and integrate the obtained qualified coordinates of the center point of the reflective contour.

[0074] The method for calculating the three-dimensional coordinates of the center point of the reflective profile in the three-dimensional space model in S5.6 is:

[0075] S5.6.1. Based on the two-dimensional depth map of the center point, the spatial coordinates of the center point of the reflective contour in the three-dimensional space model are obtained by collecting the initial point Y (0, 0, 0) of the spatial observation point where the MR glasses enter the three-dimensional space model and the distance value d from the center point of the reflective contour to the MR glasses, as well as the coordinate difference between the MR glasses in the real space and the center point of the reflective contour.

[0076] For example, the coordinates of the MR glasses in the real space are (3, 5, 9), and the calculated coordinates of the center point of the reflective profile in the real space are (8, 9, 3). The difference between their coordinates is (5, 4, -6). At this time, the coordinates of the MR glasses are transferred to the three-dimensional space model. At this time, the MR glasses are at the initial point Y (0, 0, 0). According to the coordinate difference between the MR glasses and the center point in the real space is (5, 4, -6), the position of the center point in the three-dimensional space model is calculated to be the coordinate difference (5, 4, -6). The distance value d from the center point of the reflective profile to the MR glasses is used to measure whether the distance data from the center point to the MR glasses in the three-dimensional space model is correct.

[0077] S5.6.2. After integrating the center point coordinates of the obtained qualified reflective contours, multiple reflective contours are connected in the three-dimensional space model according to the coordinate positions in the real space, so that the multiple reflective contours are combined to form the coordinates of a puncture needle in the three-dimensional space model.

[0078] By determining the coordinates of the puncture needle in the real space in the three-dimensional space model, the three-dimensional space model can update the movement of the puncture needle.

[0079] S6. When the puncture needle and the spatial observation point move, the MR glasses match the three-dimensional model of the puncture needle after movement based on the position of the puncture needle in the real space in the three-dimensional spatial model and the distance from the MR glasses to the initial spatial observation point, adjust the position of the three-dimensional model of the puncture needle at this time, and plan the three-dimensional model of the puncture needle to be close to the nearest route of the puncture path of the puncture needle planned in advance.

[0080] In S6, the MR glasses match the three-dimensional model of the moved puncture needle according to the position of the puncture needle in the real space in the three-dimensional space model and the distance from the MR glasses to the initial space observation point, as follows:

[0081] S6.1. After the MR glasses move, they scan the device in the real space, obtain scene information in the real space and a two-dimensional depth map during movement, and bring the scene in the real space from the MR glasses into the three-dimensional space model, match the model of the scene in the three-dimensional space model corresponding to the scene in the real space, and then find multiple fixed-distance points on a fixed object in the scene model according to the matched scene model;

[0082] S6.2, then using the positions of the two-dimensional depth map of the multiple fixed-distance points when moving, calculate the distance data from the MR glasses after moving to the fixed-distance points, and use the calculated distance data to determine the position of the MR glasses after moving in the three-dimensional space model;

[0083] S6.3, repeat step S5 to obtain the coordinates of the puncture needle after the movement in the three-dimensional space model;

[0084] S6.4. Calculate the movement of the puncture needle in the three-dimensional space model based on the coordinates of the puncture needle after the movement and the coordinates of the puncture needle before the movement. By calculating the movement of the puncture needle in the three-dimensional space model, the MR glasses can report the movement of the puncture needle, so as to provide a prompt to the doctor.

[0085] S6.5. According to the coordinates of the puncture needle after being moved in the three-dimensional space model and the coordinates of several key nodes on the puncture path of the puncture needle planned in advance, obtain the spatial points between the coordinates of several key nodes and the coordinates of the puncture needle after being moved in the three-dimensional space model, and use a matching algorithm to calculate the shortest route from the puncture needle in the three-dimensional space model to the puncture path of the pre-planned puncture needle.

[0086] The specific steps of S6.5 are:

[0087] S6.5. Count a number of marking points on the puncture needle and a number of key nodes on the puncture path of the puncture needle planned in advance, and record the spatial points of the relevant three-dimensional coordinates involved between the number of marking points and the key nodes;

[0088] S6.5, using the Dijkstra algorithm to obtain the shortest route from each marking point to a key node among several key nodes on the puncture path of the pre-planned puncture needle;

[0089] S6.5. Integrate the nearest routes from the acquired several marking points to the pre-planned puncture path of the puncture needle, select the nearest one as the navigation route for the puncture needle to move toward the pre-planned puncture path of the puncture needle in the three-dimensional space model, and display the route in the three-dimensional space model.

[0090] The doctor adjusts the puncture position of the puncture needle by observing the route displayed in the three-dimensional space model, eliminating the trouble of repeated positioning for the doctor, so that the puncture needle can be quickly moved to the designated puncture path planned in advance, ensuring that the doctor can accurately puncture the patient with the puncture needle, speeding up the doctor's operation and shortening the patient's operation time.

[0091] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for surgical puncture navigation using virtual reality technology, characterized in that: The following steps are involved: S1. Collect content data of the real space, and define the objects in the real space as fixed objects and animal objects according to whether the objects in the real space move, collect specification information of the puncture needle and information of the puncture path of the puncture needle planned in advance, and obtain target information of the puncture needle for puncturing the human body; S2, creating a three-dimensional space model that is consistent with the real space, and at the same time creating a three-dimensional model of the puncture needle, and also establishing the puncture path of the puncture needle planned in advance, and importing the three-dimensional model of the puncture needle and the puncture path of the puncture needle planned in advance into the three-dimensional space model; S3. Mark several points on the puncture path of the pre-planned puncture needle, and define the several points as key nodes. Set multiple marking points at the tail end of the 3D model of the puncture needle. Set multiple markers linearly at the tail end of the puncture needle in reality. The positions of the markers correspond to the positions of the marking points on the 3D model of the puncture needle. Reflective stickers are used as markers. S4, establishing an initial position point in the three-dimensional space, which is represented by the zero point coordinate P (0, 0, 0) of the three-dimensional space model, defining the coordinate point inside the three-dimensional space model, and setting a space observation point, which is the space position of the MR glasses worn on the human head in the three-dimensional space model; S5, the MR glasses scan the real space, obtain the position of the spatial observation point in the three-dimensional space model and the real space, and simultaneously identify the position of the puncture needle in the real space in the three-dimensional space model; S6. When the puncture needle and the spatial observation point move, the MR glasses match the three-dimensional model of the puncture needle after movement based on the position of the puncture needle in the real space in the three-dimensional spatial model and the distance from the MR glasses to the initial spatial observation point, adjust the position of the three-dimensional model of the puncture needle at this time, and plan the three-dimensional model of the puncture needle to be close to the nearest route of the puncture path of the puncture needle planned in advance.

2. The method for surgical puncture navigation using virtual reality technology according to claim 1, characterized in that: The specific steps of defining the coordinate points inside the three-dimensional space model and setting the space observation points in S4 are as follows: S4.

1. First, several distance points are set on the model in the three-dimensional space model. According to the models of fixed objects and animal objects in the real space corresponding to the distance points, fixed distance points and dynamic distance points are defined. At the same time, a standard scale is established. The standard scale is the same as the real scale. The spatial coordinates P of the distance point on the model in the three-dimensional space model to the initial position point are marked according to the distance from the initial position point to the spatial coordinates of the distance point. n (x n ,y n 、z n ); S4.

2. Set a spatial observation point. The spatial observation point is a movable point. At the same time, the spatial observation point when the MR glasses and the three-dimensional space model are just connected is the initial point Y (0, 0, 0) of the MR glasses in the three-dimensional space model. At the same time, match the initial point Y (0, 0, 0) with the initial position point in the three-dimensional space model to determine the position of the initial point Y (0, 0, 0) in the three-dimensional space model.

3. The method for surgical puncture navigation using virtual reality technology according to claim 2, characterized in that: In S5, the MR glasses scan the real space to obtain the position of the spatial observation point in the three-dimensional space model and the real space. The method steps are: S5.

1. In the real space, the MR glasses emit light to scan the scene in the real space, obtain a two-dimensional depth map, and obtain the distance value from the MR glasses to the scene in the real space; S5.

2. Bring the scene in the real space scanned by the MR glasses into the three-dimensional space model, match the model of the scene in the real space in the three-dimensional space model, and after determining the model of the scene, bring the distance value from the MR glasses to the scene in the real space into the three-dimensional space model to determine the coordinate position of the spatial observation point corresponding to the MR glasses in the three-dimensional space model.

4. The method for surgical puncture navigation using virtual reality technology according to claim 3, characterized in that: The method steps for identifying the position of the puncture needle in the real space in the three-dimensional space model in S5 are: S5.3, the MR glasses in the real space emit light, capture reflected light, and extract the reflective contour whose brightness exceeds the value V from the intensity of the captured reflected light; S5.4, filtering the extracted reflective contours, retaining qualified reflective contours, and calculating the center points of the polygons of the retained reflective contours; S5.5, bringing the obtained center point into the two-dimensional depth map, finding the plane coordinate position of the center point of the reflective profile in the two-dimensional depth map, and then using the formula of homogeneous coordinates to calculate the three-dimensional coordinates of the center point in the real space, and calculate the distance value d from the MR glasses to the center point; S5.

6. Calculate the spatial coordinates of the center point of the reflective contour in the three-dimensional space model through the real three-dimensional coordinates of the center point of the reflective contour and the distance value d to the MR glasses, and integrate the obtained qualified coordinates of the center point of the reflective contour.

5. The method for surgical puncture navigation using virtual reality technology according to claim 4, characterized in that: The step of filtering the extracted reflective contour in S5.4 is: S5.4.

1. Calculate the area of ​​the reflective contour and extract the reflective contours whose contour area is in the range of 1-100; S5.4.

2. Calculate the circularity of the reflective profile using the formula: Wherein α is the circularity, and its value range is 0.3-1.

0. The reflective contours whose circularity is not within the value range are eliminated. Area refers to the area of ​​the polygon of the reflective contour, and perimeter refers to the perimeter of the reflective contour.

6. The method for surgical puncture navigation using virtual reality technology according to claim 4, characterized in that: The method for calculating the three-dimensional coordinates of the center point of the reflective profile in the three-dimensional space model in S5.6 is: S5.6.

1. Based on the two-dimensional depth map of the center point, the spatial coordinates of the center point of the reflective contour in the three-dimensional space model are obtained by collecting the initial point Y (0, 0, 0) of the spatial observation point where the MR glasses enter the three-dimensional space model and the distance value d from the center point of the reflective contour to the MR glasses, as well as the coordinate difference between the MR glasses in the real space and the center point of the reflective contour.

7. The method for surgical puncture navigation using virtual reality technology according to claim 4, characterized in that: S5.6.2, after integrating the center point coordinates of the obtained qualified reflective contours, multiple reflective contours are connected in the three-dimensional space model according to the coordinate positions in the real space, so that the multiple reflective contours are combined to form the coordinates of a puncture needle in the three-dimensional space model.

8. The method for surgical puncture navigation using virtual reality technology according to claim 7, characterized in that: The method steps of the MR glasses in S6 matching the three-dimensional model of the moved puncture needle according to the position of the puncture needle in the real space in the three-dimensional space model and the distance from the MR glasses to the initial space observation point are as follows: S6.

1. After the MR glasses move, they scan the device in the real space, obtain scene information in the real space and a two-dimensional depth map during movement, and bring the scene in the real space from the MR glasses into the three-dimensional space model, match the model of the scene in the real space in the three-dimensional space model, and then find multiple fixed-distance points on a fixed object in the scene model according to the matched scene model; S6.2, then using the positions of the two-dimensional depth map of the multiple fixed-distance points when moving, calculate the distance data from the MR glasses after moving to the fixed-distance points, and use the calculated distance data to determine the position of the MR glasses after moving in the three-dimensional space model; S6.3, repeat step S5 to obtain the coordinates of the puncture needle after the movement in the three-dimensional space model; S6.

4. Calculate the movement of the puncture needle in the three-dimensional space model based on the coordinates of the puncture needle in the three-dimensional space model after the movement and the coordinates of the puncture needle in the three-dimensional space model before the movement; S6.

5. According to the coordinates of the puncture needle after being moved in the three-dimensional space model and the coordinates of several key nodes on the puncture path of the puncture needle planned in advance, obtain the spatial points between the coordinates of several key nodes and the coordinates of the puncture needle after being moved in the three-dimensional space model, and use a matching algorithm to calculate the shortest route from the puncture needle in the three-dimensional space model to the puncture path of the pre-planned puncture needle.

9. The method for surgical puncture navigation using virtual reality technology according to claim 8, characterized in that: The specific steps of S6.5 are: S6.

5. Count a number of marking points on the puncture needle and a number of key nodes on the puncture path of the puncture needle planned in advance, and record the spatial points of the relevant three-dimensional coordinates involved between the number of marking points and the key nodes; S6.5, using the Dijkstra algorithm to obtain the shortest route from each marking point to a key node among several key nodes on the puncture path of the pre-planned puncture needle; S6.

5. Integrate the nearest routes from the acquired several marking points to the pre-planned puncture path of the puncture needle, select the nearest one as the navigation route for the puncture needle to move toward the pre-planned puncture path of the puncture needle in the three-dimensional space model, and display the route in the three-dimensional space model.

Images