Method and system for navigation in intraspinal puncture based on mixed reality technology

Abstract

Provided are a method and system for navigation in intraspinal puncture based on mixed reality technology. The method comprises: step S1: receiving a click operation on a skin surface (10) in a three-dimensional image to give a collision point (a), wherein the three-dimensional image is formed by three-dimensional reconstruction based on mixed reality technology; step S2: generating a ray from the collision point (a) to a target point (b) in the spinal canal in the three-dimensional image, and detecting whether the ray collides with a bone (20) or a blood vessel before reaching the target point (b), and when it is detected that the ray does not collide with the bone (20) or the blood vessel before reaching the target point (b), determining that the collision point is an optimal needle insertion point; and S3: when the optimal needle insertion point is present, generating a visual navigation path from the optimal needle insertion point to the target point. The method and system provide a fast, real-time, accurate, and minimally invasive navigation scheme for intraspinal puncture.

Description

A navigation method and system for intraspinal puncture based on mixed reality technology Technical Field

[0001] The present invention relates to the field of digital medical technology, and in particular to navigation technology for intraspinal puncture. Background Art

[0002] Currently, most spinal punctures are performed blindly, relying on the physician's understanding of the anatomy and experience to locate the needle by feel. This can be difficult or even unsuccessful in elderly patients with degenerative changes in the lumbar spine, such as bone hyperplasia. In recent years, studies have demonstrated improved puncture success rates using traditional two-dimensional imaging (X-rays, ultrasound, etc.). However, this method poses the risk of radiation contamination to the physician performing the procedure. Furthermore, there are challenges with unintuitive touch positioning, high variability in marker positioning, and the need for repeated interruptions to locate the needle.

[0003] Summary of the Invention

[0004] In order to overcome the above technical deficiencies, the first aspect of the present invention provides a navigation method for spinal puncture based on mixed reality technology, which comprises:

[0005] Step S1: receiving a click operation on a skin surface of a three-dimensional image to obtain a collision point, wherein the three-dimensional image is three-dimensionally reconstructed based on mixed reality technology;

[0006] Step S2: Draw a ray from the collision point to the target point in the spinal canal within the three-dimensional image, and detect whether the ray collides with bones or blood vessels before reaching the target point; if it is detected that the ray does not collide with bones or blood vessels before reaching the target point, the collision point is determined to be the optimal needle insertion point;

[0007] Step S3: When an optimal needle insertion point exists, a visual navigation path from the optimal needle insertion point to the target point is generated.

[0008] Furthermore, in step S2, when it is detected that the ray collides with bones or blood vessels before reaching the target point, the method further includes:

[0009] Step S4: determining that the collision point is invalid;

[0010] Step S5: Find the direction vector from the collision point to the target point as a reference vector;

[0011] Step S6: With the target point as the center, find 100 to 500 vectors near the reference vector whose angles with the reference vector are 0 to 1 degrees as candidate vectors;

[0012] Step S7: Calculate the position point on the skin surface of the three-dimensional image corresponding to the candidate vector as the candidate needle insertion point;

[0013] Step S8: A ray is drawn from the candidate needle insertion point to the target point in the spinal canal within the three-dimensional image, and it is detected whether the ray collides with bones or blood vessels before reaching the target point;

[0014] Step S9: If the optimal needle insertion point has not been found after 100 to 500 ray detections, the angle between the candidate vector and the reference vector is further increased by 1 degree, and then steps S5 to S8 are repeated, and so on. The maximum angle between the candidate vector and the reference vector does not exceed 5 degrees.

[0015] Step S10: When it is detected that the ray does not collide with bones or blood vessels before reaching the target point, the candidate needle insertion point corresponding to the ray is determined to be the optimal needle insertion point.

[0016] Furthermore, in step S8, when it is detected that the ray does not collide with bones or blood vessels before reaching the target point, the collision point is determined to be the optimal needle insertion point.

[0017] Furthermore, in step S9, when the angle between the candidate vector and the reference vector is 5 degrees and the optimal needle insertion point still cannot be found, it is determined that the optimal needle insertion point does not exist.

[0018] A second aspect of the present application provides a navigation system for spinal puncture based on mixed reality technology, comprising:

[0019] A collision point acquisition module, the collision point acquisition module is used to receive a click operation on the skin surface of a three-dimensional image to obtain a collision point, the three-dimensional image being three-dimensionally reconstructed based on mixed reality technology;

[0020] a ray detection module, which is used to draw a ray from the collision point to the target point in the spinal canal within the three-dimensional image, and detect whether the ray collides with bones or blood vessels before reaching the target point; if it is detected that the ray does not collide with bones or blood vessels before reaching the target point, the collision point is determined to be the optimal needle insertion point;

[0021] The navigation path generation module is used to generate a visual navigation path from the optimal needle insertion point to the target point when there is an optimal needle insertion point.

[0022] Furthermore, when it is detected that the ray collides with bones or blood vessels before reaching the target, the ray detection module is further configured to:

[0023] (1) Determine that the collision point is invalid;

[0024] (2) Find the direction vector from the collision point to the target point as a reference vector;

[0025] (3) With the target point as the center, find 100 to 500 vectors near the reference vector whose angles with the reference vector are 0 to 1 degrees as candidate vectors;

[0026] (4) calculating the position point corresponding to the candidate vector on the skin surface of the three-dimensional image as the candidate needle insertion point;

[0027] (5) A ray is drawn from the candidate needle insertion point to the target point in the spinal canal located inside the three-dimensional image to detect whether the ray hits bones or blood vessels before reaching the target point;

[0028] (6) When the best needle insertion point has not been found after 100 to 500 ray detections, the angle between the candidate vector and the reference vector is further increased by 1 degree, and then steps (2) to (5) are repeated, and so on. The maximum angle between the candidate vector and the reference vector does not exceed 5 degrees.

[0029] (7) When it is detected that the ray does not collide with bones or blood vessels before reaching the target, the candidate needle insertion point corresponding to the ray is determined to be the optimal needle insertion point.

[0030] Furthermore, the ray detection module is further configured to: when the angle between the candidate vector and the reference vector is 5 degrees and the optimal needle insertion point still cannot be found, determine that the optimal needle insertion point does not exist.

[0031] The third aspect of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the above-mentioned navigation method for intraspinal puncture based on mixed reality technology.

[0032] Compared with the existing technology, the above technical solution has the following beneficial effects:

[0033] The technical solution of the present application uses mixed reality technology to virtually reconstruct real lesions into 1:1 restored and adapted stereo images that can be directly viewed through. Through precise algorithms, it can identify the position of the hand in one second and calculate the navigation path of the puncture in seconds, thereby providing an extremely fast, real-time, accurate, and minimally invasive surgical navigation solution for puncture surgeries. The technical solution of the present application integrates the technical advantages of 1:1 three-dimensional lesion reconstruction, 1-second virtual-real adaptation (using the Unity engine, with underlying optimization, batch processing technology, multi-threading support, and lightweight components, and the algorithm can be executed about 30 times in 1 second) and millimeter-level tracking navigation (100-500 rays will be emitted within each degree range. Taking 100 rays as an example, the arc length corresponding to 1 degree is about 0.017 mm), helping surgeons to make preoperative plans, enabling doctors to see through, understand, and accurately identify surgical targets, optimize personalized intraoperative real-time decision-making, improve surgical quality and efficiency, and reduce radiation damage to doctors. It is particularly suitable for navigation of puncture surgeries in departments such as pain management, anesthesia, interventional medicine, neurosurgery, and orthopedics. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] FIG1 is a flow chart of a navigation method for intraspinal puncture based on mixed reality technology in one embodiment of the present application;

[0035] FIG2 is a schematic diagram showing a ray starting from a collision point without colliding with bones or blood vessels before reaching the target. In this case, the collision point is the optimal needle insertion point.

[0036] FIG3 is a schematic diagram showing a ray starting from a collision point colliding with bones or blood vessels before reaching the target point. In this case, the collision point is invalid.

[0037] FIG4 is a schematic diagram showing the principle of searching for candidate needle insertion points when the collision point is detected to be invalid.

[0038] Reference numerals:

[0039] 10 - skin surface of 3D image; 20 - skeleton; a - collision point; b - target point; c - calculation area. DETAILED DESCRIPTION

[0040] The advantages of the present invention are further described below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that the following specific description is illustrative rather than restrictive and should not be used to limit the scope of protection of the present invention.

[0041] This embodiment provides a navigation method for spinal puncture based on mixed reality technology, as shown below:

[0042] Step 1: Obtain a CT image of the patient.

[0043] Step 2: Perform professional 3D reconstruction and add targets within the specified area.

[0044] The method for obtaining three-dimensional images based on CT images is as follows:

[0045] 1. Import the DICOM file into MIMICS software. 2. Extract Template: Select the threshold for the area to be extracted (the threshold varies depending on the density of each area; there is no fixed threshold; it depends on the density). Depending on the image, the selected area should be within the threshold range. For example, the threshold for bone is generally between (226, 2976). The extracted area will be displayed on the image. 3. Cropping: The extracted template will often include some unwanted tissue structures. In this case, you will need to use the cropping tool to remove the unnecessary parts. You can modify it directly on the image or crop it using the previewed 3D model. 4. Solidify: The extracted template is simply an overlay of layers. Solidify the template to obtain a solid model that can be exported. Select the template to be solidified and click Calculate Part. In this option, set the post-solidification effect, such as model quality, smoothness, and gap size. After setting the parameters, click OK to generate the required solid at the Object location. For example: the skin belongs to soft tissue, but when selecting, basically select all of it, export the whole, use the cropping tool, cut out the required part and then use the mesh to generate it, remove impurities, and then copy the required skin, remove the new skin required area, delete all the unnecessary areas, and the remaining is the skin part. 4. Adding targets: Targets are added using 3D balls in the analysis bar tool of MIMICS. In the three-dimensional view, place the balls at the required parts. The placed balls can be adjusted to the required positions by moving. Right-click to view the ball properties, set the size and color of the balls, etc. 5. Export STL files: Save the extracted model and the placed targets by exporting STL files. During the extraction process, the solid model and the added balls need to be extracted separately, and the name of each part must be marked during the extraction.

[0046] The above method can be used to virtually reconstruct the real lesion into a 1:1 restored and adapted stereoscopic image that can be directly viewed through.

[0047] Step 3: Upload the STL file to the mixed reality glasses. Preferably, the mixed reality glasses of this embodiment use the Unity engine, which features underlying optimization, batch processing technology, multi-threading support, and lightweight components. The algorithm can be executed approximately 30 times per second, achieving a 1-second virtual-reality adaptation effect.

[0048] Step 4: Open the intraspinal puncture navigation software installed in the mixed reality smart glasses. For example, the mixed reality smart glasses include a memory, a processor, and a computer program stored in the memory and executable on the processor (i.e., the intraspinal puncture navigation software corresponding to the intraspinal puncture navigation method based on mixed reality technology of the present application). When the computer program is executed by the processor, the intraspinal puncture navigation method based on mixed reality technology of the present application (i.e., steps 4.1 to 4.3 below) is implemented, as shown in FIG1 :

[0049] Step 4.1: Receive a click operation on the surface of the 3D image skin 10 to obtain a collision point a.

[0050] Specifically, the position of the hand in space is captured in one second by the camera on the mixed reality glasses, and a position point where the finger clicks on the skin in the stereoscopic image is captured, which is the collision point a.

[0051] Step 4.2: Draw a ray from the collision point a to the target point b located in the spinal canal inside the 3D image, and detect whether the ray collides with bones or blood vessels before reaching the target point b (ray detection is sent from point a to point b to determine whether it intersects with the levels of other organs).

[0052] If the ray detects that it has not collided with bone or blood vessels before reaching target point b, then collision point a is determined to be the optimal insertion point. Figure 2 shows a 3D image reconstructed using mixed reality technology and capable of direct perspective. 10 represents the skin surface in the 3D image, 20 represents the bone, a represents the collision point, and b represents the target point (target b is pre-determined by the surgeon within the surgical field on the awl). In this figure, the ray emitted from a does not collide with bone or blood vessels before reaching b, thus determining a to be the optimal insertion point.

[0053] When it is detected that the ray collides with bones or blood vessels before reaching the target point b, the following steps are further performed:

[0054] (1) The collision point a is determined to be invalid. As shown in Figure 3, the ray emitted from a collides with the bone before reaching b. Therefore, the current collision point a is determined to be invalid.

[0055] (2) Find the direction vector from the collision point a to the target point b (the direction from the needle insertion point a to the target point b, that is, the ab vector) as the reference vector;

[0056] (3) With the target point b as the center, 100 rays in the calculation area c near the reference vector whose angle with the reference vector is 1 degree are selected as candidate vectors, as shown in Figure 4;

[0057] 100 rays are emitted within each 1-degree angle range, and the arc length corresponding to 1 degree is approximately 0.017 mm. Therefore, the technical solution of the present application can achieve millimeter-level tracking and navigation.

[0058] (4) calculating the position point on the surface of the three-dimensional image skin 10 corresponding to the candidate vector as a candidate needle insertion point;

[0059] (5) A ray is drawn from the candidate needle insertion point to the target point b located in the spinal canal inside the three-dimensional image, and whether the ray collides with bones or blood vessels before reaching the target point b is detected; if it is detected that the ray does not collide with bones or blood vessels before reaching the target point b, the collision point a is determined to be the optimal needle insertion point;

[0060] (6) When the optimal needle entry point has not been found after 100 ray detections, the angle between the candidate vector and the reference vector is further enlarged by 1 degree, and then the same method is repeated, and so on. The maximum angle between the candidate vector and the reference vector does not exceed 5 degrees. When the angle between the candidate vector and the reference vector is 5 degrees and the optimal needle entry point is still not found, it is judged that there is no optimal needle entry point. When it is detected that the ray does not collide with the bone or blood vessel before reaching the target point b, the candidate needle entry point corresponding to the ray is judged to be the optimal needle entry point.

[0061] Step 4.3: When an optimal needle insertion point exists, a visual navigation path is generated from the optimal needle insertion point to the target point b.

[0062] The visual navigation path is generated by an algorithm, which controls the zooming in and out length to achieve the path effect.

[0063] It should be noted that the embodiments of the present invention have better practicability and do not impose any form of limitation on the present invention. Any technician familiar with the field may use the technical content disclosed above to change or modify it into an equivalent effective embodiment. However, any modification or equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A navigation method for spinal canal puncture based on mixed reality technology, characterized in that, it includes: Step S1: Receive a click operation on the skin surface of the three-dimensional image to obtain a collision point, and the three-dimensional image is reconstructed in three-dimensional stereo based on mixed reality technology; Step S2: Make a ray from the collision point to the target point in the spinal canal inside the three-dimensional image, and detect whether the ray collides with bones or blood vessels before reaching the target point; when it is detected that the ray does not collide with bones or blood vessels before reaching the target point, then determine that this collision point is the optimal needle insertion point; Step S3: When there is an optimal needle insertion point, generate a visual navigation path between the optimal needle insertion point and the target point.

2. The navigation method for spinal canal puncture based on mixed reality technology according to claim 1, characterized in that, in Step S2, when it is detected that the ray collides with bones or blood vessels before reaching the target point, it further includes: Step S4: Determine that this collision point is invalid; Step S5: Find the direction vector from this collision point to the target point as a reference vector; Step S6: With the target point as the center, find 100 - 500 vectors with an angle of 0 - 1 degree with the reference vector near the reference vector as candidate vectors; Step S7: Calculate the position points corresponding to the candidate vectors on the skin surface of the three-dimensional image as candidate needle insertion points; Step S8: Make a ray from the candidate needle insertion point to the target point in the spinal canal inside the three-dimensional image, and detect whether the ray collides with bones or blood vessels before reaching the target point; Step S9: When no optimal needle insertion point is found after 100 - 500 ray detections, further expand the angle between the candidate vector and the reference vector by 1 degree, and then repeat Steps S5 - S8, and so on, and the maximum angle between the candidate vector and the reference vector does not exceed 5 degrees; Step S10: When it is detected that the ray does not collide with bones or blood vessels before reaching the target point, then determine that the candidate needle insertion point corresponding to this ray is the optimal needle insertion point.

3. The navigation method for spinal canal puncture based on mixed reality technology according to claim 2, characterized in that, in Step S8, when it is detected that the ray does not collide with bones or blood vessels before reaching the target point, then determine that the collision point is the optimal needle insertion point.

4. The navigation method for spinal canal puncture based on mixed reality technology according to claim 2, characterized in that, in Step S9, when the angle between the candidate vector and the reference vector is 5 degrees and still no optimal needle insertion point is found, then determine that there is no optimal needle insertion point.

5. A navigation system for spinal canal puncture based on mixed reality technology, characterized in that, it includes: A collision point acquisition module, which is used to receive a click operation on the skin surface of the three-dimensional image to obtain a collision point, and the three-dimensional image is reconstructed in three-dimensional stereo based on mixed reality technology; A ray detection module, which is used to draw a ray from the collision point to the target point in the spinal canal inside the three-dimensional image, and detect whether the ray collides with bones or blood vessels before reaching the target point; when it is detected that the ray does not collide with bones or blood vessels before reaching the target point, then it is determined that this collision point is the optimal needle insertion point; A navigation path generation module, which is used to generate a visual navigation path from the optimal needle insertion point to the target point when there is an optimal needle insertion point.

6. The navigation system for spinal canal puncture based on mixed reality technology according to claim 5, characterized in that, when it is detected that the ray collides with bones or blood vessels before reaching the target point, the ray detection module is further used for: (1) Determine that this collision point is invalid; (2) Find the direction vector from this collision point to the target point as a reference vector; (3) With the target point as the center, find 100 to 500 vectors with an angle of 0 to 1 degree with the reference vector near the reference vector as candidate vectors; (4) Calculate the position points corresponding to the candidate vectors on the skin surface of the three-dimensional image as candidate needle insertion points; (5) Draw a ray from the candidate needle insertion point to the target point in the spinal canal inside the three-dimensional image, and detect whether the ray collides with bones or blood vessels before reaching the target point; (6) When no optimal needle insertion point is found after 100 to 500 ray detections, further expand the angle between the candidate vector and the reference vector by 1 degree, and then repeat steps (2) to (5), and so on, and the maximum angle between the candidate vector and the reference vector does not exceed 5 degrees; (7) When it is detected that the ray does not collide with bones or blood vessels before reaching the target point, then it is determined that the candidate needle insertion point corresponding to this ray is the optimal needle insertion point.

7. The navigation system for spinal canal puncture based on mixed reality technology according to claim 6, characterized in that, the ray detection module is further used for: when the angle between the candidate vector and the reference vector is 5 degrees and still no optimal needle insertion point is found, then it is determined that there is no optimal needle insertion point.

8. An electronic device, characterized in that, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, it implements the navigation method for spinal canal puncture based on mixed reality technology according to any one of claims 1 to 4.

Images