Puncture path planning method and system

Abstract

The invention discloses a puncture path planning method and system, and aims to quickly calculate optimal puncture path information based on optical and computer rendering thoughts so as to provide reference for medical staff and provide an auxiliary means for clinically acquiring information. The method comprises the following steps: constructing a three-dimensional model through a medical image; the method comprises the following steps: defining organs and tissues as units with incompletely same light absorbance, and assigning light absorption coefficients to each unit according to whether a biopsy needle is allowed to pass or not and the risk degree of puncture; a light source is arranged in a puncture target area, gray information of the skin surface is obtained by analyzing the light absorbance on different paths, and the gray information is used for feeding back the recommendation degree of each puncture starting point. According to the method, selection of the puncture path is converted into searching of the optimal puncture starting point, a brand-new concept of planning and recommending the puncture path is provided by combining the optical propagation principle, and the position information of the optimal puncture starting point can be visually displayed and is more comprehensive and reliable.

Description

Technical Field

[0001] The present invention relates to the fields of medical technology and computer technology, and in particular to a puncture path planning method and a planning system. Background Art

[0002] In clinical practice, in order to perform biopsy operations on organs or tissues inside the body, a biopsy needle is often used for sampling. However, the biopsy needle will puncture the skin, muscles and other organs or tissues in the body during use, which will inevitably cause a certain degree of damage to the human body. It is particularly important that some organs or tissues cannot be penetrated by the biopsy needle, such as hard bone tissues such as the ribs, sternum, and spine (unless the goal is to perform a bone marrow puncture), as well as dangerous areas such as the heart, large blood vessels, and trachea, as well as unreasonable puncture sites (for example, when the puncture target is in the lungs, it should not pass through the liver and other parts).

[0003] The selection of puncture paths in traditional puncture surgery mostly depends on the doctor's experience, and the reliability of puncture path selection still has a lot of room for improvement. In order to improve the reliability of puncture path selection, an improved method has been proposed to build a three-dimensional model of the human body based on the patient's medical images, set restricted areas and safe areas in the three-dimensional model, and use the computer's data processing capabilities and pre-set screening conditions to automatically select the optimal puncture path. This improved method can certainly accurately confirm the location of organs and tissues in the body of the puncture subject, but there are still some deficiencies in path planning, such as:

[0004] 1) The puncture needle passes through different organs and tissues, which brings different risks. Simply dividing organs and tissues into safe and prohibited puncture sites may not provide objective and accurate path options;

[0005] 2) The computer directly outputs the optimal screening results based on the preset conditions, lacking the presentation of comparisons between various paths and failing to provide more comprehensive reference information for the operator;

[0006] 3) The path is planned for all positions on the skin in a brute force exhaustive manner, which results in large amount of calculation and low efficiency. Summary of the invention

[0007] The technical purpose of the present invention is to design a new puncture path planning method and planning system to improve the deficiencies of the existing technology and provide more comprehensive and reliable reference information for clinical puncture operations.

[0008] In order to achieve the above technical objectives, the technical solution provided by the present invention is:

[0009] A puncture path planning method, characterized by comprising:

[0010] Acquiring a medical image of the puncture subject, and constructing a three-dimensional model of the puncture subject's body based on the medical image;

[0011] Determine the position of the target area for puncture in the three-dimensional model, and set a virtual light source in the target area, wherein the virtual light source outputs the same light intensity in all directions;

[0012] Screening candidate areas on the skin surface of the three-dimensional model where the puncture starting point is located, and determining the light propagation path from the virtual light source to each voxel point in the candidate area;

[0013] The organs and tissues included in the three-dimensional model are defined as units with different light absorption degrees, and light absorption coefficients are assigned to the corresponding units according to whether the biopsy needle is allowed to pass through and the risk level when the biopsy needle passes through;

[0014] Combined with the light absorption coefficient of the unit passed by each propagation path, the light intensity reaching each voxel point in the candidate area is calculated, and a grayscale image of the candidate area is generated based on this, and the recommendation degree of each voxel point in the candidate area as the puncture starting point is described by grayscale.

[0015] Based on the above scheme, further improved schemes include:

[0016] As a preferred embodiment, the method for screening the candidate area of ​​the puncture starting point on the surface of the three-dimensional model skin is:

[0017] When the target area for puncture is a target point, a sphere is determined with the target point as the center and the maximum puncture depth of the biopsy needle as the radius, the sphere intersects with the three-dimensional model, and the skin surface of the three-dimensional model located in the sphere is the candidate area;

[0018] When the target area for puncture is a target polyhedron, expansion is performed at equal distances along the outer surface of the target polyhedron, and the expansion distance is the maximum puncture depth of the biopsy needle to obtain an enlarged polyhedron. The enlarged polyhedron intersects with the three-dimensional model, and the skin surface of the three-dimensional model located in the enlarged polyhedron is the candidate area.

[0019] As a preferred implementation, when the boundary of the sphere or the expanded polyhedron exceeds the scanning range of medical images such as CT or MR, the image is expanded by a zero-padding operation.

[0020] As a preferred implementation, the following formula is used to calculate the transmitted light intensity of each unit on each light propagation path:

[0021]

[0022] In the above formula, i is the unit number on the light propagation path, I 0i is the incident light intensity per unit i, Iti is the transmitted light intensity per unit i, k i is the light absorption coefficient of unit i, l i The length of the current light propagation path through unit i.

[0023] As a preferred embodiment, when the method of the present invention performs the assignment of the light absorption coefficient, each voxel contained in each unit is assigned a value respectively;

[0024] Preset high-risk units and set the light absorption coefficient of each voxel in all high-risk units;

[0025] For other units other than high-risk units that can pass through the biopsy needle, voxels belonging to the same unit but located at different positions are assigned different light absorption coefficients according to their distance from the high-risk unit. The principle of assignment here is that the farther the distance, the smaller the voxel light absorption coefficient.

[0026] As a preferred implementation, the method for calculating the light intensity reaching each voxel point in the candidate area is:

[0027] The voxel data of the skin surface of the 3D model is converted into a geometric mesh composed of triangular patches through the Marching Cubes algorithm. The surface normal vector N of each triangular patch is calculated using the coordinates of the three vertices of each triangular patch to determine the orientation of each triangular patch.

[0028] For a triangular patch at any position in the candidate area, its illumination intensity is expressed as:

[0029] I=I t *max(N·L,0) or I=w*I t *max(N·L, 0), where L represents the direction vector from the current triangle patch to the virtual light source, and w represents the weight coefficient;

[0030] I t =I0·10-∑ i k i ·l i , where i is the unit number on the light propagation path, I0 is the light intensity output by the virtual light source, and k i is the light absorption coefficient of unit i, l i The length of the current light propagation path through unit i.

[0031] As a preferred embodiment, the method of the present invention accelerates the volume rendering process of the three-dimensional model by eliminating empty volumes, wherein the empty volumes include completely transparent areas and completely opaque areas;

[0032] The completely transparent area corresponds to the background area of ​​the three-dimensional model, the invalid area or the preset low-risk unit when the biopsy needle passes through. When the light propagation path reaches the completely transparent area, the calculation of the area is skipped;

[0033] The completely opaque area corresponds to the area that the biopsy needle cannot pass through. When the light propagation path encounters the completely opaque area, the subsequent calculation is skipped and the light intensity of the corresponding voxel point on the skin surface of the three-dimensional model reached by the current propagation path is directly output as 0.

[0034] As a preferred embodiment, the method of the present invention further comprises the step of mapping the skin surface of the three-dimensional model onto a plane map for viewing by means of equidistant cylindrical projection.

[0035] As a preferred embodiment, the method of the present invention also includes the steps of identifying and classifying organs and tissues, and / or detecting lesions through a trained neural network model.

[0036] A puncture path planning system, characterized by comprising the following components:

[0037] A model building module, used for building a three-dimensional model of the body of the puncture object based on the medical image of the puncture object;

[0038] A coefficient assignment module, used to define the organs and tissues included in the three-dimensional model as units with different light absorption degrees, and assign light absorption coefficients to each unit according to whether the biopsy needle is allowed to pass through and the risk level when the biopsy needle passes through;

[0039] A virtual light source configuration module generates a virtual light source in the target area according to the position of the puncture target area in the three-dimensional model;

[0040] A candidate area screening module, based on the maximum puncture depth of the biopsy needle, screens out candidate areas where the puncture starting point is on the surface of the three-dimensional model skin, so as to determine the light propagation path from the virtual light source to each position of the candidate area;

[0041] The gray value calculation module calculates the light intensity reaching each voxel point in the candidate area in combination with the light absorption coefficient of the unit through which each light propagation path passes, and generates a gray image of the candidate area based on this, and describes the recommendation degree of each voxel point in the reasonable candidate area as the puncture starting point through grayscale description.

[0042] Beneficial effects:

[0043] The present invention is based on medical imaging and computer technology, and the three-dimensional model rendered in a specific way is used to provide medical staff with reference information of the preferred puncture path, which is an auxiliary means of obtaining information clinically. After transforming the selection of the puncture path into the problem of finding the best puncture starting point, the present invention combines the principle of optical propagation to propose a new concept of planning the puncture path and recommending the path. The design is scientific and reasonable, which can not only visually display the location of the preferred puncture starting point, but also present the comparison results of all candidate puncture starting points, providing more comprehensive and reliable reference information for clinical puncture operators, in order to further improve the accuracy of the puncture results. At the same time, the method and system of the present invention also improve the calculation of candidate paths, reduce the difficulty of calculating organs or tissues such as blood vessels, and can use the existing ray tracing calculation library or hardware for acceleration, reducing the amount of calculation while improving the computer output efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A schematic diagram for comparing different puncture paths;

[0045] Figure 2 Schematic diagram of the operation of screening candidate areas;

[0046] Figure 3 A schematic diagram of the expansion operation;

[0047] Figure 4 Rendering effect for 3D models Figure 1 ;

[0048] Figure 5 Rendering effect for 3D models Figure 2 ;

[0049] Figure 6 Rendering effect for 3D models Figure 3 . DETAILED DESCRIPTION

[0050] In order to clarify the design scheme and technical principle of the present invention, the present invention is explained and illustrated in more detail below in conjunction with the accompanying drawings and specific embodiments.

[0051] Embodiment 1:

[0052] Regarding the selection of puncture path, the current common scheme is mainly divided into two stages. The first stage is to screen out feasible paths, and the second stage is to select the best path.

[0053] In the first stage, the constraints of the screening path used are generally divided into hard constraints and soft constraints. The hard constraints, for example, define the impassable area and set the angle between the puncture path and the skin to not be less than a threshold. Soft constraints, for example, the planned path should be as far away as possible from important organs and tissues that should not be touched, and the percutaneous puncture length of the planned path should be as short as possible. Soft constraints are priority conditions that need to be considered when planning the puncture path. They help optimize the puncture path to improve the safety and effectiveness of treatment. Different operations can be designed according to specific circumstances.

[0054] In the first stage of screening, the existing technology generally uses brute force to calculate whether each candidate path solution violates the hard constraints. The main drawbacks are that the amount of calculation is large and the output result is slow. If the amount of calculation is reduced by downsampling or reducing the number of candidate points, the resolution and accuracy of the screening results will also be reduced. Calculating the distance between the puncture path and the risk organ or tissue is relatively cumbersome. For example, the farther the biopsy needle is from the risk areas such as the ribs, sternum, spine, heart, and large blood vessels, the better. Therefore, it is necessary to calculate the distance between all paths and all risk areas. It is difficult to handle areas such as small blood vessels that are allowed to pass through but have a higher risk.

[0055] The existing technology generally has the following two ideas in the second stage:

[0056] 1) Assign different weights to soft constraints, obtain scores, and select the best scores;

[0057] 2) Use multi-objective optimization algorithms for screening, such as Pareto optimization;

[0058] Since it is generally difficult to determine whether the weights assigned to soft constraints are appropriate, multi-objective optimization methods are often used.

[0059] Different from the prior art, the present invention proposes a new puncture path planning method, which combines optical ideas into the planning of the puncture path.

[0060] The main implementation process of the method of the present invention comprises the following steps:

[0061] S1: Acquire a medical image of the puncture object, and construct a three-dimensional model of the puncture object's body based on the medical image;

[0062] S2: determining the position of the target area for puncture in the three-dimensional model, and setting a virtual light source in the target area, wherein the virtual light source outputs the same light intensity in all directions;

[0063] S3: screening the candidate area of ​​the puncture starting point on the skin surface of the three-dimensional model, and determining the light propagation path from the virtual light source to each voxel point in the candidate area;

[0064] S4: defining the organs and tissues included in the three-dimensional model as units with different light absorption degrees, and assigning light absorption coefficients to the units according to whether the biopsy needle is allowed to pass through and the risk level when the biopsy needle passes through;

[0065] S5: Calculate the light intensity that finally reaches each voxel point in the candidate area by combining the light absorption coefficient of each unit through which the light propagation path passes, and generate a grayscale image of the candidate area based on this, and describe the recommendation of each voxel point in the candidate area as the puncture starting point through grayscale.

[0066] In the above process:

[0067] The medical image in step S1 can be a picture obtained by a CT scan. In order to construct a three-dimensional model based on the CT image, it is necessary to detect and identify organs and tissues. This step is generally achieved by segmenting through a trained neural network. In theory, the machine vision (CV) method can also work, but it is quite difficult. If the CV method is used, it is necessary to set the HU value range of different organs and tissues, such as selecting [x, y] as bones and [z, w] as lungs, and then use some morphological operations to trim. If it is achieved by training a neural network, it generally has the following steps: 1) Manual labeling, using itk-snap or 3D Slicer and other labeling software, based on medical knowledge, to paint and mark different organs and tissues in the CT image, and cover different organs or tissues with different masks according to established coding rules, and save them as labeling data; 2) Model building, using 2D or 3D segmentation networks, such as U-net or V-net; 3) Model training, training the labeling data together with the CT image; 4) Use of the model: input is the CT image to be analyzed, segmentation is performed using the trained model, and the output is the segmentation mask of the organ or tissue, which is saved as a file for future use. Step S1 can be achieved by existing technology to build a three-dimensional model, which will not be repeated here.

[0068] Regarding steps S2 to S4, the design ideas are as follows:

[0069] We can imagine the human body as a three-dimensional object. Different tissues or organs are given different transmittance or absorption coefficients according to whether they allow the puncture needle to pass through, forming a "light box". For tissues or organs that do not allow the puncture needle to pass through, such as ribs, sternum, and non-target organs, a very large absorption coefficient is given to make it opaque (or very little translucent); for tissues or organs that allow the puncture needle to pass through, such as muscles, connective tissues, and target lesions, a very small absorption coefficient is given to make it highly translucent; for tissues or organs that allow the puncture needle to pass through but have a greater risk, such as blood vessels, an appropriate absorption coefficient is given to make it partially translucent. Then place a light source in the target area of ​​puncture, or turn the puncture target into a light source. The light emitted by the light source, after being absorbed by layers of the periphery, can eventually be observed on the outside with varying degrees of illumination. The area with high brightness is a good puncture path, and the area with very low brightness indicates that there are many obstacles blocking it.

[0070] In order to provide an easy-to-understand demonstration, the demonstration in this article uses two-dimensional graphics, but all calculations in the method of the present invention are actually in three-dimensional space.

[0071] like Figure 1 As shown in the figure, the figure shows a two-dimensional section of an optional area of ​​the puncture path, in which the red point represents the puncture target point. The red circle represents a sphere drawn with the maximum puncture depth R (needle length) of the biopsy needle as the radius. When the biopsy needle cannot be outside the human body (not long enough), puncture cannot be performed, so the puncture path must exist in the difference between the sphere space and the human body space (the area from point A to point B in the figure). In the figure, yellow represents an optional area, blue is an example of an area where the needle cannot be inserted because of rib obstruction, orange is an example of an area where the needle cannot be inserted because the path crosses multiple lobes, green is an example of an area where the biopsy needle is not long enough, and pink is an example of an area where the needle cannot be inserted because of obstruction by the heart or aorta.

[0072] Regarding step S5, the following formula can be used to calculate the transmitted light intensity of each unit on each light propagation path:

[0073]

[0074] In the above formula, i is the unit number on the light propagation path, I 0i is the incident light intensity per unit i, l ti is the transmitted light intensity per unit i, k i is the light absorption coefficient of unit i, l i The length of the current light propagation path through unit i.

[0075] When a light propagation path passes through the light absorption units of multiple organs or tissues, their light absorption can be superimposed.

[0076] The above formula is a simplified formula derived from the Lambert-Beer law. According to the Lambert-Beer law, if the incident light intensity is recorded as I0 and the transmitted light intensity is recorded as I t , the absorbance is recorded as A i , then A i =-log 10 (I t / I0)=K i ·l i ·c i , where K i is the absorption coefficient, l i is the length of the light path through the target tissue or organ, c i is the molar concentration. In our model, since c i There is no need for this to exist, so the formula can be simplified and the absorption coefficient of the target tissue or organ to light is recorded as k i Then we get the above formula.

[0077] Introducing this formula can simplify our calculation process. Because for the same i, this formula can be infinitely subdivided without changing the calculation result, and it is valid even for a single voxel. In this way, we can directly calculate the three-dimensional array at the voxel level, eliminating the complicated solution of "what is the diameter of the blood vessel on the path". The problem of solving the blood vessel diameter is converted into the superposition absorption effect of translucent objects on the light path.

[0078] In step S4, when assigning the light absorption coefficient, if the distance between the biopsy needle and the risk organ or tissue is also added to the constraint condition, each voxel contained in each unit can be assigned a value, that is, the absorption coefficient of different parts of the same unit may be different. The specific operations include:

[0079] 1) Preset high-risk units and set the light absorption coefficient of each voxel in all high-risk units;

[0080] 2) For other units other than high-risk units that can pass through the biopsy needle, voxels belonging to the same unit but located at different positions are assigned different light absorption coefficients according to their distance from the high-risk unit. The principle of assignment here is that the farther away from the high-risk unit, the smaller the voxel light absorption coefficient.

[0081] We can use a gradient dilation operation to solve the problem of the distance between the biopsy needle and the risk organ or tissue. In specific implementation, we can use a method based on distance transform (DistanceTransform) and mapping function to perform a gradient dilation operation on the absorption coefficient array of the organ or tissue, such as Figure 3As shown, the farther the distance is, the smaller the absorbance is. Thus, the closer the light propagation path is to the risk organ or tissue, the greater the attenuation is. Distance transform is an operation used in image (including 2D images and 3D voxel images) processing to calculate the shortest distance from each voxel (or voxels) in an image to a specific set (such as a target object, shape boundary, etc.). In the voxel model, for a given shape (such as a set of voxels in a polyhedral shape), the distance transform calculates the shortest distance from each voxel to the boundary of the shape. This distance can be the Euclidean distance (in three-dimensional space, the Euclidean distance from a voxel (x, y, z) to a boundary voxel (x0, y0, z0) is It can also be other distance metrics, such as Manhattan distance (|x-x0|+|y-y0|+|z-z0| in 3D space). Once the distance-transformed image (or voxel model) is obtained, a mapping function can be used to implement the attribute assignment of gradient expansion. The distance-transformed voxel model can be stored in an array, and then a linearly decreasing attribute assignment can be implemented (such as assigning the voxel with a distance of 0 to the maximum value V max , decreasing with a fixed step size s as the distance increases). Assume that the attribute value after gradual expansion is E, and the maximum attribute value is V max , the distance is d, the decreasing step is s, then the formula can be expressed as E = max{V max -sd, 0}. Different mapping functions (such as exponential function, logarithmic function, etc.) can be designed and implemented according to specific application requirements.

[0082] In step S5, when considering calculating the angle between the skin and the puncture path and implementing the constraint operation, the reflection model can be introduced. Since the skin has a large area, the calculation of the angle formed by the line between all points on the skin surface and the puncture target and the skin is quite large. On the other hand, the angle and length, whether it can pass, light intensity and other variables have different dimensions, which is not easy to handle when multiple factors are considered at the same time. Therefore, we introduced a simplified reflection illumination model to solve this problem.

[0083] In the model of the present invention, we do not consider the diffraction and polarization of light, nor involve the scattering phenomenon. The reflection and refraction of light are calculated only once at most. We borrowed the principle of Lambert illumination model to simplify the calculation. Lambert illumination model is a common diffuse reflection illumination model, which describes how the surface of an object interacts with the light source. Assuming that the surface is completely diffuse, the illumination intensity is proportional to the cosine value of the angle between the light direction and the surface normal vector, then there is the formula I = I light max(0, N·L), where I represents the final light intensity, I lightRepresents the intensity of the light source, N represents the surface normal vector, L represents the direction vector from the surface point to the light source, and N·L represents the dot product (cosineoftheangle) of the normal vector and the direction of the light source. The calculation of the normal vector is the core of the Lambert lighting model. The MarchingCubes technology can provide a normal vector for each surface point, thereby supporting Lambert lighting calculations. MarchingCubes is an algorithm for extracting isosurfaces from three-dimensional volume data. The result is a set of surface meshes composed of triangles, which define the surface of the object in the volume data. The key is that the MarchingCubes algorithm not only generates surfaces, but also provides vertex information for each triangular face, so that the normal vector of the vertex can be further calculated. Surface Meshes: The output of MarchingCubes is a set of triangular patches that form isosurfaces in 3D volume data. The normal vector is very important in lighting calculations because it determines the interaction between light and the surface. The normal vector of each triangle can be calculated by its vertex position.

[0084] When using MarchingCubes technology to calculate the light intensity on the skin surface, the skin surface mesh must first be extracted from the 3D volume data. The MarchingCubes algorithm converts voxel data into a geometric mesh composed of triangular patches, each of which consists of three vertices v0, v1, and v2. Using these vertices, the expression of the surface normal vector N is:

[0085]

[0086] The normal vector N represents the orientation of the skin surface at the triangle. Therefore, by generating the skin surface mesh and normal vector through the Marching Cubes technology and combining it with the Lambert lighting model, the light intensity at each point on the skin surface can be accurately calculated.

[0087] In order to speed up the lighting calculation, an optimization strategy of normal vector caching can be adopted. That is, during the calculation process, the normal vector can be calculated and cached in advance to avoid repeated calculation.

[0088] In summary, in step S5 of the present invention, the method for calculating the light intensity reaching each voxel point in the candidate area is:

[0089] The voxel data of the skin surface of the three-dimensional model is converted into a geometric mesh composed of triangular facets by using the MarchingCubes algorithm, and the surface normal vector N of each triangular facet is calculated using the coordinates of the three vertices of each triangular facet to determine the orientation of each triangular facet;

[0090] For a triangular patch at any position in the candidate area, its illumination intensity is expressed as:

[0091] I=I t *max(N·L,0) or I=w*I t *max(N·L, 0), where L represents the direction vector from the current triangle patch to the virtual light source, and w represents the weight coefficient;

[0092] Among them, i is the unit number on the light propagation path, I0 is the light intensity output by the virtual light source, k i is the light absorption coefficient of unit i, l i The length of the current light propagation path through unit i.

[0093] In the above formula, the introduction of the weight coefficient w is mainly based on parameter adjustment and optimization considerations. Regarding the influence of the angle between the skin and the biopsy needle, if we want to consider that sometimes a large angle between the puncture needle and the skin normal is acceptable and not necessarily worse than a small angle, then we can add a weight coefficient w to the formula, and then we can balance I by adjusting the value of w. t Relationship with I.

[0094] The above-mentioned parameters such as light absorption coefficient, decrement step length, weight coefficient, etc. can be manually adjusted through clinical data testing combined with clinical experience. It is also possible to manually select the best path for annotation and then use machine learning or deep learning methods to adjust the parameters.

[0095] Specific examples are as follows:

[0096] 1. Organ and tissue segmentation and modeling

[0097] First, the segmentation mask of the lung organs or tissues is obtained by using a neural network or machine vision or image morphology method. For example, the trained neural network is used to segment the lung lobes, heart, mediastinal blood vessels, ribs, sternum, liver, spleen, kidney, pancreas, trachea, skin and other organs or tissues.

[0098] 2. Lesion segmentation and modeling

[0099] The lesion coordinates are segmented or detected using a trained neural network, or the lesion coordinates are manually specified.

[0100] 3. Data preparation

[0101] According to modeling needs, different values ​​are assigned to the masks of the above organs and tissues, and the following arrays are prepared:

[0102]

[0103]

[0104] In the process of achieving the final lighting effect, we can also use the following optimization methods to improve calculation efficiency.

[0105] 1) Empty Space Skipping

[0106] Empty volume culling is an optimization technique for accelerating the volume rendering process, especially in raycasting and volume ray casting. The technique is applied in the method of the present invention, and the empty volume includes a completely transparent area and a completely opaque area.

[0107] The completely transparent area corresponds to the background area of ​​the three-dimensional model, the invalid area, or the preset low-risk unit (such as connective tissue, etc., with the opacity set to 0) when the biopsy needle passes through, etc. When the light propagation path reaches the completely transparent area, the calculation of the area is skipped;

[0108] The completely opaque area corresponds to the area that the biopsy needle cannot pass through (such as ribs, sternum, heart, etc., with the opacity set to 1). When the light propagation path encounters a completely opaque area, subsequent calculations are skipped and the light intensity of the corresponding voxel point on the skin surface of the three-dimensional model reached by the current light propagation path is directly output as 0.

[0109] 2) GPU Acceleration

[0110] GPU acceleration is a technology that improves computing speed and efficiency by utilizing graphics processing units (GPUs). GPUs are designed for parallel processing and have a large number of computing cores, making them suitable for handling large-scale data parallel tasks. GPU acceleration plays a vital role in computer graphics, deep learning, scientific computing, and other fields. Unlike traditional CPUs, GPUs have a large number of stream processors that can process multiple threads simultaneously. For applications such as volume rendering that require processing of massive voxels and voxel data, the parallel architecture of GPUs enables them to perform millions of computing operations in a short period of time. The basic idea of ​​using GPUs for acceleration is to break down large-scale computing tasks into small pieces and execute them in parallel. The architecture of GPUs is very suitable for processing matrix operations and geometric transformations, and performs well in the fields of graphics rendering and deep learning. In the process of ray casting and volume rendering, the GPU can be used to parallelly calculate the interaction between each ray and voxel data to quickly generate images.

[0111] 3) Map the skin surface of the 3D model to a plane image

[0112] In order to observe the 3D results more conveniently, we can use projection to map it to a plane map for viewing. The grayscale value of each point on the plane map represents the intensity value of the light, and the coordinates of each point can be mapped back to the 3D space, thereby realizing the reversible conversion from the 2D plane map to the 3D space, which provides convenience for further analysis and processing. The equidistant cylindrical projection (but not limited to) can be used here. Its calculation formula and principle are: The equidistant cylindrical projection is a method of projecting the surface of the earth onto a cylindrical surface, and then unfolding the cylindrical surface into a plane. In this projection, the meridians are projected as equally spaced vertical lines, and the latitudes are projected as horizontal lines. In terms of the calculation formula, assuming that the target is a sphere with a radius of R, for a point on the sphere (where θ is the longitude, is latitude), the corresponding point (x, y) on the equidistant cylindrical projection plane can be calculated by the following formula: x = Rθ,

[0113] In principle, the equidistant cylindrical projection maintains the perpendicular relationship between the meridians and the parallels, and maintains the equidistant along the meridians. This projection method maintains the relative accuracy of shape and direction to a certain extent, so that the results of geographic information or other three-dimensional data mapping can be observed and analyzed more intuitively on the projected plane. At the same time, since the coordinates of each point can be mapped from the plane back to the 3D space through the inverse formula, it provides convenience for further analysis and processing.

[0114] 4) ROI extraction

[0115] ROI is the abbreviation of Region of Interest, which refers to a specific area in an image or data that people pay special attention to and need to be analyzed in depth. The purpose of introducing ROI is to focus on the research object, reduce interference from irrelevant information, and improve analysis efficiency and accuracy.

[0116] Based on the above ideas, the following method is used to select candidate areas in step S3 of this method:

[0117] When the target area for puncture is the target point, a sphere is determined with the target point as the center and the maximum puncture depth of the biopsy needle as the radius. The sphere intersects with the three-dimensional model, and the skin surface of the three-dimensional model located in the sphere is the candidate area, such as Figure 2 As shown, in the figure, A and B represent the intersection points of the sphere and the skin, C represents the skin, and O represents the target point;

[0118] When the target area for puncture is a target polyhedron, an equal distance of expansion is performed along the outer surface of the target polyhedron, and the expansion distance is the maximum puncture depth of the biopsy needle to obtain an enlarged polyhedron, and the enlarged polyhedron intersects with the three-dimensional model, and the skin surface of the three-dimensional model located in the enlarged polyhedron is the candidate area;

[0119] Crop the mask and light absorption coefficient (transparency) array along the external bounding box of the candidate area;

[0120] When the sphere, the enlarged polyhedron or the external bounding box exceeds the scanning range of the corresponding medical image such as CT, a zero-filling operation is performed to expand the image.

[0121] This method has the following advantages:

[0122] ① By cutting, the size of the array is reduced, which reduces the amount of calculation;

[0123] ② By symmetrizing the ROI, the asymmetric calculation problem caused by the lesion not being at the center of the array is avoided (if it is asymmetric, it is necessary to consider the different distances in different directions and the different angle value ranges, which increases the calculation complexity).

[0124] It should also be noted that for the same puncture target area, each voxel point on the starting point candidate area corresponds to only one light propagation path. When the puncture target area is a polyhedron, this method only considers the shortest route from the voxel point in the candidate area to the puncture target area as the light propagation path.

[0125] like Figure 4-Figure 6 In the example of the rendered model shown:

[0126] The purple area represents the skin, and puncture can only be performed where there is skin; green represents obstacles; red represents pulmonary veins and pulmonary arteries, and the darker the red, the higher the risk of puncture; yellow represents lung nodules, which are the puncture targets; white represents safe areas.

[0127] Based on the above methods and principles, after calculating the light intensity distribution of the candidate area, observe the light intensity. The stronger the light intensity (the higher the gray value), the more recommended it is as the starting point of the puncture path. The darker the area, the less recommended the puncture starting point area. For non-candidate areas on the model surface, they can be directly represented as black areas. If perspective projection is performed, it can be observed on a two-dimensional plane.

[0128] This method is intuitive and easy to understand, providing more comprehensive reference information for clinical puncture operations. Compared with existing technologies, it can reduce the difficulty of calculating organs or tissues such as blood vessels, and can use existing ray tracing calculation libraries or hardware for acceleration, which can significantly reduce the amount of calculation and improve calculation efficiency.

[0129] At the same time, this method is more suitable for planning the puncture path of a rigid biopsy needle.

[0130] Embodiment 2:

[0131] A puncture path planning system includes the following components:

[0132] A model building module, used for building a three-dimensional model of the body of the puncture object based on the medical image of the puncture object;

[0133] A virtual light source configuration module generates a virtual light source in the target area according to the position of the puncture target area in the three-dimensional model;

[0134] The candidate region screening module screens the candidate region where the puncture starting point is on the skin surface of the three-dimensional model based on the maximum puncture depth of the biopsy needle, and determines the light propagation path from the virtual light source to each voxel point in the candidate region;

[0135] A coefficient assignment module, used to define the organs and tissues included in the three-dimensional model as units with different light absorption degrees, and assign light absorption coefficients to each unit according to whether the biopsy needle is allowed to pass through and the risk level when the biopsy needle passes through;

[0136] The grayscale value calculation module calculates the light intensity reaching each voxel point in the candidate area in combination with the light absorption coefficient of the unit through which each light propagation path passes, and generates a grayscale image of the candidate area based on this, and describes the recommendation degree of each voxel point in the candidate area as the starting point of puncture through grayscale.

[0137] This embodiment and Embodiment 1 adopt the same design concept, and the introduction of its method and principle can refer to Embodiment 1.

[0138] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present application, ordinary technicians in this field can make many forms without departing from the purpose of the present application and the scope of protection of the claims. Various changes, modifications, substitutions and variations of these embodiments without departing from the purpose of the present application and the scope of protection of the claims should all be protected by the present application.

Claims

1. A puncture path planning method, characterized in that: include: Acquiring a medical image of the puncture subject, and constructing a three-dimensional model of the puncture subject's body based on the medical image; Determine the position of the target area for puncture in the three-dimensional model, and set a virtual light source in the target area, wherein the virtual light source outputs the same light intensity in all directions; Screening candidate areas on the skin surface of the three-dimensional model where the puncture starting point is located, and determining the light propagation path from the virtual light source to each voxel point in the candidate area; The organs and tissues included in the three-dimensional model are defined as units with different light absorption degrees, and light absorption coefficients are assigned to the corresponding units according to whether the biopsy needle is allowed to pass through and the risk level when the biopsy needle passes through; Combined with the light absorption coefficient of each unit passed by each light propagation path, the light intensity reaching each voxel point in the candidate area is calculated, and a grayscale image of the candidate area is generated based on this. The grayscale is used to describe the recommendation degree of each voxel point in the candidate area as the starting point of puncture.

2. A puncture path planning method according to claim 1, characterized in that: The method for selecting the candidate area of ​​the puncture starting point on the skin surface of the three-dimensional model is: When the target area for puncture is a target point, a sphere is determined with the target point as the center and the maximum puncture depth of the biopsy needle as the radius, the sphere intersects with the three-dimensional model, and the skin surface of the three-dimensional model located in the sphere is the candidate area; When the target area for puncture is a target polyhedron, expansion is performed at equal distances along the outer surface of the target polyhedron, and the expansion distance is the maximum puncture depth of the biopsy needle to obtain an enlarged polyhedron. The enlarged polyhedron intersects with the three-dimensional model, and the skin surface of the three-dimensional model located in the enlarged polyhedron is the candidate area.

3. A puncture path planning method according to claim 1, characterized in that: When the boundary of the sphere or the expanded polyhedron exceeds the scanning range of the medical image, the image is expanded by a zero-padding operation.

4. A puncture path planning method according to claim 1, characterized in that: The following formula is used to calculate the transmitted light intensity of each unit on each light propagation path: In the above formula, i is the unit number on the light propagation path, I 0i is the incident light intensity per unit i, I ti is the transmitted light intensity per unit i, k i is the light absorption coefficient of unit i, l i The length of the current light propagation path through unit i.

5. A puncture path planning method according to claim 1, characterized in that: When assigning the light absorption coefficient, each voxel contained in each unit is assigned a value; Preset high-risk units and set the light absorption coefficient of each voxel in all high-risk units; For other units other than high-risk units that can pass through the biopsy needle, voxels belonging to the same unit but located at different positions are assigned different light absorption coefficients according to their distance from the high-risk unit. The principle of assignment here is that the farther the distance, the smaller the light absorption coefficient of the voxel.

6. A puncture path planning method according to claim 1, characterized in that: The method for calculating the light intensity reaching each voxel point in the candidate area is: The voxel data of the skin surface of the 3D model is converted into a geometric mesh composed of triangular patches through the Marching Cubes algorithm. The surface normal vector N of each triangular patch is calculated using the coordinates of the three vertices of each triangular patch to determine the orientation of each triangular patch. For a triangular patch at any position in the candidate area, its illumination intensity I is expressed as: I=I t *max(N·L,0) or I=w*I t *max(N·L, 0), where L represents the direction vector from the current triangle patch to the virtual light source, and w represents the weight coefficient; Among them, i is the unit number on the light propagation path, I0 is the light intensity output by the virtual light source, k i is the light absorption coefficient of unit i, l i The length of the current light propagation path through unit i.

7. A puncture path planning method according to claim 1, characterized in that: Accelerate the volume rendering process of the 3D model by culling empty volumes, wherein the empty volumes include completely transparent areas and completely opaque areas; The completely transparent area corresponds to the background area of ​​the three-dimensional model, the invalid area or the preset low-risk unit when the biopsy needle passes through. When the light propagation path reaches the completely transparent area, the calculation of the area is skipped; The completely opaque area corresponds to the area that the biopsy needle cannot pass through. When the light propagation path encounters the completely opaque area, the subsequent calculation is skipped and the light intensity of the corresponding voxel point at the skin surface of the three-dimensional model reached by the current light propagation path is directly output as 0.

8. A puncture path planning method according to claim 1, characterized in that: The method comprises the steps of mapping the skin surface of the three-dimensional model onto a plane map by adopting an equidistant cylindrical projection method.

9. A puncture path planning method according to claim 1, characterized in that: It includes the steps of identifying and classifying organs and tissues, and / or detecting lesions through a trained neural network model.

10. A puncture path planning system, characterized in that: include: A model building module, used for building a three-dimensional model of the body of the puncture object based on the medical image of the puncture object; A virtual light source configuration module generates a virtual light source in the target area according to the position of the puncture target area in the three-dimensional model; The candidate region screening module screens the candidate region where the puncture starting point is on the skin surface of the three-dimensional model based on the maximum puncture depth of the biopsy needle, and determines the light propagation path from the virtual light source to each voxel point in the candidate region; A coefficient assignment module, used to define the organs and tissues included in the three-dimensional model as units with different light absorption degrees, and assign light absorption coefficients to each unit according to whether the biopsy needle is allowed to pass through and the risk level when the biopsy needle passes through; The grayscale value calculation module calculates the light intensity reaching each voxel point in the candidate area in combination with the light absorption coefficient of the unit through which each light propagation path passes, and generates a grayscale image of the candidate area based on this, and describes the recommendation degree of each voxel point in the candidate area as the starting point of puncture through grayscale.

Images