Puncture path planning method and planning system
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- SHANGHAI SIMPLETOUCH ROBOT CO LTD
- Filing Date
- 2025-11-25
- Publication Date
- 2026-06-04
Smart Images
Figure CN2025137526_04062026_PF_FP_ABST
Abstract
Description
A puncture path planning method and planning system
[0001] Cross-reference to related applications
[0002] This application claims priority to Chinese Patent Application No. 2024117020260, filed on November 26, 2024, entitled "A Puncture Path Planning Method and Planning System", the entire contents of which are incorporated herein by reference. Technical Field
[0003] This application relates to the fields of medical technology and computer technology, specifically a puncture path planning method and planning system. Background Technology
[0004] In clinical practice, biopsy needles are often used to perform biopsies on internal organs or tissues. However, biopsy needles inevitably puncture the skin, muscles, and other internal organs or tissues, causing some degree of damage. Importantly, some organs or tissues cannot be penetrated by biopsy needles, such as hard bones like the ribs, sternum, and spine (unless the target is a bone marrow aspiration), dangerous sites like the heart, major blood vessels, and trachea, and inappropriate puncture sites (for example, when the target is the lungs, the needle should not pass through the liver).
[0005] Traditional puncture procedures rely heavily on the surgeon's experience for selecting the puncture path, leaving considerable room for improvement in reliability. To enhance this reliability, improved methods have been developed, such as constructing a 3D model of the body based on the patient's medical images, defining restricted and safe zones within the model, and automatically selecting the optimal puncture path using computer data processing capabilities and pre-defined filtering criteria. While this improved method can accurately pinpoint the location of organs and tissues within the body, it still has some shortcomings in path planning, such as:
[0006] 1) The risk level varies depending on the organ and tissue through which the puncture needle passes. Simply dividing organs and tissues into safe and prohibited puncture sites may not provide objective and accurate route options.
[0007] 2) The computer directly outputs the optimal filtering results based on preset conditions, lacking the presentation of comparisons between various paths, and cannot provide practitioners with more comprehensive reference information;
[0008] 3) Path planning for all locations on the skin using a brute-force approach results in high computational cost and low efficiency.
[0009] Application content
[0010] The technical objective of this application is to design a new puncture path planning method and system to improve the shortcomings of existing technologies and provide more comprehensive and reliable reference information for clinical puncture operations.
[0011] To achieve the above-mentioned technical objectives, the technical solution provided in this application is as follows:
[0012] A puncture path planning method, characterized in that it includes:
[0013] Acquire medical images of the puncture subject, and construct a three-dimensional model of the puncture subject's body based on the medical images;
[0014] The location of the puncture target area in the three-dimensional model is determined, and a virtual light source is set in the target area, which outputs the same light intensity in all directions;
[0015] Candidate regions with puncture initiation points on the skin surface of the 3D model are selected, and the light propagation path from the virtual light source to each voxel point in the candidate region is determined.
[0016] The organs and tissues contained in the three-dimensional model are defined as units with different light absorption values. The light absorption coefficient of the corresponding units is assigned according to whether the biopsy needle is allowed to pass through and the degree of risk when the biopsy needle passes through.
[0017] By combining the light absorption coefficient of each unit along the propagation path, the light intensity of each voxel point reaching the candidate region is calculated, and a grayscale image of the candidate region is generated accordingly. The grayscale value is used to describe the recommendation of each voxel point in the candidate region as the puncture starting point.
[0018] Based on the above solutions, further improvements include:
[0019] As an optional implementation, the method for screening candidate regions on the skin surface of the 3D model for puncture initiation points is as follows:
[0020] When the target area for puncture is the target point, a sphere is determined with the target point as the center and the maximum puncture depth of the biopsy needle as the radius. The sphere intersects with the three-dimensional model, and the skin surface of the three-dimensional model located inside the sphere is the candidate area.
[0021] When the target area for puncture is a target polyhedron, the outer surface of the target polyhedron is expanded at equal distances. The expansion distance is the maximum puncture depth of the biopsy needle, resulting in an expanded polyhedron. The expanded polyhedron intersects with the three-dimensional model, and the skin surface of the three-dimensional model located within the expanded polyhedron is the candidate region.
[0022] As an optional implementation, when the boundary of the sphere or enlarged polyhedron exceeds the scanning range of medical images such as CT or MR, image expansion is performed by zero-padding.
[0023] As an optional implementation, the transmitted light intensity per unit along each light propagation path is calculated using the following formula:
[0024]
[0025] In the above formula, i is the unit number along the path of light propagation. The incident light intensity is expressed as unit i. The transmitted light intensity is expressed in units of i. The light absorption coefficient is expressed as a unit of i. This represents the length of the current light propagation path, which is traversed by a unit i.
[0026] As an optional implementation, when assigning values to the light absorption coefficient, the method of this application assigns a value to each voxel contained in each unit separately;
[0027] Preset high-risk units and set the light absorption coefficient of each voxel in all high-risk units;
[0028] For units other than high-risk units that can be biopsied, voxels belonging to the same unit but located in different positions are assigned different light absorption coefficients based on their distance from the high-risk unit. The principle here is that the farther the distance, the smaller the voxel's light absorption coefficient.
[0029] As an optional implementation, the method for calculating the light intensity of each voxel reaching the candidate region is as follows:
[0030] The Marching Cubes algorithm is used to convert the voxel data of the skin surface of the 3D model into a geometric mesh composed of triangular facets. The surface normal vector N of each triangular facet is calculated using the coordinates of the three vertices of each triangular facet, and the orientation of each triangular facet is determined.
[0031] For any triangular facet in the candidate region, its illumination intensity is expressed as:
[0032] or Where L represents the direction vector from the current triangular patch to the virtual light source. Indicates the weighting coefficient;
[0033] Where i is the unit number along the path of light propagation. The light intensity output by the virtual light source, The light absorption coefficient is expressed as a unit of i. This represents the length of the current light propagation path, which is traversed by a unit i.
[0034] As an optional implementation, the method of this application accelerates the volume rendering process of a 3D model by empty volume culling, wherein the empty volume includes completely transparent areas and completely opaque areas;
[0035] The completely transparent area corresponds to the background area, invalid area, or low-risk unit when the biopsy needle passes through the 3D model. When the light propagation path reaches the completely transparent area, the calculation of that area is skipped.
[0036] The completely opaque area corresponds to the area that the biopsy needle cannot pass through. When the light propagation path encounters the completely opaque area, the subsequent calculations are skipped, and the light intensity of the corresponding voxel point on the skin surface of the three-dimensional model that the current propagation path reaches is 0.
[0037] As an optional implementation, the method of this application also includes the step of mapping the skin surface of the three-dimensional model onto a planar view using an equidistant cylindrical projection.
[0038] As an optional implementation, the method of this application also includes the steps of identifying and classifying organs and tissues and / or detecting lesions using a trained neural network model.
[0039] A puncture path planning system, characterized by comprising the following components:
[0040] The model building module is configured to build a 3D model of the puncture object's body based on the medical images of the puncture object.
[0041] The coefficient assignment module is configured to define the organs and tissues contained in the three-dimensional model as units with different light absorption values, and assign light absorption coefficient values to each unit according to whether the biopsy needle is allowed to pass through and the degree of risk when the biopsy needle passes through.
[0042] The virtual light source configuration module generates a virtual light source in the target area based on the position of the puncture target area in the three-dimensional model.
[0043] The candidate region screening module, based on the maximum puncture depth of the biopsy needle, filters out candidate regions on the skin surface of the 3D model where the puncture starting point is located, in order to determine the light propagation path from the virtual light source to each position in the candidate region.
[0044] The grayscale value calculation module calculates the light intensity of each voxel point reaching the candidate region by combining the light absorption coefficient of each unit along the propagation path of each light ray, and generates a grayscale image of the candidate region accordingly. The grayscale value describes the recommendation degree of each voxel point in the reasonable candidate region as the puncture starting point.
[0045] Beneficial effects:
[0046] This application utilizes medical imaging and computer technology to create a 3D model rendered in a specific manner, configured to provide medical personnel with preferred puncture path information for reference, serving as an auxiliary means for obtaining information in clinical practice. This application transforms the selection of puncture path into the problem of finding the optimal puncture starting point. Combining the principles of optical propagation, it proposes a novel concept for planning and recommending puncture paths. The design is scientifically sound and can not only visually and intuitively display the location of the preferred puncture starting point but also present the comparison results of all candidate puncture starting points, providing clinical puncture operators with more comprehensive and reliable reference information, aiming to further improve the accuracy of puncture results. Simultaneously, the calculation of candidate paths in this application's method and system is improved, reducing the computational difficulty for organs or tissues such as blood vessels, and can be accelerated using existing ray tracing libraries or hardware, reducing computational load while improving computer output efficiency. Attached Figure Description
[0047] Figure 1 is a schematic diagram comparing different puncture paths;
[0048] Figure 2 is a schematic diagram of the process of filtering candidate regions;
[0049] Figure 3 is a schematic diagram of the expansion operation;
[0050] Figure 4 shows the rendering effect of the 3D model;
[0051] Figure 5 shows the second rendering of the 3D model;
[0052] Figure 6 shows the rendering effect of the 3D model. Embodiments of the present invention
[0053] To clarify the design scheme and technical principles of this application, the following detailed explanation and description of this application are provided in conjunction with the accompanying drawings and specific embodiments.
[0054] Regarding the selection of puncture routes, the current common approach is to divide it into two stages: the first stage is to screen out feasible routes, and the second stage is to select the best route.
[0055] In the first stage, the constraints used in the screening path are generally divided into hard constraints and soft constraints. Hard constraints include, for example, defining impassable areas and setting a threshold for the angle between the puncture path and the skin. Soft constraints include, for example, that the planned path should be as far away as possible from important organs and tissues that should not be touched, and that the percutaneous puncture length of the planned path should be as short as possible. Soft constraints are priority conditions that need to be considered when planning the puncture path; they help optimize the puncture path to improve treatment effectiveness. Different procedures can be designed according to specific circumstances.
[0056] In the first stage of screening, existing technologies typically employ brute-force enumeration to calculate whether each candidate path violates hard constraints. The main drawbacks are the high computational cost and slow output. Reducing the computational cost by downsampling or decreasing the number of candidate points also reduces resolution and accuracy. Calculating the distance between the puncture path and the risky organ or tissue is cumbersome. For example, the further the biopsy needle is from risky areas such as the ribs, sternum, spine, heart, and major blood vessels, the better, requiring calculation of the distance between all paths and all risky areas. Furthermore, it is difficult to handle areas like small blood vessels that are permissible to pass through but pose a high risk.
[0057] Existing technologies typically employ two approaches in the second phase:
[0058] 1) Assign different weights to soft constraints, obtain scores, and select the best score;
[0059] 2) Use multi-objective optimization algorithms for screening, such as Pareto optimization;
[0060] Since it is generally difficult to determine whether the weights assigned to soft constraints are appropriate, multi-objective optimization methods are more commonly used.
[0061] Unlike existing technologies, this application proposes a novel puncture path planning method that incorporates optical concepts into the planning of puncture paths.
[0062] The main implementation process of the method in this application includes the following steps:
[0063] S1: Acquire medical images of the puncture object and construct a three-dimensional model of the puncture object's body based on the medical images;
[0064] S2: Determine the location of the puncture target area in the three-dimensional model, and set a virtual light source in the target area, wherein the virtual light source outputs the same light intensity in all directions;
[0065] S3: Filter candidate areas on the skin surface of the 3D model where the puncture starting point is located, and determine the light propagation path from the virtual light source to each voxel point in the candidate area;
[0066] S4: Define the organs and tissues contained in the three-dimensional model as units with different light absorption values, and assign light absorption coefficient values to each unit according to whether the biopsy needle is allowed to pass through and the degree of risk when the biopsy needle passes through.
[0067] S5: Calculate the light intensity of each voxel point in the candidate region by combining the light absorption coefficient of each unit along the propagation path of each light ray, and generate a grayscale image of the candidate region accordingly. Use the grayscale to describe the recommendation degree of each voxel point in the candidate region as the puncture starting point.
[0068] In the above process:
[0069] The medical images used in step S1 can be images obtained from CT scans. Building a 3D model based on CT images requires the detection and identification of organs and tissues, a step typically achieved through a trained neural network for segmentation. Theoretically, machine vision (CV) methods can also be used, but they are significantly more difficult. If CV methods are employed, it is necessary to set the Hounsfield Unit (HU) value range for different organs and tissues, for example, selecting HU ∈ [x1, x2] for bone, HU ∈ [x3, x4] for lungs, etc., and then using morphological operations for refinement. If implemented through training a neural network, the general steps are as follows: 1) Manual annotation: Using annotation software such as itk-snap or 3D Slicer, based on medical knowledge, different organs and tissues in the CT image are marked with masks. Different organs or tissues are covered with different masks according to predetermined encoding rules, and this is saved as annotation data; 2) Model building: Using 2D or 3D segmentation networks, such as U-net or V-net; 3) Model training: Training the model together with the annotation data and CT images; 4) Model usage: The input is the CT image to be analyzed, the trained model is used for segmentation, and the output is the segmentation mask of the organ or tissue, which is saved as a file for later use. Step S1, the construction of the 3D model, can be achieved using existing technology, and will not be elaborated here.
[0070] The design concept for steps S2 to S4 is as follows:
[0071] We can imagine the human body as a three-dimensional object. Different tissues or organs are assigned different light transmittance or absorption coefficients according to whether they allow the puncture needle to pass through, forming a "lightbox." For tissues or organs that do not allow the puncture needle to pass through, such as ribs, sternum, and non-target organs, a very high absorption coefficient is applied, making them opaque (or with very little light transmittance). For tissues or organs that allow the puncture needle to pass through, such as muscles, connective tissue, and target lesions, a very low absorption coefficient is applied, making them highly transparent. For tissues or organs that allow the puncture needle to pass through but pose a greater risk, such as blood vessels, an appropriate absorption coefficient is applied, allowing them to be partially transparent. Then, a light source is placed in the target area for puncture, or the puncture target is transformed into a light source. The light emitted by the light source is absorbed by layers of surrounding tissues, and finally, varying degrees of illumination can be observed externally. Areas with high brightness indicate a good puncture path, while areas with low brightness indicate many obstacles blocking the light.
[0072] To provide an easy-to-understand demonstration, this paper uses two-dimensional graphics, but all calculations in the method of this application are actually performed in three-dimensional space.
[0073] Figure 1 shows a two-dimensional cross-section of a selectable puncture path region. Point O (the center of the circle with six sector regions in Figure 1) represents the puncture target point. The circle centered at point O (the circle with six sector regions in Figure 1) represents a sphere drawn with the maximum puncture depth R (needle length) of the biopsy needle as its radius. When the biopsy needle cannot be placed outside the human body (insufficient length), puncture cannot be performed. Therefore, the puncture path must exist in the difference set between the spherical space and the human body space (the region from point A to point B in the figure). Sector region 1 in the figure represents the selectable region, sector region 2 is an example of a region where needle insertion is impossible due to rib obstruction, sector region 3 is an example of a region where needle insertion is impossible because the path crosses multiple lung lobes, sector region 4 is an example of a region where the biopsy needle length is insufficient, and sector region 5 is an example of a region where needle insertion is impossible due to obstruction by the heart or a major artery.
[0074] Regarding step S5, the transmitted light intensity per unit along each light propagation path can be calculated using the following formula:
[0075]
[0076] In the above formula, i is the unit number along the path of light propagation. The incident light intensity is expressed as unit i. The transmitted light intensity is expressed in units of i. The light absorption coefficient is expressed as a unit of i. This represents the length of the current light propagation path, which is traversed by a unit i.
[0077] When a light beam passes through multiple light-absorbing units in an organ or tissue, their light absorption can be superimposed.
[0078] The formula above is a simplified formula derived from the Lambert-Beer law. According to the Lambert-Beer law, if the emitted light intensity is denoted as... The transmitted light intensity is denoted as Absorbance is recorded as ,but ,in, It is the absorption coefficient. The length of the target tissue or organ that light travels along its path. It is molar concentration. In our model, because... Since it is unnecessary, the formula can be simplified, and the absorption coefficient of the target tissue or organ to light can be denoted as... The above formula was then obtained.
[0079] Introducing this formula simplifies our calculation process. Because this formula can be infinitely subdivided for the same value *i* without changing the result, it is effective even for a single voxel. This allows us to directly calculate voxel-level 3D arrays, eliminating the complex problem of determining the diameter of blood vessels along the path. The problem of calculating blood vessel diameter is transformed into the superposition and absorption effect of translucent objects along the light path.
[0080] In step S4, when assigning the light absorption coefficient, if the distance between the biopsy needle and the at-risk organ or tissue is also added to the constraints, a value can be assigned to each voxel within each unit separately. This means that the absorption coefficient of the same unit may differ at different locations. Specific operations include:
[0081] 1) Preset high-risk units and set the light absorption coefficient of each voxel in all high-risk units;
[0082] 2) For other units that can be biopsied by a needle, besides high-risk units, voxels belonging to the same unit but located in different positions are assigned different light absorption coefficients based on their distance from the high-risk unit. The principle here is that the farther away from the high-risk unit, the smaller the voxel's light absorption coefficient.
[0083] For the problem of the distance between the biopsy needle and the organ or tissue at risk, we can use a gradual dilation operation. Specifically, we can use a method based on distance transformation and a mapping function to gradually dilate the absorption coefficient array of the organ or tissue, as shown in Figure 3. The greater the distance, the smaller the absorbance, so the closer the light propagation path is to the organ or tissue at risk, the greater the attenuation. Distance transformation is an operation used in image processing (including 2D and 3D voxel images) to calculate the shortest distance from each voxel (or voxel) in an image to a specific set (such as a target object, shape boundary, etc.). In a voxel model, for a given shape (e.g., a set of voxels in a polyhedral shape), the distance transformation calculates the shortest distance for each voxel to the boundary of that shape. This distance can be the Euclidean distance (in three-dimensional space, for voxels...). To the boundary voxel The Euclidean distance is It can also be other distance metrics, such as Manhattan distance (which is 10 ... Once the distance-transformed image (or voxel model) is obtained, a mapping function can be used to assign attributes for gradual dilation. The distance-transformed voxel model can be stored in an array, and then a linearly decreasing attribute assignment can be implemented (e.g., voxels with a distance of 0 are assigned the maximum value). With a fixed step size as the distance increases (Decrease). Assuming the attribute value after gradient expansion is... The maximum attribute value is The distance is Decrease step size is Then the formula can be expressed as Different mapping functions (such as exponential functions, logarithmic functions, etc.) can be designed and implemented according to specific application requirements.
[0084] In step S5, when calculating and constraining the angle between the skin and the puncture path, a reflection model can be introduced. Since the skin has a large area, calculating the angle between the line connecting all points on the skin surface and the puncture target and the skin itself is computationally intensive. Furthermore, variables such as angle, length, passability, and light intensity have different dimensions, making it difficult to handle when considering multiple factors simultaneously. Therefore, we introduce a simplified reflection lighting model to address this problem.
[0085] In the model of this application, we do not consider diffraction, polarization, or scattering of light; reflection and refraction are calculated at most once. We borrow the principle of the Lambert lighting model to simplify the calculation. The Lambert lighting model is a common diffuse reflection lighting model that describes how an object's surface interacts with a light source. Assuming the surface is perfectly diffuse, the light intensity is proportional to the cosine of the angle between the light direction and the surface normal vector, hence the formula... ,in, Indicates the final light intensity. Indicates the intensity of the light source. Represents the surface normal vector. This represents the direction vector from a surface point to the light source. This represents the dot product of the normal vector and the direction of the light source (cosine of the angle). The calculation of the normal vector is central to the Lambert lighting model, and the Marching Cubes technique provides a normal vector for each surface point, thus supporting Lambert lighting calculations. Marching Cubes is an algorithm for extracting isosurfaces from 3D volume data. The result is a set of surface meshes composed of triangles, which define the object's surface in the volume data. Crucially, the Marching Cubes algorithm not only generates surfaces but also provides vertex information for each triangle facet, allowing for further calculation of the vertex normal vectors. Surface Meshes: The output of Marching Cubes is a set of triangular faces that form the isosurfaces in the 3D volume data. Normal vectors are crucial in lighting calculations because they determine the interaction between light rays and the surface; the normal vector for each triangle can be calculated from its vertex positions.
[0086] When using Marching Cubes to calculate the light intensity on the skin surface, the first step is to extract the skin's surface mesh from the 3D volumetric data. The Marching Cubes algorithm transforms the voxel data into a geometric mesh composed of triangular facets, each triangle consisting of three vertices. , , Composition. Using these vertices, the surface normal vectors... The expression is:
[0087]
[0088] The normal vector N represents the orientation of the skin surface at that triangle. Therefore, by generating the skin surface mesh and normal vector using Marching Cubes technology and combining it with the Lambert lighting model, the light intensity at each point on the skin surface can be accurately calculated.
[0089] To accelerate illumination calculations, a normal vector caching optimization strategy can be adopted. That is, during the calculation process, the normal vector can be calculated and cached in advance to avoid repeated calculations.
[0090] In summary, the method for calculating the light intensity of each voxel point reaching the candidate region in step S5 of this application is as follows:
[0091] The Marching Cubes algorithm is used to convert the voxel data of the skin surface of the 3D model into a geometric mesh composed of triangular facets. The surface normal vector N of each triangular facet is calculated using the coordinates of the three vertices of each triangular facet, and the orientation of each triangular facet is determined.
[0092] For any triangular facet in the candidate region, its illumination intensity is expressed as:
[0093] or Where L represents the direction vector from the current triangular patch to the virtual light source. Indicates the weighting coefficient;
[0094] Where i is the unit number along the path of light propagation. The light intensity output by the virtual light source, The light absorption coefficient is expressed as a unit of i. This represents the length of the current light propagation path, which is traversed by a unit i.
[0095] In the above formula, for the weighting coefficient The introduction of this factor is primarily based on considerations of parameter adjustment and optimization. Regarding the influence of the angle between the skin and the biopsy needle, if we consider that sometimes a larger angle between the puncture needle and the skin normal is acceptable and not necessarily worse than a smaller angle, then we can add a weighting coefficient to the formula. Then we can adjust The value, balance and The size relationship.
[0096] The settings for parameters such as the light absorption coefficient, decreasing step size, and weighting coefficient can be manually adjusted through clinical data testing combined with clinical experience. More preferably, to address the subjectivity of manual parameter tuning and achieve fully automated optimal path recommendation, the optimal path can be manually selected and labeled, and then an automated path recommendation model based on machine learning or deep learning can be constructed for parameter optimization and path selection. The specific implementation is as follows: First, the three-dimensional spatial light intensity distribution data calculated based on optical principles is mapped into a two-dimensional feature image (e.g., using sinusoidal projection to unfold a spherical light curtain into a two-dimensional planar image). When constructing this two-dimensional image, different physical constraint information is encoded using the RGB channels of the image: for example, the R channel (red component) is used to map the normalized tissue puncture depth (corresponding to the light absorption term); the G channel (green component) is used to map the normalized product of the skin incident angle cosine and the obstacle indicator; and the B channel (blue component) is used to map the normalized vascular risk absorption term. Thus, an RGB feature image containing all puncture constraint information is generated. Secondly, based on the aforementioned feature images, low-risk candidate puncture regions (i.e., "safe islands") are segmented using statistical thresholding, and multidimensional features of each connected region are extracted. These features include geometric features (such as region area, major axis length, eccentricity, etc.), intensity statistical features (such as the mean and standard deviation of each R, G, and B channel), and location features. Finally, a classification model (such as Random Forest, XGBoost, or Multilayer Perceptron MLP) is trained using "optimal puncture points" labeled by experienced physicians as ground truth. During model training, Synthetic Minority Oversampling Technique (SMOTE) can be used to address the imbalance between positive and negative samples. In practical applications, the feature data of the patient to be punctured is input into the trained model, the model outputs scores for each candidate region, the system automatically selects the region with the highest predicted probability, and inversely maps its geometric center back to three-dimensional space to obtain the optimal puncture path.
[0097] Specific examples are as follows:
[0098] 1. Organ and tissue segmentation and modeling
[0099] First, a segmentation mask for lung organs or tissues is obtained using neural networks, machine vision, or image morphology methods. For example, a trained neural network can be used to segment organs or tissues such as lung lobes, heart, mediastinal blood vessels, ribs, sternum, liver, spleen, kidneys, pancreas, trachea, and skin.
[0100] 2. Lesion segmentation and modeling
[0101] The lesion coordinates can be segmented or detected using a trained neural network, or the lesion coordinates can be manually specified.
[0102] 3. Data Preparation
[0103] Based on the modeling requirements, the masks for the aforementioned organs and tissues are assigned different values, and the following arrays are prepared:
[0104]
[0105] In achieving the final lighting effect, we can also use the following optimization methods to improve computational efficiency.
[0106] 1) Empty Space Skipping
[0107] Empty volume culling is an optimization technique configured to accelerate the volumetric rendering process, especially in ray casting and volume ray casting. When this technique is applied to the method of this application, the empty volume includes completely transparent regions and completely opaque regions.
[0108] The completely transparent area corresponds to the background area of the 3D model, the invalid area, or the low-risk unit (such as connective tissue, with opacity set to 0) when the biopsy needle passes through. When the light propagation path reaches the completely transparent area, the calculation of that area is skipped.
[0109] The completely opaque area corresponds to the area that the biopsy needle cannot pass through (such as ribs, sternum, heart, etc., with opacity set to 1). When the light propagation path encounters a completely opaque area, subsequent calculations are skipped, and the light intensity of the corresponding voxel point on the skin surface of the current light propagation path reaching the three-dimensional model is directly output as 0.
[0110] 2) GPU acceleration (GPU Acceleration)
[0111] GPU acceleration is a technique that utilizes graphics processing units (GPUs) to improve computational speed and efficiency. Designed for parallel processing, GPUs have a large number of computing cores, making them suitable for handling large-scale data-parallel tasks. GPU acceleration plays a crucial role in fields such as computer graphics, deep learning, and scientific computing. Unlike traditional CPUs, GPUs have a large number of stream processors, enabling them to handle multiple threads simultaneously. For applications such as volumetric rendering that require processing massive amounts of voxel data, the parallel architecture of GPUs allows them to perform millions of computational operations in a short time. The basic idea behind GPU acceleration is to break down large-scale computational tasks into smaller chunks and execute them in parallel. The architecture of GPUs is well-suited for handling matrix operations and geometric transformations, performing exceptionally well in graphics rendering and deep learning. In ray casting and volumetric rendering, GPUs can compute the interaction between each ray and voxel data in parallel, rapidly generating images.
[0112] 3) Map the skin surface of the 3D model to a 2D image.
[0113] To more easily observe the 3D results and provide standardized input data for subsequent machine learning algorithms, we can project them onto a 2D plane. The grayscale value (or RGB value) of each point on the plane represents the intensity of the light or other physical properties, and the coordinates of each point can be mapped back to 3D space, thus achieving a reversible transformation from a 2D plane to 3D space.
[0114] In different embodiments of this application, different projection methods can be selected according to specific needs:
[0115] Method 1 (Suitable for visual observation): Equidistant Cylindrical Projection. This projection method unfolds the sphere into a rectangular plane, suitable for medical staff to view intuitively. Its calculation formula and principle are: assuming the target is a sphere with a radius of... For a point on the surface of a sphere... (in For the sake of measurement, (latitude, all in radians), corresponding points on the projection plane It can be done through formula , Calculation. Among them... The longitude of the central meridian. Based on the latitude, These are standard parallels of latitude. In the most commonly used grid projection mode, let... =0, =0, =1, then the formula simplifies to: , In principle, equidistant cylindrical projection maintains the perpendicular relationship between meridians and parallels, but there is area stretching deformation in the polar regions.
[0116] Method 2 (Applicable to Machine Learning Recommendation): Sinusoidal Projection. As described in the aforementioned "Automated Path Recommendation" example, to ensure the accuracy of features (such as region area) extracted by the machine learning model, sinusoidal projection is preferred. This is an equal-area projection method that maintains the relative area size, avoiding feature extraction errors caused by projection distortion. Its transformation formula is typically... , Similarly, assuming =0, When =0, it simplifies to , .
[0117] In summary, this application allows for the flexible selection of any of the above projection methods or combinations thereof, depending on the actual processing requirements (whether the focus is on human visual observation or algorithm accuracy).
[0118] 4) ROI Extraction
[0119] ROI is an abbreviation for Region of Interest, which refers to a specific area in an image or data that people are particularly interested in and that needs to be analyzed in detail. Introducing ROI is to focus on the research object, reduce interference from irrelevant information, and improve the efficiency and accuracy of analysis.
[0120] Based on the above ideas, the following method is used to filter candidate regions in step S3 of this method:
[0121] When the target area for puncture is the target point, a sphere is determined with the target point as the center and the maximum puncture depth of the biopsy needle as the radius. The sphere intersects with the three-dimensional model, and the skin surface of the three-dimensional model located inside the sphere is the candidate area, as shown in Figure 2. In the figure, A and B represent the intersection points of the sphere and the skin, C represents the skin, and O represents the target point.
[0122] When the target area for puncture is a target polyhedron, the outer surface of the target polyhedron is expanded outward at equal distances. The outward expansion distance is the maximum puncture depth of the biopsy needle, resulting in an expanded polyhedron. The expanded polyhedron intersects with the three-dimensional model, and the skin surface of the three-dimensional model located within the expanded polyhedron is the candidate area.
[0123] The mask and light absorption coefficient (transparency) arrays are clipped along the bounding box of the candidate region;
[0124] When the sphere, the enlarged polyhedron, or the circumscribed box exceeds the scanning range of the corresponding medical image such as a CT scan, a zero-padding operation is performed to expand the image.
[0125] This method has the following advantages:
[0126] ① By pruning, the size of the array is reduced, thus reducing the amount of computation;
[0127] ② By symmetric processing of ROI, the asymmetric calculation problem caused by the lesion not being at the center of the array is avoided (if it is asymmetric, the problem of different distances and different ranges of angle values in different directions must be considered, which increases the computational complexity).
[0128] It should also be noted that for the same puncture target area, each voxel point on the candidate starting point area corresponds to only one light propagation path. When the puncture target area is a polyhedron, this method only considers the shortest route from the candidate voxel point to the puncture target area as the light propagation path.
[0129] The rendering model examples shown in Figures 4-6:
[0130] Area 6 represents skin; puncture can only be performed where there is skin. Area 7 represents obstacles. Area 8 represents pulmonary veins and pulmonary arteries; the darker the area in area 8, the higher the puncture risk. Area 9 represents pulmonary nodules, which are the puncture targets. White represents safe areas.
[0131] Based on the aforementioned methods and principles, after calculating the light intensity distribution of the candidate regions, the light intensity is observed. Areas with stronger light intensity (higher grayscale values) are more recommended as starting points for puncture paths, while darker areas are not recommended as puncture starting points. Non-candidate regions on the model surface can be directly represented as black areas. If perspective projection is performed, they can be observed on a two-dimensional plane.
[0132] This method is visually intuitive and easy to understand, providing more comprehensive reference information for clinical puncture procedures. Compared with existing technologies, it can reduce the difficulty of calculating blood vessels and other organs or tissues, and can be accelerated using existing ray tracing libraries or hardware, which can significantly reduce the amount of computation and improve computational efficiency.
[0133] Furthermore, this method is more suitable for planning the puncture path of rigid biopsy needles.
[0134] A puncture path planning system includes the following components:
[0135] The model building module is configured to build a 3D model of the puncture object's body based on the medical images of the puncture object.
[0136] The virtual light source configuration module generates a virtual light source in the target area based on the position of the puncture target area in the three-dimensional model.
[0137] The candidate region screening module, based on the maximum puncture depth of the biopsy needle, filters out candidate regions on the surface of the 3D model skin where the puncture starting point is located, and determines the light propagation path from the virtual light source to each voxel point in the candidate region.
[0138] The coefficient assignment module is configured to define the organs and tissues contained in the three-dimensional model as units with different light absorption values, and assign light absorption coefficient values to each unit according to whether the biopsy needle is allowed to pass through and the degree of risk when the biopsy needle passes through.
[0139] The grayscale value calculation module calculates the light intensity of each voxel point in the candidate region by combining the light absorption coefficient of each unit along the propagation path of each light ray, and generates a grayscale image of the candidate region accordingly. The grayscale value describes the recommendation degree of each voxel point in the candidate region as the puncture starting point.
[0140] This embodiment adopts the same design concept as the previous embodiment, and the method and principle can be described in the previous embodiment.
[0141] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many variations, modifications, substitutions, and alterations to these embodiments without departing from the spirit and scope of the claims. All such variations, modifications, substitutions, and alterations should be protected within the scope of this application. Industrial applicability
[0142] This application presents a puncture path planning method and system. Based on the concepts of optics and computer rendering, it rapidly calculates optimal puncture path information to provide reference for medical personnel and serve as an auxiliary means for obtaining information in clinical practice. It transforms the selection of the puncture path into the problem of finding the optimal puncture starting point. Combining the principles of optical propagation, it proposes a novel concept for planning and recommending puncture paths, which can intuitively display the location information of the optimal puncture starting point and is more comprehensive and reliable.
Claims
1. A puncture path planning method, characterized in that, include: Acquire medical images of the puncture subject, and construct a three-dimensional model of the puncture subject's body based on the medical images; The location of the puncture target area in the three-dimensional model is determined, and a virtual light source is set in the target area, which outputs the same light intensity in all directions; Candidate regions with puncture initiation points on the surface of the 3D model's skin are selected, and the light propagation path from the virtual light source to each voxel point in the candidate region is determined. The organs and tissues contained in the three-dimensional model are defined as units with different light absorption values. The light absorption coefficient of the corresponding units is assigned according to whether the biopsy needle is allowed to pass through and the degree of risk when the biopsy needle passes through. By combining the light absorption coefficients of each unit along each light propagation path, the light intensity of each voxel point reaching the candidate region is calculated, and a grayscale image of the candidate region is generated accordingly. The grayscale value is used to describe the recommendation of each voxel point in the candidate region as the puncture starting point.
2. The puncture path planning method as described in claim 1, characterized in that, The method for selecting candidate regions for puncture initiation points on the skin surface of a 3D model is as follows: When the target area for puncture is the target point, a sphere is determined with the target point as the center and the maximum puncture depth of the biopsy needle as the radius. The sphere intersects with the three-dimensional model, and the skin surface of the three-dimensional model located inside the sphere is the candidate area. When the target area for puncture is a target polyhedron, the outer surface of the target polyhedron is expanded at equal distances. The expansion distance is the maximum puncture depth of the biopsy needle, resulting in an expanded polyhedron. The expanded polyhedron intersects with the three-dimensional model, and the skin surface of the three-dimensional model located within the expanded polyhedron is the candidate region.
3. The puncture path planning method as described in claim 1, characterized in that, When the boundary of the sphere or enlarged polyhedron exceeds the scanning range of the medical image, the image is expanded by padding with zeros.
4. The puncture path planning method as described in claim 1, characterized in that, The transmitted light intensity per unit along each ray propagation path is calculated using the following formula: In the above formula, i is the unit number along the path of light propagation. The incident light intensity is expressed as unit i. The transmitted light intensity is expressed in units of i. The light absorption coefficient is expressed as a unit of i. This represents the length of the current light propagation path, which is traversed by a unit i.
5. The puncture path planning method as described in claim 1, characterized in that, When assigning values to the light absorption coefficient, each voxel contained in each unit is assigned a value separately; Preset high-risk units and set the light absorption coefficient of each voxel in all high-risk units; For units other than high-risk units that can be biopsied, voxels belonging to the same unit but located in different positions are assigned different light absorption coefficients based on their distance from the high-risk unit. The principle here is that the farther the distance, the smaller the light absorption coefficient of the voxel.
6. The puncture path planning method as described in claim 1, characterized in that, The method for calculating the light intensity of each voxel point reaching the candidate region is as follows: The Marching Cubes algorithm is used to convert the voxel data of the skin surface of the 3D model into a geometric mesh composed of triangular facets. The surface normal vector N of each triangular facet is calculated using the coordinates of the three vertices of each triangular facet, and the orientation of each triangular facet is determined. For any triangular facet in the candidate region, its illumination intensity I is expressed as: or Where L represents the direction vector from the current triangular patch to the virtual light source. Indicates the weighting coefficient; Where i is the unit number along the path of light propagation. The light intensity output by the virtual light source, The light absorption coefficient is expressed as a unit of i. This represents the length of the current light propagation path, which is traversed by a unit i.
7. The puncture path planning method as described in claim 1, characterized in that, The volume rendering process of 3D models is accelerated by empty volume culling, where the empty volume includes completely transparent areas and completely opaque areas; The completely transparent area corresponds to the background area, invalid area, or low-risk unit when the biopsy needle passes through the 3D model. When the light propagation path reaches the completely transparent area, the calculation of that area is skipped. The completely opaque area corresponds to the area that the biopsy needle cannot pass through. When the light propagation path encounters the completely opaque area, the subsequent calculations are skipped, and the light intensity of the corresponding voxel point on the skin surface of the three-dimensional model that the current light propagation path reaches is directly output as 0.
8. The puncture path planning method as described in claim 1, characterized in that, This includes the step of mapping the skin surface of a 3D model onto a planar view using equidistant cylindrical projection.
9. The puncture path planning method as described in claim 1, characterized in that, This includes steps such as identifying and classifying organs and tissues, and / or detecting lesions, using a trained neural network model.
10. A puncture path planning system, characterized in that, include: The model building module is configured to build a 3D model of the puncture object's body based on the medical images of the puncture object. The virtual light source configuration module generates a virtual light source in the target area based on the position of the puncture target area in the three-dimensional model. The candidate region screening module, based on the maximum puncture depth of the biopsy needle, filters out candidate regions on the surface of the 3D model skin where the puncture starting point is located, and determines the light propagation path from the virtual light source to each voxel point in the candidate region. The coefficient assignment module is configured to define the organs and tissues contained in the three-dimensional model as units with different light absorption values, and assign light absorption coefficient values to each unit according to whether the biopsy needle is allowed to pass through and the degree of risk when the biopsy needle passes through. The grayscale value calculation module calculates the light intensity of each voxel point in the candidate region by combining the light absorption coefficient of each unit along the propagation path of each light ray, and generates a grayscale image of the candidate region accordingly. The grayscale value describes the recommendation degree of each voxel point in the candidate region as the puncture starting point.