Intelligent puncture path planning method and system under ultrasonic guidance

Abstract

The present invention relates to the technical field of intelligent puncture path planning methods, and in particular to an intelligent puncture path planning method under ultrasound guidance and a system thereof. The method comprises the following steps: obtaining a real-time ultrasound image sequence; preprocessing and enhancing the real-time ultrasound image sequence; performing multi-scale deep learning image segmentation based on the enhanced ultrasound image to obtain a segmented anatomical structure label map; performing target positioning and feature extraction based on the segmented anatomical structure label map; performing path planning using deep reinforcement learning based on the segmented anatomical structure label map and target position coordinates to obtain an optimal puncture path; performing dynamic risk assessment based on the optimal puncture path and the enhanced ultrasound image to obtain a risk assessment result; and generating an augmented reality guidance interface based on the optimal puncture path, the risk assessment result, and the enhanced ultrasound image, which can find an optimal balance between multiple goals and improve the accuracy and safety of planning.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent puncture path planning methods, and in particular to an intelligent puncture path planning method and system under ultrasound guidance. Background Art

[0002] With the rapid development of minimally invasive surgical techniques in recent years, ultrasound-guided puncture procedures have played an increasingly important role in clinical practice. This technique offers patients less trauma, faster recovery time, and a lower risk of complications. However, despite significant advances in ultrasound-guided techniques, performing precise puncture procedures within complex anatomical structures still presents numerous challenges.

[0003] Traditional ultrasound-guided puncture methods rely heavily on the doctor's experience and skills. As Kaiyu Wu et al. pointed out in "Review of research on path planning and control methods of flexible steerable needle puncture robot" (Computer Assisted Surgery, 2022, 27(1):91-112), conventional rigid straight needles have difficulty navigating obstacles during puncture and are unable to adapt to complex anatomical structures. This method often struggles to achieve precise positioning and safe operation when dealing with deep organs or when avoiding important blood vessels and nerves.

[0004] To address this problem, researchers have proposed various intelligent path planning methods. Li et al. introduced an ultrasound-guided puncture robot based on adaptive control and reinforcement learning in “An Adaptive Control Method and Learning Strategy for Ultrasound-Guided Puncture Robot” (Electronics, 2024, 13:580). Although this method has improved the accuracy of puncture to a certain extent, it still has some limitations. First, the method focuses mainly on the robot's control strategy, but pays insufficient attention to complex anatomical structure recognition and risk assessment. Second, the method's real-time performance needs to be improved, making it difficult to meet the needs of dynamic surgical environments.

[0005] Furthermore, while existing deep learning-based methods have made progress in image recognition, they often overlook dynamic risk assessment during surgery. These methods typically utilize static pre-trained models, making them difficult to adapt to the challenges posed by intraoperative tissue deformation and individual patient differences.

[0006] In summary, the existing technology still has obvious deficiencies in the following aspects:

[0007] 1. Lack of accurate recognition and segmentation capabilities for complex anatomical structures;

[0008] 2. Path planning algorithms struggle to strike a good balance between safety, efficiency, and operability;

[0009] 3. Lack of real-time and dynamic risk assessment mechanisms;

[0010] 4. The real-time and robustness of the system cannot meet the requirements of complex surgical environments.

[0011] To address these problems, the present invention proposes an intelligent puncture path planning method and system under ultrasound guidance. Summary of the Invention

[0012] The purpose of the present invention is to provide an intelligent puncture path planning method and system under ultrasound guidance to solve the above problems.

[0013] The present invention provides an intelligent puncture path planning method under ultrasound guidance, comprising the following steps: obtaining a real-time ultrasound image sequence; preprocessing and enhancing the real-time ultrasound image sequence to obtain an enhanced ultrasound image; performing multi-scale deep learning image segmentation based on the enhanced ultrasound image to obtain a segmented anatomical structure label map; performing target positioning and feature extraction based on the segmented anatomical structure label map to obtain target position coordinates and feature descriptors; performing path planning using deep reinforcement learning based on the segmented anatomical structure label map and the target position coordinates to obtain an optimal puncture path; performing dynamic risk assessment based on the optimal puncture path and the enhanced ultrasound image to obtain a risk assessment result; and generating an augmented reality guidance interface based on the optimal puncture path, the risk assessment result, and the enhanced ultrasound image.

[0014] Specifically, the steps of preprocessing and enhancing the real-time ultrasound image sequence include: calculating the local standard deviation and the global standard deviation based on the local area of each pixel point in the real-time ultrasound image sequence; determining the adaptive filtering window size based on the ratio of the local standard deviation to the global standard deviation; using the adaptive filtering window size, performing median filtering on the real-time ultrasound image sequence to obtain a denoised image sequence; calculating a local histogram for each image in the denoised image sequence; determining an adaptive clipping threshold based on the maximum and minimum values of the local histogram; using the adaptive clipping threshold, performing contrast-limited adaptive histogram equalization on the denoised image sequence to obtain an equalized image sequence; calculating the local mean and the global mean of each image in the equalized image sequence; determining an adaptive gamma correction parameter based on the difference between the local mean and the global mean; and using the adaptive gamma correction parameter, performing gamma correction on the equalized image sequence to obtain the enhanced ultrasound image.

[0015] Specifically, the step of performing multi-scale deep learning image segmentation based on the enhanced ultrasound image includes: constructing a multi-scale U-Net++ network model, including an encoder and a decoder; in the encoder, extracting the multi-scale features of the enhanced ultrasound image through continuous convolution layers and pooling layers; in the decoder, gradually restoring the spatial resolution of the feature map through transposed convolution layers; adding dense jump connections between the encoder and the decoder to achieve fusion of features of different scales; calculating the spatial attention map of the fused feature map, which reflects the importance of different areas of the image; calculating the channel attention vector of the fused feature map, which reflects the importance of different feature channels; multiplying the spatial attention map and the channel attention vector with the fused feature map to obtain a weighted feature map; based on the weighted feature map, generating a multi-category segmentation probability map through a 1x1 convolution layer and a softmax activation function; calculating Dice loss, boundary loss and Focal loss, and weightedly combining these three losses to obtain a total loss function; optimizing the parameters of the network model through back propagation and gradient descent, and finally obtaining the segmented anatomical structure label map.

[0016] Specifically, the step of performing target positioning and feature extraction based on the segmented anatomical structure label map includes: obtaining the segmented anatomical structure label maps of the current frame and the previous k frames; performing temporal consistency analysis on the k+1 frame label map to screen out stable connected regions; calculating the area, perimeter and shape complexity of the stable connected regions, and screening out candidate target regions according to a preset threshold; calculating weights for pixels within the candidate target region according to their grayscale values in the enhanced ultrasound image; using the weights, calculating the weighted center of mass of the candidate target region to obtain the target position coordinates; extracting a local region of fixed size with the target position coordinates as the center; calculating shape features, including roundness, eccentricity and directionality, for the local region; calculating texture features, including grayscale co-occurrence matrix features and local binary pattern features, for the local region; using a pre-trained convolutional neural network, extracting depth features of the local region as context features; and splicing the shape features, the texture features and the context features to form the feature descriptor.

[0017] Specifically, the step of using deep reinforcement learning to perform path planning based on the segmented anatomical structure label map and the target position coordinates includes: calculating the distance from each position to the nearest obstacle according to the segmented anatomical structure label map to obtain an obstacle distance map; generating an uncertainty map based on the confidence of the segmented anatomical structure label map; estimating the tissue deformation field using the registration results between consecutive frames to obtain a deformation degree map; calculating a local vascular density map based on the vascular segmentation results; weightedly combining the obstacle distance map, the uncertainty map, the deformation degree map and the vascular density map to obtain a comprehensive risk map; constructing a reinforcement learning environment based on the comprehensive risk map and the target position coordinates; in the reinforcement learning environment, using an improved Deep Q-Network algorithm for global path planning; based on the global path, using a newly designed Advantage The Actor-Critic algorithm is used to optimize local paths; an adaptive reward function is designed based on the safety, efficiency and accuracy of the path; multiple agents are introduced, each responsible for different subtasks, such as safety, efficiency and accuracy; a knowledge distillation mechanism is used to enable knowledge sharing and collaborative learning among the agents; an integrated teacher strategy is constructed based on the strategies of all agents; the integrated teacher strategy is used to guide the learning process of each agent; combined with the learned Q value, an improved Rapidly-exploring Random Trees algorithm is used for path sampling and optimization; the optimization is iterated until convergence or a preset number of iterations is reached to obtain the optimal puncture path.

[0018] Specifically, the steps for constructing the dynamic risk map include: initializing weight coefficients based on the obstacle distance map, the uncertainty map, the deformation degree map, and the vascular density map; calculating the difference between the current risk map and the target risk distribution; updating the weight coefficients based on the difference using the gradient descent method; recalculating the comprehensive risk value using the updated weight coefficients; repeating the above steps until the risk map converges or reaches a preset number of iterations; and outputting the final dynamic risk map.

[0019] Specifically, the step of performing dynamic risk assessment based on the optimal puncture path and the enhanced ultrasound image includes: calculating the path curvature along the optimal puncture path; determining the sampling interval based on the path curvature and the dynamic risk map; extracting path key points according to the sampling interval; for each path key point, extracting the obstacle distance, uncertainty, tissue deformation and vascular density information of the local area; using fuzzy logic rules, converting the extracted information into a fuzzy set; based on evidence theory, calculating the basic probability distribution of each risk factor; using the Dempster combination rule, fusing the evidence of multiple risk factors to obtain a local risk assessment result; constructing a recursive neural network, taking the local risk assessment result sequence as input; extracting the global features of the optimal puncture path; splicing the global features with the hidden state of the recursive neural network; and outputting the overall path risk assessment result through a fully connected layer and a Sigmoid activation function.

[0020] Specifically, the step of generating an augmented reality guidance interface based on the optimal puncture path, risk assessment results and enhanced ultrasound image includes: calculating the saliency map and contrast map of the enhanced ultrasound image; generating an adaptive fusion weight map based on the saliency map and the contrast map; using the adaptive fusion weight map to superimpose the optimal puncture path on the enhanced ultrasound image; calculating the global mean and standard deviation of the dynamic risk map; determining the adaptive risk threshold based on the global mean and the standard deviation; marking high-risk areas on the superimposed image based on the adaptive risk threshold; constructing a Kalman filter to initialize the needle tip state estimation; extracting the needle tip feature template in the enhanced ultrasound image; using the normalized cross-correlation method to perform template matching in the predicted area; updating the needle tip position estimation by combining the Kalman filter prediction and template matching results; calculating the distance from the needle tip to the target, the current local risk value and the needle tip movement speed; dynamically adjusting the color and transparency of the visual feedback based on the above calculation results; and at the same time, adjusting the frequency and volume of the audio feedback based on the above calculation results; synthesizing all visual elements into a single layer to generate the final augmented reality guidance interface.

[0021] Specifically, before executing the method, the following data preparation steps are also included: obtaining the model and parameter information of the ultrasound equipment to ensure that its frame rate is not less than 30fps and the resolution is not less than 1024x768; collecting relevant medical information of the patient, including age, gender, medical history and relevant examination results; obtaining the surgical plan, including puncture target, entry point selection and avoidance structure; loading pre-trained deep learning model parameters, including image segmentation model and feature extraction model; initializing the policy network and value network parameters of the reinforcement learning agent; calibrating the spatial position and direction of the ultrasound probe; and performing system self-test to ensure that all modules are working properly.

[0022] An ultrasound-guided intelligent puncture path planning system for implementing the method includes: an image acquisition module for obtaining a real-time ultrasound image sequence; an image preprocessing module for preprocessing and enhancing the real-time ultrasound image sequence; an image segmentation module for performing multi-scale deep learning image segmentation on the enhanced ultrasound image; a target positioning module for performing target positioning and feature extraction based on the segmentation results; a path planning module for performing path planning using deep reinforcement learning; a risk assessment module for performing dynamic risk assessment; and a visual guidance module for generating an augmented reality guidance interface; wherein the image acquisition module includes an ultrasound probe and an image acquisition card; the image preprocessing module, image segmentation module, target positioning module, path planning module, and risk assessment module are all implemented by a computer with GPU acceleration function; and the visual guidance module includes a display and a stereo sound system.

[0023] The method first acquires a real-time ultrasound image sequence, preprocesses and enhances it, then uses multi-scale deep learning for image segmentation to precisely locate targets and key structures. Next, a deep reinforcement learning algorithm is used for path planning, combined with a dynamic risk assessment mechanism to adjust the puncture strategy in real time. Finally, the system generates an intuitive augmented reality guidance interface, providing doctors with real-time, precise operational guidance.

[0024] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0025] 1. The multi-scale deep learning segmentation algorithm significantly improves the recognition accuracy of complex anatomical structures, laying the foundation for precise path planning;

[0026] 2. Using deep reinforcement learning for path planning can find the optimal balance between multiple goals, improving planning accuracy and safety;

[0027] 3. The introduction of a dynamic risk assessment mechanism enables real-time assessment and adjustment of surgical risks, greatly improving surgical safety;

[0028] 4. Through optimized hardware configuration and parallel computing technology, the real-time operation of complex algorithms is achieved, meeting the needs of dynamic surgical environments;

[0029] 5. The integration of augmented reality technology provides doctors with intuitive and real-time operation guidance, effectively reducing the difficulty and risk of operation.

[0030] In general, the present invention provides a comprehensive and efficient solution for ultrasound-guided precision puncture surgery, which is expected to significantly improve the success rate and safety of minimally invasive surgery and bring better treatment effects to patients. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is the overall logic block diagram of the method of the present invention.

[0032] Figure 2 This is a logic block diagram of the image preprocessing and enhancement steps of the present invention.

[0033] Figure 3 This is a logic block diagram of the multi-scale deep learning image segmentation steps of the present invention.

[0034] Figure 4 This is a logic block diagram of the deep reinforcement learning path planning steps of the present invention.

[0035] Figure 5 This is a logic block diagram of the dynamic risk assessment steps of the present invention.

[0036] Figure 6 This is a logic block diagram of the steps for generating an augmented reality guidance interface of the present invention.

[0037] Figure 7 It is the overall logic block diagram of the system of the present invention.

[0038] Figure 8 This is a multi-dimensional dynamic decision tree diagram of the present invention and comparative examples 1-3.

[0039] Figure 9 3D puncture trajectory comparison diagram of the present invention and comparative examples 1-3. DETAILED DESCRIPTION

[0040] To further illustrate the technical means and effects employed by the present invention to achieve its intended objectives, the following detailed description, along with the accompanying drawings and preferred embodiments, includes a detailed description of the specific implementations, structures, features, and effects thereof. In the following description, references to "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0041] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0042] Please refer to Figure 1-9 This invention discloses an intelligent ultrasound-guided puncture path planning method and system, particularly suitable for precise puncture procedures of deep-seated organs such as the liver and prostate. By combining real-time ultrasound image processing, deep learning, and reinforcement learning techniques, this method provides physicians with safe and efficient puncture path planning and real-time guidance. The following describes specific implementations of this invention in detail.

[0043] Step S1: Acquire a real-time ultrasound image sequence.

[0044] First, the present invention uses a high-frequency ultrasound probe to acquire real-time ultrasound image sequences of the target area. In a preferred embodiment, a Philips iU22 ultrasound system with a frequency range of 5-12 MHz is used. To ensure image quality and real-time performance, the acquisition frame rate is set to no less than 30 fps and the resolution is set to no less than 1024x768. These parameters were selected based on clinical experience to ensure image detail while meeting the requirements of real-time processing.

[0045] Step S2: Preprocessing and enhancing the real-time ultrasound image sequence

[0046] This step aims to improve the quality of ultrasound images and lay the foundation for subsequent analysis. First, we use the adaptive median filter algorithm to denoise the image. The core of this algorithm is to dynamically adjust the filter window size to adapt to the noise level in different areas. Specifically, we calculate the local standard deviation σ of each pixel local (x,y) and the global standard deviation σ global To determine the appropriate window size. The formula for calculating the window size is:

[0047]

[0048] Here, we limit the window size to between 3 and 11. This range is based on empirical values derived from a large number of experiments and can achieve a good balance between denoising effect and detail preservation.

[0049] Next, we perform contrast-limited adaptive histogram equalization (CL-AHE) on the denoised image. The goal of this step is to enhance image contrast while avoiding over-enhancing noise. To this end, we introduce the concept of adaptive clipping threshold, which is calculated as:

[0050] P clip(x,y)=α+(1-α)·(max(H(x,y))-min(H(x,y))) / max(H(x,y))

[0051] Where H(x, y) is the local histogram, α is the basic clipping threshold, and the default value is set to 0.01. This value is optimized through repeated experiments and can achieve a good balance between enhancing image contrast and suppressing noise enhancement.

[0052] Finally, we perform adaptive gamma correction to further optimize the brightness and contrast of the image. The calculation formula of the gamma correction parameter is as follows

[0053] γ(x, y)=1+β·(μ local (x, y)-μ global ) / σ global

[0054] Here, μ local (x, y) is the local mean, μ global is the global mean, and β is a tuning parameter with a default value of 0.5. The choice of this parameter value is based on the trade-off between contrast and brightness and can be fine-tuned according to the specific application scenario.

[0055] Through the above steps, we obtain the enhanced ultrasound image I with significantly improved quality. enhanced (t), laying a solid foundation for subsequent image segmentation and feature extraction.

[0056] Step S3: Perform multi-scale deep learning image segmentation based on the enhanced ultrasound image.

[0057] In this step, we use a modified U-Net++ network model for image segmentation. First, we construct an encoder-decoder network structure with five scales. The encoder uses a ResNet-50 with dilated convolutions as the backbone network to increase the receptive field and capture more contextual information. The decoder uses transposed convolutions for upsampling and fuses features at different scales through dense skip connections to preserve more detailed information.

[0058] To further improve the model's performance, we introduced spatial and channel-wise attention mechanisms in the feature fusion module. Spatial attention, calculated through 3x3 convolutions, highlights key image regions; channel-wise attention, implemented using global average pooling and 1x1 convolutions, helps emphasize important feature channels. This dual attention mechanism significantly improves the model's ability to recognize key anatomical structures.

[0059] In the design of the loss function, we adopted a multi-task learning strategy, comprehensively considering the overall segmentation accuracy, boundary accuracy and category balance. Specifically, our loss function consists of three parts:

[0060] L total =λ1·L dice +λ2·L boundary +λ3·L focal

[0061] Among them, L dice is the Dice loss, which is used to optimize the overall segmentation accuracy; L boundary is the boundary loss, focusing on improving the accuracy of boundary segmentation; L focal Focal loss is used to address class imbalance. λ1, λ2, and λ3 are weight coefficients, with default values set to 0.5, 0.3, and 0.2, respectively. These weights were optimized through extensive experimentation and achieve a good balance between different task objectives.

[0062] Through this multi-scale, multi-task deep learning method, we can obtain high-quality segmentation results L(t), which provides reliable anatomical structure information for subsequent target positioning and path planning.

[0063] Step S4: Target positioning and feature extraction are performed based on the segmented anatomical structure label map.

[0064] In this step, we first need to accurately locate the target area. To this end, we designed an improved connected domain analysis algorithm, which not only considers the segmentation results of the current frame, but also integrates the information of the previous k frames (default k = 5) to improve the stability of positioning. Specifically, we first perform pixel-by-pixel AND operations on the k+1 frame label map to obtain a stable region mask. Then, we perform a morphological opening operation on this mask to remove small noise areas. Next, we calculate the area, perimeter and shape complexity of the remaining connected regions. Finally, candidate target areas are screened out according to preset thresholds (for example, an area greater than 100 pixels and a shape complexity less than 0.8).

[0065] After determining the candidate target region, we use the weighted centroid algorithm to accurately locate the target position. Here, we calculate the weight based on the grayscale value of the pixels in the candidate region in the enhanced ultrasound image:

[0066]

[0067] We then use these weights to calculate the weighted centroid and get the target position coordinates P target :

[0068]

[0069] This weighted centroid method takes into account the grayscale distribution of pixels within the target area and can locate the target center more accurately, especially for targets with irregular shapes or complex internal structures.

[0070] Next, we perform feature extraction. We extract a 64x64 pixel local region centered on the target location. This size is chosen to strike a balance between including sufficient contextual information and maintaining computational efficiency. Within this local region, we extract three types of features:

[0071] The first is shape features, including roundness, eccentricity, and directionality. These features can describe the basic geometric properties of the target and are very helpful in distinguishing different types of anatomical structures (such as blood vessels, tumors, etc.).

[0072] Next, we use texture features. We use the Gray Level Co-occurrence Matrix (GLCM) to calculate statistics such as energy, contrast, and correlation. The GLCM calculation typically considers four directions (0°, 45°, 90°, and 135°) and distances of 1-5 pixels to capture texture information at different scales and directions. We also use Local Binary Patterns (LBP) to extract texture features. LBP features are rotationally invariant and can effectively describe local texture patterns.

[0073] Finally, we extract deep features. Here, we use a pretrained ResNet-18 network to extract 512-dimensional deep features. ResNet-18 was chosen because it maintains high feature extraction capabilities while maintaining relatively low computational complexity, making it suitable for real-time processing. The pretrained model uses weights trained on a large-scale natural image dataset. We have found that these general features also perform well in medical image analysis tasks.

[0074] By stitching these features together, we get a comprehensive feature descriptor D target This feature descriptor contains the shape, texture and semantic information of the target, providing a rich feature representation for subsequent path planning.

[0075] Step S5: Based on the segmented anatomical structure label map and target position coordinates, deep reinforcement learning is used for path planning.

[0076] This step is the core innovation of this invention. We use an improved deep reinforcement learning algorithm for path planning. First, we need to build a dynamic risk map. The formula for building the risk map is as follows:

[0077] R(xy,t)=w1(t)·D(x,y,t)+w2(t)·U(x,y,t)+w3(t)·C(x,y,t)+w4(t)·V(x,y,t)

[0078] Where R(x, y, t) is the combined risk at location (x, y) and time t, D(x, y, t) represents the distance to the nearest obstacle, U(x, y, t) represents uncertainty, C(x, y, t) represents the degree of tissue deformation, and V(x, y, t) represents vascular density. w1(t), w2(t), w3(t), and w4(t) are dynamically adjusted weight coefficients. To dynamically adjust these weights, we use a gradient descent-based method:

[0079]

[0080] Here, w i (t) is the i-th weight at time t, is the gradient of the risk value R to the weight, η is the learning rate, usually set between 0.01 and 0.1, R target is the target risk value, R current is the current risk value. This dynamic adjustment mechanism enables the system to continuously optimize risk assessment strategies based on real-time feedback.

[0081] Next, we adopt a hierarchical reinforcement learning framework for path planning. At the global planning level, we use the improved Deep Q-Network (DQN) algorithm

[0082] Q(sa)=(1-α)Q(s,a)+α[r+γ·max(Q′(s′,a′)+ε·H(π(s′)))]

[0083] Here, Q(s, a) is the Q-value of taking action a in state s, α is the learning rate, r is the immediate reward, γ is the discount factor, Q′(s′, a′) is the Q-value of taking action a′ in the next state s′, ε is the entropy regularization coefficient, and H(π(s′)) is the entropy of policy π in state s′. Introducing policy entropy can encourage exploration and prevent the algorithm from converging to a suboptimal solution too early.

[0084] At the local optimization level, we designed a new Advantage Actor-Critic (A2C) algorithm:

[0085] A(sa)=Q(s,a)-V(s)

[0086]

[0087]

[0088] Among them, A(s, a) is the advantage function, V(s) is the state value function, θ π are policy network parameters, θ v is the value network parameter, J(θ) is the policy objective function, L(θ) is the value function loss, and b(s) is the baseline function used to reduce variance. This actor-critic structure can improve learning efficiency while ensuring stability.

[0089] To further improve the performance of the algorithm, we introduced a multi-agent collaborative learning mechanism. Each agent is responsible for different subtasks (such as security, efficiency, etc.), and collaborative learning is carried out through the knowledge distillation mechanism:

[0090] To further improve the performance of the algorithm, we introduced a multi-agent collaborative learning mechanism. Each agent is responsible for different subtasks (such as security, efficiency, etc.), and collaborative learning is carried out through the knowledge distillation mechanism:

[0091] Q i (sa)=Q i (s, a)+α i [r i +γ·max(Q i (s′, a′))-Q i (s, a)]+β i ·KL(π teacher ||π i )

[0092] Here, Q i (s, a) is the Q value of the i-th agent taking action a in state s, α i is the learning rate of the ith agent, r i is the reward obtained by the i-th agent, β i is the knowledge distillation coefficient, KL(π teacher ||π i ) is the KL divergence between the teacher’s policy and the policy of agent i. The teacher’s policy is obtained by integrating the policies of all agents:

[0093] π teacher (a|s)=softmax(∑w i Q i (s, a) / τ)

[0094] Among them, π teacher (a|s) is the probability that the teacher strategy chooses action a in state s, w i The weight of the i-th agent, τ is the temperature parameter, which controls the degree of "softening" of the strategy.

[0095] Finally, we use the improved Rapidly-exploring RandomTrees (RRT*) algorithm for path optimization. During the sampling process, we combine the learned Q value for biased sampling:

[0096] P(x random )∝exp(Q(s x , a optimal ) / T)

[0097] Here, P(x random ) is to select a random point X random The probability of Q(s x , a optimal ) is the state s x The Q-value of the optimal action is obtained, and T is the temperature parameter, which controls the balance between exploration and exploitation. In this way, we can maintain the rapid exploration capability of the RRT* algorithm while using the knowledge gained from reinforcement learning to guide sampling and improve planning efficiency.

[0098] Through the above steps, we finally get the optimal puncture path Path optimal This path not only takes into account spatial geometric constraints, but also integrates multiple factors such as safety and efficiency, and can provide doctors with the best puncture recommendations.

[0099] Step S6: Perform dynamic risk assessment based on the optimal puncture path and the enhanced ultrasound image.

[0100] After obtaining the optimal puncture path, we need to perform a dynamic risk assessment to ensure surgical safety. First, we use an adaptive sampling algorithm to extract key points along the path. The core principle of this algorithm is to dynamically determine the sampling interval based on the path curvature and risk distribution. Specifically, we increase the sampling density in areas with large path curvature or high risk; conversely, we appropriately reduce the sampling points in straight segments or low-risk areas to balance computational efficiency and assessment accuracy.

[0101] For each sampled key point, we perform a local risk assessment. This assessment takes into account multiple factors, including obstacle distance, uncertainty, tissue deformation, and vascular density. We use a combination of fuzzy logic and evidence theory to handle the uncertainty of these factors. First, we convert each risk factor into a fuzzy set using fuzzy rules. For example, for obstacle distance, we may have fuzzy sets such as "very close," "close," "medium," "far," and "very far." Then, we calculate the basic probability allocation (BPA) for each risk factor based on evidence theory. Finally, we fuse the evidence of multiple risk factors using the Dempster combination rule to obtain a comprehensive local risk assessment result.

[0102] The advantage of this approach is that it can effectively handle the uncertainty and ambiguity common in medical images and provide more robust and interpretable risk assessment results.

[0103] To obtain the overall path risk assessment, we designed a risk aggregation function based on a recurrent neural network (RNN). Specifically, we used a long short-term memory network (LSTM) as the basic unit of the RNN and took the sequence of local risk assessment results as input:

[0104] h t =LSTM(Risk local (P t ), h t-l )

[0105] Among them, h t is the hidden state at time t, Risk local (P t ) is the local risk assessment result of the key point at time t.

[0106] In addition to the local risk sequence, we also extract global features of the path, such as path length, average curvature, etc. These global features are concatenated with the final hidden state of the LSTM, and through a fully connected layer and a Sigmoid activation function, the overall path risk assessment result is output:

[0107] Risk global =σ(W·[h final , Path features ]+b)

[0108] Among them, Risk global is the result of global risk assessment, Path features is the path global feature, σ is the Sigmoid function, W and b are the weight matrix and bias term respectively, h final is the final hidden state of the LSTM.

[0109] This approach, which combines local and global information, enables a comprehensive assessment of path risk, taking into account the local risk of each point on the path without neglecting the overall characteristics of the path.

[0110] Step S7: Generate an augmented reality guidance interface based on the optimal puncture path, risk assessment results, and enhanced ultrasound image.

[0111] The final step is to integrate all this information and generate an intuitive augmented reality guidance interface to assist doctors in performing precise punctures. First, we need to overlay the planned optimal puncture path onto the real-time ultrasound image. To this end, we designed an improved image fusion algorithm that takes into account image features and human eye perception:

[0112] I fusion =λ(x, y)·I enhanced +(1-λ(x, y))·I path

[0113] Among them, I fusion is the fused image, I enhanced is the enhanced ultrasound image, I path is the path image, λ(x, y) is the adaptive fusion weight, which is calculated by the following formula:

[0114] λ(xy)=sigmoid(α·S(x,y)+β·C(x,y))

[0115] Here, S(x, y) is the saliency map, C(x, y) is the contrast map, and α and β are weight coefficients. This adaptive fusion method can clearly display the planned path while maintaining the details of the original ultrasound image.

[0116] Next, we need to mark the high-risk areas. We use an adaptive threshold algorithm to dynamically determine the threshold of high-risk areas based on the overall risk distribution:

[0117] Threshold risk =μ risk +k·σ risk

[0118] Among them, μ risk and σ risk are the mean and standard deviation of the risk values, respectively, and k is an adjustable parameter, usually set between 1.5 and 2.5. Areas above this threshold are marked as high-risk areas and marked with a striking color (such as red) on the interface.

[0119] In order to achieve real-time needle tip tracking, we designed a robust needle tip tracking algorithm that combines Kalman filtering and template matching. Kalman filtering is used to predict the motion state of the needle tip:

[0120] X t =F·X t-1 +B·u t +w t

[0121] Z t =H·X t +v t

[0122] Among them, X t It is in the needle tip state, Z t is the observation value, F is the state transfer matrix, H is the observation matrix, w t and v tare process noise and observation noise respectively, B is the control input matrix, u t is the control input.

[0123] Based on the prediction, we use template matching method for precise positioning. The matching score is calculated by normalized cross-correlation coefficient (NCC):

[0124] Among them, Template is a predefined needle tip template, I roi (x, y) is the image patch of the region of interest. Finally, we designed a multimodal feedback system to dynamically adjust the visual and audio feedback strength according to the puncture status:

[0125] Score(x,y)=NCC(Template,I roi (x, y)

[0126] Visual feedback =h v (dist to_target , Risk local , needle orientation )

[0127] Audio feedback =h a (dist to_target , Risk local , needle speed )

[0128] Where Score(x, y) is the matching score of position (x, y), NCC is the normalized cross-correlation coefficient, Template is the predefined tip template, and I roi (x, y) is the image of the region of interest at position (x, y). Visual_feedback is the visual feedback strength, ranging from [0, 1], where 0 means no visual feedback and 1 means the strongest visual feedback; Audio_feedback is the audio feedback strength, ranging from [0, 1], where 0 means no audio feedback and 1 means the loudest audio feedback; h v is the nonlinear mapping function of visual feedback, which maps the input parameters to the visual feedback intensity. a It is a nonlinear mapping function for audio feedback, mapping the input parameters to the audio feedback intensity. For example, when approaching the target or entering a high-risk area, the visual cue will become more obvious and the audio warning will be strengthened accordingly. to_target is the Euclidean distance from the needle tip to the target position, in millimeters (mm); Risk local The local risk value of the current location is in the range [0, 1], where 0 represents the lowest risk and 1 represents the highest risk. needle orientationThe needle is the direction vector of the needle, expressed as a three-dimensional unit vector [x, y, z]. speed is the moving speed of the needle, in millimeters per second (mm / s).

[0129] These nonlinear mapping functions h v and h a Can be defined as:

[0130] h v (dist,risk,orient)=sigmoid(w1·(1-dist / d max )+w2·risk+w3·(1-dot(orient,target o rient)))

[0131] h a (dist, risk, speed) = sigmoid (w4·(1-dist / d max )+w5·risk+w6·(speed / speed max ))

[0132] Among them, sigmoid(x) = 1 / (1+exp(-x)) is the Sigmoid function, which is used to map the output to the range of [0, 1]. w1, w2, w3, w4, w5, w6 are weight parameters used to adjust the importance of each factor. max is the preset maximum distance threshold. When dist_to_target exceeds this value, the distance factor is 0. det(orient, target_orient) is the dot product of the current direction of the needle and the target direction, which is used to measure the degree of direction matching. target_orient is the target direction, expressed as a three-dimensional unit vector [x, y, z]. speed max is the preset maximum speed threshold. When needle_speed exceeds this value, the speed factor is 1. These definitions enable the feedback system to dynamically adjust the intensity of visual and audio feedback based on the distance from the needle tip to the target, local risk, direction of the needle, and speed. For example, when the needle tip approaches the target (dist_to_target decreases), enters a high-risk area (Risk_local increases), the direction of the needle deviates from the target (det(orient, target_orient) decreases), or the speed is too fast (needle_speed increases), the system will increase the corresponding feedback intensity. In this way, the system can provide the operator with intuitive and real-time multimodal feedback, effectively improving the accuracy and safety of the surgery.

[0133] Through the above steps, we generated a comprehensive augmented reality guidance interface that not only displays the optimal puncture path and real-time needle tip position, but also provides intuitive risk warnings through multimodal feedback. This interface can greatly enhance the surgeon's operating precision and confidence, thereby significantly improving surgical outcomes.

[0134] In summary, this invention combines advanced image processing, deep learning, reinforcement learning, and augmented reality technologies to provide a comprehensive and innovative ultrasound-guided intelligent puncture path planning method and system. This method not only accurately locates the target and plans the optimal path, but also provides real-time risk assessment and intuitive operational guidance. It is expected to play a significant role in clinical practice, improving the safety and success rate of minimally invasive surgery.

[0135] After describing the steps of the above method in detail, we will now introduce the system for implementing this method. The ultrasound-guided intelligent puncture path planning system of the present invention is a complex system integrating multiple functional modules, each of which plays a key role in the entire workflow.

[0136] First, the core of the system is the image acquisition module 1. This module consists of a high-frequency ultrasound probe and a high-performance image acquisition card. In one embodiment, we selected a Philips iU22 ultrasound system equipped with a 5-12 MHz linear array probe. This probe provides high-resolution real-time images and is particularly suitable for detailed scanning of deep organs. It is paired with an image acquisition card that supports high frame rate acquisition, such as the Matrox Radient eV-CXP frame grabber, which can achieve image acquisition rates of up to 60 fps, fully meeting our real-time processing requirements.

[0137] Next comes the image preprocessing module 2. This module is responsible for denoising and enhancing the raw ultrasound images. Given the unique characteristics and complexity of medical image processing, we employ GPU acceleration technology in this module. Specifically, we use an NVIDIA Tesla V100 GPU to execute image processing algorithms. This high-performance GPU, with 5120 CUDA cores and 640 Tensor cores, significantly accelerates algorithms such as adaptive median filtering and contrast-limited adaptive histogram equalization, ensuring real-time performance.

[0138] The image segmentation module 3 is another key component of the system. This module runs the multi-scale U-Net++ network described in detail earlier. To support the efficient execution of deep learning models, we also use GPU acceleration. In practice, we found that a single GPU may not be able to meet the requirements of real-time segmentation, so we adopted a multi-GPU parallel computing solution. Specifically, we used two NVIDIA RTX 3090 GPUs, each equipped with 24GB of video memory, sufficient to accommodate our large neural network model. Using PyTorch's DistributedDataParallel feature, we achieved efficient parallel training and inference of the model.

[0139] Both the target localization module 4 and the path planning module 5 are computationally intensive components. These modules primarily run on the CPU because they involve extensive logical reasoning and complex mathematical operations. For our system, we chose the Intel Xeon Gold 6248R processor, a server-grade CPU with 24 cores, 48 threads, and a clock speed of up to 4.0 GHz. This high-performance CPU can quickly process complex connected domain analysis, feature extraction, and reinforcement learning algorithms.

[0140] Risk Assessment Module 6 is a complex module that integrates multiple technologies. It requires simultaneous image data processing and numerical computation. Therefore, we employed a collaborative approach using both the CPU and GPU. Image-related processing tasks, such as generating local risk maps, are performed on the GPU, while risk aggregation based on a recurrent neural network is executed on the CPU. This heterogeneous computing architecture fully leverages the strengths of different hardware types to improve overall computational efficiency.

[0141] Finally, the Visual Guidance module 7 is responsible for generating the augmented reality interface. This module's output is directly intended for the user, so both real-time performance and visual quality are crucial. We use OpenGL for efficient graphics rendering, and NVIDIA's CUDA-OpenGL interoperability enables rapid transfer of GPU computation results to the graphics display. For output, we chose a 4K monitor with a resolution of 3840x2160 and a refresh rate of 144Hz to provide a clear and smooth visual experience. A high-quality stereo sound system also provides precise audio feedback.

[0142] It’s worth noting that the entire system’s modules communicate with each other via a high-speed internal network. We utilize InfiniBand networking technology, which boasts a bandwidth of up to 200Gb / s. This ensures the real-time transmission of large amounts of image and computational data, minimizing system latency.

[0143] Furthermore, given the unique characteristics of the medical environment, our system incorporates comprehensive fault tolerance and backup mechanisms. All critical hardware features redundant design, and a RAID storage system is employed to protect critical data. The system is also equipped with an uninterruptible power supply (UPS) to mitigate potential power outages.

[0144] Overall, this ultrasound-guided intelligent puncture path planning system is a highly integrated and powerful medical assistance system. It seamlessly combines advanced hardware technology with innovative software algorithms, providing physicians with a reliable, efficient, and intuitive surgical assistance tool. Through this system, we hope to significantly improve the precision and safety of minimally invasive surgery, ultimately benefiting more patients.

[0145] To validate the superiority of the ultrasound-guided intelligent puncture path planning method and system of the present invention, we designed a series of comparative experiments. These experiments not only demonstrate the effectiveness of our method but also provide a comprehensive comparison with existing technologies. Below, we will detail the setup of the examples and comparative examples, along with the corresponding experimental results and analysis.

[0146] Example 1: Method of the present invention

[0147] In this example, we fully implemented the proposed method, including all steps of adaptive image enhancement, multi-scale deep learning segmentation, deep reinforcement learning-based path planning, and dynamic risk assessment. We used the hardware configuration described above, including a Philips iU22 ultrasound system and an NVIDIA Tesla V100 GPU.

[0148] Comparative Example 1: Traditional Manual Planning Method

[0149] This comparison example represents the traditional method currently widely used in clinical practice, in which doctors rely entirely on their own experience and judgment to plan the puncture path without the support of any intelligent assistance system.

[0150] Comparative Example 2: Method based on a single deep learning model

[0151] This comparative example uses an end-to-end deep learning model to predict the puncture path directly from ultrasound images. Specifically, we use a modified U-Net model that uses ResNet-50 as the backbone network in the encoder part to improve feature extraction capabilities.

[0152] Comparative Example 3: Method based on traditional image processing and search algorithms

[0153] This example combines traditional image processing techniques with path finding algorithms. We use an adaptive threshold segmentation method to identify targets and obstacles, and then use the A* algorithm to plan the path.

[0154] In order to comprehensively evaluate the performance of these methods, we designed the following key indicators:

[0155] 1. Path Planning Accuracy: This measures whether the planned path accurately reaches the target location. We use Euclidean distance to calculate the distance between the planned path endpoint and the actual target point. If the distance is less than 5mm, the plan is considered accurate.

[0156] 2. Safety Index: This evaluates the minimum distance between the planned path and critical structures (such as large blood vessels and nerves). We calculate the distance from all points on the path to the nearest critical structure and take the minimum value as the safety index.

[0157] 3. Path smoothness: This is measured using the average curvature of the path. The higher the smoothness, the easier the path is to operate.

[0158] 4. Computation time: The total time from input image to output planned path, reflecting the real-time performance of the method.

[0159] 5. Risk assessment accuracy: Compare the system’s risk assessment results with the expert scores and calculate their correlation coefficient.

[0160] We tested each method on 100 real liver biopsy cases, running them under the same hardware environment. The test results are shown in the following table:

[0161] index Example 1 Comparative Example 1 Comparative Example 2 Comparative Example 3 Path planning accuracy 95％ 78％ 87％ 82％ Safety index (mm) 8.7 5.2 7.1 6.5 Path smoothness 0.92 0.75 0.83 0.79 Computation time (s) 0.8 N / A 1.2 2.5 Risk assessment accuracy 0.91 0.62 0.78 0.73

[0162] From these results, we can clearly see that the method of the present invention (Example 1) performs well in all indicators and is significantly better than other comparative methods.

[0163] First, in terms of path planning accuracy, our method achieved a high accuracy of 95%, far exceeding other methods. This is primarily due to our multi-scale deep learning segmentation and deep reinforcement learning-based path planning algorithms. These advanced algorithms can better understand complex anatomical structures and find the optimal balance between multiple objectives.

[0164] The method also performed well in terms of safety index, with an average safety distance of 8.7 mm. This reflects the effectiveness of our dynamic risk assessment mechanism, which can assess and adjust path risks in real time, significantly improving surgical safety.

[0165] The high score of path smoothness (0.92) shows that our method not only finds safe paths but also ensures the operability of the paths. This is crucial for actual clinical practice because smooth paths are easier for doctors to execute accurately.

[0166] In terms of computational time, despite the complex deep learning and reinforcement learning algorithms involved in our approach, we were able to achieve this by optimizing hardware configuration and using parallel computing techniques, keeping the computational time to just 0.8 seconds, fully meeting the requirements of real-time operation. This is particularly important because, in dynamic surgical environments, rapid response is directly related to surgical success.

[0167] Finally, our approach also demonstrated significant advantages in risk assessment accuracy, with a correlation coefficient of 0.91 with expert scores. This means our system can provide risk assessments comparable to those of experts, providing a reliable reference for physicians' decision-making.

[0168] Please refer to Figure 8 , a multi-dimensional dynamic decision tree diagram shows the complexity and accuracy of real-time decision-making during the puncture process. The following is an explanation of the figure: The main path represents the overall path planning from the starting point to the end point of the puncture. This main path shows the key decision nodes in the entire process, and each node is an important decision point. The size of the node reflects the uncertainty at the decision point. A larger node indicates that there is greater uncertainty at the decision point, which means that the decision may be more complex or have more variables in actual operation. The node color indicates safety, ranging from green (safe) to red (dangerous). The closer the color is to red, the higher the decision risk of the node. The thickness of the branch indicates the priority of the path or the possibility of selection. A thicker branch means that the path is more likely to be selected or more important during the operation. The color change of the branch also indicates safety. As the path advances from the starting point to the end point, the color gradient reflects the change in risk level. Example 1 has more decision nodes, a finer branch structure, and an overall greener main path in the figure, showing higher safety and accuracy. Comparative methods may exhibit fewer decision nodes, fewer branch options, or a color shift toward yellow or red in the decision tree, indicating deficiencies in safety and accuracy. This decision tree diagram intuitively demonstrates the advantages and disadvantages of different methods for complex surgical procedures. The structure of the decision tree, the size and color of the nodes, and the thickness and safety distribution of the paths clearly demonstrate the significant advantages of the present method in terms of path planning precision, safety, and reliability.

[0169] Overall, the multi-dimensional dynamic decision tree diagram not only demonstrates the complexity of decision-making at each step during the surgical process, but also intuitively reflects the differences in safety, accuracy, and operability between different methods. This provides an important reference for evaluating and selecting the optimal surgical path.

[0170] Please refer to Figure 9 , this 3D puncture trajectory comparison diagram shows the comparison of puncture paths of four different methods in a complex anatomical environment. The following is an explanation of the figure: Example 1 (the green trajectory represents the method proposed by the present invention. This path is relatively thin, indicating that the positioning of the puncture path is very precise and avoids important anatomical structures. The green color also shows that the path is relatively safe. Comparative Example 1, Comparative Example 2, Comparative Example 3 (blue, orange and red trajectories), these trajectories represent comparisons with other methods. The thickness and color changes of the path show different degrees of positioning uncertainty and safety. For example, the red trajectory indicates that the method has a higher risk in certain areas, and the thickness of the path shows that the positioning is not as accurate as Example 1. Important blood vessels (marked in purple) and target lesions (marked in black), the figure marks the key anatomical structures that require special attention during the puncture process. The puncture path of Example 1 obviously avoids important blood vessels, This shows that this method has obvious advantages in safety. At the same time, the path eventually reaches the target lesion, indicating that its target achievement rate is high. The trajectory of Example 1 not only avoids high-risk areas, but also shows higher path smoothness and safety, which is very important in actual clinical operations. A smoother path means that the surgical operation is easier and safer. It can be seen from the thickness of the path that the path of Example 1 has higher accuracy and smaller positioning error, while the path of the comparative method shows greater uncertainty and risk. This figure intuitively demonstrates the superiority of the method of the present invention in complex anatomical structures, including higher safety, more accurate path planning, and better clinical applicability. Through these comparisons, it can be clearly seen that this method is superior to traditional or other comparative methods in all indicators.

[0171] Overall, these experimental results fully demonstrate the superiority of our method. It not only excels in accuracy and safety, but also provides high-quality path planning and risk assessment while ensuring real-time performance. This comprehensive performance advantage makes our method particularly suitable for complex minimally invasive surgical scenarios, such as targeted puncture treatment of liver tumors.

[0172] It's worth noting that while Comparative Example 2 (based on a single deep learning model) performs well on some metrics, it still suffers from significant deficiencies in risk assessment and path smoothness. This demonstrates that relying solely on a single deep learning model is insufficient; it requires comprehensive consideration of multiple factors, as our approach does, and the introduction of a more complex algorithmic framework.

[0173] In summary, these experimental results not only validate the effectiveness of our method but also point the way for the future development of intelligent surgical planning systems. We believe that with further refinement and widespread adoption of this technology, it will revolutionize minimally invasive surgery and ultimately benefit even more patients.

[0174] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An intelligent puncture path planning method under ultrasound guidance, characterized in that: The following steps are involved: Acquire real-time ultrasound image sequences; Preprocessing and enhancing the real-time ultrasound image sequence to obtain an enhanced ultrasound image; Based on the enhanced ultrasound image, multi-scale deep learning image segmentation is performed to obtain a segmented anatomical structure label map; based on the segmented anatomical structure label map, target positioning and feature extraction are performed to obtain target position coordinates and feature descriptors; based on the segmented anatomical structure label map and the target position coordinates, deep reinforcement learning is used to perform path planning to obtain an optimal puncture path; based on the optimal puncture path and the enhanced ultrasound image, dynamic risk assessment is performed to obtain a risk assessment result; generating an augmented reality guidance interface based on the optimal puncture path, the risk assessment result, and the enhanced ultrasound image; The steps of using deep reinforcement learning to perform path planning based on the segmented anatomical structure label map and the target position coordinates include: calculating the distance from each position to the nearest obstacle based on the segmented anatomical structure label map to obtain an obstacle distance map; generating an uncertainty map based on the confidence of the segmented anatomical structure label map; estimating the tissue deformation field using the registration results between consecutive frames to obtain a deformation degree map; calculating a local vascular density map based on the vascular segmentation result; weightedly combining the obstacle distance map, the uncertainty map, the deformation degree map and the vascular density map to obtain a comprehensive risk map; constructing a reinforcement learning environment based on the comprehensive risk map and the target position coordinates; using the Deep Q-Network algorithm in the reinforcement learning environment to perform global path planning; based on the global path, using the Advantage An Actor-Critic algorithm is used to optimize local paths; an adaptive reward function is designed based on the safety, efficiency, and accuracy of the path; multiple agents are introduced, each responsible for different subtasks, namely safety, efficiency, and accuracy; a knowledge distillation mechanism is used to enable knowledge sharing and collaborative learning among the agents; a teacher strategy is constructed by integrating the strategies of all agents; the teacher strategy is used to guide the learning process of each agent; a Rapidly-exploring Random Trees algorithm is used to sample and optimize the path in combination with the learned Q value, and biased sampling is performed during the sampling process in combination with the learned Q value; the optimization is iterated until convergence or a preset number of iterations is reached to obtain the optimal puncture path.

2. The method according to claim 1, characterized in that The steps of preprocessing and enhancing the real-time ultrasound image sequence include: calculating a local standard deviation and a global standard deviation based on a local area of each pixel in the real-time ultrasound image sequence; determining an adaptive filtering window size based on a ratio of the local standard deviation to the global standard deviation; performing median filtering on the real-time ultrasound image sequence using the adaptive filtering window size to obtain a denoised image sequence; calculating a local histogram for each image in the denoised image sequence; determining an adaptive clipping threshold based on the maximum and minimum values of the local histogram; performing contrast-limited adaptive histogram equalization on the denoised image sequence using the adaptive clipping threshold to obtain an equalized image sequence; calculating a local mean and a global mean for each image in the equalized image sequence; determining an adaptive gamma correction parameter based on a difference between the local mean and the global mean; and performing gamma correction on the equalized image sequence using the adaptive gamma correction parameter to obtain the enhanced ultrasound image.

3. The method according to claim 1, characterized in that The steps of performing multi-scale deep learning image segmentation based on the enhanced ultrasound image include: constructing a multi-scale U-Net++ network model, including an encoder and a decoder; in the encoder, extracting multi-scale features of the enhanced ultrasound image through continuous convolution layers and pooling layers; in the decoder, gradually restoring the spatial resolution of the feature map through transposed convolution layers; adding dense skip connections between the encoder and the decoder to achieve fusion of features of different scales; calculating a spatial attention map of the fused feature map, where the spatial attention map reflects the importance of different image regions; calculating a channel attention vector of the fused feature map, where the channel attention vector reflects the importance of different feature channels; multiplying the spatial attention map and the channel attention vector with the fused feature map to obtain a weighted feature map; based on the weighted feature map, generating a multi-category segmentation probability map through a 1x1 convolution layer and a softmax activation function; calculating Dice loss, boundary loss and Focal loss, and weightedly combining these three losses to obtain a total loss function; optimizing the parameters of the network model through back propagation and gradient descent, and finally obtaining the segmented anatomical structure label map.

4. The method according to claim 1, wherein The steps of performing target positioning and feature extraction based on the segmented anatomical structure label map include: obtaining the segmented anatomical structure label maps of the current frame and the previous k frames; performing temporal consistency analysis on the label map of the k+1 frame to screen out stable connected regions; calculating the area, perimeter, and shape complexity of the stable connected regions, and screening out candidate target regions according to a preset threshold; For the pixels within the candidate target area, weights are calculated based on their grayscale values in the enhanced ultrasound image; using the weights, the weighted centroid of the candidate target area is calculated to obtain the target position coordinates; with the target position coordinates as the center, a local area of fixed size is extracted; shape features, including roundness, eccentricity and directionality, are calculated for the local area; texture features, including grayscale co-occurrence matrix features and local binary pattern features, are calculated for the local area; using a pre-trained convolutional neural network, depth features of the local area are extracted as context features; the shape features, the texture features and the context features are spliced to form the feature descriptor.

5. The method according to claim 1, wherein The steps of constructing the comprehensive risk map include: initializing weight coefficients based on the obstacle distance map, the uncertainty map, the deformation degree map and the blood vessel density map; calculating the difference between the current risk map and the target risk distribution; updating the weight coefficients based on the difference using the gradient descent method; using the updated weight coefficients to recalculate the comprehensive risk value; repeating the above steps until the risk map converges or reaches a preset number of iterations; and outputting the final comprehensive risk map.

6. The method according to claim 1, characterized in that The steps of performing dynamic risk assessment based on the optimal puncture path and the enhanced ultrasound image include: calculating the path curvature along the optimal puncture path; determining the sampling interval based on the path curvature and the comprehensive risk map; extracting path key points according to the sampling interval; extracting obstacle distance, uncertainty, tissue deformation and vascular density information of the local area for each path key point; converting the extracted information into a fuzzy set based on fuzzy logic rules; calculating the basic probability distribution of each risk factor based on evidence theory; fusing the evidence of multiple risk factors through the Dempster combination rule to obtain a local risk assessment result; constructing a recursive neural network and taking the local risk assessment result sequence as input; extracting the global features of the optimal puncture path; splicing the global features with the hidden state of the recursive neural network; and outputting the overall path risk assessment result through a fully connected layer and a Sigmoid activation function.

7. The method according to claim 6, characterized in that The steps of generating an augmented reality guidance interface based on the optimal puncture path, risk assessment results and enhanced ultrasound image include: calculating a saliency map and a contrast map of the enhanced ultrasound image; generating an adaptive fusion weight map based on the saliency map and the contrast map; superimposing the optimal puncture path on the enhanced ultrasound image using the adaptive fusion weight map; calculating the global mean and standard deviation of the comprehensive risk map; determining an adaptive risk threshold based on the global mean and the standard deviation; marking high-risk areas on the superimposed image based on the adaptive risk threshold; and constructing a Kalman filter. , initialize the needle tip state estimation; extract the needle tip feature template in the enhanced ultrasound image; use the normalized cross-correlation method to perform template matching in the prediction area; update the needle tip position estimation by combining the Kalman filter prediction and template matching results; calculate the distance from the needle tip to the target, the current local risk value and the needle tip movement speed; dynamically adjust the color and transparency of the visual feedback based on the calculation results of the distance from the needle tip to the target, the current local risk value and the needle tip movement speed; at the same time, adjust the frequency and volume of the audio feedback based on the above calculation results; synthesize all visual elements into a single layer to generate the final augmented reality guidance interface.

8. The method according to any one of claims 1 to 7, characterized in that Before executing the method, the following data preparation steps are also included: obtaining the model and parameter information of the ultrasound equipment, ensuring that its frame rate is not less than 30fps and its resolution is not less than 1024x768; collecting relevant medical information of the patient, including age, gender, medical history and relevant examination results; obtaining the surgical plan, including puncture target, entry point selection and avoidance structure; loading pre-trained deep learning model parameters, including image segmentation model and feature extraction model; Initialize the policy network and value network parameters of the reinforcement learning agent; calibrate the spatial position and orientation of the ultrasound probe; and perform a system self-test to ensure that all modules are functioning properly.

9. An intelligent puncture path planning system under ultrasound guidance for implementing the method of claim 8, characterized in that: include: An image acquisition module, used for obtaining real-time ultrasound image sequences; An image preprocessing module, configured to preprocess and enhance the real-time ultrasound image sequence; An image segmentation module is used to perform multi-scale deep learning image segmentation on the enhanced ultrasound image; Target positioning module, used for target positioning and feature extraction based on segmentation results; A path planning module is used to perform path planning using deep reinforcement learning; a risk assessment module is used to perform dynamic risk assessment; and a visual guidance module is used to generate an augmented reality guidance interface. The image acquisition module includes an ultrasound probe and an image acquisition card. The image preprocessing module, image segmentation module, target positioning module, path planning module, and risk assessment module are all implemented by a computer with GPU acceleration capabilities. The visual guidance module includes a display and a stereo sound system.

Images