Anesthesia puncture auxiliary positioning method and system based on ultrasonic image recognition

Abstract

The invention discloses an anesthesia puncture auxiliary positioning method and system based on ultrasonic image recognition. The method comprises the following steps: acquiring an initial ultrasonic image in real time; segmenting a network model based on improved deep learning, introducing a time sequence module into the network model, and performing real-time semantic segmentation on the initial ultrasonic image to obtain a segmented ultrasonic image; according to the segmented ultrasonic image, calculating an optimal puncture path of a puncture needle from a skin needle insertion point to a target position by using a multi-objective optimized path planning algorithm; and acquiring pose data of a puncture needle in a three-dimensional space, displaying the optimal puncture path and the pose data on a display image in an overlapping manner for anesthesia puncture auxiliary positioning, and adjusting the puncture path according to real-time data in the puncture process until a needle point reaches a target position. And the accuracy and efficiency of the needle point reaching the target area are improved.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to an anesthesia puncture-assisted positioning method and system based on ultrasound image recognition. Background Technology

[0002] The accuracy of puncture in anesthesia directly determines the anesthetic effect and surgical safety. This is especially true for procedures like nerve blocks, which require avoiding dangerous structures such as blood vessels and nerves, demanding extremely high precision. Traditional punctures rely on the physician's clinical experience and anatomical knowledge, locating the target site through palpation. This method is highly subjective, prone to individual variations, and has a high complication rate. Although ultrasound-guided technology has been applied to puncture localization, providing real-time visualization of the target anatomical structure, it still has many limitations: images are susceptible to noise and artifacts, manual image selection is time-consuming and lacks precision, and dynamic path adjustment is difficult, resulting in low puncture safety and efficiency. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing an anesthesia puncture-assisted positioning method and system based on ultrasound image recognition.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned anesthesia puncture-assisted localization method based on ultrasound image recognition: The ultrasound video stream image of the target location is acquired in real time, and the ultrasound video stream image is subjected to adaptive filtering, enhancement and normalization processing to obtain an initial ultrasound image; Based on the improved DeepLabv3+ deep learning segmentation network model, and by introducing a ConvLSTM time series module into the network model, the initial ultrasound image is semantically segmented in real time to obtain a segmented ultrasound image. Based on the segmented ultrasound image, the optimal puncture path of the puncture needle from the skin entry point to the target location is calculated using the Dijkstra path planning algorithm with multi-objective optimization. The positional data of the puncture needle in three-dimensional space is acquired, and the optimal puncture path and the positional data are superimposed on the display image for anesthesia puncture assisted positioning. The puncture path is adjusted according to real-time data during the puncture process until the needle tip reaches the target position.

[0005] Furthermore, in the above-mentioned anesthesia puncture-assisted localization method based on ultrasound image recognition, the step of acquiring the ultrasound video stream image of the target location in real time, and performing adaptive filtering, enhancement, and normalization processing on the ultrasound video stream image to obtain an initial ultrasound image includes: The ultrasound video stream image of the target location is acquired in real time. Gaussian noise in the ultrasound video stream image is smoothed by Gaussian filtering, and then salt-and-pepper noise is removed by median filtering, while retaining the edge information of nerve and blood vessel structures, to obtain a denoised ultrasound image. The contrast-limited CLAHE adaptive histogram equalization algorithm is used to divide the denoised ultrasound image, and local gray-level stretching is performed on the low-echo region. By calculating the mean and variance of the sub-blocks, the gray-level values ​​are mapped to the standard range of 0-255 to obtain the enhanced ultrasound image. The enhanced ultrasound images are standardized in grayscale. The mean μ and standard deviation σ of the grayscale of all preprocessed images are calculated. The image grayscale is normalized to a distribution with a mean of 0 and a standard deviation of 1 to obtain the initial ultrasound image.

[0006] Furthermore, in the aforementioned anesthesia puncture-assisted localization method based on ultrasound image recognition, the improved DeepLabv3+ deep learning segmentation network model, with a ConvLSTM time series module introduced into the network model, performs real-time semantic segmentation on the initial ultrasound image to obtain a segmented ultrasound image, including: Based on the DeepLabv3+ framework, the backbone network uses ResNet-50, and dilated convolutions are added to the conv2 to conv5 layers to expand the receptive field while maintaining the feature map resolution. On the decoding side, the fusion ratio of low-level feature maps and high-level semantic feature maps is adjusted to 1:3 to enhance the segmentation accuracy of small targets. A ConvLSTM module is inserted after the conv5 layer of the backbone network to receive the initial ultrasound image features of 3 consecutive frames as input. The inter-frame temporal correlation is captured through a gating mechanism to alleviate the segmentation error caused by motion blur or artifacts in a single frame image. The output feature map of ConvLSTM is concatenated with the output of the ASPP module of DeepLabv3+, and then input into the decoder for feature fusion.

[0007] Furthermore, in the aforementioned anesthesia puncture-assisted localization method based on ultrasound image recognition, the improved DeepLabv3+ deep learning segmentation network model, with a ConvLSTM time series module introduced into the network model, performs real-time semantic segmentation on the initial ultrasound image to obtain a segmented ultrasound image, including: The preprocessed initial ultrasound image is input frame by frame into the trained DeepLabv3+ deep learning segmentation network model. TensorRT is used to accelerate the model. The output segmentation results are post-processed. Small holes are filled by morphological closing operation, and pseudo-segmentation regions with an area of ​​<50 pixels are removed by connected component analysis to obtain the segmented ultrasound image. The segmented ultrasound images include at least the nerve plexus, adjacent blood vessels, muscle fascia layer, bone surface echoes, and puncture risk areas.

[0008] Furthermore, in the above-mentioned anesthesia puncture-assisted localization method based on ultrasound image recognition, the step of calculating the optimal puncture path from the skin entry point to the target location using the segmented ultrasound image and the multi-objective optimization Dijkstra path planning algorithm includes: The two-dimensional segmented ultrasound images are converted into three-dimensional spatial coordinates by using the calibration parameters of the ultrasound probe; the two-dimensional contours of the nerve plexus and skeletal structures are reconstructed in three dimensions to obtain the position range of the ultrasound probe in three-dimensional space. Based on the segmentation results, the three-dimensional space is rasterized, and a weight is assigned to each raster: the target location has a weight of 0, the safe area has a weight of 1, and the dangerous area has a weight of 100, thus performing weight modeling of the puncture environment.

[0009] Furthermore, in the above-mentioned anesthesia puncture-assisted localization method based on ultrasound image recognition, the step of calculating the optimal puncture path from the skin entry point to the target location using the segmented ultrasound image and the multi-objective optimization Dijkstra path planning algorithm includes: The optimization objectives include ensuring the path does not cross dangerous areas, minimizing path length, and maintaining a reasonable puncture angle. Based on Dijkstra's algorithm, a multi-objective cost function is introduced, using all grids within the operating area on the skin surface as candidate starting points. Using the center grid of the target location as the endpoint, calculate the cost of all feasible paths and select the path with the lowest cost as the optimal puncture path.

[0010] Furthermore, in the above-mentioned anesthesia puncture-assisted positioning method based on ultrasound image recognition, the step of acquiring the pose data of the puncture needle in three-dimensional space, superimposing the optimal puncture path and the pose data on the display image for anesthesia puncture-assisted positioning, and adjusting the puncture path according to real-time data during the puncture process until the needle tip reaches the target position includes: The system compares the actual position of the puncture needle tip with the optimal path in real time. When the deviation is ≤2mm, the original path is maintained. When the deviation is >2mm and ≤5mm, the system prompts the doctor to fine-tune the puncture direction through the system interface. When the deviation is >5mm or the target position shown in the ultrasound image changes, the system automatically triggers a replanning process to generate a new optimal path and update the display.

[0011] Furthermore, in the anesthesia puncture-assisted positioning system based on ultrasound image recognition, the system includes the following modules: The ultrasound image acquisition module is used to acquire ultrasound video stream images of the target location in real time, and perform adaptive filtering, enhancement and normalization processing on the ultrasound video stream images to obtain an initial ultrasound image; An ultrasound image segmentation module is used to perform real-time semantic segmentation on the initial ultrasound image based on an improved DeepLabv3+ deep learning segmentation network model, and introduces a ConvLSTM time series module into the network model to obtain a segmented ultrasound image. The puncture path calculation module is used to calculate the optimal puncture path of the puncture needle from the skin entry point to the target position based on the segmented ultrasound image using the multi-objective optimized Dijkstra path planning algorithm. The puncture-assisted positioning module is used to acquire the position and pose data of the puncture needle in three-dimensional space, overlay the optimal puncture path and the position and pose data on the display image for anesthesia puncture-assisted positioning, and adjust the puncture path according to real-time data during the puncture process until the needle tip reaches the target position.

[0012] Furthermore, in the anesthesia puncture-assisted positioning system based on ultrasound image recognition, the puncture path calculation module includes the following sub-modules: A submodule is introduced to optimize objectives including that the path does not pass through dangerous areas, the path length is minimized, and the puncture angle is reasonable. A multi-objective cost function is introduced on the basis of Dijkstra's algorithm, with all grids in the operation area on the skin surface as candidate starting points. The calculation submodule is used to calculate the cost of all feasible paths with the center grid of the target location as the endpoint, and select the path with the minimum cost as the optimal puncture path.

[0013] Furthermore, in the anesthesia puncture-assisted positioning system based on ultrasound image recognition, the puncture-assisted positioning module includes the following sub-modules: The auxiliary submodule is used to compare the actual position of the puncture needle tip with the optimal path in real time. When the deviation is ≤2mm, the original path is maintained; when the deviation is >2mm and ≤5mm, the system interface prompts the doctor to fine-tune the puncture direction; when the deviation is >5mm or the target position shown in the ultrasound image changes, the replanning process is automatically triggered to generate a new optimal path and update the display.

[0014] Its beneficial effects lie in the effective reduction of noise interference through the combination of adaptive filtering and the CLAHE enhancement algorithm, the elimination of differences in acquisition conditions through standardized processing, and the provision of high-quality data support for subsequent segmentation, resulting in improved efficiency compared to traditional manual processing. The improved DeepLabv3+ network introduces a ConvLSTM module combined with TensorRT acceleration technology, achieving real-time segmentation at over 25fps, with a Dice coefficient ≥0.85 for key structures such as neural plexuses and blood vessels, representing a 15% improvement in segmentation accuracy compared to a single model. The multi-objective optimization Dijkstra algorithm, through weight modeling and multi-objective cost function design, plans a path that avoids dangerous areas while meeting operational angle requirements, with a path planning error ≤1mm, reducing the risk of complications such as vascular injury. The precise registration of the optical tracking system with ultrasound images and the dynamic path adjustment mechanism enable real-time synchronization and deviation correction of the puncture needle pose, improving the accuracy and efficiency of the needle tip reaching the target area. Attached Figure Description

[0015] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0016] Figure 1 This is a schematic diagram of the first embodiment of the anesthesia puncture-assisted positioning method based on ultrasound image recognition in this invention. Figure 2 This is a schematic diagram of the second embodiment of the anesthesia puncture-assisted positioning method based on ultrasound image recognition in this invention. Figure 3 This is a schematic diagram of the first embodiment of the anesthesia puncture-assisted positioning system based on ultrasound image recognition in this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0018] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1As shown, the anesthesia puncture-assisted localization method based on ultrasound image recognition includes the following steps: Step 101: Acquire ultrasound video stream images of the target location in real time, and perform adaptive filtering, enhancement and normalization processing on the ultrasound video stream images to obtain the initial ultrasound image; Specifically, in this embodiment, ultrasound video stream images of the target location are acquired in real time. Gaussian noise in the ultrasound video stream image is smoothed by Gaussian filtering, and salt-and-pepper noise is removed by median filtering, while retaining the edge information of nerve and blood vessel structures to obtain a denoised ultrasound image. The contrast-limited CLAHE adaptive histogram equalization algorithm is used to divide the denoised ultrasound image, and local gray-level stretching is performed on the low-echo regions. By calculating the mean and variance of the sub-blocks, the gray-level values ​​are mapped to the standard range of 0-255 to obtain an enhanced ultrasound image. The enhanced ultrasound image is then gray-level normalized by calculating the mean μ and standard deviation σ of the gray-level values ​​of all preprocessed images, and the image gray-level is normalized to a distribution with a mean of 0 and a standard deviation of 1 to obtain the initial ultrasound image.

[0020] For equipment selection and connection, a high-frequency linear array ultrasound probe with a frequency range of 5-12MHz is selected to be suitable for common anesthesia puncture scenarios such as peripheral nerve blocks, or a convex array probe with a frequency of 2-5MHz is selected to be suitable for deep nerve plexus puncture. A data transmission link is established with the image processing host through the HDMI or USB 3.0 interface of the ultrasound equipment to ensure a transmission bandwidth of ≥100Mbps to meet real-time requirements.

[0021] The acquisition parameters are configured by setting the probe focal length according to the depth of the target area: 3-5cm for shallow areas and 8-12cm for deep areas. The gain is adjusted to 40-60dB and the dynamic range to 60-80dB. The frame rate is fixed at 30fps to balance real-time performance and image quality. The acquisition timestamp is calibrated through the host software to ensure accurate synchronization of the time information of each frame.

[0022] The system uses target area localization guidance, combined with clinical anatomical landmarks such as the intermuscular groove and axillary crease, to guide the doctor to move the probe to the target nerve plexus area. The host displays a real-time ultrasound image preview. When the target area features, such as the honeycomb echo of the nerve plexus, appear in the image, the automatic acquisition mechanism is triggered to start continuously acquiring video streams.

[0023] Adaptive filtering denoising employs a combined Gaussian filtering and median filtering algorithm. First, Gaussian filtering is used, with the standard deviation σ adaptively adjusted based on the intra-frame noise intensity, ranging from 0.5 to 2.0, to smooth Gaussian noise in the ultrasound image. Then, median filtering is applied, with a window size of 3×3 or 5×5, adaptively selected based on the image resolution to remove salt-and-pepper noise while preserving edge information of key structures such as nerves and blood vessels. During the filtering process, the intra-frame signal-to-noise ratio (SNR) is calculated in real-time, and the filtering intensity is automatically increased when SNR < 20dB.

[0024] Image enhancement processing addresses the issue of uneven grayscale distribution in ultrasound images by employing contrast-limited adaptive histogram equalization using the CLAHE algorithm. This divides the image into 8×8 sub-blocks, with the contrast gain of each sub-block limited to between 2.0 and 4.0 to avoid excessive noise enhancement. Simultaneously, local grayscale stretching is performed on low-echo regions such as neural plexuses. By calculating the mean and variance of the sub-blocks, the grayscale values ​​are mapped to a standard range of 0-255, highlighting the difference between the target structure and the background.

[0025] Standardization and unification are achieved by first standardizing the image size, scaling all frames to 512×512 pixels to balance processing speed and detail preservation, and using a bilinear interpolation algorithm to avoid scaling distortion. Then, grayscale standardization is performed by calculating the mean μ and standard deviation σ of the grayscale values ​​of all preprocessed images. The image grayscale values ​​are then normalized to a distribution with a mean of 0 and a standard deviation of 1 using the formula f'(x,y)=(f(x,y)-μ) / σ, eliminating the influence of grayscale shifts under different acquisition conditions and obtaining the standardized initial ultrasound image.

[0026] Step 102: Based on the improved DeepLabv3+ deep learning segmentation network model, and by introducing the ConvLSTM time series module into the network model, the initial ultrasound image is semantically segmented in real time to obtain the segmented ultrasound image. Specifically, this embodiment uses DeepLabv3+ as the basic framework, with ResNet-50 as the backbone network. Dilated convolutions are added to layers conv2 to conv5 to expand the receptive field while maintaining the feature map resolution. At the decoding end, the fusion ratio of low-level feature maps to high-level semantic feature maps is adjusted to 1:3 to enhance the segmentation accuracy of small targets. A ConvLSTM module is inserted after the conv5 layer of the backbone network to receive the initial ultrasound image features of three consecutive frames as input. The gating mechanism captures the inter-frame temporal correlation to alleviate the segmentation error caused by motion blur or artifacts in a single frame. The output feature map of ConvLSTM is concatenated with the output of the ASPP module of DeepLabv3+ and then input into the decoding end for feature fusion. The preprocessed initial ultrasound images are input frame by frame into the trained DeepLabv3+ deep learning segmentation network model. TensorRT is used to accelerate the model. The output segmentation results are post-processed by filling small holes through morphological closing operations and removing pseudo-segmentation regions with an area of ​​<50 pixels through connected component analysis to obtain segmented ultrasound images. The segmented ultrasound images include at least nerve plexuses, adjacent blood vessels, muscle fascia layers, bone surface echoes, and puncture hazard areas.

[0027] During the construction phase of the improved segmentation network, the basic network selection underwent multiple rounds of comparative experiments. ResNet-50, compared to the original ResNet-101, maintained over 90% feature extraction capability while reducing the number of parameters by 40%, effectively improving the model's inference speed. To address the detail loss problem caused by the decreased resolution of feature maps in deep networks, dilated convolutions with dilation rates of 2, 4, 8, and 16 were added to layers conv2 to conv5, respectively. By expanding the receptive field, the network was able to simultaneously capture feature information from small blood vessels, diameters of 1-2 mm, and large areas of muscle fascia. The feature fusion ratio at the decoding end was adjusted to 1:3. This setting is based on feature analysis of clinical ultrasound images. Low-level feature maps contain rich edge details, such as the boundaries of neural plexuses, while high-level semantic feature maps provide structural category information. The 1:3 fusion ratio improved the model's accuracy in recognizing small targets by 12%. The embedding of the ConvLSTM module is key to solving segmentation errors in dynamic ultrasound images. Its two hidden layers and 256×64×64 hidden state size, verified through extensive experiments, optimally capture the temporal correlation of three consecutive frames. This module, through the synergistic action of input, forget, and output gates, predicts and corrects the position of neural plexuses in motion-blurred frames. When a patient's breathing causes a displacement of ±3mm in the image of the neural plexus, the ConvLSTM module can control the segmentation error within ±0.5mm based on the positional information from the previous two frames. The concatenation operation between the module's output and the ASPP module's output enables the network to process both spatial and temporal features simultaneously, further improving the stability of the segmentation.

[0028] The loss function design employs a combination of cross-entropy loss and Dice loss. Cross-entropy loss optimizes the class probability distribution, while Dice loss focuses on the segmentation accuracy of boundary regions. The weight ratio of 0.6:0.4 was determined through training and validation on 1000 labeled images, effectively balancing the segmentation performance between different classes. For pixel-level weight mapping of key regions such as neural plexuses and blood vessels, the pixel weight for key regions is set to 2.0, and for background regions to 1.0. This addresses the class imbalance problem caused by the excessively high proportion of background pixels in the clinical dataset. Experiments show that this setting improves the Dice coefficient for neural plexus segmentation from 0.78 to over 0.85. The dataset construction for model training follows the principles of diversity and high-quality annotation, collecting 10,000 ultrasound images from five tertiary hospitals, covering different ages (from newborns to those over 80 years old), different body types (BMI 18-35), and different body parts (neck, upper limbs, lower limbs, and trunk puncture scenarios), ensuring the model has good generalization ability. The annotation work was completed collaboratively by two physicians with over 10 years of experience in anesthesia and puncture and one radiologist. The annotation results passed the Kappa consistency test; only those with a Kappa value ≥ 0.85 were included in the dataset. Data augmentation operations included random flipping, rotation (±15°), scaling (0.8-1.2x), and grayscale perturbation (±10%), effectively expanding the dataset size to 50,000 frames and preventing model overfitting. In the training parameter configuration, the initial learning rate of the Adam optimizer was set to 1e-4, decaying to 0.5 every 5 epochs. This strategy enabled rapid convergence in the early stages of training, followed by fine-tuning of parameters later. The batch size was set to 8 to accommodate 16GB of GPU memory, with a total training time of 50 epochs and an early stopping strategy, ensuring sufficient model training while avoiding unnecessary training time. TensorRT acceleration technology in the real-time segmentation inference stage, by converting the model to FP16 precision, increased the inference speed from 15fps to over 25fps, meeting clinical real-time requirements. The morphological closing operation in the post-processing stage effectively fills the small gaps in the nerve plexus segmentation with 3×3 structuring elements, with an area of ​​<30 pixels. The connected region analysis removes pseudo-segmentation regions with an area of ​​<50 pixels. The final output segmented ultrasound image can clearly mark the location and extent of nerve plexuses, blood vessels, muscle fascia, bone surfaces, and puncture danger areas, providing accurate structural information for subsequent path planning.

[0029] Step 103: Based on the segmented ultrasound image, calculate the optimal puncture path of the puncture needle from the skin entry point to the target location using the multi-objective optimization Dijkstra path planning algorithm. Specifically, in this embodiment, the two-dimensional segmented ultrasound image is converted into three-dimensional spatial coordinates through the calibration parameters of the ultrasound probe; the two-dimensional contours of the nerve plexus and skeletal structure are reconstructed in three dimensions to obtain the position range of the ultrasound probe in three-dimensional space; the three-dimensional space is rasterized according to the segmentation results, and a weight is assigned to each grid, with a weight of 0 for the target position, 1 for the safe area, and 100 for the danger area, to perform weight modeling of the puncture environment. The optimization objectives include that the path does not pass through the danger area, the path length is minimized, and the puncture angle is reasonable. A multi-objective cost function is introduced based on Dijkstra's algorithm, using all grids within the operating area on the skin surface as candidate starting points; and using the center grid of the target position as the endpoint, the cost of all feasible paths is calculated, and the path with the minimum cost is selected as the optimal puncture path.

[0030] In the spatial modeling stage of image segmentation, the mapping from 2D images to 3D space is a crucial link connecting image information with physical space, and its accuracy depends on the precise calibration of the ultrasound probe. A checkerboard calibration method was used to obtain the probe's intrinsic parameters (focal length f, principal point coordinates (u0, v0) and extrinsic parameters, as well as the attitude matrix. During calibration, 20 checkerboard images from different angles were acquired, and the calibration parameters were calculated using Zhang Zhengyou's calibration algorithm, keeping the calibration error within ±0.1mm. The established xyz coordinate system has the probe center as the origin, with the x-axis along the probe's major axis, the y-axis perpendicular to the probe surface, and the z-axis along the probe's minor axis. This coordinate system setting closely matches clinical operating habits, facilitating doctors' understanding of the spatial location of the puncture path. When reconstructing the 2D contours of structures such as nerve plexuses and bones into 3D, a contour-based stereo matching algorithm was used. By analyzing the contour correspondences of multiple consecutive frames, a 3D point cloud model of the structure was constructed, keeping the spatial position error of the nerve plexus within ±0.3mm. The regional weighting process employs a 1×1×1mm grid, with the grid size matching the diameter of the puncture needle (0.3-0.8mm), accurately reflecting the positional relationship between the puncture path and surrounding structures. Weight settings are based on clinical risk assessment: in the target area, the weight for nerve plexus targets is set to 0, prioritizing the guiding path towards this area; in the safe area, the weight for muscles and fascia is set to 1, ensuring the path selects tissues with minimal resistance; in the danger area, the weight for blood vessels, within 5mm of the bone surface, and non-target areas of nerve plexuses is set to 100, forming an impassable barrier. This weighting is based on extensive clinical complication data, effectively avoiding serious complications such as vascular and nerve damage. The multi-objective optimized path calculation prioritizes safety while considering efficiency and convenience, clearly defining the priorities of three core optimization objectives: the path not crossing danger areas as a hard constraint to ensure puncture safety; the shortest path length as a secondary objective to reduce puncture operation time; and a reasonable puncture angle (30°-60°) as a tertiary objective to improve the ease of operation for physicians. In the multi-objective cost function C = a×L + b×A, the weight coefficients a=1.0 and b=0.5 were determined through clinical trials. Ten anesthesiologists were invited to score the paths under different weight coefficients, and the weight combination with the highest comprehensive score was selected. The angle penalty term A is set such that when the path angle deviates from the range of 30°-60°, a penalty value of 0.1 is added for every 1° deviation, effectively avoiding the problems of excessive steepness (>60° leading to operational instability) and excessive flatness (<30° leading to excessively long paths). The candidate starting point range is set to a 5×5cm operable area of ​​the skin surface specified by the doctor, which provides the doctor with a certain degree of operational flexibility while avoiding a surge in computational load due to an excessively large starting point range. During the path calculation process, the search efficiency of Dijkstra's algorithm is optimized by using a priority queue, reducing the calculation time from 500ms of the traditional algorithm to less than 100ms, meeting the requirements of real-time planning.The path verification and correction process forms a closed-loop control. The minimum distance between the path and the blood vessel is checked by a distance detection algorithm. When the distance is less than 3mm, a greedy algorithm is used to replan the path segment so that the distance between the corrected path and the blood vessel is greater than or equal to 5mm. The manual adjustment interface allows doctors to fine-tune the needle insertion point within ±5mm and adjust the angle within ±10°. This respects the doctor's clinical experience and ensures that the adjusted path is still within a safe range through parameter constraints.

[0031] Step 104: Obtain the position and pose data of the puncture needle in three-dimensional space, overlay the optimal puncture path and position and pose data on the display image for anesthesia puncture assisted positioning, and adjust the puncture path according to the real-time data during the puncture process until the needle tip reaches the target position.

[0032] Specifically, in this embodiment, the deviation between the actual position of the puncture needle tip and the optimal path is compared in real time. When the deviation is ≤2mm, the original path is maintained; when the deviation is >2mm and ≤5mm, the system interface prompts the doctor to fine-tune the puncture direction; when the deviation is >5mm or the target position shown in the ultrasound image changes, the replanning process is automatically triggered to generate a new optimal path and update the display.

[0033] For the deployment and calibration of the tracking equipment, an optical tracking system, such as OptiTrack Prime 13W, is used. Three reflective markers are attached to the surface of the puncture needle to form a non-collinear triangle with a side length of 2-3 cm. Two reflective markers are attached to the ultrasound probe. The system is calibrated in advance to register the coordinate system of the optical tracking system with the three-dimensional coordinate system of the ultrasound image, with an error of ≤0.5 mm, to ensure the consistency between the pose data and the image coordinates.

[0034] Real-time pose acquisition and synchronization: The optical tracking system acquires the pose data of the puncture needle at a frequency of 100fps, including three-dimensional coordinates (x, y, z) and attitude angles (pitch angle, yaw angle, roll angle), and transmits it to the host via UDP protocol; The host synchronizes the pose data with the ultrasound video stream image through a timestamp matching algorithm, with a synchronization error ≤10ms, to ensure the real-time correspondence between the puncture needle position and the ultrasound image.

[0035] Pose data preprocessing involves smoothing the acquired pose data using a sliding window filter with a window size of 5 frames to remove high-frequency noise. When occlusion occurs in the tracking system, causing data loss, Kalman filtering is used for prediction, with a prediction error ≤1mm and a prediction time ≤200ms to avoid navigation interruption.

[0036] The system displays multiple information overlays, showing three types of information superimposed on the ultrasound video stream: ① Optimal puncture path, displayed as a solid red line with a green dot marking the starting point and a yellow cross marking the ending point; ② Real-time needle position, displayed as a blue line showing the 3D projection of the needle with a red dot marking the needle tip position; ③ Danger zone warning, marked with a semi-transparent red mask. When the needle tip is less than 5mm from the danger zone, an audible and visual alarm is triggered. The display interface supports multi-view switching, including ultrasound cross-section, longitudinal section, and 3D stereoscopic image. The puncture path is dynamically adjusted, comparing the actual position of the needle tip with the optimal path in real time. When the deviation is ≤2mm, the original path is maintained; when the deviation is >2mm and ≤5mm, the system prompts the doctor to fine-tune the puncture direction without replanning the path; when the deviation is >5mm or the target area position changes in the ultrasound image, such as nerve plexus displacement due to patient respiration, the replanning process is automatically triggered, repeating steps one to three to generate a new optimal path and update the display. The system confirms and records the endpoint of the puncture. When the needle tip reaches the target area, the ultrasound image shows that the overlap between the target area and the needle tip is ≥80%, and the system issues a prompt sound indicating that the target has been reached. After the doctor confirms that the position is correct by aspirating blood and injecting a small amount of saline, the puncture is completed. The system automatically records the data of the entire puncture process, including ultrasound video, puncture path parameters, position change curve, and operation time, and saves them to the patient's electronic medical record system for subsequent traceability and analysis.

[0037] Its beneficial effects lie in the effective reduction of noise interference through the combination of adaptive filtering and the CLAHE enhancement algorithm, the elimination of differences in acquisition conditions through standardized processing, and the provision of high-quality data support for subsequent segmentation, resulting in improved efficiency compared to traditional manual processing. The improved DeepLabv3+ network introduces a ConvLSTM module combined with TensorRT acceleration technology, achieving real-time segmentation at over 25fps, with a Dice coefficient ≥0.85 for key structures such as neural plexuses and blood vessels, representing a 15% improvement in segmentation accuracy compared to a single model. The multi-objective optimization Dijkstra algorithm, through weight modeling and multi-objective cost function design, plans a path that avoids dangerous areas while meeting operational angle requirements, with a path planning error ≤1mm, reducing the risk of complications such as vascular injury. The precise registration of the optical tracking system with ultrasound images and the dynamic path adjustment mechanism enable real-time synchronization and deviation correction of the puncture needle pose, improving the accuracy and efficiency of the needle tip reaching the target area.

[0038] Please see Figure 2 In the anesthesia puncture-assisted positioning method based on ultrasound image recognition, the ultrasound video stream image of the target location is acquired in real time, and the ultrasound video stream image is subjected to adaptive filtering, enhancement and standardization to obtain the initial ultrasound image, including the following steps; Step 201: Acquire ultrasound video stream images of the target location in real time, smooth Gaussian noise in the ultrasound video stream images by Gaussian filtering, and then remove salt-and-pepper noise by median filtering, while preserving the edge information of nerve and blood vessel structures to obtain a denoised ultrasound image. Step 202: The contrast-limited CLAHE adaptive histogram equalization algorithm is used to divide the denoised ultrasound image, and local gray-level stretching is performed on the low-echo region. By calculating the mean and variance of the sub-blocks, the gray-level values ​​are mapped to the standard range of 0-255 to obtain the enhanced ultrasound image. Step 203: Standardize the grayscale of the enhanced ultrasound image, calculate the mean μ and standard deviation σ of the grayscale of all preprocessed images, and normalize the image grayscale to a distribution with a mean of 0 and a standard deviation of 1 to obtain the initial ultrasound image.

[0039] The above describes embodiments of the anesthesia puncture-assisted localization method based on ultrasound image recognition of the present invention. Please refer to [link to relevant documentation]. Figure 3 In the anesthesia puncture-assisted positioning system based on ultrasound image recognition, the anesthesia puncture-assisted positioning system based on ultrasound image recognition includes the following modules; The ultrasound image acquisition module is used to acquire ultrasound video stream images of the target location in real time, and perform adaptive filtering, enhancement and normalization processing on the ultrasound video stream images to obtain the initial ultrasound image; The ultrasound image segmentation module is used to perform real-time semantic segmentation of the initial ultrasound image based on the improved DeepLabv3+ deep learning segmentation network model, and introduces the ConvLSTM time series module into the network model to obtain the segmented ultrasound image. The puncture path calculation module is used to calculate the optimal puncture path of the puncture needle from the skin entry point to the target position based on the segmented ultrasound image using the multi-objective optimized Dijkstra path planning algorithm. The puncture-assisted positioning module is used to acquire the positional data of the puncture needle in three-dimensional space, overlay the optimal puncture path and positional data on the display image for anesthesia puncture-assisted positioning, and adjust the puncture path according to real-time data during the puncture process until the needle tip reaches the target position.

[0040] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended technical solutions and their equivalents.

Claims

1. A method for assisted localization of anesthesia puncture based on ultrasound image recognition, characterized in that, The anesthesia puncture-assisted localization method based on ultrasound image recognition includes the following steps: The ultrasound video stream image of the target location is acquired in real time, and the ultrasound video stream image is subjected to adaptive filtering, enhancement and normalization processing to obtain an initial ultrasound image; Based on the improved DeepLabv3+ deep learning segmentation network model, and by introducing a ConvLSTM time series module into the network model, the initial ultrasound image is semantically segmented in real time to obtain a segmented ultrasound image. Based on the segmented ultrasound image, the optimal puncture path of the puncture needle from the skin entry point to the target location is calculated using the Dijkstra path planning algorithm with multi-objective optimization. The positional data of the puncture needle in three-dimensional space is acquired, and the optimal puncture path and the positional data are superimposed on the display image for anesthesia puncture assisted positioning. The puncture path is adjusted according to real-time data during the puncture process until the needle tip reaches the target position.

2. The anesthesia puncture-assisted localization method based on ultrasound image recognition as described in claim 1, characterized in that, The real-time acquisition of ultrasound video stream images of the target location, followed by adaptive filtering, enhancement, and normalization of the ultrasound video stream images to obtain an initial ultrasound image, includes: The ultrasound video stream image of the target location is acquired in real time. Gaussian noise in the ultrasound video stream image is smoothed by Gaussian filtering, and then salt-and-pepper noise is removed by median filtering, while retaining the edge information of nerve and blood vessel structures, to obtain a denoised ultrasound image. The contrast-limited CLAHE adaptive histogram equalization algorithm is used to divide the denoised ultrasound image, and local gray-level stretching is performed on the low-echo region. By calculating the mean and variance of the sub-blocks, the gray-level values ​​are mapped to the standard range of 0-255 to obtain the enhanced ultrasound image. The enhanced ultrasound images are standardized in grayscale. The mean μ and standard deviation σ of the grayscale of all preprocessed images are calculated. The image grayscale is normalized to a distribution with a mean of 0 and a standard deviation of 1 to obtain the initial ultrasound image.

3. The anesthesia puncture-assisted positioning method based on ultrasound image recognition as described in claim 1, characterized in that, The improved DeepLabv3+ deep learning segmentation network model, which incorporates a ConvLSTM time series module, performs real-time semantic segmentation on the initial ultrasound image to obtain a segmented ultrasound image, including: Based on the DeepLabv3+ framework, the backbone network uses ResNet-50, and dilated convolutions are added to the conv2 to conv5 layers to expand the receptive field while maintaining the feature map resolution. On the decoding side, the fusion ratio of low-level feature maps and high-level semantic feature maps is adjusted to 1:3 to enhance the segmentation accuracy of small targets. A ConvLSTM module is inserted after the conv5 layer of the backbone network to receive the initial ultrasound image features of 3 consecutive frames as input. The inter-frame temporal correlation is captured through a gating mechanism to alleviate the segmentation error caused by motion blur or artifacts in a single frame image. The output feature map of ConvLSTM is concatenated with the output of the ASPP module of DeepLabv3+, and then input into the decoder for feature fusion.

4. The anesthesia puncture-assisted localization method based on ultrasound image recognition as described in claim 1, characterized in that, The improved DeepLabv3+ deep learning segmentation network model, which incorporates a ConvLSTM time series module, performs real-time semantic segmentation on the initial ultrasound image to obtain a segmented ultrasound image, including: The preprocessed initial ultrasound image is input frame by frame into the trained DeepLabv3+ deep learning segmentation network model. TensorRT is used to accelerate the model. The output segmentation results are post-processed. Small holes are filled by morphological closing operation, and pseudo-segmentation regions with an area of ​​<50 pixels are removed by connected component analysis to obtain the segmented ultrasound image. The segmented ultrasound images include at least the nerve plexus, adjacent blood vessels, muscle fascia layer, bone surface echoes, and puncture risk areas.

5. The anesthesia puncture-assisted positioning method based on ultrasound image recognition as described in claim 1, characterized in that, The step of calculating the optimal puncture path from the skin entry point to the target location using the segmented ultrasound image and the Dijkstra path planning algorithm with multi-objective optimization includes: The two-dimensional segmented ultrasound images are converted into three-dimensional spatial coordinates by using the calibration parameters of the ultrasound probe; the two-dimensional contours of the nerve plexus and skeletal structures are reconstructed in three dimensions to obtain the position range of the ultrasound probe in three-dimensional space. Based on the segmentation results, the three-dimensional space is rasterized, and a weight is assigned to each raster: the target location has a weight of 0, the safe area has a weight of 1, and the dangerous area has a weight of 100, thus performing weight modeling of the puncture environment.

6. The anesthesia puncture-assisted localization method based on ultrasound image recognition as described in claim 1, characterized in that, The step of calculating the optimal puncture path from the skin entry point to the target location using the segmented ultrasound image and the Dijkstra path planning algorithm with multi-objective optimization includes: The optimization objectives include ensuring the path does not cross dangerous areas, minimizing path length, and maintaining a reasonable puncture angle. Based on Dijkstra's algorithm, a multi-objective cost function is introduced, using all grids within the operating area on the skin surface as candidate starting points. Using the center grid of the target location as the endpoint, calculate the cost of all feasible paths and select the path with the lowest cost as the optimal puncture path.

7. The anesthesia puncture-assisted positioning method based on ultrasound image recognition as described in claim 1, characterized in that, The process of acquiring the pose data of the puncture needle in three-dimensional space, overlaying the optimal puncture path and the pose data onto the display image for anesthesia puncture-assisted positioning, and adjusting the puncture path based on real-time data during the puncture process until the needle tip reaches the target position includes: The system compares the actual position of the puncture needle tip with the optimal path in real time. When the deviation is ≤2mm, the original path is maintained. When the deviation is >2mm and ≤5mm, the system prompts the doctor to fine-tune the puncture direction through the system interface. When the deviation is >5mm or the target position shown in the ultrasound image changes, the system automatically triggers a replanning process to generate a new optimal path and update the display.

8. An anesthesia puncture-assisted positioning system based on ultrasound image recognition, characterized in that, The anesthesia puncture-assisted positioning system based on ultrasound image recognition includes the following modules: The ultrasound image acquisition module is used to acquire ultrasound video stream images of the target location in real time, and perform adaptive filtering, enhancement and normalization processing on the ultrasound video stream images to obtain an initial ultrasound image; An ultrasound image segmentation module is used to perform real-time semantic segmentation on the initial ultrasound image based on an improved DeepLabv3+ deep learning segmentation network model, and introduces a ConvLSTM time series module into the network model to obtain a segmented ultrasound image. The puncture path calculation module is used to calculate the optimal puncture path of the puncture needle from the skin entry point to the target position based on the segmented ultrasound image using the multi-objective optimized Dijkstra path planning algorithm. The puncture-assisted positioning module is used to acquire the position and pose data of the puncture needle in three-dimensional space, overlay the optimal puncture path and the position and pose data on the display image for anesthesia puncture-assisted positioning, and adjust the puncture path according to real-time data during the puncture process until the needle tip reaches the target position.

9. The anesthesia puncture-assisted positioning system based on ultrasound image recognition as described in claim 8, characterized in that, The puncture path calculation module includes the following sub-modules: A submodule is introduced to optimize objectives including that the path does not pass through dangerous areas, the path length is minimized, and the puncture angle is reasonable. A multi-objective cost function is introduced on the basis of Dijkstra's algorithm, with all grids in the operation area on the skin surface as candidate starting points. The calculation submodule is used to calculate the cost of all feasible paths with the center grid of the target location as the endpoint, and select the path with the minimum cost as the optimal puncture path.

10. The anesthesia puncture-assisted positioning system based on ultrasound image recognition as described in claim 8, characterized in that, The puncture-assisted positioning module includes the following sub-modules: The auxiliary submodule is used to compare the actual position of the puncture needle tip with the optimal path in real time. When the deviation is ≤2mm, the original path is maintained; when the deviation is >2mm and ≤5mm, the system interface prompts the doctor to fine-tune the puncture direction; when the deviation is >5mm or the target position shown in the ultrasound image changes, the replanning process is automatically triggered to generate a new optimal path and update the display.

Images