Anesthesia puncture auxiliary positioning method

Abstract

The invention discloses an anesthesia puncture auxiliary positioning method, and relates to the technical field of medical image processing, and the method comprises the steps: obtaining multi-modal image data of an area to be anesthetized, and carrying out the image synchronization and fusion of the multi-modal image data; uniformly dividing the fused multi-modal comprehensive image into a plurality of image blocks according to a preset number, and calculating the gray dynamic change degree of each image block; based on the gray dynamic change degree of each image block, performing noise removal and feature enhancement processing on each image block; generating an accurate three-dimensional model of a spinal canal area through a volume rendering method by utilizing the processed multi-modal image data, and displaying the three-dimensional model in real time in an overlapping manner through augmented reality equipment; and marking an important anatomical structure on the generated three-dimensional model, and calculating an optimal puncture path avoiding a key structure in a three-dimensional space by adopting an A * algorithm. Through multi-modal image fusion, three-dimensional reconstruction and path planning, the precision and operability of anesthesia puncture auxiliary positioning are improved.

Description

Technical Field

[0001] The invention relates to the technical field of medical image processing, in particular to an anesthesia puncture auxiliary positioning method. Background Art

[0002] Anesthesia puncture technology is an indispensable and important part of modern medicine. Its accuracy and safety are directly related to the success rate of the operation and the patient's recovery. Traditional anesthesia puncture positioning methods mainly rely on the doctor's experience and single-modality image guidance, such as ultrasound or X-ray guidance. However, these methods have certain limitations in the identification and precise positioning of complex anatomical structures. In recent years, multimodal image fusion technology can present anatomical structure information more comprehensively and improve the accuracy and safety of puncture by integrating multiple imaging data such as ultrasound, magnetic resonance imaging, and CT. However, there are still some problems that need to be solved.

[0003] First, the synchronization and fusion of multimodal imaging data still face technical challenges, and the accuracy of spatiotemporal registration between different modalities needs to be improved. Secondly, the existing image processing methods are not ideal in terms of noise removal and feature enhancement, and it is difficult to fully extract and utilize the key information in multimodal images. Furthermore, the application of three-dimensional visualization technology in anesthesia puncture auxiliary positioning is not mature enough, and it is difficult to achieve accurate three-dimensional reconstruction and real-time display of anatomical structures. Finally, the existing puncture path planning algorithms often only consider the two-dimensional plane, lack of comprehensive consideration of complex anatomical structures in three-dimensional space, resulting in less than optimized puncture path selection. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an anesthesia puncture assisted positioning method to solve the accuracy and safety problems of multimodal medical image fusion, three-dimensional visualization reconstruction and real-time display, and intelligent puncture path planning in anesthesia puncture assisted positioning.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In the first aspect, an embodiment of the present invention provides an anesthesia puncture auxiliary positioning method, which includes obtaining multimodal image data of the area to be anesthetized, synchronizing and fusing the multimodal image data, and generating a multimodal comprehensive image; evenly dividing the fused multimodal comprehensive image into a plurality of image blocks according to a preset number, and calculating the degree of dynamic change of grayscale of each image block; based on the degree of dynamic change of grayscale of each image block, performing noise removal and feature enhancement processing on each image block; using the processed multimodal image data, generating an accurate three-dimensional model of the spinal canal area by a volume rendering method, and superimposing and displaying the three-dimensional model in real time by an augmented reality device; marking important anatomical structures on the generated three-dimensional model, and using the A* algorithm to calculate the optimal puncture path avoiding key structures in three-dimensional space.

[0007] As a preferred solution of the anesthesia puncture auxiliary positioning method of the present invention, wherein: the acquisition of multimodal image data of the area to be anesthetized includes the following steps: Ultrasonic images, near-infrared light images and elastic imaging data of the target puncture area on the patient's waist are collected through ultrasonic probes, near-infrared light sensors and elastic imaging equipment.

[0008] As a preferred solution of the anesthesia puncture auxiliary positioning method of the present invention, wherein: performing image synchronization and fusion on multimodal image data to generate a multimodal comprehensive image includes the following steps: Use a unified time base to generate timestamps, automatically embedding accurate timestamps when acquiring image data for each modality; Use image processing software to extract timestamp information from the metadata of each modality image; Calculate the time difference between different modality images, identify and record fixed delays, and adjust the time series of each modality image through interpolation and cropping methods; SIFT algorithm is applied to ultrasound images, NIR images and elastic imaging images to detect key feature points, and the coordinates and descriptors of each feature point are recorded; Initialize the FLANN matcher, set the index parameters and search parameters, perform feature point matching on NIR-ultrasound and elastic imaging-ultrasound, generate potential matching pairs, and use ratio test to filter out false matches; Extract the coordinate set of the matching point pairs, calculate the affine transformation matrices of NIR-ultrasound and elastic imaging-ultrasound respectively, use the calculated affine transformation matrices to perform spatial transformation on the NIR image and the elastic imaging image, and generate the NIR image and the elastic imaging image that are spatially aligned with the ultrasound image; The aligned ultrasound images, NIR images and elastic imaging data are normalized to unify the grayscale range. The normalized multimodal image data are input into the pre-trained fusion U-Net model for forward propagation to generate a fused multimodal comprehensive image.

[0009] As a preferred solution of the anesthesia puncture auxiliary positioning method of the present invention, the fused multimodal comprehensive image is evenly divided into a number of image blocks according to a preset number, and the degree of dynamic change of the grayscale of each image block is calculated, including the following steps: Set the number of image blocks and image block size, use a fixed grid method, start from the upper left corner of the image, divide it row by row, and verify whether each image block covers different anatomical areas; Before the puncture operation begins, at least two ultrasound images are acquired as the initial image set, and the baseline grayscale distribution is recorded; During the actual puncture process, ultrasound images are collected in real time at fixed time intervals, and new images are added to the image group in sequence to form a time-series image data set; For each image block, the grayscale distribution sequence is extracted from the time series image dataset, and the mean grayscale sequence in the baseline phase is calculated; The DTW algorithm is implemented by using the dtaidistance library in Python to calculate the DTW distance between the grayscale distribution sequence and the mean sequence of each image block at each moment; The grayscale fluctuation index and DTW distance are combined to calculate the dynamic change degree of grayscale of the image block.

[0010] As a preferred solution of the anesthesia puncture auxiliary positioning method of the present invention, the noise removal and feature enhancement are performed on each image block based on the degree of dynamic grayscale change of each image block, including the following steps: For each pair of image blocks, calculate the difference in the degree of dynamic change of their grayscale; Calculate the similarity weights between image blocks and construct a similarity weight matrix between image blocks; Define the size of the neighborhood range, locate the image block where each pixel is located, and select the image block with high similarity in the similarity weight matrix; The weights of all neighboring image blocks are accumulated to obtain the normalization factor, and the denoised pixel value is calculated to complete the denoising process of the entire image, thus obtaining the filtered denoised multimodal image. The denoised multimodal image is normalized and input into the pre-trained fusion U-Net model for forward propagation to generate an enhanced multimodal image.

[0011] As a preferred solution of the anesthesia puncture auxiliary positioning method of the present invention, wherein: using the denoised multimodal image data, an accurate three-dimensional model of the spinal canal area is generated by a volume rendering method, and the three-dimensional model is superimposed and displayed in real time by an augmented reality device, including the following steps: The three-dimensional resolution of voxels is set according to the depth of the puncture area and the organ structure, and the blank areas between the two-dimensional images are filled using the linear interpolation method; Input the voxel data into the Marching Cubes algorithm, set a threshold, and determine whether the vertex of each small cube in the voxel data is above and below the threshold; According to the state of the cube, the table is looked up to determine the triangle combination that needs to be generated, and the precise position of the polygon vertices is calculated through linear interpolation to form a grid structure; Use the Quadratic Decimation method to optimize the mesh structure by reducing the number of polygons, and apply the Laplacian smoothing algorithm to reduce mesh noise and smooth surface curves; Microsoft HoloLens 2 was selected as the AR device, using optical tracking technology, with multiple LED markers installed on the patient's waist and the augmented reality device, and a high-frame rate camera configured for real-time marker recognition and position calculation; Perform initial calibration of the optical tracking system to correct the spatial relationship between the AR device and the actual anatomical structure; Import the optimized 3D surface mesh model into the visualization software of the AR device, and adjust the position and orientation of the 3D model according to the calibrated spatial transformation matrix; Continuously receive data from the optical tracking system and adjust the position and orientation of the 3D model in real time.

[0012] As a preferred solution of the anesthesia puncture auxiliary positioning method of the present invention, in which: on the generated three-dimensional model, important anatomical structures are marked, and the A* algorithm is used to calculate the optimal puncture path avoiding key structures in three-dimensional space, including the following steps: Use image segmentation algorithms to process the fused multimodal image data, automatically identify and segment key anatomical structures, and manually correct the automatic segmentation results; Discretize the three-dimensional space of the puncture area into grid nodes, each node represents a point in the space; The 26-neighborhood connection method is used to define the connection relationship between nodes; Choose Euclidean distance as the heuristic function to estimate the distance from the current node The straight-line distance to the target node; Combine the actual cost and the heuristic cost to calculate the total cost of each node in the A* algorithm; Add the starting node to the open list and select the node with the smallest total cost from the open list , remove it from the open list and add it to the closed list; Traversing nodes All neighbor nodes of ; If the node Already in the closed list, skip; if the node If it is not in the open list, add it to the open list and record its parent node as ; If the node If it is already in the open list but the new actual cost is smaller, update its total cost and parent node; Repeat the steps until the target node is selected from the open list and the path search is completed; if the open list is empty, no valid path can be found; Starting from the target node, backtrack along the parent node step by step until the starting point node is reached to form the entire path. The node sequence obtained by backtracking is reversed to obtain a complete puncture path from the starting point to the target point.

[0013] In a second aspect, an embodiment of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the anesthesia puncture assisted positioning method as described in the first aspect of the present invention is implemented.

[0014] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the anesthesia puncture assisted positioning method as described in the first aspect of the present invention is implemented.

[0015] The beneficial effects of the present invention are as follows: by acquiring multimodal image data of the area to be anesthetized, synchronizing and fusing them, a multimodal comprehensive image is generated, thereby improving the comprehensiveness and accuracy of the image data; the fused multimodal comprehensive image is evenly divided into a number of image blocks, and the degree of dynamic change of the grayscale of each image block is calculated, thereby effectively capturing the characteristic changes of different anatomical regions; targeted noise removal and feature enhancement processing are performed on each image block, thereby effectively retaining key anatomical structure information and enhancing the readability and interpretability of the image; an accurate three-dimensional model is generated by a volume rendering method, and real-time display is combined with augmented reality technology, thereby realizing the intuitive presentation of complex anatomical structures, marking important anatomical structures on the three-dimensional model, and using the A* algorithm to calculate the optimal puncture path, thereby improving the accuracy and safety of anesthesia puncture. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 This is a flow chart of the anesthesia puncture assisted positioning method in Example 1.

[0018] Figure 2 Flow chart of the A* algorithm in Example 1. DETAILED DESCRIPTION

[0019] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0020] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0021] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0022] Example 1, reference Figure 1 and Figure 2 , which is the first embodiment of the present invention, provides an anesthesia puncture auxiliary positioning method, comprising the following steps: S1, obtaining multimodal imaging data of the area to be anesthetized; Specifically, a high-resolution real-time ultrasound probe of model XYZ-5000 was selected, and the operating frequency was set to 5 MHz to ensure sufficient penetration depth (about 5 cm) and high spatial resolution (less than 0.5 mm). It has real-time imaging capability and high frame rate (≥60 frames / second), supports multi-plane imaging, and is convenient for real-time observation of the dynamic position of the puncture needle; the NIR-800 near-infrared light reflection sensor was selected, with an operating wavelength range of 700-900 nm, high spectral resolution and low noise characteristics, which can enhance the visibility of subcutaneous blood vessels and tissues, and provide rich vascular structure information in combination with ultrasound images; the ElasticPro-300 elastic imaging module was used, with a compression force range set between 20-50 grams and a resolution of up to 0.2 mm, which is suitable for high-precision acquisition of soft tissue elastic characteristics, can display tissue elasticity distribution in real time, and assist in identifying changes in tissue hardness in the puncture area; Ensure that all modules (ultrasound probe, near-infrared light sensor and elastic imaging module) are interconnected through USB 3.0 interfaces to ensure high-speed data transmission, and use a unified control software platform (such as MedFusion Control Suite) to achieve unified control and synchronous operation of various modality devices; Integrate various modality devices through a main controller (such as a high-performance computer or embedded system), set a unified time base, ensure the time synchronization of image data of different modalities, and use hardware trigger signals (such as TTL signals) to synchronize the image acquisition of various devices to avoid data misalignment caused by time deviation; Use calibration templates with known geometric shapes and physical properties (such as standard geometric bodies and elastic phantoms) to perform spatial calibration of the integrated device to ensure the data alignment accuracy of each modality image, and apply registration algorithms (such as rigid transformation based on landmark matching) to calculate and optimize calibration parameters, with the error controlled within ±0.5 mm; synchronize the internal clock of the device and align it with the time base of the main controller to ensure accurate time synchronization of multi-modal image data, verify the time consistency of each modality image data through timestamp verification, and ensure the accuracy of synchronous acquisition; The L4-L5 intervertebral space in the patient's waist was selected as the target puncture area. The anatomical structure of this area is clear and the movement path of the puncture needle is relatively safe. The ultrasound guidance method is used to preliminarily confirm the accurate location of the puncture point to avoid critical locations of blood vessels and nerves. Use a dedicated fixing device (such as a fixing plate and a fixing clip) to firmly fix the integrated device at the predetermined position on the patient's waist. Adjust the position and angle of the fixing device to ensure that the field of view of the ultrasound probe, NIR sensor, and elastic imaging module covers the target puncture area to prevent the device from shifting during the puncture process. Check the stability of the device fixation and verify the reliability of the fixing device by slightly moving the patient's position to ensure the stability of the device during the entire puncture process. Start all modules of the integrated device, including the ultrasonic probe, near-infrared light sensor and elastic imaging module, confirm that each module is in good working condition, and check the real-time display of the image data through the control software; select the "synchronous acquisition" mode in the control software to ensure that the image data of different modalities are synchronously acquired according to a unified time reference, start the acquisition, record the timestamp of each image, and ensure the accurate synchronization of the data in time and space.

[0023] S2, synchronizing and fusing the multimodal image data to generate a multimodal comprehensive image, including the following steps: S2.1. Use a unified time base to generate timestamps to ensure that the image data of each modality in the multimodal device configuration is automatically embedded with accurate timestamps when it is acquired; use image processing software (such as OpenCV or MATLAB) to read the metadata fields of the image files from each modality image, extract the timestamps, and convert the timestamps into a comparable time series format for subsequent synchronization processing, using the YYYY-MM-DD HH:MM:SS:MS format to ensure millisecond-level accuracy to meet real-time synchronization requirements; Calculate the time difference between images of different modalities, identify and record fixed delays (such as trigger signal delays), and adjust the time series of images of each modality through interpolation and cropping methods to ensure that the image data are strictly aligned in time; use visualization tools (such as Matplotlib or VTK) to overlay and display images of different modalities at the same time point, check the alignment effect of images of different modalities in time, and confirm that key anatomical structures and puncture needle positions are consistent in images of each modality.

[0024] S2.2, SIFT (Scale Invariant Feature Transform) algorithm is used for feature point detection, because it is invariant in scale and rotation, and is suitable for complex registration requirements of multimodal images; SIFT algorithm is applied to ultrasound images and NIR images to detect key feature points respectively. The SIFT detector is initialized using the cv2.SIFT_create() function in OpenCV. The detectAndCompute method is called for each modality image to extract feature points and descriptors. The coordinates and descriptors of each feature point are recorded for subsequent matching. The FLANN (Fast Nearest Neighbor Library) matching algorithm is used because of its high efficiency and accuracy in processing a large number of feature points. The FLANN matcher is initialized, the index parameters and search parameters are set, and feature point matching is performed on NIR-ultrasound and elastic imaging-ultrasound, respectively, to generate potential matching pairs, and the ratio test (Lowe's ratio test) is used to filter out false matches and retain reliable feature point pairs. The coordinate sets of matching point pairs are extracted, and the affine transformation matrices of NIR-ultrasound and elastic imaging-ultrasound are calculated respectively. The optimal affine transformation matrix is ​​calculated using the cv2.estimateAffine2D() function of OpenCV. The calculated affine transformation matrix is ​​used to perform spatial transformation on the NIR image and elastic imaging image to align them with the ultrasound image.

[0025] S2.3, using a pre-trained fusion U-Net model, which is based on the classic U-Net architecture and adds multi-modal data input channels, suitable for the fusion of ultrasound, near-infrared light and elastic imaging data; The input layer is a three-channel input, corresponding to ultrasound, NIR and elastic imaging data respectively. Each layer contains convolution, batch normalization (BN), activation function (ReLU) and pooling operations. Upsampling deconvolution and skip connections are used to fuse high-resolution features with low-resolution features to generate a fused high-quality comprehensive image. The sigmoid activation function is used to ensure that the grayscale value of the output image is within the range of 0 and 1. The aligned ultrasound images, NIR images and elastic imaging data are normalized to unify the grayscale range. The normalized multimodal image data are input into the pre-trained fusion U-Net model for forward propagation to generate a fused multimodal comprehensive image.

[0026] S3, evenly dividing the fused multimodal comprehensive image into a number of image blocks according to a preset number, and calculating the degree of dynamic change of the grayscale of each image block, including the following steps: S3.1. Set the number of image blocks to 50, ensuring that each image block covers approximately the same anatomical area. The size of each image block is set to 100×100 pixels, which is adjusted according to the image resolution and clinical needs. Use a fixed grid method, starting from the upper left corner of the image, and divide it row by row to ensure that there is no overlap between each image block and that it completely covers the target puncture area. Verify whether each image block covers different anatomical areas to avoid excessive or insufficient coverage of a certain area, ensure the comprehensiveness of dynamic change analysis, display the divided image blocks through visualization tools (such as Matplotlib), confirm the rationality of each covered area, calculate the proportion of tissue types covered by each block, and ensure the uniform distribution of each modality image information among image blocks.

[0027] S3.2. Before the puncture operation begins, collect at least two ultrasound images as the initial image group and record the baseline grayscale distribution to provide a reference for subsequent dynamic change analysis. First, ensure that the equipment is working properly and the sensor data is stable; second, collect two ultrasound images at a fixed time interval (such as every 1 second), store the image data in the image database, and associate the timestamp and image block number; finally, pre-process the collected baseline images (such as denoising, enhancement) to ensure the accuracy of the grayscale distribution; During the actual puncture process, ultrasound images are collected in real time at fixed time intervals, and new images are added to the image group in sequence to form a time-series image dataset. Specifically, when the puncture operation begins, the real-time image acquisition module is started to ensure that the ultrasound probe and the puncture needle move synchronously to avoid image blur. A timer (such as the time module in Python) is used to trigger image acquisition every 0.5 seconds, and the collected image blocks are added to the corresponding image block group in sequence to form a time-series image dataset.

[0028] S3.3. In order to accurately identify the image changes caused by the puncture needle, the dynamic time warping (DTW) algorithm is used to measure the deviation between the grayscale distribution sequence and the mean sequence, thereby calculating the dynamic change degree of the grayscale of each image block; for each image block , extracting grayscale distribution sequence from time series image dataset ,in =1,2,…, , is the number of time series images, and the mean grayscale sequence of the baseline phase is calculated , the expression is, in, is the number of baseline time series images (e.g., 2 images); The DTW algorithm is used to measure the grayscale sequence of image blocks. With mean series The similarity between them is calculated by minimizing the Euclidean distance to find the best matching path and calculate the DTW distance. , the expression is, in, is the grayscale distribution sequence of the current image block, is the reference grayscale distribution sequence (mean), Representation sequence Middle The gray value of the position, is the sequence length, i.e. the total number of sequential images; By implementing the DTW algorithm using the dtaidistance library in Python, we calculate the At the moment Grayscale distribution sequence With mean series The DTW distance between them; Combined with Grayscale Fluctuation Index and DTW distance , calculate the dynamic change degree of grayscale of the image block, calculate the image block The degree of dynamic change of grayscale , the expression is, in, For image blocks The degree of dynamic change of grayscale, For the Moment image block Grayscale fluctuation indicators (such as standard deviation of grayscale values); Grayscale Fluctuation Index To measure the Moment image block The grayscale change degree can usually be measured by using the standard deviation or variance of the grayscale value as an indicator; For all image blocks , according to the calculated grayscale dynamic change degree Sort by high to low, select the degree of change The highest several image blocks are used to infer the possible area where the puncture needle is located, and a threshold is set or the top five image blocks with the highest degree of change are selected as the key area for puncture needle positioning.

[0029] S4, based on the degree of dynamic grayscale change of each image block, noise removal and feature enhancement are performed on each image block, including the following steps: S4.1. Each image block obtained by calculation The degree of dynamic change of grayscale , as the basis for similarity weight calculation, set the adjustment parameters =0.001 to control the weight decay speed and ensure the sensitivity and accuracy of weight calculation; For each pair of image blocks, calculate the difference in the degree of dynamic change of their grayscale , calculate the similarity weight between image blocks, the expression is, in, For image blocks With image blocks The larger the weight value, the higher the similarity between the image block pairs, and the smaller the weight value, the lower the similarity. , For image blocks and image blocks The degree of dynamic change of grayscale; generate the similarity weight matrix W between image blocks for subsequent filtering processing.

[0030] S4.2, by combining dynamic similarity weights and non-local mean filtering, efficient noise removal is achieved while retaining key anatomical structures and puncture needle features; Specifically, define the size of the neighborhood range, such as 20×20 pixels, covering related similar image blocks, for each pixel , locate the image block where it is located, and select the image block with high similarity in the similarity weight matrix W ; Accumulate the weights of all neighboring image blocks to obtain the normalization factor and calculate the denoised pixel value , the expression is, in, is the pixel value after denoising, is the normalization factor to ensure that the pixel value after filtering is within a reasonable range. For image blocks With the neighborhood image patch The similarity weight of Pixel The non-local neighborhood range of Neighborhood pixels The original pixel value of After the denoising of the entire image is completed, the filtered denoised multimodal image is obtained. ; Furthermore, efficient matrix operation libraries (such as NumPy and OpenCL) are used to accelerate the filtering process and ensure real-time processing requirements. Through parallel computing technology, image blocks are filtered in parallel to significantly improve the processing speed. In the filtering process, gradient weights or edge detection algorithms (such as Sobel operator) are used to protect image edges, prevent edge blur, and retain the sharp edges of key structures such as puncture needles. According to the actual image noise level, similarity weights and neighborhood range parameters are adjusted to optimize the filtering effect. The optimal parameter combination (such as = 0.001, neighborhood size = 20 × 20 pixels) to balance noise removal and structure preservation.

[0031] S4.3. Use the pre-trained Fusion U-Net model, which is based on the classic U-Net architecture and has multimodal data fusion capabilities. It is suitable for feature enhancement of ultrasound, near-infrared light, and elastic imaging data. The input layer is a 3-channel input, corresponding to the denoised ultrasound, NIR light, and elastic imaging data. Each layer contains convolution, batch normalization (Batch Normalization, BN) and ReLU activation functions. Upsampling deconvolution and skip connections are used to fuse high-resolution features with low-resolution features. The Sigmoid activation function is used to ensure that the output image grayscale value is in the range of 0,1. The denoised multimodal image is normalized, and the normalized multimodal image is input into the pre-trained fusion U-Net model for forward propagation to generate an enhanced multimodal image; Through the skip connection mechanism of the U-Net network, high-resolution features are retained, the edges and contrast of the puncture needle are enhanced, and the detailed features of the puncture needle and key anatomical structures are extracted through the convolutional layer to ensure that they are clearly visible in the enhanced image; the enhanced image is denormalized to restore the grayscale value range, and morphological processing (such as dilation or corrosion) is used to further enhance the features of the puncture needle and improve its contrast and visibility in the image.

[0032] S5. Using the denoised multimodal image data, an accurate three-dimensional model of the spinal canal region is generated by volume rendering, and the three-dimensional model is superimposed and displayed in real time by an augmented reality device, including the following steps: S5.1. Convert the registered two-dimensional image sequence into three-dimensional voxel data to achieve spatial information expansion from two-dimensional to three-dimensional; Specifically, the three-dimensional resolution of voxels is set according to the depth of the puncture area and the organ structure, such as 1×1×1 mm, to ensure the details and accuracy of the three-dimensional model, and the linear interpolation method is used to fill the blank areas between the two-dimensional images to ensure the consistency and integrity of the voxel data; In order to ensure that the three-dimensional data after voxelization has no obvious faults and holes and to maintain the coherence and integrity of the three-dimensional model, three-dimensional visualization tools (such as VTK or Mayavi) can be used to visualize the voxel data, check the coherence and accuracy of the model, and identify and repair holes and faults in the voxel data through threshold segmentation and connected component analysis.

[0033] S5.2. According to clinical needs and the purpose of the 3D model, the Marching Cubes algorithm was selected for surface reconstruction because it can effectively extract the surface mesh model from the voxel data and is suitable for accurate modeling of anatomical structures. MarchingCubes is a classic 3D surface reconstruction algorithm that traverses each small cube in the voxel data and determines the position and shape of the surface polygon according to the combination of the voxel value thresholds inside it to generate a mesh model. By detecting whether the vertex of each small cube in the voxel data is above or below the threshold, the position and shape of the surface polygon are determined according to the vertex state, thereby constructing a three-dimensional surface mesh. The specific process is as follows: The cube state is defined as, in, =1 if vertex The voxel value Vi ≥ threshold, otherwise =0; According to the state of the cube, the table is looked up to determine the triangle combination that needs to be generated, and the precise position of the polygon vertices is calculated through linear interpolation to form a grid structure; Use the Quadratic Decimation method to optimize the mesh structure by reducing the number of polygons, so as to improve rendering and computing efficiency while maintaining the original shape. The Quadratic Decimation method is used to simplify the triangular mesh to reduce the number of polygons while maintaining the details of the geometric shape as much as possible. This method uses quadratic simplification technology to reduce the complexity of the mesh and is often used to optimize computing and rendering performance; Apply Laplacian smoothing algorithm to reduce mesh noise and smooth surface curves.

[0034] S5.3. To realize the real-time superimposition display of the 3D model in the doctor's field of view, select a suitable augmented reality device and perform precise positioning and tracking configuration; Furthermore, Microsoft HoloLens 2 was selected as the AR device because it has high-resolution display, accurate head tracking and gesture recognition capabilities, suitable for complex medical environments; Use optical tracking technology (such as the LED marker-based Motion Capture system) to achieve high-precision synchronization between the device and the patient's position. Install multiple LED markers on the patient's waist and augmented reality device for optical tracking camera recognition, and configure a high frame rate camera (such as 60 FPS) for real-time marker recognition and position calculation; Perform initial calibration of the optical tracking system to correct the spatial relationship between the AR device and the actual anatomical structure to ensure accurate superposition of the 3D model and the patient's position. Specifically, calibrate multiple known position points on the patient's waist, record the corresponding points in the AR device, achieve precise spatial alignment, and use the spatial transformation matrix to adjust the position and direction of the 3D model to achieve high-precision superposition; Import the optimized 3D surface mesh model into the visualization software of the AR device, such as Unity 3D or UnrealEngine, and adjust the position and orientation of the 3D model according to the calibrated spatial transformation matrix so that it is accurately aligned with the patient's actual anatomical position; Continuously receive data from the optical tracking system, parse the data from the optical tracking system, obtain the spatial transformation matrix at the current time point, dynamically update the spatial transformation of the three-dimensional model in the AR device, and adjust the position and orientation of the three-dimensional model in real time to ensure its synchronous movement with the patient's anatomical structure.

[0035] S6. On the generated three-dimensional model, important anatomical structures are marked, and the optimal puncture path avoiding key structures is calculated in three-dimensional space using the A* algorithm, including the following steps: S6.1. In order to generate a three-dimensional model with key structure labels and provide obstacle information for path planning, use an image segmentation algorithm (such as U-Net based on deep learning) to process the fused multimodal image data, automatically identify and segment key anatomical structures (such as nerves and blood vessels), and manually correct the automatic segmentation results to ensure the accuracy of the annotation.

[0036] S6.2. Determine the three-dimensional boundary of the puncture area, set the resolution of the grid (e.g., 1 mm³ per voxel), and use a uniform grid division method to divide the puncture area into a number of grid nodes, each node representing a point in space; Assign position coordinates to each node ( , , ), marking whether the node is located inside the key anatomical structure as an obstacle node, providing the spatial position and obstacle information of the node for path planning; Define the connection relationship between nodes, using 26-neighborhood connection mode, that is, each node is connected to at most 26 neighboring nodes (including all adjacent up, down, left, right and diagonal directions), for each node ( , , ), identify all possible adjacent nodes ( , , )satisfy , exclude all obstacle nodes and ensure that only feasible neighbor nodes are connected.

[0037] S6.3. Select Euclidean distance as the heuristic function to estimate the straight-line distance from the current node n to the target node. The expression is: in,( , , ) is the coordinate of the target node, ( , , ) is the current node The coordinates of Extract the current node The coordinates of ( , , ) and its parent node The coordinates of ( , , ), calculate the Euclidean distance between the current node and its parent node, the expression is, The actual cost of the current node Update to the actual cost of the parent node Add the distance between nodes: The actual cost of the starting node 0; By calculating the total cost of each node , combined with the actual cost and the heuristic cost , determine the optimal path, the expression is, in, From the starting point to the current node The actual cost is usually the Euclidean distance or Manhattan distance between nodes. For the current node Heuristically estimated cost to the target node (e.g., Euclidean distance); Use the A* algorithm to search for the optimal path from the starting point to the target point in the three-dimensional space grid, add the starting point node to the open list, and set its total cost Calculate and record, and start the search process of the A* algorithm; Select the node with the smallest total cost from the open list , remove it from the open list and add it to the closed list (Closed List); traverse the node All neighbor nodes of , skip all neighbor nodes located in the key anatomical structure, and calculate the distance from the starting point to the neighbor node The actual cost distance( , ), and calculate the heuristic cost and total cost ; If the node Already in the closed list, skip; if the node If it is not in the open list, add it to the open list and record its parent node as ; If the node Already in the open list, but new If it is smaller, update its total cost and parent node; Repeat the steps until the target node is selected from the open list and the path search is completed; if the open list is empty, no valid path can be found; Starting from the target node, backtrack along the parent node step by step until the starting point node is reached to form the entire path. The node sequence obtained by backtracking is reversed to obtain a complete puncture path from the starting point to the target point.

[0038] This embodiment also provides a computer device, which is suitable for the case of anesthesia puncture assisted positioning method, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the anesthesia puncture assisted positioning method proposed in the above embodiment.

[0039] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0040] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for implementing anesthesia puncture auxiliary positioning as proposed in the above embodiment is implemented; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0041] In summary, the present invention acquires multimodal image data of the area to be anesthetized, synchronizes and fuses them, and generates a multimodal comprehensive image, thereby realizing the complementary advantages of different imaging technologies and significantly improving the comprehensiveness and accuracy of the image data; the fused multimodal comprehensive image is evenly divided into a number of image blocks, and the degree of dynamic change of the grayscale of each image block is calculated, thereby realizing localized analysis of the image data and effectively capturing the characteristic changes of different anatomical regions; based on the degree of dynamic change of the grayscale of each image block, each image block is subjected to targeted noise removal and feature enhancement processing, thereby realizing a significant improvement in image quality, effectively retaining key anatomical structure information, while suppressing interference factors, providing high-quality data input for subsequent three-dimensional reconstruction, and enhancing the readability and interpretability of the image; using the processed multimodal image data, an accurate three-dimensional model is generated by a volume rendering method, and combined with augmented reality technology for real-time display, thereby realizing an intuitive presentation of complex anatomical structures, further, important anatomical structures are marked on the three-dimensional model, and the A* algorithm is used to calculate the optimal puncture path, thereby realizing safe and efficient puncture navigation, and significantly improving the accuracy and safety of anesthesia puncture.

[0042] Example 2, referring to Table 1, is the second example of the present invention. To further verify the technical solution of the present invention, experimental simulation data of an anesthesia puncture auxiliary positioning method is provided.

[0043] This example is aimed at assisting positioning of lumbar anesthesia puncture. The experimental preparation and implementation process are as follows: First, 50 patients who needed lumbar anesthesia were selected, with an age range of 25-65 years old and a body mass index (BMI) between 18.5 and 30. The patients were randomly divided into two groups: an experimental group (25 people) and a control group (25 people). The experimental group adopted the anesthesia puncture assisted positioning method of the present invention, and the control group adopted the traditional ultrasound guidance method.

[0044] For the experimental group, a Philips EPIQ Elite ultrasound system equipped with an L12-3 linear array probe was used for ultrasound imaging, a FLIR One Pro near-infrared light sensor was used to obtain near-infrared light images, and the Virtual Touch elastic imaging function of the Siemens ACUSON S2000HELX Evolution ultrasound system was used to collect elastic imaging data. All devices were synchronized through a high-precision time synchronization system to ensure the time consistency of multimodal data.

[0045] The image processing and fusion stage uses a pre-trained U-Net model for image fusion, which is trained on a dataset containing 5,000 multimodal medical images. The image blocks are divided into 8x8 grids, and each image block is 64x64 pixels in size.

[0046] In order to generate an accurate 3D model, a high-precision lumbar spine model was made using a Voxel8 3D printer for system calibration and accuracy verification. Microsoft HoloLens 2 was used as an AR device, and the OptiTrack Prime 13W optical tracking system was used for real-time positioning.

[0047] During the actual puncture process, the following parameters were recorded: puncture success rate, number of puncture attempts, puncture time, patient pain score (VAS score), and complication rate. At the same time, three experienced anesthesiologists were invited to score the difficulty of the puncture process (1-10 points, 10 points means the easiest).

[0048] The details are shown in Table 1 below: Table 1 Experimental data table By comparing the data of the experimental group and the control group, it can be clearly seen that the method of the present invention is significantly superior to the traditional ultrasound-guided method in many aspects.

[0049] The puncture success rate of the experimental group reached 96%, which was 14.29% higher than the 84% of the control group. This shows that the method of the present invention can provide more accurate positioning information and greatly improve the accuracy of puncture. For the average number of puncture attempts, the experimental group only needed 1.2 attempts on average to successfully puncture, while the control group needed 2.3 attempts. The method of the present invention reduced the number of puncture attempts by 47.83%, which not only improved the efficiency but also reduced the trauma to the patient. The average puncture time of the experimental group was 3.5 minutes, which was 51.39% less than the 7.2 minutes of the control group. This significant time saving not only improved the efficiency of the operation, but also reduced the discomfort time of the patient on the operating table. In terms of the average patient pain score, the VAS score of the experimental group was 2.1, which was 51.16% better than the 4.3 of the control group. This shows that the method of the present invention greatly reduced the pain of the patient during the puncture process and improved the patient's comfort. The complication rate of the experimental group was only 2%, while that of the control group was 8%, a decrease of 75%. This improvement shows that the method of the present invention greatly improves the safety of the operation and reduces potential medical risks. In the anesthesiologist evaluation, the experimental group scored 8.7 points, 40.32% higher than the control group's 6.2 points. This shows that the method of the present invention is not only excellent in objective indicators, but also recognized by professional physicians in terms of subjective operation experience.

[0050] In summary, the present invention has achieved remarkable results in improving the success rate of puncture, reducing operation time, reducing patient pain and the risk of complications by integrating multiple advanced technologies. This not only improves medical efficiency, but also greatly improves patient experience.

[0051] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An anesthesia puncture auxiliary positioning method, characterized in that: include, Acquire multimodal image data of the area to be anesthetized, synchronize and fuse the multimodal image data, and generate a multimodal comprehensive image; The fused multimodal comprehensive image is evenly divided into a number of image blocks according to a preset number, and the degree of dynamic grayscale change of each image block is calculated; Based on the degree of dynamic grayscale change of each image block, noise removal and feature enhancement processing are performed on each image block; Using the processed multimodal imaging data, an accurate three-dimensional model of the spinal canal region is generated through volume rendering, and the three-dimensional model is superimposed and displayed in real time through augmented reality equipment; Important anatomical structures are marked on the generated three-dimensional model, and the A* algorithm is used to calculate the optimal puncture path avoiding key structures in three-dimensional space.

2. The anesthesia puncture auxiliary positioning method according to claim 1, characterized in that: The step of obtaining multimodal image data of the area to be anesthetized comprises the following steps: Ultrasonic images, near-infrared light images and elastic imaging data of the patient's target puncture area are collected through ultrasonic probes, near-infrared light sensors and elastic imaging equipment.

3. The anesthesia puncture auxiliary positioning method according to claim 2, characterized in that: The step of synchronizing and fusing the multimodal image data to generate a multimodal comprehensive image comprises the following steps: Use a unified time base to generate timestamps, automatically embedding accurate timestamps when acquiring image data for each modality; Use image processing software to extract timestamp information from the metadata of each modality image; Calculate the time difference between different modality images, identify and record fixed delays, and adjust the time series of each modality image through interpolation and cropping methods; SIFT algorithm is applied to ultrasound images, NIR images and elastic imaging images to detect key feature points, and the coordinates and descriptors of each feature point are recorded; Initialize the FLANN matcher, set the index parameters and search parameters, perform feature point matching on NIR-ultrasound and elastic imaging-ultrasound, generate potential matching pairs, and use ratio test to filter out false matches; Extract the coordinate set of the matching point pairs, calculate the affine transformation matrices of NIR-ultrasound and elastic imaging-ultrasound respectively, use the calculated affine transformation matrices to perform spatial transformation on the NIR image and the elastic imaging image, and generate the NIR image and the elastic imaging image that are spatially aligned with the ultrasound image; The aligned ultrasound images, NIR images and elastic imaging data are normalized to unify the grayscale range. The normalized multimodal image data are input into the pre-trained fusion U-Net model for forward propagation to generate a fused multimodal comprehensive image.

4. The anesthesia puncture auxiliary positioning method according to claim 3, characterized in that: The fused multimodal comprehensive image is evenly divided into a number of image blocks according to a preset number, and the degree of dynamic change of the grayscale of each image block is calculated, including the following steps: Set the number of image blocks and image block size, use a fixed grid method, start from the upper left corner of the image, divide it row by row, and verify whether each image block covers different anatomical areas; Before the puncture operation begins, at least two ultrasound images are acquired as the initial image set, and the baseline grayscale distribution is recorded; During the actual puncture process, ultrasound images are collected in real time at fixed time intervals, and new images are added to the image group in sequence to form a time-series image data set; For each image block, the grayscale distribution sequence is extracted from the time series image dataset, and the mean grayscale sequence in the baseline phase is calculated; The DTW algorithm is implemented by using the dtaidistance library in Python to calculate the DTW distance between the grayscale distribution sequence and the mean sequence of each image block at each moment; The grayscale fluctuation index and DTW distance are combined to calculate the dynamic change degree of grayscale of the image block.

5. The anesthesia puncture auxiliary positioning method according to claim 4, characterized in that: Based on the degree of dynamic grayscale change of each image block, noise removal and feature enhancement are performed on each image block, including the following steps: For each pair of image blocks, calculate the difference in the degree of dynamic change of their grayscale; Calculate the similarity weights between image blocks and construct a similarity weight matrix between image blocks; Define the size of the neighborhood range, locate the image block where each pixel is located, and select the image block with high similarity in the similarity weight matrix; The weights of all neighboring image blocks are accumulated to obtain the normalization factor, and the denoised pixel value is calculated to complete the denoising process of the entire image, thus obtaining the filtered denoised multimodal image. The denoised multimodal image is normalized and input into the pre-trained fusion U-Net model for forward propagation to generate an enhanced multimodal image.

6. The anesthesia puncture auxiliary positioning method according to claim 5, characterized in that: Using the denoised multimodal imaging data, an accurate three-dimensional model of the spinal canal region is generated by volume rendering, and the three-dimensional model is superimposed and displayed in real time through augmented reality equipment, including the following steps: The three-dimensional resolution of voxels is set according to the depth of the puncture area and the organ structure, and the blank areas between the two-dimensional images are filled using the linear interpolation method; Input the voxel data into the Marching Cubes algorithm, set a threshold, and determine whether the vertex of each small cube in the voxel data is above and below the threshold; According to the state of the cube, the table is looked up to determine the triangle combination that needs to be generated, and the precise position of the polygon vertices is calculated through linear interpolation to form a grid structure; Use the Quadratic Decimation method to optimize the mesh structure by reducing the number of polygons, and apply the Laplacian smoothing algorithm to reduce mesh noise and smooth surface curves; Microsoft HoloLens 2 was selected as the AR device, using optical tracking technology, with multiple LED markers installed on the patient's waist and the augmented reality device, and a high-frame rate camera configured for real-time marker recognition and position calculation; Perform initial calibration of the optical tracking system to correct the spatial relationship between the AR device and the actual anatomical structure; Import the optimized 3D surface mesh model into the visualization software of the AR device, and adjust the position and orientation of the 3D model according to the calibrated spatial transformation matrix; Continuously receive data from the optical tracking system and adjust the position and orientation of the 3D model in real time.

7. The anesthesia puncture auxiliary positioning method according to claim 6, characterized in that: On the generated 3D model, important anatomical structures are marked, and the A* algorithm is used to calculate the optimal puncture path in 3D space to avoid key structures, including the following steps: Use image segmentation algorithms to process the fused multimodal image data, automatically identify and segment key anatomical structures, and manually correct the automatic segmentation results; Discretize the three-dimensional space of the puncture area into grid nodes, each node represents a point in the space; The 26-neighborhood connection method is used to define the connection relationship between nodes; Choose Euclidean distance as the heuristic function to estimate the distance from the current node The straight-line distance to the target node; Combine the actual cost and the heuristic cost to calculate the total cost of each node in the A* algorithm; Add the starting node to the open list and select the node with the smallest total cost from the open list , remove it from the open list and add it to the closed list; Traversing nodes All neighbor nodes of ; If the node Already in the closed list, skip; if the node If it is not in the open list, add it to the open list and record its parent node as ; If the node If it is already in the open list but the new actual cost is smaller, update its total cost and parent node; Repeat the steps until the target node is selected from the open list and the path search is completed; If the open list is empty, no valid path can be found; Starting from the target node, backtrack along the parent node step by step until the starting point node is reached to form the entire path. The node sequence obtained by backtracking is reversed to obtain a complete puncture path from the starting point to the target point.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the anesthesia puncture assisted positioning method described in any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the anesthesia puncture assisted positioning method described in any one of claims 1 to 7 are implemented.

Images