Intelligent human body part recognition implementation method based on local muscle feature recognition

Abstract

The invention provides an intelligent human body part recognition implementation method based on local muscle feature recognition, aiming at solving the problem that anesthesia / surgery operation faces great challenges due to complexity of local limb nerves and individual differences of the local limb nerves, so that the safety and the success rate of local limb nerve block anesthesia / surgery are remarkably improved. A new solution is provided for the development of clinical anesthesia / operation technology, and the limitation of a traditional method is effectively avoided. Meanwhile, machine learning and data analysis technologies are combined, the needle path is automatically optimized on the basis of muscle feature recognition in combination with anatomical features and historical data of an individual patient, the complication occurrence rate is reduced to the maximum extent, and the anesthesia / operation success rate is increased. According to the innovative solution, powerful support is provided for automatic positioning of complex local limb nerves, and the overall effect and safety of clinical anesthesia / operation are improved.

Description

Technical Field

[0001] The invention relates to a method for realizing automatic identification of various parts of a human body in an efficient and accurate manner, and belongs to the field of medical technology, in particular to the fields of anesthesia / surgery and neuro-navigation. Background Art

[0002] Most of the current local nerve block anesthesia / surgery methods rely on static images (such as ultrasound images) and the experience of anesthesiologists / surgeons, and cannot adapt to changes in the patient's anatomical structure in real time. Especially in a dynamic surgical environment, it is difficult to ensure that the needle accurately enters the target nerve area during local anesthesia / surgery. This technology can assist anesthesiologists in more accurately identifying and locating specific areas that require anesthesia, especially in complex anatomical structures. Summary of the invention

[0003] The purpose of the present invention is to provide a method that can effectively identify local muscle features, capture and map local anatomical structure changes, and determine nerve positions based on this.

[0004] In order to achieve the above object, the technical solution of the present invention is to disclose a method for realizing intelligent nerve positioning based on local muscle group recognition, which is characterized by comprising the following steps:

[0005] Step 1: Dynamic image processing and feature extraction, further including:

[0006] Step 101: Get dynamic image:

[0007] Use high frame rate and high resolution real-time ultrasound equipment to obtain dynamic images of local limb anatomical structures;

[0008] Step 102, image enhancement and feature extraction, includes the following steps:

[0009] Step 1021, local noise statistics calculation:

[0010] Use a sliding window of 3*3 to 5*5 to process the dynamic image and calculate the mean and variance within the window, as shown in the following formula:

[0011]

[0012] Where f(x,y) is the pixel value in the current dynamic image frame being processed, k is the parameter for adjusting the noise estimation, M is the number of pixels in the sliding window used for mean and variance calculation, E{f(x,y)} is the mean, and Var{f(x,y)} is the variance;

[0013] Step 1022: Adaptively adjust the noise estimation value k according to the local variance through a dynamic window adjustment strategy, thereby optimizing the noise filtering effect;

[0014] Step 1023, using contrast limited adaptive histogram equalization (CLAHE) for contrast enhancement;

[0015] Step 1024: Use a deep learning model to perform image analysis, and use YOLO and Mask R-CNN to perform real-time feature detection and segmentation;

[0016] Step 103: Dynamic 3D reconstruction:

[0017] Using continuous ultrasound images and dynamic capture data, combined with computer vision technology, a dynamic local limb three-dimensional model is generated to reflect the anatomical changes of the local limb in real time, including the following steps:

[0018] Step 1031, extracting a disparity map from the multiple registered images, and calculating a three-dimensional point cloud based on the disparity;

[0019] Step 1032, voxel grid:

[0020] For scenes with slight dynamic displacement, the point cloud is voxelized using the voxel grid method to reduce noise and computational complexity;

[0021] Step 104: Dynamic update and rendering:

[0022] Use the dynamically captured data to update the point cloud and 3D model, and combine the Kalman filter to predict the dynamic state and compensate for dynamic changes;

[0023] Step 2: Local muscle group identification, further including:

[0024] Step 201, muscle group image segmentation, including:

[0025] Step 2011, data set construction and enhancement: using a domain-specific high-quality ultrasound image set and using data enhancement technology to expand the training set;

[0026] Step 2022, build an improved U-Net architecture: introduce an attention mechanism based on the U-Net network, as shown in the following formula:

[0027]

[0028] Where Q, K, and V are query, key, and value respectively. k The dimension of the key.

[0029] Step 223, deep supervised learning: Introduce a deep supervision mechanism in each downsampling layer of U-Net and adopt a multi-scale loss function. The formula is as follows:

[0030]

[0031] in, is the l-layer segmentation loss, λ l is the weight coefficient of each layer;

[0032] Step 202, complex feature extraction and representation learning of muscle groups, including:

[0033] Step 221, feature pyramid network and residual connection: process image features of different sizes through the FPN structure, combine residual learning, and use Skip Connection to reduce the risk of gradient disappearance;

[0034] Step 2022, graph convolutional network: after feature extraction, further apply graph convolutional network to model the complex relationship between the muscle group and the nerves, wherein local structure modeling is achieved by constructing an adjacency matrix and a feature matrix;

[0035] Step 2023, multi-task learning: simultaneously learning muscle group segmentation and nerve localization, and improving the model's ability to understand different tasks by sharing feature extraction layers;

[0036] Step 2024, deep segmentation and classification: at the output end of the model, a deep segmentation head is set to predict the muscle group category to which each pixel belongs, and a weighted cross entropy loss function is used to optimize the segmentation accuracy;

[0037] Apply the Softmax function on the segmentation result of each pixel and map the output into the probability distribution of different muscle groups;

[0038] Step 203, efficient nerve position determination, includes:

[0039] Step 2031, spatial structure inference: combining anatomical knowledge and using graph-based deep learning methods to establish a spatial relationship diagram of muscle groups and nerves;

[0040] Step 2032, regression-based nerve position prediction: using a deep regression network, taking into account the shape, texture and relative position of the local muscle group, and outputting the three-dimensional coordinates of the nerve;

[0041] Step 2033, error feedback and optimization adjustment: using a feedback mechanism based on Bayesian optimization to dynamically adjust model parameters and network structure to reduce prediction errors;

[0042] Introducing a spatial transformation network to perform geometric transformation on the input image, improving the model's robustness to changes in anatomical structure position and improving prediction accuracy;

[0043] Step 3, intelligent path optimization and navigation algorithm, further comprising:

[0044] Step 301, algorithm input:

[0045] Combined with the generated 3D anatomical model, the coordinate information of muscles, nerves and blood vessels is obtained in real time to form a 3D anatomical model, and the data is fused with the image data to generate a dynamically updated 3D spatial model to ensure that the path planning is based on the real-time anatomical structure;

[0046] Step 301: Dynamic path planning:

[0047] A deep learning-based path planning algorithm is used to calculate the optimal needle path in real time, taking into account the current local limb status and changes in anatomical structure;

[0048] Step 4: Real-time feedback and monitoring mechanism:

[0049] Needle displacement monitoring: Use an optical tracking system or IMU sensor to monitor the needle position in real time to ensure that the needle is always on the optimal path, further including:

[0050] Step 401, error feedback mechanism:

[0051] Calculates the deviation between the needle's current displacement and the optimized path and provides instant audible and visual warnings when the deviation exceeds a threshold.

[0052] Preferably, in step 101, the ultrasound probe is optimized and selected according to different anatomical structures and dynamic states, and multiple imaging modes are applied simultaneously to perform ultrasound imaging of a specific area to obtain static image data from multiple angles to ensure coverage of target muscle groups, nerves and blood vessels.

[0053] Preferably, in step 1022, the noise estimation value k is adaptively adjusted using the following formula:

[0054] k=max(σ min ,α·Var{f(x,y)})

[0055] Among them, σ min is the minimum noise threshold, and α is an adjustable coefficient.

[0056] Preferably, in step 1023, the image is divided into overlapping small blocks, each small block is independently subjected to histogram equalization, and the contrast is limited to avoid over-enhancement:

[0057]

[0058] Among them, H(i) is the frequency of pixels with pixel intensity i, which is used to count the number of pixels at different gray levels, N is the total number of pixels in the image, and CDF is the function for calculating the cumulative probability of gray levels. It calculates the cumulative distribution from intensity 0 to k, which represents the proportion of all pixels at and below the current gray level;

[0059] The processed small blocks are merged into a global image, and Gaussian blur or bilateral filtering is applied to smooth the edges to eliminate obvious visual differences at the seams.

[0060] Preferably, in step 1024, the YOLO is subjected to:

[0061] Network architecture optimization, including: introducing the CSPNet network, which divides the feature map into two parts and transmits them separately, reduces the amount of calculation through cross-stage connections while maintaining high accuracy;

[0062] Replace the standard convolution with a depth-wise separable convolution, where the depth-wise separable convolution is divided into two stages: depth-wise convolution and point-wise convolution, which are calculated as follows:

[0063]

[0064] Y final =W point ·Y out

[0065] Where W depth is the convolution kernel in the deep convolution, the weight matrix, X is a multi-channel two-dimensional matrix containing the input information of the convolution layer, W depth,k is the convolution kernel used for the kth channel, X k is the kth channel in the input data, W point is the convolution kernel in point-by-point convolution, Y final is the final output result of the depth-separable convolution;

[0066] The SE module is introduced in the feature extraction layer to adaptively adjust the weights of channel features through global information, thereby enhancing the representation ability of important features;

[0067] Anchor box optimization includes: using the K-Means clustering algorithm to analyze the target box size in the training set and automatically generate an anchor box size suitable for the data set, which is achieved through the following steps:

[0068] Calculate bounding box size: extract the width and height of each object in the training set; Apply K-Means clustering: perform K-Means clustering on the extracted sizes and choose a suitable K value to maximize the IoU of each anchor box with its corresponding object.

[0069] Loss function optimization, including: YOLO uses a combined loss function to simultaneously optimize the coordinates, confidence, and category probabilities of the bounding box:

[0070] L=L coord +λ obj L obj +λ nobj Lnobj +λ class L class

[0071] Where, L coord is the part of the loss function used to optimize the bounding box coordinates, L obj To optimize the confidence loss of the target object, L nobj To minimize the confidence loss of the non-existence of the target object, L class To optimize the loss of class probability, λ obj , nobj , class It is the weight coefficient of different parts in the loss function, which controls the contribution of each part of the loss to the total loss.

[0072] Preferably, in step 104, OpenGL is used to render the point cloud and the three-dimensional model in real time, and the rendering process is accelerated by a GPU to improve rendering efficiency and smoothness.

[0073] Preferably, in step 2031, spatial feature modeling is implemented through a graph neural network, and the formula is as follows:

[0074]

[0075] in: is the neighbor set of node i, W is the trainable weight matrix, is the feature representation of node j in the kth layer, which is used to aggregate information from neighboring nodes.

[0076] Preferably, in step 2032, the regression network structure is:

[0077]

[0078] in: is the feature map, θ is the model parameter, and b is the bias of the regression model, which is used to adjust the fixed error of the predicted output.

[0079] Preferably, in step 401, the deviation is calculated using the following formula:

[0080]

[0081] Among them, (x t ,y t , z t ) is the real-time needle coordinate, (x p ,y p , z p ) is the coordinate of the target point on the path;

[0082] When the deviation exceeds the set threshold, the system automatically issues audible and visual alarms.

[0083] In modern medical image analysis, intelligent human body part recognition based on local muscle features has important application value. Through deep learning and feature extraction of local human muscles, the present invention constructs an efficient and accurate method for automatic recognition of various parts of the human body. Through local muscle group recognition, combined with AR and AI technology, real-time and dynamic needle path guidance is provided to assist in improving the accuracy and safety of local nerve block anesthesia / surgery.

[0084] The complexity of local limb nerves and their individual differences make anesthesia / surgical operations face great challenges. The present invention proposes an intelligent human body part recognition implementation method based on local muscle feature recognition. This method will significantly improve the safety and success rate of local limb nerve block anesthesia / surgery, provide new solutions for the development of clinical anesthesia / surgery technology, and effectively avoid the limitations of traditional methods. At the same time, combined with machine learning and data analysis technology, the needle path is automatically optimized based on muscle feature recognition combined with the patient's individual anatomical characteristics and historical data, to minimize the incidence of complications and improve the success rate of anesthesia / surgery. This innovative solution will provide strong support for the automatic positioning of complex local limb nerves and improve the overall effect and safety of clinical anesthesia / surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0086] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0087] The present invention discloses a method for realizing intelligent nerve positioning based on local muscle group recognition, which specifically includes the following steps:

[0088] Step 1: Dynamic image processing and feature extraction

[0089] (1) Dynamic image acquisition:

[0090] In this step, high-end real-time ultrasound equipment (such as GE Logiq E9, Philips EPIQ 7 or Siemens ACUSON) is used to obtain dynamic images of local limb anatomical structures. These devices have high frame rates (≥30 frames per second) and high resolutions (≥1920x1080) to ensure that the details of muscles, blood vessels and nerves are captured. In order to maximize the capture of image details, the ultrasound probe is optimized according to different anatomical structures and dynamic states, and multiple imaging modes such as B mode and color Doppler are applied. Ultrasound imaging is performed on specific areas (such as thighs, arms, etc.) before surgery to obtain static image data from multiple angles to ensure coverage of target muscle groups, nerves and blood vessels.

[0091] (2) Image enhancement and feature extraction:

[0092] A deep learning model is used to analyze local limb dynamic images to identify and segment nerves, blood vessels and other important anatomical structures in real time. Adaptive Wiener filter is used to calculate the local mean and variance of the image and use the mean and variance to estimate noise. The detailed steps are as follows:

[0093] (a) Local noise statistics calculation:

[0094] The image is processed using a sliding window of 3*3 to 5*5, and the mean and variance within the window are calculated to adapt to areas with different noise levels:

[0095]

[0096] Where f(x,y) is the pixel value at position (x,y) in the image. This value represents the intensity of the image brightness or color at that position, k is the estimated value of the noise, M is the total number of pixels in the sliding window, E{f(x,y)} is the mean, and Var{f(x,y)} is the variance.

[0097] (b) Adaptive noise estimation:

[0098] Through the dynamic window adjustment strategy, the noise estimation value k is adaptively adjusted according to the local variance to optimize the noise filtering effect. The formula is:

[0099] k=max(σ min ,α·Var{f(x,y)})

[0100] Among them, σ min is the minimum noise threshold, and α is an adjustable coefficient.

[0101] Image enhancement uses contrast-limited adaptive histogram equalization (CLAHE) contrast enhancement. The specific steps are as follows:

[0102] (c) Small block segmentation and local equalization:

[0103] The ultrasound image is divided into overlapping small blocks (such as 8x8), each small block is independently histogram equalized, and the contrast is limited to avoid over-enhancement:

[0104]

[0105] Among them, H(i) is the frequency of occurrence of pixel intensity value i, that is, the height of the histogram, N is the total number of pixels in the image, and CDF is the cumulative distribution calculated from intensity 0 to k, which represents the proportion of all pixels at and below the current gray level, and represents the cumulative probability distribution in the histogram equalization process.

[0106] By limiting the cumulative histogram below a certain threshold, local overexposure is avoided. The processed small blocks are merged into a global image, and Gaussian blur (or bilateral filtering) is applied to smooth the edges to eliminate obvious visual differences at the seams.

[0107] (d) Feature extraction:

[0108] A deep learning model is used for image analysis. YOLO (You Only Look Once) and Mask R-CNN are selected for real-time feature detection and segmentation. The following is the specific implementation and optimization process:

[0109] (e)Using U-Net for image segmentation:

[0110] U-Net consists of an encoder and a decoder, and uses a deep residual network (ResNet) as an encoder to improve feature extraction capabilities. By introducing SE Block to adaptively adjust weights in feature channels, the focus on important features is further enhanced. The processed ultrasound image is input and trained through the U-Net structure. The U-Net structure includes an encoder and a decoder, which can effectively extract features.

[0111] During the training process, the processed ultrasound images are input and the training data set is expanded by combining data augmentation techniques (such as rotation, translation, and scaling). The cross entropy loss function is used for optimization.

[0112]

[0113] Among them, y i is the true label of the i-th pixel, is the predicted probability of the i-th pixel, and N is the total number of all pixels in the image.

[0114] At the same time, the Dice coefficient is introduced as an auxiliary loss to improve the segmentation progress:

[0115]

[0116] Where X is the target region set predicted by the model, including all pixels predicted as target regions, and Y is the real target region set, including all pixels actually belonging to the target region.

[0117] Improving segmentation performance through joint optimization.

[0118] (2) YOLO model optimization:

[0119] Uniform size of input images: All input images are uniformly scaled to a specific size (such as 416x416 or 608x608). This fixed input size helps the model achieve high efficiency during training and inference. Images are randomly cropped to increase the robustness of the model. The brightness, contrast, saturation, and hue of the images are adjusted to enhance the model's adaptability to color changes. Images are horizontally flipped with a probability of 50% to increase the diversity of training samples.

[0120] (a) Network architecture optimization: CSPNet (Cross Stage Partial Network) is introduced to enhance the network's feature expression ability and gradient transfer efficiency. CSPNet divides the feature map into two parts and transmits them separately, reducing the amount of calculation through cross-stage connections while maintaining high accuracy. The standard convolution is replaced by the depthwise separable convolution to reduce the number of parameters and the amount of calculation while maintaining the ability to extract features. The depthwise separable convolution is divided into two stages: depthwise convolution and pointwise convolution. The specific calculation method is as follows:

[0121]

[0122] Y final =W point ·Y out

[0123] Where W depth is the convolution kernel in the deep convolution, the weight matrix, X is a multi-channel two-dimensional matrix containing the input information of the convolution layer, W depth,k is the convolution kernel used for the kth channel, X k is the kth channel in the input data, W point is the convolution kernel in point-by-point convolution, Y final is the final output of the depthwise separable convolution.

[0124] The SE module (Squeeze-and-Excitation) is introduced in the feature extraction layer to adaptively adjust the weights of channel features through global information, thereby enhancing the representation ability of important features.

[0125] (b) Anchor Box Optimization: Use the K-Means clustering algorithm to analyze the target box size in the training set and automatically generate an anchor box size suitable for the data set. This method can be implemented through the following steps:

[0126] Calculate bounding box size: Extract the width and height of each object in the training set. Apply K-Means clustering: Perform K-Means clustering on the extracted sizes and choose a suitable K value (usually 9 anchor boxes) to maximize the IoU between each anchor box and its corresponding object.

[0127] (c) Loss function optimization: YOLO uses a combined loss function to simultaneously optimize the coordinates, confidence, and category probability of the bounding box:

[0128] L=L coord +λ obj L obj +λ nobj L nobj +λ class L class

[0129] Where, L coord is used in the loss function to optimize the bounding box coordinates, L obj To optimize the confidence loss of the target object, L nobj To optimize the confidence loss of the non-existent target object, L class To optimize the loss of class probability, λ obj , nobj , class It is the weight coefficient of different parts in the loss function, which controls the contribution of each part of the loss to the total loss.

[0130] (3) Mask R-CNN Implementation

[0131] An extension of Faster R-CNN, in addition to object detection, it also implements instance segmentation. The following are technical details and implementation steps. Mask R-CNN adds a branch based on Faster R-CNN to generate segmentation masks for the target. Region Proposal Network (RPN) is used to generate candidate region proposals, generate anchor boxes on the feature map through a sliding window method, and use a binary classification network to predict whether each anchor box contains the target. The mask branch inputs the feature map of ROI Align, outputs a fixed-size mask (such as 28x28), and uses a sigmoid activation function for binarization.

[0132] In Faster R-CNN, RoIPooling is used to map the features of each candidate box to a fixed size, but in Mask R-CNN, ROI Align is used to maintain the spatial accuracy of the features. The specific implementation is as follows:

[0133] Bilinear interpolation is performed on the feature map for each RoI to obtain the exact location of the feature.

[0134] Map the RoI region to a fixed output size.

[0135] Total loss function design:

[0136] L=L class +L box +L mask

[0137] L class : Category prediction loss, L box Bounding box regression loss, usually using Smooth L1 loss. mask : Mask segmentation loss, which uses cross entropy loss to calculate the difference between the generated mask and the true mask.

[0138] (4) Dynamic 3D reconstruction:

[0139] Continuous ultrasound images and dynamic capture data are used in combination with computer vision technology to generate a dynamic three-dimensional model of the local limb, which reflects the anatomical changes of the local limb in real time.

[0140] (a) Multi-view Stereo (MVS): Extract disparity maps from multiple registered images and calculate 3D point clouds based on disparity. The following steps are used in the implementation process:

[0141] Disparity calculation: The disparity map is calculated using a stereo matching algorithm (such as SGBM or BM).

[0142] Point cloud generation: Convert the disparity map into a 3D coordinate point cloud using the 3D reconstruction formula:

[0143]

[0144] Where f is the focal length of the camera, B is the baseline distance between the two cameras, and d is the disparity value

[0145] (b) Voxel Grid:

[0146] For scenes with slight dynamic displacement, the voxel grid method is used to voxelize the point cloud to reduce noise and computational complexity. The specific methods include:

[0147] Divide the space into a uniform grid of cubes (Voxel).

[0148] Calculate the mean of the point cloud within each grid to generate a sparse point cloud representation.

[0149] (5) Dynamic update and rendering

[0150] The point cloud and 3D model are updated using the dynamically captured data, and the dynamic state is predicted and compensated for dynamic changes in combination with the Kalman filter.

[0151] Use OpenGL or similar graphics libraries to render point clouds and 3D models in real time, and use GPU to accelerate the rendering process to improve rendering efficiency and smoothness.

[0152] Step 2: Local muscle group identification

[0153] (1) Muscle group image segmentation

[0154] (a) Dataset construction and enhancement: A domain-specific high-quality ultrasound image set is used, and data augmentation techniques are used to expand the training set, including random rotation, mirror flipping, scale transformation, etc., to improve the generalization ability and robustness of the model.

[0155] (b) Improved U-Net architecture: Based on the traditional U-Net, the present invention introduces an attention mechanism to enable the network to pay more attention to important feature areas and enhance segmentation accuracy. The specific structure is:

[0156]

[0157] Where Q, K, and V are query, key, and value respectively. k The dimension of the key.

[0158] (c) Deep supervised learning: A deep supervision mechanism is introduced in each downsampling layer of U-Net, and a multi-scale loss function is used. The formula is as follows:

[0159]

[0160] in, is the l-layer segmentation loss, λ l is the weighting coefficient of each layer.

[0161] (2) Complex feature extraction and representation learning of muscle groups

[0162] (a) Feature pyramid network and residual connection: The FPN structure is used to process image features of different sizes, and combined with residual learning, Skip Connection is used to reduce the risk of gradient disappearance:

[0163] F out =F in+F(x)

[0164] Among them, F in is the input feature, and F(x) is the feature after convolution processing.

[0165] (b) Graph Convolutional Network: After feature extraction, the graph convolutional network is further applied to model the complex relationship between the muscle group and its nerves. Local structure modeling is achieved by constructing the adjacency matrix and feature matrix:

[0166]

[0167] in, is the normalized adjacency matrix, H (l) is the l-th layer feature, W (l) is the weight matrix of the lth layer, and σ is the activation function.

[0168] (c) Multi-task learning: Muscle group segmentation and nerve localization are learned simultaneously, and the model's ability to understand different tasks is improved by sharing feature extraction layers. The loss function is:

[0169] L total =L seg +λ·L loc

[0170] Among them, L loc is the neural localization loss, λ is the balance coefficient, L seg is the loss function for the segmentation task performance.

[0171] (d) Deep segmentation and classification: At the output of the model, a deep segmentation head is set to predict the muscle group category for each pixel, and the weighted cross entropy loss function is used to optimize the segmentation accuracy:

[0172]

[0173] Where: w yi is the category weight, which helps deal with category imbalance; p yi is the predicted probability value, indicating the probability that pixel i belongs to category yi; N is the total number of pixels. Apply the Softmax function to the segmentation result of each pixel and map the output to the probability distribution of different muscle groups:

[0174]

[0175] Among them, y i is the true category label of pixel i, x i is the feature input of pixel i, z k is the score of category k in the output layer, z j is the score of category j in the output layer.

[0176] (3) Efficient neural position judgment

[0177] (a) Spatial structure inference: Combined with anatomical knowledge, a graph-based deep learning method is used to build a spatial relationship diagram of muscle groups and nerves. Spatial feature modeling is achieved through a graph neural network (GNN), and the formula is as follows:

[0178]

[0179] in: is the neighbor set of node i, W is the trainable weight matrix, is the feature representation of neighbor node j in the kth layer.

[0180] (b) Regression-based nerve position prediction: A deep regression network is used to comprehensively consider the shape, texture and relative position of local muscle groups and output the three-dimensional coordinates of the nerve. The regression network structure is:

[0181]

[0182] in: is the feature map, θ is the model parameter, and b is the bias term or intercept term.

[0183] (c) Error feedback and optimization adjustment: Use a feedback mechanism based on Bayesian optimization to dynamically adjust model parameters and network structure to reduce prediction errors:

[0184]

[0185] Among them, J is the loss function, α is the learning efficiency, and θ t is the model parameter at time step t, and y is the target output value, that is, the actual neural position or label.

[0186] The spatial transformation network is introduced to perform geometric transformation on the input image, which improves the robustness of the model to changes in the position of anatomical structures and improves the prediction accuracy. The transformation function of STN is:

[0187] y=T(x;θ)

[0188] Where T is the transformation function and θ is the transformation function.

[0189] Step 3: Intelligent path optimization and navigation algorithm

[0190] (1) Algorithm input:

[0191] Combined with the 3D anatomical model generated by the image processing module, the coordinate information of muscles, nerves and blood vessels is obtained in real time to form a 3D anatomical model. The data is fused with the image data to generate a dynamically updated 3D spatial model to ensure that the path planning is based on the real-time anatomical structure.

[0192] (2) Dynamic path planning:

[0193] A deep learning-based path planning algorithm is used to calculate the optimal needle path in real time, taking into account the current local limb status and changes in anatomical structure.

[0194] (a) Genetic algorithm optimization implementation: The initial path population is randomly generated, and each path consists of a series of points (nodes) that define the dynamic trajectory of the needle in three-dimensional space.

[0195] Fitness function definition:

[0196] F(p)=w1.L(p)+w2·R(p)+w3·τ(p)

[0197] Among them, L(p) is the path length, R(p) is the tissue damage risk, A(p) is the rationality of the incident angle, and w1, w2, and w3 are weight coefficients.

[0198] Selection, crossover and mutation:

[0199] Selection: Roulette wheel selection is used to select paths based on fitness.

[0200] Crossover: Generate new paths using single-point crossover or multi-point crossover.

[0201] Mutation: Randomly change certain points in the path to increase diversity.

[0202] The iterative operation is performed until the set number of iterations or fitness threshold is met.

[0203] (b) Ant colony algorithm optimization implementation:

[0204] Pheromone initialization: Initialize the pheromone concentration of each path to a small value to ensure that the initial paths have a certain amount of exploration opportunities.

[0205] Path selection probability:

[0206] The probability of choosing a path at each node:

[0207]

[0208] Among them, τ ij is the pheromone concentration, η ij is the heuristic function (the inverse of the path length), and α and β are adjustment parameters.

[0209] (c) Pheromone update: After each iteration, update:

[0210] τ ij =(1-ρ)·τ ij +Δτij

[0211] Where ρ is the volatility coefficient, Δτ ij Additional pheromones for the path.

[0212] (d) Path evaluation and selection: The fitness of candidate paths is evaluated, and the path with the highest fitness is selected and updated to the AR interface in real time to provide navigation for anesthesiologists / surgeons.

[0213] Step 4: Real-time feedback and monitoring mechanism

[0214] Needle displacement monitoring: Use an optical tracking system or IMU sensor to monitor the needle position in real time to ensure that the needle is always on the optimal path.

[0215] (1) Error feedback mechanism:

[0216] Calculates the deviation between the needle's current displacement and the optimized path and provides instant audible and visual warnings when the deviation exceeds a threshold.

[0217] Compare the real-time monitored needle position to the optimized path:

[0218] (a) Calculation deviation:

[0219]

[0220] Among them, (x t ,y t , z t ) is the real-time needle coordinate, (x p ,y p , z p ) are the coordinates of the target point on the path.

[0221] When the deviation exceeds the set threshold, the system automatically issues audible and visual alarms.

Claims

1. A method for realizing intelligent nerve positioning based on local muscle group recognition, characterized in that: The following steps are involved: Step 1: Dynamic image processing and feature extraction, further including: Step 101: Get dynamic image: Use high frame rate and high resolution real-time ultrasound equipment to obtain dynamic images of local limb anatomical structures; Step 102, image enhancement and feature extraction, includes the following steps: Step 1021, local noise statistics calculation: Use a sliding window of 3*3 to 5*5 to process the dynamic image and calculate the mean and variance within the window, as shown in the following formula: Where f(x,y) is the pixel intensity value at the image coordinate (x,y), k is the estimated value of the noise, M is the total number of pixels in the sliding window, E{f(x,y)} is the mean, and Var{f(x,y)} is the variance; Step 1022: Adaptively adjust the noise estimation value k according to the local variance through a dynamic window adjustment strategy, thereby optimizing the noise filtering effect; Step 1023, using contrast limited adaptive histogram equalization (CLAHE) for contrast enhancement; Step 1024: Use a deep learning model to perform image analysis, and use YOLO and Mask R-CNN to perform real-time feature detection and segmentation; Step 103: Dynamic 3D reconstruction: Using continuous ultrasound images and dynamic capture data, combined with computer vision technology, a dynamic local limb three-dimensional model is generated to reflect the anatomical changes of the local limb in real time, including the following steps: Step 1031, extracting a disparity map from the multiple registered images, and calculating a three-dimensional point cloud based on the disparity; Step 1032, voxel grid: For scenes with slight dynamic displacement, the point cloud is voxelized using the voxel grid method to reduce noise and computational complexity; Step 104: Dynamic update and rendering: Use the dynamically captured data to update the point cloud and 3D model, and combine the Kalman filter to predict the dynamic state and compensate for dynamic changes; Step 2: Local muscle group identification, further including: Step 201, muscle group image segmentation, including: Step 2011, data set construction and enhancement: using a domain-specific high-quality ultrasound image set and using data enhancement technology to expand the training set; Step 2022, build an improved U-Net architecture: introduce an attention mechanism based on the U-Net network, as shown in the following formula: Where Q, K, and V are query, key, and value respectively. k The dimension of the key. Step 223, deep supervised learning: Introduce a deep supervision mechanism in each downsampling layer of U-Net and adopt a multi-scale loss function. The formula is as follows: in, is the l-layer segmentation loss, λ l is the weight coefficient of each layer; Step 202, complex feature extraction and representation learning of muscle groups, including: Step 221, feature pyramid network and residual connection: process image features of different sizes through the FPN structure, combine residual learning, and use Skip Connection to reduce the risk of gradient disappearance; Step 2022, graph convolutional network: after feature extraction, further apply graph convolutional network to model the complex relationship between the muscle group and the nerves, wherein local structure modeling is achieved by constructing an adjacency matrix and a feature matrix; Step 2023, multi-task learning: simultaneously learning muscle group segmentation and nerve localization, and improving the model's ability to understand different tasks by sharing feature extraction layers; Step 2024, deep segmentation and classification: at the output end of the model, a deep segmentation head is set to predict the muscle group category to which each pixel belongs, and a weighted cross entropy loss function is used to optimize the segmentation accuracy; Apply the Softmax function on the segmentation result of each pixel and map the output into the probability distribution of different muscle groups; Step 203, efficient nerve position determination, includes: Step 2031, spatial structure inference: combining anatomical knowledge and using graph-based deep learning methods to establish a spatial relationship diagram of muscle groups and nerves; Step 2032, regression-based nerve position prediction: using a deep regression network, taking into account the shape, texture and relative position of the local muscle group, and outputting the three-dimensional coordinates of the nerve; Step 2033, error feedback and optimization adjustment: using a feedback mechanism based on Bayesian optimization to dynamically adjust model parameters and network structure to reduce prediction errors; Introducing a spatial transformation network to perform geometric transformation on the input image, improving the model's robustness to changes in anatomical structure position and improving prediction accuracy; Step 3, intelligent path optimization and navigation algorithm, further comprising: Step 301, algorithm input: Combined with the generated 3D anatomical model, the coordinate information of muscles, nerves and blood vessels is obtained in real time to form a 3D anatomical model, and the data is fused with the image data to generate a dynamically updated 3D spatial model to ensure that the path planning is based on the real-time anatomical structure; Step 301: Dynamic path planning: A deep learning-based path planning algorithm is used to calculate the optimal needle path in real time, taking into account the current local limb status and changes in anatomical structure; Step 4: Real-time feedback and monitoring mechanism: Needle displacement monitoring: Use an optical tracking system or IMU sensor to monitor the needle position in real time to ensure that the needle is always on the optimal path, further including: Step 401, error feedback mechanism: Calculates the deviation between the needle's current displacement and the optimized path and provides instant audible and visual warnings when the deviation exceeds a threshold.

2. The method for realizing intelligent nerve positioning based on local muscle group recognition according to claim 1, characterized in that: In step 101, the ultrasound probe is optimized and selected according to different anatomical structures and dynamic states, and multiple imaging modes are applied to perform ultrasound imaging of specific areas to obtain static image data from multiple angles to ensure coverage of target muscle groups, nerves and blood vessels.

3. The method for realizing intelligent nerve positioning based on local muscle group identification according to claim 1, characterized in that: In step 1022, the noise estimation value k is adaptively adjusted using the following formula: k=max(σ min ,α·Var{f(x,y)}) Among them, σ min is the minimum noise threshold, and α is an adjustable coefficient.

4. The method for realizing intelligent nerve positioning based on local muscle group recognition according to claim 1, characterized in that: In step 1023, the image is divided into overlapping small blocks, and histogram equalization is performed independently on each small block, and the contrast is limited to avoid over-enhancement: Among them, H(i) is the frequency of occurrence of pixel intensity value i, that is, the height of the histogram, N is the total number of pixels in the image, CDF is the cumulative distribution calculated from intensity 0 to k, which represents the proportion of all pixels at and below the current gray level, and represents the cumulative probability distribution in the histogram equalization process; The processed small blocks are merged into a global image, and Gaussian blur or bilateral filtering is applied to smooth the edges to eliminate obvious visual differences at the seams.

5. The method for realizing intelligent nerve positioning based on local muscle group identification according to claim 1, characterized in that: In step 1024, the YOLO is subjected to: Network architecture optimization, including: introducing the CSPNet network, which divides the feature map into two parts and transmits them separately, reduces the amount of calculation through cross-stage connections while maintaining high accuracy; Replace the standard convolution with a depth-wise separable convolution, where the depth-wise separable convolution is divided into two stages: depth-wise convolution and point-wise convolution, which are calculated as follows: Y final =W point ·Y out Where W depth is the convolution kernel in the deep convolution, the weight matrix, X is a multi-channel two-dimensional matrix containing the input information of the convolution layer, W depth,k is the convolution kernel used for the kth channel, X k is the kth channel in the input data, W point is the convolution kernel in the point convolution, Y final is the final output result of the depth-separable convolution; The SE module is introduced in the feature extraction layer to adaptively adjust the weights of channel features through global information, thereby enhancing the representation ability of important features; Anchor box optimization includes: using the K-Means clustering algorithm to analyze the target box size in the training set and automatically generate an anchor box size suitable for the data set, which is achieved through the following steps: Calculate bounding box size: extract the width and height of each object in the training set; Apply K-Means clustering: perform K-Means clustering on the extracted sizes and choose a suitable K value to maximize the IoU of each anchor box with its corresponding object. Loss function optimization, including: YOLO uses a combined loss function to simultaneously optimize the coordinates, confidence, and category probabilities of the bounding box: L=L coord +λ obj L obj +λ nobj L nobj +λ class L class Where, L coord Please add L for optimizing the bounding box coordinates in the loss function. obj To optimize the confidence loss of the target object, L nobj To optimize the confidence loss of the target object, L class To optimize the loss of class probability, λ obj , nobj , class are the weight coefficients of different parts in the loss function.

6. The method for realizing intelligent nerve positioning based on local muscle group identification according to claim 1, characterized in that: In step 104, OpenGL is used to render the point cloud and the three-dimensional model in real time, and the rendering process is accelerated by the GPU to improve the rendering efficiency and smoothness.

7. The method for realizing intelligent nerve positioning based on local muscle group identification according to claim 1, characterized in that: In step 2031, spatial feature modeling is implemented through a graph neural network, and the formula is as follows: in: is the neighbor set of node i, W is the trainable weight matrix, is the feature representation of neighbor node j in the kth layer.

8. The method for realizing intelligent nerve positioning based on local muscle group identification according to claim 1, characterized in that: In step 2032, the regression network structure is: in: is the feature map, θ is the model parameter, and b is the bias term or intercept term.

9. The method for realizing intelligent nerve positioning based on local muscle group identification according to claim 1, characterized in that: In step 401, the deviation is calculated using the following formula: Among them, (x t ,y t , z t ) is the real-time needle coordinate, (x p ,y p , z p ) is the coordinate of the target point on the path; When the deviation exceeds the set threshold, the system automatically issues audible and visual alarms.

Images