Multi-objective segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus image

Abstract

The application discloses a multi-target segmentation method based on multi-scale discrete optimization of ultrasonic brachial plexus nerve images, and relates to the technical field of image processing. The multi-target balance and discrete point optimization semantic segmentation network comprises a down-sampling feature coding module, a target balance attention module, a category information supervision point vector re-classification module and an up-sampling multi-scale fusion module. The target balance attention module can dynamically adjust the attention weight of different targets on the feature map according to the features and context information of the targets. The category information supervision point vector re-classification module can re-predict the local pixel points in the non-obvious area of the image, and optimize the fuzzy area such as the boundary through the supervision of the category information. The up-sampling multi-scale fusion module integrates multi-scale information, and enhances the global information in the up-sampling process. The application can efficiently and accurately simultaneously segment four anatomical tissues in the ultrasonic brachial plexus nerve image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically, to a multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images. Background Technology

[0002] Nerve regions are not readily apparent structures in images, and the target outline is unclear, making it difficult for surgeons to pinpoint the precise location of the nerves for surgery. Automated segmentation of brachial plexus ultrasound images to obtain the specific location of multiple anatomy features is crucial for surgical success.

[0003] To assist doctors in identifying anatomical tissues in brachial plexus ultrasound images, several machine learning methods can be used to guide surgical anesthesia. Threshold-based segmentation compares pixel values ​​in the ultrasound image with a set threshold to divide the image into different regions. This method is simple and easy to implement, but it is sensitive to noise and lighting changes. Edge detection-based methods segment by detecting edges in the ultrasound image. Common edge detection algorithms include the Sobel operator and the Canny operator. This method is robust to noise and lighting changes, but it is prone to missegmentation when weak edges or complex backgrounds are present. In these traditional machine learning methods, feature processing and representation are overly simplistic, relying on manually designed features and rules, and requiring high-quality ultrasound images. Segmentation results often suffer from inaccuracies and instability, failing to extract useful information for identification. Currently, deep learning algorithms such as convolutional neural networks are widely used in medical and natural image analysis. Extracting image features through convolution not only saves workload but also improves prediction accuracy.

[0004] Multi-object segmentation refers to the simultaneous segmentation of multiple objects in an image and the determination of the semantic category of each region. This enables a bottom-up semantic reasoning process, ultimately yielding a segmented image with pixel-level semantic annotations. Instance segmentation and semantic segmentation are two popular multi-object segmentation methods in recent years. For example, in object segmentation, methods based on the Mask R-CNN meta-architecture perform well in many object segmentation tasks, first detecting and segmenting different objects within bounding boxes. Due to the development of transformers in various NLP tasks, the recent work ISTR uses a recursive refinement strategy for detection and segmentation, avoiding the decomposition of instance segmentation into a dependent task. Researchers proposed the PoolFormer network, which improves the performance of transformer models by replacing the attention module in the transformer with a spatial pooling operator. Compared to multi-instance segmentation, multi-object segmentation tasks do not concern themselves with the classification of specific instances, but rather with the classification of image pixels. To address this problem, encoder-decoder structures such as Fully Convolutional Neural Networks (FCN), U-Ne, and Deeplab have been proposed. Transformers have also been introduced into encoder-decoder structures to recover local spatial information, achieving superior performance in multi-organ segmentation. In existing research, PointRend is used to output clear object boundaries for more efficient segmentation of images of objects and scenes. Swin-Unet uses a hierarchical Swin Transformer with shifted windows as an encoder to extract contextual features in multi-organ (synapse) and heart (ACDC) segmentation tasks.

[0005] For brachial plexus ultrasound images containing targets of different scales, they can be divided into three main categories: small targets (neural structures), medium targets (arteries and veins), and large targets (muscle tissue). Muscle and nerves account for 12.97% and 0.93% of the training dataset, respectively, further confirming the problem of target size disparity and class imbalance. This phenomenon leads neural networks to focus more on information from large targets and less on small targets when processing these images. The key to addressing this scale difference among multiple targets is achieving target scale balance to ensure accurate capture of boundary and detail information of targets at different scales during segmentation. Furthermore, the raw ultrasound images and labeled images often exhibit unclear edges in their unprocessed state, making the prediction of some discrete edges difficult.

[0006] In recent deep learning research, various techniques have been used to improve the segmentation performance of multiple targets in ultrasound images, such as segmenting the cardiac ultrasound region. For ultrasound images with well-defined anatomical targets and edges, relatively simple feature fusion and spatial / channel attention mechanisms can achieve good target localization and edge optimization. However, for the brachial plexus-specific dataset, there are issues such as dispersed anatomical tissues, varying target sizes, and irregular edges, making it difficult to achieve good results using these simple modules. Furthermore, achieving a balanced segmentation of large targets (muscle tissue) and small targets (nerves) is challenging. Therefore, it is necessary to address this challenge and accurately segment different tissue structures in brachial plexus images, aiming to provide more precise information for clinical diagnosis and treatment, and better reference for anesthesia clinical teaching. In summary, the limitations of current image segmentation methods include:

[0007] 1) Due to the difficulty of the brachial plexus in processing complex and unevenly distributed image structures, as well as low-resolution images, it is difficult to accurately locate targets from ultrasound images.

[0008] 2) In brachial plexus ultrasound images, various targets are scattered and vary in size. Ordinary deep learning methods are not effective, and attention mechanisms are concentrated in large target areas, making it impossible to reasonably optimize small target areas and balance the relationship between large and small targets.

[0009] To overcome these limitations, a novel approach is needed to balance segmentation attention to targets of different sizes, thereby improving the overall accuracy and reliability of ultrasound medical image segmentation. Summary of the Invention

[0010] To address the issues of different target scales, unbalanced categories, and unclear semantic features in multi-target ultrasound images, this invention provides a multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images. This method can efficiently and accurately segment four anatomical tissues in ultrasound images simultaneously: nerves, arteries, veins, and muscles.

[0011] The technical solution adopted in this invention is as follows:

[0012] This invention provides a multi-objective segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images. This method employs a multi-objective balanced and discrete-point optimized semantic segmentation network to segment ultrasound brachial plexus images. The multi-objective balanced and discrete-point optimized semantic segmentation network includes a downsampling feature encoding module, an object balancing attention module, a category information supervised point vector reclassification module, and an upsampling multi-scale fusion module. The method includes:

[0013] Preprocessing ultrasound brachial plexus images to match them with multi-objective balance and discrete point optimized semantic segmentation networks;

[0014] The preprocessed ultrasound brachial plexus nerve image was downsampled and encoded using a downsampling feature encoding module to obtain several layers of encoded feature maps.

[0015] The target equalization attention module is used to process the low-level feature map in the encoded feature map, and the output is a feature map with multi-scale equalization attention added.

[0016] The feature map with balanced attention is processed by the category information supervised point vector reclassification module, and the output is a discretely optimized feature map.

[0017] The upsampling multi-scale fusion module upsamples and fuses the feature map with multi-scale balanced attention and the feature map with discrete optimization. The final segmentation result is generated by the segmentation head using the feature map after multi-scale fusion.

[0018] In a preferred embodiment of the present invention, the target equalization attention module generates a coarse prediction result, i.e., a coarsely predicted feature map M, with the same height and width as the underlying feature map E5, by using a 1×1 convolutional kernel and softmax.

[0019] M = softmax(f 1×1 (E5))

[0020] F i =E5·M i

[0021] Where M represents a coarse prediction of the image, f 1×1 This represents a 1×1 convolutional layer. The coarsely predicted feature map M is then copied three times as supervision information and placed into four branches to enhance the semantic information of each branch. Within each branch, M is selected... i As the mask of the i-th object, the i-th augmented feature F is generated. i ;

[0022] The target balanced attention module includes a feature refinement module, which weights features along the channel dimension, allowing the model to focus more on important channel information. In each FR module, F... i The results after multiplying with the coarse feature map are max-pooled and mean-pooled, respectively. These are combined and processed through a 3×3 convolution and activation layer. The feature refinement module is defined as follows:

[0023]

[0024] Among them, f 3×3 This represents a 3×3 convolution operation, f A and f MThese represent max pooling and average pooling, respectively.

[0025] In a preferred embodiment of the present invention, the category information supervised point vector reclassification module includes a point selection module and a point regression module;

[0026] In the point selection module, upsampled feature maps are used as fine-grained features to achieve point selection. The confidence level formula for point prediction is defined as:

[0027] P u =|max(P coarse )-P threshold |

[0028] In the formula, P u Represents coarse image P coarse The distance between points, P threshold Indicates the classification threshold;

[0029] Suppose N points are selected from the coarse prediction map of multiple targets obtained from the upsampled feature map, then these N points will be re-predicted;

[0030] After generating N uncertain points, the point regression module is used to regress the classification result of each point.

[0031] In a preferred embodiment of the present invention, the point selection strategy of the point selection module includes oversampling and importance filtering, specifically:

[0032] 1) Oversampling: The point selection module randomly generates kN candidate points in the coarse prediction map;

[0033] 2) Importance screening: The importance of a point is judged by the maximum probability of the point. According to the confidence level formula of the point prediction, the uncertainty values ​​of the oversampled kN points are sorted, and the N points with the lowest importance are selected as candidate points for re-prediction.

[0034] In a preferred embodiment of the present invention, the process of regressing the classification result of each point using the point regression module includes: contacting the feature vector corresponding to the selected point with the five-channel probability vector of the coarse prediction result to construct a point-directed mixed feature vector; given the point-directed mixed feature vector of each selected point; using a multilayer perceptron to perform pixel-level prediction on each point-directed mixed feature vector; each multilayer perceptron sharing weights; and using a classification loss function to calculate and sum the classification loss of each point.

[0035] In a preferred embodiment of the present invention, in the upsampling multi-scale fusion module, D i This represents the i-th decoder feature map during the upsampling operation, where D represents the feature maps of different sizes. iWe use upsampling at different scales, then perform 3×3 convolutions, and resize them to a uniform size, as defined below:

[0036] D′ i =f 3×3 [f u (D i )]

[0037]

[0038] Among them, f u F represents the upsampling operation applied to feature maps of different scales. m The output is the fused multi-scale feature;

[0039] Then, the results of Avg-pool, Max-pool, and Soft-pool with multi-layer perception are combined to obtain the attention weights for each channel, which are defined as follows:

[0040]

[0041] W i (F m )=σ(θ(F m AP )+θ(F m MP )+θ(F m SP ))

[0042] Where Θ represents a multilayer perceptron, σ is the sigmoid activation function, and F m AP F m MP F m SP These represent the results of Average Pooling, Max Pooling, and Soft Pooling for the i-th channel of the feature map, respectively.

[0043] ′

[0044] Finally, the enhanced channel attention weight W will be output. i (F m ) and input features F m Connection is F m At this point, the feature map after multi-scale fusion can be used to generate the final segmentation result through the segmentation head.

[0045] In a preferred embodiment of the present invention, the multi-objective balanced and discrete-point optimized semantic segmentation network optimizes the network parameters through gradient descent, using the following loss function:

[0046] Ltotal =L c +L s +λL p

[0047] L s =(1-λ)L ce (X,Y)+λL Dice (X,Y)

[0048] L c =(1-λ)L ce (M,M l )+L Dice (M,M l )

[0049]

[0050] Where λ is the weight, L total For the total loss, L p For point prediction loss, L s For the final segmentation mapping loss, L ce (X,Y) represents the cross-entropy loss between the predicted result X and the true value Y, and L... Dice (X,Y) represents the DICE loss between the predicted result X and the true value Y, L ce (M,M l ) represents the coarse segmentation result M and the coarse segmentation result label M. l Cross-entropy loss, L Dice (M,M l ) represents the coarse segmentation result M and the coarse segmentation result label M. l DICE loss, L ce (X points ,Y points X is the prediction result of the discrete point set. points and corresponding discrete point label Y points Cross-entropy loss between them.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] 1) The Target Balanced Attention Module (TEAM) can dynamically adjust the attention weights of different targets on the feature map based on the features and context information of the target, thereby achieving personalized enhancement of the target and more accurately capturing and locating the target; the Category Information Supervised Point Vector Reclassification (CPR) module can re-predict local pixels in inconspicuous areas of the image and optimize blurred areas such as boundaries through supervised category information; the Upsampling Multi-Scale Fusion (UMF) module integrates multi-scale information and enhances the global information in the upsampling process, which ensures that the final output of the model fully reflects the feature reduction at each level in the decoding path.

[0053] 2) The method of this invention achieves the best performance on all evaluation metrics (DSC of nerves, DSC of muscles, DSC of arteries, and DSC of veins). At the same time, the method of this invention has far fewer parameters than some conventional multi-object segmentation methods (Mask-RCNN). Compared with real-time segmentation methods (SegFormer, STDC), the accuracy of this invention is much higher, which improves the efficiency of anesthesiologists in brachial plexus block anesthesia.

[0054] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, embodiments of the present invention are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0055] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart of the segmentation process of the multi-objective balanced and discrete-point optimized semantic segmentation network described in this invention.

[0057] Figure 2 This is a flowchart of the point vector reclassification module with category information supervision as described in this invention;

[0058] Figure 3 This is a flowchart of the upsampling multi-scale fusion module described in this invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0060] This invention discloses a multi-objective segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images. This method is implemented using a multi-objective balanced and discrete-point optimized semantic segmentation network, and is used to accurately segment anatomical structures in ultrasound images, such as... Figure 1 As shown.

[0061] The multi-objective balanced and discrete-point optimized semantic segmentation network is built on an encoder-decoder semantic segmentation architecture, which includes a downsampling feature encoding module, an objective balanced attention module (TEAM), a category-supervised point vector reclassification (CPR) module, and an upsampling multi-scale fusion (UMF) module.

[0062] 1. The downsampling feature encoding module adopts the Unet encoder structure and uses four downsampling blocks to perform convolution and pooling on the image, thereby reducing the size of the feature map, which helps to extract higher-level features and expand the receptive field. Its input is E1, which is downsampled four times to obtain E2, E3, E4 and E5, thus obtaining the lowest-level feature map E5.

[0063] 2. The Target Balanced Attention Module (TEAM) is used to process the low-level feature maps in the encoded feature maps and output a feature map E′5 with multi-scale balanced attention. After processing by the Target Balanced Attention Module, each type of target is enhanced and refined, enabling more equal attention to multi-scale targets. In this module, the object is decomposed into multiple branches and the features are refined using the coarse segmentation results.

[0064] Smaller feature maps are generally more sensitive to capturing high-level semantic information. This is because smaller size characteristics provide key cues about the target's location, size, and shape. On the other hand, larger feature maps may contain more irrelevant details and background noise, in addition to more contextual information. Therefore, the lowest-level feature maps are not only used for image information compression and extraction but also serve as an important reference for image restoration during upsampling. To ensure that each target receives different attention weights from its location and shape information, this invention proposes a TEAM module.

[0065] The TEAM module uses four different branches in the low-level feature map E5 to enhance four different anatomical structures to be segmented. Since the feature map contains the richest structural information, the downsampled low-level feature map E5 is not only used for image information compression and extraction, but also serves as an important reference for image restoration during the upsampling process.

[0066] The TEAM module uses four different branches to give each target different attention weights in order to enhance four different anatomical structures to be segmented.

[0067] Specifically, the structure of the TEAM module is as follows: Figure 1 As shown. Given the input encoder feature map E5, a coarse prediction result, i.e., a coarsely predicted feature map M, is generated by using a convolution kernel size (1×1) and softmax, which has the same height and width as the underlying feature map E5 (32×32).

[0068] M = softmax(f 1×1(E5))

[0069] F i =E5·M i

[0070] Where M represents a coarse prediction of the image, f 1×1 This represents a 1×1 convolutional layer. The coarsely predicted feature map M is then copied three times as supervision information and placed into four branches to enhance the semantic information of each branch. In each branch, M is selected... i As the mask of the i-th object, the i-th augmented feature F is generated. i .

[0071] However, using a coarse prediction map for simple multiplication only enhances the probability of each channel in the feature map to the same level, failing to refine the features on each channel. To refine the feature map F... i Further refinement is needed; the TEAM module design includes... Figure 1 The feature refinement module (FR) in the model weights features along the channel dimension, allowing the model to focus more on important channel information. Within each FR module, F... i The results of multiplying with the coarse feature (i.e., the low-level feature map E5) are max-pooled and mean-pooled, respectively. These are then combined and processed through a 3×3 convolution and activation layer. The feature refinement module is defined as follows:

[0072]

[0073] Among them, f 3×3 This represents a 3×3 convolution operation, f A and f M These represent max pooling and average pooling, respectively. Finally, the feature maps refined by multiple branches are linearly added together to obtain the enhanced feature map E′5.

[0074] 3. The Category Information Supervised Point Vector Reclassification (CPR) module processes the feature map with balanced attention, performs discrete point optimization with brachial plexus category information supervision, and outputs a discretely optimized feature map. This improves the prediction results in uncertain regions, enhances the neural network's learning ability for ultrasound images, and reclassifies each pixel by supervising the category information, thus solving the problem of inaccurate pixel classification caused by unclear semantic information.

[0075] CNN-based image segmentation methods are widely used to satisfy the translation invariance requirement, enabling the network to preserve invariant features and positional information of the image. However, by sampling the predicted values ​​of different targets separately, the image can be treated as a series of discrete points without positional information, effectively transforming the overall image segmentation into a discrete vector classification problem, thus redefining the image segmentation problem from a new perspective. Since discrete vectors are uncorrelated, the influence of positional relationships between different targets is reduced during feature map refitting, focusing only on the classification result of that pixel, thereby improving the accuracy of the final result. Furthermore, the expected probability of pixels is often unevenly distributed in pixel-by-pixel classification. Near the threshold edge where ultrasonic image noise exists, a large number of pixels appear at the target edge. Image segmentation with continuously distributed pixels is difficult to optimize to discrete edge pixels. Therefore, this invention proposes a category-information-supervised point vector reclassification (CPR) module.

[0076] The CPR module aims to refine the boundaries of the target and also hopes to solve the problem of small targets being unable to be optimized during upsampling, such as... Figure 2 As shown. The Category Information Supervised Point Vector Reclassification (CPR) module is divided into two parts: a point selection module and a point regression module. The category supervision module selects upsampled feature maps and outputs a coarse feature map P. coarse For vector re-prediction, considering both efficiency and semantic information, D3 was ultimately chosen as the output of the coarse prediction map.

[0077] 1) In the point selection module, upsampling feature mapping is used as fine-grained features to achieve point selection. To quantify the uncertainty of pixel points, the confidence level formula for point prediction is defined as:

[0078] P u =|max(P coarse )-P threshold |

[0079] In the formula, P u Represents coarse image P coarse The distance between points, P threshold The confidence level of a point prediction indicates the classification threshold. The higher the confidence level of the prediction, the more obvious the predicted features are and the easier it is to be classified.

[0080] Suppose N points are selected from the coarse multi-object prediction map obtained from the upsampled feature map, and these N points will be re-predicted. Point selection strategies can be further divided into oversampling and importance filtering, as shown below:

[0081] (i) Oversampling: The point selection module randomly generates kN candidate points in the coarse prediction map, where k is the number of oversampled points defined by the user.

[0082] (ii) Importance Screening: The importance of a point is determined by its maximum probability. The uncertainty values ​​of the oversampled kN points are ranked according to the rules shown in the confidence level formula for point prediction. The N points with the lowest importance among the kN points are selected as candidate points for re-prediction.

[0083] 2) After generating N uncertain points, the prediction task is transformed into a reclassification task of a series of discrete points. Specifically, a point regression module is used to regress the classification result of each point. The feature vector corresponding to the selected point is coupled with the five-channel probability vectors (background classification probability and four anatomical tissues) of the coarse prediction result to construct a point-directed mixed feature vector. Given the point-directed mixed feature vector for each selected point, a multilayer perceptron (MLP) is used to perform pixel-level prediction for each vector. Similar to the vector classification task, MLP blocks share weights, and the classification loss for each point is calculated and summed using the standard classification loss. After supervised learning of the corresponding point features, the network can classify the coarse feature map more accurately, thus enabling the upsampling process to more accurately recover the feature map.

[0084] 4. The Upsampled Multi-Scale Fusion (UMF) module is used to upsample and fuse feature maps with added multi-scale balanced attention and discrete optimization. Then, the segmentation head generates the final segmentation result using the multi-scale fused feature map. The UMF module uses multi-scale information to segment the target, helping the network learn richer global information about the target of different sizes, thereby improving the accuracy of the segmentation result.

[0085] Multi-scale fusion is widely used in medical image segmentation. During upsampling, only the top-level features are needed to recover the final segmentation result, effectively recovering large targets in the image. However, directly predicting using the restored large feature map makes it difficult for the network to balance features at different scales to segment targets, especially small targets. To fuse more information obtained during downsampling, this invention proposes an upsampling multi-scale fusion (UMF) module to address the imbalance problem.

[0086] In the UMF module, D i Let D represent the i-th decoder feature map during the upsampling operation. Feature maps D of different sizes... i (D1, D2, D3, D4) are upsampled at different scales, then convolved with 3×3, and resized to a uniform size, as defined below:

[0087] D′ i =f 3×3 [f u (D i )]

[0088]

[0089] Among them, f u This represents the upsampling operation applied to feature maps at different scales. F m The output is the fused multi-scale feature.

[0090] Then, the results of average pooling, max pooling, and soft pooling with multilayer perception (MLP) are combined to obtain the attention weights for each channel, which are defined as follows:

[0091]

[0092] W i (F m )=σ(θ(F m AP )+θ(F m MP )+θ(F m SP ))

[0093] Where Θ represents a multilayer perceptron, σ is the sigmoid activation function, and F m AP F m MP F m SP These represent the results of Average Pooling, Max Pooling, and Soft Pooling for the i-th channel of the feature map, respectively.

[0094] Finally, the enhanced channel attention weight W will be output. i (F m ) and input features F m Connect as F′ m ,like Figure 3 As shown, at this point, the feature map after multi-scale fusion can be used to generate the final segmentation result through the segmentation head.

[0095] 5. Loss function for segmentation

[0096] This invention optimizes neural network parameters through backpropagation, gradually bringing the segmentation results closer to the correct outcome. It introduces cross-entropy loss to optimize image pixel classification, resulting in more accurate classification of pixels with clearly defined targets such as neurons. Furthermore, it introduces Dice coefficient loss to emphasize the similarity of predicted results, effectively addressing class imbalance. Combining cross-entropy loss and Dice loss in image segmentation leverages the advantages of both, improving both performance and stability. Simultaneously, in the segmentation of discrete pixels, cross-entropy loss is used to optimize pixel-level classification.

[0097] More specifically, used to compute the final segmentation map L s The loss function, which applies supervised loss to the final segmentation result, combines DSC loss and cross-entropy loss, where λ is a manually set weight. In the following text, all values ​​of λ are set to λ = 0.5 based on empirical values.

[0098] L s =(1-λ)L ce (X,Y)+λL Dice (X,Y)

[0099] In order to provide some degree of supervision over the coarse prediction result M, M l The ground truth value of the coarse feature map is the same as the final segmentation result, and it also uses a combination of two losses:

[0100] L c =(1-λ)L ce (M,M l )+L Dice (M,M l )

[0101] In addition, the cross-entropy loss is used to calculate the classification of discrete points in the CPR module, named the point prediction loss L. p .

[0102]

[0103] Finally, the total loss function of the network can be calculated. During backpropagation, the three loss functions simultaneously guide the network gradient descent, optimizing the parameters of each part of the network:

[0104] L total =L c +L s +λL p .

[0105] The experimental results of the multi-target segmentation method based on multi-scale discrete point optimization of ultrasound brachial plexus images described in this invention are as follows:

[0106]

[0107]

[0108] Illustration of metrics: In the experiment, three metrics—dice similarity coefficient (DSC), Jaccard index (JI), and recall (RC)—were used to evaluate the multi-object segmentation performance of different methods.

[0109]

[0110]

[0111]

[0112] Where A and B represent the predicted result and the true value, respectively, and TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative, respectively.

[0113] The multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images mentioned in the embodiments of this invention has the following advantages:

[0114] 1) Within the framework proposed in this invention, the Target Equalization Attention Module (TEAM) dynamically adjusts the attention weights of different targets on the feature map based on the target's features and contextual information, achieving personalized enhancement of the target and thus more accurately capturing and locating it. Building upon this, a Category Information-Supervised Point Vector Reclassification Module (CPR) is proposed to re-predict local pixels in inconspicuous areas of the image and optimize blurred regions such as boundaries using supervised category information. Furthermore, the Upsampling Multi-Scale Fusion Module (UMF) integrates multi-scale information, enhancing global information during the upsampling process. This ensures that the model's final output fully reflects the feature reduction at each level of the decoding path.

[0115] 2) The table presents a comparison of the segmentation results for the four targets (nerve, muscle, artery, and vein) with current classical and innovative methods. It is clear that the method proposed in this invention achieves the best performance across all evaluation metrics compared to other state-of-the-art methods: DSC of 0.536 for nerves, 0.682 for muscles, 0.872 for arteries, and 0.893 for veins. Furthermore, the method in this study has significantly fewer parameters than some conventional multi-target segmentation methods (Mask-RCNN), and its accuracy is much higher than real-time segmentation methods (SegFormer, STDC), thus improving the efficiency of anesthesiologists in brachial plexus block anesthesia.

[0116] 3) This implementation expands upon the private UBPD dataset, the first multi-object segmentation dataset based on the brachial plexus. The current UBPD dataset (now named UBPD 2023) is more than twice the size of its predecessor. It includes ground truth and labeled ultrasound images from 2020 patients with four anatomical tissues (arteries, veins, muscles, and nerves). This dataset expansion facilitates advancements in automated ultrasound-guided anesthesia and also aids in virtual simulation demonstrations aimed at improving the fundamental experimental and clinical skills of anesthesiologists.

[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images, characterized in that, This method employs a multi-objective balanced and discrete-point optimized semantic segmentation network to segment ultrasound brachial plexus images. The multi-objective balanced and discrete-point optimized semantic segmentation network includes a downsampling feature encoding module, an object-balanced attention module, a category-information-supervised point vector reclassification module, and an upsampling multi-scale fusion module. The method includes: Preprocessing ultrasound brachial plexus images to match them with multi-objective balance and discrete point optimized semantic segmentation networks; The preprocessed ultrasound brachial plexus nerve image was downsampled and encoded using a downsampling feature encoding module to obtain several layers of encoded feature maps. The target equalization attention module is used to process the low-level feature map in the encoded feature map, and the output is a feature map with multi-scale equalization attention added. The feature map with balanced attention is processed by the category information supervised point vector reclassification module, and the output is a discretely optimized feature map. The upsampling multi-scale fusion module upsamples and fuses the feature map with multi-scale balanced attention and the feature map with discrete optimization. The final segmentation result is generated by the segmentation head using the feature map after multi-scale fusion. The goal-balanced attention module adopts The size of the convolutional kernel and softmax generate the low-level feature map. The coarse prediction results with the same height and width, i.e., the coarsely predicted feature map M, M= softmax(f 1×1 (E5)) F i =E5·M i Where M represents a coarse prediction of the image, express The convolutional layer then copies the coarsely predicted feature map M three times as supervision information and feeds it into four branches to enhance the semantic information of each branch. Within each branch, a selection is made... As the first The nth object mask is used to generate the nth object mask. augmented features ; The target-balanced attention module includes a feature refinement module, which weights features along the channel dimension, allowing the model to focus more on important channel information. Within each feature refinement module... The results of multiplying with the coarse feature map are max pooling and average pooling, respectively. These are combined and processed through a 3×3 convolution and activation layer. The feature refinement module is defined as follows: F ′ =f 3×3 ([f A (M i ·F i )]⊕[f M (M i ·F i )])·F i Among them, f 3×3 This represents a 3×3 convolution operation, f A and f M These represent max pooling and average pooling, respectively. The category information supervised point vector reclassification module includes a point selection module and a point regression module; In the point selection module, upsampled feature maps are used as fine-grained features to achieve point selection. The confidence level formula for point prediction is defined as: In the formula, Represents a rough image Distance between points Indicates the classification threshold; Suppose N points are selected from the coarse prediction map of multiple targets obtained from the upsampled feature map, then these N points will be re-predicted; After generating N uncertain points, the point regression module is used to regress the classification result of each point.

2. The multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images according to claim 1, characterized in that, The point selection strategy of the point selection module includes oversampling and importance filtering, specifically: 1) Oversampling: The point selection module randomly generates kN candidate points in the coarse prediction map; 2) Importance screening: The importance of a point is determined by the maximum probability of the point. Based on the confidence level formula for point prediction, the uncertainty values ​​of the oversampled kN points are sorted, and the N points with the lowest importance among the kN points are selected as candidate points for re-prediction.

3. The multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images according to claim 1, characterized in that, The process of regressing the classification result of each point using the point regression module includes: connecting the feature vector corresponding to the selected point with the five-channel probability vector of the coarse prediction result to construct a point-to-point mixed feature vector; given the point-to-point mixed feature vector of each selected point, using a multilayer perceptron to perform pixel-level prediction on each point-to-point mixed feature vector; each multilayer perceptron sharing weights; and using a classification loss function to calculate and sum the classification loss of each point.

4. The multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images according to claim 1, characterized in that, In the upsampling multi-scale fusion module Indicates the first step of the upsampling operation. Each decoder feature map, feature maps of different sizes Use upsampling at different scales, and then perform... Convolution, resized to a uniform size, is defined as follows: in, This indicates upsampling operations applied to feature maps at different scales. The output is the fused multi-scale feature; Then, the results of Avg-pool, Max-pool, and Soft-pool with multi-layer perception are combined to obtain the attention weights for each channel, which are defined as follows: in, Represents a multi-layer sensing block. It is the sigmoid activation function. , , These represent the results of Average Pooling, Max Pooling, and Soft Pooling for the i-th channel of the feature map, respectively. Finally, the enhanced channel attention weights will be output. Input features Connection as At this point, the feature map after multi-scale fusion can be used to generate the final segmentation result through the segmentation head.

5. The multi-target segmentation method based on multi-scale discrete optimization of ultrasound brachial plexus images according to claim 4, characterized in that, Multi-objective balancing and discrete-point optimization semantic segmentation networks optimize network parameters through gradient descent, using the following loss function: in, As weight, For the total loss, For point prediction loss, For the final segmentation mapping loss, To calculate the cross-entropy loss between the predicted result X and the true value Y, To calculate the DICE loss between the predicted outcome X and the true value Y, For the coarse segmentation result M and the coarse segmentation result label Cross-entropy loss, For the coarse segmentation result M and the coarse segmentation result label DICE losses, Prediction results for discrete point sets and corresponding discrete point labels Cross-entropy loss between them.

Images