A method and system for automatic prompting ultrasound image segmentation based on SAM

Abstract

The application discloses a kind of automatic prompting ultrasound image segmentation method and system based on SAM, the method of the present application includes using ultrasound image segmentation network model to carry out ultrasound image segmentation, including CNN encoder, classification head module, low-level feature refining module, SAM's Transform encoder, cross-branch attention module, high-level feature enhancement module, pixel decoder, prompt encoder and mask decoder, pixel decoder decodes and generates preliminary segmentation prediction map, and mask decoder is based on the point, mask embedding vector generated by prompt encoder and the image embedding obtained by CNN encoder and Transform encoder Decoding obtains the final segmentation prediction map.The present application aims to solve the problem that most of the existing segmentation methods based on SAM rely on artificial prompting and have poor adaptability to ultrasound images, reduce the false segmentation affected by ultrasound noise and shadow, and improve the segmentation performance and generalization performance of the network.

Description

Technical Field

[0001] This invention relates to the field of image processing technology for ultrasound (US) images, and specifically to an automatic prompting ultrasound image segmentation method and system based on SAM. Background Technology

[0002] Ultrasound imaging, a non-invasive medical imaging technique, plays a crucial role in the diagnosis of various diseases due to its high safety, low cost, and real-time performance. Currently, ultrasound imaging is widely used in areas such as breast tumors, neurological disorders, thyroid nodules, and fetal detection. However, the common problems in ultrasound images, such as speckle noise, shadow artifacts, and signal attenuation, cause many segmentation methods to misclassify these areas as target regions, severely affecting the accuracy of segmentation results. In recent years, with the rapid development of deep learning technology, its application in medical image analysis has become a research hotspot, and many deep learning-based ultrasound image segmentation methods have emerged. These methods, through the design of corresponding models and training strategies, have demonstrated excellent performance in specific segmentation tasks, greatly promoting research progress in the field of ultrasound image segmentation. However, these methods are usually customized for specific tasks and have limited generalization ability to other ultrasound image segmentation tasks. Recently, the Segment Anything Model (SAM) has attracted widespread attention, significantly advancing the development of fundamental computer vision models. SAM has demonstrated excellent performance in natural image segmentation tasks and possesses the ability to generalize to new image distributions and tasks with zero samples. However, due to the scarcity of medical images in the training data for SAM, its zero-shot generalization ability in medical images is poor. Furthermore, SAM's semi-automatic segmentation mode, which relies on human prompts, not only interrupts the automatic segmentation process but also requires expert guidance, which is both time-consuming and impractical in clinical settings. Therefore, further research and innovation are needed to address SAM's dependence on human prompts and enable it to efficiently adapt to ultrasound image segmentation tasks. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide an automatic prompting ultrasound image segmentation method and system based on SAM, which addresses the above-mentioned problems in the prior art. This invention aims to solve the problem that most existing SAM-based segmentation methods rely on manual prompting and have poor adaptability to ultrasound images, reduce erroneous segmentation affected by ultrasound noise and shadows, and improve the segmentation performance and generalization performance of the network.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: An automatic prompting ultrasound image segmentation method based on SAM includes the following steps: Segmenting the input ultrasound image using an ultrasound image segmentation network model to obtain a segmentation prediction map. The ultrasound image segmentation network model includes a CNN encoder, a classification head module, a low-level feature refinement module, a SAM Transformer encoder, a cross-branch attention module, a high-level feature enhancement module, a pixel decoder, a prompting encoder, and a mask decoder. Segmenting the input ultrasound image to obtain the segmentation prediction map includes: extracting local detail features from the input ultrasound image using the CNN encoder and performing target existence classification using the classification head module; optimizing and enhancing the local detail features using the low-level feature refinement module and outputting the result to the pixel decoder; extracting global features from the input ultrasound image using the SAM Transformer encoder... After supplementing local detail features through the cross-branch attention module, an image embedding is obtained. Then, the high-level feature enhancement module uses multi-scale feature extraction to achieve semantic feature enhancement before outputting it to the pixel decoder. The pixel decoder performs the first stage of decoding to generate a preliminary segmentation prediction map. After thresholding the preliminary segmentation prediction map, a mask cue is generated. If the target existence classification result indicates the presence of a target, the top n points with the highest confidence in the preliminary segmentation prediction map are selected as positive cue points; otherwise, the top n points with the lowest confidence in the preliminary segmentation prediction map are selected as negative cue points. The positive or negative cue points are encoded by the cue encoder to generate point embedding vectors, and the mask cue points are encoded by the cue encoder to generate mask embedding vectors. The point embedding vectors, mask embedding vectors, and the image obtained after supplementing local detail features are embedded into the mask decoder to perform the second stage of decoding to obtain the final segmentation prediction map.

[0005] Optionally, the CNN encoder includes five convolutional modules (convolutional modules 1 to 5) connected in a cascaded manner. The local detail features output by the first three convolutional modules (convolutional modules 1 to 3) serve as inputs to the next-level convolutional modules in the CNN encoder, and also as inputs to corresponding low-level feature refinement modules. The local detail features output by convolutional module 5 serve as inputs to the classification head module for target presence classification. The SAM's Transformer encoder includes a multi-level window Transformer layer. The input ultrasound image is first divided into blocks by an image block embedding layer, and then combined with position encoding before being used as input to the multi-level window Transformer layer. The process of supplementing local detail features through a cross-branch attention module to obtain image embedding includes: normalizing the global features extracted by the multi-level window Transformer layer and inputting them to the cross-branch attention module; and using the local detail features output by the CNN encoder as keys and values, and the global features as queries, performing cross-attention. The image embedding is obtained by performing force operations and extracting attention features through a multi-head attention mechanism. The extracted attention features are added to the global features to obtain fused features. The local detail features output by the CNN encoder are extracted through multiple single convolutional modules. The fused features are then normalized by layers, processed by a multi-layer perceptron, added to the original fused features, and then added to the features extracted by multiple single convolutional modules to obtain image embedding with supplemented local detail features. During training, the Transformer encoder of the SAM uses a low-rank adaptive fine-tuning method to fine-tune the Transformer encoder, including: introducing trainable low-rank bypasses into the query and value projection layers in the multi-level window Transformer layer and the multi-head attention mechanism. The low-rank bypasses consist of two low-rank matrices and an activation function to perform low-rank decomposition on parameter updates. The low-rank matrices are linear layers. During training, only the parameters of the low-rank bypasses are optimized, while the weights of the original query and value projection layers are frozen. During the inference phase, the low-rank matrices are added to the original weights of the query and value projection layers and merged into new weights.

[0006] Optionally, the step of using the low-level feature refinement module to optimize and enhance local detail features before outputting them to the pixel decoder includes: taking local detail features as input feature maps, extracting features from the input feature maps using a channel attention mechanism, and passing the input feature maps in parallel through average pooling and max pooling operations with a kernel size of 2×2, and average pooling and max pooling operations with a kernel size of 8×8, respectively. Then, the results of the four pooling operations are upsampled to the same resolution and concatenated along the channel dimension. A 1×1 convolution module is then used to generate a refined feature map. The refined feature map and the residual of the original input feature map are concatenated, and then concatenated along the channel dimension with the features extracted by the channel attention mechanism. Finally, a 1×1 convolution module is used to generate the final output feature map and output it to the pixel decoder.

[0007] Optionally, the step of using the advanced feature enhancement module to achieve semantic feature enhancement through multi-scale feature extraction and then outputting it to the pixel decoder includes: embedding the image as an input feature map, extracting features from the input feature map using a channel attention mechanism, and passing the input feature map in parallel through four dilated convolution branches used to capture features at different scales in the spatial dimension; the four dilated convolution branches are as follows: the first dilated convolution branch consists of three 3×3 dilated convolutions with dilation rates of 1, 3, and 5, and a 1×1 convolution; the second dilated convolution branch consists of two 3×3 dilated convolutions with dilation rates of 1 and 3, and a 1×1 convolution; the third dilated convolution branch consists of a 3×3 dilated convolution with dilation rate of 3 and a 1×1 convolution; the fourth dilated convolution branch consists of a 3×3 dilated convolution with dilation rate of 1; the features extracted by the channel attention mechanism and the features extracted by the four dilated convolution branches are concatenated along the channel dimension to generate the final output feature map and output to the pixel decoder.

[0008] Optionally, the pixel decoder includes three pixel decoding modules. Each pixel decoding module consists of a transposed convolution, a channel dimension connection module, and two convolution modules. The convolution module is composed of a 3×3 convolutional layer, a batch normalization layer, and a GELU activation function connected in sequence. The output features of the advanced feature enhancement module are used as the input feature map of the pixel decoder and input to the transposed convolution of the first layer to obtain a double-upsampled feature map. The shallow features output by the first low-level feature refinement module are connected to the double-upsampled feature map through the channel dimension connection module and then passed through the convolution module to obtain the output feature map of the first layer. The output feature map of the first layer is transposed convolution of the second layer to obtain a quadruple-upsampled feature map. The shallow features output by the second low-level feature refinement module are connected to the quadruple-upsampled feature map through the channel dimension connection module and then passed through the convolution module to obtain the output feature map of the second layer. The output feature map of the second layer is transposed convolution of the third layer to obtain an eight-upsampled feature map. The shallow features output by the third low-level feature refinement module are connected to the eight-upsampled feature map through the channel dimension connection module and then passed through the convolution module to obtain the output feature map of the third layer. The output feature map of the third layer is then passed through a 1×1 convolutional layer and a sigmoid activation function to obtain a preliminary segmentation prediction map.

[0009] Optionally, the mask decoder includes two mask decoding units, a multi-level transposed convolution module, an anti-cross attention module, a multilayer perceptron, and a dot product module. Each mask decoding unit includes a self-attention module, an anti-cross attention module, a perceptron, and a cross attention module connected sequentially. The dot embedding vector serves as the input to the self-attention module in the first mask decoding unit. The image embedding and mask embedding vectors obtained after supplementing local detail features are concatenated and used as the input to the anti-cross attention module and cross attention module in the first mask decoding unit. The output of the perceptron in the first mask decoding unit serves as the input to the self-attention module in the second mask decoding unit, and the output of the cross attention module in the first mask decoding unit serves as the input to the self-attention module. The inputs to the anti-cross attention module and cross attention module in the second mask decoding unit are used as inputs. The output of the perceptron in the second mask decoding unit serves as the first input of the anti-cross attention module outside the two mask decoding units, and the output of the cross attention module in the second mask decoding unit serves as the second input of the anti-cross attention module outside the two mask decoding units, as well as the input of the multi-level transposed convolution module. The anti-cross attention module outside the two mask decoding units performs anti-cross attention on the two inputs and then extracts features through the multi-level perceptron as the first input of the dot product module. The output of the multi-level transposed convolution module serves as the second input of the dot product module. The dot product module performs dot product operation on the two inputs to obtain the final segmentation prediction map.

[0010] Optionally, during training, the pixel decoder of the ultrasound image segmentation network model uses a binary cross-entropy loss function, the expression of which is: , in, For binary cross-entropy loss, For the sample size, The label for the i-th pixel value. This is the initial segmentation prediction map for the i-th pixel value; the loss function used by the mask decoder is a weighted sum of focus loss and dice loss, and the functional expression for the weighted sum of focus loss and dice loss is: , ,

[0011] in, The weighted sum of focus loss and dice loss. As a balance factor between focus loss and dice loss, For the focus of loss, Loss due to dice, As a positive and negative sample balance factor, This is the adjusted predicted probability for the value of the i-th pixel. Let be the final predicted probability of the i-th pixel value.

[0012] Furthermore, the present invention also provides an SAM-based automatic prompting ultrasound image segmentation system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method.

[0013] Furthermore, the present invention provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method by a processor.

[0014] Furthermore, the present invention also provides a computer program product, including a computer program or instructions that are programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method via a processor.

[0015] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: 1. The SAM-based automatic prompting ultrasound image segmentation network model of the present invention designs a dual-branch encoder and a two-stage decoding process. The dual-branch encoder can effectively extract local detail features and global dependencies, and the two-stage decoding process automatically generates prompts and embeds them into the guidance model to achieve accurate segmentation, effectively applying SAM to the field of ultrasound image segmentation.

[0016] 2. The automatic prompting design of the ultrasound image segmentation network model based on SAM in this invention solves the problem of SAM relying on manual prompting. Furthermore, the positive and negative point prompting based on CNN classification knowledge effectively reduces the erroneous segmentation caused by ultrasound noise and shadows. Mask prompting can also provide more detailed contextual information. The combination of the two types of automatic prompting improves the information capture efficiency of the input image and can enhance the ultrasound image segmentation performance of the ultrasound image segmentation network model.

[0017] 3. The high-level feature enhancement module and low-level feature refinement module of this invention effectively improve the quality of the initial segmentation prediction map and the accuracy of automatic prompting. The high-level feature enhancement module aims to capture multi-scale semantic features, while the low-level feature refinement module plays a key role at transition connections by refining the shallow features of CNN branches. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the network structure of the ultrasound image segmentation network model in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the network structure of the advanced feature enhancement module in an embodiment of the present invention.

[0021] Figure 4 This is a schematic diagram of the network structure of the low-level feature refinement module in an embodiment of the present invention.

[0022] Figure 5 This is a schematic diagram of the network structure of the first-stage pixel decoder in an embodiment of the present invention.

[0023] Figure 6 This is a schematic diagram of the network structure for the second stage decoding process in an embodiment of the present invention.

[0024] Figure 7The images shown are segmentation results of BUSI ultrasound images obtained by different segmentation methods in this embodiment of the invention. (a-1) to (e-1) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the first sample, respectively; (a-2) to (e-2) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the second sample, respectively; and (a-3) to (e-3) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the third sample, respectively.

[0025] Figure 8 These are the incorrect segmentation results of normal (lesion-free) images in the BUSI dataset by different existing segmentation methods in the embodiments of the present invention. Among them, (a1) to (e1) are the first set of segmentation prediction maps obtained by using the U-Net method, AttU-Net method, SAMed method, SAMUS method and H-SAM method, respectively, and (a2) to (e2) are the second set of segmentation prediction maps obtained by using the U-Net method, AttU-Net method, SAMed method, SAMUS method and H-SAM method, respectively.

[0026] Figure 9 The images shown are the segmentation results of UNS ultrasound images obtained by different segmentation methods in this embodiment of the invention. (a-1) to (e-1) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the first sample, respectively; (a-2) to (e-2) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the second sample, respectively; and (a-3) to (e-3) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the third sample, respectively.

[0027] Figure 10The images shown are the segmentation results of STU ultrasound images obtained by different segmentation methods in this embodiment of the invention. (a-1) to (e-1) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the first sample, respectively; (a-2) to (e-2) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the second sample, respectively; and (a-3) to (e-3) are the segmentation prediction images and label images obtained by the U-Net method, TransUNet method, H-SAM method, and the method of this embodiment for the third sample, respectively. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0029] like Figure 1 and Figure 2As shown, the automatic prompting ultrasound image segmentation method based on SAM in this embodiment includes the following steps: The input ultrasound image is segmented using an ultrasound image segmentation network model to obtain a segmentation prediction map. The ultrasound image segmentation network model includes a CNN encoder, a classification head module, a low-level feature refinement module, a SAM Transformer encoder, a cross-branch attention module, a high-level feature enhancement module, a pixel decoder, a prompting encoder, and a mask decoder. The step of segmenting the input ultrasound image to obtain the segmentation prediction map includes: extracting local detail features from the input ultrasound image using the CNN encoder and performing target existence classification using the classification head module; optimizing and enhancing the local detail features using the low-level feature refinement module and then outputting the results to the pixel decoder; and extracting global features from the input ultrasound image using the SAM Transformer encoder. The image embedding is obtained by supplementing local detail features through a cross-branch attention module. Then, a high-level feature enhancement module is used to enhance semantic features through multi-scale feature extraction before outputting it to the pixel decoder. The pixel decoder performs the first stage of decoding to generate a preliminary segmentation prediction map. This preliminary segmentation prediction map is thresholded to generate a mask cue. If the target existence classification result indicates the presence of a target, the top n points with the highest confidence in the preliminary segmentation prediction map are selected as positive cue points; otherwise, the top n points with the lowest confidence are selected as negative cue points. The positive or negative cue points are encoded by the cue encoder to generate point embedding vectors, and the mask cue points are encoded by the cue encoder to generate mask embedding vectors. The point embedding vectors, mask embedding vectors, and the image obtained after supplementing local detail features are embedded into the mask decoder to perform the second stage of decoding, obtaining the final segmentation prediction map. In this embodiment, the CNN encoder and the SAM's Transformer encoder serve as the CNN encoding branch and Transformer encoding branch, respectively, forming a dual-branch encoder. The pixel decoder and mask decoder together constitute a two-stage decoding process. The decoding stage includes: inputting the semantic features processed by the high-level feature enhancement module and the shallow features processed by the low-level feature refinement module into the pixel decoder to generate a preliminary segmentation prediction map, and generating automatic prompts based on this map; inputting the prompt embedding processed by the prompt encoder and the image embedding output by the dual-branch encoder into the mask decoder to generate the final segmentation prediction map.

[0030] like Figure 2As shown, the CNN encoder in this embodiment includes five convolutional modules, from convolutional module 1 to convolutional module 5, connected in a cascaded manner. The local detail features output by the first three convolutional modules (1-3) serve as inputs to the next-level convolutional modules in the CNN encoder, and also as inputs to corresponding low-level feature refinement modules. The local detail features output by convolutional module 5 serve as inputs to the classification head module for object presence classification. As an optional implementation, the CNN encoder in this embodiment uses an EfficientNetV2-S network, which includes seven stages, from stage 0 to stage 7. Stages 0-1 correspond to... Figure 2 Convolution module 1, stage 2 corresponds to Figure 2 Convolutional module 2 and stage 3 in the middle correspond to Figure 2 Convolution module 3, stages 4-5 correspond to Figure 2 Convolutional module 4 and stage 6 in the middle correspond to Figure 2 Convolution module 5 and stage 7 in the middle correspond to Figure 2 The classification head module is used. Alternatively, other CNN encoder structures can be employed as needed. In this embodiment, the CNN encoder is pre-trained on the target presence classification task. During segmentation training, to improve training efficiency and ensure the stability of the classification results, the CNN encoder in this embodiment is kept in a frozen state.

[0031] like Figure 2As shown, the SAM Transformer encoder in this embodiment includes a multi-level window Transformer layer. The input ultrasound image is first divided into blocks by the image block embedding layer and then combined with position encoding as the input to the multi-level window Transformer layer. The step of obtaining the image embedding by supplementing local detail features through the cross-branch attention module includes: normalizing the global features extracted by the multi-level window Transformer layer and then inputting them into the cross-branch attention module. The cross-branch attention module is used to inject the local detail features extracted by the CNN coding branch into the Transformer coding branch to solve the problem of insufficient local features in the Transformer branch. The cross-branch attention module achieves feature fusion in the following way: It uses local detail features from the CNN encoder output as keys and values, and global features as queries to perform cross-attention operations. Then, it extracts attention features through a multi-head attention mechanism. The extracted attention features are added to the global features to obtain the fused features. The local detail features from the CNN encoder output are extracted through multiple single convolutional modules. The fused features are then normalized through layers, processed by a multi-layer perceptron, added to the original fused features, and finally added to the features extracted by multiple single convolutional modules to obtain the image embedding with supplemented local detail features. The SAM model's excellent segmentation performance is mainly due to its powerful image encoder architecture, which gives it excellent feature extraction and cross-domain generalization capabilities. However, directly training this image encoder is computationally expensive. To achieve efficient fine-tuning of the SAM's Transformer encoder, such as... Figure 2 As shown, in this embodiment, the SAM Transformer encoder is fine-tuned using a low-rank adaptive fine-tuning method during training. This includes: introducing trainable low-rank bypasses into the query and value projection layers in the multi-level window Transformer layer and the multi-head attention mechanism. The low-rank bypasses consist of two low-rank matrices and an activation function to perform low-rank decomposition on parameter updates. The low-rank matrices are linear layers. During training, only the parameters of the low-rank bypasses are optimized, while the weights of the original query and value projection layers are frozen. During the inference phase, the low-rank matrices are added to the original weights of the query and value projection layers and merged into new weights.

[0032] like Figure 2 As shown, the multiple single convolutional modules in this embodiment specifically consist of four single convolutional modules, the number of which can be configured as needed. Each single convolutional module is a trainable network module, and each module contains a 3×3 convolutional layer, a layer normalization layer, and a GELU activation function connected in sequence.

[0033] Target regions in ultrasound images often exhibit varying sizes and irregular shapes. To address these issues, this embodiment designs an advanced feature enhancement module to effectively capture high-level semantic features from multiple scales. Considering the specific characteristics and extraction difficulties of features in medical images, this embodiment utilizes 3×3 dilated convolutions with different dilation rates (1, 3, 5) to effectively capture features at different scales in the spatial dimension. For example... Figure 3 As shown, the semantic feature enhancement achieved through multi-scale feature extraction using the advanced feature enhancement module and then output to the pixel decoder includes: embedding the image as the input feature map; extracting features from the input feature map using a channel attention mechanism (enhancing channel features by capturing the importance of different channels); and passing the input feature map in parallel through four dilated convolution branches used to capture features at different scales in the spatial dimension. The four dilated convolution branches are as follows: the first dilated convolution branch consists of three 3×3 dilated convolutions with dilation rates of 1, 3, and 5, and one 1×1 convolution; the second dilated convolution branch consists of two 3×3 dilated convolutions with dilation rates of 1 and 3, and one 1×1 convolution; the third dilated convolution branch consists of one 3×3 dilated convolution with dilation rate of 3 and one 1×1 convolution; and the fourth dilated convolution branch consists of one 3×3 dilated convolution with dilation rate of 1. The features extracted by the channel attention mechanism and the features extracted by the four dilated convolution branches are concatenated along the channel dimension to generate the final output feature map, which is then output to the pixel decoder. The channel attention mechanism and four parallel dilated convolution branches constitute five branches. The feature maps obtained from the five branches are connected and refined by a 1×1 convolution to generate the final output feature map and output it to the pixel decoder.

[0034] In encoder-decoder architectures, skip connections play a crucial role in improving the accuracy and robustness of segmentation networks. In ultrasound image segmentation, the low-level features carried by skip connections simultaneously contain edge details and interference noise; direct connections may introduce noise into subsequent decoding. Therefore, this embodiment designs a low-level feature refinement module to optimize the shallow features at skip connections. For example... Figure 4As shown, the process of optimizing and enhancing local detail features using a low-level feature refinement module before outputting them to the pixel decoder includes: taking local detail features as input feature maps; extracting features from the input feature maps using a channel attention mechanism; and passing the input feature maps in parallel through average pooling and max pooling operations with a kernel size of 2×2, and average pooling and max pooling operations with a kernel size of 8×8. Average pooling preserves overall average information, while max pooling highlights prominent information. By combining these pooling operations with different kernel sizes, this module captures the diversity and importance of the input feature maps and suppresses irrelevant features. Then, the results of the four pooling operations are upsampled to the same resolution and concatenated along the channel dimension. A 1×1 convolution module is then used to generate a refined feature map. The refined feature map is concatenated with the residuals of the original input feature map and then concatenated along the channel dimension with the features extracted by the channel attention mechanism (which further enhances the module's processing power along the channel dimension and suppresses unimportant or redundant features). Finally, a 1×1 convolution module is used to generate the final output feature map, which is then output to the pixel decoder.

[0035] The first stage of the decoding process is implemented using a pixel decoder, such as... Figure 5 As shown, the pixel decoder includes three pixel decoding modules. Each pixel decoding module consists of a transposed convolution, a channel dimension connection module, and two convolution modules. The convolution module is composed of a 3×3 convolutional layer, a batch normalization layer, and a GELU activation function connected in sequence. The output features of the advanced feature enhancement module are used as the input feature map of the pixel decoder. This input is then fed into the transposed convolution of the first layer to obtain a double-upsampled feature map. The shallow features output from the first low-level feature refinement module are concatenated with the double-upsampled feature map via a channel-dimensional connection module and then passed through a convolution module to obtain the output feature map of the first layer. The output feature map of the first layer is then transposed into a second layer to obtain a quadruple-upsampled feature map. The shallow features output from the second low-level feature refinement module are concatenated with the quadruple-upsampled feature map via a channel-dimensional connection module and then passed through a convolution module to obtain the output feature map of the second layer. The output feature map of the second layer is then transposed into a third layer to obtain an eight-upsampled feature map. The shallow features output from the third low-level feature refinement module are concatenated with the eight-upsampled feature map via a channel-dimensional connection module and then passed through a convolution module to obtain the output feature map of the third layer. The output feature map of the third layer is processed by a 1×1 convolutional layer and a sigmoid activation function to generate a preliminary segmentation prediction map. This prediction map will serve as the basis for automatic prompting in the second-stage decoding process.

[0036] The second-stage decoding process consists of a cue encoder and a mask decoder, which optimizes the final segmentation output by fusing automatic cue information. In this embodiment, point cues and mask cues are automatically generated from the preliminary segmentation prediction map generated in the first-stage decoding process. Point cues are generated by the cue encoder using point embedding vectors. The positional encoding of positive sample points is added to the learnable foreground embedding vector, and the positional encoding of negative sample points is added to the learnable background embedding vector. The foreground and background embedding vectors are trainable parameters used to assist the model in accurately identifying target and background regions. Since the accuracy of point cue labels extracted directly from the preliminary segmentation mask is difficult to guarantee, this invention pre-trains the CNN branch on a target-presence classification task. By using a classification network to identify targets, the accuracy of point cues can be improved. Point cues are selected from the preliminary segmentation prediction map. If the classification result indicates the presence of a target, the top n points with the highest confidence in the preliminary segmentation prediction map are selected as positive point cues; otherwise, the top n points with the lowest confidence in the preliminary segmentation prediction map are selected as negative point cues. After thresholding, the preliminary segmentation prediction map generates mask cues, thereby filtering out some low-confidence regions that may lead to errors.

[0037] like Figure 6As shown, the mask decoder in this embodiment includes two mask decoding units, a multi-level transposed convolution module, an anti-cross attention module, a multilayer perceptron, and a dot product module. Each mask decoding unit includes a self-attention module, an anti-cross attention module, a perceptron, and a cross attention module connected in sequence. The dot embedding vectors serve as inputs to the self-attention module in the first mask decoding unit. The image embedding and mask embedding vectors obtained after supplementing local detail features are concatenated and used as inputs to the anti-cross attention module and cross attention module in the first mask decoding unit, respectively. The output of the perceptron in the first mask decoding unit serves as input to the self-attention module in the second mask decoding unit. The cross attention module in the first mask decoding unit... The output of the first input is used as the input to the anti-cross attention module and the cross attention module in the second mask decoding unit. The output of the perceptron in the second mask decoding unit is used as the first input to the anti-cross attention module outside the two mask decoding units. The output of the cross attention module in the second mask decoding unit is used as the second input to the anti-cross attention module outside the two mask decoding units and the input to the multi-level transposed convolution module. The anti-cross attention module outside the two mask decoding units performs anti-cross attention on the two inputs and then extracts features through the multi-layer perceptron as the first input to the dot product module. The output of the multi-level transposed convolution module is used as the second input to the dot product module. The dot product module performs dot product operation on the two inputs to obtain the final segmentation prediction map. Automatic point cues and automatic mask cues are jointly guided by a cue encoder and a mask decoder to segment the model. The cue encoder encodes point cues and mask cues into point embedding vectors and mask embedding vectors, respectively. The mask embedding vectors are fused with the image embedding vectors generated by the dual-branch encoder to enhance the foreground context information of the original image embedding. After being processed by a self-attention mechanism, the point embedding vectors interact with the enhanced image embedding vectors through cross-attention, thereby enabling the model to better identify target regions and reduce the segmentation of non-target regions.

[0038] In this embodiment, the parameters of the dual-branch encoder, high-level feature enhancement module, low-level feature refinement module, pixel decoder, cue encoder, and mask decoder network are determined by optimizing the loss function to minimize it during training using the training set. The training set includes ultrasound images and corresponding label images. The loss function refers to the error between the segmentation prediction image obtained from the ultrasound image segmentation network model and the corresponding label image. As an optional implementation, in this embodiment, the pixel decoder uses a binary cross-entropy loss function during training of the ultrasound image segmentation network model. The function expression of the binary cross-entropy loss is: , in, For binary cross-entropy loss, For the sample size, Let the label be the value of the i-th pixel. This is the initial segmentation prediction map for the i-th pixel value; the loss function used by the mask decoder is a weighted sum of focus loss and dice loss, and the functional expression for the weighted sum of focus loss and dice loss is: , ,

[0039] in, The weighted sum of focus loss and dice loss. As a balance factor between focus loss and dice loss, For the focus of loss, Loss due to dice, As a positive and negative sample balance factor, This is the adjusted predicted probability for the value of the i-th pixel. The predicted probability of the i-th pixel value. Loss function. Used in the first stage of decoding to obtain a high-quality preliminary segmentation prediction map, loss function This is used in the second stage of decoding to obtain the final accurate segmentation prediction map, achieving excellent segmentation performance.

[0040] To verify the effectiveness of the method in this embodiment, the BUSI breast ultrasound image dataset and the UNS neuro-ultrasound image dataset were used for testing, and the STU breast ultrasound image dataset was used to verify the zero-shot generalization performance of the method. In the experiment, an ultrasound image dataset with a one-to-one correspondence between the original image and the label was used. The number of point cues n in the example method was set to 10, and the mask cue threshold was set to 0.95. The positive and negative sample balance factor in the training loss function was set to 0.8, the easy and difficult sample balance factor was set to 2, and the balance factors for focus loss and dice loss were set to 0.2. The method in this embodiment is mainly compared with three classic segmentation methods: UNet, UNet++, and Attention-UNet; two Transformer-based segmentation methods: TransUNet and TransFuse; two advanced ultrasound image segmentation methods: AAU-Net and CMU-Net; three SAM-based fine-tuning methods: SAM, MSA, and SAMed; and three SAM-based automatic segmentation methods: AutoSAM, UN-SAM, and H-SAM. The objective evaluation metrics used include Dice coefficient, Hausdorff distance, intersection-over-union ratio (IOU), accuracy, recall, and precision. The results are shown in Tables 1 and 2.

[0041] Table 1: Objective evaluation metrics for ultrasound image segmentation results in the BUSI dataset

[0042] Table 2: Objective Evaluation Indicators for Ultrasound Image Segmentation Results in the UNS Dataset

[0043] Referring to the experimental data shown in Tables 1 and 2, the method of this embodiment demonstrates excellent image segmentation performance on both benchmark datasets. Specifically, on the BUSI breast ultrasound dataset, the method of this embodiment achieved the following results: Dice coefficient of 81.82%, Hausdorff distance of 29.71, Intersection over Union (IoU) of 75.84%, accuracy of 96.49%, recall of 84.13%, and precision of 85.52%. Among these, the four evaluation metrics—Dice coefficient, IoU, accuracy, and recall—all reached optimal values. It is worth noting that, in the most representative metrics of image segmentation—Dice coefficient and IoU—the method of this embodiment improved by 3.37% and 3.49% respectively compared to the second-best method, demonstrating a significant advantage. The Hausdorff distance metric, with a result of 29.71, ranked second, indicating that this method performs well in ultrasound image edge segmentation. Furthermore, while achieving optimal results in accuracy and recall, it also achieved third place in precision.

[0044] On the more challenging UNS neurosound dataset, the evaluation metrics of the method in this embodiment are as follows: Dice coefficient 74.27%, Hausdorff distance 10.16, intersection-over-union ratio 71.10%, accuracy 99.15%, recall 83.36%, and precision 86.53%. Except for the recall, which is slightly lower than the optimal value, the other five metrics (Dice coefficient, Hausdorff distance, intersection-over-union ratio, accuracy, and precision) all achieve the best results, with improvements of 0.21%, 0.04%, 0.15%, 0.02%, and 0.04% respectively compared to the second-best model. Compared to the BUSI dataset, ultrasound images in the UNS dataset have more significant noise interference and more complex surrounding tissue features. Existing SAM-based models perform poorly in such complex contexts, while the improvements to SAM in this embodiment effectively enhance the model's ability to capture local features and identify target tissues, significantly reducing the false segmentation rate of non-target tissues, thus surpassing traditional methods and existing SAM-based methods in segmentation performance.

[0045] To further verify the generalization ability of the method in this embodiment for ultrasound image segmentation tasks, this embodiment evaluates the zero-shot generalization ability of various segmentation methods on the STU breast ultrasound dataset. This experiment directly uses the model weights trained on the BUSI dataset without any fine-tuning or additional training. The results are shown in Table 3.

[0046] Table 3: Objective Evaluation Indicators for Ultrasound Image Segmentation Results in the STU Dataset

[0047] As shown in Table 3, the method in this embodiment exhibits excellent zero-shot generalization performance. Specifically, among the six evaluation metrics, the Dice coefficient, intersection-over-union ratio, accuracy, and recall all reached optimal values, improving upon the suboptimal model by 0.07%, 0.23%, 0.22%, and 0.14%, respectively. In the zero-shot generalization experiment, the SAM-based segmentation method significantly outperformed the traditional segmentation model. This further demonstrates the powerful generalization ability of the SAM model and its great potential in medical image segmentation.

[0048] The comparison results of partial samples of the method in this embodiment and existing methods on the BUSI dataset are as follows: Figure 7 As shown. In Figure 7In the figure, the ultrasound images in the first to third rows are benign and malignant breast lesion images from the BUSI dataset. It is clear from the three different lesion regions shown in the figure that the breast lesion regions in the ultrasound images exhibit significant differences in size and shape. Non-SAM-based segmentation methods (e.g., U-Net, TransUNet) showed both false negatives and false positives in the three lesion regions. The automatic SAM (H-SAM) segmentation method performed well in the second lesion region, but showed a small number of false negatives in the first lesion region and a large number of false negatives in the third lesion region. The method in this embodiment achieved perfect segmentation results in the first and second lesion regions. Although the segmentation result in the third lesion region was incomplete, the false negative rate of the method in this embodiment was significantly reduced compared to other methods. Overall, the proposed method outperforms other methods in terms of overall performance in breast tumor segmentation. Figure 8 This paper demonstrates the results of some existing methods on the missegmentation of normal images on the BUSI dataset. Due to the influence of ultrasound noise, some irrelevant background regions are incorrectly segmented as lesions. The method in this embodiment introduces a CNN branch to provide classification knowledge regarding the presence or absence of lesions. Automatic point suggestions, combined with classification knowledge and the preliminary segmentation prediction map, guide the model to avoid segmenting non-target regions. The probability of missegmenting normal images on the BUSI dataset using this embodiment is zero.

[0049] The comparison results of partial samples of the method in this embodiment and existing methods on the UNS dataset are as follows: Figure 9 As shown, the ultrasound images in rows one through three are brachial plexus images from the UNS dataset. On the first sample, traditional segmentation methods U-Net, TransUNet, and the H-SAM model (an improvement on SAM) all suffer from incomplete target region segmentation. The method in this embodiment achieves complete target region segmentation. On the second sample, the segmentation performance of all comparison methods is not ideal, with the method in this embodiment exhibiting the lowest error rate. On the third sample, the brachial plexus is not present in the image, and existing comparison methods are all subject to interference from irrelevant tissues, resulting in incorrect segmentation. The method in this embodiment effectively avoids incorrect segmentation due to its unique ultrasound image adaptive mechanism. Comprehensive experimental data demonstrates that the segmentation accuracy of the method in this embodiment on ultrasound neural images is significantly superior to existing techniques.

[0050] The comparison results of partial samples of the method in this embodiment and existing methods on the STU dataset are as follows: Figure 10As shown, some classic segmentation methods exhibit poor zero-shot generalization ability, with experimental results showing widespread missegmentation or incomplete segmentation. While SAM-based comparative methods can avoid widespread missegmentation, they also face issues of incomplete segmentation and insufficient edge details. In contrast, the segmentation prediction map of the method in this embodiment is closer to the true label, demonstrating advantages in capturing edge details and target region size during the segmentation process.

[0051] To evaluate the effectiveness of the high-level feature enhancement module and low-level feature refinement module in the ultrasound image segmentation network model of this embodiment, an ablation experiment was conducted on the BUSI dataset to verify the method. The method of this embodiment without these two modules was used as the baseline network, and the results are shown in Table 4.

[0052] Table 4: Objective Evaluation Indicators of Ablation Experiment Segmentation Results of High-Level Feature Enhancement Module and Low-Level Feature Refinement Module

[0053] As shown in Table 4, the experimental data demonstrates that introducing the low-level feature refinement module (+LFR), the high-level feature enhancement module (+HFE), and the high-level feature enhancement module and low-level feature refinement module (+LFR+HFE) sequentially effectively improved the model's segmentation performance. Specifically, the results in the second row show that the low-level feature refinement module significantly improved segmentation performance, increasing the Dice coefficient and intersection-over-union (IoU) ratio by 1.30% and 1.32%, respectively. Furthermore, the low-level feature refinement module effectively improved the segmentation results for edge details, increasing the Hausdorff distance by 1.26. The results in the third row show that the addition of the high-level feature enhancement module improved the Dice coefficient, Hausdorff distance, IoU ratio, accuracy, and precision by 1.36%, 0.12%, 1.37%, 0.34%, and 2.88%, respectively. These results fully validate the effectiveness of the high-level feature enhancement module in enhancing semantic features. Finally, compared with the baseline network, the addition of two design modules improved the six evaluation metrics (Dice coefficient, Hausdorff distance, crossover ratio, accuracy, recall, and precision) by 2.06%, 1.6%, 2.34%, 0.46%, 0.81%, and 2.38%, respectively.

[0054] Furthermore, to verify the contribution of the automatic point hinting and automatic mask hinting strategies to the overall method in this embodiment, a detailed ablation experiment was designed. First, regarding the automatic point hinting strategy, by comparing the experimental results with different numbers of point hints (0, 1, 3, 5, 8, 10, 15, 20), it was found that the model achieved optimal segmentation performance when 10 automatic point hints were used. The results are shown in Table 5.

[0055] Table 5: Objective Evaluation Indicators for Ablation Experiment Segmentation Results Based on the Number of Automatic Prompts

[0056] As shown in Table 5, compared to the baseline without cueing, the configuration with 10 cueing points improved the Dice coefficient, Hausdorff distance, intersection-union ratio, accuracy, recall, and precision by 2.8%, 2.4%, 2.66%, 0.44%, 2.07%, and 0.67%, respectively, demonstrating that this strategy can effectively suppress the interference of noise and shadows in ultrasound images.

[0057] Furthermore, this experiment combined the number of optimal prompts (10 points) with automatic mask prompts (threshold 0.95) for testing, and the results are shown in Table 6.

[0058] Table 6: Objective Evaluation Indicators for Ablation Experiment Segmentation Results Based on Automatic Point Hints and Automatic Mask Hints

[0059] As shown in Table 6, the effective combination of the dual-hint strategy achieved significant improvements of 3.71%, 2.71%, 3.91%, 0.48%, 1.86%, and 3.28% in six metrics (Dice coefficient, Hausdorff distance, intersection-union ratio, precision, recall, and accuracy) compared to the no-hint baseline.

[0060] In summary, this embodiment employs a dual-branch encoder and a two-stage decoding architecture. The dual-branch encoder combines the Transformer branch of SAM for global feature extraction and the CNN branch for local feature extraction. During the first-stage decoding process, a high-level feature enhancement module strengthens multi-scale semantic feature representation, and a low-level feature refinement module optimizes shallow feature details, jointly improving the quality of the initial segmentation prediction map. Regarding prompt generation, an innovative prompting strategy combining automatic point prompts and automatic mask prompts is designed: automatic point prompts are generated by combining the classification knowledge of the CNN branch with the spatial information of the initial segmentation prediction map, effectively suppressing the interference of noise and shadows in the ultrasound image; automatic mask prompts are obtained by thresholding the initial segmentation prediction map, providing detailed contextual guidance for the model. Extensive experiments show that the method in this embodiment outperforms other segmentation models. Without manual prompting, the SAM-based automatic prompting ultrasound image segmentation method achieves excellent performance in the field of ultrasound image segmentation and exhibits strong generalization ability. Compared with existing technologies, this embodiment has the following significant advantages: 1. This embodiment of the SAM-based automatic prompting ultrasound image segmentation network model designs a dual-branch encoder and a two-stage decoding process. The dual-branch encoder can effectively extract local detail features and global dependencies, while the two-stage decoding process automatically generates prompts to embed and guide the model to achieve accurate segmentation, effectively applying SAM to the field of ultrasound image segmentation. 2. The automatic prompting design of the SAM-based ultrasound image segmentation network model in this embodiment solves the problem of SAM relying on manual prompts. Furthermore, the positive and negative point prompts based on CNN classification knowledge effectively reduce the segmentation errors caused by ultrasound noise and shadows. Mask prompts can also provide more detailed contextual information. The combination of these two types of automatic prompts improves the information capture efficiency of the input image and enhances the ultrasound image segmentation performance of the ultrasound image segmentation network model. 3. The high-level feature enhancement module and low-level feature refinement module in this embodiment effectively improve the quality of the initial segmentation prediction map and the accuracy of automatic prompts. The high-level feature enhancement module aims to capture multi-scale semantic features, while the low-level feature refinement module plays a key role at the transition connections by refining the shallow features of CNN branches.

[0061] Furthermore, this embodiment also provides an SAM-based automatic prompting ultrasound image segmentation system, including a microprocessor and a memory interconnected, wherein the microprocessor is programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method. Additionally, this embodiment provides a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method via a processor. Furthermore, this embodiment also provides a computer program product, including a computer program or instructions programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method via a processor.

[0062] Those skilled in the art will understand that the technical solutions provided by this invention may take the form of a method, system, or computer program product. Therefore, this invention may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention may take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, produce an implementation of the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0063] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. An automatic prompting ultrasound image segmentation method based on SAM, characterized in that, The process includes the following steps: Segmenting the input ultrasound image using an ultrasound image segmentation network model to obtain a segmentation prediction map. This ultrasound image segmentation network model includes a CNN encoder, a classification head module, a low-level feature refinement module, a SAM Transformer encoder, a cross-branch attention module, a high-level feature enhancement module, a pixel decoder, a cue encoder, and a mask decoder. The step of segmenting the input ultrasound image to obtain the segmentation prediction map includes: extracting local detail features from the input ultrasound image using the CNN encoder and performing target existence classification using the classification head module; optimizing and enhancing the local detail features using the low-level feature refinement module and then outputting the result to the pixel decoder; extracting global features from the input ultrasound image using the SAM Transformer encoder and supplementing them using the cross-branch attention module. After supplementing local detail features, an image embedding is obtained. Then, the high-level feature enhancement module uses multi-scale feature extraction to enhance semantic features before outputting it to the pixel decoder. The pixel decoder performs the first stage of decoding to generate a preliminary segmentation prediction map. After thresholding the preliminary segmentation prediction map, a mask cue is generated. If the target existence classification result indicates the presence of a target, the top n points with the highest confidence in the preliminary segmentation prediction map are selected as positive cue points; otherwise, the top n points with the lowest confidence in the preliminary segmentation prediction map are selected as negative cue points. The positive or negative cue points are encoded by the cue encoder to generate point embedding vectors, and the mask cue points are encoded by the cue encoder to generate mask embedding vectors. The point embedding vectors, mask embedding vectors, and the image obtained after supplementing local detail features are embedded into the mask decoder to perform the second stage of decoding to obtain the final segmentation prediction map.

2. The automatic prompting ultrasound image segmentation method based on SAM according to claim 1, characterized in that, The CNN encoder includes five convolutional modules (convolutional modules 1 to 5) connected in a cascaded manner. The local detail features output by the first three convolutional modules (convolutional modules 1 to 3) serve as inputs to the next level convolutional modules in the CNN encoder, and also as inputs to corresponding low-level feature refinement modules. The local detail features output by convolutional module 5 serve as inputs to the classification head module for target presence classification. The SAM's Transformer encoder includes a multi-level window Transformer layer. The input ultrasound image is first divided into blocks by an image block embedding layer, and then combined with position encoding before being used as input to the multi-level window Transformer layer. The image embedding obtained after supplementing local detail features through a cross-branch attention module includes: normalizing the global features extracted by the multi-level window Transformer layer and inputting them to the cross-branch attention module; and using the local detail features output by the CNN encoder as keys and values, and the global features as queries, performing cross-attention operations through the cross-branch attention module. The image embedding is obtained by extracting attention features through a multi-head attention mechanism, adding the extracted attention features to the global features, and then extracting local detail features from the CNN encoder output through multiple single convolutional modules. The fused features are then normalized through layers, processed by a multi-layer perceptron, added to the original fused features, and then added to the features extracted by multiple single convolutional modules to obtain image embedding with supplemented local detail features. During training, the Transformer encoder of the SAM uses a low-rank adaptive fine-tuning method to fine-tune the Transformer encoder, including: introducing trainable low-rank bypasses into the multi-level window Transformer layer and the query and value projection layers in the multi-head attention mechanism. The low-rank bypasses consist of two low-rank matrices and an activation function to perform low-rank decomposition on parameter updates. The low-rank matrices are linear layers. During training, only the parameters of the low-rank bypasses are optimized, while the weights of the original query and value projection layers are frozen. During the inference phase, the low-rank matrices are added to the original weights of the query and value projection layers and merged into new weights.

3. The automatic prompting ultrasound image segmentation method based on SAM according to claim 1, characterized in that, The step of using a low-level feature refinement module to optimize and enhance local detail features before outputting them to the pixel decoder includes: taking local detail features as input feature maps; extracting features from the input feature maps using a channel attention mechanism; passing the input feature maps in parallel through average pooling and max pooling operations with a kernel size of 2×2, and average pooling and max pooling operations with a kernel size of 8×8; upsampling the results of the four pooling operations to the same resolution and concatenating them along the channel dimension; generating a refined feature map through a 1×1 convolution module; concatenating the refined feature map with the residuals of the original input feature map; concatenating it along the channel dimension with the features extracted by the channel attention mechanism; and finally generating the final output feature map through a 1×1 convolution module and outputting it to the pixel decoder.

4. The automatic prompting ultrasound image segmentation method based on SAM according to claim 1, characterized in that, The process of using an advanced feature enhancement module to enhance semantic features through multi-scale feature extraction and then outputting the result to the pixel decoder includes: embedding an image as an input feature map; extracting features from the input feature map using a channel attention mechanism; and passing the input feature map in parallel through four dilated convolution branches used to capture features at different scales in the spatial dimension. The four dilated convolution branches are as follows: the first branch consists of three 3×3 dilated convolutions with dilation rates of 1, 3, and 5, and one 1×1 convolution; the second branch consists of two 3×3 dilated convolutions with dilation rates of 1 and 3, and one 1×1 convolution; the third branch consists of one 3×3 dilated convolution with dilation rate of 3 and one 1×1 convolution; and the fourth branch consists of one 3×3 dilated convolution with dilation rate of 1. The features extracted by the channel attention mechanism and the features extracted by the four dilated convolution branches are concatenated along the channel dimension to generate the final output feature map, which is then output to the pixel decoder.

5. The automatic prompting ultrasound image segmentation method based on SAM according to claim 1, characterized in that, The pixel decoder includes three pixel decoding modules. Each pixel decoding module consists of a transposed convolution, a channel-dimension connection module, and two convolutional modules. Each convolutional module is composed of a 3×3 convolutional layer, a batch normalization layer, and a GELU activation function connected sequentially. The output features of the high-level feature enhancement module are used as the input feature map of the pixel decoder and input to the transposed convolution of the first layer to obtain a double-upsampled feature map. The shallow features output by the first low-level feature refinement module are concatenated with the double-upsampled feature map through the channel-dimension connection module and then passed through the convolutional module to obtain the output feature map of the first layer. The output feature map of the first layer is then processed by... The transposed convolution of the second layer yields a 4x upsampled feature map. The shallow features output from the second low-level feature refinement module are connected to the 4x upsampled feature map via a channel dimension connection module and then passed through a convolution module to obtain the output feature map of the second layer. The output feature map of the second layer is transposed convolution of the third layer to obtain an 8x upsampled feature map. The shallow features output from the third low-level feature refinement module are connected to the 8x upsampled feature map via a channel dimension connection module and then passed through a convolution module to obtain the output feature map of the third layer. The output feature map of the third layer is then passed through a 1×1 convolutional layer and a sigmoid activation function to obtain a preliminary segmentation prediction map.

6. The automatic prompting ultrasound image segmentation method based on SAM according to claim 1, characterized in that, The mask decoder includes two mask decoding units, a multi-level transposed convolution module, an anti-cross attention module, a multilayer perceptron, and a dot product module. Each mask decoding unit comprises a self-attention module, an anti-cross attention module, a perceptron, and a cross attention module connected sequentially. The dot embedding vector serves as the input to the self-attention module in the first mask decoding unit. The image embedding and mask embedding vectors obtained after supplementing local detail features are concatenated and used as the input to the anti-cross attention and cross attention modules in the first mask decoding unit. The output of the perceptron in the first mask decoding unit serves as the input to the self-attention module in the second mask decoding unit, and the output of the cross attention module in the first mask decoding unit serves as the input to the... The inputs to the anti-cross attention module and the cross attention module in the two mask decoding units are as follows: the output of the perceptron in the second mask decoding unit serves as the first input to the anti-cross attention module outside the two mask decoding units; the output of the cross attention module in the second mask decoding unit serves as the second input to the anti-cross attention module outside the two mask decoding units; and the input to the multi-level transposed convolution module is also included. The anti-cross attention module outside the two mask decoding units performs anti-cross attention on the two inputs and then extracts features through the multi-level perceptron as the first input to the dot product module. The output of the multi-level transposed convolution module serves as the second input to the dot product module. The dot product module performs dot product operations on the two inputs to obtain the final segmentation prediction map.

7. The automatic prompting ultrasound image segmentation method based on SAM according to claim 2, characterized in that, During training, the pixel decoder of the ultrasound image segmentation network model uses a binary cross-entropy loss function, the expression of which is: ， in, For binary cross-entropy loss, For the sample size, Let the label be the value of the i-th pixel. This is the initial segmentation prediction map for the i-th pixel value; the loss function used by the mask decoder is a weighted sum of focus loss and dice loss, and the functional expression for the weighted sum of focus loss and dice loss is: ， ， in, The weighted sum of focus loss and dice loss. As a balance factor between focus loss and dice loss, For the focus of loss, Loss due to dice, As a positive and negative sample balance factor, This is the adjusted predicted probability for the value of the i-th pixel. Let be the predicted probability of the i-th pixel value.

8. An automatic prompting ultrasound image segmentation system based on SAM, comprising a microprocessor and a memory interconnected, characterized in that, The microprocessor is programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method according to any one of claims 1 to 7 via a processor.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to execute the SAM-based automatic prompting ultrasound image segmentation method according to any one of claims 1 to 7 via a processor.

Images