Ultrasonic image segmentation method and system based on multistage feature extraction

Abstract

The invention discloses an ultrasonic image segmentation method and system based on multistage feature extraction, and the method comprises the steps: carrying out the ultrasonic image segmentation of an input ultrasonic image through an ultrasonic image segmentation network model, the ultrasonic image segmentation network model comprises a double-branch encoder, a multi-scale feature fusion module, a multi-scale feature attention module and a decoder, and the double-branch encoder is used for extracting small detail features, large-scale context features and global features of different scales; the multi-scale feature fusion module fuses features and then inputs the fused features into the decoder, the multi-scale feature attention module enhances each layer of features of the CNN encoder and then inputs the enhanced features into the decoder, and a segmentation prediction map is obtained through decoding of the decoder. The method aims at solving the problem that most of existing ultrasonic image segmentation methods based on deep learning only pay attention to capturing local details and global correlation and ignore large-scale context information, the information capturing efficiency of an input image is improved, and the segmentation performance of a network is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing of ultrasound (US) images, and in particular to an ultrasound image segmentation method and system based on multi-level feature extraction. Background Art

[0002] Ultrasound imaging is a common method for clinical diagnosis. It can observe the internal structure and position of organs and nerves, the contraction and peristalsis of organs. Due to its real-time, easy operation and harmlessness, ultrasound imaging is more acceptable than other medical imaging methods such as computed tomography (CT), magnetic resonance imaging (MRI) and positron emission tomography (PET). Ultrasound image analysis and lesion area annotation are highly dependent on the doctor's experience, resulting in low efficiency. Ultrasound image segmentation plays an important role in the early diagnosis and subsequent treatment of diseases. It improves the efficiency of clinical diagnosis and reduces the workload of doctors. Traditional segmentation methods such as region growing and graph cutting can achieve automatic segmentation of targets, however, these methods have unsatisfactory segmentation accuracy and limited generalization ability. In recent years, with the development of deep learning, it has achieved great success in the field of computer vision. Compared with traditional segmentation methods, image segmentation methods based on deep learning have better performance. The commonly used image segmentation methods are based on deep learning models, such as UNet, TransUNet, and ConvNext. Although the segmentation effect is significantly improved compared with traditional segmentation methods, most existing methods only focus on extracting local features and global features, ignoring the large-scale context features necessary for large targets, and using large kernel convolution networks. The number of model parameters is too large, and the network training effect is not ideal. How to make the network have better feature extraction capabilities and improve segmentation performance, image segmentation methods based on deep learning models still need continuous improvement and innovation. Summary of the invention

[0003] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a method and system for ultrasound image segmentation based on multi-level feature extraction are provided. The present invention aims to solve the problem that most existing ultrasound image segmentation methods based on deep learning only focus on capturing local details and global correlations while ignoring large-scale contextual information, thereby improving the information capture efficiency of the input image and enhancing the segmentation performance of the network.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: A method for ultrasonic image segmentation based on multi-level feature extraction includes using an ultrasonic image segmentation network model to perform ultrasonic image segmentation on an input ultrasonic image to obtain a segmentation prediction map, the ultrasonic image segmentation network model includes a dual-branch encoder, a multi-scale feature fusion module, a multi-scale feature attention module and a decoder, the dual-branch encoder includes a CNN encoder and a Transformer encoder, and using the ultrasonic image segmentation network to perform ultrasonic image segmentation on the input ultrasonic image to obtain a segmentation prediction map includes: extracting small detail features and large-scale context features of different scales from the input ultrasonic image through a CNN encoder and using adaptive weighted weights to adaptively select and enhance the small detail features and large-scale context features of different scales; extracting global features from the input ultrasonic image through a Transformer encoder; using a multi-scale feature attention module to enhance the shallow features output by each layer of the CNN encoder and output them to the decoder, and decoding the fused multi-level features and the shallow features enhanced by the multi-scale feature attention module layer by layer through the decoder to obtain a final segmentation prediction map.

[0005] Optionally, the CNN encoder includes encoding branches of N scales, the encoding branch of the first scale includes a 1×1 convolution layer and a large and small kernel attention module, and the encoding branches of the remaining N-1 scales all include a downsampling module, a channel dimension splicing module and a large and small kernel attention module. The output of the 1×1 convolution layer in the encoding branch of the first scale is used as the input of the downsampling module in the encoding branch of the second scale, and the input of the remaining downsampling modules is the output of the channel dimension splicing module in the encoding branch of the previous scale. The outputs of all channel dimension splicing modules are also used as the input of the large and small kernel attention modules in the encoding branches of the same scale. The first N-1 The output of the large and small kernel attention modules in the encoding branch of the scale is used as one input of the channel dimension splicing module in the encoding branch of the next scale, and the output of the large and small kernel attention modules in the encoding branch of the Nth scale is used as the input of the multi-scale feature fusion module CNN encoder end. The decoder includes N-1 levels of decoding modules connected in cascade, and the multi-scale feature fusion module is used as the input of the first level decoding module. Another input of each level of decoding module is the output of the large and small kernel attention modules in the encoding branch of the corresponding scale, and the output feature is extracted by the multi-scale attention module, and the final segmentation prediction map is decoded and output by the N-1th level decoding module.

[0006] Optionally, the processing of the input feature map by the large and small core attention module includes: extracting large-scale context features from the input feature map of the large and small core attention module through a large-scale feature selection module, and extracting small detail features from the input feature map of the large and small core attention module through a small-core convolution module, and then adding the large-scale context features and the small detail features element by element; then the features obtained by element-by-element addition are passed through a feature interaction module to obtain fully integrated small detail and large-scale context output data, and the feature interaction module performs maximum pooling and average pooling on the features obtained by element-by-element addition and then splices them in the channel dimension, and then passes through a convolution layer with a convolution kernel size of 7×7 and a Sigmoid activation function, and then performs matrix multiplication operation with the features obtained by element-by-element addition, and finally adds the features obtained by the matrix multiplication operation and the small detail features element by element, and then passes through a 3×3 convolution layer, a 3×3 dilated convolution layer with a dilation rate d of 2, and a 1×1 convolution layer to obtain fully integrated small detail and large-scale context output data.

[0007] Optionally, the large-scale feature selection module extracts large-scale context features, including: extracting preliminary features from the input feature map of the large-scale feature selection module through a 1×1 convolution layer and a 3×3 convolution layer; obtaining two large-scale context features from the preliminary features through large-core decomposition modules with different receptive fields, adding the two large-scale context features element by element, and then performing feature adaptive enhancement through a feature adaptive enhancement module, wherein the feature adaptive enhancement module includes a global average pooling, two linear layers and a Softmax function connected in sequence, wherein the two linear layers are used for dimensionality reduction and dimensionality increase, respectively; then performing matrix multiplication of the two large-scale context features obtained by the large-core decomposition module with the weights output by the Softmax function, and then adding them element by element to obtain the final large-scale context features; the small-core convolution module consists of a 1×1 convolution layer, a 5 ×5 convolutional layers are connected in series; the said passing the preliminary features through large kernel decomposition modules with different receptive fields respectively means that the preliminary features are respectively passed through a first large kernel decomposition module and a second large kernel decomposition module with different receptive fields for large-scale context feature extraction, the first large kernel decomposition module and the second large kernel decomposition module each include a 5×5 depthwise separable convolutional layer, a 5×5 dilated convolutional module, a 7×7 dilated convolutional module, a 3×3 dilated convolutional module and a 1×1 convolutional layer connected in sequence, wherein the number of groups of dilated group convolutions is 8, the dilated rates d of the 5×5 dilated convolutional modules, the 7×7 dilated convolutional modules and the 3×3 dilated convolutional modules in the first large kernel decomposition module are all 1, and the dilated rates d of the 5×5 dilated convolutional modules, the 7×7 dilated convolutional modules and the 3×3 dilated convolutional modules in the second large kernel decomposition module are all 3.

[0008] Optionally, the multi-scale feature fusion module fuses the multi-level features extracted by the CNN encoder and the Transformer encoder, including: splicing the multi-level features extracted by the CNN encoder and the Transformer encoder through the channel dimension, and then passing the channel dimension through a 1×1 convolution layer to convert the channel dimension from 2048 dimensions to 1024-dimensional features, and then extracting features from the 1024-dimensional features through three branches of a 1×1 convolution layer, a 3×3 convolution layer, and a 3×3 void convolution layer with a void rate d of 2, and then splicing the features extracted by the three branches with the original 1024-dimensional features in the channel dimension, and then passing the 1×1 convolution layer to reduce the channel dimension to 1024 dimensions to obtain the fused multi-level features.

[0009] Optionally, the multi-scale attention module extracts output features including: extracting features of the input features of the multi-scale attention module through a first branch consisting of a 1×1 convolution layer and a 3×3 convolution layer, and extracting features through a second branch consisting of a 1×1 convolution layer and a 3×3 dilated convolution layer with a dilation rate d of 2, concatenating the channel dimensions of the features extracted by the two branches, and then reducing the channel dimensions through a 1×1 convolution layer, and then performing maximum pooling and average pooling on the features after reducing the channel dimensions, and then concatenating the channel dimensions, and then performing matrix multiplication on the features after reducing the channel dimensions after linear layer dimensionality reduction, linear layer dimensionality increase, and Sigmoid activation function activation to obtain output features.

[0010] Optionally, the Transformer encoder is composed of a patch embedding layer, a linear layer and a plurality of transformer layers connected in sequence, wherein the patch embedding is a convolution layer with a convolution kernel size of 16×16 and a step size of 16, and an ultrasound image with a dimension of H×W×3 is passed through the patch embedding layer to obtain a feature with a dimension of H / 16×W / 16×D, wherein H and W are the height and width of the ultrasound image, and D is the output data dimension. The transformer layer is composed of an embedded patch layer (embedded The invention relates to a novel image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention. The invention relates to an image processing method for the present invention.

[0011] In addition, the present invention also provides an ultrasonic image segmentation system based on multi-level feature extraction, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the ultrasonic image segmentation method based on multi-level feature extraction.

[0012] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction through a processor.

[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, wherein the computer program or instructions are programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction through a processor.

[0014] Compared with the prior art, the present invention mainly has the following advantages: 1. The multi-level feature extraction design of the ultrasound image segmentation network model of the present invention realizes the extraction of small detail features, large-scale context features to global features, improves the information capture efficiency of the input image, and can enhance the ultrasound image segmentation performance of the ultrasound image segmentation network model.

[0015] 2. The large and small kernel attention modules of the present invention can capture the features of objects of different sizes in the input image. The small kernel convolution layer is used to extract the edge details of the object, and the decomposed large kernel module is used to capture the large-scale context features of large-sized objects more accurately. The feature interaction module has made an important contribution to improving the fusion expression ability between different scale features in the network and avoiding overfitting.

[0016] 3. The large-scale feature selection module of the present invention utilizes convolutional layers of multiple different scales to extract features of different scales, and adaptively selects and enhances important scales, thereby improving the feature extraction capability of the network and the effective expression of features of different scales in the image, and obtaining richer and more accurate feature data.

[0017] 4. The large kernel decomposition of the present invention can reduce the number of parameters of the ultrasound image segmentation network model and improve the trainability of the ultrasound image segmentation network model by effectively decomposing the large-scale context features while fully extracting the large-scale context features. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0019] Figure 2 Schematic diagram of the network structure of the ultrasound image segmentation network model in an embodiment of the present invention.

[0020] Figure 3 Schematic diagram of the network structure of the large and small core attention modules in an embodiment of the present invention.

[0021] Figure 4 Schematic diagram of the network structure of the large-scale feature selection module in an embodiment of the present invention.

[0022] Figure 5 Schematic diagram of the network structure of the first and second large core decomposition modules in an embodiment of the present invention, wherein (a) is a schematic diagram of the network structure of the first large core decomposition module, and (b) is a schematic diagram of the network structure of the second large core decomposition module.

[0023] Figure 6 Schematic diagram of the network structure of the multi-scale feature fusion module in an embodiment of the present invention.

[0024] Figure 7 Schematic diagram of the network structure of the multi-scale attention module in an embodiment of the present invention.

[0025] Figure 8 Schematic diagram of the network structure of the transformer layer in the transformer encoder in an embodiment of the present invention.

[0026] Fig. 9 1 is the first part of the segmentation results of BUSI and BUSC ultrasound images using different segmentation methods in the embodiments of the present invention, where (a-1) to (e-1) are the label images of the first sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method; (a-2) to (e-2) are the label images of the second sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method; (a-3) to (e-3) are the label images of the third sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method. tion-UNet method; (a-4) to (e-4) are the label images of the 4th sample and the segmentation prediction maps obtained by using the method of this embodiment, UNet method, UNet++ method and Attention-UNet method respectively; (a-5) to (e-5) are the label images of the 5th sample and the segmentation prediction maps obtained by using the method of this embodiment, UNet method, UNet++ method and Attention-UNet method respectively; (a-6) to (e-6) are the label images of the 6th sample and the segmentation prediction maps obtained by using the method of this embodiment, UNet method, UNet++ method and Attention-UNet method respectively.

[0027] Fig.10The second part of the segmentation results of BUSI and BUSC ultrasound images by different segmentation methods in the embodiment of the present invention, where (a-1) to (e-1) are the segmentation prediction images obtained by using the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the first sample respectively; (a-2) to (e-2) are the segmentation prediction images obtained by using the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the second sample respectively; (a-3) to (e-3) are the segmentation prediction images obtained by using the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the third sample respectively. The segmentation prediction images obtained by the SKUNet method and the MDA-Net method; (a-4) to (e-4) are the segmentation prediction images obtained by the 4th sample using the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method respectively; (a-5) to (e-5) are the segmentation prediction images obtained by the 5th sample using the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method respectively; (a-6) to (e-6) are the segmentation prediction images obtained by the 6th sample using the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method respectively.

[0028] Fig.11 The first part of the segmentation results of the UNS ultrasound image using different segmentation methods in the embodiments of the present invention, wherein (a-1) to (e-1) are respectively the label images of the first sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method; (a-2) to (e-2) are respectively the label images of the second sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method; (a-3) to (e-3) are respectively the label images of the third sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method; (a-4) to (e-4) are respectively the label images of the fourth sample and the segmentation prediction maps obtained using the method of this embodiment, the UNet method, the UNet++ method and the Attention-UNet method.

[0029] Fig.12The second part of the segmentation results of the UNS neural ultrasound image using different segmentation methods in the embodiment of the present invention, where (a-1) to (e-1) are the segmentation prediction images obtained by the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the first sample respectively; (a-2) to (e-2) are the segmentation prediction images obtained by the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the second sample respectively; (a-3) to (e-3) are the segmentation prediction images obtained by the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the third sample respectively; (a-4) to (e-4) are the segmentation prediction images obtained by the SKUNet method, TransUNet method, CMU-Net method, CMUNext method and MDA-Net method for the fourth sample respectively. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0031] like Figure 1 As shown, the ultrasonic image segmentation method based on multi-level feature extraction in this embodiment includes using an ultrasonic image segmentation network model to perform ultrasonic image segmentation on the input ultrasonic image to obtain a segmentation prediction map, the ultrasonic image segmentation network model includes a dual-branch encoder, a multi-scale feature fusion module, a multi-scale feature attention module and a decoder, the dual-branch encoder includes a CNN encoder and a Transformer encoder, and the ultrasonic image segmentation network is used to perform ultrasonic image segmentation on the input ultrasonic image to obtain a segmentation prediction map. The method includes: extracting small detail features and large-scale context features of different scales from the input ultrasonic image through the CNN encoder and using adaptive weighted weights to adaptively select and enhance small detail features and large-scale context features of different scales; extracting global features from the input ultrasonic image through the Transformer encoder; fusing the multi-level features extracted by the CNN encoder and the Transformer encoder through the multi-scale feature fusion module; using the multi-scale feature attention module to enhance the shallow features output by each layer of the CNN encoder and output them to the decoder, and the decoder decodes the fused multi-level features and the shallow features enhanced by the multi-scale feature attention module layer by layer to obtain the final segmentation prediction map.

[0032] like Figure 2As shown in Figure 1, the CNN encoder includes N scale encoding branches. The encoding branch of the first scale includes a 1×1 convolution layer and a large and small kernel attention module. The encoding branches of the remaining N-1 scales all include a downsampling module, a channel dimension splicing module, and a large and small kernel attention module. The output of the 1×1 convolution layer in the encoding branch of the first scale is used as the input of the downsampling module in the encoding branch of the second scale. The input of the remaining downsampling modules is the output of the channel dimension splicing module in the encoding branch of the previous scale. The output of all channel dimension splicing modules is also used as the input of the large and small kernel attention modules in the encoding branch of the same scale. The output of the large and small core attention modules in the encoding branch of the next scale is used as one input of the channel dimension splicing module in the encoding branch of the next scale. The output of the large and small core attention modules in the encoding branch of the Nth scale is used as the input of the multi-scale feature fusion module CNN encoder. The decoder includes N-1 levels of decoding modules connected in cascade. The multi-scale feature fusion module is used as the input of the first level decoding module. The other input of each level of decoding module is the output of the large and small core attention modules in the encoding branch of the corresponding scale. The output feature is extracted by the multi-scale attention module, and the final segmentation prediction map is decoded and output by the N-1th level decoding module. Figure 2 In this embodiment, N=5. Figure 2 Among the five scale coding branches in the middle part, from top to bottom are the coding branch of the first scale, the coding branch of the second scale, the coding branch of the third scale, the coding branch of the fourth scale and the coding branch of the fifth scale. Figure 2 In the four-level decoding module on the right side of the figure, from bottom to top, there are the first-level decoding module, the second-level decoding module, the third-level decoding module, and the fourth-level decoding module. The method of this embodiment uses the CNN encoder to extract small detail features and large-scale context features and adaptively selects and enhances features of different scales. The encoding branch constitutes a multi-scale input of the CNN encoder-side features, and the Transformer encoder is used to extract global features; the multi-scale feature fusion module is used to perform information interaction between the multi-level features extracted by the dual encoder; the multi-scale feature attention module embedded in the jump connection is used to enhance the important features of the shallow layer of each layer output of the CNN encoder to promote the subsequent deep and shallow feature fusion; the decoder decodes the fused multi-level features layer by layer to output the final segmentation prediction map; the small details and large-scale context features extracted by the large and small kernel attention modules of the CNN encoder, and the global features extracted by the Transformer encoder are used to interact with each other through the multi-scale feature fusion module, and the output features are passed through the decoder composed of convolutional layers. The deep features of the decoder are combined with the enhanced shallow features of the CNN encoder, and the final segmentation prediction map is obtained through layer-by-layer decoding.

[0033] The large and small kernel attention module uses the small kernel convolution module and the large-scale feature selection module to extract small detail features and large-scale context features, and adaptively selects and enhances large-scale context features of appropriate scales, and performs information interaction between features to obtain local small detail features and large-scale context features after feature fusion. The large and small kernel attention module includes a small kernel convolution module, a large-scale feature selection module, and a feature interaction module, wherein the small kernel convolution module and the large-scale feature selection module are dual-branch parallel structures, and the features of the two branches are added and fused, and then the information interaction between features of different scales is realized through the feature interaction module. Figure 3 As shown, the processing of the input feature map by the large and small core attention modules includes: extracting large-scale context features from the input feature maps of the large and small core attention modules through a large-scale feature selection module, and extracting small detail features from the input feature maps of the large and small core attention modules through a small-core convolution module, and then adding the large-scale context features and the small detail features element by element; then the features obtained by element-by-element addition are passed through a feature interaction module to obtain fully integrated small detail and large-scale context output data, and the feature interaction module performs maximum pooling and average pooling on the features obtained by element-by-element addition and then splices them in the channel dimension, and then passes through a convolution layer with a convolution kernel size of 7×7 and a Sigmoid activation function to perform matrix multiplication with the features obtained by element-by-element addition, and finally adds the features obtained by the matrix multiplication operation to the small detail features element by element, and then passes through a 3×3 convolution layer, a 3×3 dilated convolution layer with a dilation rate d of 2, and a 1×1 convolution layer to obtain fully integrated small detail and large-scale context output data. The feature interaction module includes a parallel maximum pooling layer and an average pooling layer. The data obtained from the two branches are concatenated in the channel dimension and then reduced in dimension using a 7×7 convolution layer. The data channel dimension is converted from two dimensions to one dimension. The output data is activated by the sigmoid function to obtain the corresponding spatial weight, and the weight is multiplied by the data input into the module. The output data is then added to the data output by the small-core convolution module and further feature fused through a 3×3 convolution layer, a 3×3 dilation group convolution layer with a dilation rate of 2, and a 1×1 convolution layer to obtain fully fused small details and large-scale context output data. The large and small-core attention modules achieve a wider and deeper capture of the features of objects of different scales in the input image. The small-core convolution layer is used to extract features such as edge details of the object. For large-size objects, the decomposed large-core module is used to capture large-scale context features more accurately. The feature interaction module has made an important contribution to improving the fusion expression ability between features of different scales in the network and avoiding overfitting.

[0034] like Figure 4As shown, the large-scale feature selection module extracts large-scale context features including: extracting preliminary features from the input feature map of the large-scale feature selection module through a 1×1 convolution layer and a 3×3 convolution layer; obtaining two large-scale context features from the preliminary features through large kernel decomposition modules with different receptive fields, adding the two large-scale context features obtained from the large kernel decomposition module element by element, and then performing feature adaptive enhancement through a feature adaptive enhancement module, wherein the feature adaptive enhancement module includes a global average pooling, two linear layers and a Softmax function connected in sequence, wherein the two linear layers are used for dimensionality reduction and dimensionality increase, respectively; and then performing matrix multiplication of the two large-scale context features obtained from the large kernel decomposition with the weights output by the Softmax function, and then adding the two large-scale context features element by element to obtain the final large-scale context features. The large-scale feature selection module is implemented by two large-core decomposition modules of different scales and a feature adaptive enhancement module; the large-core decomposition module consists of depthwise separable convolution, hole group convolution and 1×1 convolution, with two scales in total, and the overall receptive fields are 17 and 41 respectively; the input data is input into the two large-core decomposition modules after passing through a 1×1 convolution layer and a 3×3 convolution layer, and the obtained features are adaptively enhanced to the appropriate scale through the feature adaptive enhancement module to obtain the enhanced large-scale context output data; the feature adaptive enhancement module consists of a global average pooling layer and two linear layers, which are used for downscaling respectively. The matrix dimension of the input data is N×H×W×C. After the global average pooling layer, the output matrix dimension is N×1×1×C. The two linear layers convert the channel dimension of the data from 2 / C to C in turn. Then, the activation function outputs two channel weights α and β. The weights are multiplied by the two large core decomposition modules respectively and then the data of the two are added to obtain the output data extracted by the large-scale feature selection module. Among them, N represents the batch size of the input image, H and W represent the height and width of the input image, and the matrix dimension of is N×1×1×C. The batch size is the number of images taken for training once. For example, when the batch size is 8, it means that the network trains 8 ultrasound images at a time. Here, it refers to the batch of input images. In addition, the size of the original ultrasound image and the label image is the same. The large-scale feature selection module uses convolutional layers of multiple different scales to extract features of different scales, and adaptively selects and enhances the important scales, which improves the feature extraction ability of the network and the effective expression of features of different scales in the image, and obtains richer and more accurate feature data.

[0035] like Figure 3 As shown in the figure, the small-core convolution module is composed of a 1×1 convolution layer and a 5×5 convolution layer in series; the input data of the small-core convolution module is processed by a 1×1 convolution layer and a 5×5 convolution layer in sequence, and then added to the input data of the large-scale feature selection module to obtain output data of different scale features.

[0036] In this embodiment, decomposing the preliminary large-scale context features through large kernels with different receptive fields respectively means decomposing the preliminary large-scale context features through a first large kernel decomposition module (LKDM1) and a second large kernel decomposition module (LKDM2) with different receptive fields respectively. Figure 5 As shown in FIG. 1 , the first large core decomposition module and the second large core decomposition module both include a 5×5 depthwise separable convolution layer, a 5×5 dilated convolution module, a 7×7 dilated convolution module, a 3×3 dilated convolution module, and a 1×1 convolution layer connected in sequence, wherein the number of dilated group convolutions is 8, the dilated ratio d of the 5×5 dilated convolution module, the 7×7 dilated convolution module, and the 3×3 dilated convolution module in the first large core decomposition module is 1, and the dilated ratio d of the 5×5 dilated convolution module, the 7×7 dilated convolution module, and the 3×3 dilated convolution module in the second large core decomposition module is 3. The input data is subjected to depthwise separable convolution to extract spatial feature data, and the large-scale context features are further extracted through dilated group convolutions of three different convolution kernel sizes, and the features are sent to the 1×1 convolution layer for feature interaction in the channel dimension to obtain the large-scale context feature data output by the large core decomposition module. The first large kernel decomposition module and the second large kernel decomposition module are different from the conventional standard large kernel convolution. The appropriate and reasonable decomposition method enables the module to fully express the features of various levels of the image, greatly improving the feature extraction and generalization capabilities of the network.

[0037] like Figure 6 As shown in FIG. 1 , the multi-scale feature fusion module fuses the multi-level features extracted by the CNN encoder and the Transformer encoder, including: splicing the multi-level features extracted by the CNN encoder and the Transformer encoder through the channel dimension, and then passing through a 1×1 convolution layer to convert the channel dimension from 2048 dimensions to 1024-dimensional features, and then extracting features from the 1024-dimensional features through three branches, namely, a 1×1 convolution layer, a 3×3 convolution layer, and a 3×3 dilated convolution layer with a dilation rate d of 2, and then splicing the features extracted by the three branches with the original 1024-dimensional features in the channel dimension and passing through a 1×1 convolution layer to obtain the fused multi-level features. In this embodiment, the multi-scale feature fusion module splices the data output by the dual-branch encoder in channel dimension, uses a 1×1 convolution layer to convert the channel dimension from 2048 to 1024, and then inputs them into a 1×1 convolution layer, a 3×3 convolution layer, a 3×3 dilated convolution layer with a dilation rate of 2, and a direct connection operation, respectively. After splicing the data output by each convolution layer in channel dimension, a 1×1 convolution layer is used to reduce the channel dimension to obtain the output data of the multi-scale feature fusion module.

[0038] like Figure 7As shown, the multi-scale attention module extracts output features including: extracting features of the input features of the multi-scale attention module through a first branch consisting of a 1×1 convolution layer and a 3×3 convolution layer, extracting features through a second branch consisting of a 1×1 convolution layer and a 3×3 dilated convolution layer with a dilation rate d of 2, concatenating the channel dimensions of the features extracted by the two branches, and then reducing the channel dimension through a 1×1 convolution layer, and then performing maximum pooling and average pooling on the features after reducing the channel dimension, and then performing matrix multiplication on the features after reducing the channel dimension after linear layer dimensionality reduction, linear layer dimensionality increase, and Sigmoid activation function activation, and obtaining the output features. In this embodiment, the multi-scale attention module inputs the output data from each layer of the CNN encoder into the module. First, the data is input into a branch consisting of a 1×1 convolution layer, a 3×3 convolution layer, a 1×1 convolution layer, and a 3×3 dilated convolution layer with a dilated rate of 2. The output features are concatenated in the channel dimension and then a 1×1 convolution layer is used to reduce the channel dimension. The output features are input into a channel attention layer to enhance the channel dimension of the feature data to obtain the output data of the multi-scale attention module.

[0039] like Figure 2 As shown in the figure, the Transformer encoder consists of a patch embedding layer, a linear layer, and n transformer layers connected in sequence. The patch embedding layer is a convolution layer with a convolution kernel size of 16×16 and a stride of 16. The ultrasound image with a dimension of H×W×3 is passed through the patch embedding layer to obtain features with dimensions of H / 16×W / 16×D, where H and W are the height and width of the ultrasound image, and D is the output data dimension (the default is 768). Figure 2 , wherein transformer layer (1) to transformer layer (n) are n transformer layers, and in this embodiment, n=12, that is, the transformer encoder used in the present invention includes 12 transformer layers.

[0040] like Figure 8 As shown in the figure, the transformer layer consists of an embedded patch layer, two normalization layers, a multi-head attention layer, a multi-layer perceptron layer and two residual connections. The input feature data passes through the embedded patch layer, the first normalization layer and the multi-head attention layer, and then is added element-by-element to the feature data output by the embedded patch layer through the first residual connection. The obtained feature data then passes through the second normalization layer and the multi-layer perceptron layer, and then is added element-by-element to the first residual connection through the second residual connection as the output data of the transformer layer.

[0041] In this embodiment, the decoding module is composed of a transposed convolution with a convolution kernel size of 2×2 and two 3×3 convolution layers connected in sequence. In this embodiment, the decoder side decodes the extracted features through the decoding module to achieve accurate positioning of the target. The decoder side applies four identical decoding modules to restore the image size and output the final segmentation prediction map of the target area.

[0042] In this embodiment, the parameters of the convolutional layers in the large and small kernel attention modules, the Transformer encoder, the multi-scale feature fusion module, the multi-scale attention module, and the decoder network are determined by optimizing the loss function when the training set is used for training, so that the loss function is minimized; wherein, the training set includes ultrasound images and corresponding label images; the loss function refers to the error between the segmentation prediction map obtained by using the ultrasound image segmentation network model and the corresponding label image. As an optional implementation, the loss function used in this embodiment when training the ultrasound image segmentation network model is The function expression is: , , , In the above formula, is the binary cross entropy loss, is the dice loss, is the sample size, is the label of the i-th pixel value, is the pixel value of the i-th pixel, is the label graph, is the final segmentation prediction map.

[0043] In order to verify the effectiveness of the method of this embodiment, the BUSI and BUSC breast ultrasound image datasets and the UNS nerve ultrasound image dataset are used for testing. In the experiment, an ultrasound image dataset with one-to-one correspondence between the original image and the label is used. The method of this embodiment is mainly compared with four classic segmentation methods: UNet, UNet++, Attention-UNet, and SKUNet, two segmentation methods combining CNN and Transformer: TransUNet and CMUNext, and two large kernel convolution segmentation methods: CMU-Net and MDA-Net. The objective evaluation indicators used include intersection over union (IOU), Dice coefficient, Hausdorff distance, accuracy, and precision. The results are shown in Tables 1, 2, and 3.

[0044] Table 1: Objective evaluation indicators of ultrasound image segmentation results of BUSI dataset

[0045] Table 2: Objective evaluation indicators of ultrasound image segmentation results of BUSC dataset

[0046] Table 3: Objective evaluation indicators of ultrasound image segmentation results of UNS dataset

[0047] As shown in Table 1, Table 2 and Table 3, compared with the comparison method, the method of this embodiment has achieved relatively excellent performance in five indicators of intersection over union (IOU), Dice coefficient, Hausdorff distance, accuracy and precision on the BUSI, BUSC and UNS ultrasound image datasets.

[0048] In addition, the comparison results of some samples between the method of this embodiment and the existing method are as follows: Fig. 9 , Fig.10 , Fig.11 and Fig.12 As shown in Figure 9 and Fig.10In the figure, the ultrasound images in the first row (a-1) to (e-1) to the fourth row (a-4) to (e-4) are from the BUSI dataset, and the categories are benign, benign, malignant, and normal, respectively. In the first row (a-1) to (e-1), the label image (a-1) has two lesion regions. Some networks (such as the TransUNet method and the CMUNext method) can only predict one of the lesion targets, while most other networks (such as the SKUNet method and the CMU-Net method) have low accuracy in predicting the lesion region. In the second row (a-2) to (e-2), the target region of the label image is solid. More than half of the regions in the segmentation prediction map of the comparison method are hollow. In the third row (a-3) to (e-3), the segmentation prediction maps of half of the segmentation methods (i.e., the UNet method, the UNet++ method, the Attention-UNet method, the TransUNet method, and the MDA-Net method) contain redundant regions. The remaining methods have a rough prediction of the lesion region boundary. For the normal images in the fourth row (a-4) to (e-4), there are no lesions in the label image. However, all the comparison methods predicted the lesion area. The prediction errors of half of the segmentation methods (i.e., Attention-UNet method, SKUNet method, TransUNet method, CMU-Net method, and CMUNext method) are too large. It is worth emphasizing that the method of this embodiment is completely correct in predicting normal images in the BUSI dataset. Compared with other methods, the method of this embodiment has obvious advantages. The input images of the fifth row (a-5) to (e-5) and the sixth row (a-6) to (e-6) are from the BUSC dataset. The input images of the fifth row (a-5) to (e-5) contain benign breast areas, and the inputs of the sixth row (a-6) to (e-6) are malignant breast images. It can be seen that the predicted masks of many comparison methods are relatively rough for the edge details of the target area. Fig.11 and Fig.12In the first row (a-1) to (e-1), some comparison methods (i.e., Attention-UNet method, SKUNet method, TransUNet method, CMUNext method, and MDA-Net method) predict that there is an extra brachial plexus region in the upper right corner. In the second row (a-2) to (e-2), the segmentation prediction maps of many comparison methods (i.e., UNet method, UNet++ method, SKUNet method, and CMU-Net method) show incomplete brachial plexus regions, resulting in the loss of many edge details in the target. In the last two rows without brachial plexus regions, it can be observed that most segmentation methods have incorrect predictions. Compared with other segmentation methods, the method of this embodiment has better segmentation performance in the brachial plexus region. It can effectively capture image edge details, suppress interference from irrelevant tissues around the target, and distinguish regions similar to the target in a global range. The method of this embodiment enhances the robustness of the target segmentation network under complex backgrounds.

[0049] In order to verify the contribution of each module in the ultrasound image segmentation network model of the method of this embodiment to the whole, this embodiment makes adjustments on the baseline model respectively, and conducts ablation experiments by comparing the four improved parts of the large and small kernel attention module (LSKA), multi-scale attention module (MSA), Transformer module and multi-scale feature fusion module (MSF2). The target segmentation accuracy of the BUSI dataset is experimentally tested to evaluate the effectiveness of each part. The experimental results are shown in Table 4.

[0050] Table 4: Objective evaluation indexes of ablation experiment segmentation results of the method of the present invention on BUSI ultrasound images

[0051] It can be clearly seen from Table 4 that the performance of the segmentation network proposed in this embodiment is significantly improved compared with the baseline network (Unet). Specifically, the intersection over union (IOU) index is improved by 7.80%, the Dice coefficient index is improved by 6.80%, the Hausdorff distance index is improved by 5.26, the accuracy index is improved by 0.36%, and the precision index is improved by 1.97%. In the second row of Table 3, the intersection over union index of the proposed LSKA module is improved by 4.86%. This shows that the combination of large and small kernel feature extraction modules enables the segmentation algorithm to learn more reliable breast ultrasound image details and large-scale context features.

[0052] In addition, in order to verify the contribution of different levels of feature extraction modules in the method of this embodiment to the whole, this embodiment conducts ablation experiments on the ultrasound image segmentation network model of the method of this embodiment by comparing and removing the small core module (SKM), multi-scale feature selection module (LSFS) and Transformer module. The target segmentation accuracy of the BUSI dataset is experimentally tested to evaluate the effectiveness of each part. The experimental results are shown in Table 5.

[0053] Table 5: Objective evaluation indicators of ablation experiment segmentation results of different levels of feature extraction modules on BUSI ultrasound images

[0054] The first row of Table 5 is the baseline network without the small kernel module (SKM), multi-scale feature selection module (LSFS), and Transformer module. The three levels of feature extraction modules can extract local small details, large-scale context, and global features. The second to fourth rows of Table 5 are ablation experiments of the baseline combined with three separate feature extraction modules. Figure 5 It can be observed that each independent module presents better segmentation effect than the baseline module. The fifth to seventh rows are the network results of the baseline combined with the two modules. The results show that the network performance of the two modules is better than that of a single module. The last row is the method proposed by combining the baseline with three levels of feature extraction modules. The segmentation performance of the method in this embodiment is better than that of other networks. The ablation experiment results confirm that the multi-level feature extraction network has an impact on improving the segmentation accuracy of ultrasound images.

[0055] In summary, the ultrasound image segmentation network model of the ultrasound image segmentation method based on multi-level feature extraction in this embodiment is a dual encoder and single decoder structure. The input image extracts small detail features and large-scale local context features through the large and small kernel attention modules of the CNN encoder, and the small detail features and large-scale context features of different scales are adaptively selected and enhanced through adaptive weighted weights. The global features of the image are extracted based on the Transformer encoder; then the multi-level features extracted by the two encoders are used for information interaction between different levels of features through the multi-scale feature fusion module; in addition, this embodiment designs a multi-scale attention module on the jump connection to retain important shallow features, so as to facilitate the subsequent deep and shallow feature fusion; the fused multi-level features are decoded layer by layer by the decoder to obtain the final segmentation prediction map. The present invention adopts large and small kernel attention modules to adaptively select and enhance features of different scales. The large kernel decomposition method reduces the model parameters while achieving a large receptive field, and also improves the network's feature extraction ability and model generalization ability; the multi-level feature extraction method designed by the method makes the obtained features more sufficient, which is more conducive to the improvement of image segmentation performance. The method of this embodiment fully extracts small detail features, large-scale context features and global features in the image through a parallel dual encoder structure, uses a small convolution kernel module and a large kernel decomposition module of different scales to extract small detail features and large-scale features, and adaptively selects and enhances large-scale features, and interacts information between features through convolution and attention layers, uses two large kernel decomposition modules with different receptive fields to extract large-scale context features, adds the output data and obtains two different attention weights through a channel weight generation layer, multiplies the weights by the corresponding large kernel decomposition module results, obtains adaptively selected enhanced large-scale context features, decomposes the large kernel into a combination of convolutions of different types and scales, and achieves large-scale feature extraction while significantly reducing the number of model parameters. And compared with the traditional decomposition form, the design of this unit is also more efficient and lightweight for feature extraction in the image. Compared with the prior art, the method of this embodiment has the following significant advantages: 1. The multi-level feature extraction design of the ultrasound image segmentation network model of this embodiment realizes the extraction of small detail features, large-scale context features to global features, improves the information capture efficiency of the input image, and further improves the segmentation performance of the network. 2. The large and small kernel attention modules of this embodiment can capture the features of objects of different sizes in the input image, where the edge details of the object are extracted using a small kernel convolution layer, and the large-scale context features of large-scale objects are captured more accurately using a decomposed large kernel module. The feature interaction module has made an important contribution to improving the fusion expression ability between features of different scales in the network and avoiding overfitting.3. The large-scale feature selection module of this embodiment uses convolutional layers of multiple different scales to extract features of different scales, and adaptively selects and enhances important scales, thereby improving the network's feature extraction capability and the effective expression of features of different scales in the image, and obtaining richer and more accurate feature data. 4. The large-core decomposition module of this embodiment reduces the number of model parameters and improves the trainability of the model while fully extracting large-scale context features through an effective decomposition method.

[0056] In addition, this embodiment also provides an ultrasound image segmentation system based on multi-level feature extraction, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction.

[0057] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored. The computer program or instruction is programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction through a processor.

[0058] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction through a processor.

[0059] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present application may be in the form of methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0060] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. An ultrasonic image segmentation method based on multi-level feature extraction, characterized in that: The method comprises performing ultrasonic image segmentation on an input ultrasonic image using an ultrasonic image segmentation network model to obtain a segmentation prediction map, wherein the ultrasonic image segmentation network model comprises a dual-branch encoder, a multi-scale feature fusion module, a multi-scale feature attention module and a decoder, wherein the dual-branch encoder comprises a CNN encoder and a Transformer encoder, and performing ultrasonic image segmentation on an input ultrasonic image using an ultrasonic image segmentation network to obtain a segmentation prediction map comprises: extracting small detail features and large-scale context features of different scales from the input ultrasonic image through a CNN encoder and adaptively selecting and enhancing small detail features and large-scale context features of different scales using adaptive weighted weights; extracting global features from the input ultrasonic image through a Transformer encoder; fusing multi-level features extracted by the CNN encoder and the Transformer encoder through a multi-scale feature fusion module; enhancing shallow features outputted by each layer of the CNN encoder using a multi-scale feature attention module and outputting the enhanced shallow features to the decoder, and decoding the fused multi-level features and the shallow features enhanced by the multi-scale feature attention module layer by layer through the decoder to obtain a final segmentation prediction map.

2. The ultrasonic image segmentation method based on multi-level feature extraction according to claim 1, characterized in that: The CNN encoder includes encoding branches of N scales, the encoding branch of the first scale includes a 1×1 convolution layer and a large and small kernel attention module, and the encoding branches of the remaining N-1 scales all include a downsampling module, a channel dimension splicing module and a large and small kernel attention module. The output of the 1×1 convolution layer in the encoding branch of the first scale is used as the input of the downsampling module in the encoding branch of the second scale, and the input of the remaining downsampling modules is the output of the channel dimension splicing module in the encoding branch of the previous scale. The outputs of all channel dimension splicing modules are also used as the input of the large and small kernel attention modules in the encoding branch of the same scale. The output of the large and small kernel attention modules in the encoding branch of the next scale is used as one input of the channel dimension splicing module in the encoding branch of the next scale, and the output of the large and small kernel attention modules in the encoding branch of the Nth scale is used as the input of the multi-scale feature fusion module CNN encoder end. The decoder includes N-1 levels of decoding modules connected in cascade, and the multi-scale feature fusion module is used as the input of the first level decoding module. Another input of each level of decoding module is the output of the large and small kernel attention modules in the encoding branch of the corresponding scale, and the output feature is extracted by the multi-scale attention module, and the final segmentation prediction map is decoded and output by the N-1th level decoding module.

3. The ultrasonic image segmentation method based on multi-level feature extraction according to claim 2, characterized in that: The processing of the input feature map by the large and small core attention module includes: extracting large-scale context features from the input feature map of the large and small core attention module through a large-scale feature selection module, and extracting small detail features from the input feature map of the large and small core attention module through a small-core convolution module, and then adding the large-scale context features and the small detail features element by element; then obtaining the features obtained by element-by-element addition through a feature interaction module to obtain fully integrated small detail and large-scale context output data, and the feature interaction module performs maximum pooling and average pooling on the features obtained by element-by-element addition and then splices them in the channel dimension, and then passes through a convolution layer with a convolution kernel size of 7×7 and a Sigmoid activation function to perform matrix multiplication operation with the features obtained by element-by-element addition, and finally adds the features obtained by the matrix multiplication operation and the small detail features element by element, and then passes through a 3×3 convolution layer, a 3×3 void convolution layer with a void rate d of 2, and a 1×1 convolution layer to obtain fully integrated small detail and large-scale context output data.

4. The ultrasonic image segmentation method based on multi-level feature extraction according to claim 3, characterized in that: The large-scale feature selection module extracts large-scale context features, including: extracting preliminary features from the input feature map of the large-scale feature selection module through a 1×1 convolution layer and a 3×3 convolution layer; obtaining two large-scale context features through large-core decomposition modules with different receptive fields, adding the two large-scale context features element by element, and then performing feature adaptive enhancement through a feature adaptive enhancement module, wherein the feature adaptive enhancement module includes a global average pooling, two linear layers, and a Softmax function connected in sequence, wherein the two linear layers are used for dimensionality reduction and dimensionality increase, respectively; then performing matrix multiplication of the two large-scale context features obtained by the large-core decomposition with the weights output by the Softmax function, and then adding the two large-scale context features element by element to obtain the final large-scale context features; the small-core convolution module is composed of a 1×1 convolution layer, a 5×5 convolution layer, and a Softmax function connected in sequence. The method comprises the following steps: the first large kernel decomposition module and the second large kernel decomposition module are connected to form a large-scale context feature extraction module; the method comprises the following steps: extracting the preliminary large-scale context feature through a first large kernel decomposition module and a second large kernel decomposition module with different receptive fields, wherein the first large kernel decomposition module and the second large kernel decomposition module each comprise a 5×5 depthwise separable convolution layer, a 5×5 dilated convolution module, a 7×7 dilated convolution module, a 3×3 dilated convolution module and a 1×1 convolution layer connected in sequence, wherein the number of groups of the dilated group convolution is 8, the dilated rates d of the 5×5 dilated convolution module, the 7×7 dilated convolution module and the 3×3 dilated convolution module in the first large kernel decomposition module are all 1, and the dilated rates d of the 5×5 dilated convolution module, the 7×7 dilated convolution module and the 3×3 dilated convolution module in the second large kernel decomposition module are all 3.

5. The ultrasonic image segmentation method based on multi-level feature extraction according to claim 2, characterized in that: The multi-scale feature fusion module fuses the multi-level features extracted by the CNN encoder and the Transformer encoder, including: splicing the multi-level features extracted by the CNN encoder and the Transformer encoder through the channel dimension, converting the channel dimension from 2048 dimensions to 1024-dimensional features through a 1×1 convolution layer, and then extracting features from the 1024-dimensional features through three branches of a 1×1 convolution layer, a 3×3 convolution layer, and a 3×3 hole convolution layer with a hole rate d of 2, and then splicing the features extracted by the three branches with the original 1024-dimensional features in the channel dimension, and then reducing the channel dimension to 1024 dimensions through a 1×1 convolution layer to obtain the fused multi-level features.

6. The ultrasonic image segmentation method based on multi-level feature extraction according to claim 2, characterized in that: The multi-scale attention module extracts output features including: extracting features of the input features of the multi-scale attention module through a first branch consisting of a 1×1 convolution layer and a 3×3 convolution layer, extracting features through a second branch consisting of a 1×1 convolution layer and a 3×3 dilated convolution layer with a dilation rate d of 2, splicing the channel dimensions of the features extracted by the two branches, and then reducing the channel dimension through a 1×1 convolution layer, and then performing maximum pooling and average pooling on the features after the channel dimension reduction, and then splicing the channel dimension, and then performing matrix multiplication operation on the features after the channel dimension reduction after linear layer dimensionality reduction, linear layer dimensionality increase, and Sigmoid activation function activation to obtain output features.

7. The ultrasonic image segmentation method based on multi-level feature extraction according to claim 1, characterized in that: The Transformer encoder consists of an image block embedding layer, a linear layer and multiple transformer layers connected in sequence, wherein the image block is embedded as a convolution layer with a convolution kernel size of 16×16 and a step size of 16. The ultrasound image with a dimension of H×W×3 is passed through the image block embedding layer to obtain features with a dimension of H / 16×W / 16×D, wherein H and W are the height and width of the ultrasound image, and D is the output data dimension. The transformer layer consists of an embedded image block layer, two normalization layers, a multi-head attention layer, The invention is composed of a multi-layer perceptron layer and two residual connections. The input feature data passes through the embedded image block layer, the first normalization layer, and the multi-head attention layer, and then is added element by element with the feature data output by the embedded image block layer through the first residual connection. The obtained feature data passes through the second normalization layer and the multi-layer perceptron layer, and then is added element by element with the first residual connection through the second residual connection as the output data of the transformer layer. The decoding module is composed of a transposed convolution with a convolution kernel size of 2×2 and two 3×3 convolution layers connected in sequence.

8. An ultrasound image segmentation system based on multi-level feature extraction, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction according to any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the ultrasound image segmentation method based on multi-level feature extraction according to any one of claims 1 to 7 through a processor.

Images