Ultrasound image segmentation method, system, device and medium based on deep learning

Abstract

This invention relates to the fields of image segmentation and medical imaging technology, and particularly to a method, system, device, and medium for ultrasound image segmentation based on deep learning. The method includes the following steps: extracting multiple feature maps with different spatial resolutions from the current ultrasound image to be detected; optimizing the missing feature information in each feature map layer using complementary information between the different layers, resulting in optimized multi-layer feature maps; and performing an upsampling convolution operation on the optimized multi-layer feature maps to obtain an ultrasound image segmentation map at the same scale as the RGB image of the current ultrasound image to be detected. This method fully utilizes the complementary information between different layers of feature maps, performing targeted optimization on each layer to improve the segmentation results, and achieving higher detection accuracy.

Description

Technical Field

[0001] This invention relates to the fields of image segmentation and medical imaging technology, and in particular to a method, system, device and medium for ultrasound image segmentation based on deep learning. Background Technology

[0002] Accurate segmentation of lesion areas in ultrasound images using automated computers is crucial for computer-aided clinical examination and treatment. This task can be formulated as a problem of binary annotation of a single ultrasound image, i.e., automatically annotating lesion areas in ultrasound data at the pixel level using a computer-aided system.

[0003] Currently, in clinical treatment, doctors rely on accurate tumor segmentation results for patient diagnosis. However, due to the thin boundaries, large variations in shape, and uneven intensity distribution within liver lesions, accurate ultrasound segmentation of liver lesion regions is extremely difficult. Traditional ultrasound image segmentation methods often first remove speckle noise before segmenting the ultrasound image, or rely on manual feature extraction. However, due to the limited discriminative power of the images, these methods often cannot handle ultrasound images with strong artifacts, making ultrasound video segmentation a technically challenging task. Recently, deep learning-based image processing algorithms have shown superior performance compared to traditional algorithms. However, current deep learning-based image segmentation algorithms simply concatenate the feature maps captured by the network without performing targeted optimization and completion for each layer of feature maps, thus their segmentation accuracy remains unsatisfactory. Summary of the Invention

[0004] This invention provides a deep learning-based ultrasound image segmentation method, system, device, and medium that intelligently processes input ultrasound images and automatically acquires lesion masks, thereby improving the speed and accuracy of automatic lesion detection in computer-assisted ultrasound images.

[0005] To address the aforementioned technical problems, the first aspect of this invention provides a deep learning-based ultrasound image segmentation method, comprising the following steps:

[0006] Based on the current ultrasound image to be detected, multiple feature maps with different spatial resolutions are extracted.

[0007] By utilizing the complementary information between the feature maps of different layers, the missing feature information of each layer of the feature map is optimized to obtain an optimized multi-layer feature map.

[0008] An upsampling convolution operation is performed on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected.

[0009] In some exemplary embodiments, the step of optimizing the missing feature information in each layer of feature maps by utilizing complementary information between the feature maps of different layers to obtain optimized multi-layer feature maps specifically includes:

[0010] Each layer of feature map is selected as the target feature map in sequence, and the remaining layer feature maps other than the target feature map are aggregated to generate an aggregated feature map.

[0011] Dual attention weights are learned from the aggregated feature map to optimize each layer of the target feature map, resulting in an optimized multi-layer feature map.

[0012] In some exemplary embodiments, the step of sequentially selecting each layer of feature maps as the target feature map and aggregating the remaining layer feature maps (excluding the target feature map) to generate an aggregated feature map specifically includes:

[0013] Select the i-th layer feature map as the target feature layer Fi; where i = 1, 2, 3, 4;

[0014] Feature maps Ta, Tb, and Tc are generated using the remaining three feature layers; among them, Ta has the largest spatial resolution and Tc has the smallest spatial resolution.

[0015] After downsampling Ta to give it the same spatial resolution as Tb, Ta is optimized using a 3×3 convolutional layer to obtain the optimized Ta. Then, the optimized Ta is multiplied element-wise with Tb to generate the feature map Tab.

[0016] We optimize Tab using another 3×3 convolutional layer to obtain the optimized Tab; then we perform element-wise addition between Tab and Ta to generate the feature map Taba, which is defined as follows:

[0017] Taba=Conv(Conv(Ta)+Tb×Conv(Ta)) (1)

[0018] Where Conv represents a 3×3 convolutional layer;

[0019] Next, Ta and Tb are downsampled to the same spatial resolution as Tc, and then two 3×3 convolutional layers are applied on the two downsampled layers of Ta and Tb to obtain the convolution result;

[0020] Multiply the convolution result by Tc to obtain the feature map Tabc:

[0021] Tabc=Tc×Conv(Tb)×Conv(Ta) (2)

[0022] Add Taba and Tabc to generate the aggregated feature map Fg:

[0023] Fg=Conv(Taba+Tabc) (3).

[0024] In some exemplary embodiments, the step of learning dual attention weights from the aggregated feature map to optimize each layer of the target feature map and obtain an optimized multi-layer feature map specifically includes:

[0025] A global attention weight map Za is learned by utilizing a 3×3 convolutional layer and a normalized exponential activation function on the aggregated feature map Fg, and then subtracted from 1 to produce an inverse self-attention map Zr:

[0026] Za = Sigmoid(Conv(Fg)) (4)

[0027] Zr=1-Za (5)

[0028] Wherein, Sigmoid represents the Sigmoid layer, Conv represents the 3×3 convolutional layer, and the dual attention weight map includes the global attention weight map Za and the anti-self attention map Zr;

[0029] Two 3×3 convolutional layers are applied to the inverse self-attention map Zr to obtain Kr and Vr; where Kr is H×W×C / 8 and Vr is H×W×C / 2;

[0030] Two 3×3 convolutional layers on the global attention weight map Za are applied to obtain Ka and Va; where Ka is H×W×C / 8 and Va is H×W×C / 2.

[0031] A 3×3 convolutional layer is used on the target feature layer Fi to obtain the vector Kf of H×W×C / 8 and the vector Vf of H×W×C / 2;

[0032] The spatial scale of Kf is adjusted to C / 8×HW, and the spatial scale of Kr is adjusted to HW×C / 8. These two integer results are then multiplied by a matrix, and the Softmax activation function is applied to calculate the nonlocal similarity mapping Sr of the key mapping vectors Kf and Kr over all pixels. Sr is calculated as follows:

[0033] Sr=Softmax(Kr×Kf) (6)

[0034] Transform Ka into HW×C / 8, and perform matrix multiplication between Ka and Kf to obtain the nonlocal similarity mapping weight Sa:

[0035] Sa = Softmax(Ka × Kf) (7)

[0036] A new similarity weight map Sd is generated by multiplying Sr and Sa element by element, and Sd is multiplied by Vf to obtain the multiplication result.

[0037] The multiplication result is concatenated with Vr and Va to obtain the concatenated result;

[0038] The concatenated result is convolved to obtain the optimized mapping of the i-th DSR module. The mathematical formula for the optimized feature map is as follows:

[0039] Fi=Conv(Concate(Vr,Va,Vf×(Sr*Sa))) (8)

[0040] Where * represents element-wise matrix multiplication, × represents matrix dot product, Concate is the concatenation operation of feature map channel dimensions, and Conv is a convolutional layer with a kernel size of 3×3.

[0041] Generate an optimized multi-layer feature map; wherein the optimized multi-layer feature map includes a first optimized feature layer D1, a second optimized feature layer D2, a third optimized feature layer D3, and a fourth optimized feature layer D4.

[0042] In some exemplary embodiments, the step of performing an upsampling convolution operation on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected specifically includes:

[0043] The fourth optimized feature layer D4 is upsampled to the same resolution as the third optimized feature layer D3, and then concatenated with the third optimized feature layer D3 along the channel dimension. Then, a new feature map D3' is obtained through a convolution operation.

[0044] Then, D3' is upsampled to the same resolution as the second optimized feature layer D2, and concatenated with the second optimized feature layer D2 along the channel dimension. Then, a new feature map D2' is obtained through a convolution operation.

[0045] Then D2' is upsampled to the same resolution as the second optimized feature layer D1, and concatenated with the first optimized feature layer D1 along the channel dimension. Then, a new feature map D1' is obtained through a convolution operation.

[0046] Then, an upsampling operation is performed on D1' so that the RGB image scale of D1' is the same as that of the current ultrasound image to be detected, and it is output as an ultrasound image segmentation map.

[0047] In some exemplary embodiments, the step of extracting multiple feature maps with different spatial resolutions based on the current ultrasound image to be detected specifically includes:

[0048] Using the current ultrasound image to be detected as input, the first layer feature map F1 is generated;

[0049] The first layer feature map F1 is used as input to generate the second layer feature map F2;

[0050] Using the second-layer feature map F2 as input, the third-layer feature map F3 is generated;

[0051] Using the third-layer feature map F3 as input, the fourth-layer feature map F4 is generated.

[0052] In some exemplary embodiments, after the step of extracting multiple layers of feature maps with different spatial resolutions based on the current ultrasound image to be detected, the method further includes: dimensionality compression of each layer of the feature maps. This is used to accelerate subsequent feature extraction.

[0053] A second aspect of the present invention provides a deep learning-based ultrasound image segmentation system, the system comprising:

[0054] The encoding module is used to extract multiple feature maps with different spatial resolutions based on the current ultrasound image to be detected.

[0055] The aggregation module is used to sequentially select each layer of feature map as the target feature map, and perform aggregation processing on the remaining layer feature maps other than the target feature map to generate an aggregated feature map.

[0056] A dual attention module is used to learn dual attention weights from the aggregated feature map to optimize each layer of the target feature map, thereby obtaining an optimized multi-layer feature map.

[0057] The decoding module is used to perform upsampling convolution on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected.

[0058] A third aspect of the present invention provides an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the deep learning-based ultrasound image segmentation method as described above.

[0059] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described deep learning-based ultrasound image segmentation method.

[0060] The technical solution provided by this invention has at least the following advantages:

[0061] This invention proposes the first lesion segmentation method applicable to ultrasound images. Unlike traditional image segmentation methods, this invention fully utilizes the complementary information between feature maps of different layers, performing targeted optimization on each layer to improve segmentation results and achieve higher detection accuracy. Furthermore, this invention proposes a dual attention module. Compared to convolutional neural networks, this module can specifically optimize missing features in the feature maps during feature learning, filling in the missing key information in each feature map and suppressing redundant tumor regions. Attached Figure Description

[0062] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations do not constitute a limitation on the embodiments, and unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0063] Figure 1 This is a schematic diagram of the PVT network framework in the prior art;

[0064] Figure 2 A schematic diagram of the model framework for a deep learning-based ultrasound image segmentation method provided by the present invention;

[0065] Figure 3 A schematic diagram of the i-th layer DSR module of the Transformer in a deep learning-based ultrasound image segmentation method provided by the present invention;

[0066] Figure 4 This is a schematic diagram of the feature map aggregation module in a deep learning-based ultrasound image segmentation method provided by the present invention.

[0067] Figure 5 A schematic diagram of an ultrasound image segmentation system based on deep learning provided by the present invention;

[0068] Figure 6 This is a schematic diagram of the structure of an electronic device provided by the present invention. Detailed Implementation

[0069] Currently, one image segmentation method disclosed in the prior art is the Pyramid Vision Transformer (PVT) model, the overall architecture of which is as follows: Figure 1As shown, PVT consists of two parts: an encoder and a decoder. The encoder consists of four cascaded Transformer layers. For a given RGB image, the image first passes through the first Transformer layer to generate a first-layer feature map F1. Then, feature map F1 is directly input into the second Transformer layer to obtain the second-layer feature map F2, and so on, resulting in the third-layer feature map F3 and the fourth-layer feature map F4. The decoder directly takes the four feature maps (F1, F2, F3, F4) generated by the encoder as input and uses progressive upsampling convolution to obtain the final segmentation result. Specifically, the decoder upsamples F4 to the same resolution as F3 and concatenates it with F3 along the channel dimension. Then, a convolution operation is performed to obtain a new feature map F3'. F3' is then upsampled to the same resolution as F2 and concatenated with F2 along the channel dimension. A convolution operation is then performed to obtain a new feature map F2', and so on, until F1' is obtained. Finally, F1' is upsampled to the same scale as the input RGB image to become the model's segmentation result. The Transformer mentioned is a conventional structure, consisting of a linear layer, a regularization layer, a multi-head attention mechanism, and a feedforward layer.

[0070] The inventors discovered that the PVT algorithm, proposed for natural images, is a multi-layer network structure. For a given input image, the encoder generates feature maps (F1, F2, F3, F4) with different spatial resolutions. However, the decoder directly concatenates these feature maps to obtain the segmentation result for the current input image. This operation inevitably introduces noise from the encoder, affecting the final segmentation accuracy. Furthermore, each feature map layer cannot fully represent the abnormal tumor regions in the current input image; therefore, the direct concatenation operation used by the decoder further impacts the boundary delineation of the tumor regions.

[0071] To this end, this invention proposes an ultrasound image segmentation model based on a dual attention mechanism, combining the characteristics of the lesion region. This model specifically optimizes and completes the feature maps of each layer in the multi-layer network, which greatly improves the accuracy of lesion segmentation and provides more possibilities for the application of deep learning-based computer-aided systems in the field of clinical medical imaging.

[0072] The present invention provides a first aspect of a deep learning-based ultrasound image segmentation method, comprising the following steps:

[0073] S1. Based on the current ultrasound image to be detected, extract multiple feature maps with different spatial resolutions.

[0074] S2. Utilize the complementary information between the feature maps of different layers to optimize the missing feature information of each layer of feature map, and obtain the optimized multi-layer feature map.

[0075] S3. Perform an upsampling convolution operation on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected.

[0076] More specifically, S2 includes:

[0077] S21. Select each layer of feature map as the target feature map in sequence, and perform aggregation processing on the remaining layer feature maps other than the target feature map to generate an aggregated feature map.

[0078] S22. Learn dual attention weights from the aggregated feature map to optimize each layer of the target feature map, and obtain the optimized multi-layer feature map.

[0079] The second invention provides a deep learning-based ultrasound image segmentation system, such as... Figure 5 As shown, the system includes:

[0080] The encoding module 301 is used to extract multiple feature maps with different spatial resolutions based on the current ultrasound image to be detected.

[0081] The aggregation module 302 is used to sequentially select each layer of feature map as the target feature map, and perform aggregation processing on the remaining layer feature maps other than the target feature map to generate an aggregated feature map.

[0082] Dual attention module 303 is used to learn dual attention weights from the aggregated feature map to optimize each layer of target feature map, thereby obtaining an optimized multi-layer feature map;

[0083] The decoding module 304 is used to perform upsampling convolution operation on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected.

[0084] Specifically, the ultrasound video segmentation model based on a dual attention mechanism proposed in this invention, such as... Figure 2 As shown.

[0085] (I) Encoding Module

[0086] This invention uses the input to label the current ultrasound image to be detected, the encoder is denoted as Transformer Encoder, the decoder as Decoder, and the result generated by the deep learning model is denoted as output. This invention employs a multi-layer cascaded Transformer structure as the backbone network. The first Transformer layer takes the current ultrasound image to be detected as input and generates the first layer feature map F1. The subsequent three layers of feature maps... Figure 1 The feature map from the previous layer is used as input to generate feature maps F2, F3, and F4 for each layer. Subsequently, the feature maps (F1, F2, F3, F4) generated by the encoder are input into the dual attention module (DSRModule).

[0087] It should be noted that the encoding module of the present invention can specifically be an encoder, and the encoder can adopt other popular feature extraction networks, such as Unet, FPN, GGNet, etc.

[0088] (II) Dual Attention Module

[0089] This invention designs a dual self-attention optimization (DSR) module, which optimizes the feature map of each Transformer layer by utilizing complementary information between different Transformer layers. Figure 2 This diagram illustrates the DSR module positioned at the i-th Transformer layer. This DSR module extracts missing feature information from the feature maps of the other three Transformer layers and optimizes the feature map. Specifically, the feature maps generated by the other three Transformer layers are first passed to the aggregation module (…). Figure 3 These are integrated in the model to generate an aggregated feature map Fg. Then, dual attention weights are learned from Fg to optimize the current target feature map Fi. To do this, a global attention weight map Za is learned by utilizing a 3×3 convolutional layer and a normalized exponential (Softmax) activation function on Fg, and then Za is subtracted from 1 to produce an anti-self-attention map Zr.

[0090] Za = Sigmoid(Conv(Fg)) (4)

[0091] Zr=1-Za (5)

[0092] Here, Sigmoid represents a Sigmoid layer, and Conv represents a 3×3 convolutional layer. The dual attention weight maps include Za and Zr.

[0093] To fuse the two weight maps Za and Zr, two 3×3 convolutional layers are first applied to Zr to obtain a pair of key-value mapping vectors, Kr(H×W×C / 8) and Vr(H×W×C / 2). Two more 3×3 convolutional layers are applied to Za to obtain another pair of key-value mapping vectors (Ka(H×W×C / 8) and Va(H×W×C / 2)). Simultaneously, two more 3×3 convolutional layers are applied to Fi to obtain vectors Kf (H×W×C / 8) and Vf (H×W×C / 2). Then, the spatial scale of Kf is adjusted to C / 8×HW, and the spatial scale of Kr is adjusted to HW×C / 8. These two rounded results are then multiplied by a matrix, and the Softmax activation function is applied to calculate the non-local similarity mapping Sr of the two key mapping vectors Kf and Kr across all pixels. Therefore, Sr is calculated as follows:

[0094] Sr=Softmax(Kr×Kf) (6)

[0095] Simultaneously, Ka is transformed into HW×C / 8, and then multiplied by Kf to calculate another nonlocal similarity mapping weight Sa:

[0096] Sa = Softmax(Ka × Kf) (7)

[0097] Next, the two similarity weight maps Sr and Sa are multiplied element-wise to generate a new similarity weight map Sd, which is then multiplied by Vf. The multiplication result is then concatenated with Vr and Va. Finally, the concatenated result is convolved to obtain the optimized mapping of the i-th DSR module. The mathematical formula for this optimized feature map is as follows:

[0098] Fi=Conv(Concate(Vr,Va,Vf×(Sr*Sa))) (8)

[0099] Where * denotes element-wise matrix multiplication. × denotes matrix dot product. Concate is a concatenation operation of feature map channels, and Conv is a convolutional layer with a kernel size of 3×3.

[0100] Through the above operations, the initial feature maps (F1, F2, F3, F4) generated by the Transformer encoder layer are optimized to generate (D1, D2, D3, D4).

[0101] In this invention, the combination of global attention and anti-self attention in the dual attention module can be performed in parallel or in a serial manner. Moreover, the temporal and spatial attention modules can be superimposed N times, but this example uses it once.

[0102] (III) Aggregation Module

[0103] Figure 4 The structure of the feature map fusion module used to integrate feature maps from the other three Transformer layers is shown. Ta, Tb, and Tc represent the feature maps generated by the other three Transformer layers. Among them, Ta has the largest spatial resolution, and Tc has the smallest spatial resolution.

[0104] Specifically, first, Ta is downsampled to have the same spatial resolution as Tb, then optimized using a 3×3 convolutional layer, and finally multiplied element-wise with Tb to generate feature map Tab. Simultaneously, another 3×3 layer optimizes the newly generated feature map Tab, and then adds it element-wise with Ta to generate feature map Taba. The feature map Taba is defined as follows:

[0105] Taba=Conv(Conv(Ta)+Tb×Conv(Ta)) (1)

[0106] Here, Conv represents a 3×3 convolutional layer. Furthermore, Ta and Tb are downsampled to the same spatial resolution as Tc, and then two 3×3 convolutional layers are applied to the two downsampled layers of Ta and Tb. Finally, the two convolutional results are multiplied by Tc to obtain the feature map Tabc.

[0107] Tabc=Tc×Conv(Tb)×Conv(Ta) (2)

[0108] Finally, Taba and Tabc are added together to generate the output feature map (Fg) of the feature map aggregation module:

[0109] Fg=Conv(Taba+Tabc) (3)

[0110] (iv) Decoding Module

[0111] This invention borrows from the PVT decoder structure, gradually amplifying the feature map through a series of upsampling convolution operations. Specifically, the decoder upsamples D4 to the same resolution as D3 and concatenates it with D3 along the channel dimension. Then, a convolution operation is performed to obtain a new feature map D3'. Next, D3' is upsampled to the same resolution as D2 and concatenated with D2 along the channel dimension. Then, a convolution operation is performed to obtain a new feature map D2', and so on to obtain D1'. Finally, D1' is upsampled to the same scale as the input RGB image as the model's segmentation result.

[0112] loss function

[0113] This invention minimizes the following loss function, which includes the predicted mask. The pixel-level binary cross-entropy (BCE) loss function and Dice loss function for labeled Y.

[0114]

[0115] Where α is a hyperparameter greater than 0, which measures the weight of the two losses, and is set to 1 in this invention. Y represents the final prediction result generated by the proposed deep learning model, where Y is the ground truth result. m represents the binary mask corresponding to the final prediction result generated by the proposed deep learning model. This represents the binary mask corresponding to the actual ground result, log is the logarithmic function, and ∩ represents the intersection operator of sets.

[0116] The present invention has been tested and verified. The experimental results show that, compared with the prior art, the present invention not only has higher detection accuracy, but also faster detection speed, and can achieve real-time detection.

[0117] In the verification test of this invention, 519 clinical liver tumor ultrasound images with a resolution of 240×240 were used to test the invention. For quantitative comparison, in addition to frame rate, five convincing indicators were used for comparison: Dice (Dice similarity coefficient), Accuracy, Jaccard (Jaccard similarity coefficient), APD (mean vertical distance), and HD (Hausdorff distance).

[0118] The experimental results are shown below:

[0119] Dice Accuracy Jaccard APD HD PVT 80.85±0.23 98.20±0.10 69.23±0.41 5.04±1.37 4.04±10.05 This method 82.80±0.18 98.42±0.09 71.81±0.33 4.42±1.08 3.82±0.05

[0120] As can be seen from the table, the segmentation method provided by this invention is superior to existing methods in terms of detection accuracy and detection speed.

[0121] refer to Figure 6 Another embodiment of this application provides an electronic device, including: at least one processor 110; and a memory 111 communicatively connected to the at least one processor; wherein the memory 111 stores instructions executable by the at least one processor 110, the instructions being executed by the at least one processor 110 to enable the at least one processor 110 to perform any of the above method embodiments.

[0122] The memory 111 and processor 110 are connected via a bus, which may include any number of interconnecting buses and bridges, connecting various circuits of one or more processors 110 and memory 111. The bus may also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver may be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 110 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 110.

[0123] Processor 110 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 111 can be used to store data used by processor 110 during operation.

[0124] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0125] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0126] Those skilled in the art will understand that the above-described embodiments are specific examples of implementing this application, and in practical applications, various changes in form and detail may be made without departing from the spirit and scope of this application. Any person skilled in the art can make their own modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application should be determined by the scope defined in the claims.

Claims

1. A deep learning-based ultrasound image segmentation method, characterized in that, Includes the following steps: Based on the current ultrasound image to be detected, multiple feature maps with different spatial resolutions are extracted. The method utilizes complementary information between feature maps of different layers to optimize the missing feature information in each layer, resulting in an optimized multi-layer feature map. Specifically, this includes: sequentially selecting each layer's feature map as the target feature map; aggregating the remaining layer feature maps (excluding the target feature map) to generate an aggregated feature map; and learning dual attention weights from the aggregated feature map to optimize each layer's target feature map, thus obtaining the optimized multi-layer feature map. A global attention weight map Za is learned by utilizing a 3×3 convolutional layer and a normalized exponential activation function on the aggregated feature map Fg, and then subtracted from 1 to produce an inverse self-attention map Zr: Za = Sigmoid(Conv(Fg))(3) Zr = 1 - Za(4) Wherein, Sigmoid represents the Sigmoid layer, Conv represents the 3×3 convolutional layer, and the dual attention weight map includes the global attention weight map Za and the anti-self attention map Zr; Two 3×3 convolutional layers are applied to the inverse self-attention map Zr to obtain Kr and Vr; where Kr is H × W × C / 8 and Vr is H × W × C / 2; Two 3×3 convolutional layers on the global attention weight map Za are applied to obtain Ka and Va; where Ka is H×W×C / 8 and Va is H × W × C / 2. A 3 × 3 convolutional layer is used on the target feature map Fi to obtain a vector Kf of H × W × C / 8 and a vector Vf of H × W × C / 2; The spatial scale of Kf is adjusted to C / 8 × HW, and the spatial scale of Kr is adjusted to HW × C / 8. These two integer results are then multiplied by a matrix, and the Softmax activation function is applied to calculate the nonlocal similarity mapping Sr of the key mapping vectors Kf and Kr across all pixels. Sr is calculated as follows: Sr = Softmax(Kr × Kf)(5) Transform Ka into HW × C / 8, and perform matrix multiplication between Ka and Kf to obtain the nonlocal similarity mapping weight Sa: Sa = Softmax(Ka × Kf)(6) A new similarity weight map Sd is generated by multiplying Sr and Sa element by element, and Sd is multiplied by Vf to obtain the multiplication result. The multiplication result is concatenated with Vr and Va to obtain the concatenated result; The concatenated result is convolved to obtain the optimized mapping of the i-th dual attention module. The mathematical formula for the optimized feature map is as follows: Fi = Conv(Concate(Vr, Va, Vf × (Sr * Sa))) (7) Where * represents element-wise matrix multiplication, × represents matrix dot product, Concate is the concatenation operation of feature map channel dimensions, and Conv is a convolutional layer with a kernel size of 3×3. Generate optimized multi-layer feature maps; An upsampling convolution operation is performed on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected. The optimized multi-layer feature map includes a first optimized feature layer D1, a second optimized feature layer D2, a third optimized feature layer D3, and a fourth optimized feature layer D4.

2. The ultrasound image segmentation method based on deep learning according to claim 1, characterized in that, The step of sequentially selecting each layer of feature maps as the target feature map and aggregating the remaining layer feature maps (excluding the target feature map) to generate an aggregated feature map specifically includes: Select the i-th layer feature map as the target feature map Fi; where i = 1, 2, 3, 4; Feature maps Ta, Tb, and Tc are generated using the remaining three feature layers; among them, Ta has the largest spatial resolution and Tc has the smallest spatial resolution. After downsampling Ta to give it the same spatial resolution as Tb, Ta is optimized using a 3×3 convolutional layer to obtain the optimized Ta. Then, the optimized Ta is multiplied element-wise with Tb to generate the feature map Tab. The Tab is optimized using another 3×3 convolutional layer to obtain the optimized Tab; the Tab and Ta are then added element-wise to generate the feature map Taba; Next, Ta and Tb are downsampled to the same spatial resolution as Tc, and then two 3×3 convolutional layers are applied on the two downsampled layers of Ta and Tb to obtain the convolution result; Multiply the convolution result by Tc to obtain the feature map Tabc: Tabc = Tc × Conv(Tb) × Conv(Ta) (1) Where Conv represents a 3×3 convolutional layer; Add Taba and Tabc to generate the aggregated feature map Fg: Fg = Conv(Taba + Tabc)(2).

3. The ultrasound image segmentation method based on deep learning according to claim 1, characterized in that, The step of performing an upsampling convolution operation on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected specifically includes: The fourth optimized feature layer D4 is upsampled to the same resolution as the third optimized feature layer D3, and then concatenated with the third optimized feature layer D3 along the channel dimension. Then, a new feature map D3' is obtained through a convolution operation. Then, D3' is upsampled to the same resolution as the second optimized feature layer D2, and concatenated with the second optimized feature layer D2 along the channel dimension. Then, a new feature map D2' is obtained through a convolution operation. Then D2' is upsampled to the same resolution as the first optimized feature layer D1, and concatenated with the first optimized feature layer D1 along the channel dimension. Then, a new feature map D1' is obtained through a convolution operation. Then, an upsampling operation is performed on D1' so that the RGB image scale of D1' is the same as that of the current ultrasound image to be detected, and it is output as an ultrasound image segmentation map.

4. The ultrasound image segmentation method based on deep learning according to claim 1, characterized in that, The step of extracting multiple feature maps with different spatial resolutions based on the current ultrasound image to be detected specifically includes: using the current ultrasound image to be detected as input to generate a first-layer feature map F1; using the first-layer feature map F1 as input to generate a second-layer feature map F2; using the second-layer feature map F2 as input to generate a third-layer feature map F3; and using the third-layer feature map F3 as input to generate a fourth-layer feature map F4.

5. The ultrasound image segmentation method based on deep learning according to claim 1, characterized in that, After the step of extracting multiple feature maps with different spatial resolutions based on the current ultrasound image to be detected, the following steps are also included: The feature maps of each layer are dimensionally compressed.

6. A deep learning-based ultrasound image segmentation system, wherein the system is used to implement the deep learning-based ultrasound image segmentation method as described in any one of claims 1 to 5, characterized in that, The system includes: The encoding module is used to extract multiple feature maps with different spatial resolutions based on the current ultrasound image to be detected. The aggregation module is used to sequentially select each layer of feature map as the target feature map, and perform aggregation processing on the remaining layer feature maps other than the target feature map to generate an aggregated feature map. A dual attention module is used to learn dual attention weights from the aggregated feature map to optimize each layer of the target feature map, thereby obtaining an optimized multi-layer feature map. The decoding module is used to perform upsampling convolution on the optimized multi-layer feature map to obtain an ultrasound image segmentation map of the same scale as the RGB image of the current ultrasound image to be detected.

7. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the deep learning-based ultrasound image segmentation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the deep learning-based ultrasound image segmentation method according to any one of claims 1 to 5.

Images