Breast ultrasonic tumor segmentation method, system and related equipment

Abstract

The invention provides a breast ultrasonic tumor segmentation method and system and related equipment. The method comprises the following steps: acquiring preprocessed input image information; inputting the input image information into the breast ultrasonic tumor segmentation model to obtain a corresponding tumor segmentation prediction map; wherein after the breast ultrasonic tumor segmentation model obtains input image information, edge information in low-level features is dynamically extracted through an edge differential convolution module and serves as supplement of global features, and then the capacity of the breast ultrasonic tumor segmentation model on multi-scale feature capture is enhanced through a multi-scale aggregation module; and the breast ultrasonic tumor segmentation model adopts a supervision strategy based on foreground feature and background feature alignment in training. According to the method, additional supervision signals are introduced, alignment supervision is carried out on foreground and background features, and richer foreground and background region learning signals are explicitly provided for the model, so that feature internal characterization is enhanced, and segmentation edge fuzziness or false detection is avoided.

Description

Technical Field

[0001] The present application relates to the technical field of medical image processing, and in particular to a breast ultrasound tumor segmentation method, system and related equipment. Background Art

[0002] Breast cancer poses a serious threat to human health, especially women's health. Ultrasound imaging, as a medical imaging technology with high safety, low cost and high efficiency, has been widely used in the diagnosis of breast cancer. Accurate segmentation of tumors in breast ultrasound images is of great significance for early diagnosis and lesion analysis. At present, ultrasound physicians usually further quantify lesions by manually segmenting breast tumors in images. However, this manual segmentation process is not only time-consuming and labor-intensive, but also needs to be performed by experienced professionals. For this reason, the technology that can automatically segment tumors in breast ultrasound images has important research value and clinical significance.

[0003] Traditional image segmentation methods, such as edge detection, threshold segmentation, and region growing, can achieve good results in some cases. However, these methods usually require manual parameter setting and rely heavily on expert experience. In addition, the low contrast, blurred edges, and diverse shapes and sizes of tumors in breast ultrasound images further increase the difficulty of segmentation. Therefore, using traditional methods to achieve accurate segmentation of breast ultrasound tumors still faces huge challenges.

[0004] With the rapid development of deep learning technology, many related methods have been widely used in the field of medical image segmentation, greatly improving the performance and efficiency of segmentation technology. Among them, the U-net model has become a classic method in the field of medical image segmentation due to its superior structural design and excellent segmentation effect. By combining the encoder and decoder structure, the model can capture global context information while retaining the detailed information of the target, thereby significantly improving the segmentation accuracy.

[0005] However, both U-net and other methods based on it in recent years, although they perform well in overall segmentation, lack fine-grained supervision of feature space representation. This defect makes it difficult to obtain consistent semantic representations for foreground and background internal pixels, which may lead to problems such as blurred segmentation edges or false detection. Based on this, the present application provides a breast ultrasound tumor segmentation method, system and related equipment. Summary of the invention

[0006] In view of the problems existing in the prior art, the present application provides a breast ultrasound tumor segmentation method, system and related equipment.

[0007] The purpose of this application is achieved by the following technical solutions:

[0008] In a first aspect, the present application provides a method for breast ultrasound tumor segmentation, the method comprising:

[0009] Acquire preprocessed input image information, wherein the preprocessing includes scaling the original image to a fixed size, applying Gaussian filtering to remove noise, normalizing pixel values, and generating diversified samples through data enhancement technology;

[0010] Inputting the input image information into a breast ultrasound tumor segmentation model to obtain a corresponding tumor segmentation prediction map;

[0011] Among them, after the breast ultrasound tumor segmentation model obtains the input image information, it dynamically extracts edge information in low-level features through the edge differential convolution module and uses it as a supplement to the global features, and then enhances the ability of the breast ultrasound tumor segmentation model in multi-scale feature capture through the multi-scale aggregation module; the breast ultrasound tumor segmentation model adopts a supervision strategy based on the alignment of foreground features and background features during training.

[0012] In some possible implementations, the dynamically extracting edge information from low-level features through an edge differential convolution module as a supplement to the global features includes:

[0013] The low-level features extracted by the first encoder and the second encoder are respectively passed through the edge differential convolution module to obtain horizontal and vertical features, and then the dynamic feature fusion is performed using the connection between the horizontal edge features and the vertical edge features to obtain the fused features and input them into the decoder; the form of the dynamic feature fusion is as follows:

[0014] F v ′ =σ(w a )×F v +(1-σ(w a ))×F h

[0015] F h ′ =σ(w b )×F v +(1-σ(w b ))×F h

[0016] F e =C(F v ′ ,F h ′ )

[0017] Among them, F v is the vertical feature obtained from the low-level features extracted from the first encoder by the edge difference convolution module, F his the horizontal feature obtained from the low-level features extracted from the second encoder by the edge difference convolution module, F v ′ is the fusion feature map in the vertical direction, F h ′ is the fusion feature map in the horizontal direction, F e is the final fusion feature; w a and w b are learnable weight parameters, representing F h and F v σ represents the attention weight; σ represents the Sigmoid function, which is used to normalize the weight to [0,1]; C represents the concatenation operation.

[0018] In some possible implementations, after edge differential convolution, the expressions of horizontal and vertical features are as follows:

[0019]

[0020] Among them, x is the input feature map, a0 represents the current center pixel position, a n is to traverse the region R h and R v , θ is used to adjust the weight of the center pixel, w is the learnable weight, and b is the bias term.

[0021] In some possible implementations, the multi-scale aggregation module is disposed between the output of the fifth encoder and the input of the fourth decoder, and the corresponding expression is as follows:

[0022] G=Conv 1×1 (F5)

[0023]

[0024] S g =C(G,X1,X2,X3)+F5

[0025] Wherein, F5 is the output of the fifth encoder; G is the feature map after dimensionality reduction by 1×1 convolution; X i is the feature map generated by the i-th branch; r represents the expansion rate, defined as r = 2 × i + 1, which is used to gradually increase the receptive field; S g It is the fused multi-scale feature map, which further strengthens the combination of global and local features by connecting with F5 residual; C represents the concatenation operation.

[0026] In some possible implementations, the breast ultrasound tumor segmentation model uses the feature map output by the first decoder to provide supervision for decoders in other layers. The corresponding expressions are as follows:

[0027]

[0028] in, represents the total L2 norm loss, which is used to measure the difference between feature maps of all layers; f l is the feature map of layer l; f L A feature map output by the first decoder; is the L2 norm, which is used to calculate the difference between feature maps of different layers and provide additional supervision signals.

[0029] In some possible implementations, the breast ultrasound tumor segmentation model adopts a supervision strategy based on alignment of foreground features and background features during training, including:

[0030] Extracting a foreground feature map and a background feature map of the feature map output by the first decoder, and calculating feature alignment losses of the foreground region and the background region respectively;

[0031] Combine the feature alignment loss of foreground and background regions with the binary cross entropy loss to obtain the total loss function;

[0032] Among them, the expression corresponding to the feature alignment loss of the foreground area is as follows:

[0033]

[0034] in, is the feature alignment loss of the foreground region, is the feature value of the i-th pixel in the foreground feature map output by the l-th decoder, is the feature value of the i-th pixel in the foreground feature map output by the first decoder; N fg is the number of foreground pixels, which is used to ensure loss normalization; the i in the expression corresponding to the feature alignment loss of the foreground area represents the foreground pixel index in the feature map;

[0035] The expression corresponding to the feature alignment loss of the background area is as follows:

[0036]

[0037] in, is the feature alignment loss of the background region, is the feature value of the i-th pixel in the background feature map output by the l-th decoder, is the feature value of the i-th pixel in the background feature map output by the first decoder, N bg is the number of background pixels; i in the expression corresponding to the feature alignment loss of the background area represents the background pixel index in the feature map;

[0038] The expression corresponding to the total loss function is as follows:

[0039]

[0040] in, is the standard binary cross entropy loss, which is used to supervise the matching between the final output and GT, λ fg and λ bg is the weight hyperparameter of the foreground and background feature alignment loss.

[0041] In some possible implementations, a method for acquiring the feature map of the foreground area and the feature map of the background area includes:

[0042] Use the foreground mask to filter the feature map and obtain the foreground feature map. The corresponding expression is as follows:

[0043] f fg =f⊙y

[0044] ⊙ represents the element-by-element multiplication operation, which is used to retain the features belonging to the foreground area in the feature map through the foreground mask, and set other areas to 0; f fg is the foreground feature map; f is the feature map used for screening; y is the foreground mask;

[0045] The feature map of the background area is extracted by 1-y. The corresponding expression is as follows:

[0046] f bg =f⊙(1-y)

[0047] f bg is the background feature map.

[0048] In a second aspect, the present application also provides a breast ultrasound tumor segmentation system, the system comprising:

[0049] An information acquisition module, the information acquisition module is used to acquire preprocessed input image information, the preprocessing includes scaling the original image to a fixed size, applying Gaussian filtering to remove noise, normalizing pixel values, and generating diversified samples through data enhancement technology;

[0050] A model prediction module, wherein the model prediction module is used to input the input image information into a breast ultrasound tumor segmentation model to obtain a corresponding tumor segmentation prediction map;

[0051] Among them, after the breast ultrasound tumor segmentation model obtains the input image information, it dynamically extracts edge information in low-level features through the edge differential convolution module and uses it as a supplement to the global features, and then enhances the ability of the breast ultrasound tumor segmentation model in multi-scale feature capture through the multi-scale aggregation module; the breast ultrasound tumor segmentation model adopts a supervision strategy based on the alignment of foreground features and background features during training.

[0052] In a third aspect, the present application further provides an electronic device comprising one or more processors and a memory; one or more programs are stored in the memory and are configured such that the one or more processors execute any of the methods described in the first aspect.

[0053] In a fourth aspect, the present application further provides a computer-readable storage medium having instructions stored thereon, wherein when the instructions are executed by a processor, the processor implements the method as described in any one of the first aspects.

[0054] In combination with the above-mentioned technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows: The breast ultrasound tumor segmentation method provided by the present application, on the one hand, by introducing the foreground and background feature alignment supervision mechanism, the model can pay more attention to the consistency of the internal features of the foreground and background regions, and strengthen the representation learning within the region. This mechanism effectively reduces the overlap between the foreground and background in the feature space, significantly reduces the problem of blurred or misclassified boundaries, and thus improves the segmentation accuracy; on the other hand, by introducing the edge differential convolution module (EDC) and the dynamic fusion method, the model's ability to learn detailed features is enhanced, especially in terms of tumor boundaries and texture differences, significantly improving the segmentation accuracy and adaptability to irregular shapes; on the other hand, through the multi-scale aggregation module (MSAM), the balance between global and local information is optimized, so that the model can capture more contextual information, further improving the segmentation performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present application is further described below in conjunction with the accompanying drawings and implementation methods.

[0056] Figure 1 It is a flow chart of a breast ultrasound tumor segmentation method provided in an embodiment of the present application.

[0057] Figure 2 It is a schematic diagram of a breast ultrasound tumor segmentation model provided in an embodiment of the present application.

[0058] Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] Below, in conjunction with the accompanying drawings and specific implementation methods, the present application is further described. It should be noted that, under the premise of no conflict, the various embodiments described below or the various technical features can be arbitrarily combined to form a new embodiment. The implementation procedures of the present application will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific implementation procedures, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be understood that the preferred embodiments are only for illustrating the present application, not for limiting the scope of protection of the present application.

[0060] The following is a brief description of the technical field and related terms of the embodiments of the present application to facilitate understanding by those skilled in the art.

[0061] Deep learning is a branch of machine learning that is based on learning algorithms of artificial neural networks. Deep learning models are able to learn multi-level representations and abstractions of data, which makes them excel in areas such as image and speech recognition, natural language processing, and medical image analysis. The encoder-decoder architecture is a common structure in deep learning, where the encoder is responsible for processing input data (such as images or sequences) and extracting features. In image segmentation, the encoder usually consists of a series of convolutional layers and downsampling layers, which gradually extract features of the image and reduce its spatial dimension. The decoder is responsible for converting the features extracted by the encoder back to the spatial dimensions of the original input for pixel-level prediction. In image segmentation, the decoder usually contains upsampling layers and convolutional layers to gradually restore the spatial resolution of the feature map.

[0062] Whether it is U-net or other methods based on it in recent years, they lack fine-grained supervision of feature space representation; in addition, they usually focus more on learning global image features and pay insufficient attention to capturing edge prior information. However, edge prior information is crucial for accurately segmenting objects with complex shapes or blurred boundaries. The above shortcomings significantly limit the further improvement of segmentation accuracy.

[0063] Based on this, the present application provides a method, system and related equipment for breast ultrasound tumor segmentation, which introduces additional supervision signals to align and supervise the foreground and background features respectively, and explicitly provides the model with richer foreground and background area learning signals, thereby strengthening the internal representation of the features. At the same time, the shallow features extracted by the encoder are used to dynamically extract edge information in the horizontal and vertical directions through the edge differential convolution module, and the global features are further refined in combination with the multi-scale aggregation module to improve the segmentation accuracy. The breast ultrasound tumor segmentation method of the present invention can generate a report containing the prediction results when a tumor is predicted, to assist the user (doctor) in making diagnosis and treatment decisions. In other words, this method is only used as an intermediate result for user reference. The following will first explain the method, and then explain the system, etc.

[0064] Method embodiment.

[0065] See also Figure 1 and Figure 2 , Figure 1 is a flow chart of a breast ultrasound tumor segmentation method provided in an embodiment of the present application, Figure 2 It is a schematic diagram of a breast ultrasound tumor segmentation model provided in an embodiment of the present application.

[0066] The present application embodiment provides a method for breast ultrasound tumor segmentation, the method comprising:

[0067] S101, obtaining preprocessed input image information, wherein the preprocessing includes scaling the original image to a fixed size, applying Gaussian filtering to remove noise, normalizing pixel values, and generating diversified samples through data enhancement technology;

[0068] S102, inputting the input image information into a breast ultrasound tumor segmentation model to obtain a corresponding tumor segmentation prediction map;

[0069] Among them, after the breast ultrasound tumor segmentation model obtains the input image information, it dynamically extracts edge information in low-level features through the edge differential convolution module and uses it as a supplement to the global features, and then enhances the ability of the breast ultrasound tumor segmentation model in multi-scale feature capture through the multi-scale aggregation module; the breast ultrasound tumor segmentation model adopts a supervision strategy based on the alignment of foreground features and background features during training.

[0070] This method dynamically extracts edge information from low-level features through the edge differential convolution module (EDC), which enhances the model's ability to recognize image edges. At the same time, through the multi-scale aggregation module (MSAM), the model can capture tumor features of different scales and improve the ability to segment tumors of different sizes.

[0071] As an example, the original breast ultrasound image is first subjected to a series of preprocessing operations including resizing, Gaussian filtering denoising, pixel value normalization and data enhancement, and the preprocessed image is input into the trained breast ultrasound tumor segmentation model. The model first uses edge differential convolution to extract edge information from low-level features. Edge information, as a supplement to global features, helps the model better understand the structure and boundaries in the image. Then, the model aggregates features of different scales through MSAM to enhance the ability to capture multi-scale tumor features, thereby improving the accuracy of segmentation.

[0072] Therefore, by integrating the edge differential convolution module to accurately extract edge features, combining the multi-scale aggregation module to enhance the ability to capture tumor features of different sizes, and adopting the supervision strategy of aligning foreground and background features, the shortcomings of traditional methods in fine-grained supervision of feature space representation are effectively solved. This method significantly improves the model's ability to distinguish between foreground and background, reduces segmentation edge blur and false detection problems, and thus achieves higher-precision tumor recognition and segmentation in the field of medical image segmentation, solving the problem of traditional methods' shortcomings in fine-grained supervision of feature space representation.

[0073] In some embodiments, the step of dynamically extracting edge information from low-level features through an edge differential convolution module as a supplement to global features includes:

[0074] The input image information is sequentially subjected to convolution and downsampling layer by layer operations of multiple convolutional layers and downsampling layers; depthwise separable convolution is introduced into the convolutional layer, and the standard convolution is decomposed into depthwise convolution and pointwise convolution; wherein the depthwise convolution is used to extract local spatial features, and the pointwise convolution is used to fuse cross-channel information;

[0075] The low-level features extracted by the first encoder and the second encoder are respectively passed through the edge differential convolution module to obtain horizontal and vertical features, and then the dynamic feature fusion is performed using the connection between the horizontal edge features and the vertical edge features to obtain the fused features and input them into the decoder; the form of the dynamic feature fusion is as follows:

[0076] F v ′ =σ(w a )×F v +(1-σ(w a ))×F h

[0077] F h ′ =σ(w b )×F v +(1-σ(w b ))×F h

[0078] F e =C(F v ′ ,F h ′ )

[0079] Among them, F v is the vertical feature obtained from the low-level features extracted from the first encoder by the edge difference convolution module, F h is the horizontal feature obtained from the low-level features extracted from the second encoder by the edge difference convolution module, F v ′ is the fusion feature map in the vertical direction, F h ′ is the fusion feature map in the horizontal direction, F e is the final fusion feature; w a and w b are learnable weight parameters, representing F h and F v σ represents the attention weight; σ represents the Sigmoid function, which is used to normalize the weight to [0,1]; C represents the concatenation operation.

[0080] This embodiment innovatively applies edge differential convolution to extract edge information from low-level features, thereby enhancing the model's ability to capture edge features. Through dynamic feature fusion, the edge features in the horizontal and vertical directions are adaptively combined to improve the expressiveness and segmentation accuracy of the features. Specifically, the input image passes through multiple convolutional layers and downsampling layers to gradually extract multi-scale features. In the convolutional layer, deep separable convolution is used to extract local spatial features and cross-channel information. The first and second encoders extract low-level features respectively, and obtain edge features in the horizontal and vertical directions through the edge differential convolution module. Using learnable weight parameters, after normalization by the Sigmoid function, the fusion ratio of the horizontal and vertical edge features is dynamically adjusted to obtain fused features. Thus, through the edge differential convolution module and dynamic feature fusion, the model can more accurately identify and segment tumor areas in breast ultrasound images.

[0081] In some embodiments, after edge differential convolution, the expressions of horizontal direction features and vertical direction features are obtained as follows:

[0082]

[0083] Among them, x is the input feature map, a0 represents the current center pixel position, a n is to traverse the region R h and R v , θ is used to adjust the weight of the center pixel, w is the learnable weight, and b is the bias term.

[0084] In this embodiment, x represents the input feature map, that is, the image data after being processed by the network convolution layer. Through the edge difference convolution module, the model can more effectively capture the edge information in the image. The horizontal and vertical features are extracted separately, so that the model can understand the structural information of the image more comprehensively.

[0085] Specifically, x represents the feature map input to the edge difference convolution layer, which contains the multi-scale feature information of the image. By designing a specific convolution kernel, the model performs convolution operations in the horizontal and vertical directions to extract edge features. By adjusting the weight of the central pixel a0 by the parameter θ, the model can adaptively emphasize or suppress the role of the central pixel in feature extraction. The bias term b is added to provide additional flexibility to the model. Therefore, by accurately capturing edge information, the model can more accurately identify and segment tumor areas in the image. Directional feature extraction enables the model to understand the structure of the image more comprehensively, thereby improving the expressiveness of the features.

[0086] In some embodiments, the multi-scale aggregation module is disposed between the output of the fifth encoder and the input of the fourth decoder, and the corresponding expression is as follows:

[0087] G=Conv 1×1 (F5)

[0088]

[0089] S g =C(G,X1,X2,X3)+F5

[0090] Wherein, F5 is the output of the fifth encoder; G is the feature map after dimensionality reduction by 1×1 convolution; X i is the feature map generated by the i-th branch; r represents the expansion rate, defined as r = 2 × i + 1, which is used to gradually increase the receptive field; S g It is the fused multi-scale feature map, which further strengthens the combination of global and local features by connecting with F5 residual; C represents the concatenation operation.

[0091] In this embodiment, a multi-scale aggregation module is introduced between the encoder and the decoder to enhance the model's ability to capture features of different scales. Using deep separable convolution (DSC) in the multi-scale aggregation module improves the computational efficiency and performance of the model. By dynamically adjusting the expansion rate (r), the model can adaptively capture features of different scales, enhancing the flexibility of the model. As a result, the fusion of multi-scale features enables the model to understand the image content more comprehensively and improve the accuracy of segmentation. Deep separable convolution reduces model parameters and improves the model's adaptability to different image qualities.

[0092] In some embodiments, the breast ultrasound tumor segmentation model uses the feature map output by the first decoder to provide supervision for decoders of other layers. The corresponding expression is as follows:

[0093]

[0094] in, represents the total L2 norm loss, which is used to measure the difference between feature maps of all layers; f l is the feature map of layer l; f L A feature map output by the first decoder; is the L2 norm, which is used to calculate the difference between feature maps of different layers and provide additional supervision signals.

[0095] The L2 norm is a method of measuring the length of a vector in a vector space, and is used to calculate the difference between feature maps as a supervisory signal to guide model learning. This supervisory mechanism provides additional supervisory signals by calculating the L2 norm difference between the feature map output by each decoder layer and the feature map output by the first decoder, which helps ensure that all decoder layers are consistent with the first decoder in feature representation, thereby improving the segmentation accuracy of the model.

[0096] In some embodiments, the breast ultrasound tumor segmentation model adopts a supervision strategy based on alignment of foreground features and background features during training, including:

[0097] Extracting a foreground feature map and a background feature map of the feature map output by the first decoder, and calculating feature alignment losses of the foreground region and the background region respectively;

[0098] Combine the feature alignment loss of foreground and background regions with the binary cross entropy loss to obtain the total loss function;

[0099] Among them, the expression corresponding to the feature alignment loss of the foreground area is as follows:

[0100]

[0101] in, is the feature alignment loss of the foreground region, is the feature value of the i-th pixel in the foreground feature map output by the l-th decoder, is the feature value of the i-th pixel in the foreground feature map output by the first decoder; N fg is the number of foreground pixels, which is used to ensure loss normalization; the i in the expression corresponding to the feature alignment loss of the foreground area represents the foreground pixel index in the feature map (i∈foreground);

[0102] The expression corresponding to the feature alignment loss of the background area is as follows:

[0103]

[0104] in, is the feature alignment loss of the background region, is the feature value of the i-th pixel in the background feature map output by the l-th decoder, is the feature value of the i-th pixel in the background feature map output by the first decoder, N bg is the number of background pixels; the i in the expression corresponding to the feature alignment loss of the background area represents the background pixel index in the feature map (i∈"background");

[0105] The expression corresponding to the total loss function is as follows:

[0106]

[0107] in, is the standard binary cross entropy loss, which is used to supervise the matching between the final output and GT, λ fg and λ bg is the weight hyperparameter of the foreground and background feature alignment loss, which is used to adjust the weight λ fg and λ bg Controlling the impact of feature alignment loss on model training enables the model to learn good segmentation capabilities and enhance the distinction between foreground and background in feature space.

[0108] The technical solution provided in this embodiment enhances the model's ability to recognize key areas in the image by supervising the foreground and background features respectively. The introduction of feature alignment loss enables the model to better distinguish the foreground and background in the feature space, thereby improving the accuracy of segmentation. Specifically, the model extracts the feature map of the image through the decoder layer, including foreground and background features. For the foreground and background areas, their feature alignment losses are calculated separately. This can be achieved by comparing the differences between the feature maps output by different (e.g., the second, third, and fourth) decoders and the feature maps output by the first decoder. The feature alignment loss of the foreground and background is combined with the binary cross entropy loss to form a total loss function for model training and optimization.

[0109] Therefore, through the feature alignment loss, the model can more accurately segment the foreground and background in the image. The feature alignment supervision strategy helps the model maintain stable performance in the face of different backgrounds and foregrounds.

[0110] In some embodiments, the method for acquiring the feature map of the foreground area and the feature map of the background area includes:

[0111] Use the foreground mask to filter the feature map and obtain the foreground feature map. The corresponding expression is as follows:

[0112] f fg =f⊙y

[0113] ⊙ represents the element-by-element multiplication operation, which is used to retain the features belonging to the foreground area in the feature map through the foreground mask, and set other areas to 0; f fg is the foreground feature map; f is the feature map used for screening; y is the foreground mask;

[0114] The feature map of the background area is extracted by 1-y. The corresponding expression is as follows:

[0115] f bg =f⊙(1-y)

[0116] f ng is the background feature map.

[0117] By using masks to separate the foreground and background features in the feature map, the model can learn and optimize these two parts of the features separately. During the training process, the feature alignment loss is calculated for the foreground and background feature maps respectively, and combined with the binary cross entropy loss to form a total loss function for model optimization.

[0118] Therefore, by supervising the foreground and background features separately, the model can learn the feature representation of the foreground and background more accurately, thereby improving the segmentation accuracy. Through feature alignment loss, the model can use data more effectively during training, accelerate convergence, and improve training efficiency.

[0119] As an example, a breast ultrasound tumor segmentation method is provided. Figure 2 In the breast ultrasound tumor segmentation model, by introducing additional supervision signals, a direct feature alignment supervision mechanism is established between the encoder and the decoder, and the semantically rich high-level feature maps are used to supervise the shallow feature maps, explicitly enhancing the model's learning ability for the foreground and background areas. In addition, an edge differential convolution module is used to obtain edge information from the shallow features extracted by the encoder, and the global features extracted by the multi-scale aggregation module are refined to improve the accuracy of the segmentation edges. This method combines feature alignment supervision and edge information enhancement to significantly improve the accuracy and robustness of breast ultrasound tumor segmentation, providing important technical support for the early diagnosis of breast cancer. Specifically, the method includes the following steps:

[0120] Data preprocessing steps: In the ultrasound tumor segmentation task, data preprocessing is a key step to improve model performance and robustness, including the following operations: First, the original image is uniformly scaled to a fixed size (such as 224×224) to ensure that the data format is consistent and adapts to the model input requirements; second, Gaussian filtering is used to smooth the image to effectively remove speckle noise and improve image quality; then, the image pixel values ​​are normalized to the [0,1] interval to reduce the impact of grayscale value differences on model training; finally, data enhancement techniques (such as random rotation, flipping, cropping, etc.) are used to generate diverse samples to further improve the generalization ability of the model.

[0121] Data input step: Input the preprocessed data (input image information) into the breast ultrasound tumor segmentation model, which includes the first encoder to the fifth encoder and the first decoder to the fourth decoder. The input image information is processed by multiple layers of convolutional layers and downsampling layers in sequence. The convolutional layer extracts local features of the input image through the local receptive field, and combines the nonlinear activation function to enhance the nonlinear expression ability of the network, so that it can learn more complex feature representations. The downsampling layer reduces the spatial resolution of the feature map, discards redundant details while retaining key information, thereby improving computational efficiency. Through the layer-by-layer operations of convolution and downsampling, the model can gradually extract multi-scale features, gradually aggregate from low-level texture information to high-level semantic information, and lay a solid feature foundation for subsequent segmentation tasks.

[0122] After the data is input into the breast ultrasound tumor segmentation model, the breast ultrasound tumor segmentation model includes a depthwise separable convolution. Depthwise separable convolution refers to the introduction of depthwise separable convolution in the convolution layer to improve efficiency and performance. Depthwise separable convolution greatly reduces the number of network parameters and computational complexity by decomposing the standard convolution into two parts: depthwise convolution and pointwise convolution, thereby reducing the complexity of the model. This lightweight design not only alleviates the risk of overfitting, especially in small sample medical data scenarios, but also improves the efficiency of model training. At the same time, the decoder uses bilinear interpolation in the upsampling process to calculate the value of the target pixel by considering the values ​​of the four nearest neighboring pixels around the target pixel. The output of the first decoder can output the segmented image after feature back projection FBP (not shown in the figure).

[0123] Among them, deep convolution focuses on extracting local spatial features, while point-by-point convolution is used to fuse cross-channel information, enabling the model to better capture complex multi-scale lesion features in medical images, including tumor boundary details and texture differences.

[0124] Edge differential convolution step: In the breast ultrasound tumor segmentation task, the low-level features extracted by Encoder1 (the first encoder) and Encoder2 (the second encoder) retain more detailed information of the input image compared to the high-level semantic features extracted by the other three layers, and can better emphasize local features and spatial relationships, which helps to accurately locate the target boundaries.

[0125] To further extract edge detail features, this example combines the Sobel operator in the traditional segmentation method with the convolutional neural network, introduces the concept of differential information to extract gradient features, and thus enhances the model's representation and generalization capabilities. Specifically, convolution layers for vertical and horizontal directions are designed, and their convolution kernel weights can be dynamically adjusted to detect vertical and horizontal edges of breast tumors. For the input feature x, after edge differential convolution, the horizontal feature F is obtained. h and vertical feature F v The expression is as follows:

[0126]

[0127] Among them, a0 represents the current center pixel position, a n Traverse region R h and R v , θ is used to adjust the weight of the center pixel, w is the learnable weight, and b is the bias term.

[0128] Since tumors vary in size and shape, it is difficult to avoid fusion operations when dealing with multi-scale problems, especially the difficulty in balancing local details and global semantic information. In addition, tumors are often located in larger background areas or complex structures, and a single convolution operation may not be able to capture enough contextual information. This example proposes a multi-scale aggregation module (MSAM) to capture multi-scale contextual information, expand the receptive field, and enable the network to understand more contextual information. The specific operations of the multi-scale aggregation module are as follows:

[0129] G=Conv 1×1 (F5)

[0130]

[0131] S g =C(G,X1,X2,X3)+F5

[0132] Among them, F5 is the output of Encoder5 (the fifth encoder), G is the feature map after dimensionality reduction through 1×1 convolution, and X i is the feature map generated by the i-th branch, r represents the expansion rate, defined as r = 2 × i + 1, to gradually increase the receptive field. gThe fused multi-scale feature map is connected with the F5 residual to further strengthen the combination of global and local features. MSAM expands the receptive field and fuses multi-scale features, providing more accurate global features for subsequent refinement.

[0133] Considering that in the segmentation task, the decoder can often accurately locate the target area, but the encoder is prone to capture some irrelevant information, causing the model's attention to be distracted outside the boundary. At the same time, the closer the decoder is to the output module (such as Decoder1, the first decoder), the higher the segmentation accuracy, while the earlier modules tend to learn irrelevant information. As the supervision signal gradually moves away from the true label, the problem of semantic loss becomes more obvious. This may cause the model to activate some areas that are not related to the segmentation task, thus affecting the overall performance.

[0134] To solve the problem of semantic loss, this example uses the most semantically informative feature map in the decoder (the output of Decoder1) to provide additional supervision for other layers. The formula is as follows:

[0135]

[0136] Among them, f l is the feature map of layer l, f L is the feature map output by Decoder1, It is the L2 norm, which is used to calculate the difference between feature maps of different layers and provide additional supervision signals.

[0137] This strategy can effectively improve global semantic consistency. Further considering that if only the overall feature map is supervised, the discrimination of the model in the foreground and background areas needs to be improved to avoid being affected when dealing with blurred edges or complex backgrounds.

[0138] Considering that in medical image segmentation, the foreground and background categories are usually unbalanced: the foreground area is small, while the background occupies a larger proportion. Simply optimizing the overall loss will cause the model to favor the background area and ignore the foreground. To solve this problem, this example proposes a supervision strategy based on the alignment of foreground and background features.

[0139] Therefore, based on the above, the foreground part of the feature map y is extracted according to the foreground area of ​​the ground truth (GT) y. The foreground mask (the real label mask) can be used to filter the feature map:

[0140] f fg =f⊙y

[0141] Here ⊙ represents the element-by-element multiplication operation, which preserves the features of the foreground area in the feature map through the mask, and sets the other areas to 0. Similarly, the feature map of the background area is extracted through 1-y:

[0142] f bg =f⊙(1-y)

[0143] The feature maps of the foreground and background are extracted separately, and then the contrast loss between them and the target features is calculated. First, the loss of the foreground region is calculated. With the foreground features of the last layer For comparison:

[0144]

[0145] Where N fg is the number of foreground pixels, ensuring the loss is normalized, and i represents the foreground pixel index in the feature map. Similarly, for the background area, the loss is as follows:

[0146]

[0147] Where N bg is the number of background pixels.

[0148] Combine the feature alignment loss of foreground and background with the binary cross entropy loss:

[0149]

[0150] in, is the standard binary cross entropy loss, which supervises the matching of the final output with GT, λ fg and λ bg is the weight hyperparameter of the foreground and background feature alignment loss. By adjusting the weight λ fg and λ bg , the impact of feature alignment loss on model training can be controlled, so that the model can not only learn good segmentation capabilities, but also enhance the distinction between foreground and background in the feature space.

[0151] In terms of gradient transfer and optimization, during the training of the segmentation model, the loss calculation of the foreground and background feature maps is directly involved in the gradient transfer of the entire network, and the intermediate and deep features of the network are optimized multiple times. This mechanism not only promotes the effective learning of foreground area features, but also can achieve significant separation of foreground and background in the feature space, making the representation of the background area more consistent and stable, thereby reducing the occurrence of misclassification.

[0152] Through this design, the network can more accurately capture the unique properties of foreground and background features during training, and enhance its ability to handle fuzzy boundaries and small target areas in complex scenes. At the same time, this gradient transfer method takes into account the optimization of global semantic information and local details, further improving the segmentation performance and stability of the model. The parameters of some formulas in this example have been explained above and will not be repeated here.

[0153] In summary, the breast ultrasound tumor segmentation method provided in this example, on the one hand, by introducing the foreground and background feature alignment supervision mechanism, the model can pay more attention to the consistency of the internal features of the foreground and background regions, and strengthen the representation learning within the region. This mechanism effectively reduces the overlap of the foreground and background in the feature space, significantly reduces the problem of blurred or misclassified boundaries, and thus improves the segmentation accuracy; on the other hand, by introducing the edge differential convolution module (EDC) and the dynamic fusion method, the model's ability to learn detailed features is enhanced, especially in terms of tumor boundaries and texture differences, significantly improving the segmentation accuracy and adaptability to irregular shapes; on the other hand, through the multi-scale aggregation module (MSAM), the balance between global and local information is optimized, so that the model can capture more contextual information, further improving the segmentation performance.

[0154] In order to verify the beneficial effect of a breast ultrasound tumor segmentation method mentioned in this application, another example is provided:

[0155] 1. Dataset and evaluation method steps:

[0156] Three widely used public breast tumor segmentation datasets were selected for evaluation. The first dataset is the BUSI dataset (Breast Ultrasound Image), which contains breast ultrasound images collected in 2018, covering 600 female patients aged 25 to 75 years. The dataset contains 780 images with an average image size of 500×500 pixels, and the images are divided into three categories: normal, benign, and malignant. We selected 210 malignant images and 437 benign images from this dataset for experiments. The BUSI dataset is one of the most challenging segmentation datasets due to the large variation in tumor size and morphology and abnormal and irregular boundaries. The second dataset is the Dataset B dataset, which contains 163 breast ultrasound images from different women with an average size of 760×570 pixels. Each image shows one or more lesions, of which 53 are cancerous masses and 110 are benign lesions. The third dataset is the STU dataset, which contains 42 breast ultrasound images with an average size of 128×128 pixels. Given the small number of images in the STU dataset, it is mainly used as an external validation dataset to evaluate the generalization ability of the segmentation network. Considering the individual differences between medical images, this paper uses five-fold cross-validation to evaluate the performance of different segmentation networks. The BUSI and Dataset B datasets are divided into five independent folds, each fold is used as a test set in turn, and the remaining folds are used for training. The results of the five rounds are averaged as the final evaluation result of the method.

[0157] 2. Evaluation index steps:

[0158] The method was evaluated based on five indicators: Jaccard coefficient, Dice coefficient, Precision, Recall, and Specificity. The Jaccard coefficient is the ratio of the intersection to the union of two sets. It is used to measure the degree of overlap between the tumor region predicted by the model and the actual tumor region. The larger the Jaccard coefficient, the more the segmentation result overlaps with the true label, and the better the segmentation effect. The Dice coefficient is a variant of the Jaccard coefficient, which calculates the similarity between the predicted region and the true region. The higher the Dice coefficient, the closer the tumor region predicted by the model is to the true region. Precision measures the proportion of the part predicted by the model as positive (tumor region) that is actually positive. It reflects the accuracy of the model. Recall reflects the sensitivity of the model. A high Recall means that the model can identify more actual tumor regions and reduce missed detections. Specificity reflects the specificity of the model or its ability to exclude non-tumor regions.

[0159] 3. Result comparison steps:

[0160] The method protected by this application is compared with several advanced ultrasound breast tumor segmentation methods, including U-net, Att U-net, AE U-net, SKU-net and AAU-net. The experimental results are shown in Tables 1 and 2. Table 1 shows the performance of different methods on the BUSI dataset. The results show that on the BUSI dataset, our method is 1.98%, 1.12%, 1.69% and 0.13% higher than the most advanced AAU-net in terms of the four indicators of Jaccard coefficient, Dice coefficient, Precision and Specificity, respectively. Table 2 shows the performance of different methods on the Dataset B dataset. It can be seen that our method achieves 61.88%, 72.61%, 73.96%, 76.47% and 95.48% in terms of the five indicators of Jaccard coefficient, Dice coefficient, Precision, Recall and Specificity, respectively. These results show that our method can still maintain excellent segmentation accuracy in complex scenes, thanks to the introduction of foreground and background region alignment supervision and the effective use of edge information.

[0161] Table 1: Results of different methods on BUSI

[0162]

[0163]

[0164] Table 2: Results of different methods on Dataset B

[0165]

[0166] 4. External verification steps:

[0167] In order to further verify the robustness and generalization ability of the breast ultrasound tumor segmentation model provided by this application, the STU dataset was used as the external validation dataset of Dataset B. The model weights with the best performance were selected from Dataset B and verified on the STU dataset, and the results are shown in Table 3. It can be seen that the method protected by this application can still achieve excellent segmentation results on the STU dataset when it performs best on Dataset B, further proving the effectiveness and generalization ability of the breast ultrasound tumor segmentation model.

[0168] Table 3: Results of different methods on the external validation dataset STU

[0169]

[0170] Obviously, the breast ultrasound tumor segmentation method based on foreground and background feature alignment proposed in this application first dynamically extracts edge information from low-level features through the edge differential convolution module as a supplement to the global features, which significantly improves the model's ability to segment tumor boundaries; then, the multi-scale aggregation module is used to further enhance the model's ability to capture multi-scale features, so that it can better adapt to the diversity of tumor morphology. On this basis, a supervision mechanism based on foreground and background feature alignment is designed to effectively ensure the distinction and consistency of foreground and background areas, and reduce misclassification and omission phenomena. The above experimental results show that the technical solution provided by this application has achieved significant performance improvement compared with existing mainstream segmentation methods on datasets such as BUSI and Dataset B, especially in terms of evaluation indicators such as Jaccard coefficient, Dice coefficient, Precision and Specificity. At the same time, through the test of the external verification dataset STU, it is proved that this method has strong generalization ability.

[0171] System Embodiments.

[0172] The embodiment of the present application provides a breast ultrasound tumor segmentation system, and its specific implementation method is consistent with the implementation method described in the above method implementation method and the technical effects achieved, and some contents are not repeated here.

[0173] The system comprises:

[0174] An information acquisition module, the information acquisition module is used to acquire preprocessed input image information, the preprocessing includes scaling the original image to a fixed size, applying Gaussian filtering to remove noise, normalizing pixel values, and generating diversified samples through data enhancement technology;

[0175] A model prediction module, wherein the model prediction module is used to input the input image information into a breast ultrasound tumor segmentation model to obtain a corresponding tumor segmentation prediction map;

[0176] Among them, after the breast ultrasound tumor segmentation model obtains the input image information, it dynamically extracts edge information in low-level features through the edge differential convolution module and uses it as a supplement to the global features, and then enhances the ability of the breast ultrasound tumor segmentation model in multi-scale feature capture through the multi-scale aggregation module; the breast ultrasound tumor segmentation model adopts a supervision strategy based on the alignment of foreground features and background features during training.

[0177] In some embodiments, the step of dynamically extracting edge information from low-level features through an edge differential convolution module as a supplement to global features includes:

[0178] The low-level features extracted by the first encoder and the second encoder are respectively passed through the edge differential convolution module to obtain horizontal and vertical features, and then the dynamic feature fusion is performed using the connection between the horizontal edge features and the vertical edge features to obtain the fused features and input them into the decoder; the form of the dynamic feature fusion is as follows:

[0179] F v ′ =σ(w a )×F v +(1-σ(w a ))×F h

[0180] F h ′ =σ(w b )×F v +(1-σ(w b ))×F h

[0181] F e =C(F v ′ ,F h ′ )

[0182] Among them, F v is the vertical feature obtained from the low-level features extracted from the first encoder by the edge difference convolution module, F h is the horizontal feature obtained from the low-level features extracted from the second encoder by the edge difference convolution module, F v ′ is the fusion feature map in the vertical direction, F h ′ is the fusion feature map in the horizontal direction, F e is the final fusion feature; w a and w b are learnable weight parameters, representing F h and F v σ represents the attention weight; σ represents the Sigmoid function, which is used to normalize the weight to [0,1]; C represents the concatenation operation.

[0183] In some embodiments, after edge differential convolution, the expressions of horizontal direction features and vertical direction features are obtained as follows:

[0184]

[0185] Among them, x is the input feature map, a0 represents the current center pixel position, a n is to traverse the region R h and R v, θ is used to adjust the weight of the center pixel, w is the learnable weight, and b is the bias term.

[0186] In some embodiments, the multi-scale aggregation module is disposed between the output of the fifth encoder and the input of the fourth decoder, and the corresponding expression is as follows:

[0187] G=Conv 1×1 (F5)

[0188]

[0189] S g =C(G,X1,X2,X3)+F5

[0190] Wherein, F5 is the output of the fifth encoder; G is the feature map after dimensionality reduction by 1×1 convolution; X i is the feature map generated by the i-th branch; r represents the expansion rate, defined as r = 2 × i + 1, which is used to gradually increase the receptive field; S g It is the fused multi-scale feature map, which further strengthens the combination of global and local features by connecting with F5 residual; C represents the concatenation operation.

[0191] In some embodiments, the breast ultrasound tumor segmentation model uses the feature map output by the first decoder to provide supervision for the decoders of other layers. The corresponding expression is as follows:

[0192]

[0193] in, represents the total L2 norm loss, which is used to measure the difference between feature maps of all layers; f l is the feature map of layer l; f L A feature map output by the first decoder; is the L2 norm, which is used to calculate the difference between feature maps of different layers and provide additional supervision signals.

[0194] In some embodiments, the breast ultrasound tumor segmentation model adopts a supervision strategy based on alignment of foreground features and background features during training, including:

[0195] Extracting a foreground feature map and a background feature map of the feature map output by the first decoder, and calculating feature alignment losses of the foreground region and the background region respectively;

[0196] Combine the feature alignment loss of foreground and background regions with the binary cross entropy loss to obtain the total loss function;

[0197] Among them, the expression corresponding to the feature alignment loss of the foreground area is as follows:

[0198]

[0199] in, is the feature alignment loss of the foreground region, is the feature value of the i-th pixel in the foreground feature map output by the l-th decoder, is the feature value of the i-th pixel in the foreground feature map output by the first decoder; N fg is the number of foreground pixels, which is used to ensure loss normalization; the i in the expression corresponding to the feature alignment loss of the foreground area represents the foreground pixel index in the feature map;

[0200] The expression corresponding to the feature alignment loss of the background area is as follows:

[0201]

[0202] in, is the feature alignment loss of the background region, is the feature value of the i-th pixel in the background feature map output by the l-th decoder, is the feature value of the i-th pixel in the background feature map output by the first decoder, N bg is the number of background pixels; i in the expression corresponding to the feature alignment loss of the background area represents the background pixel index in the feature map;

[0203] The expression corresponding to the total loss function is as follows:

[0204]

[0205] in, is the standard binary cross entropy loss, which is used to supervise the matching between the final output and GT, λ fg and λ bg is the weight hyperparameter of the foreground and background feature alignment loss.

[0206] In some embodiments, the method for acquiring the feature map of the foreground area and the feature map of the background area includes:

[0207] Use the foreground mask to filter the feature map and obtain the foreground feature map. The corresponding expression is as follows:

[0208] f fg =f⊙y

[0209] ⊙ represents the element-by-element multiplication operation, which is used to retain the features belonging to the foreground area in the feature map through the foreground mask, and set other areas to 0; f fg is the foreground feature map; f is the feature map used for screening; y is the foreground mask;

[0210] The feature map of the background area is extracted by 1-y. The corresponding expression is as follows:

[0211] f bg =f⊙(1-y)

[0212] f bg is the background feature map.

[0213] Device Embodiments.

[0214] The embodiment of the present application provides an electronic device, the specific implementation of which is consistent with the implementation and technical effects recorded in the above method and system implementation, and some contents are not repeated. The electronic device includes one or more processors and a memory; one or more programs are stored in the memory and configured to be executed by the one or more processors to perform any of the above methods.

[0215] See also Figure 3 , Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0216] The electronic device may include, for example, at least one memory 11, at least one processor 12, and a bus 13 connecting different platform systems.

[0217] The memory 11 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 111 and / or a cache memory 112 , and may further include a read-only memory (ROM) 113 .

[0218] The memory 11 also stores a computer program, which can be executed by the processor 12 so that the processor 12 implements the steps of any of the above methods.

[0219] The memory 11 may also include a utility 114 having at least one program module 115, such program module 115 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0220] Accordingly, the processor 12 may execute the above-mentioned computer program, and may execute the utility 114 .

[0221] The processor 12 may adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0222] Bus 13 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures.

[0223] The electronic device may also communicate with one or more external devices 14 such as a keyboard, a pointing device, a Bluetooth device, etc., and may also communicate with one or more devices that can interact with the electronic device, and / or communicate with any device (such as a router, a modem, etc.) that enables the electronic device to communicate with one or more other computing devices. Such communication may be performed through an input / output interface 15. In addition, the electronic device may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN) and / or a public network, such as the Internet) through a network adapter 16. The network adapter 16 may communicate with other modules of the electronic device through the bus 13. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device in actual applications, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0224] Storage Medium Embodiments.

[0225] The embodiment of the present application also provides a computer-readable storage medium, the specific embodiment of which is consistent with the embodiment recorded in the above method embodiment and the technical effects achieved, and some contents will not be repeated here.

[0226] The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the steps of any of the above methods or the functions of any of the above electronic devices are implemented.

[0227] Computer readable medium can be a computer readable signal medium or a computer readable storage medium. In an embodiment of the present application, a computer readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. Computer readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0228] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable storage medium may also be any computer-readable medium that can send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or for use in combination with it. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. The program code for performing the operation of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and also conventional procedural programming languages ​​such as C language or similar programming languages. The program code may be executed entirely on a user computing device, partially on a user device, as an independent software package, partially on a user computing device, partially on a remote computing device, or entirely on a remote computing device or server. Where a remote computing device is involved, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0229] It should be noted that, in the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, a and b, a and c, b and c or a and b and c, where a, b and c can be single or multiple. It is worth noting that "at least one" can also be interpreted as "one or more items".

[0230] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are configured to distinguish similar objects, and do not have to be configured to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0231] This application is explained from the perspectives of purpose of use, effectiveness, progress and novelty, and has met the functional enhancement and usage requirements emphasized by the Patent Law. The above description and drawings of this application are only the preferred embodiments of this application, and are not intended to limit this application. Therefore, all structures, devices, features, etc. that are similar or identical to this application, that is, all equivalent replacements or modifications made in accordance with the scope of the patent application of this application, should fall within the scope of protection of the patent application of this application.

Claims

1. A breast ultrasound tumor segmentation method, characterized in that: The method comprises: Acquire preprocessed input image information, wherein the preprocessing includes scaling the original image to a fixed size, applying Gaussian filtering to remove noise, normalizing pixel values, and generating diversified samples through data enhancement technology; Inputting the input image information into a breast ultrasound tumor segmentation model to obtain a corresponding tumor segmentation prediction map; Among them, after the breast ultrasound tumor segmentation model obtains the input image information, it dynamically extracts edge information in low-level features through the edge differential convolution module and uses it as a supplement to the global features, and then enhances the ability of the breast ultrasound tumor segmentation model in multi-scale feature capture through the multi-scale aggregation module; the breast ultrasound tumor segmentation model adopts a supervision strategy based on the alignment of foreground features and background features during training.

2. The method for breast ultrasound tumor segmentation according to claim 1, characterized in that: The edge information in the low-level features is dynamically extracted through the edge difference convolution module as a supplement to the global features, including: The low-level features extracted by the first encoder and the second encoder are respectively passed through the edge differential convolution module to obtain horizontal and vertical features, and then the dynamic feature fusion is performed using the connection between the horizontal edge features and the vertical edge features to obtain the fused features and input them into the decoder; the form of the dynamic feature fusion is as follows: F v ′ =σ(w a )×F v +(1-σ(w a ))×F h F h ′ =σ(w b )×F v +(1-σ(w b ))×F h F e =C(F v ′ ,F h ′ ) Among them, F v is the vertical feature obtained from the low-level features extracted from the first encoder by the edge difference convolution module, F h is the horizontal feature obtained from the low-level features extracted from the second encoder by the edge difference convolution module, F v ′ is the fusion feature map in the vertical direction, F h ′ is the fusion feature map in the horizontal direction, F e is the final fusion feature; w a and w b are learnable weight parameters, representing F h and F v σ represents the attention weight; σ represents the Sigmoid function, which is used to normalize the weight to [0,1]; C represents the concatenation operation.

3. The method for breast ultrasound tumor segmentation according to claim 2, characterized in that: After edge differential convolution, the expressions of horizontal and vertical features are as follows: Among them, x is the input feature map, a0 represents the current center pixel position, a n is to traverse the region R h and R v , θ is used to adjust the weight of the center pixel, w is the learnable weight, and b is the bias term.

4. The method for breast ultrasound tumor segmentation according to claim 1, characterized in that: The multi-scale aggregation module is arranged between the output end of the fifth encoder and the input end of the fourth decoder, and the corresponding expression is as follows: G=Conv 1×1 (F5) S g =C(G,X1,X2,X3)+F5 Wherein, F5 is the output of the fifth encoder; G is the feature map after dimensionality reduction by 1×1 convolution; X i is the feature map generated by the i-th branch; r represents the expansion rate, defined as r = 2 × i + 1, which is used to gradually increase the receptive field; S g It is the fused multi-scale feature map, which further strengthens the combination of global and local features by connecting with F5 residual; C represents the concatenation operation.

5. The method for breast ultrasound tumor segmentation according to claim 1, characterized in that: The breast ultrasound tumor segmentation model uses the feature map output by the first decoder to provide supervision for the decoders of other layers. The corresponding expression is as follows: in, represents the total L2 norm loss, which is used to measure the difference between feature maps of all layers; f l is the feature map of layer l; f L A feature map output by the first decoder; is the L2 norm, which is used to calculate the difference between feature maps of different layers and provide additional supervision signals.

6. The method for breast ultrasound tumor segmentation according to claim 5, characterized in that: The breast ultrasound tumor segmentation model adopts a supervision strategy based on alignment of foreground features and background features during training, including: Extracting a foreground feature map and a background feature map of the feature map output by the first decoder, and calculating feature alignment losses of the foreground region and the background region respectively; Combine the feature alignment loss of foreground and background regions with the binary cross entropy loss to obtain the total loss function; Among them, the expression corresponding to the feature alignment loss of the foreground area is as follows: in, is the feature alignment loss of the foreground region, f l fg (i) is the feature value of the i-th pixel in the foreground feature map output by the l-th decoder, is the feature value of the i-th pixel in the foreground feature map output by the first decoder; N fg is the number of foreground pixels, which is used to ensure loss normalization; the i in the expression corresponding to the feature alignment loss of the foreground area represents the foreground pixel index in the feature map; The expression corresponding to the feature alignment loss of the background area is as follows: in, is the feature alignment loss of the background region, f l bg (i) is the feature value of the i-th pixel in the background feature map output by the l-th decoder, is the feature value of the i-th pixel in the background feature map output by the first decoder, N bg is the number of background pixels; i in the expression corresponding to the feature alignment loss of the background area represents the background pixel index in the feature map; The expression corresponding to the total loss function is as follows: in, is the standard binary cross entropy loss, which is used to supervise the matching between the final output and GT, λ fg and λ bg is the weight hyperparameter of the foreground and background feature alignment loss.

7. The method for breast ultrasound tumor segmentation according to claim 6, characterized in that: The method of acquiring the feature map of the foreground area and the feature map of the background area includes: Use the foreground mask to filter the feature map and obtain the foreground feature map. The corresponding expression is as follows: f fg =f⊙y ⊙ represents the element-by-element multiplication operation, which is used to retain the features belonging to the foreground area in the feature map through the foreground mask, and set other areas to 0; f fg is the foreground feature map; f is the feature map used for screening; y is the foreground mask; The feature map of the background area is extracted by 1-y. The corresponding expression is as follows: f bg =f⊙(1-y) f bg is the background feature map.

8. A breast ultrasound tumor segmentation system, characterized in that: The system comprises: An information acquisition module, the information acquisition module is used to acquire preprocessed input image information, the preprocessing includes scaling the original image to a fixed size, applying Gaussian filtering to remove noise, normalizing pixel values, and generating diversified samples through data enhancement technology; A model prediction module, wherein the model prediction module is used to input the input image information into a breast ultrasound tumor segmentation model to obtain a corresponding tumor segmentation prediction map; Among them, after the breast ultrasound tumor segmentation model obtains the input image information, it dynamically extracts edge information in low-level features through the edge differential convolution module and uses it as a supplement to the global features, and then enhances the ability of the breast ultrasound tumor segmentation model in multi-scale feature capture through the multi-scale aggregation module; the breast ultrasound tumor segmentation model adopts a supervision strategy based on the alignment of foreground features and background features during training.

9. An electronic device, characterized in that: The method comprises one or more processors and a memory; one or more programs are stored in the memory and are configured so that the one or more processors execute the method according to any one of claims 1 to 7.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by at least one processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images