A breast ultrasound tomographic image segmentation model establishing method and a segmentation method

Abstract

The application discloses a breast ultrasound tomographic image segmentation model establishing method and a segmentation method, and belongs to the field of medical image segmentation. total = L E + alpha L vntrast + beta L uncer is a training loss function, the initial model constructed is trained by using the training set, and the establishment of the breast ultrasound tomographic image segmentation model is completed; wherein, L total represents a total loss; L E represents a segmentation error of the breast ultrasound tomographic image segmentation model; L contrast represents a contrast loss function between the intermediate result output by the decoding structure and the labeled result, L uncer is an uncertainty loss function between the intermediate result output by the decoding structure and the labeled result, alpha and beta represent weight coefficients; preferably, the model is inserted with a CrossFormer module and a multi-scale attention module in UNet. The application can effectively improve the segmentation precision of the breast ultrasound tomographic image.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of medical image segmentation, and more particularly relates to a breast ultrasound tomography image segmentation model establishment method and a segmentation method. BACKGROUND

[0002] Breast cancer is the second most deadly cancer in women after lung cancer, and regular breast screening can effectively reduce breast cancer mortality. Ultrasound tomography has the advantages of high sensitivity and standardized operation compared with traditional ultrasound, and its diagnostic results are not affected by the experience of doctors, and it has broad application prospects in breast tumor screening. Ultrasound tomography can be divided into two modalities, namely reflection mode and transmission mode, and the reflection mode image has higher resolution than the traditional ultrasound image. Transmission images can provide functional information of lesions, thereby further assisting doctors in diagnosis, so when imaging the breast, the transmission mode is often used. The ultrasound tomography system uses step-by-step scanning to obtain images of different layers of breast tissue, and then reconstructs a three-dimensional image of the breast tissue, making the lesion display more intuitive.

[0003] The probe frequency used by the ultrasound tomography system is usually 2-3MHz, which is much lower than the frequency of more than 10MHz used by the traditional ultrasound probe, resulting in lower image resolution, so it is a challenging and time-consuming task to identify lesions in breast ultrasound tomography images. Computer-aided diagnosis is widely used in medicine, mainly used to assist doctors in diagnosis and develop subsequent treatment plans, and improve the specificity and sensitivity of doctor's diagnosis. In computer-aided diagnosis of the breast, automatic medical image segmentation is the most critical step to improve diagnosis efficiency and accuracy. By segmenting the lesion from the ultrasound tomography image, it is beneficial for doctors to determine the size of the lesion and the degree of lesion, and to reconstruct an intuitive three-dimensional shape, which is convenient for doctors to make correct diagnosis and accurately develop treatment plans.

[0004] The UNet model has good effect in the application of medical image segmentation, so it is also widely used in breast ultrasound tomography image segmentation. UNet is a typical encoding-decoding structure, and its structure is as shown in Figure 1 The left convolutional network part is an encoding structure responsible for feature extraction, mainly composed of convolutional layers and down-sampling layers, and it can be seen that the feature map size is constantly decreasing; the right side is a decoding structure corresponding to the up-sampling process, which is restored to a size close to the original image through long skip connection (concat way) with different convolutional layers. In order to further improve the segmentation effect of breast ultrasound tomography images, researchers have improved the traditional UNet model and proposed some improved models, such as the Attention-UNet model, the UNet++ model, and the ResUNet model.

[0005] The above model can effectively segment the lesion area in the breast ultrasound tomographic image after training is completed, but unlike other medical images, the background of the breast ultrasound tomographic image is complex and the contrast between the lesion and other tissues is low, and in the decoding stage of the above model, a large amount of information is lost due to the upsampling operation, resulting in the problem of missegmentation in actual application. In addition, since the lesion in the breast ultrasound tomographic image is usually small, the existing model cannot accurately segment the small lesion area. In general, the segmentation accuracy of the existing breast ultrasound tomographic image segmentation method needs to be further improved. SUMMARY

[0006] In view of the defects of the prior art and the improvement needs, the present application provides a breast ultrasound tomographic image segmentation model establishing method and a segmentation method, which aims to improve the segmentation accuracy of the breast ultrasound tomographic image.

[0007] To achieve the above-mentioned purpose, according to one aspect of the present application, a breast ultrasound tomographic image segmentation model establishing method is provided, comprising:

[0008] constructing a training set, in which each training data is a breast ultrasound tomographic image with a labeled lesion area;

[0009] constructing an initial breast ultrasound tomographic image segmentation model based on a UNet model, which is used to segment the lesion area from the breast ultrasound tomographic image;

[0010] using L total =L E +αL contrast +βL uncer as the training loss function, training the initial breast ultrasound tomographic image segmentation model using the training set, and completing the establishment of the breast ultrasound tomographic image segmentation model;

[0011] wherein L total represents the total loss; L E represents the segmentation error of the breast ultrasound tomographic image segmentation model; L contrast represents the contrast loss function between the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model and the labeled result, and a represents its weight coefficient; L uncer is an uncertainty loss function, which is used to represent the difference between the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model and the labeled result, and β represents its weight coefficient; a≥0, β≥0, and a and β are not zero at the same time.

[0012] Further,

[0013]

[0014]

[0015]

[0016] wherein J represents the number of intermediate outputs of the model; L CE () represents the cross-entropy loss; l represents the label; p j represents the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model, represents the i-th channel of the intermediate result p j ; D KL represents the KL divergence, and C represents the number of categories, represents the weight assigned to the pixel p in the j-th intermediate result.

[0017] Further,

[0018]

[0019] wherein L represents the similarity between pixels; s o represents the feature in the intermediate result, s l represents the feature in the label; m and n represent the feature categories; represents the feature of the same category, represents the feature of different categories; τ represents the temperature coefficient.

[0020] Further, the breast ultrasound tomographic image segmentation model further comprises a CrossFormer module inserted between the last layer of the encoding structure and the decoding structure in the UNet model.

[0021] Further, the breast ultrasound tomographic image segmentation model further comprises a multi-scale attention module inserted in the long skip connection between the encoding structure and the decoding structure in the UNet model;

[0022] The multi-scale attention module comprises a dilated spatial convolution pooling pyramid and a feature enhancement module;

[0023] The dilated convolution pooling pyramid is used for multi-scale feature extraction on the features output by the corresponding layers in the encoding structure, to obtain multi-scale features;

[0024] The feature enhancement module takes the multi-scale features and the decoding features output by the corresponding layers in the decoding structure as inputs, and is used for enhancing the features related to the task and suppressing the features irrelevant to the task.

[0025] Further, the feature enhancement module comprises a first convolution layer, a second convolution layer, a third convolution layer, a ReLU activation layer, a Sigmoid activation layer, a pixel addition layer and a pixel multiplication layer;

[0026] The first convolution layer is used for inputting the multi-scale features and performing convolution operation;

[0027] The second convolutional layer is configured to input the decoding feature and perform a convolution operation;

[0028] The pixel addition layer is configured to perform pixel-by-pixel addition on the results output by the first convolutional layer and the second convolutional layer;

[0029] The ReLU activation layer is configured to perform activation processing on the result output by the pixel addition layer;

[0030] The third convolutional layer is configured to perform a convolution operation on the result output by the ReLU activation layer;

[0031] The Sigmoid activation layer is configured to perform activation processing on the result output by the third convolutional layer;

[0032] The pixel multiplication layer is configured to perform pixel-by-pixel multiplication on the result output by the Sigmoid activation layer and the multi-scale feature.

[0033] Further, a training set is constructed, comprising:

[0034] An original data set composed of breast ultrasound tomographic images is obtained, and a lesion region in each breast ultrasound tomographic image is labeled to obtain a corresponding lesion region mask image;

[0035] n consecutive breast ultrasound tomographic images are spliced in the channel dimension, and the spliced breast ultrasound tomographic images and the corresponding lesion region mask images form the training set;

[0036] wherein n is a positive integer greater than 1.

[0037] Further, n = 3;

[0038] And, n consecutive breast ultrasound tomographic images are spliced in the channel dimension, comprising:

[0039] Information of one channel is extracted from each breast ultrasound tomographic image to obtain information of three channels;

[0040] The information of the three channels is spliced in the channel dimension.

[0041] According to another aspect of the present application, a breast ultrasound tomographic image segmentation method is provided, comprising:

[0042] inputting the breast ultrasound tomographic image to be segmented into the breast ultrasound tomographic image segmentation model to obtain a lesion region segmentation result;

[0043] wherein the breast ultrasound tomographic image segmentation model is established by the breast ultrasound tomographic image segmentation model establishment method provided by the present application.

[0044] According to another aspect of the present application, a computer readable storage medium is provided, comprising a stored computer program; the computer program, when executed by a processor, controls a device in which the computer readable storage medium is located to perform the above-mentioned breast ultrasound tomographic image segmentation model establishing method provided by the present application, and / or the above-mentioned breast ultrasound tomographic image segmentation method provided by the present application.

[0045] Overall, the above technical solutions conceived by the present application can achieve the following beneficial effects:

[0046] (1) In the present application, at least one of the contrast loss function and the uncertainty loss function between the intermediate result output by the decoding structure and the labeled result is introduced in the training loss function when the established breast ultrasound tomographic image segmentation model is trained; the introduction of the contrast loss function enables the model to distinguish pixels belonging to different categories (i.e., lesion regions or backgrounds), more accurately classify pixels near the boundary, and thus alleviate the impact of low contrast of breast ultrasound tomographic images on segmentation accuracy; the introduction of the uncertainty loss function minimizes the difference between the intermediate result output by the decoding structure and the labeled result, enables the network to learn more discriminative and reliable knowledge at an early stage, effectively reduces the loss of detailed information caused by the upsampling operation in the decoding stage, and enables the model to accurately segment the lesion region even in the case of low contrast. Overall, the present application can effectively improve the segmentation accuracy of breast ultrasound tomographic images by improving the training loss function of the model.

[0047] (2) The difference between the model output and the label is usually measured using cross-entropy loss and Dice loss, but in breast ultrasound tomographic image segmentation, constructing an uncertainty loss function based only on cross-entropy loss has limited effect on alleviating the loss of detailed information caused by upsampling; in the preferred scheme of the present application, the designed uncertainty loss function introduces a regularization term based on KL divergence on the basis of cross-entropy loss, which can minimize the loss of detailed information caused by the upsampling operation in the decoding stage and further improve the segmentation accuracy of the model.

[0048] (3) In the preferred scheme of the present application, the designed contrast loss function maximizes the similarity of pixels belonging to the same category in the intermediate output and the labeled result, and minimizes the similarity of pixels belonging to different categories, which can further improve the ability of the model to distinguish pixels of different categories.

[0049] (4) The breast ultrasound tomographic image segmentation task belongs to a dense prediction task, and local features and global features are very important for the task; in the preferred scheme of the present application, a CrossFormer module is inserted between the last layer of the encoding structure and the decoding structure in the UNet model, on the basis of rich local information extracted by the convolutional layer and the pooling layer in the encoding structure, global information is extracted by the CrossFormer module, and by depending on the relationship between the encoded global information and the local information, the segmentation of small lesions is more accurate.

[0050] (5) In the preferred scheme of the present application, a multi-scale attention module is inserted in the long skip connection of the UNet model, which is specifically composed of a dilated spatial convolution pyramid and a feature enhancement module, wherein the dilated spatial convolution pyramid enables the network to increase the receptive field and capture multi-scale information while retaining as much local detail information as possible, and the feature enhancement module further processes the multi-scale features extracted by the dilated spatial convolution pyramid, enhances the features related to the task and suppresses the features irrelevant to the task, so that the model has high segmentation accuracy for small lesions.

[0051] (6) In the ultrasound tomographic imaging process, multiple images will be generated for the same object; in the preferred scheme of the present application, when constructing the data set for model training, the continuous multiple images are spliced in the channel dimension to serve as model input, which can effectively utilize the interlayer information of the ultrasound tomographic image and further improve the segmentation accuracy. In a further preferred scheme, three continuous images are selected to extract one channel respectively and spliced as model input, which can avoid that the spliced image contains too many background region features, and on the other hand, the dimension of the spliced image is consistent with the original image, which simplifies the processing of model input in the subsequent segmentation process. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 It is a schematic diagram of the existing UNet model structure;

[0053] Figure 2 It is a breast ultrasound tomographic image and its corresponding mask image provided by the embodiment of the present application; wherein (a) is a breast ultrasound tomographic image, and (b) is a mask image;

[0054] Figure 3 It is a schematic diagram of the breast ultrasound tomographic image segmentation model provided by the embodiment of the present application;

[0055] Figure 4 It is a schematic diagram of the existing CrossFormer module;

[0056] Figure 5 It is a schematic diagram of the multi-scale attention module provided by the embodiment of the present application;

[0057] Figure 6 The segmentation results of the segmentation method provided by the embodiment of the application and other segmentation methods based on the UNet model are shown in the following table; wherein (a) is the label of the lesion area of the breast ultrasound tomographic image, (b) is the segmentation result of the UNet model, (c) is the segmentation result of the Attention-UNet model, (d) is the segmentation result of the UNet++ model, (e) is the segmentation result of the ResUNet model, (f) is the segmentation result of the TransUNet model, and (g) is the segmentation result of the embodiment of the application. DETAILED DESCRIPTION

[0058] In order to make the objects, technical solutions and advantages of the application clearer, the application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the application and not to limit the application. In addition, the technical features involved in each embodiment of the application described below can be combined with each other as long as they do not conflict with each other.

[0059] In the application, the terms "first", "second", etc. (if any) in the application and the drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0060] In order to improve the accuracy of breast ultrasound tomographic image segmentation, the application provides a breast ultrasound tomographic image segmentation model establishment method and a segmentation method, and the overall idea is as follows: by optimizing the loss function in the model training process, the model has the ability to distinguish different categories of pixels, and / or learns more discriminative and reliable knowledge in the early stage, and reduces the loss of detailed information due to the up-sampling operation, so that the model can accurately segment the lesion area from the breast ultrasound tomographic image with low contrast. On this basis, the structure of the model is further improved, so that the model can better utilize the global information and local detailed information in the image, and further improve the segmentation accuracy.

[0061] The following is an embodiment.

[0062] Embodiment 1

[0063] A breast ultrasound tomographic image segmentation model establishment method comprises:

[0064] A training set is constructed; in the training set, each training data is a breast ultrasound tomographic image with a labeled lesion area; Figure 2 An example of the training data is shown as follows, Figure 2 In (a), the breast ultrasound tomographic image collected is shown, and (b) is a mask image with a labeled lesion area.

[0065] Optionally, in this embodiment, the specific steps of constructing the training set include the following steps:

[0066] (1) Use an ultrasonic tomography system with a frequency of 2-3 MHz to collect breast ultrasonic tomography images, a total of 372 images, and divide them into a training set, a validation set and a test set in a ratio of 7:1:2;

[0067] (2) The collected 372 images are labeled with lesions in a professional and consistent manner to obtain corresponding mask images as labels for model training;

[0068] (3) The breast ultrasonic tomography images in the training set and the corresponding labels are subjected to data augmentation through image flipping, rotation, scaling and other operations to expand the training set to 8 times the original training set, a total of 2088 images;

[0069] Through data augmentation, the robustness of the model can be improved and overfitting can be avoided;

[0070] (4) The size of the training images after data augmentation is normalized to 512x512 to meet the requirements of the model for the input image size, and then three consecutive images are spliced in the channel dimension. The specific splicing method is: extracting one channel of information from each image, then splicing the extracted three channels of information in the channel dimension, and the obtained three-channel image is used as the input of the segmentation model, that is, the input image size is 512x512x3;

[0071] In the ultrasonic tomography process, multiple images (usually 30) will be generated for the same object. In this embodiment, multiple consecutive images are spliced and used as model input, which can effectively utilize the interlayer information. The lesions in the breast ultrasonic tomography images are usually small, and the images of different layers often have a certain interlayer spacing. In addition, the original breast ultrasonic tomography image is a three-channel image, and the signals of each channel are the same. In this embodiment, one channel is extracted from each of the three consecutive ultrasonic images and then spliced into a three-channel image as the model input. On the basis of utilizing interlayer information, it not only avoids the inclusion of too much background area features in the spliced image, but also maintains the consistency of the dimension of the spliced image with the original image dimension, so that the channel dimension of the original breast ultrasonic tomography image is consistent with the model input requirement in the actual segmentation process, simplifying the processing of the model input in the subsequent segmentation process.

[0072] It should be noted that the above training set construction method is only a preferred embodiment of the present application and should not be understood as the only limitation of the present application. Other data sets of breast ultrasonic tomography images labeled with lesion regions can also be used in the present application. It is easy to understand that the image quantity, image size and other parameters in the above steps are only exemplary descriptions and should not be understood as the only limitation of the present application. They can be adjusted as needed in actual application.

[0073] The embodiment further includes: constructing an initial breast ultrasound tomographic image segmentation model based on the UNet model, for segmenting the lesion region from the breast ultrasound tomographic image.

[0074] Considering the original UNet model and the model improved based on the UNet model, since the details information will be lost in the up-sampling process in the decoding stage, it is difficult to segment the small lesions in the breast ultrasound tomographic image. The embodiment improves the UNet model to obtain a new breast ultrasound tomographic image segmentation model, and the UNet model used is the model shown in Figure 1 The related improvements include: inserting a CrossFormer module between the last layer of the encoding structure and the decoding structure in the UNet model, and inserting a multi-scale attention module (MFEModule) in the long skip connection of the UNet model.

[0075] The improved segmentation model structure is shown in Figure 3 The improved segmentation model structure is shown in

[0076] Referring to Figure 3 , in the breast ultrasound tomographic image segmentation model, the sixth layer is the inserted CrossFormer module; the CrossFormer module is composed of two consecutive CrossFormerBlock, as shown in Figure 4One of the CrossFormerBlock contains a Cross-Scale Embedding Layer (CEL), two Layer Norms, a Relative Position Encoding Layer (RPB), a Short Distance Attention Layer (SDA) and a Multi-Layer Perceptron Layer (MLP). Another CrossFormerBlock contains two Layer Norms, a Relative Position Encoding Layer (RPB), a Long Distance Attention Layer (LDA) and a Multi-Layer Perceptron Layer (MLP). The output feature map size of the sixth layer is the same as that of the fifth layer, i.e., still 32x32x512. The seventh layer is composed of a convolution layer and an up-sampling layer, and the channel number of the output feature map of the seventh layer is the same as that of the sixth layer, and the width and height are twice that of the sixth layer, i.e., the size of the output feature map of the seventh layer is 64x64x512.

[0077] In this embodiment, the CrossFormer module is inserted between the last layer of the encoding structure and the decoding structure in the UNet model. On the basis of the rich local information extracted by the convolution layer and the pooling layer in the encoding structure, the CrossFormer module extracts global information, and through the dependence between the encoding global information and the local information, the segmentation of small lesions can be more accurate.

[0078] Referring to Figure 3 In the breast ultrasound tomographic image segmentation model, the eighth to tenth layers are each composed of two convolution layers and an up-sampling layer, and the channel number of the output feature map of each layer becomes 1 / 2 of the channel number of the input feature map, and the width and height become twice the width and height of the input feature map. The width and height of the output feature map of the tenth layer are restored to the size of the input image, i.e., 512x512. The eleventh layer is composed of two convolution layers, and the twelfth layer is composed of a 1x1 convolution layer, and the output channel number is the number of categories, and the size of the output feature map is 512x512x2.

[0079] The CrossFormer module is taken as a distinction, and the left structure corresponds to the encoding structure, and the right structure corresponds to the decoding structure.

[0080] Referring to Figure 3In the breast ultrasound tomographic image segmentation model, the first, second, third and fourth layers in the encoding structure are inserted into the multi-scale attention module (MFEModule) and spliced with the tenth, ninth, eighth and seventh layers in the decoding structure using a long skip connection. The size of the output feature maps of the first layer and the tenth layer is 512x512x64, the size of the output feature maps of the second layer and the ninth layer is 256x256x128, the size of the output feature maps of the third layer and the eighth layer is 128x128x256, and the size of the output feature maps of the fourth layer and the seventh layer is 64x64x512. That is, the output feature maps of the first layer and the tenth layer are spliced to 512x512x128 and then convolution is performed, the output feature maps of the second layer and the ninth layer are spliced to 256x256x256 and then convolution is performed, the output feature maps of the third layer and the eighth layer are spliced to 128x128x512 and then convolution is performed, and the output feature maps of the fourth layer and the seventh layer are spliced to 64x64x1024 and then convolution is performed.

[0081] In this embodiment, the structure of the multi-scale attention module (MFEModule) is as shown in Figure 5 , which includes two branches. One branch is an atrous spatial pyramid pooling (ASPPModule) for multi-scale feature extraction of the features output by the corresponding layer in the encoding structure to obtain multi-scale features. The ASPPModule includes a 1x1 convolution, three 3x3 atrous spatial convolutions with expansion coefficients of 6, 12 and 18 respectively, and a 1x1 pooling layer. The structure of the ASPP enables the network to increase the receptive field and capture multi-scale information while preserving as much local detail information as possible. The other branch is a feature enhancement module, which includes three 1x1 convolutions (i.e., a first convolution layer, a second convolution layer and a third convolution layer), a ReLU activation layer, a Sigmoid activation layer, a pixel addition layer and a pixel multiplication layer, as shown in Figure 5 .

[0082] The first convolution layer is used to input multi-scale features and perform convolution operation.

[0083] The second convolution layer is used to input decoding features and perform convolution operation.

[0084] The pixel addition layer is used to perform pixel-by-pixel addition on the results output by the first convolution layer and the second convolution layer.

[0085] The ReLU activation layer is used to activate the result output by the pixel addition layer.

[0086] The third convolution layer is used to perform convolution operation on the result output by the ReLU activation layer.

[0087] The sigmoid activation layer is configured to activate the output of the third convolutional layer;

[0088] The pixel multiplication layer is configured to multiply the output of the sigmoid activation layer and the multi-scale features pixel by pixel, and the multiplied result is taken as the input of the next layer.

[0089] The feature enhancement module is configured to further process the multi-scale features extracted by the ASPP, so as to enhance the features related to the task and suppress the features irrelevant to the task. It should be noted that the structure of the feature enhancement module is an optional solution, and other modules based on the attention mechanism and capable of enhancing the features related to the task and suppressing the features irrelevant to the task can also be used in the present application.

[0090] In general, the present embodiment can improve the segmentation accuracy of small lesions by inserting the feature enhancement module in the long skip connection of the UNet model.

[0091] It should be noted that the above model is only an optional model of the present application, and should not be understood as the only limitation of the present application. In other embodiments of the present application, when the segmentation accuracy meets the application requirements, the CrossFormer module can be inserted into the corresponding position of the UNet model, or only the multi-scale attention module can be inserted, or the existing UNet model or Attention-UNet model, UNet++ model, ResUNet model and other improved models can be directly used.

[0092] It is easy to understand that when the structure of the UNet model changes, the number of layers of the improved breast ultrasound tomographic image segmentation model and the parameters in each layer can also change accordingly. The above description of the model parameters is only an exemplary description and should not be understood as the only limitation of the present application.

[0093] The initial breast ultrasound tomographic image segmentation model needs to be trained before it can perform actual segmentation tasks. The difference between the model output and the label is usually measured by cross-entropy loss and Dice loss. However, due to the low contrast between the lesion area and the background in the breast ultrasound tomographic image, and the large amount of information lost in the up-sampling operation in the decoding stage of the UNet model and its improved models, the model trained by the existing training method still has the problem of missegmentation. To solve this problem, the present embodiment introduces a contrast loss function and an uncertainty loss function between the intermediate results (i.e., O2, O3, O4 in the decoding structure output) and the annotation results in the training loss function of the model, and the improved loss function is as follows: Figure 3

[0094] L​total = L E + aL contrast + bL uncer

[0095] wherein, L total represents the total loss; L E represents the segmentation error of the breast ultrasound tomographic image segmentation model; L contrast represents the contrast loss function between the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model and the labeled result, and a represents the weight coefficient thereof; L uncer is the uncertainty loss function, which is used to represent the difference between the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model and the labeled result, and b represents the weight coefficient thereof; a > 0, b > 0, and a and b are not 0 at the same time; the specific values of a and b can be flexibly adjusted according to the actual segmentation effect, and optionally, in the embodiment, a and b are respectively set to 0.2 and 0.3.

[0096] In the above loss function designed in the embodiment, by introducing the uncertainty loss function, the difference between the intermediate result output by the decoding structure and the labeled result is minimized, so that the network can learn more discriminative and reliable knowledge in the early stage, and effectively reduce the loss of detailed information caused by the up-sampling operation in the decoding stage, so that the model can accurately segment the lesion area even in the case of low contrast. It is found that, based on the cross-entropy loss, the uncertainty loss function has limited effect on alleviating the loss of detailed information caused by up-sampling. In order to minimize the loss of detailed information caused by the up-sampling operation in the decoding stage, the embodiment introduces a regularization term based on KL divergence on the basis of the cross-entropy loss. Finally, the expression of the uncertainty loss function is specifically:

[0097]

[0098]

[0099]

[0100] wherein, J represents the number of intermediate outputs of the model; L CE () represents the cross-entropy loss; l represents the labeled result; p j represents the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model, represents the i-th channel of the intermediate result p j ; D KL represents the KL divergence, C represents the number of categories, represents the weight assigned to the pixel p in the j-th intermediate result.

[0101] The introduction of the contrast loss function enables the model to distinguish pixels belonging to different categories (i.e., lesion regions or backgrounds), more accurately classify pixels near the boundary, and thus alleviate the influence of low contrast of the breast ultrasound tomographic image on the segmentation accuracy. In this embodiment, the expression of the contrast loss function is as follows:

[0102]

[0103] wherein L represents the similarity between pixels; s o represents the features in the intermediate result, s l represents the features in the labeled result; m and n represent the feature categories, represents the m-th feature in the labeled result, and respectively represent the m-th and n-th features in the intermediate result s o . represents the features of the same category, represents the features of different categories; τ represents a temperature coefficient, which is 2 in this embodiment.

[0104] Optionally, in this embodiment, the cosine similarity is used to measure the similarity, and the corresponding expression is as follows:

[0105]

[0106] It should be noted that the cosine similarity is only an optional feature similarity measurement method, and should not be understood as the only limitation of the present application. Other measurement methods, such as Pearson correlation coefficient, Mahalanobis distance, Euclidean distance, etc., can also be used in the present application.

[0107] Based on the contrast loss function, the similarity between the intermediate output and the labeled result is maximized for pixels belonging to the same category, and the similarity between pixels belonging to different categories is minimized, which can further improve the ability of the model to distinguish pixels of different categories.

[0108] Optionally, in the above loss function, the segmentation error L E of the breast ultrasound tomographic image segmentation model is represented as the error between the segmentation result O1 of the lesion region and the label, which is still represented by the cross-entropy loss function L CE and the Dice loss function L Dice .

[0109] Finally, the overall loss function expression is as follows:

[0110]

[0111] L CE = -[ylogl + (1-y)log(1-l)]

[0112]

[0113] wherein, y represents a segmentation result output by the breast ultrasound tomographic image segmentation model, and l represents a labeled result;

[0114] Based on the above analysis, after the data set and the initial breast ultrasound tomographic image segmentation model are constructed, the embodiment further includes:

[0115] L total L E + aL contrast + bL uncer is used as a training loss function, the initial breast ultrasound tomographic image segmentation model is trained by using the training set, and the establishment of the breast ultrasound tomographic image segmentation model is completed.

[0116] It should be noted that in some other embodiments of the present application, one of the hyperparameters a and b can be 0.

[0117] It is easy to understand that after the model training is completed, in order to ensure that the segmentation result meets the requirements, the trained model will be further tested and verified by using the test set and the verification set. In the testing process, the Dice coefficient and the IoU can be used as evaluation indexes to evaluate the segmentation result.

[0118] Embodiment 2:

[0119] A breast ultrasound tomographic image segmentation method includes:

[0120] The breast ultrasound tomographic image to be segmented is input into the breast ultrasound tomographic image segmentation model to obtain a lesion region segmentation result.

[0121] The breast ultrasound tomographic image segmentation model is established by the breast ultrasound tomographic image segmentation model establishment method provided by the present application.

[0122] Embodiment 3:

[0123] A computer readable storage medium includes a stored computer program. When the computer program is executed by a processor, the computer readable storage medium controls the device where the computer readable storage medium is located to execute the breast ultrasound tomographic image segmentation model establishment method provided by the present application, and / or the breast ultrasound tomographic image segmentation method provided by the present application.

[0124] The beneficial effects that can be achieved by the present application are further analyzed and described below in combination with the segmentation effects of different models.

[0125] For Figure 3The breast ultrasound tomographic image segmentation model shown uses Pytorch as a deep learning framework, the experimental environment is Nvidia GeForce RTX 2080Ti (11GB) GPU, and the UNet network structure provided in Embodiment 1 above (i.e. Figure 3 The breast ultrasound tomographic image segmentation model shown) is used as the segmentation model, C0=2, C1=64, C2=128, C3=256, and C4=512 in the model. The input image size is 512x512x3. The random gradient descent algorithm (weight decay=0.0001, momentum=0.9) is used as the optimizer, the initial learning rate is set to 0.01, the learning rate is reduced to 0.1 times the original value after every 100 epochs, a total of 300 epochs are trained, and the batch size is set to 4. The total number of data sets used is 327, the data set is divided into a training set, a validation set, and a test set according to a ratio of 7:1:2, and the training set is expanded to 8 times the original size. The loss function uses the cross-entropy function, the Dice loss function, the contrast loss function, and the uncertainty loss function, and the Dice coefficient and the intersection over union (IoU) are used as evaluation indicators.

[0126] The existing UNet model, Attention-UNet model, UNet++ model, ResUNet model, and TransUNet model are used as comparative models, and different models are used to segment the breast ultrasound tomographic images, and the results are shown in Figure 6 . Figure 6 In the above, (a) is a plurality of breast ultrasound tomographic image lesion region labels, (b) is the segmentation result of the UNet model, (c) is the segmentation result of the Attention-UNet model, (d) is the segmentation result of the UNet++ model, (e) is the segmentation result of the ResUNet model, (f) is the segmentation result of the TransUNet model, and (g) is the segmentation result of the present embodiment. According to the results shown in Figure 6 It can be seen that the boundary segmented by the breast ultrasound tomographic image segmentation model established based on the present application is closer to the true boundary, and the error segmentation is relatively less. For ease of description, the model established in Embodiment 1 above will be referred to as "MFE-DSCrossUNet" hereinafter, and the Dice coefficients and IoU coefficients of the segmentation results of each model are shown in Table 1.

[0127] Table 1 Dice coefficients and IoU coefficients of segmentation results of different models

[0128]

[0129] According to the evaluation indexes shown in Table 1, the Dice coefficient of the segmentation result of the present application reaches 0.8883, and the IoU coefficient reaches 0.7990. Compared with the segmentation result of UNet, the Dice coefficient is increased by 3.68%, and the IoU coefficient is increased by 5.76%. It is fully proved that the breast ultrasound tomographic image segmentation model established in the present application is effective and superior.

[0130] Those skilled in the art will easily understand that the above description is only the preferred embodiment of the present application, and is not used to limit the present application, and any modification, equivalent replacement and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A breast ultrasound tomogram segmentation model establishment method, characterized in that, The method comprises the following steps: constructing a training set; each piece of training data in the training set is a breast ultrasound tomogram with a labeled lesion region; constructing an initial breast ultrasound tomogram segmentation model based on a UNet model, which is used to segment the lesion region from the breast ultrasound tomogram; With an initial breast ultrasound tomographic image segmentation model is trained using the training set as a training loss function, and the establishment of the breast ultrasound tomographic image segmentation model is completed. wherein, represents the overall loss; represents the segmentation error of the breast ultrasound tomographic image segmentation model; represents a contrast loss function between the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model and the labeled result, α represents a weight coefficient thereof; is an uncertainty loss function, used to represent the difference between the intermediate result output by the decoding structure in the breast ultrasound tomographic image segmentation model and the labeled result, β represents a weight coefficient thereof; α ≥ 0, β ≥ 0, and α , β are not simultaneously 0. wherein, represents the number of intermediate outputs of the model; represents the cross-entropy loss; represents the label result; represents the intermediate result of the decoding structure output in the breast ultrasound tomographic image segmentation model, represents the intermediate result of the first channel; represents the KL divergence, represents the number of classes, represents the weight assigned to the pixel in the first intermediate result; pixel in the first intermediate result. wherein, L represents the similarity between pixels; represents a feature in an intermediate result, represents a feature in a labeled result; m and n represents a feature class, represents a feature of the same class, represents a feature of a different class; represents a temperature coefficient.

2. The breast ultrasound tomogram segmentation model establishing method of claim 1, wherein, the breast ultrasound tomogram segmentation model further comprises a CrossFormer module inserted between the last layer of the encoding structure and the decoding structure in the UNet model.

3. The breast ultrasound tomogram segmentation model establishing method according to claim 1 or 2, characterized by, The breast ultrasound tomogram segmentation model further comprises a multi-scale attention module inserted in the long skip connection between the encoding structure and the decoding structure in the UNet model. The multi-scale attention module comprises a cavity spatial convolution pooling pyramid and a feature enhancement module. The cavity spatial convolution pooling pyramid is used to perform multi-scale feature extraction on the features output by the corresponding layer in the encoding structure to obtain multi-scale features. The feature enhancement module takes the multi-scale features and the decoding features output by the corresponding layer in the decoding structure as inputs, and is used to enhance the features related to the task and suppress the features irrelevant to the task.

4. The breast ultrasound tomogram segmentation model establishing method of claim 3, wherein, The feature enhancement module comprises a first convolution layer, a second convolution layer, a third convolution layer, a ReLU activation layer, a Sigmoid activation layer, a pixel addition layer and a pixel multiplication layer. The first convolution layer is used to input the multi-scale features and perform convolution operation. The second convolution layer is used to input the decoding features and perform convolution operation. The pixel addition layer is used to perform pixel-by-pixel addition on the results output by the first convolution layer and the second convolution layer. The ReLU activation layer is used to perform activation processing on the result output by the pixel addition layer. The third convolution layer is used to perform convolution operation on the result output by the ReLU activation layer. The Sigmoid activation layer is used to perform activation processing on the result output by the third convolution layer. The pixel multiplication layer is used to perform pixel-by-pixel multiplication on the result output by the Sigmoid activation layer and the multi-scale features.

5. The breast ultrasound tomogram segmentation model establishing method of claim 1, wherein, The method for constructing a training set comprises the following steps: obtaining an original data set composed of breast ultrasound tomograms, and labeling the lesion region in each breast ultrasound tomogram to obtain a corresponding lesion region mask image; The continuous n The breast ultrasound tomographic images are spliced in the channel dimension, and the training set is constituted by the spliced breast ultrasound tomographic images and the corresponding lesion region mask images. wherein n is a positive integer greater than 1.

6. The breast ultrasound tomogram segmentation model establishing method of claim 5, wherein, n =3； and, concatenating three consecutive breast ultrasound tomograms in the channel dimension, comprising: extracting one channel of information from each breast ultrasound tomogram to obtain three channels of information; concatenating the three channels of information in the channel dimension.

7. A breast ultrasound tomogram segmentation method characterized by, The method comprises the following steps: inputting the breast ultrasound tomogram to be segmented into the breast ultrasound tomogram segmentation model to obtain a lesion region segmentation result; wherein the breast ultrasound tomogram segmentation model is established by the breast ultrasound tomogram segmentation model establishment method of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer program is stored in the computer readable storage medium, and when executed by a processor, controls the device where the computer readable storage medium is located to execute the breast ultrasound tomogram segmentation model establishment method of any one of claims 1-6, and / or the breast ultrasound tomogram segmentation method of claim 7.

Images