A training method, device and electronic device for breast image classification model

Abstract

The present disclosure provides a training method, device and electronic device for a breast image classification model, the method comprising: obtaining labeled breast image data and unlabeled breast image data as training samples, training the model based on the labeled breast image data, determining a first loss value, then classifying and predicting the unlabeled breast image data through the model, obtaining the prediction probability of the unlabeled breast image data and the corresponding pseudo-label, determining the loss weight of the unlabeled breast image data, determining a second loss value according to the loss weight, and finally determining the total loss value of the breast image classification model, judging whether the total loss value meets the preset requirements, and determining whether to continue iterative training of the model. By applying this method, by determining the loss weight of the unlabeled breast image data and determining the second loss value of the model, the noise that may be caused by the pseudo-labels generated in semi-supervised learning is alleviated, and the utilization rate of the unlabeled data is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a training method, device and electronic equipment for a breast image classification model. Background Art

[0002] In breast image diagnosis, the judgment of whether the lesion is benign or malignant is of great significance. For malignant lesions, early detection and timely treatment can significantly improve the cure rate. For benign lesions, accurate classification can avoid unnecessary surgery or treatment and reduce the physical and mental burden of patients. Therefore, by analyzing the benign or malignant nature of the lesion, patients can be treated more accurately and in a targeted manner.

[0003] In the breast image classification task, for medical images of different types and modalities, special types of lesion images, etc., it is difficult to train the model through supervised learning with the current public data sets. Therefore, in the existing breast image classification, semi-supervised learning is often used to generate pseudo labels to train the model, and the generated pseudo labels are used as labels for unlabeled data. However, directly using pseudo labels as labels for unlabeled data for training will introduce noise. Therefore, a dynamic threshold adjustment strategy is proposed. Only when the model prediction probability is greater than the threshold will the data be included in the learning. However, after processing using the dynamic threshold method, more hyperparameters still need to be manually designed, which is inefficient. Summary of the invention

[0004] The present disclosure provides a training method, device, electronic device and storage medium for a breast image classification model to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present disclosure, a method for training a breast image classification model is provided, the method comprising:

[0006] Obtaining labeled breast image data and unlabeled breast image data as training sample data;

[0007] Training the breast image classification model based on the labeled breast image data;

[0008] Determining a first loss value for the labeled breast image data;

[0009] Using the breast image classification model trained by the labeled breast image data to perform classification prediction on the unlabeled breast image data, to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label;

[0010] Determining a loss weight of the unlabeled breast image data according to a predicted probability of the unlabeled breast image data;

[0011] Determining a second loss value of the unlabeled breast image data according to the loss weight;

[0012] Determine a total loss value corresponding to a current training round of the breast image classification model according to the first loss value and the second loss value;

[0013] Determining whether the total loss value meets preset requirements;

[0014] If the preset requirements are met, the breast image classification model obtained in the current training round is determined as the target breast image classification model; if the preset requirements are not met, the unlabeled breast image data with pseudo-labels obtained in the current training round is determined as labeled breast image data, and the step of training the breast image classification model based on the labeled breast image data is returned until the total loss value of the breast image classification model meets the preset requirements.

[0015] In one possible implementation, the training of the breast image classification model based on the labeled breast image data includes: obtaining a first breast image and a second breast image of the labeled breast image data; performing feature extraction on the first breast image and the second breast image; inputting the obtained features into a self-attention module for a first weight distribution to obtain a first feature vector; segmenting the first feature vector to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image; inputting the first high-dimensional feature vector and the second high-dimensional feature vector into a cross-attention module for a second weight distribution to obtain a second feature vector and a third feature vector; performing maximum pooling processing on the first high-dimensional feature vector to obtain a first feature, performing linear projection processing on the second high-dimensional feature vector to obtain a second feature, and performing average pooling processing on the second feature vector and the third feature vector to obtain a third feature; obtaining a fused feature based on the first feature, the second feature and the third feature; and inputting the fused feature into a fully connected layer to obtain the lesion type of the labeled breast image data.

[0016] In one possible implementation, determining the first loss value of the labeled breast image data includes: determining the cross loss value of the labeled breast image data according to the following cross loss function: Among them, L ce represents the cross loss value of the labeled breast image data, N represents the number of labeled breast image data, K represents the number of categories of breast image data, i represents labeled breast image data, j represents the category of breast image data, y ij represents the true label of the i-th labeled breast image data in the j-th category, p ijrepresents the predicted probability of the i-th labeled breast image data in the j-th category; the multi-view loss value of the labeled breast image data is determined according to the following multi-view loss function: Among them, L mv represents the multi-view loss value of the labeled breast image data, O f is the first high-dimensional feature vector, R f is the second high-dimensional feature vector, O a is the second eigenvector, R a is the third eigenvector; and the first loss value is determined according to the cross loss value and the multi-view loss value.

[0017] In one possible implementation, determining the loss weight of the unlabeled breast image data based on the predicted probability of the unlabeled breast image data includes: determining the mean and variance of the unlabeled breast image data based on the predicted probability of the unlabeled breast image data; updating the mean and variance through a smoothing algorithm; and determining the loss weight of the unlabeled breast image data based on the updated mean and variance of the unlabeled breast image data.

[0018] In one possible implementation manner, after obtaining the predicted probability of the unlabeled image data, the method further includes: performing a standardized alignment process on the predicted probability.

[0019] In one possible implementation, after obtaining the labeled breast image data and the unlabeled breast image data as training sample data, the method further includes: performing image enhancement processing on the unlabeled breast image data; and performing standardization processing on the unlabeled breast image data and the labeled breast image data after the image enhancement processing.

[0020] According to a second aspect of the present disclosure, a method for classifying breast images is provided, which is applied to the above-mentioned target breast image classification model, and the method comprises: inputting breast image data to be classified into the target breast image classification model; and determining that the output of the target breast image classification model is the classification result of the breast image data to be classified.

[0021] According to a third aspect of the present disclosure, a training device for a breast image classification model is provided, the device comprising:

[0022] An acquisition module, used for acquiring labeled breast image data and unlabeled breast image data as training sample data;

[0023] A training module, used for training the breast image classification model based on the labeled breast image data;

[0024] A first determination module, used to determine a first loss value of the labeled breast image data;

[0025] The training module is further used to classify and predict the unlabeled breast image data using the breast image classification model trained by the labeled breast image data to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label;

[0026] The first determination module is further used to determine the loss weight of the unlabeled breast image data according to the predicted probability of the unlabeled breast image data;

[0027] The first determination module is further used to determine a second loss value of the unlabeled breast image data according to the loss weight;

[0028] The first determination module is further used to determine a total loss value corresponding to a current training round of the breast image classification model according to the first loss value and the second loss value;

[0029] A judgment module, used to determine whether the total loss value meets the preset requirements;

[0030] The judgment module is also used to determine the breast image classification model obtained in the current training round as the target breast image classification model if the preset requirements are met; if the preset requirements are not met, determine the unlabeled breast image data with pseudo labels obtained in the current training round as labeled breast image data, and return to the step of training the breast image classification model based on the labeled breast image data until the total loss value of the breast image classification model meets the preset requirements.

[0031] In one embodiment, the training module includes: an acquisition submodule for acquiring a first breast image and a second breast image of the labeled breast image data; a feature extraction submodule for extracting features from the first breast image and the second breast image; a processing submodule for inputting the obtained features into a self-attention module for a first weight distribution to obtain a first feature vector; a segmentation submodule for segmenting the first feature vector to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image; the processing submodule is also used to extract the first high-dimensional feature vector and the second high-dimensional feature vector corresponding to the first breast image. The second high-dimensional feature vector is input into the cross-attention module for a second weight distribution to obtain a second feature vector and a third feature vector; the processing submodule is further used to perform maximum pooling processing on the first high-dimensional feature vector to obtain a first feature, perform linear projection processing on the second high-dimensional feature vector to obtain a second feature, and perform average pooling processing on the second feature vector and the third feature vector to obtain a third feature; a fusion submodule is used to obtain a fused feature based on the first feature, the second feature and the third feature; the processing submodule is further used to input the fused feature into a fully connected layer to obtain the lesion type of the labeled breast image data.

[0032] In one embodiment, the first determination module includes: a first determination submodule, configured to determine the cross loss value of the labeled breast image data according to the following cross loss function: Among them, L ce represents the cross loss value of the labeled breast image data, N represents the number of labeled breast image data, K represents the number of categories of breast image data, i represents labeled breast image data, j represents the category of breast image data, y ij represents the true label of the i-th labeled breast image data in the j-th category, p ij Represents the predicted probability of the i-th labeled breast image data in the j-th category;

[0033] The first determination submodule is further configured to determine the multi-view loss value of the labeled breast image data according to the following multi-view loss function: Among them, L mv represents the multi-view loss value of the labeled breast image data, O f is the first high-dimensional feature vector, R f is the second high-dimensional feature vector, O a is the second eigenvector, R a is the third eigenvector;

[0034] The first determination submodule is further used to determine the first loss value according to the cross loss value and the multi-view loss value.

[0035] In one possible implementation, the first determination module also includes: a second determination submodule, used to determine the mean and variance of the unlabeled breast image data based on the predicted probability of the unlabeled breast image; an update submodule, used to update the mean and variance through a smoothing algorithm; and a third determination submodule, used to determine the loss weight of the unlabeled breast image data based on the updated mean and variance of the unlabeled breast image data.

[0036] In one possible implementation manner, the device further includes: a processing module, configured to perform a standardized alignment process on the predicted probability after obtaining the predicted probability of the unlabeled image data.

[0037] In one possible implementation manner, the processing module is further used to perform image enhancement processing on the unlabeled breast image data after acquiring the labeled breast image data and the unlabeled breast image data as sample data; and to perform standardization processing on the unlabeled breast image data and the labeled breast image data after the image enhancement processing.

[0038] According to a fourth aspect of the present disclosure, a breast image classification device is provided, which is applied to the above-mentioned target breast image classification model, and the device comprises:

[0039] An input module, used for inputting the breast image data to be classified into the target breast image classification model;

[0040] The second determination module is used to determine that the output of the target breast image classification model is the classification result of the breast image data to be classified.

[0041] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0042] at least one processor; and

[0043] a memory communicatively connected to the at least one processor; wherein,

[0044] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0045] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present disclosure.

[0046] The present invention discloses a training method, device, electronic device and storage medium for a breast image classification model. The method first obtains labeled breast image data and unlabeled breast image data as training sample data. The breast image classification model is trained based on the labeled breast image data to determine a first loss value of the labeled breast image data. The breast image classification model trained with the labeled breast image data is used to perform classification prediction on the unlabeled breast images to obtain a prediction probability of the unlabeled breast images and a corresponding pseudo label. The loss weight of the unlabeled breast image data is determined according to the prediction probability of the unlabeled breast image data. The loss weight of the unlabeled breast image data is determined according to the loss weight. Re-determine the second loss value of the unlabeled breast image data, determine the total loss value corresponding to the current training round of the breast image classification model according to the first loss value and the second loss value, and judge whether the total loss value meets the preset requirements. If so, determine the breast image classification model obtained in the current round as the target breast image classification model. If not, determine the unlabeled breast image data with pseudo labels obtained in the current training round as labeled breast image data, and return to iterative training of the breast image classification model based on the labeled breast image data until the total loss value of the breast image classification model meets the preset requirements.

[0047] By applying this method, a semi-supervised method is used to train a breast image classification model. By determining the loss weight of unlabeled breast image data, a second loss value of the unlabeled breast image data is determined. Instead of simply determining the pseudo labels that can be used for model training by means of a threshold, the noise that may be caused by the pseudo labels generated in semi-supervised learning is alleviated, the utilization rate of unlabeled data is improved, and no additional parameters need to be introduced during the model training process. The model is optimized by optimizing the loss of unlabeled breast image data.

[0048] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0050] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0051] Figure 1 The schematic diagram shows the implementation process of a training method for a breast image classification model according to an embodiment of the present disclosure. Figure 1 ;

[0052] Figure 2 The schematic diagram shows the implementation process of a training method for a breast image classification model according to an embodiment of the present disclosure. Figure 2 ;

[0053] Figure 3 The schematic diagram shows the implementation process of a training method for a breast image classification model according to an embodiment of the present disclosure. Figure 3 ;

[0054] Figure 4 A schematic diagram of the implementation process of a breast image classification method according to an embodiment of the present disclosure is shown;

[0055] Figure 5 A module schematic diagram of a training device for a breast image classification model according to an embodiment of the present disclosure is shown;

[0056] Figure 6 A schematic diagram of a module of a breast image classification device according to an embodiment of the present disclosure is shown;

[0057] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0058] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0059] Figure 1 The schematic diagram shows the implementation process of a training method for a breast image classification model according to an embodiment of the present disclosure. Figure 1 ,include:

[0060] Step 101: Obtain labeled breast image data and unlabeled breast image data as training sample data.

[0061] First, training sample data is obtained. The present application adopts a semi-supervised training method to train the breast image classification model. Therefore, the obtained training sample data includes labeled breast image data and unlabeled breast image data, wherein the labeled breast image data and the unlabeled breast image data are both breast image data containing lesions, and the label of the labeled breast image data represents the type of lesions in the breast image data.

[0062] Step 102, training a breast image classification model based on labeled breast image data;

[0063] Step 103, determining a first loss value of the labeled breast image data.

[0064] After obtaining the training sample data, the breast image classification model is first trained based on the labeled breast image data, and the labeled breast image data is input into the breast image classification model to obtain the predicted probability of the labeled breast image data and the corresponding lesion type, and a first loss value of the labeled breast image data is determined based on the predicted probability of the labeled breast image data. The first loss value represents the difference between the lesion type predicted and determined by the breast image classification model and the actual lesion type.

[0065] Step 104 , classify and predict the unlabeled breast image data using the breast image classification model trained with the labeled breast image data to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label.

[0066] The breast image classification model is trained by semi-supervised training. After the model is trained with labeled breast image data, the unlabeled breast image data is input into the breast image classification model trained with labeled breast image data. The breast image classification model performs classification prediction on the unlabeled breast image data to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label. The pseudo label can use the maximum prediction probability as the prediction category.

[0067] Step 105 : determining the loss weight of the unlabeled breast image data according to the predicted probability of the unlabeled breast image data.

[0068] The present application does not directly discard the unlabeled breast image data with low prediction probability, but controls the pseudo-labels in a weighted manner. The loss weight of the unlabeled breast image data is related to the prediction probability of the unlabeled breast image data. The greater the prediction probability, the greater the loss weight, and the smaller the prediction probability, the smaller the loss weight. A lower loss weight is given to the unlabeled breast image data with low prediction probability. Through this loss weight, the loss value of the unlabeled breast images with high prediction probability can be increased, and the loss value brought by the unlabeled breast images with low prediction probability can be reduced. By determining the loss weight of the unlabeled breast image data, the impact of the noise generated by introducing erroneous pseudo-label data into model training in semi-supervised learning can be alleviated, and the utilization rate of the unlabeled breast image data can be improved.

[0069] Step 106: Determine a second loss value of the unlabeled breast image data according to the loss weight.

[0070] The second loss value of the unlabeled breast image data is determined according to the following formula:

[0071] Where L uis the second loss value of the unlabeled breast image data, N u is the number of unlabeled breast image data in the current training round, i represents the i-th unlabeled breast image data, λ(P i ) represents the loss weight of the i-th unlabeled breast image data, K represents the number of categories of breast image data, j represents the category of breast image data, P ij represents the predicted probability of the i-th unlabeled breast image data in the j-th category, Pseudo labels for unlabeled breast image data.

[0072] Step 107: Determine a total loss value corresponding to a current training round of the breast image classification model according to the first loss value and the second loss value.

[0073] The first loss value and the second loss value are added and the sum of the first loss value and the second loss value is used as the total loss value corresponding to the current round of the breast image classification model.

[0074] Step 108, determining whether the total loss value meets the preset requirements; if it meets the preset requirements, determining the breast image classification model obtained in the current training round as the target breast image classification model; if it does not meet the preset requirements, determining the unlabeled breast image data with pseudo labels obtained in the current training round as labeled breast image data, and returning to the step of training the breast image classification model based on the labeled breast image data until the total loss value of the breast image classification model meets the preset requirements.

[0075] The total loss value is judged to determine whether the total loss value meets the preset requirements. The preset requirements can be whether the difference between the total loss value of the current round and the total loss value of the previous round is within a preset range or whether the breast image classification model converges according to the total loss value. When the total loss value meets the preset requirements, the breast image classification model obtained in the current round is determined as the target breast image classification model, and the breast image data to be predicted can be predicted according to the target breast image classification model to determine the benign and malignant nature of the breast image data to be predicted; when the total loss value does not meet the preset requirements, the unlabeled breast image data with labels is used as labeled breast image data, and the steps of training the model based on the labeled breast image data are returned to iterate repeatedly until the total loss value of the breast image classification model meets the preset requirements, and the target breast image classification model is obtained. The number of unlabeled breast image data with pseudo-labels used for model training and the corresponding loss weights are updated by the total loss value, so that the model can be better optimized.

[0076] The disclosed embodiment provides a training method for a breast image classification model, which obtains labeled breast image data and unlabeled breast image data as training samples, and uses a semi-supervised training method to train the breast image classification model. First, the model is trained based on the labeled breast image data to determine a first loss value, and then the unlabeled breast image data is classified and predicted by the model to obtain the prediction probability and pseudo-label corresponding to the unlabeled breast image data, and the loss weight corresponding to the unlabeled breast image data is determined. According to the loss weight, a second loss value is determined, and finally the total loss value of the breast image classification model is determined. The total loss value is judged to determine whether to iteratively train the model. By applying this method, the second loss value of the model is determined by determining the loss weight of the unlabeled breast image data, thereby alleviating the noise that may be caused by the pseudo-labels generated in semi-supervised learning and improving the utilization rate of the unlabeled data.

[0077] In one possible implementation, Figure 2 As shown, the breast image classification model is trained based on labeled breast image data, including:

[0078] Step 201: Acquire a first breast image and a second breast image with labeled breast image data.

[0079] First, labeled breast image data is acquired, and first breast image data and second breast image data of the labeled breast image data are acquired. Breast image data usually include shots of four annotated body positions, namely, Left Cranio-Caudal (LCC), Right Cranio-Caudal (RCC), Left Medio-Lateral Oblique (LMLO) and Right Medio-Lateral Oblique (RMLO). When the lesion can be observed at different projection positions of the ipsilateral breast, the breast image data observed at different projection positions of the ipsilateral breast are mapped together, that is, when shooting the breast image data, if the lesion can be observed at both LCC and LMLO, the breast image data observed at LCC and the breast image data observed at LMLO are mapped together and stored as unilateral breast image data of the breast image data; when the lesion can only be observed at either LCC or LMLO, the breast image data observed at LCC or LMLO is used as unilateral breast image data of the breast image data. In addition, an index table between lesions may be established, in which multiple breast image data of the breast image data are stored.

[0080] When training a breast image classification model with labeled breast image data, first breast image data and second breast image data with labeled breast image data are obtained as the base image and the counter image based on the index table, wherein the first breast image data is the cranio-caudal image data or the medial-lateral oblique image data, and the second breast image is the medial-lateral oblique image data or the cranio-caudal image belonging to the same side as the first breast image. If the first breast image is LCC, the second breast image is LMLO. It is understandable that after acquiring the first breast image, the second breast image corresponding to the first breast image can be determined based on the index table. If the labeled breast image data in the index table only includes the first breast image but not the second breast image, the first breast image is flipped and the flipped image is used as the second breast image.

[0081] Step 202: extract features from the first breast image and the second breast image.

[0082] The size of the first breast image and the second breast image is (3, 16, 224, 224), where 3 represents the number of channels, 16 represents the size of the image in the depth dimension, and 224 represents the size of the image in the height dimension and the width dimension. The first breast image and the second breast image are spliced ​​in the depth dimension to obtain a spliced ​​image with a size of (3, 32, 224, 224). The spliced ​​image is input into the feature extraction module of the breast image classification model for feature extraction. In the present application, the Resnet3D model can be used as the image feature extraction module for feature extraction. The Resnet3D model can make the breast image classification model more lightweight. Of course, other feature extraction models can also be selected as the feature extraction module for feature extraction according to actual conditions.

[0083] Step 203: Input the obtained features into the self-attention module for the first weight distribution to obtain the first feature vector.

[0084] The features extracted from the stitched image by the feature extraction module are input into the self-attention module of the breast image classification model in the depth dimension. This module can extract the key information of the features in the depth layer according to the correlation between different depth layers of the stitched image, and perform the first information weight allocation on the obtained features to obtain the first feature vector. In this way, more attention can be paid to image frames with important information, which can improve the classification performance of the breast image classification model.

[0085] Step 204 , segmenting the first feature vector to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image.

[0086] Since the first eigenvector is determined by feature extraction and weight redistribution after the first breast image and the second breast image are spliced, the first eigenvector is segmented to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image.

[0087] Step 205 , input the first high-dimensional feature vector and the second high-dimensional feature vector into the cross-attention module for a second weight distribution to obtain a second feature vector and a third feature vector.

[0088] The first high-dimensional feature vector and the second high-dimensional feature vector are input into the cross-attention module in the breast image classification model for fusion. Through the cross-attention module, the correspondence between different high-dimensional feature vectors can be learned, and a second information weight allocation is performed based on the correlation between different high-dimensional feature vectors. In this way, a high-dimensional feature vector can be weightedly fused according to the position in another high-dimensional feature vector to achieve information interaction between images of different perspectives, so that the breast image classification model can obtain richer semantic information and better utilize the information between the two high-dimensional feature vectors to obtain a second feature vector corresponding to the first high-dimensional feature vector and a third feature vector corresponding to the second high-dimensional feature vector.

[0089] Step 206, performing maximum pooling processing on the first high-dimensional feature vector to obtain a first feature, performing linear projection processing on the second high-dimensional feature vector to obtain a second feature, and performing average pooling processing on the second feature vector and the third feature vector to obtain a third feature.

[0090] The first high-dimensional feature vector is input into the maximum pooling layer of the breast image classification model for maximum pooling processing to obtain the first feature, the second high-dimensional feature vector is input into the linear projection layer of the breast image classification model for linear projection processing to obtain the second feature, and the second feature vector and the third feature vector are input into the global average pooling layer of the breast image classification model for global average pooling processing to obtain the third feature.

[0091] Step 207, obtaining a fusion feature according to the first feature, the second feature and the third feature.

[0092] The first feature, the second feature and the third feature are input into the feature fusion layer of the breast image classification model. In the feature fusion layer, the first feature and the second feature are firstly weighted summed, specifically: S′=S1+λ*S2, where S′ is the feature after the weighted summation of the first feature and the second feature, S1 is the first feature, S2 is the second feature, and λ is a learnable parameter, which can reflect the weight of the feature of the second breast image when it acts on the feature of the first breast image. This process mainly fuses the global features of the first breast image and the second breast image, and can remove redundant information while strengthening the key features by weighted summation. The obtained S′ is spliced ​​with the third feature along the last dimension to obtain the fused feature.

[0093] Step 208, input the fused features into the fully connected layer to obtain the lesion type of the labeled breast image data.

[0094] The fused features are input into the fully connected layer of the breast image classification model, and after being processed by the fully connected layer, the output is the type of lesions in the labeled breast image data.

[0095] Since breast images may contain noise, artifacts, or deformation, the features of a single perspective may be disturbed. We can make full use of image information from different projection positions to train the model, reduce the images of the above problems, and improve the robustness of the breast image classification model.

[0096] In one embodiment, determining a first loss value for a labeled breast image includes:

[0097] The cross loss value of the labeled breast image data is determined according to the following cross loss function:

[0098]

[0099] Among them, L ce represents the cross loss value of labeled breast image data, N represents the number of labeled breast image data, K represents the number of categories of breast image data, i represents labeled breast image data, j represents the category of breast image data, y ij represents the true label of the i-th labeled breast image data in the j-th category, p ij Represents the predicted probability of the i-th labeled breast image data in the j-th category;

[0100] The multi-view loss value of labeled breast image data is determined according to the following multi-view loss function:

[0101]

[0102] Among them, L mvrepresents the multi-view loss value of labeled breast image data, O f is the first high-dimensional eigenvector, R f is the second highest dimensional eigenvector, O a is the second eigenvector, R a is the third eigenvector;

[0103] The first loss value is determined according to the cross loss value and the multi-view loss value.

[0104] After the breast image classification model is trained by the labeled breast image data, a first loss value of the labeled breast image data is calculated, and the first loss value includes a cross loss value and a multi-view loss value, wherein the cross loss value is used to measure the difference between the benign and malignant prediction categories of the labeled breast image data obtained by the breast image classification model and the true labels of the labeled breast image data. Specifically, the cross loss value of the labeled breast image data is determined according to the following cross loss function: Among them, L ce represents the cross loss value of labeled breast image data, B represents the number of labeled breast image data, K represents the number of categories of breast image data, the categories of breast image data in this application include benign and malignant, i represents labeled breast image data, j represents the category of breast image data, y ij represents the true label of the i-th labeled breast image data in the j-th category. In this application, y ij The value of includes 0 and 1. ij Represents the predicted probability of the i-th labeled breast image data in the j-th category.

[0105] The multi-view loss value is used to measure the similarity between the first breast image and the second breast image, enhance the ability of the breast image classification model to understand different input data, and improve the generalization of the breast image classification model. Specifically, the multi-view loss value of the labeled breast image data is determined according to the following multi-view loss function: Among them, L mv represents the multi-view loss value of labeled breast image data, O f is the first high-dimensional eigenvector, R f is the second highest dimensional eigenvector, O a is the second eigenvector, R a is the third eigenvector.

[0106] Finally, the cross loss value and the multi-view loss value are added to obtain the first loss value of the labeled breast image data.

[0107] In one possible implementation, Figure 3 As shown, the loss weight of the unlabeled breast image data is determined according to the predicted probability of the unlabeled breast image data, including:

[0108] Step 301, determining the mean and variance of the unlabeled breast image according to the predicted probability of the unlabeled breast image;

[0109] Step 302, updating the mean and variance by a smoothing algorithm;

[0110] Step 303: determine the loss weight of the unlabeled breast image data according to the updated mean and variance of the unlabeled breast image data.

[0111] The mean of the current training round of unlabeled breast image data is determined by the following formula: where μ u is the mean of the current training round of unlabeled breast image data, N u is the number of unlabeled breast image data in the current training round, i represents the i-th unlabeled breast image data, P i Represents the predicted probability of the i-th unlabeled breast image data predicted by the breast image classification model.

[0112] The variance of the unlabeled breast image is determined by the following formula: where σ u 2 represents the variance of the current training round of unlabeled breast image data, N u is the number of unlabeled breast image data in the current training round, i represents the i-th unlabeled breast image data, P i represents the predicted probability of the i-th unlabeled breast image data predicted by the breast image classification model, μ u is the mean of the current training round of unlabeled breast image data.

[0113] The mean and variance are updated by a smoothing algorithm. The smoothing algorithm used in this application is an exponential moving average (EMA) data smoothing algorithm to update the mean and variance, specifically: where μ t is the updated mean of the current training round, m is a hyperparameter, μ t-1 is the updated mean value of the previous training round, μ u is the mean of the current training round of unlabeled breast image data, σ t 2 is the updated variance of the current training round, σ t-1 2 is the variance after the last training round update, N u is the number of unlabeled breast image data in the current training round, Nu-1 is the number of unlabeled breast image data in the previous round of training, σ u 2 Represents the variance of the current training round of the unlabeled breast image data.

[0114] Determine the loss weights for the unlabeled breast image data based on the updated mean and variance: Where λ(P i ) represents the loss weight of the i-th unlabeled breast image data, P i represents the predicted probability of the i-th unlabeled breast image data predicted by the breast image classification model, μ t is the updated mean of the current training round, σ t 2 is the updated variance of the current training round. i )<μ t hour, When max(P i )≥μ t When λ(P i )=1.0.

[0115] In one possible implementation manner, after obtaining the predicted probability of the unlabeled image data, the method further includes: performing a standardized alignment process on the predicted probability.

[0116] In order to handle the situation where there are different levels of sample learning difficulty in the training sample data, the predicted probabilities can be normalized (Normalized, Nor), and then the loss weights can be calculated using the predicted probabilities that have undergone normalization and alignment. Compared with directly calculating the loss weights using the original predicted probability values, the table conversion and alignment process can assign higher weights to samples that are difficult to learn or have fewer categories, alleviating the problem of category imbalance to a certain extent.

[0117] Loss weights for unlabeled breast image data after normalizing and aligning the predicted probabilities:

[0118] In one possible implementation, after obtaining labeled breast image data and unlabeled breast image data as training sample data, the method further includes: performing image enhancement processing on the unlabeled breast image data; and performing standardization processing on the unlabeled breast image data and labeled breast image data after the image enhancement processing.

[0119] After obtaining labeled breast image data and unlabeled breast image data as training samples, the lesions in the breast image data are cropped along the lesion frame as the region of interest. In order to reduce the content and existing content occupied during model training, the cropping size can be set to (16, 256, 256). The three dimensions in this size represent the depth, height and width of the lesion, respectively. The depth direction is formed by taking 8 frames above and below the center of the lesion. After the lesion data is cropped, the obtained region of interest image is enhanced. The semi-supervised learning method is based on the consistency regularization assumption, that is, the model's prediction results for similar data with added interference should be the same as the original data.

[0120] Therefore, the region of interest obtained from the unlabeled image data can be enhanced. The specific enhancement processing includes two methods, strong enhancement and weak enhancement. The prediction results output by the breast image classification model for the unlabeled image data after weak enhancement and strong enhancement are also the same. Among them, in this scheme, the weak enhancement processing can adopt methods such as random flipping, random rotation, random center cropping, etc. Random center cropping can specify the cropping size as (16, 224, 224), and its purpose is to limit the range of random cropping. Strong enhancement processing can adopt methods such as random flipping, random rotation, random center cropping, contrast adjustment and Gaussian additive noise. Random center cropping can also be executed with a cropping size of (16, 224, 224). By adding Gaussian noise for perturbation, the diversity of training samples can be expanded, so that the model can better capture the distribution characteristics of the data, which helps to improve the robustness and generalization ability of the breast image classification model.

[0121] After image enhancement, the labeled breast image data and the unlabeled breast image data after image enhancement are standardized. First, the pixel value range of the breast image is determined according to the window width and window position of the breast image data. The specific pixel value range is The pixel value range is 0-255, where WC is the window level of the breast image data and WW is the window width of the breast image data. The adjusted pixel value is then normalized, specifically: image1 is the normalized pixel value, image is the pixel value range of 0-255, and vmin is the minimum value of the pixel value range. vmax is the maximum value of the pixel value range, that is, After the pixel values ​​are normalized, the normalized pixel values ​​are then standardized according to the mean and variance of the image pixel values ​​that can be processed by the breast image classification model to be trained. image2 is the standardized pixel value, pixel_mean is the mean of the image pixel values ​​that the breast image classification model can process, and pixel_stdn is the variance of the image pixel values ​​that the breast image classification model can process. The standardized image pixel values ​​have a centralized distribution pattern, which can make the breast image classification model easier to converge.

[0122] Figure 4 A schematic diagram of the implementation process of a breast image classification method according to an embodiment of the present disclosure is shown, including:

[0123] Step 401, inputting the breast image data to be classified into a target breast image classification model;

[0124] Step 402: determine that the output of the target breast image classification model is the classification result of the breast image data to be classified.

[0125] Obtain breast image data to be classified, determine a first breast image to be predicted and a second breast image to be predicted of the breast image data to be classified, extract features of the first breast image to be predicted and the second breast image to be predicted by an image feature extraction module in a target breast classification model, input the extracted features into a self-attention module in the target breast classification model, perform a first weight distribution, obtain a first feature vector to be predicted, segment the first feature vector to be predicted, obtain a first high-dimensional feature vector to be predicted corresponding to the first breast image to be predicted and a second high-dimensional feature vector to be predicted corresponding to the second breast image to be predicted, and divide the first high-dimensional feature vector to be predicted and the second high-dimensional feature vector to be predicted into a plurality of high-dimensional feature vectors. The feature vector is input into the cross-attention module in the target breast classification model for a second weight distribution to obtain the second feature vector to be predicted and the third feature vector to be predicted, the first high-dimensional feature vector to be predicted is subjected to maximum pooling processing to obtain the first feature to be predicted, the second high-dimensional feature vector to be predicted is subjected to linear projection processing to obtain the second feature to be predicted, the second feature vector to be predicted and the third feature vector to be predicted are subjected to average pooling processing to obtain the third feature to be predicted, and a fused feature to be predicted is obtained based on the first feature to be predicted, the second feature to be predicted and the third feature to be predicted, and the fused feature to be predicted is input into the fully connected layer of the target breast classification model to obtain the lesion type of the breast image data to be classified.

[0126] Figure 5 A module schematic diagram of a training device for a classification model of breast images according to an embodiment of the present disclosure is shown.

[0127] See also Figure 5 According to a third aspect of the embodiments of the present disclosure, a training device for a breast image classification model is provided, the device comprising:

[0128] An acquisition module 501 is used to acquire labeled breast image data and unlabeled breast image data as training sample data;

[0129] A training module 502, configured to train the breast image classification model based on the labeled breast image data;

[0130] A first determination module 503, used to determine a first loss value of the labeled breast image data;

[0131] The training module 502 is further used to classify and predict the unlabeled breast image data using the breast image classification model trained by the labeled breast image data to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label;

[0132] The first determination module 503 is further configured to determine the loss weight of the unlabeled breast image data according to the predicted probability of the unlabeled breast image data;

[0133] The first determination module 503 is further configured to determine a second loss value of the unlabeled breast image data according to the loss weight;

[0134] The first determination module 503 is further configured to determine a total loss value corresponding to a current training round of the breast image classification model according to the first loss value and the second loss value;

[0135] A judgment module 504 is used to determine whether the total loss value meets a preset requirement;

[0136] The judgment module is also used to determine the breast image classification model obtained in the current training round as the target breast image classification model if the preset requirements are met; if the preset requirements are not met, determine the unlabeled breast image data with pseudo labels obtained in the current training round as labeled breast image data, and return to the step of training the breast image classification model based on the labeled breast image data until the total loss value of the breast image classification model meets the preset requirements.

[0137] In one embodiment, the training module 502 includes: an acquisition submodule 5021, used to acquire the first breast image and the second breast image of the labeled breast image data; a feature extraction submodule 5022, used to extract features from the first breast image and the second breast image; a processing submodule 5023, used to input the obtained features into the self-attention module for the first weight distribution to obtain a first feature vector; a segmentation submodule 5024, used to segment the first feature vector to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image; the processing submodule 5023 is also used to input the first high-dimensional feature vector into the self-attention module for the first weight distribution to obtain a first feature vector; The high-dimensional feature vector and the second high-dimensional feature vector are input into the cross-attention module for a second weight distribution to obtain a second feature vector and a third feature vector; the processing submodule 5023 is also used to perform maximum pooling processing on the first high-dimensional feature vector to obtain a first feature, perform linear projection processing on the second high-dimensional feature vector to obtain a second feature, and perform average pooling processing on the second feature vector and the third feature vector to obtain a third feature; the fusion submodule 5025 is used to obtain a fused feature based on the first feature, the second feature and the third feature; the processing submodule 5023 is also used to input the fused feature into the fully connected layer to obtain the lesion type of the labeled breast image data.

[0138] In one embodiment, the first determination module 503 includes: a first determination submodule 5031, configured to determine the cross loss value of the labeled breast image data according to the following cross loss function: Among them, L ce represents the cross loss value of the labeled breast image data, N represents the number of labeled breast image data, K represents the number of categories of breast image data, i represents labeled breast image data, j represents the category of breast image data, y ij represents the true label of the i-th labeled breast image data in the j-th category, p ij Represents the predicted probability of the i-th labeled breast image data in the j-th category;

[0139] The first determination submodule 5031 is further configured to determine the multi-view loss value of the labeled breast image data according to the following multi-view loss function: Among them, L mv represents the multi-view loss value of the labeled breast image data, O f is the first high-dimensional feature vector, R f is the second high-dimensional feature vector, O a is the second eigenvector, R a is the third eigenvector;

[0140] The first determining submodule 5031 is further configured to determine the first loss value according to the cross loss value and the multi-view loss value.

[0141] In one possible implementation, the first determination module 503 further includes: a second determination submodule 5032, used to determine the mean and variance of the unlabeled breast image data based on the predicted probability of the unlabeled breast image; an updating submodule 5033, used to update the mean and variance through a smoothing algorithm; and a third determination submodule 5034, used to determine the loss weight of the unlabeled breast image data based on the updated mean and variance of the unlabeled breast image data.

[0142] In one possible implementation, the device further includes: a processing module 505, configured to perform a standardized alignment process on the predicted probability after obtaining the predicted probability of the unlabeled image data.

[0143] In one possible implementation, the processing module 505 is further used to perform image enhancement processing on the unlabeled breast image data after acquiring the labeled breast image data and the unlabeled breast image data as sample data; and to perform standardization processing on the unlabeled breast image data and the labeled breast image data after the image enhancement processing.

[0144] Figure 6 A module schematic diagram of a breast image classification device according to an embodiment of the present disclosure is shown.

[0145] See also Figure 6 According to the fourth aspect of the embodiments of the present disclosure, a breast image classification device is provided, the device comprising: an input module 601, used to input breast image data to be classified into the target breast image classification model; a second determination module 602, used to determine that the output of the target breast image classification model is the classification result of the breast image data to be classified.

[0146] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0147] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0148] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0149] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0150] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a training method for a breast image classification model. For example, in some embodiments, a training method for a breast image classification model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the training method for a breast image classification model described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute a training method for a breast image classification model in any other appropriate manner (eg, by means of firmware).

[0151] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0152] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0153] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0154] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0155] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0156] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0157] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0158] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0159] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A method for training a breast image classification model, characterized in that: The method comprises: Obtaining labeled breast image data and unlabeled breast image data as training sample data; Training the breast image classification model based on the labeled breast image data; Determining a first loss value for the labeled breast image data; Using the breast image classification model trained by the labeled breast image data to perform classification prediction on the unlabeled breast image data, to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label; Determining a loss weight of the unlabeled breast image data according to a predicted probability of the unlabeled breast image data; Determining a second loss value of the unlabeled breast image data according to the loss weight; Determine a total loss value corresponding to a current training round of the breast image classification model according to the first loss value and the second loss value; Determining whether the total loss value meets preset requirements; If the preset requirements are met, the breast image classification model obtained in the current training round is determined as the target breast image classification model; If the preset requirements are not met, the unlabeled breast image data with pseudo labels obtained in the current training round are determined as labeled breast image data, and the step of training the breast image classification model based on the labeled breast image data is returned until the total loss value of the breast image classification model meets the preset requirements; The training of the breast image classification model based on the labeled breast image data includes: obtaining a first breast image and a second breast image of the labeled breast image data; performing feature extraction on the first breast image and the second breast image; inputting the obtained features into a self-attention module for a first weight distribution to obtain a first feature vector; segmenting the first feature vector to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image; inputting the first high-dimensional feature vector and the second high-dimensional feature vector into a cross-attention module for a second weight distribution to obtain a second feature vector and a third feature vector; performing maximum pooling processing on the first high-dimensional feature vector to obtain a first feature, performing linear projection processing on the second high-dimensional feature vector to obtain a second feature, and performing average pooling processing on the second feature vector and the third feature vector to obtain a third feature; obtaining a fusion feature based on the first feature, the second feature and the third feature; and inputting the fusion feature into a fully connected layer to obtain the lesion type of the labeled breast image data.

2. The method according to claim 1, characterized in that Determining a first loss value of the labeled breast image data includes: The cross loss value of the labeled breast image data is determined according to the following cross loss function: Among them, L ce represents the cross loss value of the labeled breast image data, N represents the number of labeled breast image data, K represents the number of categories of breast image data, i represents labeled breast image data, j represents the category of breast image data, y ij represents the true label of the i-th labeled breast image data in the j-th category, p ij Represents the predicted probability of the i-th labeled breast image data in the j-th category; The multi-view loss value of the labeled breast image data is determined according to the following multi-view loss function: Among them, L mv represents the multi-view loss value of the labeled breast image data, O f is the first high-dimensional feature vector, R f is the second high-dimensional feature vector, O a is the second eigenvector, R a is the third eigenvector; The first loss value is determined according to the cross loss value and the multi-view loss value.

3. The method according to claim 1, characterized in that: The step of determining the loss weight of the unlabeled breast image data according to the predicted probability of the unlabeled breast image data comprises: Determining the mean and variance of the unlabeled breast image data according to the predicted probability of the unlabeled breast image; Updating the mean and variance by a smoothing algorithm; The loss weight of the unlabeled breast image data is determined according to the updated mean and variance of the unlabeled breast image data.

4. The method according to claim 1, characterized in that: After obtaining the predicted probability of the unlabeled image data, the method further includes: The predicted probabilities are subjected to a standardized alignment process.

5. The method according to claim 1, characterized in that After obtaining the labeled breast image data and the unlabeled breast image data as training sample data, the method further includes: Performing image enhancement processing on the unlabeled breast image data; The unlabeled breast image data and the labeled breast image data after the image enhancement processing are standardized.

6. A method for classifying breast images, characterized in that: Applied to any one of the target breast image classification models of claims 1-5, the method comprising: Inputting the breast image data to be classified into the target breast image classification model; Determine that the output of the target breast image classification model is the classification result of the breast image data to be classified.

7. A training device for a breast image classification model, characterized in that: The device comprises: An acquisition module, used for acquiring labeled breast image data and unlabeled breast image data as training sample data; A training module, used for training the breast image classification model based on the labeled breast image data; A first determination module, used to determine a first loss value of the labeled breast image data; The training module is further used to classify and predict the unlabeled breast image data using the breast image classification model trained by the labeled breast image data to obtain the prediction probability of the unlabeled breast image data and the corresponding pseudo label; The first determination module is further used to determine the loss weight of the unlabeled breast image data according to the predicted probability of the unlabeled breast image data; The first determination module is further used to determine a second loss value of the unlabeled breast image data according to the loss weight; The first determination module is further used to determine a total loss value corresponding to a current training round of the breast image classification model according to the first loss value and the second loss value; A judgment module, used to determine whether the total loss value meets the preset requirements; The judgment module is further configured to determine the breast image classification model obtained in the current training round as the target breast image classification model if the preset requirements are met; if the preset requirements are not met, determine the unlabeled breast image data with pseudo labels obtained in the current training round as labeled breast image data, and return to the step of training the breast image classification model based on the labeled breast image data until the total loss value of the breast image classification model meets the preset requirements; The training module includes: an acquisition submodule for acquiring a first breast image and a second breast image of the labeled breast image data; a feature extraction submodule for extracting features from the first breast image and the second breast image; a processing submodule for inputting the obtained features into a self-attention module for a first weight distribution to obtain a first feature vector; a segmentation submodule for segmenting the first feature vector to obtain a first high-dimensional feature vector corresponding to the first breast image and a second high-dimensional feature vector corresponding to the second breast image; the processing submodule is further used to input the first high-dimensional feature vector and the second high-dimensional feature vector into a cross-attention module for a second weight distribution to obtain a second feature vector and a third feature vector; the processing submodule is further used to perform maximum pooling processing on the first high-dimensional feature vector to obtain a first feature, perform linear projection processing on the second high-dimensional feature vector to obtain a second feature, and perform average pooling processing on the second feature vector and the third feature vector to obtain a third feature; a fusion submodule for obtaining a fused feature based on the first feature, the second feature and the third feature; the processing submodule is further used to input the fused feature into a fully connected layer to obtain the lesion type of the labeled breast image data.

8. A breast image classification device, characterized in that: Applied to any one of the target breast image classification models in claims 1-5, the device comprising: An input module, used for inputting the breast image data to be classified into the target breast image classification model; The second determination module is used to determine that the output of the target breast image classification model is the classification result of the breast image data to be classified.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

Images