Robust processing method for noisy labels in medical images based on dynamic weight learning network

Abstract

The present invention discloses a method for robustly processing medical image noise labels based on a dynamic weight learning network. The present invention obtains a medical image data set containing noise labels by first acquiring a number of medical images to be labeled and collecting their corresponding labels; then, preprocessing the medical images in the data set and acquiring their deep features and classification confidence; then, calculating the dynamic loss weight of the current t-th training according to the deep features and label information; finally, constructing an adaptive robust loss function based on the dynamic loss weight, training the model on the medical image data set containing noise labels, and obtaining a medical image classification model with robust noise labels. The present invention fuses deep features and label information through a dynamic weight learning network, adaptively learns the weights of loss functions on different samples, and dynamically balances the training advantages of traditional active loss functions and the robustness of symmetric passive loss functions to reduce the performance degradation caused by noise labels.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and in particular to a medical image noise label robust processing method based on a dynamic weight learning network. Background Art

[0002] Deep neural networks can be used for tasks such as detection, classification, and segmentation, and are widely used in medical image analysis tasks, but they rely on large-scale and high-quality labeled datasets. However, in practical applications, it is difficult to collect enough datasets with high-quality labels. The label quality of datasets obtained by directly mining imaging reports, clinical diagnoses, etc. from hospital information systems is usually low. The acquisition of high-quality labels often requires experienced medical experts to label, which is time-consuming and laborious. In addition, due to differences in experience and subjective judgment among different experts, the labeling results are prone to differences. Especially in the case of complex lesions, different annotators have different interpretations of images, which inevitably leads to the existence of "noise labels" in the dataset. Noise labels refer to the situation where the labeled labels of samples in the dataset are inconsistent with the true labels. They can be learned and fitted by the model during the training process of the deep learning model, affecting the prediction accuracy of the model on the test set and reducing the generalization ability of the model; especially in medical image analysis tasks, incorrect classification results may have a serious impact on clinical decision-making.

[0003] The method of selecting samples with correct labels to train the model can alleviate the impact of noisy labels on deep neural network models, but the selected training samples may still contain wrong labels, and directly discarding samples suspected of having wrong labels and not using them for training will result in a large loss of valuable sample information. Even if the suspected wrongly labeled samples are utilized through label correction to improve data utilization, noisy labels will be introduced. In addition, robust loss functions can alleviate the performance degradation caused by noisy labels. Symmetric loss functions are robust to noisy labels in multi-classification problems, but relying solely on symmetric loss to train the model usually makes it difficult for the model to converge, especially for complex data sets and deep models. In order to take into account the training advantages of fast convergence of traditional asymmetric active loss functions (such as cross entropy loss and focal loss) and the robustness of symmetric passive loss functions (such as mean absolute error and inverse cross entropy loss), many studies have attempted to merge these two types of loss functions. For example, methods such as generalized cross entropy GCE, symmetric cross entropy SCE, active passive loss APL and asymmetric loss function ALFs, but these methods rely on hyperparameters to balance the ratio of the two types of loss functions. However, the hyperparameters are sensitive to different data sets and noise ratios, and the weight coefficients need to be manually adjusted according to the training effect, which limits their applicability and stability in clinical practice applications. Summary of the invention

[0004] The purpose of the present invention is to design a medical image noise label robust processing method based on a dynamic weight learning network in order to solve the above problems. The dynamic weight learning network is combined with the current learning state and label information of the deep learning model to adaptively balance the training proportion of active loss and passive loss.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions:

[0006] The medical image noise label robust processing method based on dynamic weight learning network comprises the following steps:

[0007] S1. Obtain several medical images to be annotated, and collect the labels of each of the medical images to obtain a medical image dataset containing noisy labels. ,in represents the i-th medical image, Indicates its corresponding noise label;

[0008] S2. Preprocessing the medical images in the medical image dataset and obtaining the deep features and classification confidence of each of the medical images;

[0009] S3, calculating the dynamic loss weight of the current t-th training according to the depth features and noise labels;

[0010] S4. Construct an adaptive robust loss function based on the dynamic loss weight, train the model on the medical image dataset containing noise labels, and obtain a medical image classification model that is robust to noise labels.

[0011] The beneficial effects of the present invention are:

[0012] A robust processing method for medical image noise labels based on dynamic weight learning network obtains a number of medical images to be annotated and collects the labels of each medical image to obtain a medical image dataset with noise labels. ,in represents the i-th medical image, Represents its corresponding noise label; then, the medical images in the medical image dataset are preprocessed, and the deep features and classification confidence of each medical image are obtained; the dynamic loss weight of the current t-th training is calculated according to the deep features and noise labels; finally, an adaptive robust loss function based on the dynamic loss weight is constructed, and the model is trained on a medical image dataset containing noise labels to obtain a medical image classification model that is robust to noise labels, thereby adaptively learning the loss function weights on different samples, dynamically balancing the training advantages of the traditional active loss function's fast convergence and the robustness of the symmetric passive loss function, so as to reduce the performance degradation caused by noise labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a flow chart of the method for robustly processing medical image noise labels based on a dynamic weight learning network according to the present invention;

[0014] Figure 2 The overall block diagram of the dynamic weight learning network structure of the present invention;

[0015] Figure 3 The following are the effects of different proportions of cross entropy loss and inverse cross entropy loss under different noise ratios on dermoscopic images. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0017] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0019] In the description of the present invention, it should be understood that the terms "upper", "lower", "inside", "outside", "left", "right", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings, or are directions or positional relationships in which the product of the invention is usually placed when in use, or are directions or positional relationships commonly understood by those skilled in the art. These directions or positional relationships are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as a limitation on the present invention.

[0020] Furthermore, the terms “first”, “second”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0021] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms such as "setting" and "connection" should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0022] The present invention provides a method for robustly processing noisy labels in medical images based on a dynamic weight learning network, which aims to adaptively learn the weights of loss functions on different samples according to the learning state of a deep neural network model and the characteristics of a data set, and dynamically balance the training advantages of an active loss function and the robustness of a symmetric passive loss function to reduce the performance degradation caused by noisy labels. The specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0023] like Figure 1-3 As shown, the medical image noise label robust processing method based on dynamic weight learning network includes the following steps:

[0024] S1. Obtain several medical images to be annotated, and collect the labels of each of the medical images to obtain a medical image dataset containing noisy labels. ,in represents the i-th medical image, Indicates its corresponding noise label;

[0025] S2. Preprocessing the medical images in the medical image dataset and obtaining the deep features and classification confidence of each of the medical images;

[0026] S3, calculating the dynamic loss weight of the current t-th training according to the depth features and noise labels;

[0027] S4. Construct an adaptive robust loss function based on the dynamic loss weight, train the model on the medical image dataset containing noise labels, and obtain a medical image classification model that is robust to noise labels.

[0028] In one embodiment, the method for calculating the dynamic loss weight of the current t-th training according to the deep features and the noise labels in step S3 includes the following steps:

[0029] The deep features are extracted by a fully connected feature extraction module. Perform global average pooling to obtain the feature vector , and extract global features through the first fully connected layer , where BN stands for batch normalization, is the network weight of the first fully connected layer;

[0030] The noise label is fused by the label feature fusion module Perform category sparse coding to obtain label embedding vector , and the global features With the label embedding vector Concatenate to form input features , and then the fusion features are obtained through the second fully connected layer, batch normalization layer and Sigmoid activation function ,in is the network weight of the second fully connected layer;

[0031] The dynamic weight regression module is used to Calculating a dynamic weight of the medical image .

[0032] In one embodiment, the method for calculating the dynamic loss weight of the current t-th training according to the depth feature and the noise label in step S3 further includes the following steps:

[0033] Combine the current dynamic weight with the historical dynamic weight to generate the dynamic loss weight for the current t-th training. ,in, is the historical dynamic weight of the previous moment, It is a smoothing factor, which is used to adjust the ratio of the current dynamic weight to the historical dynamic weight.

[0034] In one embodiment, the method for constructing the adaptive robust loss function based on the dynamic loss weight in step S4 is specifically:

[0035] Calculate the active loss of each medical image according to the classification confidence and robust passive losses ;

[0036] Calculate an adaptive robust loss function based on the dynamic loss weight ,in It is a sample The classification confidence.

[0037] In one embodiment, the method for obtaining the depth feature and classification confidence of each medical image in step S2 is:

[0038] The deep features are extracted from the preprocessed medical image by a feature extractor to obtain the deep features. , where Aug represents the preprocessing process;

[0039] The classification confidence of the medical image in different categories is obtained by the classifier ;in, express The confidence level of the predicted label is k, where K represents the number of categories.

[0040] In this embodiment, the label collection method of medical images can be automatic collection or manual labeling; automatic collection is to automatically obtain labels based on the relevant fields in the imaging report, electronic medical record or diagnosis report of the medical image. This method can reduce the labeling cost, but it will increase the proportion of noise labels; manual labeling is a single labeling by a professional doctor or multi-level labeling by multiple experts and the final label is determined by voting. However, due to the differences in the professional level of doctors, when encountering medical images that are difficult to identify, the labeling results of different doctors may be inconsistent, which will also lead to a certain proportion of noise labels in the data set. Assume that the true label of the medical image is ,when When is a correctly labeled sample; otherwise, , then the sample Mislabeled samples.

[0041] In this embodiment, the feature extractor in step S2 can be implemented using deep neural network models such as ResNet, Inception, EfficientNet, Vision Transformer, etc. In this embodiment, the feature extractor uses the first 4 residual blocks of ResNet18 to extract the deep features of the medical image; the classifier uses the classification layer of ResNet18. It can be understood that the medical image is first preprocessed, that is, data augmentation is performed through operations such as random scaling, random horizontal flipping, random vertical flipping, random rotation and random cropping to increase the diversity of training data to enhance the model's robustness to changes in lesion features in terms of direction, size, position, etc.; and the medical image is standardized and normalized to reduce the impact of imaging differences caused by factors such as lighting and equipment during the acquisition process. Then, the medical image after data preprocessing is input into the feature extractor for deep feature extraction to obtain a sample. Deep Features , where Aug represents the data preprocessing process. Finally, the classification confidence of medical images in different categories is obtained through the classifier. ,in Representation sample The confidence level of the predicted label is k, where K represents the number of categories.

[0042] In this embodiment, the dynamic weight learning network is based on the deep features of the input samples. and mark tags Calculate the dynamic loss weight at the current tth training ,like Figure 2 As shown in the figure, the dynamic weight learning network structure includes a fully connected feature extraction module, a label feature fusion module and a dynamic weight regression module; it can be understood that the fully connected feature extraction module is used to extract the deep features obtained in step S2. Perform global average pooling to obtain the feature vector , and further extract global features through a single-layer fully connected neural network; specifically, the feature vector Mapped to the global feature space through the first fully connected layer, the output dimension is , followed by batch normalization and ReLU activation function, the extracted global features The specific calculation formula is as follows:

[0043] ,

[0044] In the formula, BN represents batch normalization, FullConnectL1 represents the first fully connected layer, is the network weight of the first fully connected layer. The label feature fusion module fuses the category information contained in the sample’s noise label into the global feature Specifically, firstly, the noise label Perform category sparse coding to obtain label embedding vector ; Then, the global features of the first layer Concatenate with the label embedding vector to form a new input feature ; Then, the feature size is obtained through a feature fuser including a second fully connected layer, a batch normalization layer, and a Sigmoid activation function. Fusion features , the specific calculation formula is as follows:

[0045] ,

[0046] Where FullConnectL2 represents the second fully connected layer, is the network weight of the second fully connected layer. The dynamic weight regression module calculates the dynamic weight based on the fusion features. Specifically, the fusion features are mapped to the dynamic weight space through the third fully connected layer, and the Softmax function (normalized exponential function) is used to normalize the result to between 0 and 1 to obtain the dynamic weight of the medical image. .

[0047] In order to enhance the stability of model training, this embodiment adopts the exponential moving average method to combine the current dynamic weight with the historical dynamic weight to generate the dynamic loss weight of the current training. , the specific calculation formula is as follows:

[0048] ,

[0049] In the formula, is the historical dynamic weight of the previous moment, is the dynamic weight at the current moment, Is a smoothing factor used to adjust the ratio of current weight to historical weight, usually , the smaller The value will make the model pay more attention to the current weights, and a larger A lower value will make the model more dependent on historical information.

[0050] In this embodiment, an adaptive robust loss function based on dynamic weights is constructed, and a medical image classification model with robust noise labels is obtained by training the model on a medical image dataset containing noise labels. It can be understood that, first, the active loss and robust symmetric passive loss of each image are calculated based on the classification confidence of each image obtained by the classifier; then, the dynamic loss weight of each image obtained in the dynamic weight learning network in step S3 and the adaptive robust loss function are combined to calculate its loss. The specific loss function calculation formula is as follows:

[0051] ,

[0052] In the formula is the sample obtained in step S3 The dynamic loss weight at time t, is the sample prediction probability obtained by the classifier in step S2; It is an active loss function, which can use cross entropy loss function, focal loss function, etc. It is a symmetric passive loss function, which can use mean absolute error, inverse cross entropy loss function, etc.

[0053] Since the prediction output of the classifier uses the Softmax function, the sum of the predicted values ​​is 1, that is, ; Since the symmetric passive loss function must satisfy, for any sample , the sum of its losses is a constant, namely:

[0054] ,

[0055] Where C is a constant, such as the mean absolute error MAE exists:

[0056] ,

[0057] in is the probability of predicting class k, is the probability that the sample label is c, that is, when k=c, =1, otherwise hour, = 0. Therefore, the ratio of noise labels is Dataset Any sample in , the true label is When constructing a medical image classification model Loss Expectation exist:

[0058]

[0059] in is the expected loss without noise labels, i.e. , it can be deduced that for the constructed medical image classification model , symmetric passive loss function exist:

[0060] ,

[0061] It can be shown that in the presence of noisy labels, minimizing the symmetric passive loss function What you get It is also the minimum value under the noise-free label. Therefore, the symmetric passive loss function is a robust loss function. Furthermore, the adaptive robust function It combines the training advantages of active loss functions and the robustness of symmetric passive loss functions to noisy labels.

[0062] Finally, the constructed adaptive robust loss function is used to calculate the loss value in each medical image, and the constructed medical image classification model is trained, such as Figure 1 As shown in the figure, the specific training process includes: first input the training data with noise labels into the classification model, and generate a set of deep features through the feature extractor; then pass the extracted deep features through the classifier to obtain the classification results, and pass them through the dynamic weight learning network with the encoded noise labels to obtain the dynamic loss weights; then, use the prediction results of the classifier and the dynamic loss weights to calculate the loss value based on the constructed adaptive robust loss function, and train the model through back propagation. Among them, since the purpose of the dynamic weight learning network is to adaptively learn the loss weights according to the learning state of the network and the marked labels, the reverse gradient of the dynamic weight learning network is not back-propagated to the backbone feature extractor, and the feature extractor and classifier are directly used to classify the input medical images in the test phase.

[0063] In order to verify the effectiveness of the robust processing method for medical image noise labels based on the dynamic weight learning network of the present invention, an experimental verification was carried out on the public dataset DermaMNIST, which consists of 10015 dermoscopic images divided into 7 categories. Noise labels were added to the original training set by randomly flipping the labels to other categories according to the noise ratio; the first 4 residual blocks of ResNet18 were used as feature extractors in the experiment to extract the deep features of medical images; the classification layer of ResNet18 was used as a classifier; the first fully connected layer of the dynamic weight learning network had an input dimension of 512 and an output dimension of 256, the second fully connected layer had an input dimension of 263 and an output dimension of 64, the third fully connected layer had an input dimension of 64 and an output dimension of 1, and the smoothing factor of the historical weight and the current weight is 0.9; the active loss function uses the cross entropy loss function, the passive loss function uses the inverse cross entropy loss, the training uses the Adam optimizer, the learning rate is 0.01, the momentum is set to 0.9, 128 samples are trained in each batch, and the training is performed 100 times on the entire dataset.

[0064] The dynamic weight loss function proposed in the present invention is compared with the traditional cross entropy loss function, the robust inverse cross entropy loss function, and the symmetric loss function method combining cross entropy loss and inverse cross entropy loss, and the average accuracy of the last 10 times on the test set and the area under the receiver operating characteristic curve are used to evaluate the model performance. These two indicators reflect the overall classification accuracy and the model's ability to distinguish different categories. The results are shown in Table 1. In the absence of noise, the cross entropy loss performs best. During the training process, the cross entropy loss will strengthen the learning of samples whose predicted labels are inconsistent with the marked labels, ensuring that the model can learn all samples in the training set, thereby achieving the best test accuracy in a noise-free environment; however, in the case of a large number of noise labels, since the noise labels deviate from the true distribution, the cross entropy loss will learn the wrong labels as true labels, resulting in the problem of overfitting noise in the model, thereby reducing the robustness of the model.

[0065] Table 1 is a table showing the experimental results of the comparison between the medical image noise label robust processing method based on dynamic weight learning network and the traditional method:

[0066]

[0067] Since the inverse cross entropy loss pays equal attention to all prediction results, it effectively avoids the model from over-learning the wrong labels and is more robust. However, the same accuracy under different noise ratios indicates that its training has not converged. The symmetric cross entropy loss function that combines cross entropy and inverse cross entropy losses neutralizes the advantages and disadvantages of both and achieves better results overall, but the proportion of the two needs to be adjusted for different data sets and noise ratios, which lacks flexibility. Figure 3 As shown, different ratios of cross entropy and inverse cross entropy loss combinations are suitable for different noise ratios. The symmetric cross entropy in Table 1 uses the (6.0, 0.1) ratio with better overall results as the final result. Compared with the symmetric cross entropy loss function, the present invention has achieved better results and shows better adaptability when the noise ratio changes. It can be seen that in the case of noisy labels, the medical image classification model based on the adaptive robust loss function proposed in the present invention can effectively suppress the interference of noise labels on the model, adaptively suppress the influence of noise labels on classification performance, significantly improve the classification accuracy and discrimination ability, and enhance the practical application ability in medical image classification tasks.

[0068] Compared with the traditional fixed-weight loss function design, the present invention achieves better classification performance in a noisy label environment by dynamically adjusting the ratio of active loss to passive loss; the dynamic loss weight adjustment mechanism of the present invention introduces a dynamic weight learning network into the network, and adaptively calculates the weights of active loss and passive loss according to the current learning state of the model and the noise characteristics of the data, thereby optimizing the loss combination in real time in different noise environments; the dynamic weight learning network based on sample labels and deep features of the present invention designs a multi-layer fully connected network to extract the deep features of input samples and fuse the sample label features to generate dynamic weights; and introduces historical weight information to improve the stability of training, so that the weights of different loss functions can be adaptively adjusted according to the noise characteristics of the samples and the current state of the model, providing a loss that is more noise-robust than fixed weights.

[0069] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for robustly processing medical image noise labels based on a dynamic weight learning network, characterized in that: The medical image noise label robust processing method based on dynamic weight learning network comprises the following steps: S1. Obtain several medical images to be annotated, and collect the labels of each of the medical images to obtain a medical image dataset containing noisy labels. , where x i represents the i-th medical image, Indicates its corresponding noise label; S2. Preprocessing the medical images in the medical image dataset and obtaining the deep features and classification confidence of each of the medical images; S3, calculating the dynamic loss weight of the current t-th training according to the depth features and noise labels; S4, constructing an adaptive robust loss function based on the dynamic loss weight, training a model on the medical image data set containing noise labels, and obtaining a medical image classification model that is robust to noise labels; The method for constructing the adaptive robust loss function based on the dynamic loss weight in step S4 is specifically: Calculate the active loss of each medical image according to the classification confidence and robust passive losses ; Calculate an adaptive robust loss function based on the dynamic loss weight ,in, is the dynamic loss weight of the current t-th training, is the active loss function, is the passive loss function, p i is the sample x i The classification confidence.

2. The method for robustly processing medical image noise labels based on dynamic weight learning network according to claim 1 is characterized in that: The method for calculating the dynamic loss weight of the current t-th training according to the depth feature and the noise label in step S3 comprises the following steps: The deep features are extracted by a fully connected feature extraction module. Perform global average pooling to obtain the feature vector , and extract global features through the first fully connected layer , where BN stands for batch normalization, is the network weight of the first fully connected layer; The noise label is fused by the label feature fusion module Perform category sparse coding to obtain label embedding vector , and the global features With the label embedding vector Concatenate to form input features , and then the fusion features are obtained through the second fully connected layer, batch normalization layer and Sigmoid activation function ,in is the network weight of the second fully connected layer; The dynamic weight regression module is used to Calculating a dynamic weight of the medical image .

3. The method for robustly processing medical image noise labels based on dynamic weight learning network according to claim 2 is characterized in that: The method for calculating the dynamic loss weight of the current t-th training according to the depth feature and the noise label in step S3 also includes the following steps: Combine the current dynamic weight with the historical dynamic weight to generate the dynamic loss weight for the current t-th training. ,in, is the historical dynamic weight of the previous moment, It is a smoothing factor, which is used to adjust the ratio of the current dynamic weight to the historical dynamic weight.

4. The method for robustly processing medical image noise labels based on dynamic weight learning network according to claim 1, characterized in that: The method for obtaining the depth feature and classification confidence of each medical image in step S2 is: The deep features are extracted from the preprocessed medical image by a feature extractor to obtain the deep features. , where Aug represents the preprocessing process; The classification confidence of the medical image in different categories is obtained by the classifier ;in, Represents x i The confidence level of the predicted label is k, where K represents the number of categories.

Images