Single-positive multi-label chest x-ray image classification method based on generalized nesterov iteration fast gradient sign method and pseudo-label consistency regularization

Abstract

The application discloses to the medical image classification technical field, specifically for single positive multi-label chest X-ray image classification method based on generalized Nesterov iteration fast gradient sign method and pseudo label consistency regularization, including specific steps as follows: single positive multi-label chest X-ray image is input into submodel g' and uses generalized Nesterov iteration fast gradient sign method, and cumulative gradient and optimized perturbation step length are used to generate adversarial samples as strong enhanced images;Single positive multi-label chest X-ray image is preprocessed by random horizontal flip and input into main model g as weak enhanced image, wherein high-level features of the image are obtained in the backbone network, and then the model prediction of weak enhanced image is obtained according to the features using classification head, the pseudo label generated by weak enhanced image is used to supervise the strong enhanced version of the same image, which is beneficial to reduce the influence of false negative label problem and enhance the generalization performance of the model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image classification, in particular to a single-positive multi-label chest X-ray image classification method based on generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization. BACKGROUND

[0002] Deep learning methods for multi-label chest X-ray image classification often require large-scale datasets, however, obtaining fully labeled datasets is expensive and time-consuming, and errors may be introduced during the labeling process. Consider an extreme case where an annotator only provides one relevant label for each image, i.e., only one positive label is observed, and other class labels are unknown. Therefore, each image in the training set will have only one positive label, and there is no explicit negative label, which is called single-positive multi-label chest X-ray image classification. Therefore, how to maintain the accuracy of model classification under the condition of limited dataset labeling is a challenging problem.

[0003] There are three traditional methods to implement the single-positive multi-label chest X-ray image classification task. The first method simply sets the loss term corresponding to the unobserved label to 0 to adjust the model parameters according to the loss value. However, this loss implicitly assumes that the unobserved label is correctly predicted, and the error of the calculated loss value is large. The second method assumes that the unobserved label is negative, which is the most common method in single-positive label classification tasks. Since in the chest X-ray image classification task, the number of negative labels contained in the image is usually more than the number of positive labels, assuming that the unobserved label is negative is consistent with reality. However, this method introduces some false negative labels, i.e., the originally positive labels are incorrectly assumed to be negative labels and participate in the calculation of the loss function. The third method further reduces the weight of the loss term corresponding to the negative label based on the assumption that the unobserved label is negative. This method reduces the impact of false negative labels, but the impact of false negative labels is still not reasonably addressed. Therefore, the present application proposes a single-positive multi-label chest X-ray image classification method based on generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization. SUMMARY

[0004] In view of the above and / or existing problems in the single-positive multi-label chest X-ray image classification method based on generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization, the present application is proposed.

[0005] Therefore, the purpose of the present application is to provide a single-positive multi-label chest X-ray image classification method based on generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization, by introducing the idea of generating adversarial samples into the chest X-ray image classification task, and further optimizing the Nesterov Accelerated Gradient algorithm. The method first inputs the single-positive multi-label chest X-ray image into the sub-model to generate adversarial samples as strong enhanced images using the optimized generalized Nesterov iterative fast gradient sign method. Then, the original image is preprocessed by random horizontal flipping to obtain a weak enhanced image, and the pseudo-label threshold τ is used to generate pseudo-labels by combining the prediction value of the weak enhanced image and the single-positive label of the image. Then, the strong enhanced image is input into the main model and the pseudo-label is used to supervise the model to predict the strong enhanced version of the same image to realize the pseudo-label consistency regularization of the image-level perturbation. Finally, the model is trained using the regularization to realize the single-positive multi-label chest X-ray image classification, which can solve the above-mentioned existing problems.

[0006] To solve the above technical problems, according to one aspect of the present application, the present application provides the following technical solutions:

[0007] The single-positive multi-label chest X-ray image classification method based on generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization includes the following specific steps:

[0008] S1: inputting the single-positive multi-label chest X-ray image into the sub-model g' using the generalized Nesterov iterative fast gradient sign method to generate adversarial samples as strong enhanced images using the cumulative gradient and the optimized perturbation step;

[0009] S2: preprocessing the single-positive multi-label chest X-ray image by random horizontal flipping as a weak enhanced image and inputting it into the main model g, wherein the high-level features of the image are obtained in the backbone network, and then the model prediction of the weak enhanced image is obtained using the classification head according to these features;

[0010] S3: combining the pseudo-label threshold to screen the model prediction of the weak enhanced image, and selecting the label with high model prediction confidence as the pseudo-label of the image;

[0011] S4: performing consistency regularization calculation on the obtained pseudo-label and the model prediction of the strong enhanced image, training the main network by minimizing the error between the pseudo-label and the prediction of the strong enhanced image, and using the trained main network to realize accurate classification of the single-positive multi-label chest X-ray image.

[0012] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization of the application, wherein the specific process of S1 is as follows:

[0013] S11: input the single-positive multi-label chest X-ray image into the sub-model g', add perturbation to the image according to the hyperparameter perturbation step size α and the gradient attenuation factor μ to obtain a forward-looking perturbed image, and then pass through the backbone network and the classification head in turn to obtain the prediction value of the model for the image; for the setting of the iteration step size parameter, the Nesterov Accelerated Gradient algorithm is combined to analyze and study the influence of the optimization parameter on the convergence, and the generalized Nesterov Accelerated Gradient algorithm is further calculated, and the formula is as follows:

[0014]

[0015] Where T is a transformation operator used to update or adjust the direction of the solution to the expected target, θ k is a momentum parameter, t k is the iteration number;

[0016] The generalized Nesterov Accelerated Gradient algorithm is combined to realize the generalized Nesterov iterative fast gradient sign method, and the perturbation step size α of the sub-model is set as follows:

[0017]

[0018] Where t is the current iteration number;

[0019] The generation formula of the forward-looking perturbed image is as follows:

[0020]

[0021] Where is the forward-looking perturbed image obtained in this round of iteration, is the perturbed image obtained in the last round of iteration, α is the perturbation step size, and μ is the gradient attenuation factor;

[0022] S12: use the prediction value of the forward-looking perturbed image and the corresponding single-positive label to calculate the cross-entropy loss loss, and obtain the gradient g t relative to the image;

[0023] S13: calculate and update the cumulative gradient g t+1 according to the gradient g t calculated in this round of iteration and the attenuation factor μ, and the cumulative gradient update formula is as follows:

[0024]

[0025] wherein g t is the cumulative gradient at iteration t rounds, g0=0, μ is the decay factor, and J() is the cross-entropy loss function;

[0026] The constraint of using the direction of the cumulative gradient g t+1 to perturb the input image is used to increase the loss, and the perturbed image obtained by adding the constraint is used as the perturbed image of this round of iteration, and the generation formula of the perturbed image is as follows:

[0027]

[0028] wherein is a pruning function used to constrain the pixel value of the generated adversarial sample;

[0029] The perturbed image is input into the sub-model as the input image for the next round of iteration, and according to the condition of the loop, the final adversarial sample is generated after multiple rounds of iteration.

[0030] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization according to the application, wherein: the backbone network of the sub-model comprises a primary convolutional layer with a convolution kernel size of 7x7 and 8 residual block groups, and each residual block group comprises two convolutional layers with a convolution kernel size of 3x3.

[0031] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization according to the application, wherein: the classification head is composed of a global average pooling layer and a fully connected layer.

[0032] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization according to the application, wherein: the main model network structure in S2 has a total of 50 layers from input to output, including an initial convolutional layer composed of a 7x7 convolutional layer and a max-pooling layer, and 4 groups of residual blocks.

[0033] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization according to the application, wherein: the number of residual blocks in each group is 3, 4, 6, and 3, respectively, and each residual block is composed of 3 convolutional layers.

[0034] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iteration fast gradient sign method and pseudo-label consistency regularization, in the S3, the model prediction result of the weak enhanced image is combined with the hyperparameter pseudo-label threshold τ to screen the model prediction, and the class with a prediction value greater than τ is set as positive, and the class with a prediction value less than 1-τ is set as negative, that is, the label with high model prediction confidence is screened as a pseudo-label.

[0035] As a preferred scheme of the single-positive multi-label chest X-ray image classification method based on the generalized Nesterov iteration fast gradient sign method and pseudo-label consistency regularization, in the S4, the adversarial sample generated in the S1 is input into the main model g, and the prediction value of the model for the strong enhanced image is obtained by sequentially passing through the backbone network and the classification head, and the pseudo-label obtained in the S3 is combined to calculate the regularization loss, and the regularization loss formula is:

[0036]

[0037] Wherein is the value of the lth class in the pseudo-label of the ith weak enhanced image, is the value of the lth class in the model prediction of the ith strong enhanced image, and τ is the pseudo-label threshold; according to the regularization loss formula, the main model is trained to minimize the loss to realize accurate classification of single-positive multi-label chest X-ray images.

[0038] Compared with the prior art:

[0039] The adversarial sample is used as the strong enhanced image, by adding a small and accurate disturbance, the model can explore and learn deeper and more abstract features in the training process, which is beneficial to the model to form a smoother and more stable decision boundary, and can more effectively generalize and correctly classify when processing new data with fuzzy boundaries between classes. In addition, the generalized Nesterov Accelerated Gradient algorithm is used in the process of generating adversarial samples to stabilize the update direction of the disturbance, and the prospective idea is used to correct the previously accumulated gradient using the expected update, which is beneficial to more easily and quickly escape from the poor local extremum in the loss function, thereby improving the quality of the adversarial sample. Secondly, the pseudo-label is generated by randomly horizontally flipping the preprocessed weak enhanced image for subsequent supervision. This method effectively utilizes unlabeled data, reduces labeling cost, fully recovers potential positive labels, and improves the effect of model training. Finally, the pseudo-label consistency regularization uses the pseudo-label generated by the weak enhanced image to supervise the strong enhanced version of the same image, which is beneficial to reduce the influence of false negative label problems and enhance the generalization performance of the model. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The schematic diagram of the whole model framework of the application is shown in the figure;

[0041] Figure 2 The flow chart of the application is shown in the figure;

[0042] Figure 3 The architecture diagram of the sub-model network of the application is shown in the figure;

[0043] Figure 4 The structure diagram of the main model network of the application is shown in the figure. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical scheme and advantages of the application more clear, the embodiments of the application will be further described in detail below with reference to the drawings.

[0045] The application provides a single-positive multi-label chest X-ray image classification method based on generalized Nesterov iterative fast gradient sign method and pseudo-label consistency regularization, please refer to Figures 1-4 ;

[0046] The specific steps include the following:

[0047] S1: input the single-positive multi-label chest X-ray image into the sub-model g' to use the generalized Nesterov iterative fast gradient sign method to generate an adversarial sample as a strong enhanced image by using the accumulated gradient and the optimized perturbation step length, wherein the single-positive multi-label chest X-ray image refers to an image with 14 categories in total, but only one category is labeled as positive, and the rest categories are unknown;

[0048] The specific process of S1 is as follows:

[0049] S11: input the single-positive multi-label chest X-ray image into the sub-model g', add perturbation to the image according to the hyperparameter perturbation step length a and the gradient attenuation factor m to obtain a forward-looking perturbation image, and then pass through the backbone network and the classification head in sequence to obtain the prediction value of the model on the image; for the setting of the iteration step length parameter, the Nesterov Accelerated Gradient algorithm is combined to analyze and study the influence of the optimization parameter on the convergence condition, and the generalized Nesterov Accelerated Gradient algorithm is further calculated, and the formula is as follows:

[0050]

[0051] Wherein T is a transformation operator used to update or adjust the direction of the solution to the expected target, and k is a momentum parameter, t k is the number of iterations.

[0052] The generalized Nesterov iteration fast gradient sign method is realized in combination with the generalized Nesterov accelerated gradient algorithm, and the disturbance step length a of the sub-model is set as follows:

[0053]

[0054] where t is the current iteration number;

[0055] The generation formula of the forward disturbance image is as follows:

[0056]

[0057] where is the forward disturbance image obtained in the current iteration, is the disturbance image obtained in the last iteration, a is the disturbance step length, and μ is the gradient attenuation factor;

[0058] S12: The cross-entropy loss loss of the prediction value of the forward disturbance image and the corresponding single positive label is calculated, and the gradient g t of the image is calculated.

[0059] S13: The gradient g t calculated in the current iteration and the attenuation factor μ are used to calculate and update the cumulative gradient g t+1 , and the cumulative gradient update formula is as follows:

[0060]

[0061] where g t is the cumulative gradient in the iteration t, g0=0, μ is the attenuation factor, and J() is the cross-entropy loss function.

[0062] The input image is disturbed in the direction of the cumulative gradient g t+1 for the purpose of increasing the loss, and the disturbance image in the current iteration is obtained by constraint and addition, and the generation formula of the disturbance image is as follows:

[0063]

[0064] where is a clipping function used to constrain the pixel value of the generated adversarial sample;

[0065] The disturbance image is input into the sub-model as the input image for the next iteration, and according to the condition of the loop, the final adversarial sample is generated after multiple iterations;

[0066] Wherein: the backbone network of the sub-model comprises a primary convolutional layer with a convolution kernel size of 7x7 and 8 residual block groups, each residual block group comprising two convolutional layers with a convolution kernel size of 3x3; the classification head is composed of a global average pooling layer and a fully connected layer;

[0067] S2: inputting the single-positive multi-label chest X-ray image after random horizontal flip preprocessing as a weakly augmented image into the main model g, wherein high-level features of the image are obtained in the backbone network, and then the model prediction of the weakly augmented image is obtained according to the features using the classification head; wherein the main model network structure has 50 layers from input to output, including an initial convolutional layer composed of a 7x7 convolutional layer and a max-pooling layer, and 4 groups of residual blocks, the number of each group of residual blocks being 3, 4, 6, and 3 respectively, and each residual block being composed of 3 convolutional layers;

[0068] S3: screening the model prediction of the weakly augmented image in combination with the hyperparameter pseudo-label threshold, and selecting the label with high model prediction confidence as the pseudo-label of the image; wherein for the model prediction result of the weakly augmented image, the model prediction is screened in combination with the hyperparameter pseudo-label threshold τ, the class with a prediction value greater than τ is set as positive, and the class with a prediction value lower than 1-τ is set as negative, that is, the label with high model prediction confidence is selected as the pseudo-label;

[0069] S4: performing consistency regularization calculation on the obtained pseudo-label and the model prediction of the strongly augmented image, training the main network by minimizing the error between the pseudo-label and the prediction of the strongly augmented image, and using the trained main network to realize accurate classification of the single-positive multi-label chest X-ray image;

[0070] Wherein, the adversarial sample generated in S1 is input into the main model g, and the prediction value of the main model for the strongly augmented image is obtained by sequentially passing through the backbone network and the classification head, and regularization loss calculation is performed in combination with the pseudo-label obtained in S3, and the regularization loss formula is:

[0071]

[0072] Wherein is the value of the lth class in the pseudo-label of the ith weakly augmented image, is the value of the lth class in the model prediction of the ith strongly augmented image, and τ is the pseudo-label threshold; according to the regularization loss formula, the main model is trained to minimize the loss to realize accurate classification of the single-positive multi-label chest X-ray image.

[0073] The experimental data set used by the present application is the CheXpert data set, which is a large data set for chest radiograph diagnosis, consisting of 224316 multi-view chest radiographs of 65240 patients, which are labeled as positive, negative or unannotated 14 pathologies. The size of all input CXR images is 224x224, the data set is divided into training set, validation set and test set in the ratio of 7:2:1, and the image batch size is 64 during model training. The image label uses single positive label, that is, each image only contains one correct positive class, and the remaining classes are marked as unknown.

[0074] In addition, the entire algorithm of the present application is tested in a CentOS7(Core)(64-bit) system, wherein the cpu is Intel Xeon Silver 4210R(2.4GHz), and the graphics card model is GeForce RTX 3090(24GB memory). In programming, the Pytorch 2.1.2 platform is used to build the network model.

[0075] Although the present application has been described with reference to the embodiments above, various improvements can be made thereto and equivalents can be substituted for elements thereof without departing from the scope of the present application. In particular, features of the disclosed embodiments of the present application can be combined together in any manner as long as there is no structural conflict, and the combinations are not exhaustively described in the specification only for the purpose of omitting the length and saving resources. Therefore, the present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A single-positive multi-label chest X-ray image classification method based on generalized Nesterov iteration fast gradient sign method and pseudo-label consistency regularization, characterized in that, The specific steps include: S1: input the single-positive multi-label chest X-ray image into the sub-model g' to use the generalized Nesterov accelerated gradient sign method to generate an adversarial sample as a strong enhanced image by using accumulated gradient and an optimized perturbation step length, characterized in that, S11: input the single-positive multi-label chest X-ray image into the sub-model g', add perturbation to the image according to the hyperparameter perturbation step length α and the gradient decay factor μ to obtain a forward perturbation image, and then sequentially pass through the backbone network and the classification head to obtain the prediction value of the model for the image; for the setting of the iteration step length parameter, the Nesterov Accelerated Gradient algorithm is combined to analyze and study the influence of the optimization parameter on the convergence, and the algorithm formula is as follows: with t k-1 = a(k - 1) ω + b, ω = 1, b = 5 where T is a transformation operator to update or adjust the direction of the solution towards the intended target, θ k is a momentum parameter, t k is the number of iterations; The generalized Nesterov accelerated gradient sign method is realized by combining the generalized Nesterov Accelerated Gradient algorithm, and the perturbation step length α of the sub-model is set as follows: Where t is the current iteration number; The generation formula of the forward perturbation image is as follows: wherein is the forward-looking perturbed image obtained for the present iteration, is the perturbed image obtained for the previous iteration, a is the perturbation step, and μ is the gradient decay factor. S2: the single-positive multi-label chest X-ray image is preprocessed by random horizontal flip as a weak enhanced image and input into the main model g, wherein the high-level features of the image are obtained in the backbone network, and then the model prediction of the weak enhanced image is obtained according to these features using the classification head; S3: the model prediction of the weak enhanced image is screened according to the hyperparameter pseudo-label threshold, and the label with high model prediction confidence is selected as the pseudo-label of the image; S4: the pseudo-label obtained is calculated for consistency regularization with the model prediction of the strong enhanced image, the main network is trained by minimizing the error between the pseudo-label and the prediction of the strong enhanced image, and the trained main network is used to realize accurate classification of the single-positive multi-label chest X-ray image.

2. The single-positive multi-label chest X-ray image classification method based on the generalized Nesterov accelerated gradient sign method and pseudo-label consistency regularization according to claim 1, characterized in that, S12: Calculate the cross-entropy loss loss using the predicted value of the forward-perturbed image and the corresponding single-positive label, and find its gradient g with respect to the image t ; S13: Gradient g calculated according to the current iteration t With the decay factor μ, the cumulative gradient g is calculated and updated t+1 The cumulative gradient update formula is as follows: where g t is the accumulated gradient over t iterations, g0= 0, μ is a decay factor, and J() is the cross-entropy loss function. Using accumulated gradient g for the purpose of increasing loss t+1 The constraint of perturbing the input image in the direction of the accumulated gradient g and adding the perturbed image obtained in this round of iteration, the generation formula of the perturbed image is as follows: wherein is a clipping function for constraining pixel values of the generated adversarial samples; The perturbed image is input into the sub-model as the input image for the next round of iteration, and the final adversarial sample is generated after multiple rounds of iteration according to the conditions of the loop.

3. The method according to claim 2, wherein, The backbone network of the sub-model includes a primary convolutional layer with a convolution kernel size of 7x7 and 8 residual blocks, and each residual block includes two convolutional layers with a convolution kernel size of 3x3.

4. The method of claim 2, wherein the method is a single-positive multi-label chest X-ray image classification method based on generalized Nesterov acceleration fast gradient sign method and pseudo-label consistency regularization, characterized in that, The classification head is composed of a global average pooling layer and a fully connected layer.

5. The method according to claim 1, wherein, The main model network structure in S2 has a total of 50 layers from input to output, including an initial convolutional layer composed of a 7x7 convolutional layer and a max pooling layer, and 4 groups of residual blocks.

6. The method according to claim 5, wherein, The number of residual blocks in each group is 3, 4, 6, and 3, respectively, and each residual block is composed of 3 convolutional layers.

7. The method according to claim 1, wherein, In S3, the model prediction of the weak enhanced image is screened according to the hyperparameter pseudo-label threshold τ, and the classes with prediction values greater than τ are set as positive, and the classes with prediction values less than 1-τ are set as negative, that is, the labels with high model prediction confidence are selected as pseudo-labels.

8. The method according to claim 1, wherein the method is characterized in that, The S4 inputs the adversarial sample generated by the S1 into the main model g, sequentially passes through the backbone network and the classification head to obtain a prediction value of the model on the strongly enhanced image, and combines the pseudo label obtained by the S3 to calculate a regularization loss, and the regularization loss formula is: wherein is the value of the i-th weakly augmented image for the l-th class of the pseudo label, is the value of the i-th strongly augmented image for the l-th class of the model prediction, and τ is the pseudo label threshold; according to the regularization loss formula, the main model is trained to minimize the loss to achieve accurate classification for single-positive multi-label chest X-ray images.

Images