A semi-supervised medical image classification method based on a secure contrastive self-ensemble framework

Abstract

The present invention provides a semi-supervised medical image classification method based on a secure contrast self-integration framework, comprising the following steps: first, a batch of data consisting of labeled and unlabeled data is selected to add data perturbation to it, and the process is repeated twice to obtain two groups of data, which are respectively input into the student and teacher networks; then, a weight function is designed and the weight function is updated using supervised loss to automatically assign weights to each unlabeled data; a reliable perceived consistency loss is established by combining the weight parameters and the probability output of the two networks; further, a reliable perceived contrast loss is established by combining the weight parameters and using a projection network to obtain a normalized low-dimensional representation of the output of the two networks; finally, all loss functions are weighted and summed to form a final loss function, and network parameters and weight function parameters are updated alternately. The method proposed by the present invention can simultaneously learn reliable information at the data level and the data structure level, thereby improving the robustness and generalization of the model.

Description

Technical Field

[0001] The invention relates to a semi-supervised medical image classification method based on a secure contrast self-integrated framework, and belongs to the field of medical image classification. Background Art

[0002] In recent years, deep learning has made great breakthroughs in the field of clinical medicine and has been widely used in pathology, medical imaging and diagnosis, bioinformatics, etc. In general, deep learning requires a large amount of labeled data for network learning. Since the collection of a large amount of high-quality labeled medical data is time-consuming and labor-intensive and requires professional medical knowledge, the demonstration application of deep learning methods in clinical practice is hindered. Deep semi-supervised learning methods can use a large amount of unlabeled medical data to assist in the establishment of deep networks. Among them, self-ensemble models have been shown to achieve good semi-supervised medical image classification performance. However, current self-ensemble models all ignore the reliability of unlabeled medical images. There are inevitably differences between unlabeled medical images obtained from different populations, equipment, and environments. Directly using unlabeled medical images and assigning the same weight to all unlabeled data may have a negative impact on the classification performance of the self-ensemble method. Existing methods are always limited in terms of security and generalization in practical applications.

[0003] In response to the above problems, the present invention proposes a security-aware contrastive self-integration framework. First, the weight function is introduced into the average teacher model, which can map the probability prediction of unlabeled medical data into a weight that reflects its reliability, thereby safely using the probability prediction of unlabeled data under different perturbations to build a reliable-aware consistency loss for the network model. In order to utilize the information at the data structure level of medical images, the present invention further designs a reliable-aware contrastive loss to encourage reliable unlabeled data to have better intra-class compactness and inter-class separability. The constructed network framework can simultaneously capture reliable data-level and data structure-level information, thereby improving the robustness and generalization ability of the model. Summary of the invention

[0004] The present invention provides a semi-supervised medical image classification method based on a secure contrast self-integrated framework. On the basis of an average teacher model, a densely connected convolutional network is used as the backbone network of the average teacher model, and a weight function is introduced to map the probability prediction of each unlabeled data into a unique weight that can measure its reliability. The weights of all unlabeled data are combined to establish a reliable perceived contrast loss and a reliable perceived consistency loss, and together with the cross entropy loss function, a final loss function is formed, and the network parameters and the weight function parameters are updated through alternating iterations.

[0005] The present invention adopts the following technical solutions to solve the above problems:

[0006] A semi-supervised medical image classification method based on a secure contrast self-ensemble framework comprises the following steps:

[0007] Step 1: Divide the medical image training data into labeled data With unlabeled data Among them, x i represents the i-th medical image sample, y i Represents the true label of the i-th sample;

[0008] Step 2: Add data perturbation by performing random rotation and affine transformation on a batch of training data consisting of unlabeled samples and labeled samples. After repeating the process twice, two sets of input data with added perturbation are obtained. The two sets of data are input into the student network and teacher network of the average teacher model respectively.

[0009] Step 3: Use the probability output obtained by the softmax function of the output of the student network in step 2 and the true label to calculate the supervised loss;

[0010] Step 4: Design a weight function and use the supervised loss of labeled data on the student network to update the weight function and obtain the weight β of each unlabeled data. i ;

[0011] Step 5: Use the probability output obtained by the softmax function of the output of the student and teacher networks in step 2, combined with the weight of the unlabeled data calculated in step 3, to calculate the reliable perceptual consistency loss of the average teacher model;

[0012] Step 6: Use the projection network to normalize the output of the student network and the teacher network in step 2 into a low-dimensional representation, and obtain Z s and Z t , combined with the weights of the unlabeled data calculated in step 4, calculate the reliable perceptual contrast loss of the average teacher model;

[0013] Step 7: The supervised loss calculated in step 3, the reliable perceptual consistency loss calculated in step 5, and the reliable perceptual contrast loss calculated in step 6 are weighted and summed to form the final loss function;

[0014] Step 8: Construct an optimization method for the inner and outer layers, and alternately update the parameters w of the weight function and the network parameters θ.

[0015] Preferably, the present invention proposes to introduce a weight function into the average teacher model, automatically assign a unique weight to each unlabeled data, and prevent the network model performance from degrading after using weighted unlabeled data by continuously tracking the classification performance of labeled data. The weight function is specifically expressed as:

[0016] β i=g(f(x i ;θ,η),w) (1)

[0017] Preferably, the reliable perceptual consistency loss and the reliable perceptual contrast loss designed by the present invention are used as the core of the network model, which are specifically expressed as follows:

[0018] L rcs =β i L C (2)

[0019]

[0020] Among them, L rcs represents the reliable perceptual consistency loss, L rct represents the reliable perceptual contrast loss, L C is the minimum mean square error metric, for the normalized low-dimensional feature representation z of the i-th unlabeled data i , the pseudo label obtained by the teacher network is expressed as A(i)≡Z s ∪Z t \{i}, and Represents the index set of all positive classes in A(i), and τ is a temperature parameter with a value of 0.5.

[0021] Beneficial effects:

[0022] 1. The present invention introduces a weight function into the average teacher model, maps the probability prediction of each unlabeled data into a corresponding weight that can reflect its reliability, and can alleviate the negative impact that indiscriminate use of unlabeled medical images may have on the classification performance of the average teacher model.

[0023] 2. The reliable perception consistency constraint designed by the present invention alleviates the performance degradation problem caused by the average teacher model using unreliable unlabeled data by penalizing the inconsistent probability predictions of labeled data and reliable unlabeled data under different input perturbations, and improves the network model's ability to learn data-level information. Secondly, the deep features of medical image data are normalized into a low-dimensional representation using a projection network. Through reliable perception contrast constraints, labeled data and reliable unlabeled data are encouraged to have better intra-class compactness and inter-class separability, and reliable information at the data structure level can be learned. In summary, the two constraints jointly improve the security and generalization of the network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a network framework diagram of the semi-supervised medical image classification method based on the secure comparison self-integration framework in the present invention. DETAILED DESCRIPTION

[0025] See also Figure 1 As shown:

[0026] The present invention is further explained below in conjunction with examples.

[0027] The main implementation process of the present invention is as follows, and the relevant network framework is shown in Figure 1 .

[0028] Step 1: Divide images into labeled data With unlabeled data Among them, x i represents the i-th medical image sample, y i Represents the true label of the i-th sample;

[0029] Step 2: Take a batch of training data consisting of unlabeled samples and labeled samples, perform random rotation and affine transformation to increase data disturbance. The present invention horizontally flips the image with a probability of 0.5, and randomly rotates the angle range (-10 degrees, 10 degrees), and the corresponding translation interval parameters of the length and width dimensions are (0.02, 0.02). After repeating the process twice, two sets of input data with increased disturbance are obtained, and the two sets of data are respectively input into the student network and the teacher network of the average teacher model.

[0030] Step 3: Use the probability output obtained by the softmax function of the output of the student network in step 2 together with the true label to calculate the supervised loss, which is specifically expressed as:

[0031]

[0032] Where L sup represents the supervised loss and θ the student network parameters.

[0033] Step 4: Design a weight function and use the supervised loss of labeled data on the student network to update the weight function. The weight function automatically assigns a unique weight β to each unlabeled data. i , using Parameterized weight function Map the probability prediction of unlabeled data to the corresponding weight β i , specifically expressed as:

[0034] β i =g(f(x i ;θ,η),w) (5)

[0035] Step 5: Use the probability output obtained by the softmax function of the output of the student and teacher networks in step 2, combined with the weight β of the unlabeled data calculated in step 3 i , calculate the reliable perceived consistency loss of the average teacher model, which is specifically expressed as:

[0036] L rcs =g(f(x i ;θ,η),w)·L C =β i L C (6)

[0037] Among them, L C It is a distance metric. The present invention adopts the minimum mean square error. Formula (6) penalizes inconsistent predictions of labeled data and reliable unlabeled data under different input perturbations.

[0038] Step 6: Input the results of the student network and the teacher network in step 2 into the projection network to obtain B unlabeled data and The normalized low-dimensional representation is combined with the weights of the unlabeled data calculated in step 4 to calculate the reliable perceived contrast loss of the average teacher model, which is specifically expressed as:

[0039]

[0040] Among them, β i , i = 1, 2, ..., 2B is z i The weight parameter of the teacher network is expressed as follows: A(i)≡Z s ∪Z t \{i}, Represents the index set of all positive classes in A(i), τ is a temperature parameter, and the present invention takes it as 0.5.

[0041] Step 7: The supervised loss, reliable perception consistency loss, and reliable perception contrast loss are weighted and summed to form the final loss function. The final objective function is expressed as the following two-layer optimization problem:

[0042]

[0043] where β contains and β L With β U is automatically calculated by formula (5). s (·;θ) represents the supervision loss.

[0044] Step 8: In order to solve the two-layer optimization problem in step 7, the present invention represents the outer layer target as L outer (θ), the inner target is represented as L inner (θ, w), alternately update the parameters w of the weight function and the network parameters θ. First, update the network parameters θ by fixing the parameters w of the weight function g t ,θt+1 It can be optimized as follows:

[0045]

[0046] where η θ is the step size; then use the network parameter θ t+1 , update the parameters w of the weight function g according to the gradient of the supervised loss:

[0047]

[0048] where η w is the step size, and L is calculated using the automatic differentiation technique of the deep learning framework. outer (θ t+1 ) relative to the parameter w t gradient.

[0049] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in the relevant technical field, are also included in the patent protection scope of the present invention.

Claims

1. A semi-supervised medical image classification method based on a secure contrast self-ensemble framework, characterized in that: Here are the steps: Step 1: Divide the medical image training data into labeled data With unlabeled data Among them, x i represents the i-th medical image sample, y i Represents the true label of the i-th sample; Step 2: Add data perturbation by performing random rotation and affine transformation on a batch of training data consisting of unlabeled samples and labeled samples. After repeating the process twice, two sets of input data with added perturbation are obtained. The two sets of data are input into the student network and teacher network of the average teacher model respectively. Step 3: Use the probability output obtained by the softmax function of the output of the student network in step 2 and the true label to calculate the supervised loss; In step 3, the probability output obtained by passing the output of the student network through the softmax function is calculated together with the true label to calculate the supervised loss, which is specifically expressed as: Where L sup represents the supervised loss, θ is the student network parameter; Step 4: Design a weight function and use the supervised loss of labeled data on the student network to update the weight function and obtain the weight β of each unlabeled data. i ; In step 4, a weight function is designed and the supervised loss of the labeled data on the student network is used to update the weight function. The weight function automatically assigns a unique weight β to each unlabeled data. i , using Parameterized weight function g: Map the probability prediction of unlabeled data to the corresponding weight β i , specifically expressed as: b i =g(f(x i ;θ,η),w)(2); Step 5: Use the probability output obtained by the softmax function of the output of the student and teacher networks in step 2, combined with the weight of the unlabeled data calculated in step 3, to calculate the reliable perceptual consistency loss of the average teacher model; In step 5, the probability output obtained by passing the output of the student and teacher networks through the softmax function is combined with the weight β of the unlabeled data calculated by formula (2) i , calculate the reliable perceived consistency loss of the average teacher model, which is specifically expressed as: L rcs =g(f(x i ;θ,η),w)·L C =b i L C (3) Among them, L C is a distance metric. Equation (3) penalizes inconsistent predictions between labeled data and reliable unlabeled data under different input perturbations; Step 6: Use the projection network to normalize the output of the student network and the teacher network in step 2 into a low-dimensional representation, and obtain Z s and Z t , combined with the weights of the unlabeled data calculated in step 4, calculate the reliable perceptual contrast loss of the average teacher model; In step 6, the results of the student network and the teacher network are input into the projection network to obtain B unlabeled data and The normalized low-dimensional representation is combined with the weights of the calculated unlabeled data to calculate the reliable perceived contrast loss of the average teacher model, which is specifically expressed as: Among them, β i , i = 1, 2, ..., 2B is z i The weight parameter of the teacher network is expressed as follows: A(i)≡Z s ∪Z t \{i}, represents the index set of all positive classes in A(i), τ is a temperature parameter; Step 7: The supervised loss calculated in step 3, the reliable perceptual consistency loss calculated in step 5, and the reliable perceptual contrast loss calculated in step 6 are weighted and summed to form the final loss function; In step 7, the supervised loss, reliable perceptual consistency loss, and reliable perceptual contrast loss are weighted and summed to form the final loss function. The final objective function is expressed as the following two-layer optimization problem: where β contains and β L With β U is automatically calculated by formula (2); L s (D L ; θ) represents the supervised loss; Step 8: Construct an optimization method for the inner and outer layers, and alternately update the parameters w of the weight function and the network parameters θ; In step 8, to solve the two-layer optimization problem, the outer layer objective is represented as L outer (θ), the inner target is represented as L inner (θ, w), alternately update the parameters w of the weight function and the network parameters θ; first, update the network parameters θ, by fixing the parameters w of the weight function g t ,θ t+1 It can be optimized as follows: where η θ is the step size; then use the network parameter θ t+1 , update the parameters w of the weight function g according to the gradient of the supervised loss: where η w is the step size, and L is calculated by the automatic differentiation technology of the deep learning framework outer (θ t+1 ) relative to the parameter w t The gradient of Step 9: Input the medical image data to be classified into the final network model to obtain its category label.

Images