Red blood cell test time domain adaptation detection method based on average teacher model

Abstract

The invention provides a red blood cell test time domain adaptation detection method based on an average teacher model, which belongs to the field of deep learning and comprises the steps of pre-training, data enhancement, random recovery, loss calculation and update of a teacher model and a student model. First, a benchmark detection model is pre-trained on source domain data to obtain initial features. In the test stage, target domain data is input into the teacher model and the student model after being subjected to weak enhancement and strong enhancement respectively. A pseudo label is generated after the teacher model is input, so that the influence of error accumulation in the continuous adaptation process is reduced; the student model is optimized through the cross entropy loss of the prediction result of the teacher model, and the teacher model is updated through an index moving average method. In addition, a random recovery method is adopted, knowledge is explicitly recovered from a source domain pre-training model to a student model, and the problem of disastrous forgetting is effectively relieved. And the final model can keep good performance for target domain data during testing.

Description

Technical Field

[0001] The invention belongs to the technical field of image detection, and in particular relates to a time-domain adaptive detection method for red blood cell testing based on an average teacher model. Background Art

[0002] In the medical field, cell detection is an important medical examination method. Counting the number of cells of different morphologies in a patient's blood cells can provide guidance for diseases such as anemia. Usually, experienced doctors add annotations to blood cell datasets, which are then divided into training and test sets. After training the neural network model on the training set, its performance is tested on the test set.

[0003] In actual applications, blood cell images may be collected by different devices, which may come from different manufacturers and different device models. The results of collecting the same sample on different devices are different. This difference will cause the performance of the existing model to decline.

[0004] In order to solve the problem of inconsistent performance of the model on data collected by different devices and make the model perform well on data collected by different devices, we can use the test-time domain adaptation method. This method aims to adapt the pre-trained model on the source data by learning from the unlabeled target data at test time. Due to the domain gap between the source training data and the target test data, adaptation is required to obtain good performance. In reality, due to patient privacy issues or legal restrictions, the source data is usually inaccessible during test time, which makes it a more challenging and realistic problem than unsupervised domain adaptation. In many scenarios, adaptation also needs to be performed online. Therefore, test-time adaptation is crucial to the success of blood cell detection applications under domain transfer.

[0005] Existing test-time domain adaptation methods usually use pseudo-labels or entropy to update the parameters of the model to deal with changes in data distribution between the source and target domains. Such self-training methods are effective when the test data comes from a fixed domain. However, they become unstable when the domain of the test data keeps changing. There are two reasons for this: First, as the domain changes, the data distribution also changes, which makes the pseudo-labels noisier, so early prediction errors are more likely to lead to gradual accumulation of errors. Second, the model adapts to the data distribution of different domains for a long time, and the original knowledge in the source domain is difficult to preserve, which may lead to catastrophic forgetting of knowledge. Summary of the invention

[0006] In this context, the purpose of the present invention is to provide a time-domain adaptive detection method for red blood cell testing based on the average teacher model. Experimental results show that this method can significantly improve the performance of the source model on other target domain data, providing an effective solution for the medical application of red blood cell detection.

[0007] First, we pre-train a source detection model on the source domain data and use it to initialize the teacher and student models. Next, the target domain data is weakly enhanced and then input into the teacher model, and then strongly enhanced and then input into the student model. The student model is updated with cross entropy loss, and the teacher model is updated with the exponential moving average method. Finally, some student model weights are randomly restored, and the pseudo-label output by the teacher model is used as the final prediction result. We selected the largest public blood cell dataset Pathologic for experiments and divided it into four domains. The experimental results show that this method can significantly improve the performance of the source model on other target domain data, providing an effective solution for the medical application of red blood cell detection.

[0008] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] The present invention is a time-domain adaptive detection method for red blood cell testing based on an average teacher model, characterized in that it comprises the following steps:

[0010] 1) Pre-train a baseline detection model on the source domain data as the source detection model to learn universal feature representation;

[0011] 2) Initializing the teacher model and the student model respectively using the weights of the source detection model;

[0012] 3) Perform different degrees of data augmentation on the target domain data, where weak augmentation data is input into the teacher model and strong augmentation data is input into the student model;

[0013] 4) Get pseudo labels based on the teacher model and predict labels based on the student model;

[0014] 5) Calculate the cross entropy loss based on the predictions of the teacher model and the student model and update the student model;

[0015] 6) Update the teacher model weights using the exponential moving average method;

[0016] 7) Randomly restore some weights from the source detection model to the student model.

[0017] Furthermore, the step 1) specifically includes:

[0018] Set the source domain dataset to in represents the input image in the source domain, Represents the label information corresponding to the input image, and N represents the number of samples of the source domain data;

[0019] Use the source domain dataset to train the benchmark detection model and optimize the loss function

[0020]

[0021] in is the classification loss, which is used to identify the category to which the target belongs. is the regression loss;

[0022] In the pre-training phase, the model first calculates the predicted output on each input sample through forward propagation. Then according to the annotation Calculating Losses Optimize θ by backpropagation:

[0023]

[0024] Where η is the learning rate, Represents the gradient of the loss function with respect to the model parameters.

[0025] Furthermore, the step 3) specifically includes: weakly enhancing the target domain data and inputting it into the teacher model:

[0026] For each input image x, weak data augmentation involves random horizontal flipping, expressed as follows:

[0027]

[0028] Where flip(·) represents the horizontal flip operation, p is a random number sampled from the uniform distribution u(0, 1), and p flip is the preset reversal probability;

[0029] The target domain data is strongly enhanced and then input into the student model:

[0030] For each input image x, color jitter is expressed as:

[0031] x′=ColorJitter(x;Δ b , Δ c , Δ s , Δ h )

[0032] where Δ b , Δ c , Δ s , Δ hare the perturbation amplitudes of brightness, contrast, saturation, and hue, respectively, and their values ​​are randomly sampled from a preset range;

[0033] Grayscale conversion is to convert the image into a grayscale image. The expression is:

[0034] x′=Gray(x)=0.299·x R +0.587 x G +0.114 x B

[0035] where x R , x G , x B Represent the red, green, and blue channels of the image respectively;

[0036] Gaussian blur processing is performed on the image data to simulate blurred scenes at different focal lengths. The operation is expressed as: x′=x*Kσ;

[0037] Where Kσ is a Gaussian kernel function with a standard deviation of σ, and * represents a convolution operation;

[0038] The image data is cut into patches to enable the model to learn more global features. The operation is expressed as: x′=x⊙M;

[0039] Where ⊙ represents element-wise multiplication, M is a binary mask matrix, where the position where its value is 0 represents the blocked area, and the position where its value is 1 represents the retained area.

[0040] Furthermore, the step 5) specifically includes: setting the target data to The teacher model is f θ′ ; At time step t, the teacher model first generates a pseudo label Then the student model f θ The cross entropy loss calculated by the prediction results of the student model and the teacher model is updated. The cross entropy loss formula is as follows:

[0041]

[0042] in is the probability that the pseudo-label predicted by the teacher model successfully predicts category c, is the prediction result of the student model.

[0043] Furthermore, in step 6), the teacher model update method is the exponential moving average EMA method, which is performed after the student model is updated, and the formula is as follows:

[0044] θ′ t+1 =αθ′ t +(1-α)θ t+1

[0045] where α is the smoothing factor, θ t+1 is the updated student model weight, θ′ t is the teacher model weight before update, θ′ t+1 are the updated teacher model weights.

[0046] Furthermore, the step 7) specifically includes: randomly restoring the weight of the student model after the teacher model is updated;

[0047] Update the student model f by gradient descent using the cross entropy formula θ When , the convolutional layer has: l+1 =W t+1 ★x l ;

[0048] Where ★ represents the convolution operation, x l and x l+1 Represents the input and output of this layer, W t+1 represents a trainable convolutional filter;

[0049] Update the weight W by:

[0050] M~bernoulli(p),

[0051] W t+1 =M⊙W 0 +(1-M)⊙W t+1 ;

[0052] where ⊙ represents element-by-element multiplication, p is the small recovery probability, and M is the sum of W t+1 A mask tensor of the same shape.

[0053] Beneficial effects of the present invention:

[0054] The present invention can save the cost of separately training detection models on each domain in the traditional medical image field, and no longer requires consuming a lot of time and resources; the present invention only needs to train one source model, and the remaining domain adaptation steps can be completed during testing, which can significantly reduce the workload of professionals for data set annotation; the present invention can promote a detection model to different medical equipment to meet the needs of different doctors in different hospitals during disease diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is the overall framework diagram of the method of the present invention;

[0056] Figure 2 It is an algorithm flow chart of the method of the present invention;

[0057] Figure 3 Schematic diagram of cells in different domains. DETAILED DESCRIPTION

[0058] In order to facilitate the understanding of those skilled in the art, the present invention is further described below in conjunction with embodiments and drawings. The contents mentioned in the implementation modes are not intended to limit the present invention.

[0059] Reference Figure 1 As shown, the present invention is a time-domain adaptive detection method for red blood cell testing based on an average teacher model, comprising the following steps:

[0060] 1) A source detection model is pre-trained on the source domain data of red blood cells. The model is updated by gradient descent. After training, it is used as the source detection model to provide initialization parameters for the target domain model, aiming to extract the basic features of red blood cell images.

[0061] 2) The source detection model is used to initialize the teacher and student models, which have the same model structure. In this way, the teacher model can quickly adapt to the red blood cell data in the target domain, and the student model can also be adjusted on the target domain data to better adapt to the differences in red blood cell morphology and distribution in clinical data.

[0062] 3) Perform weak enhancement and strong enhancement on the target domain data of the current batch of red blood cells. The purpose of weak enhancement is to provide more stable input for the teacher model to generate reliable pseudo labels; strong enhancement enhances the adaptability of the student model to changes in the distribution of red blood cell images by enriching sample diversity.

[0063] 4) Input the weakly enhanced target domain data into the teacher model, and use the generated prediction result as the pseudo label, which is used as the final prediction result of the red blood cell image; input the strongly enhanced target domain data into the student model and obtain the prediction result.

[0064] 5) Calculate the cross entropy loss based on the predictions of the teacher model and the student model and update the student model using gradient descent.

[0065] 6) After the student model is updated, the teacher model is updated using the exponential moving average method, and the calculation process of this method uses the updated student model parameters and teacher model parameters. The updated teacher model is used for the next round of test time domain adaptation of red blood cell images.

[0066] 7) After the teacher model is updated, some weights are randomly restored from the source detection model to the student model to prevent catastrophic forgetting. The updated student model is used for the next round of test time domain adaptation of red blood cell images.

[0067] In general, the present invention can be divided into two major stages: pre-training stage and test adaptation stage. The main task of the pre-training stage is to train a model on labeled data, with the aim of giving it basic prior knowledge so that the model can be more easily adapted to other domains. The main task of the test adaptation stage is to input the unlabeled data of other domains into the average teacher model framework in batches, and continuously adapt the model parameters to the data distribution of the new domain during the test process, so that it can also have good performance in the new domain, and finally output the prediction results of the test data and the updated model. The overall algorithm flow chart is as follows: Figure 2 shown.

[0068] Furthermore, the step 1) of pre-training the detection model is an important step to improve the model's domain adaptability. This process aims to extract common features and provide initialization parameter information for the target domain model by training the model on the source domain data.

[0069] Let the source domain dataset be in represents the input image in the source domain, Indicates the label information corresponding to the input image, and N represents the number of samples in the source domain. Model f θ The learning goal is to optimize the loss function To learn the model parameters θ:

[0070]

[0071] in is the classification loss, which is used to identify the category to which the target belongs. It is a regression loss, which is used to optimize the position accuracy of the predicted box. In common target detection models, the classification loss usually adopts the focal loss, while the regression loss is based on the L1 loss or IoU loss of the bounding box offset.

[0072] Specifically, in the pre-training phase, the model is first propagated through forward propagation to calculate the predicted output of each input sample.

[0073] Then according to the annotation Calculating Losses Optimize θ by backpropagation:

[0074]

[0075] Where η is the learning rate, Represents the gradient of the loss function with respect to the model parameters.

[0076] A significant advantage of pre-training on blood cell datasets is that the distribution of objects and background information in the images can provide rich prior knowledge for subsequent domain adaptation. Blood cell microscopic images usually contain high-density and diverse objects, whose characteristics are similar to real scenes in many object detection tasks, thus helping the model to extract more generalizable features.

[0077] Furthermore, the initialization in step 2) refers to copying the parameters of the source detection model to the teacher and student models, because they have the same network structure and the same parameter structure.

[0078] Furthermore, the strong and weak data enhancement in step 3) refers to the preprocessing step before the image is input to the student or teacher model. The teacher model inputs weakly enhanced data, the purpose of which is to allow the teacher model to output reliable pseudo labels, while the student model inputs strongly enhanced data, the purpose of which is to enhance the diversity of training samples through large-scale image transformation. Strong enhancement can help the student model adapt to potential distribution changes in the target domain, making it more generalizable.

[0079] Weak data augmentation involves random horizontal flipping, and the formula is as follows:

[0080]

[0081] Where flip(·) represents the horizontal flip operation, p is a random number sampled from the uniform distribution u(0, 1), and p flip is the preset reversal probability.

[0082] Strong data augmentation involves randomly added color jittering, grayscale, Gaussian blur, and clipping patches. These augmentations are designed to improve the student's robustness and generalization to appearance changes and occluded objects. Color jittering changes the color distribution of an image by randomly adjusting brightness, contrast, saturation, and hue. For each input image x, color jittering can be expressed as:

[0083] x′=ColorJitter(x;Δ b , Δ c , Δ s , Δ h )

[0084] where Δ b , Δ c , Δ s , Δ h They are the perturbation amplitudes of brightness, contrast, saturation, and hue, respectively, and their values ​​are randomly sampled from a preset range.

[0085] Grayscale conversion is to convert the image into a grayscale image, which can reduce the interference of color information. The formula is:

[0086] x′=Gray(x)=0.299·x R +0.587 x G +0.114 x B

[0087] where x R , x G , x B Represent the red, green, and blue channels of the image respectively.

[0088] Gaussian blur achieves blur effect by applying Gaussian kernel function to the image to simulate blur scenes at different focal lengths. The operation can be expressed as:

[0089] x′=x*Kσ

[0090] Where Kσ is a Gaussian kernel function with a standard deviation of σ, and * represents a convolution operation.

[0091] Clipping patches are an enhancement method that randomly blocks parts of an image, aiming to force the model to learn more global features. The operation can be expressed as:

[0092] x′=x⊙M

[0093] Where ⊙ represents element-wise multiplication, M is a binary mask matrix, where the position where its value is 0 represents the blocked area, and the position where its value is 1 represents the retained area.

[0094] Furthermore, in step 4), the teacher model generates pseudo labels. This is because the distribution of target data is also changing in the scenario where the target domain is constantly changing. At this time, the quality of pseudo label generation will decrease with the change in distribution, and the average model usually performs better than the normal model. At the same time, it can reduce the impact of accumulated errors in the model during the continuous domain adaptation process. Therefore, the teacher model is used in the present invention to generate more accurate pseudo label predictions.

[0095] Furthermore, the student model updating method in step 5) is completed by calculating the cross entropy loss of the prediction results of the student and teacher models. Let the target data be The teacher model is f θ′ At time step t, the teacher model first generates pseudo labels Then the student model f θ The cross entropy loss calculated by the prediction results of the student model and the teacher model is updated. The cross entropy loss formula is as follows:

[0096]

[0097] in is the probability that the pseudo-label predicted by the teacher model successfully predicts category c, is the prediction result of the student model, and this loss can strengthen the consistency between the teacher model and the student model.

[0098] Furthermore, the teacher model update method in step 6) is an exponential moving average (EMA) method. It is performed after the student model is updated, and the formula is as follows:

[0099] θ′ t+1 =αθ′ t +(1-α)θ t+1

[0100] where α is the smoothing factor, θ t+1 is the updated student model weight, θ′ t is the teacher model weight before update, θ′ t+1 are the updated teacher model weights.

[0101] Furthermore, the random recovery in step 7) refers to randomly restoring a small number of trainable weights in the model to the initial weights of the pre-trained source model during the model training process, so as to prevent the model from deviating too far from the initial source model during the domain adaptation process, thereby effectively alleviating the catastrophic forgetting problem. Long-term domain adaptation inevitably leads to error accumulation and knowledge forgetting, especially when the target domain distribution changes drastically, self-training may aggravate the degree of prediction error. Even if the target domain distribution does not change significantly, continuous adaptation may still cause the model to lose its memory of source domain knowledge. Through random recovery, the model can explicitly introduce knowledge from the source detection model, thereby retaining source domain information while preventing model collapse, enhancing adaptability and ensuring effective updating of trainable parameters.

[0102] Therefore, we randomly restore the weights of the student model after the teacher model is updated, which explicitly restores the knowledge from the source detection model. When updating the student model f using the above cross entropy formula for gradient descent θ When , the convolutional layer has:

[0103] x l+1 =W t+1 ★x l

[0104] Where ★ represents the convolution operation, x l and x l+1 Represents the input and output of this layer, W t+1 represents a trainable convolution filter. The present invention updates the weight W by the following formula:

[0105] M~bernoulli(p)

[0106] W t+1 =M⊙W0 +(1-M)⊙W t+1

[0107] Where ⊙ represents element-by-element multiplication. p is the small recovery probability, M is the sum of W t+1 A mask tensor of the same shape. The mask tensor determines the t+1 Which element in is restored to the source weight W 0 .

[0108] Target domain input data during testing The final prediction result is the output of the teacher model The category with the highest probability value.

[0109] The above is a detailed description of the algorithm flow. The overall algorithm flow is as follows:

[0110]

[0111] The present invention is based on the largest public blood cell dataset, Pathologics, which was collected and sorted by two blood experts, who manually prepared and stained peripheral blood and bone marrow images of some patients. They used four digital pathology scanners to generate 25 whole-slide images (WSIs), each equipped with a different optical microscope, control unit, integrated camera, and patch stitching software. The 25 slides were divided into four subsets according to the scanner, and each scanner was used to scan a different subset. Figure 3 The cell images captured by the four scanners are shown, and Table 1 shows the distribution of these 25 slides in the four scanners.

[0112] Table 1

[0113]

[0114]

[0115] The four scanners represent four different domains. In order to show the differences in data distribution in different domains, we conducted a cross-domain model test. The test first pre-trained the model on one domain, and then tested the data of the remaining four domains on this pre-trained model. And so on, a total of four groups of experiments were conducted. The evaluation indicators used in the experiment were Average Precision (AP) and Intersection over Union (IoU) = 0.50:0.95. The experimental results are shown in Table 2. It can be seen from the table that when the model was tested on other domains, the performance declined to varying degrees. When tested on the domain where the training set was located, the performance was optimal. It can be seen that the difference in data distribution between domains will lead to a decline in model performance.

[0116] Table 2

[0117]

[0118] The experimental design of the present invention is based on a data set divided into four sub-domains Each subdomain can be represented as x j is the input image, y j is the corresponding annotation, N i Indicates subdomain D i The experiment is divided into four groups, and the specific process is as follows: In each group of experiments, from the four subdomains {D i} select a subdomain as the source domain D s =D i (i=1,2,3,4), pre-trained on the source domain, and the source detection model f is obtained θ . The remaining three subdomains As the target domain dataset, each target domain image x k ∈D j During testing, the inference is performed through the model adaptation process. The target domain image is weakly enhanced and then fed into the teacher model. Get predicted pseudo labels At the same time, the image is strongly enhanced and then input into the student model The student model updates its weights based on its own predictions and pseudo labels, and updates the teacher model using the EMA method, and finally randomly restores some of the weights of the student model. The performance of each subdomain is evaluated using the average precision (AP, IoU = 50:95). Four groups of experiments test temporal adaptation with each subdomain as the source domain, and compare the performance of the adapted model with the direct prediction performance of the source detection model.

[0119] The present invention has a significant effect in solving the problem of cross-domain model performance degradation. Through test time domain adaptation, the method can adjust the model weights to the target domain online, so as to directly obtain accurate detection results in the test phase. After test time adaptation, the detection accuracy of the target domain data has been improved to varying degrees, approaching more than 90% of the performance of its own training model. This shows that the present invention has good scalability, practicality and effectiveness for the massive cell slice images scanned and generated by different hospitals and different instruments.

[0120] The present invention proposes a test-time domain adaptation method, which can enable the model to have equivalent performance to labeled training in the absence of labels. It first allows the model to learn the basic knowledge of the field through pre-training, and then performs domain adaptation during testing. First, it completes the adaptation task based on the average teacher model framework, and second, it combines weighted averaging and random recovery to improve the model's adaptability and help it output more accurate and reliable prediction results.

[0121] There are many practical application ways of the present invention. The above is only the preferred implementation mode of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements can be made without departing from the principle of the present invention. These improvements should also be regarded as the scope of protection of the present invention.

Claims

1. A time-domain adaptive detection method for red blood cell testing based on an average teacher model, characterized in that: The steps include: 1) Pre-train a baseline detection model on the source domain data as the source detection model to learn universal feature representation; 2) Initializing the teacher model and the student model respectively using the weights of the source detection model; 3) Perform different degrees of data augmentation on the target domain data, where weak augmentation data is input into the teacher model and strong augmentation data is input into the student model; 4) Get pseudo labels based on the teacher model and predict labels based on the student model; 5) Calculate the cross entropy loss based on the predictions of the teacher model and the student model and update the student model; 6) Update the teacher model weights using the exponential moving average method; 7) Randomly restore some weights from the source detection model to the student model.

2. The time-domain adaptive detection method for red blood cell testing based on the average teacher model according to claim 1 is characterized in that: The step 1) specifically includes: Set the source domain dataset to in represents the input image in the source domain, Represents the label information corresponding to the input image, and N represents the number of samples of the source domain data; Use the source domain dataset to train the benchmark detection model and optimize the loss function in is the classification loss, which is used to identify the category to which the target belongs. is the regression loss; In the pre-training phase, the model first calculates the predicted output on each input sample through forward propagation. Then according to the annotation Calculating Losses Optimize θ by backpropagation: Where η is the learning rate, Represents the gradient of the loss function with respect to the model parameters.

3. The time-domain adaptive detection method for red blood cell testing based on the average teacher model according to claim 1 is characterized in that: The step 3) specifically includes: weakly enhancing the target domain data and inputting it into the teacher model: For each input image x, weak data augmentation involves random horizontal flipping, expressed as follows: Where flip(·) represents the horizontal flip operation, p is a random number sampled from the uniform distribution u(0,1), and p flip is the preset reversal probability; The target domain data is strongly enhanced and then input into the student model: For each input image x, color jitter is expressed as: x′=ColorJitter(x;Δ b ,D c ,D s ,D h ) Where Δ b ,Δ c ,Δ s ,Δ h are the perturbation amplitudes of brightness, contrast, saturation, and hue, respectively, and their values ​​are randomly sampled from a preset range; Grayscale conversion is to convert the image into a grayscale image. The expression is: x′=Gray(x)=0.299·x R +0.587·x G +0.114·x B where x R ,x G ,x B Represent the red, green, and blue channels of the image respectively; Gaussian blur is performed on the image data to simulate blurred scenes at different focal lengths. The operation is expressed as: x′=x*Kσ; Where Kσ is a Gaussian kernel function with a standard deviation of σ, and * represents a convolution operation; The image data is cut into patches to enable the model to learn more global features. The operation is expressed as: x′=x⊙M; Where ⊙ represents element-wise multiplication, M is a binary mask matrix, where the position where its value is 0 represents the blocked area, and the position where its value is 1 represents the retained area.

4. The time-domain adaptive detection method for red blood cell testing based on the average teacher model according to claim 1 is characterized in that: The step 5) specifically includes: setting the target data to The teacher model is f θ′ ; At time step t, the teacher model first generates a pseudo label Then the student model f θ The cross entropy loss calculated by the prediction results of the student model and the teacher model is updated. The cross entropy loss formula is as follows: in is the probability that the pseudo-label predicted by the teacher model successfully predicts category c, is the prediction result of the student model.

5. The time-domain adaptive detection method for red blood cell testing based on the average teacher model according to claim 1 is characterized in that: In step 6), the teacher model update method is the exponential moving average EMA method, which is performed after the student model is updated. The formula is as follows: θ′ t+1 =θ′ t +(1-α)θ t+1 where α is the smoothing factor, θ t+1 is the updated student model weight, θ′ t is the teacher model weight before update, θ′ t+1 are the updated teacher model weights.

6. The time-domain adaptive detection method for red blood cell testing based on the average teacher model according to claim 1, characterized in that: The step 7) specifically includes: randomly restoring the weight of the student model after the teacher model is updated; Update the student model f by gradient descent using the cross entropy formula θ When , the convolutional layer has: x l+1 =W t+1 *x l ; Where * represents the convolution operation, x l and x l+1 Represents the input and output of this layer, W t+1 represents a trainable convolution filter; the weight W is updated by: M~bernoulli(p), W t+1 =M⊙W0+(1-M)⊙W t+1 ; where ⊙ represents element-by-element multiplication, p is the small recovery probability, and M is the sum of W t+1 A mask tensor of the same shape.

Images