Lightweight lung CT image COVID-19 classification method based on knowledge distillation

Abstract

The invention discloses a lightweight lung CT image COVID-19 classification method based on knowledge distillation, and aims to solve the problem that extreme compression and high precision of a model are difficult to consider in the prior art. The method comprises the following steps: firstly, on the basis of OfficientNet-B0 pre-trained by ImageNet, constructing a high-precision teacher model through classifier improvement and hierarchical fine tuning; an improved U-Net lightweight student model containing three coding and decoding stages, depth separable convolution and cross-layer residual connection is designed; teacher knowledge is migrated through a mixed distillation loss function of alpha = 0.7 and T = 4, and strategy training such as an AdamW optimizer and a dynamic learning rate is matched. Experiments show that the parameter quantity of the student model is reduced by 97.4% compared with that of the teacher model, the test accuracy reaches 91.96%, the efficiency is improved by 35.9 times, and the AUC is 0.965. The method realizes the balance of light weight and high precision, can be deployed in resource-constrained medical equipment, and provides technical support for COVID-19 rapid auxiliary diagnosis.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of artificial intelligence, computer vision and digital healthcare, and specifically relates to a lightweight lung CT image COVID-19 classification method based on knowledge distillation. Background Technology

[0002] Deep learning technology, especially deep convolutional neural networks, has made breakthroughs in medical imaging tasks such as lung CT image classification and lesion detection due to its powerful hierarchical feature extraction capabilities. The diagnostic accuracy of some models has reached or surpassed that of professional physicians. Among them, high-performance CNN models, represented by EfficientNet-B0, have achieved certain lightweight advantages compared to traditional CNN models while maintaining classification accuracy through efficient network scaling strategies. However, they still have the following drawbacks: 1. High number of parameters and computational complexity: EfficientNet-B0 typically has millions of parameters, resulting in high model storage requirements and slow inference speed, making it difficult to adapt to resource-constrained scenarios such as portable CT equipment and mobile diagnostic terminals. 2. Bottlenecks in the application of knowledge distillation technology: Existing knowledge distillation methods struggle to balance the contradiction between "extreme compression" and "high accuracy preservation" in medical image classification tasks. When the number of parameters in the student model is compressed to less than 5% of that in the teacher model, the "knowledge transfer gap" caused by the difference in model structure between teachers and students (the teacher uses a deep CNN classification architecture, while the student uses a lightweight simplified architecture) leads to a significant drop in the classification accuracy of the student model, which is usually lower than 90% of the accuracy of the teacher model and cannot meet the accuracy requirements of clinical diagnosis. 3. Limitations of loss function design: Traditional distillation methods often use a single hard loss (fitting the true label) or soft loss (mimicking the teacher's output). The former is difficult to fully utilize the deep semantic knowledge of the teacher model, while the latter is easily affected by the local prediction bias of the teacher model. Neither can achieve efficient knowledge transfer.

[0003] Therefore, there is an urgent need for a lightweight medical image classification scheme that can overcome the limitations of the "structural gap" and optimize the efficiency of knowledge transfer, while maintaining classification accuracy that meets clinical needs and achieving extreme model lightweighting. Summary of the Invention

[0004] To overcome the shortcomings of the existing technologies, this invention proposes a lightweight lung CT image COVID-19 classification method based on knowledge distillation, which aims to significantly reduce the number of model parameters while maintaining excellent classification performance, and effectively solve the knowledge transfer problem caused by the "structural gap" between teacher and student models.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A lightweight COVID-19 classification method for lung CT images based on knowledge distillation, comprising the following steps: Step S1: Construct and train a high-precision teacher model: Use EfficientNet-B0 pre-trained on the ImageNet dataset as the basic architecture, replace its classifier with an enhanced binary classifier, and train it to a validation accuracy of ≥95.70% through a hierarchical progressive fine-tuning strategy, which is used to extract knowledge representations of lung CT images. Step S2: Construct a lightweight student model: Design an improved U-Net architecture based on depthwise separable convolution. This model includes an encoder-decoder structure and cross-layer residual connections. The number of basic features is set to 16, and the classification results are output through a specific classification head. Its parameter count is significantly lower than that of the teacher model. Step S3: Knowledge distillation training: An improved distillation loss function combining hard and soft loss is used to transfer the knowledge of the teacher model to the student model. The training process optimizes training stability through dynamic learning rate scheduling, gradient pruning, and early stopping. Step S4: Model performance evaluation: The student model is comprehensively evaluated from three dimensions: classification accuracy, compression efficiency, and inference efficiency, using multiple indicators.

[0006] Further optimization, step S1 specifically includes: Step S1.1: The enhanced binary classifier is constructed in series in the following order: Dropout layer (dropout probability p=0.5, suppressing overfitting) → fully connected layer (input dimension 1280, output dimension 512, realizing feature dimension compression and nonlinear transformation) → batch normalization layer (standardize feature distribution, stabilize the training process) → SiLU activation function (introduce smooth nonlinear characteristics, enhance feature interaction ability) → Dropout layer (dropout probability p=0.4, further suppressing overfitting) → fully connected layer (input dimension 512, output dimension 2, output binary classification probability).

[0007] Step S1.2, the layered progressive fine-tuning strategy, specifically involves freezing the parameters of features.0 to features.4 layers of EfficientNet-B0. This part corresponds to the shallow layers of the model and is responsible for extracting general low-level features. It does not need to be adapted to medical images. Only the parameters of features.5 to features.8 layers (deep semantic feature extraction layers) and the enhanced binary classifier described in step 1.2 are updated to ensure that the model retains general features while accurately learning disease-specific features of lung CT images, such as ground-glass opacities and consolidation regions.

[0008] Step S1.3: The training process uses the cross-entropy loss function as the optimization objective, configures the Adam optimizer, sets the learning rate to 1e-4, and sets the training batch size to 32; iterates training until the accuracy on the validation set is stable at ≥95.70%, and finally the teacher model achieves an accuracy of ≥97.05% on the independent test set.

[0009] Further optimization, step S2 specifically includes: Step S2.1: The improved U-Net architecture includes 3 complete encoding stages and 3 complete decoding stages, with the basic feature number set to 16; Step S2.2: The basic computational unit for the encoding and decoding stages is a lightweight basic block. The structure of each basic block is: depthwise separable convolutional layer → batch normalization layer → ReLU activation function → depthwise separable convolutional layer → batch normalization layer → ReLU activation function, and cross-layer residual connections are set inside the basic block. Step S2.3, encoding path downsampling method: At the end of each encoding stage, a 2×2 max pooling operation with a step size of 2 is used to halve the feature map size and expand the receptive field; after each encoding stage, the number of feature map channels doubles, specifically changing from 16 to 32 to 64 to 128, gradually extracting coarse-grained disease semantic features.

[0010] Step S2.4, Decoding Path Upsampling Method: At the beginning of each decoding stage, a 2×2 transposed convolution operation (with a stride of 2) is used to restore the feature map size; after each decoding stage, the number of channels in the feature map is halved, specifically changing from 128 to 64 to 32 to 16; at the same time, the upsampled feature map is concatenated with the feature map of the corresponding level of the encoding path along the channel dimension (the number of channels after concatenation is twice the number of output channels of the decoding stage), realizing the fusion of semantic features and spatial detail features.

[0011] Step S2.5: The classification heads are constructed in the following order: an adaptive global average pooling layer is connected to the decoder output (compressing feature maps of arbitrary size into 1×1×16 feature vectors while retaining feature information in the channel dimension). The subsequent classification heads are constructed in the following order: Dropout layer (p=0.3) → fully connected layer (input dimension 16, output dimension 64) → batch normalization layer → ReLU activation function → Dropout layer (p=0.2) → fully connected layer (input dimension 64, output dimension 2). The total number of parameters in the final student model is strictly controlled to be 123,037.

[0012] Further optimization, step S3 specifically includes: Step S3.1, Dataset Division: Divide all lung CT image data into training set, validation set and test set according to 7:1.5:1.5. Use stratified sampling to ensure that the ratio of COVID-19 positive samples to negative samples in each set is consistent with the original dataset.

[0013] Step S3.2, the calculation method for the mixed distillation loss function is as follows: L total =(1-α)×L hard +α×L soft , Where: L hard The cross-entropy loss between the student model's predicted results and the actual data labels is used to calculate the difference between the student model's predicted results and the actual data labels. This guides the model to learn the objective facts of disease diagnosis and ensures the accuracy of the classification results. soft To employ the KL divergence loss function, the probability distribution of the student model's output logits after softening at temperature T is calculated, and the difference between the probability distribution of the teacher model's output logits after softening at the same temperature T is compared. The temperature parameter T=4. By softening the probability distribution, the student model can learn the teacher model's "uncertain knowledge" (such as the judgment logic for fuzzy samples), achieving the transfer of deep semantic knowledge. The weight coefficient α is fixed at 0.7 to balance the contributions of hard loss (weight 0.3) and soft loss (weight 0.7), ensuring that the student model, while imitating the teacher's knowledge, does not deviate from the constraints of the true labels.

[0014] Step S3.3, Training hyperparameter configuration: The optimizer used is AdamW, and the initial learning rate is set to 1e-3. Dynamic learning rate scheduling: The ReduceLROnPlateau strategy is adopted. When the validation loss does not decrease for 5 consecutive epochs, the learning rate is reduced by 0.5 times to improve the convergence accuracy of the model in the later stage of training. Gradient clipping: Set the maximum norm of the gradient (max). norm =1.0, which scales gradients whose sum of absolute gradient values ​​exceeds this threshold during training to prevent gradient explosion in deep lightweight networks; Early stopping mechanism: Set the early stopping patience value to 20. When the validation loss does not decrease for 20 consecutive epochs, immediately terminate the training and save the current optimal model parameters to avoid model overfitting.

[0015] Further optimization, step S4 specifically includes: Step S4.1, Classification Accuracy Evaluation: Calculate the classification accuracy, area under the ROC curve (AUC), and area under the precision-recall curve (PR-AUC) of the student model on the independent test set. The required accuracy is ≥91.96%, AUC is ≥0.965, PR-AUC is ≥0.972, and the accuracy retention is ≥94.8%. Step S4.2, Compression Performance Evaluation: The lightweighting effect of the model is quantified using the following two indicators: The percentage reduction in the number of parameters: (Number of parameters in the teacher model - Number of parameters in the student model) / Number of parameters in the teacher model × 100% ≥ 97.4%; Model compression ratio: Teacher model parameter count / Student model parameter count ≥ 37.9 times; Step S4.3, Inference Efficiency Evaluation: Define the efficiency index as "classification accuracy percentage / million parameters" to quantify the performance contribution per unit of parameters; require student model efficiency ≥ 747.33, and efficiency improvement multiple (student model efficiency / teacher model efficiency) ≥ 35.9 times to verify the feasibility of model deployment in resource-constrained scenarios.

[0016] Step S4.4, Multi-dimensional Visualization Validation: Plot the confusion matrix to analyze the false positive rate and false negative rate, and plot the ROC curve and Precision-Recall curve to verify the classification robustness of the model under different thresholds.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Breaking through the technical bottleneck of "balancing extreme compression and high accuracy": Through the fusion design of improved U-Net architecture and depthwise separable convolution, the number of parameters of the student model is reduced by 97.4 compared with the teacher model, but the classification accuracy is still maintained at 91.96% and the accuracy retention rate reaches 94.8%, which solves the defect of existing knowledge distillation methods in medical image classification that "the performance drops significantly when the compression ratio exceeds 10 times".

[0018] 2. Significantly improved knowledge transfer efficiency: The innovative use of a hybrid distillation loss function of "hard loss + soft loss" combined with the improved U-Net and the feature extraction logic of the teacher model effectively narrows the "structural gap" between teacher and student models. Hard loss ensures classification accuracy, while soft loss transfers the teacher's deep semantic knowledge, enabling the lightweight student model to fully absorb the diagnostic experience of the teacher model. 3. The inference efficiency meets the needs of resource-constrained scenarios: The efficiency index of the student model is 35.9 times that of the teacher model (20.80), and the performance contribution per unit of parameters is greatly improved. It can be quickly deployed in resource-constrained scenarios such as portable CT equipment, edge computing nodes, and mobile diagnosis and treatment terminals in primary hospitals to achieve rapid auxiliary diagnosis of COVID-19.

[0019] 4. Strong training stability and generalization ability: The combination of hierarchical progressive fine-tuning, AdamW optimizer, dynamic learning rate scheduling, gradient pruning, and early stopping mechanism effectively solves problems such as gradient vanishing, exploding, and overfitting in the training process of lightweight models; multi-dimensional evaluation verifies the robustness of the model under different sample distributions and meets the strict requirements of clinical diagnosis.

[0020] 5. High clinical application value: The lightweight design of the model lowers the hardware deployment threshold, and the high-precision classification performance can help doctors quickly distinguish between COVID-19 positive and negative samples. It is especially suitable for scenarios such as primary hospitals and epidemic prevention and control sites, which can effectively improve diagnostic efficiency, reduce the rate of missed diagnosis / misdiagnosis, and provide technical support for epidemic prevention and control. Attached Figure Description

[0021] Figure 1 This is a flowchart of a lightweight lung CT image COVID-19 classification method based on knowledge distillation, as described in this invention. Detailed Implementation

[0022] The present invention will be further described below with reference to specific embodiments.

[0023] like Figure 1 As shown, a lightweight lung CT image classification method based on knowledge distillation is used to distinguish between COVID-19 positive and negative results, including the following steps: Step (1): Build and train a teacher model that achieves high accuracy on the target medical image dataset to extract knowledge representations of medical images.

[0024] Step (2): Construct a lightweight student model with a lower structural complexity than the teacher model.

[0025] Step (3): Based on the knowledge distillation method, an improved distillation loss function combining hard loss and soft loss is used to transfer the knowledge of the teacher model to the student model.

[0026] Step (4): A lightweight lung CT image classification method based on knowledge distillation was used to distinguish between COVID-19 positive and negative results, and the method was further evaluated and identified.

[0027] Step (1) specifically refers to: 1A) Using EfficientNet-B0 pre-trained on the ImageNet dataset as the basic architecture, its classifier is replaced with an augmented binary classifier. The specific structure is: Dropout layer (p=0.5) → fully connected layer (1280→512) → batch normalization layer → SiLU activation function → Dropout layer (p=0.4) → fully connected layer (512→2).

[0028] 1B) A hierarchical progressive fine-tuning strategy is adopted, freezing the parameters of the shallow part of the model (features.0 to features.4) and training only the deep part (features.5 to features.8) and the newly constructed classifier, in order to retain general features and adapt to specific tasks.

[0029] 1C) The model was trained on the target COVID-19 lung CT image dataset using the cross-entropy loss function and the Adam optimizer (learning rate 1e-4). After training, the teacher model achieved an accuracy of 95.70% on the validation set and 97.05% on the independent test set, with a total of 4,665,470 parameters.

[0030] Step (2) specifically refers to: 2A) Construct an improved U-Net architecture based on depthwise separable convolutions as the student model. To achieve lightweight design, its base number of features is set to 16. The model consists of three complete downsampling (encoding) and upsampling (decoding) stages.

[0031] 2B) The base block (LightweightBlock) of each encoder and decoder stage consists of two cascaded depthwise separable convolutional layers, each followed by batch normalization and ReLU activation functions. Simultaneously, cross-layer residual connections are introduced to enhance gradient flow and feature reuse.

[0032] 2C) The encoder path is downsampled using max pooling (MaxPool2d), doubling the number of channels at each stage. The decoder path is upsampled using transposed convolution (ConvTranspose2d) and concatenated with the feature maps of the corresponding encoder layers in the channel dimension (skip connections) to fuse multi-scale information.

[0033] (2D) After the output of the final decoder block, a global average pooling layer (AdaptiveAvgPool2d) is connected, followed by a classifier module with the following structure: Dropout (p=0.3) → fully connected layer (16→64) → batch normalization → ReLU → Dropout (p=0.2) → fully connected layer (64→2). The total number of parameters in this student model is only 123,037.

[0034] Step (3) specifically refers to: 3A) All data were divided into training, validation and test sets in a ratio of 7:1.5:1.5, and stratified sampling was used to ensure that the proportion of classes in each set was consistent with the original dataset.

[0035] 3B) The distillation loss function is specifically: L total =(1-α)×L hard +α×L soft , Where: L hard The cross-entropy loss between the student model's predicted results and the actual data labels is used to calculate the difference between the student model's predicted results and the actual data labels. This guides the model to learn the objective facts of disease diagnosis and ensures the accuracy of the classification results. soft The KL divergence loss function is used, which is the KL divergence loss between the output probability distributions of the student model and the teacher model under temperature parameter T adjustment; wherein the weighting coefficient α is 0.7 and the temperature parameter T is 4.

[0036] 3C) Set and optimize training hyperparameters. Use AdamW as the optimizer with an initial learning rate of 1e-3, and employ the ReduceLROnPlateau dynamic learning rate scheduler (adjusted based on validation loss). During training, gradient clipping (max_norm=1.0) is applied to prevent gradient explosion, and early stopping (patience=20) is used to avoid overfitting.

[0037] Step (4) specifically refers to: 4A) The classification performance was evaluated, and the final student model achieved a classification accuracy of 91.96%, an area under the ROC curve (AUC) of 0.965, and an area under the precision-recall curve (PR-AUC) of 0.972.

[0038] 4B) The compression performance was evaluated by comparing the student model (123,037 parameters) and the teacher model (4,665,470 parameters). The calculated parameter reduction rate was 97.4%, and the model compression ratio was 37.9 times.

[0039] 4C) Efficiency evaluation was performed to calculate the performance contribution per unit of parameters. The efficiency (accuracy percentage / million parameters) of the student model was 747.33, which was much higher than the teacher model's 20.80, representing an efficiency improvement of 35.9 times.

[0040] (4D) Through multi-dimensional indicators such as confusion matrix, ROC curve, and precision-recall curve, this method comprehensively demonstrates that it achieves extreme lightweighting while maintaining high classification performance. This computational analysis needs to be combined with appropriate experimental verification to ensure the high credibility and reproducibility of the experimental and computational conclusions, while significantly improving the correct classification of images within a controllable experimental and computational range.

[0041] Based on existing datasets, this invention proposes a lightweight model-based knowledge distillation method with high classification accuracy, thus having a wide range of applications and market prospects.

[0042] This invention utilizes a lightweight deep learning model based on knowledge distillation to distinguish between COVID-19 positive and negative results, using chest X-ray images. The invention applies the U-Net architecture (encoder-decoder + skip connections), commonly used in image segmentation, to a lightweight classification task. This structure allows the student model to capture coarse-grained semantic information during downsampling, while retaining essential low-level texture and spatial details from the upsampling (decoding) process through skip connections, ultimately reconstructing an information-rich feature representation before the classification head. This enables the student model to effectively absorb the knowledge essence of structurally different teacher models, even with extremely low parameter counts, thus maintaining very high performance under extreme compression.

[0043] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A lightweight COVID-19 classification method for lung CT images based on knowledge distillation, characterized in that: Includes the following steps: Step S1: Construct and train a high-precision teacher model: Use EfficientNet-B0 pre-trained on the ImageNet dataset as the basic architecture, replace its classifier with an enhanced binary classifier, and train it to a validation accuracy of ≥95.70% through a hierarchical progressive fine-tuning strategy, which is used to extract knowledge representations of lung CT images. Step S2: Construct a lightweight student model: Design an improved U-Net architecture based on depthwise separable convolution. This model includes an encoder-decoder structure and cross-layer residual connections. The number of basic features is set to 16, and the classification results are output through a specific classification head. Its parameter count is significantly lower than that of the teacher model. Step S3: Knowledge distillation training: An improved distillation loss function combining hard and soft loss is used to transfer the knowledge of the teacher model to the student model. The training process optimizes training stability through dynamic learning rate scheduling, gradient pruning, and early stopping. Step S4: Model performance evaluation: The student model is comprehensively evaluated from three dimensions: classification accuracy, compression efficiency, and inference efficiency, using multiple indicators.

2. The lightweight lung CT image COVID-19 classification method based on knowledge distillation according to claim 1, characterized in that: Step S1 specifically includes: Step S1.1: The enhanced binary classifier is constructed in series in the following order: Dropout layer → fully connected layer → batch normalization layer → SiLU activation function → Dropout → fully connected layer; Step S1.2, the hierarchical progressive fine-tuning strategy is as follows: freeze the parameters of features.0 to features.4 layers of EfficientNet-B0, and only update the parameters of features.5 to features.8 layers and the enhanced binary classifier. Step S1.3: The training process uses the cross-entropy loss function as the optimization objective, configures the Adam optimizer, and iterates the training until the accuracy on the validation set is stable at ≥95.70%, and finally the teacher model achieves an accuracy of ≥97.05% on the independent test set.

3. The lightweight lung CT image COVID-19 classification method based on knowledge distillation according to claim 1, characterized in that: Step S2 specifically includes: Step S2.1: The improved U-Net architecture includes 3 complete encoding stages and 3 complete decoding stages, with the basic feature number set to 16; Step S2.2: The basic computational unit for the encoding and decoding stages is a lightweight basic block. The structure of each basic block is: depthwise separable convolutional layer → batch normalization layer → ReLU activation function → depthwise separable convolutional layer → batch normalization layer → ReLU activation function, and cross-layer residual connections are set inside the basic block. Step S2.3, Encoding path downsampling method: At the end of each encoding stage, a 2×2 max pooling operation is used, which doubles the number of channels in the feature map after pooling; Step S2.4, Decoding path upsampling method: At the beginning of each decoding stage, a 2×2 transposed convolution operation is used. After upsampling, the number of channels in the feature map is halved, and the feature map of the corresponding level of the encoding path is concatenated along the channel dimension. Step S2.5: The classification heads are constructed in the following order: adaptive global average pooling layer → Dropout layer → fully connected layer → batch normalization layer → ReLU activation function → Dropout layer.

4. The lightweight lung CT image COVID-19 classification method based on knowledge distillation according to claim 1, characterized in that: Step S3 specifically includes: Step S3.1, Dataset Division: Divide all lung CT image data into training set, validation set and test set according to 7:1.5:1.

5. Use stratified sampling to ensure that the ratio of COVID-19 positive samples to negative samples in each set is consistent with the original dataset. Step S3.2, the calculation method for the mixed distillation loss function is as follows: L total =(1-α)×L hard +α×L soft , Where: L hard The cross-entropy loss value between the student model's predicted results and the true labels in the data; L soft The KL divergence loss value between the probability distribution of logits output by the student model after softening at temperature T=4 and the probability distribution of logits output by the teacher model after softening at the same temperature is calculated; the weighting coefficient α is fixed at 0.

7. Step S3.3, Training hyperparameter configuration: The optimizer used is AdamW, and the initial learning rate is set to 1e-3. Dynamic learning rate scheduling: The ReduceLROnPlateau strategy is adopted. When the validation loss does not decrease for 5 consecutive epochs, the learning rate decays by 0.5 times. Gradient clipping: Set the maximum norm of the gradient (max). norm =1.0, which scales gradients whose sum of absolute gradient values ​​exceeds this threshold during training; Early stopping mechanism: Set the early stopping patience value to 20. When the validation loss does not decrease for 20 consecutive epochs, terminate the training and save the current optimal model parameters.

5. The lightweight lung CT image COVID-19 classification method based on knowledge distillation according to claim 1, characterized in that: Step S4 specifically includes: Step S4.1, Classification Accuracy Evaluation: Calculate the classification accuracy, area under the ROC curve (AUC), and area under the precision-recall curve (PR-AUC) of the student model on the independent test set. The required accuracy is ≥91.96%, AUC is ≥0.965, PR-AUC is ≥0.972, and the accuracy retention is ≥94.8%. Step S4.2, Compression Performance Evaluation: Calculate the parameter reduction ratio using the formula (Teacher model parameter count - Student model parameter count) / Teacher model parameter count × 100%, requiring a reduction of ≥ 97.4%; calculate the model compression ratio using the formula Teacher model parameter count / Student model parameter count, requiring a compression ratio of ≥ 37.9 times. Step S4.3, Inference Efficiency Evaluation: Define the efficiency index as "classification accuracy percentage / million parameters", calculate the efficiency values ​​of the student model and the teacher model, and require that the efficiency of the student model is ≥747.33 and the efficiency improvement factor is ≥35.9 times; Step S4.4, Multi-dimensional Visualization Validation: Plot the confusion matrix to analyze the false positive rate and false negative rate, and plot the ROC curve and Precision-Recall curve to verify the classification robustness of the model under different thresholds.

Images