A Classification Method and System for Noise-Resistant Cassava Leaf Diseases Based on Course Knowledge Distillation

Abstract

This invention discloses a noise-resistant cassava leaf disease classification method and system based on curriculum knowledge distillation, belonging to the field of knowledge distillation application technology. The invention includes: Step S1, training a well-trained student model S using dataset D based on knowledge distillation and curriculum learning; Step S2, classifying the cassava leaf disease image dataset to be classified using the trained student model S. The method utilizes a teacher model to obtain feature knowledge of clean samples, and then, through curriculum knowledge distillation, uses the teacher model as a measure of the difficulty of positive and negative samples. A training scheduler inputs samples into the model in an order of increasing difficulty. The introduction of the training scheduler and difficulty measurer effectively reduces the impact of annotation noise on the model, and supervises the improvement of the student network's generalization ability, thereby constructing a noise-resistant cassava leaf disease classification method and model.

Description

Technical Field

[0001] This invention relates to the field of knowledge distillation application technology, and in particular to a method for classifying noise-resistant cassava leaf diseases based on curriculum knowledge distillation. Background Technology

[0002] Training deep learning models requires large-scale, high-quality labeled datasets; models lacking sufficient labeled data are prone to overfitting. In agriculture, different diseases share similar characteristics, thus labeling disease images requires annotators with strong agronomical knowledge and experience. Expert annotation for dataset construction is costly and time-consuming, and cannot guarantee label homogeneity, making it difficult to meet model training needs; this inevitably leads to excessively high label production costs, making it difficult to match the actual agricultural production situation. To meet the practical needs of large-scale sample annotation, "crowdsourcing" annotation has become a mainstream approach, obtaining corresponding labels by leveraging numerous individuals to annotate samples; however, the sample labels obtained through this method often contain a significant proportion of incorrect labels, i.e., label noise. Furthermore, machine annotation methods using semi-supervised or unsupervised learning algorithms are also susceptible to low-quality samples and retraining, resulting in inaccurate pseudo-labels. All these factors contribute to label noise that affects the robustness and generalization ability of the model, thereby reducing its performance.

[0003] To mitigate the impact of annotation noise, many studies have proposed methods such as adjusting the loss function, adding regularization terms, or special training strategies to eliminate the negative effects of annotation noise on the model. However, these methods rely on accurate sample estimation. Knowledge distillation utilizes a pre-trained, high-performance large model to guide the training of a lightweight model. The teacher model acts as an auxiliary model, transmitting "implicit knowledge" to the student model and transforming it into an easily understandable form to help the student better grasp the knowledge, thereby improving the generalization ability of the student model. "Implicit knowledge," also known as tacit knowledge, refers to the non-verbal experience, thought processes, and problem-solving strategies that a teacher possesses after learning a particular domain. Knowledge distillation transforms complex models into simpler ones, reducing the risk of overfitting. However, knowledge distillation generally employs traditional neural network training strategies, making it susceptible to annotation noise, which in turn affects the classification accuracy of the student network model. Summary of the Invention

[0004] The purpose of this invention is to address the aforementioned problems by providing a noise-resistant cassava leaf disease classification method based on curriculum knowledge distillation. This method combines a knowledge distillation framework and curriculum learning strategies to mitigate the impact of annotation noise in the dataset, thereby improving model accuracy and generalization.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A classification method for noise-resistant cassava leaf diseases based on course knowledge distillation includes the following:

[0007] Step S1: Based on knowledge distillation and course learning, train the dataset D to obtain the trained student model S; including the following specific processing flow:

[0008] Step S11: Construct cassava leaf disease data as a noisy dataset D;

[0009] Step S12: Construct a training scheduler, using the teacher model T as a difficulty measure, inputting the dataset D to obtain sample difficulty scores, and then using a regularizer to determine the proportion and weight of samples selected in each round of training, selecting samples to form a training subset D. m ;

[0010] The specific processing flow of the training scheduler is as follows:

[0011] (1) In the initial stage, the original dataset D is input into the teacher model T to obtain the sample difficulty score l, and samples with difficulty scores lower than the score are selected. The samples form the first training subset D1;

[0012] (2) In the m-th stage, the dataset D is input into the difficulty measuring device T to obtain the sample difficulty score l, and the difficulty is selected that is lower than the score λ. 2 The samples form the m-th training subset D m Where, D1∪D2∪…∪D m-1 =D m ; Calculate and update the learning speed λ based on the current batch;

[0013] (3) Repeat steps (1) and (2) above until training ends or the model tends to converge.

[0014] Step S13: Construct an adaptive weight module to adaptively adjust the knowledge transfer ratio ω of the teacher model T to the student model S, using a training subset D. m The student model S obtained in this round of training is obtained;

[0015] The knowledge transfer weight ω is determined using the validation set accuracy of each training round as a measure of model discrepancy; in the e-th training round, the validation set accuracy of the student model is A. s The validation set accuracy of the teacher model is A. T The formula for calculating the weighted weight ω is as follows:

[0016]

[0017] In the formula, γ is the coefficient for controlling the gradual automatic decay of the specific gravity of knowledge distillation, with a value of [0, 1].

[0018] Step S14: Repeat steps S12-S13 until the target number of iterations is completed or the model tends to converge, and output the trained student model S.

[0019] Step S2: Use the trained student model S to classify the cassava leaf disease image dataset to be classified; the dataset includes images of 5 categories, namely images of four diseases: cassava bacterial wilt, cassava brown streak, cassava green mottle and cassava mosaic, and images of healthy cassava.

[0020] By adopting the above technical solution, the present invention has the following beneficial effects:

[0021] 1. This invention utilizes a teacher model to acquire feature knowledge of clean samples, and then uses course knowledge distillation. The teacher model is used as a measure of the difficulty of positive and negative samples. The training scheduler inputs samples into the model in order of difficulty. The introduction of the training scheduler and the difficulty measurer can effectively reduce the impact of labeling noise on the model and supervise the improvement of the generalization ability of the student network, thereby constructing a noise-resistant cassava leaf disease classification method and model.

[0022] 2. By introducing an adaptive weight module, this invention automatically adjusts the knowledge transfer ratio from the teacher model to the student model as the performance of the student model improves, further enhancing the effectiveness of the model, effectively improving the self-learning ability of the student model, and increasing the classification accuracy of categories. Attached Figure Description

[0023] Figure 1 This is a flowchart of the classification method of the present invention.

[0024] Figure 2 This is a framework diagram of course knowledge distillation for the present invention.

[0025] Figure 3 This is a graph showing the change in the weight of the self-weighting module of the present invention.

[0026] Figure 4 This is a graph showing how the weight of the selected samples changes with the difficulty level in this invention.

[0027] Figure 5 This is a curve showing the change in sample difficulty according to the present invention.

[0028] Figure 6 This is a visualization analysis result diagram of the present invention.

[0029] Figure 7 This is a system block diagram of the classification system of the present invention. Detailed Implementation

[0030] The specific implementation of the invention will be further described below with reference to the accompanying drawings.

[0031] like Figure 1 As shown, the noise-resistant cassava leaf disease classification method based on curriculum knowledge distillation of the present invention includes the following:

[0032] Step S1: Based on knowledge distillation and course learning, train the dataset D to obtain the trained student model S; including the following specific processing flow:

[0033] Step S11: Construct cassava leaf disease data as a noisy dataset D;

[0034] Step S12: Construct a training scheduler, using the teacher model T as a difficulty measure, inputting the dataset D to obtain sample difficulty scores, and then using a regularizer to determine the proportion and weight of samples selected in each round of training, selecting samples to form a training subset Dm; the specific processing flow of the training scheduler is as follows:

[0035] (1) In the initial stage, the original dataset D is input into the teacher model T to obtain the sample difficulty score l, and samples with difficulty scores lower than the score are selected. The samples form the first training subset D1;

[0036] (2) In the m-th stage, the dataset D is input into the difficulty measuring device T to obtain the sample difficulty score l, and the difficulty is selected that is lower than the score λ. 2 The samples form the m-th training subset D m Where, D1∪D2∪…∪D m-1 =D m ; Calculate and update the learning speed λ based on the current batch;

[0037] (3) Repeat steps (1) and (2) above until training ends or the model tends to converge.

[0038] Step S13: Construct an adaptive weight module to adaptively adjust the knowledge transfer ratio ω of the teacher model T to the student model S, using a training subset D. m The student model S obtained in this round of training is obtained;

[0039] Step S14: Repeat steps S12-S13 until the target number of iterations is completed or the model tends to converge, and output the trained student model S.

[0040] Step S2: The trained student model S is used to classify the cassava leaf disease image dataset to be classified. This image dataset includes images of five categories: images of four diseases (cassava bacterial wilt, cassava brown streak, cassava green mottle, and cassava mosaic) and images of healthy cassava. The knowledge distillation method used in this classification process will be explained in detail below.

[0041] The main design idea of ​​this invention is to reduce the negative impact of annotation noise on model learning by improving model generalization. Annotation noise can cause the model to tend to fit incorrect samples, leading to incorrect knowledge learning. Therefore, identifying valuable positive and negative samples for training is key to solving this problem. To alleviate this issue, this invention proposes a cassava leaf disease classification method based on course knowledge distillation (CCKD), introducing a course learning strategy into the knowledge distillation scenario. The CCKD method uses a teacher model as a measure of the difficulty of positive and negative samples and inputs samples into the model in order of increasing difficulty through a training scheduler. Introducing a training scheduler and a difficulty measurer can effectively reduce the impact of annotation noise on the model, thereby improving model generalization. The CCKD method is designed based on a knowledge distillation framework and consists of a teacher model T, a student model S, a self-adaptive weight module (adaptive weight module), and a training scheduler, as follows: Figure 2 As shown.

[0042] During training, the teacher model T acts as a difficulty measure, dynamically adjusting the learning curriculum through a training scheduler. Teacher model T extracts features from images in the noisy dataset D and its own experiential knowledge, using this as supervisory information for the student model S, guiding its supervised learning and improving its recognition accuracy and generalization ability. Student model S also learns image features from the noisy dataset, and adjusts the knowledge transfer weights using a self-adaptive weighting scheduling module based on its learning progress. Before training, teacher model T is pre-trained with clean labels, learning "dark knowledge" from the clean data to pass feature knowledge to the student model during training.

[0043] Typically, knowledge distillation methods use the entire dataset D to pre-train a teacher model, and then distill a continuously learning student model using this pre-trained teacher model. Curriculum learning methods, however, divide knowledge distillation into m stages, each using a different training set (D1, D2, ..., Dm). As training progresses, the size of the training set continuously expands to allow the student model to absorb more knowledge. In this process, the method for partitioning the training set is determined by the training scheduler, while the learning rate of knowledge distillation is determined by the self-adjusting weight module. The following section will explain the self-adjusting weight module and the training scheduler scheme and their roles from the perspective of the loss function.

[0044] 1. Loss Function Design

[0045] The loss function guides model training, determining the direction of network optimization by calculating the degree of difference between predicted and actual values. To improve the model's robustness, this invention introduces (decoupled knowledge distillation) DKDLoss as the distillation loss L. dist Non-target features are used as auxiliary information. This allows the model to learn target features while also acquiring more useful knowledge from noise.

[0046] DKDLoss divides the basic KD loss into two parts: Target Class Knowledge Distillation (TCKD) and Non-target Class Knowledge Distillation (NCKD), which improve the generalization and discriminative abilities of the student model, respectively. TCKD operates on the binary probability distribution of the target class, conveying knowledge related to the difficulty of sample fitting and indicating the model's confidence in the sample. If the teacher model's output is [0.9, 0.1] in binary classification, it indicates that the teacher model has high confidence in that sample; if the teacher model's output is [0.55, 0.45], it indicates that the model has difficulty making an accurate judgment and has low confidence in that sample. NCKD operates on the probability distribution of the non-target class, containing the "dark knowledge" conveyed by knowledge distillation. Both TCKD and NCKD losses are implemented using Kullback-Leible-Divergence (KL Divergence). KL Divergence measures the difference between two distributions, and the calculation formula for KL in knowledge distillation is shown in Equation 2.

[0047]

[0048]

[0049] In the formula, p = [p1, p2, ..., p t , ..., p C ]∈R 1×C , is the classification probability of the model output f after sotmax. Let represent the teacher model's predicted probability for the t-th class of samples. c represents the predicted probability of the student model for the t-th class sample.

[0050] Distillation loss L distThe probability distribution is learned from the teacher model output using soft distillation. The calculation process is as follows: First, the outputs of the student model and the teacher model are divided into target and non-target categories according to the true labels, and the probability distribution is softened using the softmax function, with a temperature parameter controlling the smoothness of the probability distribution. Second, the target and non-target probabilities of the teacher and students are separated. Finally, the sum of the KL divergence between the student target and teacher target category probabilities and the KL divergence between the student non-target and teacher non-target category probabilities is used as the TCKD loss function (Equation 3), and the KL divergence between the teacher non-target and student non-target category probabilities is used as the NCKD loss function (Equation 4). dist The general formula for the loss function is shown in Figure 5.

[0051]

[0052]

[0053] L dist =αTCKD+βNCKD (5)

[0054] In the formula, This represents the predicted probability of the student model for samples other than the t-th class. This represents the teacher model's predicted probability for the i-th class (excluding the t-th class). This represents the predicted probability of the student model for samples of class i (excluding class t). α and β are hyperparameters that serve as weights controlling TCKD and NCKD.

[0055] To ensure that the student model does not completely depend on the guidance of the teacher model, CCKD uses a basic loss Li. basic This measures the error between the student model and the true labels. The base loss uses GCE loss (Generalized cross entropy loss), which is suitable for training on noisy samples. It combines the advantages of cross entropy loss (CE loss) and mean absolute error loss (MAELoss). CE is the most commonly used classification loss function in convolutional neural networks, but if the sample contains too many noisy samples, it will overfit the easily classified samples and underfit the difficult-to-classify samples. MAE is not overly sensitive to noise, but the gradient is large during training and it is not easy to converge, resulting in a slow model learning speed. GCELoss integrates the robustness of MAE to noise and the efficiency of CE, and its calculation formula is shown in Equation 6 below.

[0056]

[0057] In the formula, f jThe corresponding model predicts the unique heat value of class j. q is a hyperparameter ranging from 0 to 1, used to control the balance between robustness and efficiency.

[0058] In summary, the total loss L of the CCKD method of the present invention is shown in Formula 7 below.

[0059] L=ωL dist +(1-ω)L basic (7)

[0060] In the formula, ω is a hyperparameter that controls distillation loss and basic loss, and is determined by the self-weighting module.

[0061] 2. Self-configured weight module

[0062] Student models, through self-study and guidance from teacher models, can mitigate the effects of noise to some extent. However, due to the presence of noisy annotations in the ground truth labels, the output confidence of student models is often low in the early stages of training. But as the student model learns, its recognition performance gradually improves, and the teacher model, in turn, interferes with its learning progress. Therefore, it is unreasonable for student models to rely entirely on teacher guidance. During training, students gradually learn more implicit information, exceeding the scope of the teacher's guidance. Furthermore, teacher models struggle to make accurate judgments about knowledge they haven't yet mastered. If student models continue to completely trust the judgments of the teacher model at this point, the teacher model will inhibit the student model's learning progress and may even become the upper limit of its performance.

[0063] Based on the above premises, the CCKD of this invention uses a self-adjusting weight module to avoid the aforementioned problems. As the cognitive level of the student model increases, by adjusting the ratio of distillation loss (teacher and student loss) to student loss, the proportion of knowledge transfer from the teacher model to the student model can be changed (the value of ω in Formula 8 below).

[0064] The self-adjusted weights are assigned values ​​using the validation set accuracy at each training stage as a measure of model discrepancy. In the e-th training round, the validation set accuracy of the student model is A. s The validation set accuracy of the teacher model is A. T The weighted scheme is shown in Formula 8 below.

[0065]

[0066] In the formula, γ is a parameter used to control the gradual and automatic decrease of the specific gravity of the knowledge distillation, and the coefficient ranges from [0, 1].

[0067] The weight change curve is as follows Figure 3 As shown, the weight ω decreases as the accuracy of the student model gradually improves and the difference in accuracy between it and the teacher model becomes smaller.

[0068] Using a self-configured weight module achieves the following effect: In the early stages of training, the student model relies entirely on the guidance of the teacher model. As training progresses, the student model gradually begins to learn autonomously, at which point L... basi The proportion of c in the total loss L also gradually increases. By introducing an adaptive weight module, the proportion of knowledge transferred from the teacher model to the student model will automatically adjust as the performance of the student model improves, further enhancing the effectiveness of the model.

[0069] 3. Training Scheduler (Automated Difficulty Scheduler and Training Scheduler, Self-Paced Learning (SPL): Allows students to act as their own teachers and measures the difficulty of training samples based on their loss. This strategy is similar to self-learning by human students: determining their own learning pace based on their current status)

[0070] Existing research has shown that training a model by sorting samples by difficulty or confidence can improve model performance. However, most current knowledge distillation methods use randomly selected mini-batch training sets as model inputs, neglecting the impact of sample input order on model learning. This is especially true when the dataset is noisy, where the sample learning order becomes even more crucial. Therefore, this invention's CCKD method introduces a curriculum learning strategy to control the sample input order. Introducing a curriculum learning strategy requires addressing two issues: (1) How to assess the difficulty of each sample? (2) How to design a curriculum that progresses from easy to difficult samples?

[0071] For problem (1), the CCKD method of this invention uses the teacher model T as a difficulty measure to determine the difficulty level of the samples. This is because the output of the teacher model provides the target distribution of knowledge, serving as a visualization method for sample difficulty. The teacher model is trained with clean labels and can provide corresponding difficulty scores for samples during training, without being affected by noisy data. Furthermore, most of the knowledge learned by the teacher model with fixed parameters can be directly transferred to the student model. For a single sample, the cross-entropy loss calculated between the teacher model and the sample is used as the sample's difficulty assessment; the specific calculation formulas for the difficulty score are shown in Formulas 9 and 10.

[0072]

[0073] In the formula, y represents the true label of the sample. This represents the weight. A larger value for l indicates a more difficult sample.

[0074] For problem (2), the CCKD method of this invention uses a training scheduler (sample selector) to select samples. Since the loss of simple samples is significantly less than that of complex and noisy samples, the training scheduler uses a regularizer to determine the optimal weight vector for each training stage, thus determining the proportion and weight of samples selected in each training round.

[0075] The course learning strategy introduces weights v into the initial minimization objective function, v = [v1, v2, ..., v]. D ] T ∈[0,1],l i If we represent a specific sample, then the objective function is Equation 10.

[0076]

[0077] The regularizer g(v; λ) is key to minimizing the objective function. According to the AOS policy optimization, the minimum loss can be obtained by alternately optimizing w and v. When w... * When fixed, use a hybrid regularizer to solve for v. * As shown in Formula 11. When v * When fixed, w * The optimal solution can be obtained by the gradient descent algorithm, as shown in Equation 12. v* is the representation of v in the alternating optimization process.

[0078]

[0079]

[0080] The hybrid regularizer reflects the importance of samples at a finer granularity and can better distinguish similar losses, as shown in Equation 13. Since g(v; λ) is a convex function, the optimal solution of Equation 11 can be derived as Equation 14.

[0081]

[0082]

[0083] In the formula, k is a hyperparameter that controls the sample weight strength (k>0).

[0084] In Equation 14, the first condition term in the equation... Called l1, the second condition term λ 2 This is called l2; when the loss is less than the first threshold l1, the sample is considered an easy sample; when the loss is greater than the second threshold l2, the sample is considered a hard sample, and learning on that sample stops; when the loss is between the two, the number of hard samples is gradually increased and the weight of hard samples is decreased. Sample weight v * The curve showing the change in difficulty is as follows: Figure 4 As shown in the figure, k is 1, and λ is 0.5, 1, and 1.5 from left to right.

[0085] This solves the problems of assessing the difficulty of each sample and designing a course that progresses from easy to difficult. The specific implementation of the training scheduler is described below:

[0086] (1) In the initial stage, the original dataset D is input into the teacher model (difficulty measurer) T to obtain the sample difficulty score. l1 samples with difficulty lower than the score are selected from D to form the first subset D1.

[0087] (2) In the m-th stage, the dataset D is input into the difficulty measuring device T to obtain the sample difficulty score l. The different samples in D are assigned weights according to Formula 16, which is equivalent to selecting samples with difficulty lower than score l2 to form the m-th subset D. m Where, D1∪D2∪…∪D m-1 =D m Dm includes all samples up to stage m. λ is calculated and updated based on the current batch.

[0088] (3) Repeat the above steps until training is complete or the model tends to converge.

[0089] Through the above process, the difficulty of the dataset increases sequentially, and the sample size also gradually increases. The entire optimization algorithm flow is as follows:

[0090]

[0091]

[0092] For visualization of the selection of sample difficulty, see [link to relevant documentation]. Figure 5 The expression describes the variation of λ(S). λ is related to the number of training iterations (epochs), and the sample difficulty score threshold increases with the number of model iterations. The initial value of λ for the training scheduler is 0.5, and k is 1. As the number of iterations increases, λ gradually increases by 0.05. The student model learns on a dataset containing data ranging from easy to difficult, effectively improving the model's robustness.

[0093] 4. Experimental Design

[0094] The following section will introduce the cassava leaf disease dataset and evaluation metrics used, and explain the experimental parameter settings. To demonstrate the superiority of the proposed CCKD method on noisy datasets, comparative experiments will be conducted with classical and noisy models, and an ablation experiment will be performed to investigate the impact of adding modules on CCKD, thus verifying the effectiveness of CCKD.

[0095] 4.1 Dataset

[0096] The study used image data of cassava leaf diseases under complex backgrounds published by the Kaggle competition. The dataset contains five categories: images of four diseases, namely bacterial wilt of cassava (CBB), brown streak of cassava (CBSD), green mottle of cassava (CGM), and cassava mosaic (CMD), and images of healthy cassava.

[0097] The cassava leaf disease dataset contains 21,367 images, with the number of images in each category shown in the first row of Table 1. The categories are unbalanced, with the fewest containing only 1,087 images and the most containing 13,158. The CMD category accounts for over 60%, while the other four categories have relatively small differences in the number of images. This large disparity in the number of images across categories can lead to slow model convergence, overfitting to categories with fewer images, and poor generalization. Furthermore, because the dataset contains annotation noise, using offline image augmentation methods to expand the dataset can easily exacerbate this noise pollution. Therefore, undersampling was performed on categories with an excessive number of images. Images that were otherwise unidentifiable were then removed after a preliminary screening. After these steps, the final number of images selected was 10,917.

[0098] To avoid the impact of noisy annotations on the teacher model, this experiment selected 300 correct images from each class in the original dataset as the training dataset for the teacher model, totaling 1500 images. A clean dataset was used for both training and testing of the student model, with the clean images used to train the teacher model serving as the validation set. The student training set was randomly divided into a training:test set ratio of 9:1, resulting in 9827 images in the training set and 1090 images in the test set. The specific numbers are shown in the third row of Table 1 below.

[0099] Table 1: Number of categories in the dataset

[0100]

[0101] To enhance the robustness and generalization of the model, online image enhancement is performed during training. Commonly used image enhancement methods include, but are not limited to, vertical flipping, random brightness transformation, and adaptive histogram equalization. These methods can reduce the impact of factors such as shooting angle, lighting, and background on image detection performance, allowing the model to achieve better detection results. In this experiment, vertical flipping, random brightness transformation, and adaptive histogram equalization are employed.

[0102] 4.2 Experimental Setup

[0103] In the CCKD method of this invention, the selection of the teacher model and the student model is limited by the amount of data, model size, and complexity. The teacher model needs to utilize a powerful deep network architecture to learn features at various levels of abstraction, while the student model can use a simpler and lighter network architecture to learn implicit knowledge. Among common classification models, ResNet34, which has the highest accuracy, is selected as the teacher model, and MobileViT, which balances accuracy and model size, is selected as the student model. The combination of these two models is called the baseline model.

[0104] The experimental environment used the PyTorch framework, Ubuntu with CUDA 11.1, and an NVIDIA RTX 3090 (24GB). Transfer learning was performed using parameters pre-trained on ImageNet. The loss function was optimized using a cosine annealing learning rate. Normalization was performed using the mean and variance of ImageNet to balance the effects of geometric transformations. In post-processing, the classification results were output on the test set at the original image size and compared with mainstream models.

[0105] After multiple adjustments, the optimal network parameters are as follows: all images in the dataset are scaled to 224×224×3 RGB images; the batch size of the model is 8, and the number of iterations is 200; the network is initialized using the Xavier

[25] method, and the parameters are optimized using the Adam method. The initial learning rate is 1×10. -3 The optimizer momentum is set to 0.9, the decay coefficient to 5 × 10⁻⁴, and the temperature in knowledge distillation to 1.2; the training scheduler's λ is 0.5, k is 1, and the difficulty measurer's γ is 0.8. Other hyperparameters remain consistent with their default values.

[0106] To address the issues of model parameters easily getting trapped in local optima under training noise and overfitting simple samples in deep neural networks, random deactivation (Dropout) is used to optimize deep neural networks. Dropout originates from genetic algorithms in artificial intelligence, promoting the overall evolution of the entire population through random mutation. This method iteratively resets the outputs of hidden nodes to zero, updating only a subset of the model's parameters each time to reduce implicit co-adaptation between nodes and prevent a few high-weight nodes (local optima) from determining the output, ultimately shrinking the square norm of the weights. Dropout achieves regularization by simplifying the neural network structure and compressing model weights. Dropout is fused into the final fully connected layer of the donc layer, with the deactivation probability adjusted to 0.2; dropout is disabled during testing.

[0107] 4.3 Evaluation Indicators

[0108] Four commonly used evaluation metrics for classification models are: Precision (P), Recall (R), F1-score (F1), and Accuracy (Acc). Precision calculates the proportion of correctly predicted classes; recall calculates the proportion of correctly predicted actual classes; generally, an increase in precision leads to a decrease in recall, and vice versa, therefore F1-score is used to consider both precision and recall; accuracy considers the recognition accuracy across all classes, and is also referred to as classification accuracy in this section. The formulas for these four evaluation metrics are as follows:

[0109]

[0110]

[0111]

[0112]

[0113] In the formula, TP represents the number of correctly classified categories, FP represents the number of incorrectly classified categories, TN represents the number of other correctly classified categories, and FN represents the number of other incorrectly classified categories.

[0114] 5. Experimental Results and Analysis

[0115] A series of experiments were conducted on common classification models. The experimental results for each cassava leaf disease were analyzed and summarized in detail. The impact of different noise methods on the experimental results was compared and analyzed, providing useful references and insights for further optimization and improvement of the network structure.

[0116] 5.1 Experimental Results of Commonly Used Models

[0117] The experiments included: comparative analysis of six mainstream classification models of different sizes; comparative analysis of the accuracy performance of each class in the benchmark model; and comparative analysis of three different label noise processing methods. The training set used was the student dataset (row 3 of Table 1), and the test set was row 4 of Table 1. During training, the model accuracy was verified after each batch of data, and the model from the last batch of each training round was saved after each round. After training, the model with the highest accuracy was selected as the final model of the network. The training and experimental results of each model are as follows.

[0118] 5.1.1 Results of common model test sets

[0119] This experiment used six mainstream classification models of different sizes to predict the test set: lightweight models ShuffleNetV2 and MobileNetV3, conventional models ResNet and EfficientNet, and large-scale models ConvNext and ViT. ResNet utilized three model architectures: ResNet18, ResNet34, and ResNet50. Table 2 below shows the scores of each classification model on a student dataset with noisy labels. The results show that dataset noise affects all different models.

[0120] Table 2: Accuracy (%) of the classification model on the test set

[0121]

[0122] ViT's average precision, average recall, and accuracy were 79.30%, 82.98%, and 77.87%, respectively. Its F1 score of only 79.30% indicates poor performance, likely due to its high model complexity, making it difficult to fit small datasets containing noise. The significant difference between precision and recall highlights the importance of the F1 score and overall classification accuracy. ConvNeXt's F1 score was 4.97 percentage points higher than ViT, but its performance remained poor. The experiments used the ConvNeXt-Tiny version, but the model still required a large amount of training data. ShuffleNet, MobileViT, and MobileNet are all lightweight models with similar performance metrics. The poor experimental results suggest that smaller models are more susceptible to noisy data and may learn incorrect information. Although MobileNet's average precision and average recall were low, its F1 score of 84.41% indicates a better balance between accuracy and recall. ResNet18's metrics are slightly lower than ResNet34's, and its average recall is significantly lower than its average precision, indicating poor class discrimination and generalization ability. EfficientNet and ResNet34 perform similarly, with F1 scores both above 87%. Although EfficientNet's F1 score is higher than ResNet34's, ResNet34's average precision and average recall are more balanced. The ResNet50 model performs exceptionally well, with an accuracy of 89.63% and an average precision of 89.66%. However, its F1 score is 2.99 percentage points lower than the CCKD method, indicating that residual networks, due to their unique bottleneck structure and skip connections, can reduce learning forgetting and resist some noise. The CCKD method proposed in this invention performs best, with average precision, average recall, F1 score, and accuracy all exceeding 90%, at 91.95%, 91.07%, 91.44%, and 92.57% respectively, demonstrating a good balance between accuracy and recall. Thanks to the course knowledge distillation method, the CCKD of this invention achieved an F1 score 5.73 percentage points higher than the unmodified MobileViT.

[0123] 5.1.2 Specific accuracy performance of the CCKD method on the test set

[0124] Table 3 below shows the accuracy performance of the CCKD method of this invention on various categories in the test set. The model has the lowest precision and recall for the CBB category, with a recall of only 81.27%, indicating that many samples in this category are misclassified by the classification model as other categories. This is because the lesions of this type of disease are small and their shapes are not obvious, making them easy to mistake for Healthy. The early lesions of this type of disease are similar to those of CBS, and the model's efforts to learn this type have resulted in low recall. The model's F1 score for CBS is 91.18%, because the lesion appearance of this type varies greatly at different stages of infection, and the early symptoms are easily confused with CBB. The CGM and Healthy categories perform well in terms of precision and recall, but their F1 scores are lower than those of the CMD category. The CMD category performs best in all indicators, indicating that samples in this category are easier for the classification model to identify and distinguish. The lesions are more obvious and significantly different from other categories, so the model has high recognition accuracy for this type.

[0125] Table 3: Accuracy (%) of the CCKD method on the test set

[0126]

[0127] 5.1.3 Experimental Comparison Results of Different Noise Reduction Methods

[0128] Different label noise processing methods were analyzed, and four methods, namely model regularization, robust loss, meta-learning, and semi-supervised learning, were compared. The experimental comparison is shown in Table 4 below.

[0129] Table 4: Experimental comparison of different tag noise methods (%)

[0130]

[0131]

[0132] Table 4 shows that the experimental results of methods that handle noise by changing the loss function are generally poor; simply changing the loss function is insufficient to combat noise in the dataset. GCELoss, a robust loss method, has the lowest F1 score and accuracy, at 86.09% and 88.07%, respectively. This indicates that this method is susceptible to outliers, leading to a decrease in model accuracy. Labelsmoothing, a regularization method, has an average precision of 86.22%, an average recall of 88.72%, an F1 score of 87.15%, and an accuracy of 88.72%. This method learns more distribution information by smoothing the label distribution, but the distribution of the noisy dataset differs from that of the clean dataset, and the model does not learn accurate knowledge from this difference.

[0133] Co-teaching, a meta-learning method, performed well, with an average precision of 90.70%, an average recall of 88.30%, an F1 score of 89.29%, and an accuracy of 90.37%. This method uses two identical models. When the models struggle to distinguish between samples with similar diseases and noise samples, the model outputs tend to be consistent, co-training fails, and the model cannot continue learning. The DivideMix method achieved an F1 score of 90.95% and an accuracy of 91.77%. This is slightly lower than the CCKD method's F1 score by 0.49%, because it is suitable for situations with a large amount of noise. In this chapter's dataset, the proportion of noise is relatively small, leading to many samples being incorrectly labeled as noise samples, resulting in fewer training samples for the model when partitioning subsets. The CCKD method performed best, with an F1 score of 91.44% and an accuracy of 92.57%, indicating that the CCKD method can effectively resist the influence of noise to a certain extent and has a good ability to distinguish cassava leaf disease categories.

[0134] 5.2 Ablation Test

[0135] To investigate the improvement of the model by the method described in this section, ablation experiments were conducted on the main network using MobileViT for each improvement (Table 5 below). To explore the impact of different teacher models on the model, the MobileNetV3 network, Convnext, ViT, and ResNet were first trained on the teacher training set to evaluate the accuracy of the teacher models (Table 6 below). The three best teacher models were then selected to test student models of different sizes, and the experimental results are shown in Table 7 below.

[0136] 5.2.1 Experimental Results of the Improved Module

[0137] The CCKD method of this invention uses MobilViT as the base model, and tests the effectiveness of each module to verify the improved module. The results are shown in Table 5. The first row of the table contains experimental data for the base model, and the second, third, and fourth rows contain experimental data for adding knowledge distillation, self-adjusting weights, and training scheduler modules layer by layer, respectively. The following conclusions can be drawn:

[0138] (1) After using knowledge distillation, the model's F1 score improved by 1.85 percentage points, and the accuracy improved by 1.66 percentage points. The addition of the teacher model's guidance effectively improved the model's performance. The knowledge distillation method transfers feature knowledge to the student model through the teacher model, providing effective knowledge information and improving the recognition accuracy.

[0139] (2) After using the self-adjusting weight module, the model's F1 score increased by 2.24 percentage points and the accuracy increased by 2.56 percentage points. By adjusting the ratio of distillation loss to student loss, the proportion of knowledge transfer from the teacher model to the student model was changed, which effectively improved the self-learning ability of the student model and increased the classification accuracy of the categories.

[0140] (3) After adding the training scheduler module, the model's F1 score improved by 1.64 percentage points, and its accuracy improved by 1.29 percentage points. This demonstrates the effectiveness of the course learning, and that the order of sample input improves the model's learning performance.

[0141] Table 5: Comparison of Experimental Results for Different Modules

[0142]

[0143] 5.2.2 Effectiveness of the Course Knowledge Distillation Method

[0144] To verify the effectiveness of the course knowledge distillation method, multiple models were trained using the teacher dataset. The three models with the highest F1 scores in Table 6 were selected as teacher models, and the improvement of the student model by the CCKD method was compared between different teacher models and different student models.

[0145] Table 6: Accuracy (%) of each method on the test set after training with a clean dataset

[0146]

[0147] As shown in Table 6, MobileNet and ShuffleNet, both lightweight networks, have the lowest F1 scores across all metrics. ResNet50, ResNet18, and EfficientNet models suffer from low accuracy due to a lack of training data. In contrast, ResNet34, ConvNeXt, and ViT models have the highest F1 scores. These three models learned more implicit knowledge from clean datasets, so they were chosen as teacher models to train the smaller models. In the knowledge distillation framework, distilling the smaller models using teacher models improves their generalization ability and reduces the risk of overfitting. Considering both model size and the evaluation metrics in Table 2, ResNet18 and MobileViT were selected as the smaller models. Notably, ResNet is a pure CNN model, while MobileViT is a hybrid model of CNN and ViT, allowing us to explore the effects of distillation on different model architectures.

[0148] The three teacher models ResNet34, ConvNeXt, and ViT, along with the two student models ResNet18 and MobileViT, were trained using the CCKD method. The results are shown in Table 7 below.

[0149] Table 7: Comparison of experimental results of different models using the CCKD method

[0150]

[0151] Table 7 shows that the F1 scores of all models were improved after using the CCKD method of this invention. For the student model ResNet18, the F1 score improved by 2.87 percentage points when the teacher model was ConvNeXt, followed by ResNet34 and ViT, which improved by 2.53 and 0.98 percentage points respectively. This indicates that distilling a CNN model with another CNN model is more effective than distilling a CNN model with ViT. This is because the structural and parameter differences between the two models prevent proper matching and transmission of information during distillation, leading to information loss. For the student model MobileViT, the effect was even better when the teacher model was ResNet34, with an F1 score improvement of 5.73 percentage points, followed by ConvNeXt and ViT, which improved by 5.53 and 5.23 percentage points respectively. This shows that distillation using CNNs is generally more effective than distillation using ViTs; however, with limited data, the distillation effect of larger models is not as comprehensive as that of smaller models.

[0152] In summary, when using the CCKD method, the choice of knowledge distillation model should be determined based on the size of the dataset and the proportion of noisy data.

[0153] 5.3 Results Visualization

[0154] To more intuitively demonstrate the improvement in overall classification accuracy resulting from incorporating course knowledge distillation, a class activation mapping (SCAM) method is used to generate class activation maps to display the model's regions of interest. SCAM is a method used to interpret CNN classification results, visualizing the important features of each category in the neural network. This invention utilizes the Score-CAM method, which can more precisely determine the importance of feature regions and more accurately locate the decision regions for classification results. See also... Figure 6 The pixel values ​​on the heatmap represent the degree of influence on the classification results. The red area represents the area that the model pays more attention to, while the blue area represents the area that the model considers irrelevant to the classification.

[0155] Depend on Figure 6It can be seen that the CCKD method of this invention improves the model's recognition accuracy. For CBB disease, the previous model mainly focused on some lesions and prominent light and dark areas, failing to capture disease information effectively. In contrast, the improved model reduces interference from background and irrelevant features, focusing more on lesion areas. Furthermore, the improved model is more precise in identifying lesions such as CBS, CGM, and CMD. Even in the absence of lesions, this method can comprehensively and accurately identify whether each cotyledon is diseased. The results indicate that the CCKD method plays a crucial role in improving model performance.

[0156] As described above, this invention addresses the problem of cassava leaf disease image recognition on noisy datasets. Open-source cassava leaf disease datasets suffer from noise and uneven distribution, with small differences between different disease species. Therefore, a cassava leaf disease recognition method based on curriculum knowledge distillation is proposed. This method utilizes a teacher model to acquire feature knowledge from clean samples, then distills the data according to the difficulty of the samples and feeds them into a student model for image recognition. This supervisedly improves the generalization ability of the student network, thereby constructing a noise-resistant cassava leaf disease classification model. Experiments compared the classification performance of the classic model and the noisy model, and also compared the classification performance of models with and without CCKD on the test set. Experimental results show that the method combined with CCKD significantly improves overall accuracy, achieving an F1 score of 91.44% and an accuracy of 92.57%.

[0157] like Figure 7 As shown, a noise-tolerant cassava leaf disease classification system based on curriculum knowledge distillation can be constructed according to this classification method. This system includes: a training module: used to train a well-trained student model S using dataset D based on knowledge distillation and curriculum learning; the specific processing flow includes steps S11-S14; and a classification module: used to classify the cassava leaf disease image dataset to be classified using the trained student model S. This image dataset includes images of five categories: images of four diseases (cassava bacterial wilt, cassava brown streak, cassava green mottle, and cassava mosaic) and images of healthy cassava. For other details not covered herein, please refer to the aforementioned description of this classification method.

[0158] It should be noted that the examples of the above embodiments can preferably be combined with one or more of each other according to actual needs, and the accompanying drawings of multiple examples adopt a set of combined technical features, which will not be described in detail here.

[0159] The above description is a detailed explanation and illustration of the preferred embodiments of the present invention. However, these descriptions are not intended to limit the scope of protection claimed by the present invention. All equivalent changes or modifications made under the technical teachings of the present invention should fall within the patent protection scope covered by the present invention.

Claims

1. A noise-robust cassava leaf disease classification method based on curriculum knowledge distillation, characterized by, The method comprises the following steps: Step S1, training a trained student model S using a dataset D based on knowledge distillation and curriculum learning; The method comprises the following specific processing The method comprises the following steps: Step S11, assembling cassava leaf disease data as a noisy dataset D; Step S12, constructing a training scheduler, using the teacher model T as a difficulty measurer, inputting the data set D into it to obtain a sample difficulty score, then using a regularizer to determine the sample proportion and weight selected in each round of training, and selecting samples to form a training subset D m ; Step S13, constructing an adaptive weight module to adaptively adjust the knowledge transfer proportion ω of the teacher model T to the student model S, using the training subset D m The student model S of the current training is trained. Step S14, repeatedly performing the operations of steps S12-S13 until the target iteration number of training ends or the model tends to converge, and outputting the trained student model S; Step S2, classifying a to-be-classified cassava leaf disease image dataset using the trained student model S; the image dataset comprises five categories of images, which are four disease images of cassava bacterial wilt, cassava brown stripe disease, cassava green mottle disease and cassava mosaic disease, and a healthy cassava image; In step S1, the difficulty score l is calculated by sample cross-entropy loss, and the specific calculation formula is as follows, In the formula, y represents a true label of the sample, represents a probability that the model predicts the sample belongs to the positive class; The proportion of samples and the weight v selected in each round of training are determined by the regularizer * The weight v * The specific calculation formula is as follows, In the formula, k is a hyperparameter k>0 for controlling the sample weight intensity, and λ represents the learning speed. The value of the knowledge transfer proportion ω uses the validation set accuracy of each training phase as a measure of model gap; in the e-th training, the validation set accuracy of the student model is A S , the validation set accuracy of the teacher model is A T , and the calculation formula of the knowledge transfer proportion ω is as follows: In the formula, γ is a gradually automatic attenuation coefficient for controlling the proportion of knowledge distillation, and the value range is [0, 1).

2. The noise-robust cassava leaf disease classification method based on curriculum knowledge distillation according to claim 1, characterized in that: In step S1, the training scheduler specifically processes the following procedure: (1) In the initial stage, the original dataset D is input into the teacher model T to obtain the sample difficulty score l, and samples with difficulty scores lower than the score are selected. The samples form the first training subset D1; (2) In the mth stage, input the data set D into the difficulty measurer T to obtain the sample difficulty score l, and select samples with difficulty lower than the score λ to form the mth training subset D 2 m ;​ wherein D1∪D2∪…∪D m-1 = D m ; calculate and update the learning rate λ according to the current batch; (3) repeatedly performing the above steps (1) and (2) until the training ends or the model tends to converge.

3. A noise-robust cassava leaf disease classification system based on curriculum knowledge distillation, characterized by, The method comprises the following steps: The training module is used for training a trained student model S using a dataset D based on knowledge distillation and curriculum learning; The method comprises the following specific processing The method comprises the following steps: Step S11, assembling cassava leaf disease data as a noisy dataset D; Step S12, constructing a training scheduler, using the teacher model T as a difficulty measurer, inputting the data set D into it to obtain a sample difficulty score, then using a regularizer to determine the sample proportion and weight selected in each round of training, and selecting samples to form a training subset D m ; Step S13, constructing an adaptive weight module to adaptively adjust the knowledge transfer proportion ω of the teacher model T to the student model S, using the training subset D m The student model S of the current training is trained. Step S14, repeatedly performing the operations of steps S12-S13 until the target iteration number of training ends or the model tends to converge, and outputting the trained student model S; The classification module is used for classifying a to-be-classified cassava leaf disease image dataset using the trained student model S; the image dataset comprises five categories of images, which are four disease images of cassava bacterial wilt, cassava brown stripe disease, cassava green mottle disease and cassava mosaic disease, and a healthy cassava image; In the training module, the difficulty score l is calculated by sample cross-entropy loss, and the specific calculation formula is as follows, In the formula, y represents a true label of the sample, represents a probability that the model predicts the sample belongs to the positive class; The proportion of samples and the weight v selected in each round of training are determined by a regularizer * The weight v * The specific calculation formula is as follows, In the formula, k is a hyperparameter k>0 for controlling the sample weight intensity, and λ represents the learning speed. The value of the knowledge transfer proportion ω uses the validation set accuracy of each training phase as a measure of model gap; in the e-th training, the validation set accuracy of the student model is A S , the validation set accuracy of the teacher model is A T , and the calculation formula of the knowledge transfer proportion ω is as follows: In the formula, γ is a gradually automatic attenuation coefficient for controlling the proportion of knowledge distillation, and the value range is [0, 1).

4. The noise-robust cassava leaf disease classification system based on curriculum knowledge distillation of claim 3, wherein: In the training module, the training scheduler specifically processes the following procedure: (1) In the initial stage, the original dataset D is input into the teacher model T to obtain the sample difficulty score l, and samples with difficulty scores lower than the score are selected. The samples form the first training subset D1; (2) In the mth stage, input the data set D into the difficulty measurer T to obtain the sample difficulty score l, and select the samples with difficulty lower than the score λ to form the mth training subset D 2 m ;​ wherein D1∪D2∪…∪D m-1 = D m ; calculate and update the learning rate λ according to the current batch; (3) repeatedly performing the above steps (1) and (2) until the training ends or the model tends to converge.

Images