A rice disease image classification method and system based on knowledge distillation

Abstract

The invention discloses a rice disease image classification method based on knowledge distillation and a system thereof, and relates to the technical field of image classification. The method comprises the following steps: obtaining sample data; constructing a knowledge distillation network model; and the knowledge distillation network model adopts a convolutional neural network; constructing a teacher-student model; extracting sample features; constructing class prototype features; constructing similarity prediction; constructing a compact step-by-step distillation loss within a class; constructing an adaptive soft label; constructing an adaptive self-distillation loss; constructing a total loss function; training a network model of the current training stage; and real-time classification of rice disease images. The invention constructs a class probability at a feature level by mining the category relationship between teacher and student features of models in adjacent training stages, promotes the model to form a clustering effect in the feature space; adaptively combines the class probability output by the teacher model, the class probability at the feature level, and the true label, and generates a compact soft target within the class as a self-distillation soft target.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification, and in particular to a rice disease image classification method based on knowledge distillation and a system thereof. Background Art

[0002] Knowledge distillation is an effective technology for model compression and deepening deep neural networks. It adopts a teacher-student architecture and improves a small student network by extracting knowledge from a large-scale teacher with good performance. The goal is to combine the class probability distribution of the teacher network with the true label to form a soft label to supervise the student's prediction. Self-distillation is to extract knowledge from the middle layer of the network or the pre-training stage to promote teacher-free distillation, that is, to extract knowledge from itself to train itself without the assistance of other teacher models, thereby providing effective incremental information to the student model to improve the efficiency of the student model. Existing self-distillation work usually focuses on experimenting with result-driven distillation by minimizing the final prediction between the teacher and the student using various strategies, while these methods ignore the extraction of feature information that is also valuable for self-distillation and the transfer of its structured relationship. The existing BYOT method constructs an additional classifier to form an auxiliary branch to realize the transfer of the deepest classifier knowledge to the additional classifier, thereby enhancing the ability of the intermediate module. The Tf-KD model constructs a soft target with smooth labels to obtain a virtual teacher and benefit from it. Although there is a SAD method in self-distillation that attempts to perform top-down and layer-by-layer attention distillation inside the network to extract useful contextual information from the attention map for self-learning. However, this method learns the feature knowledge of different layers within the network from the instance itself. The model tends to learn trivial solutions that are independent of the object. These solutions are difficult to generalize and therefore easily cause feature confusion between classes.

[0003] Therefore, it can be seen that the knowledge distillation methods in the existing technology have limitations: 1) The training cost of suitable teachers. Traditional knowledge distillation adopts a two-stage distillation method, which requires additional training of teacher models, and at the same time causes a lot of training costs and storage requirements to find better teachers. 2) Failure to capture rich feature information. Specifically, the existing self-distillation method only extracts knowledge from its own model output prediction, while ignoring the valuable information hidden in the features: samples have strong similarities with samples of the same category, but are distinguishable from samples of different categories. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] In view of the problems existing in the above-mentioned existing rice disease image classification method based on knowledge distillation and its system, the present invention is proposed.

[0006] Therefore, the purpose of the present invention is to provide a rice disease image classification method and system based on knowledge distillation, which is to solve the problem that the existing self-distillation technology generally uses various strategies to minimize the final prediction between the teacher and student models for distillation. These methods ignore the extraction of feature information that is also valuable for self-distillation, and at the same time ignore the problem of more similar category relationships between similar samples.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In step S1, sample data is obtained; that is, image sample data and label data are obtained;

[0009] Step S2, constructing a knowledge distillation network model; and the knowledge distillation network model adopts a convolutional neural network;

[0010] Step S3, constructing a teacher-student model; using the network model in the current training phase as the student model and the network model in the previous training phase as the teacher model;

[0011] Step S4, extracting sample features; the preprocessed rice disease image is input into the convolutional neural network to obtain the features and logits output of the current training stage, that is, the intermediate features and logits output of the student model. Similarly, the features and logits output obtained in the previous training stage are the intermediate features and logits output of the teacher model.

[0012] Step S5, constructing class prototype features;

[0013] Step S6, constructing similarity prediction;

[0014] Step S7, constructing the intra-class compact stepwise distillation loss; calculating the cross entropy loss between the student's similarity prediction and the true label;

[0015] Step S8, constructing an adaptive soft label; softening the teacher logits to form the teacher's model prediction, and taking an adaptive strategy to combine it with the teacher's similarity prediction to form an integrated teacher's prediction, and adaptively adjusting the integrated teacher's prediction and the true label to form an adaptive soft label;

[0016] Step S9, construct an adaptive self-distillation loss; calculate the cross entropy loss of the adaptive soft labels and the student logits;

[0017] Step S10, constructing a total loss function; the total loss includes intra-class compact step-wise distillation loss and adaptive self-distillation loss;

[0018] Step S11, training the network model of the current training stage; using the total loss function to train the student model, and back-propagating to update the parameters of the student model to obtain a mature student model;

[0019] In step S12, the rice disease images are classified in real time; the real-time rice disease images are acquired and input into the student model, and the student model outputs the classification results.

[0020] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, wherein: in the S1 process, the acquired rice disease image sample data is preprocessed, wherein the preprocessing method includes: image cropping, scaling, and rotation.

[0021] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, wherein: in the S5 process, a class prototype feature is constructed, and the features of the same category in the output intermediate features are summed and averaged to obtain the prototype feature of each class. The prototype feature of the current stage is called the student class prototype feature, and the prototype feature of the previous stage is called the teacher class prototype feature. The calculation formula is:

[0022]

[0023] Among them, y i represents the label, f i represents the characteristics of the student model, I(·) is the function of label matching, when the label is k, its value is 1, otherwise it is 0; the class prototype features of the student model and the teacher model are constructed in the same way.

[0024] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, wherein: in the S6 process, a similarity prediction is constructed, specifically, a similarity vector is constructed between the intermediate features of the student and the prototype features of each class of the teacher, and the similarity vector is softened to form the similarity prediction of the student, that is;

[0025] First, construct the similarity between the sample features and each class prototype;

[0026]

[0027] Reconstruct similarity predictions;

[0028]

[0029] in, represents the class prototype feature of the teacher model, f i represents the characteristics of the student model, ||·|| is the L2 norm (Euclidean norm), and d i =[d i,1,d i,2 ,…,d i,K ] is the similarity vector of the i-th student sample, τ is the temperature hyperparameter, and K is the number of sample categories.

[0030] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, in the process of s7, a compact stepwise distillation loss within the class is constructed, and its calculation formula is as follows:

[0031]

[0032] in, represents the student similarity prediction.

[0033] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, wherein: in the S8 process, constructing an adaptive soft label mainly consists of constructing a teacher model prediction, constructing an integrated teacher prediction and constructing an adaptive soft label, and the specific construction process is as follows;

[0034] First, build the model prediction;

[0035]

[0036] Among them, z i =[z i,1 ,z i,2 ,…,z i,K ] indicates logits output, p i =[p i,1 ,p i,2 ,…,p i,K ] represents model prediction, and the model prediction of students and teachers is constructed in the same way;

[0037] Reconstruct the integrated teacher's predictions;

[0038]

[0039] Among them, w i is the weight for adaptively adjusting similarity prediction; represents the teacher similarity prediction; represents the teacher model prediction;

[0040] Reconstruct adaptive soft labels;

[0041] s=(1-ε m )y+ε m e;

[0042] Among them, ε m is the weight for adaptively adjusting the integrated teacher prediction in the current training phase.

[0043] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, wherein: in the process of step S9, an adaptive self-distillation loss is constructed,

[0044] The calculation method is: L ASD =H(s, p S );

[0045] Among them, p S is the student model prediction.

[0046] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, in which: in the S10 process, a total loss function is constructed, and its calculation method is the sum of the intra-class compact step-by-step distillation loss and the adaptive self-distillation loss, that is:

[0047] L total =L ASD +λ·L PICC .

[0048] As a preferred solution of the rice disease image classification method based on knowledge distillation described in the present invention, in the process of S11, the parameters of the student model are updated in the following specific manner:

[0049] Use the step total loss function to train the student model, back propagate to calculate the gradient, and update the network parameters. The parameters are:

[0050]

[0051] Among them, θ T represents the parameters of the teacher model, θ S represents the parameters of the student model, γ represents the learning rate, Represents the total loss function L total Compute the gradients of the student model parameters.

[0052] A system applied to the above-mentioned rice disease image classification method based on knowledge distillation, wherein: the system includes the following functional modules:

[0053] Rice disease image sample data acquisition module, used to obtain rice disease image sample data and label data; knowledge distillation network model construction module, used to construct a knowledge distillation network model using a convolutional neural network; teacher-student model construction module, the network model in the current training stage is used as the student model, and the network model in the previous training stage is used as the teacher model;

[0054] Sample feature extraction module: The preprocessed rice disease images are input into the convolutional neural network to obtain the features and logits outputs of the current training stage, which are the intermediate features and logits outputs of the student model. Similarly, the features and logits outputs obtained in the previous training stage are the intermediate features and logits outputs of the teacher model.

[0055] The class prototype feature construction module sums and averages the features of the same category in the output intermediate features to obtain the prototype features of each class. The prototype features of the current stage are called student class prototype features, and the prototype features of the previous stage are called teacher class prototype features.

[0056] The similarity prediction building module builds a similarity matrix between the student’s intermediate features and the teacher’s prototype features for each class, and softens it to form the student’s similarity prediction;

[0057] The intra-class compact stepwise distillation loss building block computes the cross entropy loss between the student’s similarity predictions and the true labels;

[0058] The adaptive soft label building module softens the teacher's logits output to form the teacher's model prediction, and adopts an adaptive strategy to combine it with the teacher's similarity prediction to form the integrated teacher's prediction. The integrated teacher's prediction is adaptively adjusted with the true label to form an adaptive soft label;

[0059] Adaptive self-distillation loss building block that computes the cross entropy loss between the adaptive soft labels and the student logits;

[0060] The total loss function building module is used to build the total loss function. The total loss includes the intra-class compact step-wise distillation loss and the adaptive self-distillation loss.

[0061] The network model training module is used to train the network model of the current training stage using the total loss function, and back-propagate to update the parameters of the student model to obtain a mature network model of the current training stage;

[0062] The rice disease image real-time classification module is used to obtain real-time rice disease images and input the rice disease images into the network model trained in the current stage, and the student model outputs the classification results.

[0063] The beneficial effects of the present invention are as follows: by mining the category relationship between the teacher and student features of the models in adjacent training stages, the class probability at the feature level is constructed, and the model is promoted to form a clustering effect in the feature space. At the same time, the class probability, the feature-level class probability and the true label output by the teacher model are adaptively combined to generate a compact soft target within the class as a self-distillation soft target. As the training continues, the model is not only trained more compactly on the same category, but also can imitate the high performance of capable teachers. Moreover, the present invention performs gradual self-distillation training in the models in adjacent training stages, does not require additional teacher training, and has the advantages of low cost and high performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0065] Figure 1 This is an overall flow chart of a rice disease image classification method based on knowledge distillation proposed in the present invention. DETAILED DESCRIPTION

[0066] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0067] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0068] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0069] Reference Figure 1 , as an embodiment of the present invention, provides a rice disease image classification method based on knowledge distillation, the method comprising the following steps:

[0070] In step S1, sample data is obtained; that is, image sample data and label data are obtained, and the obtained rice disease image sample data is preprocessed, wherein the preprocessing method includes: image cropping, scaling, and rotation;

[0071] Step S2, constructing a knowledge distillation network model; and the knowledge distillation network model adopts a convolutional neural network;

[0072] Step S3, constructing a teacher-student model; using the network model in the current training phase as the student model and the network model in the previous training phase as the teacher model;

[0073] Step S4, extracting sample features; the preprocessed rice disease image is input into the convolutional neural network to obtain the features and logits output of the current training stage, that is, the intermediate features and logits output of the student model. Similarly, the features and logits output obtained in the previous training stage are the intermediate features and logits output of the teacher model.

[0074] Step S5, constructing class prototype features; the specific operation is: summing and averaging the features of the same category in the output intermediate features to obtain the prototype features of each class. The prototype features of the current stage are called student class prototype features, and the prototype features of the previous stage are called teacher class prototype features. The calculation formula is:

[0075]

[0076] Among them, y i represents the label, f i represents the characteristics of the student model, I(·) is the function of label matching, when the label is k, its value is 1, otherwise it is 0; the class prototype features of the student model and the teacher model are constructed in the same way;

[0077] Step S6, constructing similarity predictions; specifically, constructing similarity vectors from the student's intermediate features and the teacher's prototype features of each class, and softening them to form the student's similarity predictions, that is;

[0078] First, construct the similarity of sample features to each class prototype;

[0079]

[0080] Reconstruct similarity predictions;

[0081]

[0082] in, represents the class prototype feature of the teacher model, f i represents the characteristics of the student model, ||·|| is the L2 norm (Euclidean norm), and d i =[d i,1 ,d i,2 ,…,d i,K] is the similarity vector of the i-th student sample, τ is the temperature hyperparameter, and K is the number of sample categories

[0083] Step S7, construct the intra-class compact stepwise distillation loss; calculate the cross entropy loss between the student's similarity prediction and the true label, and the calculation formula is as follows:

[0084]

[0085] in, represents the student similarity prediction;

[0086] Step S8, constructing an adaptive soft label; softening the teacher logits to form the teacher's model prediction, and taking an adaptive strategy to combine it with the teacher's similarity prediction to form an integrated teacher's prediction, and adaptively adjusting the integrated teacher's prediction and the true label to form an adaptive soft label;

[0087] Specifically, constructing adaptive soft labels mainly consists of constructing teacher model predictions, constructing ensemble teacher predictions, and constructing adaptive soft labels. The specific construction process is as follows;

[0088] First, build the model prediction;

[0089]

[0090] Among them, z i =[z i,1 ,z i,2 ,…,z i,K ] indicates logits output, p i =[p i,1 ,p i,2 ,…,p i,K ] represents model prediction, and the model prediction of students and teachers is constructed in the same way;

[0091] Reconstruct the integrated teacher's predictions;

[0092]

[0093] Among them, w i is the weight for adaptively adjusting similarity prediction; represents the teacher similarity prediction; represents the teacher model prediction;

[0094] Reconstruct adaptive soft labels;

[0095] s=(1-ε m )y+ε m e;

[0096] Among them, ε mis the weight for adaptively adjusting the integrated teacher prediction in the current training phase.

[0097] Step S9, construct an adaptive self-distillation loss; calculate the cross entropy loss between the adaptive soft labels and the student logits, and construct an adaptive self-distillation loss.

[0098] The calculation method is: L ASD =H(s, p S );

[0099] Among them, p S is the student model prediction.

[0100] Step S10, constructing a total loss function; the total loss includes the intra-class compact step-by-step distillation loss and the adaptive self-distillation loss. The total loss function is calculated as the sum of the intra-class compact step-by-step distillation loss and the adaptive self-distillation loss, that is:

[0101] L total =L ASD +λ·L PICC .

[0102] Step S11, training the network model of the current training stage; using the total loss function to train the student model, and back-propagating, updating the parameters of the student model, obtaining a mature student model, and then updating the parameters of the student model, the specific method is as follows:

[0103] Use the step total loss function to train the student model, back propagate to calculate the gradient, and update the network parameters. The parameters are:

[0104]

[0105] Among them, θ T represents the parameters of the teacher model, θ S represents the parameters of the student model, γ represents the learning rate, Represents the total loss function L total Compute the gradients of the student model parameters.

[0106] In step S12, the rice disease images are classified in real time; the real-time rice disease images are acquired and input into the student model, and the student model outputs the classification results.

[0107] In addition, the present invention also discloses a system applied to the above-mentioned rice disease image classification method based on knowledge distillation, and the system includes the following functional modules:

[0108] The rice disease image sample data acquisition module is used to obtain rice disease image sample data and label data; the knowledge distillation network model construction module is used to construct the knowledge distillation network model using a convolutional neural network;

[0109] In the teacher-student model construction module, the network model in the current training stage is used as the student model, and the network model in the previous training stage is used as the teacher model. In the sample feature extraction module, the preprocessed rice disease images are input into the convolutional neural network to obtain the features and logits outputs of the current training stage, i.e., the intermediate features and logits outputs of the student model. Similarly, the features and logits outputs obtained in the previous training stage are the intermediate features and logits outputs of the teacher model.

[0110] The class prototype feature construction module sums and averages the features of the same category in the output intermediate features to obtain the prototype features of each class. The prototype features of the current stage are called student class prototype features, and the prototype features of the previous stage are called teacher class prototype features.

[0111] The similarity prediction building module constructs a similarity matrix between the student's intermediate features and the teacher's prototype features of each class, and softens it to form the student's similarity prediction; the intra-class compact step-by-step distillation loss building module calculates the cross entropy loss between the student's similarity prediction and the true label.

[0112] The adaptive soft label building module softens the teacher's logits output to form the teacher's model prediction, and adopts an adaptive strategy to combine it with the teacher's similarity prediction to form the integrated teacher's prediction. The integrated teacher's prediction is adaptively adjusted with the true label to form an adaptive soft label;

[0113] The adaptive self-distillation loss building module calculates the cross entropy loss between the adaptive soft labels and the student logits; the total loss function building module is used to construct the total loss function, which includes the intra-class compact step-by-step distillation loss and the adaptive self-distillation loss.

[0114] The network model training module is used to train the network model of the current training stage using the total loss function, and back-propagate to update the parameters of the student model to obtain a mature network model of the current training stage; the rice disease image real-time classification module is used to obtain real-time rice disease images, and input the rice disease images into the network model trained in the current stage, and the student model outputs the classification results.

[0115] In summary, by mining the category relationship between the teacher and student features of the models in adjacent training stages, the class probability at the feature level is constructed to promote the model to form a clustering effect in the feature space. At the same time, the class probability, feature-level class probability and true label output by the teacher model are adaptively combined to generate a compact soft target within the class as a self-distillation soft target. As the training continues, the model is not only trained more compactly on the same category, but also able to imitate the high performance of capable teachers. Moreover, the present invention performs gradual self-distillation training in the models in adjacent training stages, does not require additional teacher training, and has the advantages of low cost and high performance.

[0116] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A rice disease image classification method based on knowledge distillation, characterized in that: The method comprises the following steps: In step S1, sample data is obtained; that is, image sample data and label data are obtained; Step S2, constructing a knowledge distillation network model; and the knowledge distillation network model adopts a convolutional neural network; Step S3, constructing a teacher-student model; using the network model of the current training stage as the student model and the network model of the previous training stage as the teacher model; Step S4, extracting sample features; the preprocessed rice disease image is input into the convolutional neural network to obtain the features and logits output of the current training stage, that is, the intermediate features and logits output of the student model. Similarly, the features and logits output obtained in the previous training stage are the intermediate features and logits output of the teacher model. Step S5, constructing class prototype features; summing and averaging the features of the same category in the output intermediate features to obtain the prototype features of each class. The prototype features of the current stage are called student class prototype features, and the prototype features of the previous stage are called teacher class prototype features. The calculation formula is: Among them, y i represents the label, f i represents the characteristics of the student model, I(·) is the function of label matching, when the label is k, its value is 1, otherwise it is 0; the class prototype features of the student model and the teacher model are constructed in the same way; Step S6, construct similarity prediction; construct similarity vectors by combining the intermediate features of students with the prototype features of each class of teachers, and soften them to form similarity predictions of students, that is; Dynamic data acquisition method for 3D GIS and BIM modeling based on multi-source sensor fusion First, construct the similarity of sample features to each class prototype; Reconstruct similarity predictions; in, represents the class prototype feature of the teacher model, f i represents the characteristics of the student model, ||·|| is the L2 norm, and d i =[d i,1 ,d i,2 ,…,d i,K ] is the similarity vector of the i-th student sample, τ is the temperature hyperparameter, and K is the number of sample categories; Step S7, constructing the intra-class compact stepwise distillation loss; calculating the cross entropy loss between the student's similarity prediction and the true label; Step S8, constructing an adaptive soft label; softening the teacher logits to form the teacher's model prediction, and taking an adaptive strategy to combine it with the teacher's similarity prediction to form an integrated teacher's prediction, and adaptively adjusting the integrated teacher's prediction and the true label to form an adaptive soft label; Step S9, construct an adaptive self-distillation loss; calculate the cross entropy loss of the adaptive soft labels and the student logits; Step S10, constructing a total loss function; the total loss includes intra-class compact step-wise distillation loss and adaptive self-distillation loss; Step S11, training the network model of the current training stage; using the total loss function to train the student model, and back-propagating to update the parameters of the student model to obtain a mature student model; In step S12, the rice disease images are classified in real time; the real-time rice disease images are acquired and input into the student model, and the student model outputs the classification results.

2. The rice disease image classification method based on knowledge distillation according to claim 1, characterized in that: In the process S1, the acquired rice disease image sample data is preprocessed, wherein the preprocessing method includes: image cropping, scaling, and rotation.

3. The rice disease image classification method based on knowledge distillation according to claim 2 is characterized in that: In the s7 process, a compact stepwise distillation loss within a class is constructed, and its calculation formula is as follows: in, represents the student similarity prediction.

4. The rice disease image classification method based on knowledge distillation according to claim 3 is characterized by: In the S8 process, constructing an adaptive soft label mainly consists of constructing a teacher model prediction, constructing an integrated teacher prediction, and constructing an adaptive soft label. The specific construction process is as follows; First, build the teacher model prediction; Among them, z i =[z i,1 ,z i,2 ,…,z i,K ] indicates logits output, p i =[p i,1 ,p i,2 ,…,p i,K ] represents model prediction, and the model prediction of students and teachers is constructed in the same way; Reconstruct the integrated teacher's predictions; Among them, w i is the weight for adaptively adjusting similarity prediction; represents the teacher similarity prediction; represents the teacher model prediction; Reconstruct adaptive soft labels; s i =(1-e m )y i +e m ·e i ; Among them, ε m is the weight for adaptively adjusting the integrated teacher prediction in the current training phase.

5. The rice disease image classification method based on knowledge distillation according to claim 4, characterized in that: During step S9, an adaptive self-distillation loss is constructed. The calculation method is: L ASD =H(s, p S ); Among them, p S is the student model prediction.

6. The rice disease image classification method based on knowledge distillation according to claim 5, characterized in that: In the S10 process, a total loss function is constructed, which is calculated as the sum of the intra-class compact step-wise distillation loss and the adaptive self-distillation loss, that is: L total =L ASD +λ·L PICC .

7. The rice disease image classification method based on knowledge distillation according to claim 6, characterized in that: In the S11 process, the parameters of the student model are updated in the following manner: Use the step total loss function to train the student model, back propagate to calculate the gradient, and update the network parameters. The parameters are: Among them, θ T represents the parameters of the teacher model, θ S represents the parameters of the student model, γ represents the learning rate, Represents the total loss function L total Compute the gradients of the student model parameters.

8. A system applied to the rice disease image classification method based on knowledge distillation according to any one of claims 1 to 7, characterized in that: The system includes the following functional modules: A rice disease image sample data acquisition module is used to acquire rice disease image sample data and label data; Knowledge distillation network model construction module, used to construct the knowledge distillation network model using convolutional neural network; The teacher-student model construction module uses the network model of the current training phase as the student model and the network model of the previous training phase as the teacher model; Sample feature extraction module: The preprocessed rice disease images are input into the convolutional neural network to obtain the features and logits outputs of the current training stage, which are the intermediate features and logits outputs of the student model. Similarly, the features and logits outputs obtained in the previous training stage are the intermediate features and logits outputs of the teacher model. The class prototype feature construction module sums and averages the features of the same category in the output intermediate features to obtain the prototype features of each class. The prototype features of the current stage are called student class prototype features, and the prototype features of the previous stage are called teacher class prototype features. The similarity prediction building module builds a similarity matrix between the student’s intermediate features and the teacher’s prototype features for each class, and softens it to form the student’s similarity prediction; The intra-class compact stepwise distillation loss building block computes the cross entropy loss between the student’s similarity predictions and the true labels; The adaptive soft label building module softens the teacher's logits output to form the teacher's model prediction, and adopts an adaptive strategy to combine it with the teacher's similarity prediction to form the integrated teacher's prediction, and adaptively adjusts the integrated teacher's prediction with the true label to form an adaptive soft label; Adaptive self-distillation loss building block that computes the cross entropy loss between the adaptive soft labels and the student logits; The total loss function building module is used to build the total loss function. The total loss includes the intra-class compact step-wise distillation loss and the adaptive self-distillation loss. The network model training module is used to train the network model of the current training stage using the total loss function, and back-propagate to update the parameters of the student model to obtain a mature network model of the current training stage; The rice disease image real-time classification module is used to obtain real-time rice disease images and input the rice disease images into the network model trained in the current stage, and the student model outputs the classification results.

Images