A crop leaf disease identification method based on a teacher-student model multi-level knowledge distillation

Abstract

The application discloses a crop leaf disease identification method based on a teacher-student model multi-level knowledge distillation, and is characterized in that the method comprises the following steps: constructing a crop leaf disease image dataset and performing data preprocessing; constructing a cross-modal teacher model comprising an image encoder and a text encoder; constructing a lightweight student model and performing multi-level knowledge distillation training; and using the trained student model to identify crop leaf diseases. The application introduces a cross-modal feature alignment module and a contrast learning mechanism to constrain the feature space of the student model, thereby enhancing the model's ability to distinguish between fine-grained disease categories and improving the recognition performance and robustness. The application adopts a multi-level knowledge distillation strategy combining output layer distillation and feature layer distillation, and combines a progressive distillation mechanism and a dynamic weight adjustment mechanism to improve the learning efficiency of the student model for the knowledge of the teacher model.

Description

Technical Field

[0001] This invention relates to the field of intelligent identification of crop leaf diseases and computer vision technology, and in particular to a method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model. Background Technology

[0002] Leaf diseases in crops can lead to decreased photosynthetic efficiency, reduced nutrient accumulation, and yield losses. In severe cases, they can even cause disease spread and large-scale losses. Rapid and accurate disease identification is crucial for disease monitoring, early warning, and precision pesticide application. It helps identify the control window early, reduces the risk of pesticide overuse and control costs, and improves the intelligence level of field management. Traditional plant disease identification methods typically rely on expert experience or manual feature extraction, such as color, texture, and shape features, combined with classification algorithms like support vector machines and random forests. However, these methods are heavily reliant on human experience, have complex feature designs, and suffer from low accuracy in complex environments, making them unsuitable for the needs of modern intelligent agricultural management. With the widespread use of mobile terminals and the development of computer vision technology, automatic identification based on leaf images has gradually become an important research direction for crop disease identification. Deep learning models can learn image features end-to-end and typically achieve high classification accuracy under controlled acquisition conditions. However, when test images come from real field environments or differ from the distribution of training data, model performance often drops significantly, reflecting a clear lack of generalization when relying solely on a single visual modality in cross-domain scenarios. Meanwhile, large-scale deep models introduced to improve recognition accuracy typically have high computational overhead, making low-latency deployment on mobile terminals and field edge devices difficult. Recent cross-modal methods, such as visual language models, can incorporate semantic information like disease names and symptom descriptions to enhance model representation capabilities; however, these models are structurally complex, resource-intensive, and often struggle to reliably obtain text input corresponding to the image during the actual inference phase, thus limiting their direct application in edge scenarios.

[0003] Regarding patented technologies, Chinese patent CN110929610A proposes a method for plant disease identification using convolutional neural networks. This method extracts deep features from leaf images and classifies and identifies diseases, thereby achieving automatic detection of plant diseases. This technology demonstrates that deep learning methods can effectively improve the accuracy of plant disease identification and reduce the workload of manual feature design. However, the deep neural network structure used in this method has a large parameter scale, requiring significant computing resources and storage space. Deployment on resource-constrained edge devices such as agricultural IoT terminals or mobile devices remains challenging.

[0004] To reduce the computational overhead of deep learning models in practical deployments, researchers have begun exploring ways to improve disease identification performance by refining network structures or optimizing training strategies. Chinese patent CN113610163B proposes a deep learning-based method for crop disease identification, which improves model recognition performance through improvements to network structure and training strategies. However, this method primarily relies on visual image information for identification, with limited utilization of semantic information about diseases. Furthermore, the model structure remains relatively complex, posing challenges for deployment on resource-constrained edge devices.

[0005] In summary, while existing crop disease identification methods have made some progress in terms of identification accuracy and model lightweighting, the following problems still exist: First, most methods rely solely on single-modal image information for feature learning, lacking effective utilization of disease semantic information; second, while reducing computational complexity, lightweight models often weaken the model's feature representation capabilities. Therefore, how to improve the feature representation capabilities while ensuring model lightweighting, and to realize a crop leaf disease identification method suitable for deployment on edge devices, remains an urgent problem to be solved. Summary of the Invention

[0006] Purpose of the invention: The purpose of this invention is to provide a method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model.

[0007] Technical Solution: The present invention provides a method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model, characterized by the following steps:

[0008] Step 1: Construct a dataset of crop leaf disease images and perform data preprocessing;

[0009] Step 2: Construct a cross-modal teacher model that includes an image encoder and a text encoder;

[0010] Step 3: Construct a lightweight student model and perform multi-level knowledge distillation training;

[0011] Step 4: Use the trained student model to identify leaf diseases in crops.

[0012] Further, step 1 includes:

[0013] Step 1.1: Acquire raw images of crop leaves using a mobile phone camera, industrial camera, or visible light camera from a drone;

[0014] Step 1.2: Convert the input image to RGB three channels uniformly; if the original image is BGR, perform channel rearrangement;

[0015] Step 1.3: Size normalization and cropping: Scale and crop the RGB image to the preset input size H×W=224×224 to obtain an image with uniform size;

[0016] Step 1.4: Linearly normalize the image pixel values ​​from 0 to 255 to 0 to 1, and perform mean-variance normalization; then convert the image from H×W×C format to C×H×W format.

[0017] Furthermore, step 2, constructing the cross-modal teacher model, includes:

[0018] Step 2.1: Construct an image encoder. ShuffleNetV2 is used as the backbone network. The preprocessed image obtained in Step 1 is input into the image encoder. Through multi-layer convolution operations and nonlinear mapping, texture features, shape features and lesion structure features in the image are extracted layer by layer to obtain image feature representations that can characterize leaf disease information.

[0019] Step 2.2: Using a pre-trained visual-language model, the pre-processed crop leaf images are input into the model to generate a structured disease text description that includes the overall distribution features of the disease, the morphological features of local lesions, and the color and texture features. The resulting text feature sequence is then input into a pre-trained text encoder for semantic encoding.

[0020] Step 2.3: Input the visual features output by the image encoder and the semantic features output by the text encoder into the cross-modal fusion module. Through feature mapping, cross-attention mechanism or gating fusion method, feature interaction between visual information and semantic information is realized, so that image features can be combined with disease semantic information to express, thereby obtaining the fused cross-modal feature representation.

[0021] Step 2.4: Input the fused cross-modal features into the classification prediction layer and output the category prediction results of crop leaf diseases. Through continuous optimization of the teacher model parameters during the training process, the teacher model can simultaneously utilize image features and text semantic features for disease identification and generate high-quality prediction results and feature representations to guide the training of student models.

[0022] Further, step 3 includes:

[0023] Step 3.1: Construct a student model that relies solely on image input to extract visual features from leaf images and complete the disease classification task. The student model uses a lightweight convolutional neural network, ShuffleNetV2, as the image feature extraction module. The input image is processed by the image feature extraction module of the student model to obtain image feature representation, which serves as the basis for subsequent semantic feature simulation and feature fusion.

[0024] Step 3.2: Construct a multi-scale semantic feature simulation module to generate simulated semantic feature representations corresponding to the semantic feature space structure through image feature transformation. During the training phase, knowledge distillation is performed on the student model using cross-modal semantic features provided by the teacher model, so that the simulated semantic features gradually approach the real semantic representation in the teacher model.

[0025] Step 3.3: Input the image features extracted from the student model and the semantic features generated by the semantic feature simulation module into the cross-modal feature alignment module;

[0026] Step 3.4: Introduce a contrastive learning mechanism during the student model training process to constrain the feature space of the student model;

[0027] Step 3.5: During the training of the student model, the cross-modal teacher model trained in Step 2 is introduced as the guidance network, and the discriminative knowledge in the teacher model is transferred to the student model through a multi-level knowledge distillation strategy.

[0028] Step 3.6: During the distillation training process, the parameters of the student model are updated by jointly optimizing the classification loss and distillation loss of the student model, so that the student model can learn the discriminative knowledge in the teacher model while learning the real label information.

[0029] Further, step 3.1 includes inputting the preprocessed image obtained in step 1 into the image feature extraction module of the student model, performing dimensional transformation through the feature mapping layer to map it to a unified feature space. The mapped image features can be represented as a Batch×d feature matrix, where Batch represents the number of input samples, d represents the feature dimension, and d is 256.

[0030] Further, step 3.2 includes:

[0031] 3.2.1: Input the image features extracted by the student model into the semantic feature simulation module, and then send the image features into feature mapping branches of multiple scales for parallel processing;

[0032] The fine-grained feature mapping branch is used to extract local detail information in image features. It transforms the input features from Batch×d to Batch×(d / 2) through the dimensionality reduction mapping Linear(d, d / 2), simulating the enhancement of lesion texture and edge detail information.

[0033] The mesoscale feature mapping branch is used to extract mid-level semantic information from image features. By maintaining the feature dimension unchanged, the Linear(d, d) mapping structure keeps the input features as Batch×d, thus simulating the expression of the structural information of the disease area.

[0034] The coarse-grained feature mapping branch is used to extract global semantic information from image features. By using the up-dimensional mapping Linear(d, 2d), the input features are expanded from Batch×d to Batch×(2d), which simulates and enhances the overall disease semantics.

[0035] 3.2.2: The features obtained from the three scale branches are concatenated and combined so that the fused features simultaneously contain fine-grained local information, mid-level structural information, and global semantic information. The concatenated multi-scale feature dimension is Batch×(3.5d). Then, it is remapped to the unified feature space Batch×d through the feature fusion layer to obtain the simulated semantic feature representation. The feature fusion layer includes a linear mapping Linear(3.5d, d)), a non-linear activation function GELU, and a normalization layer LayerNorm.

[0036] Further, step 3.3 includes:

[0037] 3.3.1: Stack the image features Batch×d with the simulated semantic features Batch×d to form a cross-modal feature sequence Batch×2×d;

[0038] 3.3.2: The cross-modal feature sequence is input into the cross-attention module. The correlation weight between visual features and semantic features is calculated through a lightweight multi-head attention mechanism, so that image features can perceive semantic information and semantic features can perceive visual structural information, thereby realizing cross-modal information interaction. After the cross-attention is completed, the features are transformed and enhanced through residual connections and feedforward mapping networks.

[0039] 3.3.3: The obtained feature sequence is subjected to feature aggregation processing. By weighted aggregation of the sequence dimensions, the feature dimension is mapped from Batch×2×d to Batch×d, thereby obtaining the fused cross-modal feature representation, and the features are used as input for the student model's subsequent classification prediction and distillation learning.

[0040] Further, step 3.5 includes:

[0041] 3.5.1: The same input image is simultaneously input into the teacher model and the student model for forward computation. The teacher model outputs cross-modal fusion feature representation and disease category prediction results, while the student model obtains fusion feature representation through semantic feature simulation and cross-modal alignment module;

[0042] 3.5.2: The fusion feature representation output by the teacher model is represented as Batch×d, and the fusion feature representation obtained by the student model through the cross-modal feature alignment module is also represented as Batch×d;

[0043] 3.5.3: Guided training of student models is conducted using a multi-level distillation strategy. The distillation process includes output layer distillation and feature layer distillation.

[0044] Output layer distillation uses the class probability distribution output by the teacher model as a soft label to guide the student model to learn the classification decision information of the teacher model, enabling the student model to obtain a more stable class discrimination ability. Feature layer distillation aligns the fused feature representations generated by the teacher model and the student model in the feature extraction stage, enabling the student model to learn the cross-modal semantic discrimination information contained in the teacher model.

[0045] Further, step 4 includes:

[0046] Step 4.1: Input the crop leaf image to be identified into the student model, extract the high-level visual features of the image through the convolutional feature extraction network, and convert it into an image feature representation in a unified feature space through the feature mapping layer. The image feature representation is a Batch×d feature matrix, where Batch represents the number of input samples, d represents the feature dimension, and d is 256.

[0047] Step 4.2: Input the image features into the semantic feature simulation module, generate simulated semantic features through multi-scale feature mapping. During the training phase, the student model is supervised by distillation through the cross-modal semantic features provided by the teacher model. During the inference phase, even without inputting text information, the student model can still generate feature representations with semantic discriminative ability.

[0048] Step 4.3: Input the image features and simulated semantic features into the cross-modal feature alignment module, and realize the feature interaction between visual information and semantic information through a lightweight attention mechanism, thereby obtaining the fused cross-modal feature representation.

[0049] Step 4.4: Input the fused features into the classification prediction layer to output the probability of crop leaf disease categories, and determine the disease category according to the principle of maximum probability to realize the identification of crop leaf diseases.

[0050] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0051] (1) By constructing a cross-modal teacher model, image information and text semantic information are used for joint modeling during the training phase, thereby obtaining richer semantic feature expression capabilities for diseases and improving the accuracy of crop leaf disease identification;

[0052] (2) Construct a semantic feature simulation module in the student model and combine it with a cross-modal feature alignment mechanism so that the student model can learn the semantic discrimination ability in the teacher model without inputting text information during the reasoning stage, thereby realizing the effective transfer of cross-modal knowledge to the single-modal model.

[0053] (3) By introducing a cross-modal feature alignment module and a contrastive learning mechanism, the feature space of the student model is constrained, thereby enhancing the model's ability to distinguish fine-grained disease categories and improving recognition performance and robustness.

[0054] (4) A multi-level knowledge distillation strategy combining output layer distillation and feature layer distillation is adopted, and progressive distillation and dynamic weight adjustment mechanism are combined to improve the learning efficiency of student model on teacher model knowledge.

[0055] (5) By adopting a lightweight student model and retaining only the student model for identification during the inference stage, the computational complexity of the model and the inference latency are reduced, enabling the method to be efficiently deployed on edge devices such as mobile terminals and agricultural IoT. Attached Figure Description

[0056] Figure 1 This is a schematic diagram of the overall process of the method described in this invention;

[0057] Figure 2 This is a schematic diagram of the cross-modal teacher model structure of the present invention;

[0058] Figure 3 This is a schematic diagram of the single-modal student model structure;

[0059] Figure 4 This is a schematic diagram showing the comparative experimental results between the traditional distillation method and the present invention;

[0060] Figure 5 This is a schematic diagram of the ablation experiment results. Detailed Implementation

[0061] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0062] like Figure 1 As shown, the crop leaf disease identification method based on multi-level knowledge distillation using a teacher-student model according to the present invention includes the following steps:

[0063] Step 1: Construct a crop leaf disease image dataset and perform data preprocessing. This includes image acquisition, color space processing, size normalization, data augmentation, and pixel normalization. The logic is as follows: first, unify the original leaf images from different sources and with different resolutions to a fixed input size and numerical range so that the encoder can stably extract features later.

[0064] Step 1.1: Image Acquisition: Acquire raw images of crop leaves using a mobile phone camera, industrial camera, or visible light camera from a drone. The image format can be JPG, PNG, or BMP, and the original resolution is not limited.

[0065] Step 1.2: Convert the input image to RGB three channels uniformly; when the original image is BGR, perform channel rearrangement.

[0066] Step 1.3: Size normalization and cropping: Scale and crop the RGB image to the preset input size H×W=224×224 to obtain an image with uniform size.

[0067] Step 1.4: Pixel normalization: Linearly normalize the image pixel values ​​from 0 to 255 to 0 to 1, and perform mean-variance normalization; then convert the image from H×W×C format to C×H×W format.

[0068] Step 2: Construct a cross-modal teacher model containing an image encoder and a text encoder. This includes image encoder construction, text encoder construction, and cross-modal feature fusion. The logic is as follows: the image encoder extracts visual features from leaf images, while the text encoder extracts semantic features corresponding to disease category names or symptom descriptions. Then, the cross-modal fusion module interactively fuses the visual and semantic features to construct a cross-modal teacher model that can utilize both image and semantic information. Specifically, this includes:

[0069] Step 2.1: Image Encoder Construction: An image encoder is constructed, using ShuffleNetV2 as the backbone network. The preprocessed image obtained in Step 1) is input into the image encoder. Through multi-layer convolution operations and nonlinear mapping, texture features, shape features, and lesion structure features in the image are extracted layer by layer to obtain an image feature representation that can characterize leaf disease information.

[0070] Step 2.2: Text Encoder Construction: A pre-trained vision-language model (selecting a large multimodal model capable of understanding images and outputting Chinese descriptions, such as GLM-4.6V) is used. The pre-processed crop leaf images are input into the model to generate structured text descriptions of the diseases, including overall disease distribution features, local lesion morphology features, and color and texture features. The resulting text feature sequence is then input into a pre-trained text encoder (such as BLIP2) for semantic encoding.

[0071] Step 2.3: Cross-modal feature fusion: Input the visual features output by the image encoder and the semantic features output by the text encoder into the cross-modal fusion module. Through feature mapping, cross-attention mechanism or gating fusion method, feature interaction between visual information and semantic information is realized, so that image features can be combined with disease semantic information to express, thereby obtaining the fused cross-modal feature representation.

[0072] Step 2.4: Teacher Model Construction: The fused cross-modal features are input into the classification prediction layer, which outputs the category prediction results of crop leaf diseases. The teacher model parameters are continuously optimized through the training process, enabling the teacher model to simultaneously utilize image features and text semantic features for disease identification, and generate high-quality prediction results and feature representations to guide student model training.

[0073] Step 3: Construct a lightweight student model and perform multi-level knowledge distillation training. This includes constructing a lightweight student model, building a semantic feature simulation module, contrastive learning feature constraints, and multi-level knowledge distillation training. The overall logic is as follows: By constructing a lightweight student model that relies solely on image input, and using the cross-modal teacher model trained in Step 2 as the guiding network, the visual semantic knowledge in the teacher model is transferred to the student model. This allows the student model to achieve recognition performance close to that of the teacher model even without requiring text input during the inference stage.

[0074] Step 3.1: Lightweight Student Model Construction: A student model that relies solely on image input is constructed to extract visual features from leaf images and complete the disease classification task. The student model uses a lightweight convolutional neural network, ShuffleNetV2, as its image feature extraction module to reduce the model parameter size and computational overhead. The input image, after passing through the image feature extraction module of the student model, yields an image feature representation, which serves as the basis for subsequent semantic feature simulation and feature fusion.

[0075] First, the preprocessed image obtained in step 1 is input into the image feature extraction module of the student model. A feature mapping layer performs dimensionality transformation, mapping the image to a unified feature space. The mapped image features can be represented as a Batch×d feature matrix, where Batch represents the number of input samples, and d represents the feature dimension, which is 256.

[0076] Step 3.2: Construction of a Multi-Scale Semantic Feature Simulation Module: During the inference phase, the student model only receives leaf images as input, without a text encoder or text input. Therefore, it cannot directly obtain the semantic features obtained from the image-text joint encoding in the teacher model. To enable the student model to learn feature representations similar to the teacher model's semantic space using only visual information, this invention constructs a multi-scale semantic feature simulation module in the student model. This module generates simulated semantic feature representations corresponding to the semantic feature space structure through image feature transformation. During the training phase, knowledge distillation is performed on the student model using cross-modal semantic features provided by the teacher model, gradually approximating the true semantic representation in the teacher model. This achieves effective transfer of semantic information from the cross-modal teacher model to the unimodal student model.

[0077] The semantic feature simulation module adopts a multi-scale feature mapping structure, which consists of a fine-grained feature mapping branch, a medium-scale feature mapping branch, a coarse-grained feature mapping branch, and a feature fusion layer.

[0078] First, the image features extracted by the student model are input into the semantic feature simulation module, and then the image features are sent to feature mapping branches at multiple scales for parallel processing.

[0079] The fine-grained feature mapping branch is used to extract local detail information in image features. It transforms the input features from Batch×d to Batch×(d / 2) through dimensionality reduction mapping (Linear(d, d / 2)) to simulate and enhance the texture and edge details of lesions.

[0080] The mesoscale feature mapping branch is used to extract mid-level semantic information from image features. By maintaining the feature dimension unchanged by the mapping structure (Linear(d, d)), the input features are kept as Batch×d, which simulates the expression of the structural information of the disease area.

[0081] The coarse-grained feature mapping branch is used to extract global semantic information from image features. By using the dimension-upgrading mapping (Linear(d, 2d)), the input features are expanded from Batch×d to Batch×(2d), which simulates and enhances the overall disease semantics.

[0082] Subsequently, the features obtained from the three scale branches are concatenated and combined, so that the fused features simultaneously contain fine-grained local information, mid-level structural information, and global semantic information. The concatenated multi-scale feature dimension is Batch×(3.5d), which is then remapped to a unified feature space Batch×d through the feature fusion layer, thereby obtaining a simulated semantic feature representation. The feature fusion layer consists of a linear mapping (Linear(3.5d, d)), a non-linear activation function (GELU function), and a normalization layer (LayerNorm).

[0083] Step 3.3: Cross-modal feature alignment module: Input the image features Batch×d extracted by the student model and the semantic features Batch×d generated by the semantic feature simulation module into the cross-modal feature alignment module, such as... Figure 2 As shown.

[0084] First, the image features Batch×d and the simulated semantic features Batch×d are stacked to form a cross-modal feature sequence Batch×2×d.

[0085] The cross-modal feature sequence is then input into the cross-attention module, where a lightweight multi-head attention mechanism is used to calculate the correlation weights between visual and semantic features. This allows image features to perceive semantic information, while semantic features can perceive visual structural information, thus achieving cross-modal information interaction. After cross-attention is completed, residual connections and feedforward mapping networks are used to further transform and enhance the features to improve their expressive power and maintain training stability.

[0086] Finally, the obtained feature sequences are subjected to feature aggregation processing. By weighted aggregation of the sequence dimensions, the feature dimensions are mapped from Batch×2×d to Batch×d, thereby obtaining the fused cross-modal feature representation, which is then used as the input for the student model's subsequent classification prediction and distillation learning.

[0087] Step 3.4: Contrastive Learning Feature Constraints: To further enhance the discriminative ability of the student model, a contrastive learning mechanism is introduced during the training process of the student model to constrain the feature space of the student model.

[0088] First, the fused features output by the student model are used as feature representations. By constructing positive and negative sample pairs, the distance between samples of the same category in the feature space is made closer, and the distance between samples of different categories is made farther, thereby improving the model's ability to distinguish between disease categories.

[0089] By contrastive learning constraints, the discriminative structure of the student model's feature space can be further enhanced, thereby improving the model's generalization ability.

[0090] Step 3.5: Multi-level knowledge distillation: During the training of the student model, the cross-modal teacher model trained in step 2) is introduced as the guidance network. The discriminative knowledge in the teacher model is transferred to the student model through a multi-level knowledge distillation strategy, so that the student model can achieve recognition performance close to that of the teacher model while maintaining low computational complexity.

[0091] First, the same input image is simultaneously fed into both the teacher model and the student model for forward computation. The teacher model outputs a cross-modal fusion feature representation and a disease category prediction result, while the student model obtains the fusion feature representation through semantic feature simulation and the cross-modal alignment module.

[0092] The fusion feature representation output by the teacher model can be represented as Batch×d, and the fusion feature representation obtained by the student model through the cross-modal feature alignment module is also Batch×d, thus ensuring that the two features are in a unified feature space.

[0093] Subsequently, the student model was trained using a multi-level distillation strategy. The distillation process includes output layer distillation and feature layer distillation:

[0094] Output layer distillation uses the class probability distribution output by the teacher model as a soft label to guide the student model to learn the classification decision information of the teacher model, enabling the student model to obtain a more stable class discrimination ability. Feature layer distillation aligns the fused feature representations generated by the teacher model and the student model in the feature extraction stage, enabling the student model to learn the cross-modal semantic discrimination information contained in the teacher model, thereby improving the student model's ability to distinguish disease categories.

[0095] Through a multi-level distillation process, the student model can gradually approximate the feature representation space and classification decision-making behavior of the teacher model, such as... Figure 3 As shown.

[0096] Step 3.6: Progressive distillation training: During the distillation training process, the parameters of the student model are updated by jointly optimizing the classification loss and distillation loss of the student model, so that the student model learns the discriminative knowledge in the teacher model while learning the real label information.

[0097] During training, a progressive distillation strategy and a dynamic weight adjustment mechanism are employed. In each training iteration, the classification loss is calculated by combining the classification prediction results of the student model with the true labels, while the distillation loss is calculated based on the class probability distribution output by the teacher model and the fused feature representation. The classification loss and distillation loss are then weighted and combined to form the training objective function.

[0098] A progressive distillation strategy is employed to regulate the distillation process during training. In the early stages of training, the weight of output layer distillation is increased to allow the student model to prioritize learning the overall classification decision information from the teacher model. As training progresses, the weight of feature layer distillation is gradually increased to enhance the student model's ability to align across the cross-modal semantic feature space.

[0099] By employing the distillation training strategy described above, the student model achieves stronger feature representation capabilities while maintaining a lightweight structure, significantly improving the accuracy and stability of crop leaf disease identification. The final result is a lightweight student model that, during the inference phase, only requires crop leaf images as input to complete the disease identification task, thus enabling efficient deployment on mobile terminals and edge devices.

[0100] Step 4: Use the trained student model to identify crop leaf diseases. In the model deployment and practical application stage, only the lightweight student model trained in Step 3 is retained, and the cross-modal teacher model, text input module, multi-level distillation module, and contrastive learning loss module used in the training stage are removed, thereby constructing a disease identification model that relies solely on image input.

[0101] The student model used in the inference phase includes an image feature extraction module, a multi-scale semantic feature simulation module, a cross-modal feature alignment module, and a classification prediction layer. Specifically, the image feature extraction module extracts visual features from leaf images, the multi-scale semantic feature simulation module generates simulated semantic feature representations based on image features, the cross-modal feature alignment module enables interactive fusion between image features and simulated semantic features, and the classification prediction layer outputs the disease category prediction results.

[0102] Step 4.1: Input the crop leaf image to be identified into the student model, extract the high-level visual features of the image through the convolutional feature extraction network, and convert it into an image feature representation in a unified feature space through the feature mapping layer. The image feature representation is a Batch×d feature matrix, where Batch represents the number of input samples, d represents the feature dimension, and d is 256.

[0103] Step 4.2: Input the image features into the semantic feature simulation module to generate simulated semantic features through multi-scale feature mapping. It should be noted that during the training phase, the student model has already undergone distillation supervision using cross-modal semantic features provided by the teacher model, allowing the simulated semantic features to gradually approximate the semantic representation in the teacher model. Therefore, even without inputting text information during the inference phase, the student model can still generate feature representations with semantic discriminative capabilities.

[0104] Step 4.3: Input the image features and simulated semantic features into the cross-modal feature alignment module, and realize the feature interaction between visual information and semantic information through a lightweight attention mechanism, thereby obtaining the fused cross-modal feature representation.

[0105] Step 4.4: Input the fused features into the classification prediction layer to output the probability of crop leaf disease categories, and determine the disease category according to the principle of maximum probability to realize the identification of crop leaf diseases.

[0106] Therefore, during the inference phase, this invention can complete the task of identifying crop leaf diseases by relying solely on the trained lightweight student model, without the need for a teacher model to participate in the calculation or additional text information input. This significantly reduces the model parameter size and computational complexity, enabling the method to be efficiently deployed on mobile terminals, agricultural IoT devices, and other resource-constrained edge devices.

[0107] The technical solution of the present invention will be further described in detail below with reference to specific examples.

[0108] In this embodiment, the publicly available soybean leaf disease dataset (Soybean Disease dataset) is selected to verify the method of the present invention. This dataset consists of field-captured images of soybean leaves suffering from diseases, including eight disease categories: bacterial leaf blight, brown spot, downy mildew, frog-eye leaf spot, target spot, soybean rust, potassium-deficient leaves, and healthy leaves, totaling 9648 leaf images. To ensure consistency in evaluation, this embodiment divides the images into a 70% training set and a 30% test set ratio, maintaining a consistent sample ratio for each category (stratified random partitioning).

[0109] During the training phase, data augmentation was performed on leaf images, including random cropping and random horizontal flipping, to simulate scale and shooting direction changes that may occur during field collection. During the testing phase, the images were uniformly scaled to 256×256 and then cropped from the center to obtain a standard input of 224×224 to ensure consistent evaluation conditions.

[0110] To verify the performance advantages of the "multi-level knowledge distillation" technology of this invention ("cross-modal teacher-unimodal student multi-level knowledge distillation") over existing technologies, this embodiment designs a comparative experiment to compare the recognition performance of different crop disease identification methods under the same experimental conditions. The comparison methods include:

[0111] (1) Single-modal teacher model: Only the visual features of the leaf image are extracted by the image encoder, and the disease category prediction results are output by the classification network;

[0112] (2) Cross-modal teacher model: Visual and semantic features are extracted simultaneously using image encoder and text encoder, and joint modeling of visual and semantic information is achieved through cross-modal interaction and gating fusion module.

[0113] (3) Traditional knowledge distillation method: a cross-modal teacher model is used as the teacher model and a lightweight convolutional neural network is used as the student model. Knowledge transfer is achieved only through output layer distillation.

[0114] (4) The cross-modal teacher-single-modal student multi-level knowledge distillation method proposed in this invention: introduce disease semantic information through the cross-modal teacher model, and construct a multi-scale semantic feature simulation module and a cross-modal feature alignment module in the student model, and train them in combination with the contrastive learning mechanism.

[0115] like Figure 4As shown, experimental results indicate that on the soybean foliar disease dataset, the single-modal teacher model achieved a recognition accuracy of 92.21%; using the cross-modal teacher model, the recognition accuracy increased to 99.26%; to achieve lightweight model deployment, the traditional output layer knowledge distillation method was used, achieving a recognition accuracy of 94.65%, which was lower than the cross-modal teacher model; using the cross-modal teacher-single-modal student multi-level knowledge distillation method, the model's final recognition accuracy reached 99.12%, with recognition accuracy comparable to the cross-modal teacher model. These comparative experimental results demonstrate that the method of this invention can effectively improve the recognition accuracy of crop foliar diseases while maintaining a lightweight model structure, indicating that the proposed cross-modal teacher-single-modal student multi-level knowledge distillation technique has significant advantages.

[0116] To further verify the contribution of each key module of this invention to the improvement of model performance, this embodiment designs a module ablation experiment. The ablation experiment adopts a single-variable control method, that is, while keeping other modules unchanged, only one key module is removed at a time to analyze the impact of that module on model performance. Accordingly, the following five model configurations are constructed:

[0117] (1) The complete model includes a multi-scale semantic feature simulation module, a cross-modal feature alignment module, and a contrastive learning mechanism, and is trained using multi-level knowledge distillation;

[0118] (2) Remove the semantic feature simulation module and retain only the cross-modal feature alignment module and distillation training;

[0119] (3) Remove the cross-modal feature alignment module and retain only the semantic feature simulation module and distillation training;

[0120] (4) Remove the contrast learning module and retain only the semantic feature simulation module and the cross-modal feature alignment module;

[0121] (5) Remove the characteristic layer distillation and retain only the output layer distillation.

[0122] like Figure 5 As shown, the ablation experiment results indicate that the recognition accuracy of the complete model is 99.12%; the recognition accuracy is 95.37% when the semantic feature simulation module is disabled; the recognition accuracy is 96.64% when the cross-modal feature alignment module is disabled; the recognition accuracy is 98.62% when the contrastive learning module is disabled; and the model recognition accuracy is 94.65% when feature layer distillation is disabled. These experimental results demonstrate that all training optimization strategies have a positive effect on improving model performance. Among them, the multi-level distillation strategy has the most significant impact on model performance and is an important factor in improving the student's model knowledge transfer ability. Furthermore, cross-modal feature alignment, contrastive learning, and multi-scale semantic simulation can further improve the model's training stability and generalization ability.

[0123] The above examples demonstrate that the multi-level knowledge distillation technology proposed in this invention, which combines "cross-modal teacher-single-modal student" approaches, can effectively inherit the discriminative ability of the teacher model under reasoning conditions with only input images. It also achieves high recognition accuracy and good robustness in crop leaf disease identification tasks, thus verifying the feasibility and effectiveness of the technical solution of this invention.

Claims

1. A method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model, characterized in that, Includes the following steps: Step 1: Construct a dataset of crop leaf disease images and perform data preprocessing; Step 2: Construct a cross-modal teacher model that includes an image encoder and a text encoder; Step 3: Construct a lightweight student model and perform multi-level knowledge distillation training; Step 4: Use the trained student model to identify leaf diseases in crops.

2. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 1, characterized in that, Step 1 includes: Step 1.1: Acquire raw images of crop leaves using a mobile phone camera, industrial camera, or visible light camera from a drone; Step 1.2: Convert the input image to RGB three channels uniformly; if the original image is BGR, perform channel rearrangement; Step 1.3: Size normalization and cropping: Scale and crop the RGB image to the preset input size H×W=224×224 to obtain an image with uniform size; Step 1.4: Linearly normalize the image pixel values ​​from 0 to 255 to 0 to 1, and perform mean-variance normalization; then convert the image from H×W×C format to C×H×W format.

3. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 1, characterized in that, Step 2, constructing the cross-modal teacher model, includes: Step 2.1: Construct an image encoder. ShuffleNetV2 is used as the backbone network. The preprocessed image obtained in Step 1 is input into the image encoder. Through multi-layer convolution operations and nonlinear mapping, texture features, shape features and lesion structure features in the image are extracted layer by layer to obtain image feature representations that can characterize leaf disease information. Step 2.2: Using a pre-trained visual-language model, the pre-processed crop leaf images are input into the model to generate a structured disease text description that includes the overall distribution features of the disease, the morphological features of local lesions, and the color and texture features. The resulting text feature sequence is then input into a pre-trained text encoder for semantic encoding. Step 2.3: Input the visual features output by the image encoder and the semantic features output by the text encoder into the cross-modal fusion module. Through feature mapping, cross-attention mechanism or gating fusion method, feature interaction between visual information and semantic information is realized, so that image features can be combined with disease semantic information to express, thereby obtaining the fused cross-modal feature representation. Step 2.4: Input the fused cross-modal features into the classification prediction layer and output the category prediction results of crop leaf diseases. Through continuous optimization of the teacher model parameters during the training process, the teacher model can simultaneously utilize image features and text semantic features for disease identification and generate high-quality prediction results and feature representations to guide the training of student models.

4. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 1, characterized in that, Step 3 includes: Step 3.1: Construct a student model that relies solely on image input to extract visual features from leaf images and complete the disease classification task. The student model uses a lightweight convolutional neural network, ShuffleNetV2, as the image feature extraction module. The input image is processed by the image feature extraction module of the student model to obtain image feature representation, which serves as the basis for subsequent semantic feature simulation and feature fusion. Step 3.2: Construct a multi-scale semantic feature simulation module to generate simulated semantic feature representations corresponding to the semantic feature space structure through image feature transformation. During the training phase, knowledge distillation is performed on the student model using cross-modal semantic features provided by the teacher model, so that the simulated semantic features gradually approach the real semantic representation in the teacher model. Step 3.3: Input the image features extracted from the student model and the semantic features generated by the semantic feature simulation module into the cross-modal feature alignment module; Step 3.4: Introduce a contrastive learning mechanism during the student model training process to constrain the feature space of the student model; Step 3.5: During the training of the student model, the cross-modal teacher model trained in Step 2 is introduced as the guidance network, and the discriminative knowledge in the teacher model is transferred to the student model through a multi-level knowledge distillation strategy. Step 3.6: During the distillation training process, the parameters of the student model are updated by jointly optimizing the classification loss and distillation loss of the student model, so that the student model can learn the discriminative knowledge in the teacher model while learning the real label information.

5. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 4, characterized in that, Step 3.1 includes inputting the preprocessed image obtained in step 1 into the image feature extraction module of the student model, and performing dimensional transformation through the feature mapping layer to map it to a unified feature space. The mapped image features can be represented as a Batch×d feature matrix, where Batch represents the number of input samples and d represents the feature dimension.

6. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 4, characterized in that, Step 3.2 includes: 3.2.1: Input the image features extracted by the student model into the semantic feature simulation module, and then send the image features into feature mapping branches of multiple scales for parallel processing; The fine-grained feature mapping branch is used to extract local detail information in image features. The input features are transformed from Batch×d to Batch×(d / 2) through the dimensionality reduction mapping Linear(d,d / 2), which simulates the enhancement of lesion texture and edge detail information. The mesoscale feature mapping branch is used to extract mid-level semantic information from image features. By maintaining the feature dimension unchanged, the Linear(d, d) mapping structure keeps the input features as Batch×d, thus simulating the expression of the structural information of the disease area. The coarse-grained feature mapping branch is used to extract global semantic information from image features. By using the dimension-upgrading mapping Linear(d,2d), the input features are expanded from Batch×d to Batch×(2d), which simulates and enhances the overall disease semantics. 3.2.2: The features obtained from the three scale branches are concatenated and combined so that the fused features simultaneously contain fine-grained local information, mid-level structural information, and global semantic information. The concatenated multi-scale feature dimension is Batch×(3.5d). Then, it is remapped to the unified feature space Batch×d through the feature fusion layer to obtain the simulated semantic feature representation. The feature fusion layer includes a linear mapping Linear(3.5d, d)), a non-linear activation function GELU, and a normalization layer LayerNorm.

7. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 4, characterized in that, Step 3.3 includes: 3.3.1: Stack the image features Batch×d with the simulated semantic features Batch×d to form a cross-modal feature sequence Batch×2×d; 3.3.2: The cross-modal feature sequence is input into the cross-attention module. The correlation weight between visual features and semantic features is calculated through a lightweight multi-head attention mechanism, so that image features can perceive semantic information and semantic features can perceive visual structural information, thereby realizing cross-modal information interaction. After the cross-attention is completed, the features are transformed and enhanced through residual connections and feedforward mapping networks. 3.3.3: The obtained feature sequence is subjected to feature aggregation processing. By weighted aggregation of the sequence dimensions, the feature dimension is mapped from Batch×2×d to Batch×d, thereby obtaining the fused cross-modal feature representation, and the features are used as input for the student model's subsequent classification prediction and distillation learning.

8. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 4, characterized in that, Step 3.5 includes: 3.5.1: The same input image is simultaneously input into the teacher model and the student model for forward computation. The teacher model outputs cross-modal fusion feature representation and disease category prediction results, while the student model obtains fusion feature representation through semantic feature simulation and cross-modal alignment module; 3.5.2: The fusion feature representation output by the teacher model is represented as Batch×d, and the fusion feature representation obtained by the student model through the cross-modal feature alignment module is also represented as Batch×d; 3.5.3: Guided training of student models is conducted using a multi-level distillation strategy. The distillation process includes output layer distillation and feature layer distillation. Output layer distillation uses the class probability distribution output by the teacher model as a soft label to guide the student model to learn the classification decision information of the teacher model, enabling the student model to obtain a more stable class discrimination ability. Feature layer distillation aligns the fused feature representations generated by the teacher model and the student model in the feature extraction stage, enabling the student model to learn the cross-modal semantic discrimination information contained in the teacher model.

9. The method for identifying crop leaf diseases based on multi-level knowledge distillation using a teacher-student model according to claim 1, characterized in that, Step 4 includes: Step 4.1: Input the crop leaf image to be identified into the student model, extract the high-level visual features of the image through the convolutional feature extraction network, and convert it into an image feature representation in a unified feature space through the feature mapping layer. The image feature representation is a Batch×d feature matrix, where Batch represents the number of input samples and d represents the feature dimension. Step 4.2: Input the image features into the semantic feature simulation module, generate simulated semantic features through multi-scale feature mapping. During the training phase, the student model is supervised by distillation through the cross-modal semantic features provided by the teacher model. During the inference phase, even without inputting text information, the student model can still generate feature representations with semantic discriminative ability. Step 4.3: Input the image features and simulated semantic features into the cross-modal feature alignment module, and realize the feature interaction between visual information and semantic information through a lightweight attention mechanism to obtain the fused cross-modal feature representation; Step 4.4: Input the fused features into the classification prediction layer to output the probability of crop leaf disease categories, and determine the disease category according to the principle of maximum probability to realize the identification of crop leaf diseases.

Images