Crop disease and pest image recognition system based on deep learning

Abstract

The invention discloses a crop disease and pest image recognition system based on deep learning, and relates to the technical field of image processing, and the system comprises a data collection module which is used for collecting disease and pest image data of various crops under different environment conditions; according to the method, the adversarial network is designed to generate a highly vivid pest image, the image is visually similar to a real image, and consistency can be kept in pathological features, so that a training data set is greatly enriched, and effective knowledge in a large teacher model is extracted and transmitted to a student model through a knowledge distillation technology, so that the training efficiency is improved. According to the invention, the method achieves the light weight of the model, greatly reduces the parameter quantity and calculation complexity while maintaining the similar performance of the student model, and enables the student model to still maintain higher recognition accuracy while reducing the calculation burden through knowledge distillation.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and specifically to a crop pest and disease image recognition system based on deep learning. Background Art

[0002] Crop pests and diseases refer to various harmful organisms and adverse factors that affect the growth, development, and yield of crops. These pests and diseases mainly include two categories: biological pests and diseases and non-biological pests and diseases. Crop pests and diseases have a great impact on agricultural production, which can lead to crop yield reduction, quality decline, and even complete crop failure in severe cases. Therefore, preventing and controlling crop pests and diseases is an important measure to ensure national food security and the sustainable development of agriculture. The prevention and control methods include: agricultural control, biological control, physical control, and chemical control.

[0003] Existing pest and disease image recognition systems rely on a large amount of labeled data to train deep learning models. However, obtaining high-quality labeled data is both time-consuming and laborious. In some datasets of specific pests and diseases, there is often a problem of unbalanced labeled samples, resulting in insufficient recognition ability of the model for certain pests and diseases. Moreover, the model will overfit the training data when trained under specific conditions, resulting in insufficient generalization ability in new environmental conditions or different crop varieties. Therefore, we propose a crop pest and disease image recognition system based on deep learning to solve the problems mentioned above.

[0004] The above information disclosed in this background art is only used to increase the understanding of the background art of the present invention. Therefore, it may include prior art that is not known to those of ordinary skill in the art. Summary of the Invention

[0005] The purpose of the present invention is to provide a crop pest and disease image recognition system based on deep learning to solve the problems in the current market proposed in the above background art.

[0006] To achieve the above purpose, the present invention provides the following technical solutions:

[0007] A crop pest and disease image recognition system based on deep learning, comprising:

[0008] A data acquisition module, used to collect pest and disease image data of various crops under different environmental conditions;

[0009] A data enhancement and generation module, used to perform preliminary processing and enhancement on the collected image data;

[0010] A feature extractor, used to extract high-level features from the enhanced images;

[0011] A domain discriminator, used to distinguish features of the source domain and the target domain;

[0012] A conditional generative adversarial network is used to generate pest and disease images with specific attributes according to the input environmental conditions and noise;

[0013] The reinforcement learning module: is used to learn how to adjust the recognition strategy under different environmental conditions to maintain accurate pest and disease recognition;

[0014] The knowledge distillation module compresses a complex large model into a lightweight model through knowledge distillation technology.

[0015] As a further optimization scheme of the present invention, the data enhancement and generation module includes:

[0016] The image enhancement unit: is used to perform a series of transformations on the original image to generate new training samples;

[0017] The noise addition unit: is used to add different types of noise to the image to simulate interference factors in the actual environment;

[0018] The image mixing unit: creates new image samples by superimposing and fusing multiple images;

[0019] The generative adversarial network unit: uses GAN to generate new pest and disease images;

[0020] The data augmentation strategy optimization unit: is used to analyze and optimize the data enhancement strategy so that the enhanced data still conforms to the distribution of the real world.

[0021] As a further optimization scheme of the present invention, the data acquisition module includes:

[0022] The image acquisition unit: uses cameras, mobile phones, drones or image capture devices to collect pest and disease images of crops under different environmental conditions;

[0023] The image annotation unit: is used to annotate the collected images and mark the specific types, locations and degrees of pests and diseases;

[0024] The data storage and management unit: is used to store the collected image data and establish a database management system;

[0025] The data preprocessing unit: is used for preliminary preprocessing;

[0026] The data synchronization and transmission unit: is used to transmit the collected image data from the acquisition device to the storage server or directly to the data processing unit;

[0027] The environmental information recording unit: is used to record the environmental information at the time of image acquisition.

[0028] As a further optimization scheme of the present invention, the reinforcement learning module includes:

[0029] The state space, including the current model performance metrics and data processing status;

[0030] The action space, including possible data augmentation strategies and model adjustment actions;

[0031] The reward function, used to evaluate the impact of actions on model performance.

[0032] As a further optimization scheme of the present invention, the specific steps to implement the knowledge distillation module are as follows:

[0033] Train the teacher model to obtain high-precision pest and disease identification ability, and train the student model;

[0034] Train the student model through the output soft labels of the teacher model to transfer the knowledge of the teacher model;

[0035] Optimize the student model so that while maintaining a high recognition accuracy, it reduces the computational resources and storage requirements;

[0036] Deploy the optimized student model to the edge device and provide real-time pest and disease identification services through the network interface.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] The present invention designs an adversarial network to generate highly realistic pest and disease images, which are not only visually similar to real images but also maintain consistency in pathological features, thus greatly enriching the training dataset. And through the knowledge distillation technology, the effective knowledge in the large teacher model is extracted and transferred to the student model, realizing the lightweight of the model. While the student model maintains similar performance, the number of parameters and computational complexity are greatly reduced. Knowledge distillation enables the student model to maintain a high recognition accuracy while reducing the computational burden.

[0039] The above summary is only for the purpose of the specification and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the drawings and the following detailed description. Brief Description of the Drawings

[0040] Figure 1 It is a module block diagram of the crop pest and disease image recognition system based on deep learning of the present invention;

[0041] Figure 2 It is a flow step diagram of the conditional generative adversarial network in the present invention. Detailed Embodiments

[0042] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0043] Embodiment 1

[0044] Please refer to Figure 1 、 Figure 2 , a crop pest and disease image recognition system based on deep learning, including:

[0045] A data acquisition module for collecting pest and disease image data of various crops under different environmental conditions;

[0046] Specifically, the data acquisition module includes:

[0047] An image acquisition unit: using a camera, mobile phone, drone or image capture device to collect pest and disease images of crops under different environmental conditions;

[0048] Select common pests and diseases that have a greater impact on agricultural production. For common pests and diseases of wheat, including stripe rust, rice blast, potato late blight;

[0049] Select major food crops as the types of crops to be identified, such as wheat, rice, and corn.

[0050] An image annotation unit: used to annotate the collected images and mark the specific types, locations, and degrees of pest and disease;

[0051] A data storage and management unit: used to store the collected image data and establish a database management system;

[0052] A data preprocessing unit: used for preliminary preprocessing;

[0053] A data synchronization and transmission unit: used to transmit the collected image data from the acquisition device to the storage server or directly to the data processing unit;

[0054] An environmental information recording unit: used to record the environmental information during image acquisition.

[0055] A data enhancement and generation module for performing preliminary processing and enhancement on the collected image data;

[0056] Specifically, the data enhancement and generation module includes:

[0057] An image enhancement unit: used to perform a series of transformations on the original image to generate new training samples;

[0058] Specifically, randomly rotate the image at angles such as 0°, 90°, 180°, 270°, etc., randomly adjust the image size, simulate images taken at different distances, and randomly crop certain parts of the image to generate different image sizes and views of pests and diseases.

[0059] Noise addition unit: used to add different types of noise to the image to simulate interference factors in the actual environment;

[0060] Image mixing unit: creates new image samples by superimposing and fusing multiple images;

[0061] Generative adversarial network unit: uses GAN to generate new images of pests and diseases;

[0062] Specifically, the specific steps for training GAN are as follows: input the source domain image and environmental condition labels into the generator, the generator outputs an image simulating the target environmental conditions, the discriminator distinguishes the generated image from the real target domain image, and through adversarial training, optimize the parameters of the generator and the discriminator.

[0063] Data augmentation strategy optimization unit: used to analyze and optimize the data augmentation strategy so that the augmented data still conforms to the distribution of the real world.

[0064] Feature extractor, used to extract high-level features from the augmented images;

[0065] Specifically, use ResNet-50 as the network architecture. In the initial layer of the convolutional layer, use a 7x7 convolutional kernel, a stride of 2, and SAME padding, followed by batch normalization and ReLU activation function. For the max pooling layer, use a 3x3 pooling kernel with a stride of 2.

[0066] In the residual block, first, construct the residual block. ResNet-50 contains 4 residual layers, and each residual layer contains multiple residual blocks. Inside each residual block, there are: Convolutional layer 1: a 1x1 convolutional kernel for dimensionality reduction. Convolutional layer 2: a 3x3 convolutional kernel for feature extraction. Convolutional layer 3: a 1x1 convolutional kernel for dimensionality increase. Add a skip connection between the start and end of each residual block, allowing the input to be directly added to the output.

[0067] In the fully connected layer, after all residual layers, use a global average pooling layer to reduce the number of parameters. Finally, there are multiple fully connected layers for classification, and the number of nodes in the fully connected layer is equal to the number of classes.

[0068] Use the pre-trained ResNet-50 model parameters on the ImageNet dataset as initialization.

[0069] Domain discriminator, used to distinguish the features of the source domain and the target domain;

[0070] A conditional generative adversarial network for generating pest and disease images with specific properties based on input environmental conditions and noise;

[0071] Specifically, for the network architecture, an input layer is designed to receive two inputs, one is a random noise vector and the other is an environmental condition vector; for the embedding layer, the environmental condition vector is converted through an embedding layer into a dimension compatible with the noise vector; for the merging layer, the converted environmental condition vector and the noise vector are merged to form a new input vector; for the hidden layer, for non-image data, convolutional layers and deconvolutional layers are designed to gradually increase the dimension of the feature maps while introducing skip connections to maintain information transmission; for the output layer, an image of the target domain is output, and the number of filters is equal to the number of channels of the image.

[0072] For the discriminator, in the input layer, it accepts two inputs, one is the generated or real image and the other is the environmental condition vector; for the embedding layer, it is the same as the generator; for the merging layer, the converted environmental condition vector and the image features are merged; for the hidden layer, convolutional layers are designed to extract high-level features of the image; for the output layer, a single authenticity score is output.

[0073] Furthermore, the dimension of the noise vector Z is [batch_size, z_dim], and the dimension of the environmental condition vector C is [batch_size, c_dim]. The embedding layer is used to convert C into a vector with the same dimension as Z, [batch_size, z_dim], and the converted C and Z are merged along the last dimension (axis=-1).

[0074] Specifically: H = concatenate([Z, Embedding(C)], axis=-1)

[0075] Where, Embedding(C) represents converting C through the embedding layer into a vector with the same dimension as Z.

[0076] In the convolutional layer, the input feature map H generates an output feature map H' through the convolution operation Conv and the activation function activation.

[0077] Specifically: H' = activation(Conv(H, kernel_size, stride, padding))

[0078] Where, Conv represents the convolution operation, kernel_size is the size of the convolution kernel, stride is the stride of the convolution, and padding is the padding size of the input feature map before convolution.

[0079] In the transposed convolution layer, the input feature map H generates an output feature map H′ through the transposed convolution operation Deconv and the activation function activation.

[0080] Specifically, H′ = activation(Deconv(H, kernel_size′, stride′, output_padding′, padding′))

[0081] Among them, Deconv represents the transposed convolution operation, kernel_size′ is the size of the transposed convolution kernel, stride′ is the stride of the transposed convolution, output_padding′ is the additional padding size of the output feature map after transposed convolution, and padding′ is the padding size of the input feature map before transposed convolution.

[0082] Reinforcement learning module: used to learn how to adjust the recognition strategy under different environmental conditions to maintain accurate pest and disease recognition;

[0083] Among them, the state space includes the current model performance metrics and data processing status;

[0084] The action space includes possible data augmentation strategies and model adjustment actions;

[0085] The reward function is used to evaluate the impact of actions on the model performance.

[0086] Specifically, the reinforcement learning module includes:

[0087] State definition unit: used to define the state of the environment;

[0088] Action space definition unit: by defining all possible actions that the model may take.

[0089] Policy network unit: includes a neural network, which outputs a probability distribution of an action according to the current state, guiding how the model selects actions.

[0090] Value function estimation unit: responsible for estimating the value function of each state, that is, the expected return that can be obtained by taking the optimal strategy starting from this state.

[0091] Reward function design unit: used to provide feedback after the model takes an action.

[0092] Exploration and exploitation strategy unit: used to determine the balance between the model's exploration of unknown strategies and exploitation of known best strategies.

[0093] Experience replay unit: stores the historical data of the states, actions, rewards, and next states experienced by the model for subsequent batch learning.

[0094] Knowledge distillation module, which compresses a complex large model into a lightweight model through knowledge distillation technology.

[0095] Specifically, the knowledge distillation module includes:

[0096] Teacher model training unit: used to train the teacher model so that the model can learn complex features and classification tasks from pest and disease images.

[0097] Soft label generation unit: generates soft labels using the output of the teacher model.

[0098] Student model design unit: responsible for designing the structure of the student model.

[0099] Knowledge distillation loss function definition unit: used to define the fitting loss of the student model to the soft label and the fitting loss of the student model to the hard label during the knowledge distillation process.

[0100] The specific steps to implement the knowledge distillation module are as follows:

[0101] Train the teacher model to obtain high-precision pest and disease recognition ability, and train the student model;

[0102] Train the student model through the output soft labels of the teacher model to transfer the knowledge of the teacher model;

[0103] Optimize the student model so that while maintaining a high recognition accuracy, it reduces computational resources and storage requirements;

[0104] Deploy the optimized student model to edge devices and provide real-time pest and disease recognition services through a network interface.

[0105] Furthermore, forward-propagate the training dataset through the teacher model and use the activation before the output layer of the teacher model to generate soft labels. The formula used:

[0106]

[0107] where S i is the soft label of the i-th class, Z i is the original activation value of the teacher model for the i -th class, T is the temperature parameter used to adjust the smoothness of the soft label.

[0108] For the training of the student model: the training objective of the student model includes two parts, one part is the cross-entropy loss of the true label, and the other part is the soft label loss provided by the teacher model. The formula used is:

[0109]

[0110] Among them, yi is the true label, and pi is the predicted probability of the student model for the i-th class.

[0111] Soft label loss formula:

[0112]

[0113] Among them, S i is the soft label generated by the teacher model, and p i ′ is the predicted probability of the student model at temperature T.

[0114] Combined loss function: Combine the hard label loss and the soft label loss to form the total loss function of the student model.

[0115] Total loss formula:

[0116] L total = αL hard +(1 - α)L soft

[0117] Among them, α is the balance coefficient, which is used to adjust the importance of the hard label loss and the soft label loss.

[0118] Backpropagation and optimization: Perform backpropagation on the student model and update the model parameters according to the total loss function.

[0119] Student model training unit: Used to train the student model using the soft label generated by the teacher model and the hard label of the student model.

[0120] It should be understood that each part of the present invention can be implemented by hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, any one or a combination of the following well-known technologies in the art can be used: discrete logic circuits with logic gate circuits for implementing logical functions on data signals, application-specific integrated circuits with appropriate combinational logic gate circuits, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0121] Those of ordinary skill in the art of this technology can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiments.

[0122] In addition, each functional unit in various embodiments of the present invention may be integrated into a processing module, or each unit may exist physically alone, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. When the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0123] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A crop pest and disease image recognition system based on deep learning, characterized by: include: Data acquisition module, used to collect image data of pests and diseases of various crops under different environmental conditions; Data enhancement and generation module, used for preliminary processing and enhancement of collected image data; A feature extractor, used to extract high-level features from the enhanced image; Domain discriminator, used to distinguish the features of source domain and target domain; Conditional Generative Adversarial Networks, which are used to generate pest and disease images with specific attributes based on input environmental conditions and noise; Reinforcement learning module: used to learn how to adjust identification strategies under different environmental conditions to maintain accurate pest and disease identification; The knowledge distillation module compresses complex large models into lightweight models through knowledge distillation technology.

2. The crop pest image recognition system based on deep learning according to claim 1, characterized in that: The data enhancement and generation modules include: Image enhancement unit: used to perform a series of transformations on the original image to generate new training samples; Noise adding unit: used to add different types of noise to the image to simulate interference factors in the actual environment; Image blending unit: creates new image samples by superimposing and fusing multiple images; Generative Adversarial Network Unit: Generate new pest and disease images using GAN; Data augmentation strategy optimization unit: used to analyze and optimize data augmentation strategies so that the augmented data still conforms to the real-world distribution.

3. The crop pest image recognition system based on deep learning according to claim 1, characterized in that: The data acquisition module includes: Image acquisition unit: Use cameras, mobile phones, drones or image capture devices to collect images of crop pests and diseases under different environmental conditions; Image annotation unit: used to annotate the collected images and mark the specific type, location and degree of pests and diseases; Data storage and management unit: used to store the collected image data and establish a database management system; Data preprocessing unit: used for preliminary preprocessing; Data synchronization and transmission unit: used to transmit the collected image data from the collection device to the storage server or directly to the data processing unit; Environmental information recording unit: used to record environmental information during image acquisition.

4. The crop pest and disease image recognition system based on deep learning according to claim 1, characterized in that: The enhanced learning modules include: The state space contains the current model performance indicators and data processing status; The action space, which contains possible data augmentation strategies and model adjustment actions; A reward function is used to evaluate the impact of actions on model performance.

5. The crop pest and disease image recognition system based on deep learning according to claim 1, characterized in that: The specific steps to implement the knowledge distillation module are: Train the teacher model to obtain high-precision pest and disease identification capabilities, and train the student model; The student model is trained through the output soft labels of the teacher model to transfer the knowledge of the teacher model; Optimize the student model to reduce computing resources and storage requirements while maintaining high recognition accuracy; The optimized student model is deployed to edge devices and provides real-time pest and disease identification services through network interfaces.

Images