Corn disease detection method and system based on IRG-ResNet distillation model

Abstract

The invention discloses a corn disease detection method and system based on an IRG-ResNet distillation model, and relates to the technical field of corn disease recognition. The method comprises the following steps: acquiring original images including different types of corn diseases, and establishing a corn disease data set; selecting ResNet50 as a basic model of a teacher model, and selecting ResNet18 as a basic model of a student model; basic model structures of the teacher model and the student model are improved and optimized, and the optimized models are trained to obtain the teacher model and the student model; knowledge is migrated from the teacher model to the student model by using an instance relation graph distillation method composed of instance features, instance relations and feature space transformation, and an IRG-ResNet distillation model is obtained from the student model; the recognition performance of the IRG-ResNet distillation model is determined; and carrying out corn disease identification and detection by adopting an IRG-ResNet distillation model. The method can effectively meet the requirements of the agricultural Internet of Things mobile terminal for the model, and can quickly and accurately realize the automatic identification of the corn diseases.

Description

Technical Field

[0001] The present invention belongs to the technical field of corn disease identification, and in particular relates to a corn disease detection method and system based on an IRG-ResNet distillation model. Background Art

[0002] Corn is an important crop. Its growth conditions involve temperature, water, light, soil, fertilizer, and planting density. Therefore, corn faces many challenges in its growth process. Corn diseases are one of the important challenges. Corn diseases can significantly reduce its yield and quality. Rapid and accurate identification of corn diseases can effectively take appropriate preventive measures to reduce disease losses. However, corn diseases have similar or similar disease characteristics, which can easily lead to misdiagnosis of diseases.

[0003] Convolutional Neural Network (CNN) is a deep learning model that has achieved great success in the field of computer vision. Convolutional neural networks are combined with agricultural Internet of Things to realize intelligent monitoring and control of agricultural production environment, and have been widely used in the field of agricultural identification. At present, convolutional neural networks have developed rapidly in the field of crop disease identification, including but not limited to corn, and have achieved a series of research results. For example, Wang Yingying et al. collected corn disease data sets, constructed a new downsampling module based on the AlexNet network to enhance the foreground response of corn disease images, enhance the model's ability to extract detail features, build a new fully connected module to reduce network parameters, apply MISH activation function to improve the nonlinear expression of the network, introduce attention mechanism, and finally the average recognition accuracy is 99.35%. Ksibi et al. used a remote-controlled agricultural unmanned aerial vehicle equipped with a camera to shoot olive leaf disease images and proposed a hybrid deep learning model; in order to comprehensively improve the disease prediction ability, the model is a merger of ResNet50 and MobileNet models and is named MobiRes-Net. The classification accuracy of the model is 97.08%, which is better than the 92.86% of the ResNet50 model and the 94.63% of the MobileNet model. Although the convolutional neural network method has achieved ideal recognition accuracy in specific disease recognition tasks, it also has problems such as many network parameters, large amount of calculation and complex model, and poor practicality.

[0004] In order to solve the problem of model deployment on the mobile side of the agricultural Internet of Things, researchers have developed different technologies including knowledge distillation, pruning, quantization and binarization neural networks, and proposed various lightweight architectures to provide strong technical support for the deployment of crop disease identification methods. At present, the CNN model has made some breakthroughs in model lightweighting and has achieved excellent results in disease identification, but the model of agricultural Internet of Things mobile deployment with high recognition rate, low parameters and excellent real-time performance is still in the exploration stage.

[0005] In view of the lack of excellent models with low false recognition rate, low parameters and high real-time performance in the recognition field, the present invention proposes an IRG-ResNet knowledge distillation lightweight model for corn disease recognition. IRG is the instance relationship graph distillation method, which models instance features, instance relationships and feature space transformation to capture the whole network knowledge. It aims to guide the learning of the student network through the teacher network, so that the student network can extract more discriminative features under the condition of small parameter amount and improve the recognition accuracy. It can respond in real time in practical application, making it more suitable for deployment on the mobile terminal of agricultural Internet of Things. Summary of the invention

[0006] The purpose of the present invention is to provide a corn disease detection method and system based on the IRG-ResNet distillation model to solve the problem of the lack of a low error rate, low parameter, and high real-time agricultural Internet of Things mobile deployment model in the recognition field proposed in the above background technology.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] The first aspect of the present invention proposes a corn disease detection method based on the IRG-ResNet distillation model, comprising the following steps:

[0009] S1. Collect original images including different types of corn diseases, perform data enhancement processing on the original images, and obtain a corn disease dataset;

[0010] S2. Build the basic model of the teacher model and the student model, select ResNet50 as the basic model of the teacher model, and select ResNet18 as the basic model of the student model;

[0011] S3. Improve and optimize the basic model structure of the teacher model and the student model, and use the corn disease dataset to train the optimized model to obtain the teacher model and the student model;

[0012] S4, using the instance relationship graph distillation method composed of instance features, instance relationships and feature space transformation to transfer knowledge from the teacher model to the student model, and obtain the IRG-ResNet distillation model from the student model;

[0013] S5. Determine the recognition performance of the IRG-ResNet distillation model;

[0014] S6. Use IRG-ResNet distillation model for corn disease identification and detection.

[0015] Preferably, the different types of corn diseases in S1 include corn rust, corn leaf spot, corn Curvularia leaf spot, corn sheath blight, corn leaf spot, and corn brown spot.

[0016] Preferably, the basic model in S2 is as follows:

[0017] The ResNet50 and ResNet18 both include multiple convolutional layers, a fully connected layer, batch normalization and a ReLU activation function, and the multiple convolutional layers are specifically Conv1 to Conv5.

[0018] Preferably, the basic model structures of the teacher model and the student model in S3 are both improved and optimized in the same manner, as follows:

[0019] An attention mechanism module is embedded after Conv1, the activation functions of Conv2 to Conv5 are replaced with SILU activation functions, spatial pyramid pooling is added after Conv5, and a Dropout layer is added;

[0020] Use pruning to compress the model and eliminate redundant layers in Conv2-Conv5 through layer pruning.

[0021] Furthermore, the ResNet50 retains two repeated BottleNeck modules in each Conv layer through layer pruning; and the ResNet18 retains only one BasicBlock module in each Conv layer through layer pruning.

[0022] Preferably, the S4 is specifically as follows:

[0023] The instance relationship graph distillation method models three kinds of knowledge: instance features, instance relationships, and feature space transformation. First, by considering instance features and instance relationships as vertices and edges respectively, an instance relationship graph is constructed to model the extracted knowledge of a network layer. Second, an instance relationship graph transformation is proposed to model the feature space transformation across layers. Subsequently, a loss function for the instance relationship graph is formulated using the mined knowledge. Finally, an overall loss is formulated based on all loss functions.

[0024] Specifically, the distilled knowledge is transferred by adding three supervisory signals, including L IRG , L logits and L IRG-t ; L IRG Used to transmit instance features and instance relations; Llogits Represents instance characteristics, which is L IRG A special case of L IRG-t Extract feature space transformation knowledge; loss function L IRG , L logits and L IRG-t The loss function L with Softmax GT A multi-type knowledge loss is constructed to transfer all knowledge from the teacher model to the student model.

[0025] Furthermore, the loss function L IRG , as follows:

[0026] Assume that the training instance input is the x of the lth layer i The instance features and instance relationship formula of the instance feature adjacent matrix A l , IRG l Represents the feature space of the lth layer; the loss function L IRG The difference between the teacher model and the student model, including the vertex differences of the two instance relationship graphs Marginal differences Follow the formula below:

[0027]

[0028] We further use vertex differences only for the logits layer, so L IRG Follow the formula below:

[0029]

[0030] in, Represents the instance relationship diagram of the L-th layer teacher model, represents the instance features of the L-th layer teacher model, represents the feature adjacency matrix of the L-th layer teacher model instance, Represents the instance relationship diagram of the M-th layer student model, represents the instance features of the M-th layer student model, represents the instance feature adjacency matrix of the M-th layer student model, λ1 and λ2 are penalty term parameters, and x is the input image.

[0031] Furthermore, the loss function L IRG-t , as follows:

[0032] The instance relationship graph transformation represents the instance feature space transformation, which consists of vertex transformation and edge transformation; therefore, L IRG-t The loss function of includes two parts: vertex transformation and edge transformation, as shown in the following formula:

[0033]

[0034] in, and represents the transformation of vertices and edges from layer 1 to layer 2 of the teacher model, and represents the spatial transformation of the corresponding student model; and Represent the vertex transformation and edge transformation between the teacher model and the student model respectively;

[0035] Further omitting the edge part of the instance relationship graph transformation loss, we get the following formula:

[0036]

[0037] in, represents the vertex transformation from layer 1 to layer 2 of the teacher model, Represents the vertex transformation of the student model from layer 3 to layer 4.

[0038] Furthermore, the multi-type knowledge loss is shown in the following formula:

[0039]

[0040] Among them, L MTK represents multi-type knowledge loss, and λ3 is the penalty parameter.

[0041] Preferably, the S5 is specifically as follows:

[0042] The accuracy formula, precision formula, recall formula and harmonic mean formula are used to evaluate the recognition performance of the IRG-ResNet distillation model.

[0043] The second aspect of the present invention proposes a corn disease detection system based on the IRG-ResNet distillation model, including the IRG-ResNet distillation model obtained by the above method.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] (1) In view of the problem that the existing corn disease recognition model has bloated model, large consumption of computing memory resources, and is not conducive to the deployment of disease recognition agricultural Internet of Things mobile terminals in the field application, the IRG-ResNet distillation model is built in the present invention. Through this lightweight model, accurate recognition of corn diseases under complex field backgrounds is achieved. The IRG-ResNet model developed based on the ResNet model has a size of 4.9M, a recognition accuracy of 98.25%, and an average inference time of 39.79ms. The results show that the model has good recognition performance for six corn diseases. The introduction of the attention mechanism CBAM, the replacement of the activation function SILU, and the addition of spatial pyramid pooling enhance the generalization ability and robustness of the model. The application of pruning technology reduces the complexity of the model. Comparative experiments with other networks have proved the effectiveness and simplicity of this experimental model, which meets the requirements of low complexity and high performance, has practical application value, and provides strong technical support for the mobile deployment of disease recognition models.

[0046] (2) The method of the present invention is based on ResNet50, constructs the CBAM attention mechanism, modifies the SILU activation function, adds pyramid pooling, and designs a teacher model with rich knowledge, high performance and high complexity. The attention mechanism emphasizes the part that contributes most to the result and weakens the secondary part by assigning different weights to the input data. This enables the model to focus on the most important part when processing information, thereby improving the interpretability of the model. By introducing nonlinear activation functions, the neural network can perform nonlinear mapping of the input signal to better fit different types of data. Spatial pyramid pooling can extract richer features by dividing and pooling the feature map at multiple scales, so that it can better adapt to different input images.

[0047] (3) The method in the present invention refers to ResNet18 to build a student model that has learning ability, is lightweight and efficient, and is suitable for deployment on the mobile terminal of the agricultural Internet of Things.

[0048] (4) The method of the present invention applies the instance relationship graph distillation method to force the lightweight student model to learn the knowledge extracted by the teacher model, thereby improving the performance of the student model. Model distillation compresses large and complex models into small and simple models, and through the "knowledge transfer" method, enables the student model to learn the important patterns and features of the teacher model, thereby improving the model's reasoning speed, improving generalization ability, and reducing the consumption of computing resources.

[0049] (5) The method of the present invention uses pruning operation to compress the model, delete redundancy, reduce the number of parameters, reduce the amount of calculation and memory usage, obtain a streamlined model, and finally form an improved IRG-ResNet lightweight corn disease recognition and detection method. The feasibility, accuracy and effectiveness of the proposed model were verified through experiments on corn disease datasets.

[0050] (6) In order to ensure the robustness of the model in practical applications, the method of the present invention constructed a dataset of 6 types of corn diseases in a complex background. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is a structural block diagram of the IRG-ResNet distillation model in the present invention;

[0052] Figure 2 The diagram is a sample of corn diseases in the present invention (a. brown spot, b. sheath blight, c. large spot, d. small spot, e. Curvularia leaf spot, f. rust);

[0053] Figure 3 It is a structural block diagram of the residual unit of the basic model in the present invention ((a) is the two-layer residual unit BasicBlock of ResNet18, (b) is the three-layer residual unit BottleNeck of ResNet50);

[0054] Figure 4 Schematic diagram of the SILU activation function in the present invention;

[0055] Figure 5 Schematic diagram of the structure of the Dropout layer in the present invention;

[0056] Figure 6 It is a structural diagram of the student model of the present invention;

[0057] Figure 7 Schematic diagram of the test accuracy and loss curve of the ResNet model in the present invention ((a) is the test accuracy, (b) is the test loss);

[0058] Figure 8 It is a bar chart of the parameters of the student model and the teacher model of the present invention;

[0059] Fig. 9 It is a line graph of the accuracy of the student model and the teacher model of the present invention;

[0060] Fig.10 A schematic diagram of the average reasoning time of the student model and the teacher model in the present invention;

[0061] Fig.11 Schematic diagram of the confusion matrix of the IRG-ResNet network model in the present invention;

[0062] Fig.12 Schematic diagram of test accuracy and loss curves of the student model, teacher model and IRG-ResNet distillation model of the present invention ((a) is the test accuracy, (b) is the test loss);

[0063] Fig.13 It is a schematic diagram of the recognition rate of various diseases by different models in the present invention;

[0064] Fig.14 Schematic diagram for comparing the performance of different models in the present invention. DETAILED DESCRIPTION

[0065] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Embodiment 1:

[0067] The lightweight corn disease detection method in the present invention is implemented based on the IRG-ResNet distillation model. The overall architecture of the model is as follows: Figure 1 The specific method is as follows:

[0068] Step 1: Create a data set;

[0069] The experimental data were collected in the northern part of Anhui Province. They were taken by researchers with handheld cameras and obtained through expert screening, identification and classification. The shooting background was complex, including different conditions such as wild flowers and weeds, gravel and soil, or staggered leaf coverage, reflecting the actual living conditions of corn. The database contains 6 types of corn diseases, totaling 2,725 photos, including 470 photos of corn rust, 645 photos of corn leaf spot, 546 photos of corn leaf spot, 356 photos of corn sheath blight, 260 photos of corn leaf spot, and 448 photos of corn brown spot. Figure 2 Some image samples are shown.

[0070] In order to increase the number of data sets and improve the quality of data sets, a better model can be trained. The experiment performed data enhancement processing on the original data set. The main methods are as follows: random rotation, random cropping, random scaling, horizontal and vertical random translation and other operations on the original data set images. The enhanced data set images are unified in size, and the image storage format is set to JPG format. After the data enhancement operation, the number of original corn disease data sets is expanded to 10,845, and there are no duplicate images. The number of corn disease data set images before and after processing is shown in Table 1.

[0071] Table 1 Number of images in the corn disease dataset before and after processing

[0072] Types of corn diseases Number of original pictures Number of enhanced images Label Rust 470 1880 1 Small spot disease 645 1835 2 Curvularia leaf spot 546 1838 3 Sheath blight 356 1780 4 Big spot disease 260 1720 5 Brown spot 448 1792 6 total 2725 10845 /

[0073] Step 2: Build the basic model of teacher network and student network;

[0074] The corn disease images in the field environment have complex backgrounds and rich noise interference information. A model architecture with high recognition accuracy is required to detect the types of diseases. However, deep learning models with high expressiveness have problems such as easy overfitting, poor interpretability or complex parameter adjustment. The ResNet model solves the problem of gradient disappearance or explosion during deep network training. By introducing residual blocks, information can be directly transmitted from the input layer to the deep layer. Due to the existence of residual connections, the model can learn at a deeper level, reducing the dependence on feature extraction at each layer, which helps to speed up training and improve accuracy. Therefore, the present invention selects the ResNet model as the basic model.

[0075] In the present invention, the teacher network uses ResNet50 as the basic model, the basic model of the student network is ResNet18, and the ResNet network structure is shown in Table 2. In order to solve the problems of gradient disappearance and representation degradation in deep networks, ResNet adopts the residual learning strategy. Figure 3 (a) shows the two-layer residual unit BasicBlock of ResNet18. Figure 3 (b) shows the three-layer residual unit BottleNeck of ResNet50. Residual connections allow the network to skip some layers directly and add input to the output. Through residual units, the neural network can learn the difference between input and output instead of directly learning the output. This design allows the network to become very deep without the problem of difficulty in training traditional deep networks. Artificially creating an identity mapping allows the entire structure to converge in the direction of the identity mapping, ensuring that the final error rate does not deteriorate with increasing depth. ResNet contains multiple convolutional layers and a fully connected layer, using efficient convolutions such as 1×1, 3×3 and 7×7 convolution kernels, as well as batch normalization and ReLU activation functions to ensure the computational efficiency and performance of the model.

[0076] Table 2 ResNet network structure

[0077]

[0078] Step 3: Optimize the basic model structure of the teacher network;

[0079] (1) Embedded attention mechanism CBAM module;

[0080] The attention mechanism effectively captures the dependencies and importance between different tags in the input sequence by adjusting the importance of each tag to the current processing task, thereby improving the performance of the model. CBAM (Convolutional Block Attention Module) is a lightweight hybrid attention module that contains two sub-modules: the Channel Attention Module and the Spatial Attention Module. The output features of the Channel Attention Module and the Spatial Attention Module are multiplied element by element to obtain the final attention-enhanced feature. This enhanced feature will be used as the input of subsequent network layers to suppress noise and irrelevant information while retaining key information. The CBAM module adaptively learns channel and spatial attention weights to improve the feature expression ability of the convolutional neural network. By combining channel attention and spatial attention, the CBAM module is able to capture the correlation between features in different dimensions, thereby improving the performance of image recognition tasks.

[0081] (2) Modify the activation function;

[0082] The SILU (Sigmoid Linear Unit) activation function is an adaptive activation function, such as Figure 4 , as shown in formula (1). SILU is an improved function of Sigmoid and ReLU. SILU performs better than ReLU in deep models. It has the characteristics of no upper bound, lower bound, smoothness, and non-monotonicity, and can be regarded as a smooth ReLU activation function. SILU has no upper bound to avoid oversaturation; it has a lower bound and tends to 0, which has a regularization effect; it is non-monotonic and can output negative gradients, which is more robust to different initializations and learning rates; it is smooth and easier to optimize. SILU can convert between positive and negative values, providing richer information processing capabilities. In some cases, SILU can reduce the number of parameters required for the model, making the model more lightweight. The output automatically adapts to the size of the input, which helps prevent the gradient from disappearing or exploding, thereby improving the training efficiency of the network. The SILU activation function is introduced into this experiment because of its unique characteristics and potential advantages.

[0083] The function expression of SILU is:

[0084] SILU(x)=x·σ(x) (1)

[0085]

[0086] Among them, σ(x) represents the Sigmoid function, and the range of the SILU function is (-∞,+∞) because the function has output in the positive and negative range of the input value x.

[0087] (3) Add spatial pyramid pooling;

[0088] Spatial Pyramid Pooling (SPP): Spatial pyramid pooling is a strategy for processing network structures with variable input sizes. The convolutional layers in SPP are used to extract feature representations of the input image. In traditional convolutional neural networks, the input size of the fully connected layer is usually fixed, so the size of the input image is also required to be fixed. By introducing SPP, the network is allowed to process input images with different sizes. SPP divides the input feature map into regions of different sizes and performs a pooling operation on each region to obtain a feature vector of fixed length. Therefore, even if the input image has different sizes, SPP is able to generate feature representations of the same length, making the network scale invariant. After feature extraction, activation functions are usually applied to introduce nonlinearity and increase the expressive power of the network. The last few layers of SPP are fully connected layers, which are used to map the extracted features to category labels for tasks such as classification or regression. This pooling technique can effectively capture multi-scale feature information by dividing the input data into different regions and applying maximum pooling within these regions. This makes the model more robust to scale changes in the input image.

[0089] (4) Add Dropout layer;

[0090] Dropout (such as Figure 5 As shown in the figure, it is a regularization method proposed to alleviate the overfitting of CNN, which can effectively alleviate the occurrence of overfitting. Dropout has the effect of averaging. Dropout of different hidden neurons is similar to training different networks. Randomly deleting half of the hidden neurons leads to different network structures. The whole Dropout process is equivalent to averaging many different neural networks. Different networks produce different overfitting. Some "opposite" fits offset each other to achieve the overall reduction of overfitting. Dropout can reduce the complex co-adaptation relationship between neurons. The Dropout program causes two neurons to not necessarily appear in the same network every time. In this way, the update of weights no longer depends on hidden nodes with fixed relationships, preventing certain features from being effective only under other specific features. Force the network to learn more robust features, which also exist in random subsets of other neurons. Dropout can increase sparsity. When the amount of data is small, sparsity can be used to increase the distinguishability of features. The present invention applies Dropout to deal with the fact that the model does not rely too much on certain local features, and the model generalization ability will be stronger.

[0091] (5) Using pruning to compress the model;

[0092] Model pruning is a method to reduce model complexity by reducing model parameters, thereby speeding up inference and reducing memory consumption, while trying not to significantly reduce model performance. Model pruning is often applied to deep neural networks, especially convolutional neural networks. Model pruning is mainly divided into structured pruning and unstructured pruning. Unstructured pruning removes unimportant neurons. Accordingly, the connections between pruned neurons and other neurons are ignored during calculation. Corresponding to unstructured pruning is structured pruning, which usually prunes filters or entire network layers as basic units. When a filter is pruned, its previous feature map and the next feature map will change accordingly, but the structure of the model is not destroyed and can still be accelerated by GPU or other hardware.

[0093] The present invention applies structured pruning to delete unimportant network layers and prune the layers. As shown in Table 2, the last four convolutional layers of ResNet50 are very complex. Through layer pruning, two repetitions are retained in each convolutional layer, and the middle 48 layers are compressed to 24 layers.

[0094] Step 4: Optimize the basic model structure of the student network;

[0095] The complex teacher network has excellent accuracy, but it cannot meet the requirements of loading the model on mobile devices. Therefore, referring to the ResNet18 network structure, a lightweight student network with 5 convolutional modules is designed, such as Figure 6 As shown. The overall design of the student network is similar to that of the teacher network. The activation function in the network is changed to SILU, allowing some negative values ​​to flow in. The CBAM attention mechanism module is embedded after Conv1 to focus on important features, suppress unimportant features, and improve the model's representation ability. Figure 3 (a) shows the BasicBlock module. At the same time, layer pruning is used to keep only one repeated BasicBlock in each Conv layer to remove redundancy. Spatial pyramid pooling is added after Conv5 to improve the model's adaptive expression ability. Finally, a Dropout layer is added to alleviate overfitting.

[0096] Step 5: Use knowledge distillation to transfer knowledge;

[0097] Knowledge distillation is an effective model compression method that uses a complex and bloated teacher network to transfer knowledge to a lightweight student network. Knowledge is extracted from a large teacher model and condensed into a small student model.

[0098] The present invention applies the instance relationship graph IRG (Instance Relationship Graph) distillation method. It models three kinds of knowledge: instance features, instance relationships, and feature space transformation. First, by considering instance features and instance relationships as vertices and edges respectively, IRG is constructed to model the extracted knowledge of a network layer. Secondly, an IRG transformation is proposed to model cross-layer feature space transformation. It is gentler than directly imitating the features of the intermediate layers. This method effectively captures the knowledge on the entire network through IRG, so that it has stable convergence and strong robustness to different network architectures. Subsequently, using the mined knowledge, a loss function for IRG is formulated. Finally, the overall loss is formulated based on the previous loss function to supervise the training of the student network. In addition to the Softmax loss function L GT , three supervisory signals are added to transfer the distilled knowledge, including L IRG , L logits and L IRG-t These three signals all come from IRG, which represents the feature space of a certain layer. Specifically, L IRG Used to transfer instance features and instance relationships. logits Represents instance characteristics, which is L IRG A special case of L IRG-t The feature space transformation knowledge is extracted. Finally, these three loss functions and the Softmax loss function L GT It constitutes a multi-type knowledge loss (L MTK ), which transfers all knowledge from teacher to student.

[0099] The overall framework of the method proposed in the present invention is as follows Figure 1 As shown in the figure. The network above is the teacher network, and the network below is the student network. In the figure, x is the input image, Conv1-5 is the ResNet structure, SPP is the spatial pyramid pooling, Trans(v l1 ,v l2 ) is the vertex transformation, Trans(ε l1 ,ε l2 ) is the edge transformation.

[0100] In order to distill the general knowledge, moderate knowledge and sufficient knowledge in the teacher network from the teacher network to the student network, let the training instance input f l (x i ) is the x of the lth layer i The instance features and instance relationship formula of the instance feature adjacent matrix A l , IRG l Represents the feature space of the lth layer. The loss function L IRGis the difference between the teacher network and the student network. represents the IRG of the L-th layer teacher network, is the IRG of the Mth layer of the student network, and the vertex difference between the two IRGs Marginal differences Follow formula (3).

[0101]

[0102] in, Represents the instance relationship diagram of the L-th layer teacher model, represents the instance features of the L-th layer teacher model, represents the feature adjacency matrix of the L-th layer teacher model instance, Represents the instance relationship diagram of the M-th layer student model, represents the instance features of the M-th layer student model, represents the instance feature adjacency matrix of the M-th layer student model, and λ1 and λ2 are the penalty term parameters.

[0103] The dense knowledge distillation in the middle layers is over-constrained for the student network. Therefore, vertex differences are only used in the logits layer. Therefore, L IRG Follow formula (4).

[0104]

[0105] IRG transformation is a transformation of instance feature space, which consists of vertex transformation and edge transformation. IRG-t The loss function of also consists of two parts, as shown in formula (5).

[0106]

[0107] in, and is the transformation of vertices and edges of the teacher network from layer 1 to layer 2, and and Together they represent the spatial transformation of the student. and denote the vertex transformation and edge transformation between the teacher network and the student network, respectively.

[0108] Compared with the vertex part, the edge transformation part consumes more computational resources. The knowledge of the extracted vertex transformation and edge transformation is redundant. Therefore, in order to improve efficiency, the edge part of the IRG transformation loss is omitted, and formula (6) is obtained.

[0109]

[0110] Three loss functions and Softmax loss function LGT Constitutes a multi-type knowledge loss L MTK , which learns all knowledge from the teacher network to the student network, as shown in formula (7).

[0111]

[0112] Among them, L MTK represents multi-type knowledge loss, and λ3 is the penalty parameter.

[0113] Step 6: Evaluate the recognition performance of the model;

[0114] The recognition performance of the model is evaluated using the accuracy formula (8), precision formula (9), recall formula (10) and harmonic mean formula (11).

[0115]

[0116]

[0117]

[0118]

[0119] Among them, True Positive indicates the number of samples that are actually positive but predicted to be positive, False Positive indicates the number of samples that are actually negative but predicted to be positive, False Negative indicates the number of samples that are actually positive but predicted to be negative, and True Negative indicates the number of samples that are actually negative but predicted to be negative. The number of parameters and FLOPs evaluate the complexity of the model. The average inference time tests whether the model is suitable for loading into the mobile terminal of the agricultural Internet of Things. The number of parameters corresponds to the complexity of the algorithm space. Reducing the number of parameters can reduce the consumption of computer memory resources. FLOPs is the number of floating-point operations and the computational time complexity, which is used to measure the complexity of the algorithm. Inference refers to a forward propagation process of the network, which also refers to the process of inputting data into the network and obtaining output results.

[0120] Test verification:

[0121] The IRG-ResNet model in the present invention was experimentally verified, and the experiment was divided into four parts: selecting a teacher network base network and a student network base network from the ResNet network; a performance comparison test before and after the model improvement; verifying the impact of the improvement of the model in the present invention on the performance of the corn disease type recognition model by performing a network structure ablation test; and comparing the performance differences of the model with other network models in corn disease detection under the same conditions.

[0122] (1) Test configuration

[0123] The hardware and software platform configurations used in the experiment are shown in Table 3. First, in order to meet the model requirements, all images were resized to 224×224, and then the dataset was randomly divided into training set, test set, and validation set in a ratio of 8:1:1. The training set was used to train the model, the validation set was used to adjust the parameters, and the test set was used to evaluate the performance of the final model. During the training and testing process, the Adam optimizer was selected, and the initial learning rate l r The weight decay is set to 0.000001, the weight decay is set to 0.0008, and the cosine annealing decay strategy is used for update. The batch size is set to 64 and the epochs is set to 1000.

[0124] Table 3 Test platform configuration

[0125]

[0126]

[0127] (2) Experimental analysis

[0128] 1) Selected analysis of basic network:

[0129] The ResNet network experiment was conducted on the corn disease dataset, with the learning rate set to 0.0001, the batch size set to 16, and the epochs set to 50. The experimental results are shown in Table 4 and Figure 7 shown.

[0130] Table 4 ResNet network test results

[0131] Model Learning Rate Batch size Parameter quantity Accuracy FLOPs ResNet18 0.0001 16 11,693,736 94.52％ 1.83G ResNet34 0.0001 16 21,801,809 88.48％ 4.65G ResNet50 0.0001 16 25,557,032 90.68％ 4.11G ResNet101 0.0001 16 41,364,008 84.83％ 15.26G ResNet152 0.0001 16 57,894,440 82.45％ 23.14G

[0132] As can be seen from Table 4, as the network depth increases, the number of model parameters increases. ResNet18 has the least number of parameters, which is 11,693,736; ResNet152 has the most parameters, which is 57,894,440, which is 4.95 times that of ResNet18. The best accuracy of corn disease image recognition is 94.52% for ResNet18, followed by ResNet50. Figure 7 It can be seen that the accuracy curves and loss curves of ResNet34 and ResNet50 are intertwined, and the performance of the two models is similar. Table 2 shows that the residual modules used by the two models are different. ResNet34 and ResNet50 continue to train on this dataset, and ResNet50 performs better. At the same time, the number of parameters of ResNet50 is slightly larger, but the FLOPs value is 0.54G smaller. Therefore, in general, ResNet50 is selected as the teacher model and ResNet18 as the student model to jointly meet the high performance requirements of corn disease image recognition.

[0133] 2) Performance comparison analysis before and after model improvement:

[0134] The basic model and the improved model of the student network and the teacher network are compared, and the comparison results are shown in Table 5.

[0135] Table 5 Comparison of model before and after improvement

[0136]

[0137]

[0138] As shown in Table 5, on the corn disease dataset, the accuracy of the basic model ResNet50 of the teacher model is 90.68%, which is 6.83 percentage points less than the teacher model Teacher after construction. The accuracy of the student network Student is 98.25%, which is 3.73 percentage points higher than the basic model ResNet18. The FLOPs of the Teacher model is 1.75G less than the 4.11G of ResNet50. The number of parameters of the Student model is much smaller than 11,693,736 of ResNet18, which is 4,913,704.

[0139] Figure 8 and Fig. 9 It is clearly shown that the improved models Teacher and Student are superior to the original models in terms of expressiveness. The Student model has the highest accuracy, the smallest number of parameters and FLOPs, and is the most outstanding model among the four models. Therefore, it is proved that the method of model improvement is scientific, desirable, and meets the experimental requirements.

[0140] 3) Analysis of model structure ablation test results:

[0141] In corn disease recognition, in order to further verify the effectiveness of the designed model, the designed IRG distillation model was subjected to an ablation test. When different knowledge was distilled from the teacher network to the student network, the test results are shown in Table 6. Among them, "Baseline" means that the student network only uses the label as supervision information during training, and the training of the student network does not use any distilled knowledge; "Student_1", "Student_2", "Student_3", "Student_4", "Student_5", "Student_6" and "Student" refer to the use of "SPP", "SILU" and "CBAM" modules in the student network to guide the training of the student network; "Teacher" refers to the trained teacher network.

[0142] Table 6 Model structure ablation test results

[0143] Module CBAM SPP SILU Parameters Accuracy Teacher √ √ √ 15,996,298 97.51％ Baseline × × × 11,693,736 93.09％ Student_1 × √ × 11,693,736 94.66％ Student_2 × × √ 11,693,736 93.83％ Student_3 √ × × 11,694,858 94.84％ Student_4 √ √ × 11,694,858 95.76％ Student_5 √ × √ 11,694,858 95.21％ Student_6 × √ √ 11,693,736 95.40％ Student √ √ √ 4,913,704 98.25％

[0144] Note: √ indicates that the module was used in the experiment, and × indicates that the module was not used in the experiment

[0145] From the ablation test results shown in Table 6, it can be found that the "Student_1" model adds the "SPP" module compared to the "Baseline" model, and the recognition accuracy of the model is 94.66%, which is 1.57 percentage points higher than the accuracy of the "Baseline" model. The "Student_2" model adds the "SILU" module compared to the "Baseline" model, and the recognition accuracy of the model is 93.83%, which is 0.74 percentage points higher than the accuracy of the "Baseline" model. The "Student_3" model adds the "CBAM" module compared to the "Baseline" model, and the recognition accuracy of the model is 94.84%, which is 1.75 percentage points higher than the accuracy of the "Baseline" model. The parameters of the models "Baseline", "Student_1", "Student_2" and "Student_3" are consistent. Obviously, adding any module of "SPP", "SILU" and "CBAM" on the basis of the "Baseline" model will improve the model performance.

[0146] From the ablation test results shown in Table 6, it can be found that the "Student_4" model adds the "SPP+CBAM" module compared to the "Baseline" model, and the recognition accuracy of the model is 95.76%, which is 2.67 percentage points higher than the accuracy of the "Baseline" model. The "Student_5" model adds the "SILU+CBAM" module compared to the "Baseline" model, and the recognition accuracy of the model is 95.21%, which is 2.12 percentage points higher than the accuracy of the "Baseline" model. The "Student_6" model adds the "SPP+SILU" module compared to the "Baseline" model, and the recognition accuracy of the model is 95.40%, which is 2.31 percentage points higher than the accuracy of the "Baseline" model. The addition or subtraction of the "CBAM" module will change the number of model parameters by 1,122. In short, adding any two modules of "SPP", "SILU" and "CBAM" on the basis of the "Baseline" model will improve the model performance expression, and it is better than adding one module.

[0147] The recognition accuracy of the "Teacher" model is 97.51%, which is 4.42 percentage points higher than the accuracy of the "Baseline" model. The number of parameters is 15,996,298, which is 4,302,562 more than the "Baseline". The accuracy of the "Student" model is 98.25%, which is 5.91% higher than the accuracy of the "Baseline" model, and the number of parameters is 57.98% less than the "Baseline". In summary, the "Student" model can use the distillation function to transfer knowledge from the teacher network to the student network, improving the model's ability to recognize and express corn disease images.

[0148] 4) Average inference time results analysis:

[0149] In order to further observe the inference time of the proposed IRG-ResNet model, the average inference time (ms) of the teacher model and the student model for corn disease image recognition within 1 to 50 epochs is output in turn every 5 epochs, as shown in Fig.10 As shown in the figure. From the average inference time curve, we can see that the average inference time of the student network of the IRG-ResNet model is about 39.79ms each time, which is 150.88ms less than the average inference time of the teacher model. Among them, the average inference time of the student network is reduced by 73.63 percentage points, which is mainly due to the fact that the student network is a light convolutional neural model. In general, the IRG-ResNet model is stable and can meet the real-time requirements.

[0150] 5) Analysis of pruning compression model performance results:

[0151] Pruning technology is an effective technology for model compression. In the present invention, layer pruning operation is used to eliminate redundant layers to obtain a streamlined model. This method can reduce the risk of overfitting and improve the generalization ability of the model. The performance results of the original teacher model O-Teacher and the original student model O-Student are shown in Table 7.

[0152] Table 7 Model compression comparison

[0153] Model CBAM SPP SILU Pruning Parameters Accuracy Teacher √ √ √ √ 15,996,298 97.51％ O-Teacher √ √ √ × 25,558,154 97.32％ O-Student √ √ √ × 11,694,858 97.70％ IRG-ResNet √ √ √ √ 4,913,704 98.25％

[0154] Note: √ indicates that the module was used in the experiment, and × indicates that the module was not used in the experiment.

[0155] Table 7 illustrates the necessity of pruning. Compared with the pruned teacher model Teacher, the O-Teacher model has 9.56M more parameters and 0.19% lower accuracy. The O-Student model size is 11.69M, the model accuracy is 97.70%, and the overall performance is weaker than IRG-ResNet. The IRG-ResNet model is obtained by pruning the original model and fine-tuning it. It is small in size and has superior performance.

[0156] 6) Model recognition accuracy test results:

[0157] See also Figure 11-Figure 12 ,Depend on Fig.12 From the three accuracy curves of the corn disease test set, we can see that the teacher network Teacher fluctuates greatly during training and tends to stabilize with the increase in the number of training rounds. The student network converges the slowest. The IRG-ResNet model curve obtained after the student network is distilled from the teacher network is the most stable, and the final recognition accuracy is also the highest. This is mainly due to the IRG knowledge distillation in the IRG-ResNet model, which effectively captures the instance features, instance relationships, and feature space conversion, and transfers the network knowledge to the student network, improving the convergence and robustness of the student network.

[0158] In order to further observe the performance of the proposed IRG-ResNet model in each category of corn diseases, Fig.11 The confusion matrix of the six diseases is shown in the figure. Each column in the matrix represents the true label, the sum of the data in each column represents the image of that category, each row represents the predicted label of the test image, the sum of the data in each row represents the image predicted to be of that category, the values ​​in the diagonal cells represent correct recognition, and the data in the non-diagonal cells represent incorrect recognition of different categories. Table 8 summarizes the recognition accuracy of the model for the six diseases.

[0159] Table 8 Recognition accuracy of the model

[0160] Precision Recall F1-Score support class1 1.00 0.99 0.99 189 class2 1.00 0.96 0.98 183 class3 1.00 1.00 1.00 207 class4 0.94 1.00 0.97 165 class5 0.96 0.95 0.95 169 class6 1.00 1.00 1.00 172 avg / total 0.98 0.98 0.98 1085

[0161] from Fig.11 As shown in Table 8, class 4 sheath blight and class 5 large leaf spot are easily confused because they have similar disease colors. When the disease occurs, the sheath blight spots are light brown and gradually turn brown, while the large leaf spot spots are dark brown or yellow brown, which are easy to confuse. As shown in Table 8, the average recognition accuracy of the six diseases reached 0.98, and the IRG-ResNet model performed well on each disease.

[0162] 7) Analysis of disease recognition results of different networks:

[0163] Based on the corn disease dataset and experimental method, a comparative test was conducted with the classic traditional model and the lightweight model. The recognition results are shown in Table 9.

[0164] Table 9 Corn disease recognition results of different networks

[0165] Model Precision Recall F1-Score Accuracy Parameters IRG-ResNet 98.29％ 98.25％ 98.27％ 98.25％ 4,913,704 LeNet 93.93％ 93.15％ 93.54％ 93.15％ 23,868,802 VGG16 91.60％ 90.83％ 91.21％ 90.83％ 14,714,688 GoogLeNet 96.60％ 96.53％ 96.56％ 96.53％ 58,305,926 MobileNetV2 90.96％ 90.71％ 90.84％ 90.71％ 3,235,014 EfficientNetV2 84.16％ 81.78％ 82.95％ 81.78％ 5,586,346 SqueezeNet 88.19％ 87.53％ 87.86％ 87.53％ 2,395,180 ShuffleNetV2 94.00％ 93.52％ 93.76％ 93.52％ 2,543,484

[0166] As shown in Table 9, the accuracy of the seven models LeNet, VGG16, GoogleNet, MobileNetV2, EfficientNetV2, SqueezeNet and ShuffleNetV2 are all lower than that of the IRG-ResNet model. Among the three traditional classic algorithms LeNet, VGG16 and GoogleNet, the recognition accuracy of the GoogleNet model is the highest at 96.53%, and the lowest is the VGG16 model, with an accuracy of 90.83%. In the size comparison of the three models LeNet, VGG16 and GoogleNet, the LeNet model has the smallest parameter volume, which is 4.86 times that of the IRG-ResNet model. GoogLeNet is the most bloated model, with a parameter volume of 11.87 times that of IRG-ResNet. The four lightweight models MobileNetV2, EfficientNetV2, SqueezeNet and ShuffleNetV2 have overall weaker corn disease recognition performance than the first three models, but the models are streamlined and have small parameters. The three models of MobileNetV2, SqueezeNet and ShuffleNetV2 are smaller than the IRG-ResNet model built in this experiment. Both SqueezeNet and ShuffleNetV2 are about half the size of the IRG-ResNet model. EfficientNetV2 is slightly larger than the IRG-ResNet algorithm. However, the recognition accuracy of the four models of MobileNetV2, EfficientNetV2, SqueezeNet and ShuffleNetV2 is the smallest at 81.78%, which is much smaller than IRG-ResNet.

[0167] Fig.13 The figure shows the accuracy of eight models in identifying six corn diseases. It is obvious that the recognition performance of the IRG-ResNet model is generally better than that of the other seven models. Fig.14 The performance comparison of 5 models is shown. Fig.14It can be seen that the IRG-ResNet model occupies more space than the classic lightweight models MobileNetV2, SqueezeNet and ShuffleNetV2. Compared with the models of Mobile-DMSANet, the IRG-ResNet model has excellent accuracy and slightly larger number of parameters, indicating that the model may still have redundant modules.

[0168] In summary, the recognition accuracy of the IRG-ResNet model is 98.25%, the highest among the seven models. In terms of model size, IRG-ResNet is larger than MobileNetV2, SqueezeNet and ShuffleNetV2, and slightly smaller than EfficientNetV2. It meets the requirements of lightweight and high recognition rate, and is suitable for deployment in resource-constrained agricultural IoT devices.

[0169] The above description is only used to help understand the method of the present invention and its core essence, but the protection scope of the present invention is not limited thereto. For those skilled in the art in the art, equivalent replacement or change according to the technical solution and inventive concept of the present invention within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A corn disease detection method based on the IRG-ResNet distillation model, characterized in that: The steps include: S1. Collect original images including different types of corn diseases, perform data enhancement processing on the original images, and obtain a corn disease dataset; S2. Build the basic model of the teacher model and the student model, select ResNet50 as the basic model of the teacher model, and select ResNet18 as the basic model of the student model; S3. Improve and optimize the basic model structure of the teacher model and the student model, and use the corn disease dataset to train the optimized model to obtain the teacher model and the student model; S4, using the instance relationship graph distillation method composed of instance features, instance relationships and feature space transformation to transfer knowledge from the teacher model to the student model, and obtain the IRG-ResNet distillation model from the student model; S5. Determine the recognition performance of the IRG-ResNet distillation model; S6. Use IRG-ResNet distillation model for corn disease identification and detection.

2. The corn disease detection method based on the IRG-ResNet distillation model according to claim 1, characterized in that: The basic model in S2 is as follows: The ResNet50 and ResNet18 both include multiple convolutional layers, a fully connected layer, batch normalization and a ReLU activation function, and the multiple convolutional layers are specifically Conv1 to Conv5.

3. The corn disease detection method based on the IRG-ResNet distillation model according to claim 2, characterized in that: The basic model structures of the teacher model and the student model in S3 are improved and optimized in the same way, as follows: An attention mechanism module is embedded after Conv1, the activation functions of Conv2 to Conv5 are replaced with SILU activation functions, spatial pyramid pooling is added after Conv5, and a Dropout layer is added; Use pruning to compress the model and eliminate redundant layers in Conv2-Conv5 through layer pruning.

4. The corn disease detection method based on the IRG-ResNet distillation model according to claim 3, characterized in that: The ResNet50 retains two repeated BottleNeck modules in each Conv layer through layer pruning; the ResNet18 retains only one BasicBlock module in each Conv layer through layer pruning.

5. The corn disease detection method based on the IRG-ResNet distillation model according to claim 1, characterized in that: The S4 is specifically as follows: The instance relationship graph distillation method models three kinds of knowledge: instance features, instance relationships, and feature space transformation. Firstly, by considering instance features and instance relationships as vertices and edges respectively, an instance relationship graph is constructed to model the extracted knowledge of a network layer. Secondly, an instance relationship graph transformation is proposed to model the feature space transformation across layers. Then, the mined knowledge is used to formulate a loss function about the instance relationship graph; finally, the overall loss is formulated based on all the loss functions; Specifically, the distilled knowledge is transferred by adding three supervisory signals, including L IRG , L logits and L IRG-t ; L IRG Used to transmit instance features and instance relations; L logits Represents instance characteristics, which is L IRG A special case of L IRG-t Extract feature space transformation knowledge; Loss function L IRG , L logits and L IRG-t The loss function L with Softmax GT A multi-type knowledge loss is constructed to transfer all knowledge from the teacher model to the student model.

6. The corn disease detection method based on the IRG-ResNet distillation model according to claim 5, characterized in that: The loss function L IRG , as follows: Loss function L IRG The difference between the teacher model and the student model, including the vertex differences of the two instance relationship graphs Marginal differences Follow the formula below: Use vertex differences only for the logits layer, so L IRG Follow the formula below: in, Represents the instance relationship diagram of the L-th layer teacher model, represents the instance features of the L-th layer teacher model, represents the feature adjacency matrix of the L-th layer teacher model instance, Represents the instance relationship diagram of the M-th layer student model, represents the instance features of the M-th layer student model, represents the instance feature adjacency matrix of the M-th layer student model, λ1 and λ2 are penalty term parameters, and x is the input image.

7. The corn disease detection method based on the IRG-ResNet distillation model according to claim 6, characterized in that: The loss function L IRG-t , as follows: The instance relationship graph transformation represents the instance feature space transformation, which consists of vertex transformation and edge transformation. Therefore, L IRG-t The loss function of includes two parts: vertex transformation and edge transformation; omitting the edge part of the instance relationship graph transformation loss, the following formula is obtained: in, represents the vertex transformation from layer 1 to layer 2 of the teacher model, Represents the vertex transformation of the student model from layer 3 to layer 4.

8. The corn disease detection method based on the IRG-ResNet distillation model according to claim 7, characterized in that: The multi-type knowledge loss is shown in the following formula: Among them, L MTK represents multi-type knowledge loss, and λ3 is the penalty parameter.

9. The corn disease detection method based on the IRG-ResNet distillation model according to claim 1, characterized in that: The S5 is specifically as follows: The accuracy formula, precision formula, recall formula and harmonic mean formula are used to evaluate the recognition performance of the IRG-ResNet distillation model.

10. Corn disease detection system based on IRG-ResNet distillation model, characterized in that: Including the IRG-ResNet distillation model obtained by the method as described in any one of claims 1-9.

Images