Lightweight rice disease recognition method and system based on sensitivity under complex background

Abstract

The invention discloses a lightweight rice disease recognition method and system based on sensitivity under complex background, the method comprising: obtaining a data set containing real field complex background images and preprocessing the data set; constructing a lightweight network model for inter-class similarity and complex background, using adaptive global average pooling to replace the fully connected layer, dividing and reasoning the training set after each round of training, calculating the sensitive convolution kernel and the insensitive convolution kernel in the current round model according to the evaluation algorithm of the convolution kernel sensitivity to different categories of diseases; punishing the feature channels output after all insensitive convolution kernels, adding spatial attention to the output features of the sensitive convolution kernel after the feature layer; constructing an intermediate teacher model for auxiliary fine-tuning training based on the structured pruning of the channel penalty of the convolution kernel sensitivity; and outputting the trained model after testing. The invention can identify rice diseases, and reduces the complexity of the model while ensuring the accuracy of the model.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a lightweight rice disease recognition method and system based on sensitivity under complex background. Background Art

[0002] Rice is one of the most important food crops in my country and even in the world, providing food for nearly 50% of the world's population.

[0003] Rice is generally planted in large areas, and some rice diseases such as sesame leaf blight and rice smut are contagious. Once rice diseases occur and are not dealt with in time, they will pose a huge threat to farmers' income and national food security. According to the forecast of the Pest and Disease Monitoring and Forecasting Office, rice diseases and insect pests will be more serious in 2024, and the area of ​​disease occurrence will reach 390 million mu. In the early stages of rice diseases, rice leaves will develop lesions and show various symptoms of the disease. At present, in my country, the main method for identifying rice diseases is manual diagnosis, that is, relying on farmers' experience and knowledge to make a preliminary diagnosis of the disease. Manual diagnosis of rice diseases is not only inefficient, but also easily affected by subjective factors such as fatigue and emotions, and can only be detected when symptoms appear and have obvious characteristics. This manual diagnosis is subjective, labor-intensive, inefficient and expensive, and cannot be applied on a large scale. In order to ensure rice yields, it is crucial to diagnose rice diseases in a timely and accurate manner.

[0004] In recent years, due to the ability of deep learning to automatically extract image features and achieve classification, many scholars have introduced deep learning to the task of disease classification. However, on the one hand, the symptoms of different types of diseases are similar due to factors such as rice being in different developmental stages, different disease sites, and disease severity. The excessive similarity of the diseases can easily lead to incorrect classification by the deep learning model. On the other hand, the rice disease image data collected in real rice fields have complex background factors such as leaf stacking. Too much irrelevant information in the image may cause the deep learning model to make incorrect classifications. Finally, most of the current rice leaf disease classification models have very complex structures, huge parameters and calculations, and high requirements for device resources, which makes it difficult to migrate to some resource-constrained mobile or embedded devices, thereby limiting the actual implementation and promotion of disease models.

[0005] In order to solve the problem of inter-class similarity of disease symptoms, existing methods mainly strengthen feature extraction and fusion from the perspectives of multimodal data, different color gamut spaces, and the combination of deep learning and traditional machine learning methods. However, these methods generally require the addition of additional branches, which leads to an increase in model parameters and computational complexity. In addition, building and aligning multimodal data is also a challenging task. In order to solve the problem of inter-class similarity of disease symptoms, existing methods mainly use image segmentation and attention mechanisms to guide the model to focus on the diseased area and classify it. On the one hand, the disease image segmentation model makes the disease classification problem more complicated. On the other hand, the addition of channel attention and indifferent spatial attention will also introduce additional parameters and computational overhead that need to be trained. In order to solve the problem of complex model structure, existing methods mostly use lightweight neural networks and lightweight modules. Lightweight neural networks and lightweight modules may involve higher difficulty and complexity in the design and training process. At the same time, since the scale and complexity of the model are reduced by the early design, the performance of the model may be sacrificed in some complex tasks, and the universality of these designs has not been fully verified.

[0006] In summary, the existing rice leaf disease recognition methods still face challenges in actual field disease detection, such as similar diseases, complex backgrounds, and insufficient edge computing power. Therefore, this patent proposes a lightweight model construction method based on channel penalty linkage spatial attention based on convolution kernel sensitivity, focusing on the construction of small models facing inter-class similarities and complex backgrounds, and providing technical support for improving the quality and efficiency of rice production. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a lightweight rice disease identification method and system based on sensitivity under complex background in view of the defects in the prior art.

[0008] The technical solution adopted by the present invention to solve its technical problem is:

[0009] The present invention provides a lightweight rice disease identification method based on sensitivity under complex background, the method comprising the following steps:

[0010] Step 1, obtain a data set containing real field complex background images, divide the data set into a training set and a test set, and preprocess the training set using image enhancement technology;

[0011] Step 2: Build a lightweight network model for inter-class similarity and complex backgrounds. The lightweight network model selects the VGG13 network and uses adaptive global average pooling to replace the fully connected layer. After each round of training, the training set is divided according to the specified rules, and the divided data is input into the lightweight network model for reasoning. The sensitive convolution kernel and the insensitive convolution kernel in this round of model are calculated according to the evaluation algorithm of the convolution kernel's sensitivity to different types of diseases.

[0012] Step 3: Based on the output of the evaluation algorithm in step 2, on the one hand, all feature channels output after the insensitive convolution kernels are penalized, and on the contrary, the convolution kernels with high sensitivity are made to automatically focus on the difference features between similar diseases of different categories during the model training process; on the other hand, spatial attention is added only to the output features of the sensitive convolution kernels after a certain feature layer, so that the convolution kernels with high sensitivity can focus on the diseased area while paying attention to the differences between diseases of similar categories;

[0013] Step 4: Perform structured pruning based on the results of the non-sensitive convolution kernel channel penalty. At the same time, construct an intermediate teacher model based on the structured pruning strategy to perform auxiliary fine-tuning training on the lightweight network model with excessive structured pruning.

[0014] Step 5: Test the lightweight network model obtained after fine-tuning in step 4 using the test set to determine whether the model performance and model size meet the requirements. If not, re-execute step 4; if so, output a lightweight network model for inter-class similarity and complex backgrounds for rice disease identification.

[0015] Furthermore, the method in step 1 of the present invention comprises:

[0016] The dataset contains images of no less than 9 types of rice diseases and healthy rice, which were taken between 45 and 82 days of rice growth. The symptoms of different types of diseases are similar due to factors such as different developmental stages of rice, different disease sites, and disease severity, as well as complex backgrounds.

[0017] Methods for preprocessing images include: rotation, scaling, translation, flipping, and adjusting to a uniform size.

[0018] Furthermore, in step 2 of the present invention, after each round of training, the training set is divided according to a specified rule, and the divided data is input into a lightweight network model for reasoning. The specific method is:

[0019] After each Epoch training, the sensitivity of each convolution kernel is calculated once, and the entire training set is divided into batches according to the rules. The division rules include:

[0020] ① Each batch contains rice disease images of the same category and has the same number of images;

[0021] ② The images in the i-th batch and the i+1-th batch belong to different categories, that is, the two adjacent batch images in the same Epoch belong to different categories;

[0022] ③ The last batch of the i-th Epoch and the first batch of the i+1-th Epoch belong to different categories, that is, when different Epochs are connected, the images of the two batches belong to different categories respectively;

[0023] After the lightweight network model completes the i-th Epoch training, the training data set is divided according to the rules, and the training data is input into the model in the order of batch0, batch1, and batch2 for inference.

[0024] Furthermore, the evaluation algorithm for the sensitivity of different types of diseases according to the convolution kernel in step 2 of the present invention is specifically:

[0025] Batch i The image is input into the model and each convolution kernel batch is recorded i Image output i，l，j ; where i represents the input image of the i-th batch, l represents the convolution layer index, and j represents the j-th convolution kernel in the l-th layer. Similarly, batch i+1 The diseased image is input into the model and each convolution kernel batch is recorded i+1 output i+1，l，j , and then use the squared Euclidean distance to calculate the feature map output i，l，j and output i+1，l，j similarity;

[0026] The calculated similarity is divided by the width and height of each layer of feature map to get the final similarity. The formula is as follows:

[0027] similar(output i，l，j , output i+1，l，j )=sum((output i，l，j -output i+1，l，j ) 2 ) / h l / w l

[0028] Among them, h l 、w l They are the output of the convolution kernel of the lth layer respectively. i，l，j The height and width of output i，l，j and output i+1，l，j They are the outputs of the same convolution kernel and therefore have the same height and width;

[0029] In order to define sensitive convolution kernels and insensitive convolution kernels, set a ratio of insensitive convolution kernels in advance. Calculate the similarity based on the two batch output feature maps before and after the convolution kernel, and sort the similarities from high to low.

[0030] After each similarity comparison of the output feature maps of two batches, the convolution kernels that appear within the insensitive_ratio are counted. After the similarity comparison of the entire training data set is completed, the convolution kernels that appear within the insensitive_ratio are ranked from high to low according to the number of times they appear. The convolution kernels with the highest insensitive_ratio are finally determined as the insensitive convolution kernels in the model after the end of this round of training.

[0031] Furthermore, the method of penalizing the feature channel output after the insensitive convolution kernel in step 3 of the present invention includes:

[0032] Set the penalty ratio punish_ratio = insensitive_ratio; a batch of images passes through the Conv layer and then the BN layer and the ReLu layer in turn; according to the obtained insensitive convolution kernel, the [γ i , γ k , ..., γ n ] is finally determined as the channel to be punished during the i+1th Epoch training. According to the result of multiple similarity comparison statistics, the [γ i , γ k , ..., γ n ] to recalculate;

[0033] According to the determined γ index [γ i , γ k , ..., γ n ] and the calculation formula of the BN layer, give the corresponding BN [γ i , γ k , ..., γ n ] l Add an L1 regularization, the formula is:

[0034] l1_reg=l1_reg+l1([γ i , γ k , ..., γ n ] l )

[0035] Among them, l1_reg represents the selected [γ i , γ k , ..., γn ] l After calculating L1, it is accumulated, l represents the layer index, and the accumulated l1_reg is added to the loss function, the formula is:

[0036] loss=loss+l1stren*l1_reg

[0037] Among them, l1stren is a hyperparameter, which indicates the penalty strength. According to the addition derivative formula and the chain derivative rule, the selected [γ i , γ k , ..., γ n ] l The formula for gradient update is:

[0038]

[0039] Among them, α represents the learning rate, sign(*) is a sign function, and in [γ i , γ k , ..., γ n ] l After adding the L1 penalty, the scaling factor corresponding to the insensitive convolution kernel is obtained [γ i , γ k , ..., γ n ] l The value of will be additionally subtracted from l1_stren*[sign(γ i ), sign(γ k ), ..., sign(γ n )] l , let the non-sensitive convolution kernel correspond to [γ i , γ k , ..., γ n ] l Due to the additional penalty, it quickly tends to 0, thus achieving the purpose of insensitive convolution kernel channel penalty.

[0040] Furthermore, the method of adding spatial attention to the sensitive convolution kernel output features in step 3 of the present invention includes:

[0041] According to the guidance of the algorithm for evaluating the sensitivity of convolution kernels to different types of diseases, spatial attention is added to the feature map defined as the output of the sensitive convolution kernel after a certain BN layer;

[0042] The channel penalty based on the convolution kernel sensitivity directly affects the [γ i , γ k , ..., γ n ] l Punishment is carried out, and the [γ i, γ k , ..., γ n ] l The process of calculating BN is as follows:

[0043] [x1, x2, ..., x c ]·([γ w1 , γ w2 =1,...,γ wc ]·[γ1,γ2,...,γ c ])→y l

[0044] Among them, x i represents the feature output by the i-th convolution kernel of the convolution layer, c represents the number of channels of the feature layer, and γ i Indicates that the i-th feature map corresponds to the i-th γ in BN, γ wi Represents the original γ after the channel penalty of the non-sensitive convolution kernel i Compared to normal training updates, an additional scaling is performed, where γ w2 =1 means that the channel corresponds to a sensitive convolution kernel and is not additionally penalized.

[0045] Furthermore, the method for performing structured pruning in step 4 of the present invention includes:

[0046] In the model obtained by channel penalty training in step 3, the γ vectors in each BN layer are collected and sorted from small to large according to the absolute value, and prune_ratio = punish_ratio = insensitive_ratio is set. [γ vectors] ranked within the top prune_ratio are selected from the global i , γ k , ..., γ n ] channels and delete the convolution kernels corresponding to these channels from the model.

[0047] Furthermore, the method for constructing the intermediate teacher model in step 4 of the present invention includes:

[0048] After over-structural pruning of the model, knowledge distillation is used to assist in fine-tuning the model; an intermediate teacher model with a similar structure is built through the structured pruning strategy in step 4;

[0049] During the fine-tuning training process, on the one hand, the image data is passed through the small model after structured pruning to obtain the classification layer output S(x), and on the other hand, the image data is passed through the intermediate teacher model to obtain the classification layer output T(x); in the loss function, on the one hand, the cross entropy between the output of the small model and the true label y is calculated, and on the other hand, the KL divergence is used to calculate the difference between the student and teacher outputs. Finally, when the disease model after structured pruning is fine-tuned, the training loss function is:

[0050] Loss=(1-α)·CrossEntropy(S(x),y)+α·KLDiv(T(x),S(x))

[0051] Among them, α is the balance parameter of the distillation loss, which is used to adjust the weight between the two parts of the loss terms, S(x) represents the output of the student model classification layer, y represents the true label in the original classification image, CrossEntropy(.,.) represents the cross entropy loss function, and KLDiv(.,.) is the KL divergence loss for calculating the two distributions.

[0052] Furthermore, the method for determining performance in step 5 of the present invention includes:

[0053] The precision, recall, accuracy, and F1 score indicators are used to evaluate the effectiveness of the model in identifying rice leaf diseases. The lightweight network model obtained after fine-tuning in step 4 is tested. As the Epoch increases, the test accuracy of the model increases until the test accuracy stabilizes.

[0054] The present invention provides a lightweight rice disease recognition system based on sensitivity under complex background, comprising:

[0055] A data set preprocessing unit is used to obtain a data set containing real field complex background images, divide the data set into a training set and a test set, and preprocess the training set using image enhancement technology;

[0056] The evaluation algorithm unit of the sensitivity of convolution kernels to different types of diseases is used to build a lightweight network model for inter-class similarity and complex backgrounds. The lightweight network model selects the VGG13 network and uses adaptive global average pooling to replace the fully connected layer. After each round of training, the training set is divided according to the specified rules, and the divided data is input into the lightweight network model for reasoning. The sensitive convolution kernel and the insensitive convolution kernel in this round of model are calculated according to the evaluation algorithm of the sensitivity of convolution kernels to different types of diseases.

[0057] The channel penalty-linked spatial attention unit based on the sensitivity of the convolution kernel is used to penalize the feature channels output after all insensitive convolution kernels according to the output of the evaluation algorithm, and conversely, let the convolution kernels with high sensitivity automatically pay attention to the difference features between similar diseases of different categories during the model training process; on the other hand, after a certain feature layer, only add spatial attention to the output features of the sensitive convolution kernel, so that the convolution kernels with high sensitivity can focus on the diseased area while paying attention to the differences between diseases of similar categories;

[0058] The structured pruning unit based on the channel penalty of the convolution kernel sensitivity is used to perform structured pruning according to the results of the non-sensitive convolution kernel channel penalty. At the same time, an intermediate teacher model is constructed based on the structured pruning strategy to assist in fine-tuning the lightweight network model with excessive structured pruning.

[0059] The model testing and output unit is used to test the lightweight network model obtained after fine-tuning through the test set to determine whether the model performance and model size meet the requirements. If not, it will enter the structured pruning unit with channel penalty based on convolution kernel sensitivity for retraining; if it does, it will output a lightweight network model for inter-class similarity and complex background for rice disease recognition.

[0060] The beneficial effects produced by the present invention are:

[0061] 1. The present invention proposes an evaluation algorithm for the sensitivity of convolution kernels to different types of diseases, and based on the algorithm, performs channel penalties on convolution kernels with weak sensitivity, and conversely allows convolution kernels with high sensitivity to automatically focus on the difference characteristics between similar diseases of different categories during model training. By comparing the action process of the classic SeNet, it is deduced that the channel penalty based on the sensitivity of the convolution kernel contains the idea of ​​channel attention and does not introduce additional parameters that need to be trained.

[0062] 2. The present invention proposes channel-penalized linked spatial attention based on the sensitivity of convolution kernels. Spatial attention is added only to the output features of high-sensitivity convolution kernels in a certain feature layer. Compared with indiscriminately adding spatial attention to all features in the feature layer, the amount of calculation in the model training process is reduced, allowing the convolution kernels with high sensitivity to focus on the diseased area while paying attention to the differences between similar categories, thereby reducing the impact of complex backgrounds and improving model recognition accuracy.

[0063] 3. The present invention proposes a structured pruning strategy based on channel penalty of convolution kernel sensitivity. The penalized channels are re-evaluated according to the disease sensitivity evaluation algorithm after each Epoch training. Therefore, this strategy can allow the convolution kernels that were wrongly penalized or have potential in the previous Epoch to resume normal training in the next round of training. At the same time, structured pruning can truly reduce the number of convolution kernels to reduce the number of parameters and calculations, thereby reducing the complexity of the model.

[0064] 4. The present invention proposes to build an intermediate teacher model and use knowledge distillation to assist model fine-tuning training. On the one hand, using the above-mentioned structured pruning strategy to construct the intermediate teacher model can avoid the large gap between teacher and student resulting in poor auxiliary fine-tuning effect. On the other hand, the proportion of structured pruning can be further increased. The auxiliary fine-tuning technology of knowledge distillation can restore the original accuracy of the pruned small model and ensure the performance of the small model.

[0065] 5. The present invention conducts experiments on the Paddy Doctor Dataset public data set to prove the effectiveness of the proposed method. The data set contains 9 types of rice diseases and healthy rice pictures, which are taken during the period of 45 to 82 days of rice growth. The symptoms of different types of diseases are similar due to factors such as different developmental stages of rice, different disease sites and disease severity, as well as the characteristics of complex background. The experimental results show that the model parameter volume finally constructed using the method of the present invention is 0.379M, the average recognition accuracy is 98.8445, the average recall rate is 98.8445, the average accuracy rate is 98.8486, and the F1 value is 98.8440, which is higher than most deep learning models of the same period, and the confusion matrix shows that the model has significant performance in each category. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0067] Figure 1 is a partial sample of a data set in an embodiment of the present invention;

[0068] Figure 2 This is the overall process of lightweight network modeling for inter-class similarity and complex backgrounds according to an embodiment of the present invention;

[0069] Figure 3 Schematic diagram of the i-th Epoch data set division rule according to an embodiment of the present invention;

[0070] Figure 4 This is a similarity calculation process of two adjacent batch feature graphs in an embodiment of the present invention;

[0071] Figure 5 It is the relationship between each block Conv, BN, and Relu of the classic convolutional neural network in the embodiment of the present invention;

[0072] Figure 6 This is the classic channel attention SeNet workflow of an embodiment of the present invention;

[0073] Figure 7 It is the spatial attention addition based on the sensitive convolution kernel of an embodiment of the present invention;

[0074] Figure 8 is the γ distribution in the BN layer after channel penalty in an embodiment of the present invention;

[0075] Fig. 9 This is a schematic diagram of building an intermediate teacher model for auxiliary training according to an embodiment of the present invention;

[0076] Fig.10 is the change in model accuracy after each round of training of the test set of the embodiment of the present invention;

[0077] Fig.11 is the total loss change of the model on the test set after each round of training in the embodiment of the present invention

[0078] Fig.12 It is the confusion matrix of the model constructed in the embodiment of the present invention on the test set. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0080] Embodiment 1

[0081] The lightweight rice disease identification method based on sensitivity under complex background of the embodiment of the present invention comprises the following steps:

[0082] Step 1, obtain a data set containing real field complex background images, divide the data set into a training set and a test set, and preprocess the training set using image enhancement technology;

[0083] Step 2: Build a lightweight network model for inter-class similarity and complex backgrounds. The lightweight network model selects the VGG13 network and uses adaptive global average pooling to replace the fully connected layer. After each round of training, the training set is divided according to the specified rules, and the divided data is input into the lightweight network model for reasoning. The sensitive convolution kernel and the insensitive convolution kernel in this round of model are calculated according to the evaluation algorithm of the convolution kernel's sensitivity to different types of diseases.

[0084] In a preferred embodiment of the present invention, a specific method of evaluating the sensitivity of the convolution kernel to different types of diseases is provided:

[0085] In actual rice disease recognition, the symptoms of different types of diseases are similar due to factors such as the different developmental stages of rice, different disease sites, and disease severity. At the same time, considering that the contribution of each convolution kernel in image classification decision-making is different, a large number of convolution kernels with low contribution naturally make the model bloated. Therefore, the present invention designs an algorithm that can judge the sensitivity of each convolution kernel in each layer to different types of diseases, and performs channel penalties on the features extracted by the convolution kernels with weak sensitivity, and reversely allows the convolution kernels with high sensitivity to automatically learn the difference features between similar diseases.

[0086] Step 3: Based on the output of the evaluation algorithm in step 2, on the one hand, all feature channels output after the insensitive convolution kernels are penalized, and on the contrary, the convolution kernels with high sensitivity are made to automatically focus on the difference features between similar diseases of different categories during the model training process; on the other hand, spatial attention is added only to the output features of the sensitive convolution kernels after a certain feature layer, so that the convolution kernels with high sensitivity can focus on the diseased area while paying attention to the differences between diseases of similar categories;

[0087] In a preferred embodiment of the present invention, a method for channel penalty linkage spatial attention based on convolution kernel sensitivity is provided:

[0088] The rice disease image data collected in real rice fields have complex backgrounds such as stacked leaves. Too much useless information will mislead the model's judgment. Therefore, the present invention takes into account that the channel penalty based on the sensitivity evaluation algorithm contains the idea of ​​channel attention. It is intended to add spatial attention only to the features output by the convolution kernel with high sensitivity according to the results of the sensitivity evaluation algorithm, so that the convolution kernel with high sensitivity itself can automatically focus on the diseased area in the image.

[0089] Step 4: Perform structured pruning based on the results of the non-sensitive convolution kernel channel penalty. At the same time, construct an intermediate teacher model based on the structured pruning strategy to perform auxiliary fine-tuning training on the lightweight network model with excessive structured pruning.

[0090] In a preferred embodiment of the present invention, a specific method of a structured pruning strategy based on channel penalty of convolution kernel sensitivity is provided:

[0091] A large number of convolution kernels with low category sensitivity make the rice disease recognition model have many parameters and large computational complexity, making the model difficult to implement. The channel penalty based on the sensitivity of the convolution kernel makes the scaling factor corresponding to the BN layer after the insensitive convolution kernel in each layer tend to 0 during the training process. The convolution kernels corresponding to the scaling factors tending to 0 are structurally pruned from the model to reduce the complexity of the model.

[0092] In a preferred embodiment of the present invention, a method for fine-tuning a model by building an intermediate teacher model with the aid of knowledge distillation is provided:

[0093] After using structured pruning to prune off a large number of convolution kernels, the model may not be able to restore its accuracy through self-fine-tuning alone. Using a teacher model with a similar structure can assist in the fine-tuning training of the small model through knowledge distillation. However, directly using a larger teacher model may result in poor auxiliary training effect due to the large gap between the teacher and student models. Therefore, the present invention builds an intermediate teacher model based on the above-mentioned pruning strategy to assist in the fine-tuning of the over-pruned small model, thereby ensuring the model accuracy while further reducing the model complexity.

[0094] Step 5: Test the lightweight network model obtained after fine-tuning in step 4 using the test set to determine whether the model performance and model size meet the requirements. If not, re-execute step 4; if so, output a lightweight network model for inter-class similarity and complex backgrounds for rice disease identification.

[0095] The technical solution of the present invention solves the following technical problems:

[0096] (1) Solve the problem of model misjudgment caused by similar characteristics between classes of rice disease symptoms. The present invention proposes an evaluation algorithm for the sensitivity of convolution kernels to different types of diseases, which counts the sensitivity of each convolution kernel in each layer to different types of diseases, performs channel penalty on the features extracted by the convolution kernels with weak sensitivity, and reversely allows the convolution kernels with high sensitivity to automatically learn the difference features between similar diseases, thereby reducing the model misjudgment caused by similar diseases.

[0097] (2) Solve the problem of model misjudgment caused by complex backgrounds such as stacked leaves in rice disease image data collected in real rice fields. Considering that the channel penalty based on the sensitivity evaluation algorithm contains the idea of ​​channel attention, it is proposed to add spatial attention to the features output by the convolution kernel with high sensitivity according to the sensitivity evaluation algorithm at a certain layer of the network so that the convolution kernel with high sensitivity can automatically focus on the diseased area while extracting the difference features of similar diseases, thereby reducing the influence of complex background factors.

[0098] (3) Solve the problem that the rice disease model has a complex structure and cannot be deployed on resource-constrained edge terminals. The present invention first performs structured pruning on the model based on the results of the non-sensitive convolution kernel channel penalty to build an intermediate teacher model with a low pruning ratio, and further uses the intermediate teacher model with a similar structure to assist in fine-tuning the model with a high pruning ratio through knowledge distillation to restore the accuracy, and deletes a certain proportion of convolution kernels through structured pruning to reduce the complexity of the model.

[0099] Embodiment 2

[0100] 1. Training data preparation

[0101] The Paddy Doctor Dataset is a public RGB color dataset containing a total of 10,407 images, including rice diseases of 9 categories and healthy rice images, taken between 45 and 82 days of rice growth. The symptoms of different types of diseases are similar due to factors such as different developmental stages of rice, different disease sites, and disease severity. The disease images in this dataset are collected from real rice field diseases in India, so they have the characteristics of complex background factors such as leaf stacking. Figure 1 Some samples from the dataset are shown.

[0102] In order to improve the diversity of images and prevent overfitting problems during the model training process, this paper only uses image enhancement methods such as rotation, scaling, translation, and flipping on the training set, and uniformly adjusts the images to 104x104 JPG images to improve the image processing speed.

[0103] 2. Modeling lightweight networks for inter-class similarities and complex backgrounds

[0104] (1) Overview of the overall model framework

[0105] The overall process of model construction proposed in the embodiment of the present invention is as follows: Figure 2 shown.

[0106] First, the collected data set is divided into training set and test set, and then only the training set is subjected to image enhancement techniques such as rotation, scaling, and translation. The model selects the VGG13 network, and uses adaptive global average pooling to replace the fully connected layer. After each Epoch training, the training data set is divided according to the specified rules, and the divided data is input into the model for inference. The sensitive convolution kernel and the insensitive convolution kernel in this round of model are calculated according to the evaluation algorithm of the sensitivity of the convolution kernel to different categories of diseases. According to the results of the algorithm, on the one hand, the feature channels output after all insensitive convolution kernels are punished, and inversely, the convolution kernels with high sensitivity are made to automatically pay attention to the difference features between similar diseases of different categories during the model training process. On the other hand, after a certain feature layer, only the output features of the sensitive convolution kernel are added with spatial attention, so that the convolution kernels with high sensitivity can focus on the diseased area while paying attention to the differences between diseases of similar categories, reducing the impact of complex background. Since additional channel penalties are applied to non-sensitive convolution kernels during each round of training, the feature maps output by the non-sensitive convolution kernels tend to 0, and the contribution to the classification of the disease model gradually tends to 0. Therefore, structured pruning is performed based on the results of the non-sensitive convolution kernel channel penalty. At the same time, a low-pruning model with a similar structure is constructed based on the structured pruning strategy as an intermediate teacher model to assist in fine-tuning the small model with excessive structured pruning, further reducing the number of convolution kernels and the complexity of the model while ensuring model performance. Finally, the obtained lightweight model is tested to see whether the model performance and model size can comprehensively meet the actual needs. If not, the pruning, auxiliary fine-tuning training, and testing process are executed repeatedly.

[0107] (2) Design of convolution kernel sensitivity evaluation algorithm for different types of diseases

[0108] The symptoms of different types of diseases are similar due to factors such as the different developmental stages of rice, different disease sites, and disease severity. Even manual detection requires the inspectors to have very rich experience. Therefore, we consider designing an algorithm that can measure the sensitivity of each convolution kernel in each layer to different types of diseases, and at the same time provide a basis for the addition of channel penalties and spatial attention.

[0109] Considering that after each Epoch model training, the model has learned the entire training data set once and completed multiple gradient updates. At the same time, in order to accurately and timely judge the sensitivity of each convolution kernel to different types of diseases after the model training is completed, it is considered to calculate the sensitivity of each convolution kernel once after each Epoch training. In order to meet the requirements of the algorithm, the entire training set needs to be calculated as follows: Figure 3 As shown in the figure, there are three division rules for batch division:

[0110] ① Each batch contains rice disease image data of the same category and has the same number of images.

[0111] ② The i-th batch and the i+1-th batch of images belong to different categories, that is, the two adjacent batches of images in the same Epoch belong to different categories.

[0112] ③ The last batch of the i-th Epoch and the first batch of the i+1-th Epoch belong to different categories, that is, when different Epochs are connected, the image data of the two batches belong to different categories respectively.

[0113] After the model is trained in the i-th Epoch, the training data set is divided according to the above rules, and the training data is input into the model in the order of batch0, batch1, batch2, etc. for inference. The diseased images of batch1 are input into the model and the convolution kernel of each batch is recorded. i Image output i，l，j , where i represents the input image of the i-th batch, l represents the convolution layer index, and j represents the j-th convolution kernel in the l-th layer. Similarly, batch i+1 The diseased image is input into the model and each convolution kernel batch is recorded i+1 output i+1，l，j , and then use the squared Euclidean distance to calculate the feature map output i，l，j and output i+1，l，j The similarity of Figure 4 shown.

[0114] In order to judge the sensitivity of the convolution kernel from the entire convolutional neural network, and considering that the output feature maps of different convolutional layers have different sizes, the calculated similarity is divided by the width and height of each layer of feature map to obtain the final similarity. The specific formula is as follows:

[0115] similar(output i，l，j , output i+1，l，j )=sum((output i，l，j -output i+1，l，j ) 2 ) / h l / w l (1)

[0116] where h l 、w l They are the output of the convolution kernel of the lth layer respectively. i，l，j The height and width of the output i，l，j and output i+1，l，j They are the outputs of the same convolution kernel and therefore have the same height and width.

[0117] In order to define sensitive convolution kernels and insensitive convolution kernels, it is necessary to set a ratio of insensitive convolution kernels in advance, insensitive_ratio, calculate the similarity based on the two batch output feature maps before and after the convolution kernel, and sort the similarity from high to low. It is worth noting that the insensitive convolution kernel is not determined simply based on the similarity ranking of the two batch output feature maps, but the convolution kernels that appear in the insensitive_ratio are counted after the similarity comparison of the two batch output feature maps each time. After the similarity comparison of the entire training data set is completed, the convolution kernels that appear in the insensitive_ratio ratio are ranked from high to low according to the number of times they appear. The convolution kernels ranked in the top insensitive_ratio are finally determined as the insensitive convolution kernels in the model after the end of this round of training. This process of obtaining insensitive convolution kernels by statistically counting the number of indexes through multiple similarity comparisons can prevent some accidental events from misjudging the sensitivity of the convolution kernel.

[0118] In layman's terms: when the current and subsequent images of different categories are input into the same convolution kernel, but this convolution kernel always gives very similar outputs, then the present invention believes that this convolution kernel is insensitive between different categories, that is, the output of this convolution kernel does not contribute much to the model classification.

[0119] (3) Channel Penalty Linked Spatial Attention Based on Convolution Kernel Sensitivity

[0120] In the classic convolutional neural network construction, each block is usually composed of Conv, BN, and ReLu layers. The relationship between the three is as follows: Figure 5 shown.

[0121] That is, after a batch image passes through the Conv layer, it will pass through the BN layer and the ReLu layer in sequence. The calculation in the BN layer is shown in Formula 2:

[0122]

[0123] Each convolution kernel corresponds to a γ j , β j , γ j It is called the scaling factor, y and β represent the number of convolution kernels behind γ j , β j The collection of x l Represents the feature map of the l-th convolution output, Represents x l After normalization, the feature map y l Represents x l The output after the BN layer. The calculation of the ReLu layer is shown in formula 3:

[0124] f(y l )=max(0,y l ) (3)

[0125] Through formula 2, formula 3, Figure 3 And the associative law of multiplication can be obtained. Suppose we want to suppress the non-sensitive convolution kernel [conv i , conv k , ..., conv n ] can be obtained by penalizing the scaling factor [γ i , γ k , ..., γ n ], which ultimately causes the output features of the insensitive convolution kernel to tend to 0 as a whole after passing through the BN and ReLu layers due to too small weights.

[0126] Here, the penalty ratio punish_ratio = insensitive_ratio is set. According to the insensitive convolution kernel defined in the previous convolution kernel sensitivity evaluation algorithm for different types of diseases, the corresponding [γ i , γ k , ..., γ n ] is finally determined as the channel to be punished during the i+1th Epoch training. According to the design of the convolution kernel sensitivity evaluation algorithm for different types of diseases, the [γ i , γk , ..., γ n ] will be recalculated, and [γ i , γ k , ..., γ n ] is calculated based on the statistical number of multiple similarity comparisons. This channel penalty based on the sensitivity of the convolution kernel can prevent some accidental events from causing erroneous channel penalties, and the convolution kernel corresponding to the penalized channel can resume normal training in the next Epoch.

[0127] According to the previously determined γ index [γ i , γ k , ..., γ n ] and formula 2, give the corresponding BN [γ i , γ k , ..., γ n ] l Add an L1 regularization, as shown in Formula 4:

[0128] l1_reg=l1_reg+l1([γ i , γ k , ..., γ n ] l ) (4)

[0129] Where l1_reg represents the selected [γ i , γ k , ..., γ n ] l After calculating L1, it is accumulated, where l represents the layer index, and the accumulated l1reg is added to the loss function, as shown in Formula 5:

[0130] loss=loss+l1_stren*11_reg (5)

[0131] Among them, l1_stren is a hyperparameter, which indicates the penalty strength. According to the addition derivative formula and the chain derivative rule, the selected [γ i , γ k , ..., γ n ] l When the gradient is updated, as shown in Formula 6:

[0132]

[0133] Where α represents the learning rate, sign(*) is a sign function, and i , γ k , ..., γ n ] lAfter adding the L1 penalty, the scaling factors corresponding to these insensitive convolution kernels can be obtained according to Formula 6 [γ i , γ k , ..., γ n ] l The value of will be additionally subtracted from l1_stren*[sign(γ i ), sign(γ k ), ..., sign(γ n )] l , so that the non-sensitive convolution kernel corresponding to [γ i , γ k , ..., γ n ] l Due to the additional penalty, it quickly tends to 0, thus achieving the purpose of insensitive convolution kernel channel penalty.

[0134] In another preferred embodiment of the present invention, when the network continuously approaches the insensitive convolution kernel output to 0 during training and the model accuracy is improved during the test process, it is actually a disguised strengthening of the contribution of the output features of the category-sensitive convolution kernel. It can further illustrate that the channel penalty based on the sensitivity of the convolution kernel can enhance the feature extraction capability of the sensitive convolution kernel.

[0135] The schematic diagram of SeNet is as follows Figure 6 As shown, a batch of data passes through the convolution layer F tr Then we get the feature map U with height, width and channel respectively H, W and C. In order to calculate the importance of each channel, SeNet will perform global average pooling on the feature map U in the channel dimension to get a 1*1*C feature vector. The feature vector will then pass through a 1*1*C fully connected layer and then through an F scale (.,.) Normalize the output of the fully connected layer and scale the value to between 0 and 1. At this time, the weight of each channel in the feature map is obtained. After obtaining this weight, multiply this weight by the original input feature map to complete the scoring of each channel in the feature map. Figure 5 As shown in Figure 2, assuming that the convolutional layer is followed by a BN layer, the feature map weighted by SeNet will pass through the BNN layer, where Here is a description of the SeNet process, as shown in Equation 7:

[0136] ([x1, x2, ..., x c ]·[x w1 , x w2 , ..., x wc ])·[γ1,γ2,...,γ c ]→y l (7)

[0137] The channel penalty based on the sensitivity of the convolution kernel directly affects the [γ i , γ k , ..., γ n ] l Punishment is carried out, and the [γ i , γ k , ..., γ n ] l To calculate BN, the process is as shown in formula 8:

[0138] [x1, x2, ..., x c ]·([γ w1 , γ w2 =1,...,γ wc ]·[γ1,γ2,...,γ c ])→y l (8)

[0139] where x i represents the feature output by the i-th convolution kernel of the convolution layer, c represents the number of channels of the feature layer, and x wi represents the weight of the i-th feature map calculated by SeNet, γ i Indicates that the i-th feature map corresponds to the i-th γ in BN, γ wi Represents the original γ after the channel penalty of the non-sensitive convolution kernel i Compared to normal training updates, an additional scaling is performed, where γ w2 =1 means that the channel corresponds to a sensitive convolution kernel and is not additionally penalized, so the update scaling value is 1 compared to normal training. At the same time, for the sake of simplicity, equations 7 and 8 do not consider the data [x1, x2, ..., x c ], through equation 7, equation 8 and the combination rate of multiplication, it can be seen that the channel penalty based on the sensitivity of the convolution kernel can also distinguish the features in the channel dimension, thus containing the idea of ​​channel attention, and it is worth noting that from Figure 4 It can be seen that the addition of SeNet requires the introduction of an additional fully connected layer to complete the calculation of channel attention at a certain layer, while the channel penalty based on the sensitivity of the convolution kernel can calculate the channel importance of all layers in the network without introducing additional parameters.

[0140] Generally speaking, in convolutional neural networks, channel attention and spatial attention are generally used simultaneously to allow the model to automatically focus on important information from both channel and spatial dimensions. Considering that the images collected in rice fields have complex backgrounds such as stacked leaves, it is planned to add spatial attention to the feature map defined as the output of the sensitive convolution kernel after a certain BN layer according to the guidance of the convolution kernel sensitivity evaluation algorithm for different types of diseases, so that the convolution kernel with high sensitivity in the model can focus on the diseased area in the complex background when extracting features. The specific adding process is as follows: Figure 7 shown.

[0141] from Figure 7 It can be seen that, first, the sensitive convolution kernel and the insensitive convolution kernel in the convolution layer are calculated according to the convolution kernel sensitivity evaluation algorithm for different types of diseases to obtain sensitive channels and insensitive channels. Then, only the average pooling and maximum pooling are calculated for the sensitive channels in the channel dimension. Compared with the original spatial attention, which indiscriminately calculates the average and maximum values ​​in all channel dimensions, it ignores the channel features with low sensitivity and reduces a certain amount of calculation. At the same time, it can be analyzed that the sensitivity-based channel penalty is linked to the spatial attention, so that the features of the convolution kernel output are calculated in both the channel dimension and the spatial dimension, so that the sensitive convolution kernel can further focus on the diseased area and reduce the impact of complex background.

[0142] (4) Structured pruning strategy based on channel penalty of convolution kernel sensitivity

[0143] According to the previous analysis, not all convolution kernels in the model contribute equally to disease classification, that is, there are convolution kernels that are redundant for disease classification tasks. At the same time, as shown in Formula 6, the channel penalty based on the sensitivity of the convolution kernel makes the scaling factor in the BN corresponding to the insensitive convolution kernel closer to 0 due to the additional penalty during the training process compared to the scaling factor corresponding to the sensitive convolution kernel. At this time, the distribution of γ in each BN layer of the best model is roughly as follows: Figure 8 As shown, we can see that some γ i The absolute value of will be very small, and for the network these γ i The corresponding output feature map tends to 0 after ReLu. At the same time, during the training process of channel penalty, the features output by the sensitive convolution kernel will contribute more to the model classification, which will eventually lead to the deletion of γ. i The corresponding convolution kernel has little effect on the model accuracy, so the γ vectors in each BN layer of the best model obtained in the channel penalty training phase are collected and sorted from small to large according to the absolute value. Here, prune_ratio = punish_ratio = insensitive_ratio is set, and [γ] ranked within the top prune_ratio is selected from the global i , γ k , ..., γn ] channels, and finally delete the convolution kernels corresponding to these channels from the network.

[0144] At the same time, it should be noted that according to the algorithm design of channel penalty based on convolution kernel sensitivity, the scaling factor to be punished will be recalculated after each Epoch training in the channel penalty. Therefore, the pruning strategy based on the above channel penalty can ensure to a certain extent that the pruned convolution kernel is the result after multiple Epoch penalties, thereby preventing certain accidental events from causing the wrong convolution kernel to be pruned.

[0145] (5) Build an intermediate teacher model and use knowledge distillation to assist model fine-tuning training

[0146] Knowledge distillation is a teacher-student training structure. The student model is generally small and the difference in the output of the teacher-student model is calculated in the loss function to allow the student to simulate the output of the teacher model and learn the knowledge of the teacher model. The output of the classification layer of the teacher model can provide more inter-class knowledge than the original One-Hot encoded hard label in image classification. For example, when the three-class teacher model gives a picture with the probabilities of 0.5, 0.4, and 0.1 for bacterial leaf blight, bacterial stripe, and bacterial ear blight, it can be concluded from the probability that bacterial leaf blight and bacterial stripe are very similar, while bacterial leaf blight and bacterial ear blight are not very similar. This allows the small model to learn more inter-class knowledge during the distillation training process to improve performance.

[0147] After the model is over-pruned, it may not be possible to restore the original accuracy by fine-tuning the model itself. Considering that the teacher model in knowledge distillation can provide more information about different types of diseases, which is helpful for identifying similar diseases, knowledge distillation is used to assist in fine-tuning the small model. At the same time, considering that the gap between the small model and the teacher model is too large after over-pruning, resulting in poor auxiliary fine-tuning effect, this module plans to first build an intermediate teacher model with a similar structure through the above-mentioned structured pruning strategy, such as Fig. 9 shown.

[0148] During the fine-tuning training process, on the one hand, the image data is passed through the small model after structured pruning to obtain the classification layer output S(x), and on the other hand, the image data is passed through the intermediate teacher model to obtain the classification layer output T(x). In the loss function, on the one hand, the cross entropy between the output of the small model and the true label y is calculated, and on the other hand, the KL divergence is used to calculate the difference between the student and teacher outputs. Finally, when the disease model after structured pruning is fine-tuned, the training loss function is:

[0149] Loss=(1-α)·CrossEntropy(S(x),y)+α·KLDiv(T(x),S(x)) (9)

[0150] Among them, α is the balance parameter of the distillation loss, which is used to adjust the weight between the two loss terms. S(x) represents the output of the student model classification layer, y represents the true label in the original classification image, CrossEntropy(.,.) represents the cross entropy loss function, and KLDiv(.,.) is the KL divergence loss of the two distributions. The two loss terms allow the small model to learn both the original hard label knowledge and more inter-class knowledge during fine-tuning to improve performance. When fine-tuning the model after pruning, the introduction of an intermediate teacher model can assist model training and prevent the student model and the teacher model from being too different, resulting in poor model "teaching".

[0151] 3. Experimental Analysis Example

[0152] In order to demonstrate the performance of the model constructed by the method of the embodiment of the present invention and to evaluate the network overall, this study uses the precision, recall, accuracy, and F1-score indicators to evaluate the effectiveness of the model in identifying rice leaf diseases. The calculation formulas for these indicators are as follows:

[0153]

[0154]

[0155]

[0156]

[0157] Among them, T p Represents the number of samples correctly classified as the corresponding disease, F p represents the number of samples that do not belong to the disease category but are misclassified as this category, F n represents the number of samples belonging to this disease category but mistakenly classified into other categories, T n Represents the number of samples that do not belong to this type of disease and are correctly identified. Precision is the proportion of true positive examples predicted by the model; Recall is the proportion of true positive examples predicted by the model; Accuracy is the proportion of correctly classified samples to the total number of samples; F1-score is the weighted average of precision and recall, and the larger the value, the better the performance of the model.

[0158] Fig.10 , Fig.11 This paper describes the performance of the small model finally constructed by this patent in the intermediate teacher model assisted by knowledge distillation. The test accuracy of the model increases with the increase of epochs and stabilizes after about 80 epochs.

[0159] Through calculation, it is found that the model parameter volume of the patent is 0.379M, the average recognition accuracy is 98.8445, the average recall rate is 98.8445, the average accuracy is 98.8486, and the F1 value is 98.8440. At the same time, the patent method pruned the 0.6 and 0.8 scale convolution kernels, and the model performance was compared with the baseline method when the model parameters were 1.311M and 0.379M respectively. The results are shown in Table 1. It can be seen that the accuracy of the model constructed by the proposed method is higher than that of other baseline models.

[0160] Table 1 Performance comparison of VGG13 model and baseline methods on public datasets at different pruning ratios

[0161]

[0162] In addition to the overall performance of the model, plotting the confusion matrix such as Fig.12 Observe the performance of the model on each category, where the horizontal axis represents the model prediction results and the vertical axis represents the true category. It can be seen that the constructed model performs stably on all rice disease categories, with an average accuracy between 97% and 99%.

[0163] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0164] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A lightweight rice disease identification method based on sensitivity under complex background, characterized in that: The method comprises the following steps: Step 1, obtain a data set containing real field complex background images, divide the data set into a training set and a test set, and preprocess the training set using image enhancement technology; Step 2: Build a lightweight network model for inter-class similarity and complex backgrounds. The lightweight network model selects the VGG13 network and uses adaptive global average pooling to replace the fully connected layer. After each round of training, the training set is divided according to the specified rules, and the divided data is input into the lightweight network model for reasoning. The sensitive convolution kernel and the insensitive convolution kernel in this round of model are calculated according to the evaluation algorithm of the convolution kernel's sensitivity to different types of diseases. Step 3: Based on the output of the evaluation algorithm in step 2, on the one hand, all feature channels output after the insensitive convolution kernels are penalized, and on the contrary, the convolution kernels with high sensitivity are made to automatically focus on the difference features between similar diseases of different categories during the model training process; on the other hand, spatial attention is added only to the output features of the sensitive convolution kernels after a certain feature layer, so that the convolution kernels with high sensitivity can focus on the diseased area while paying attention to the differences between diseases of similar categories; Step 4: Perform structured pruning based on the results of the non-sensitive convolution kernel channel penalty. At the same time, construct an intermediate teacher model based on the structured pruning strategy to perform auxiliary fine-tuning training on the lightweight network model with excessive structured pruning. Step 5: Test the lightweight network model obtained after fine-tuning in step 4 using the test set to determine whether the model performance and model size meet the requirements. If not, re-execute step 4; if so, output a lightweight network model for inter-class similarity and complex backgrounds for rice disease recognition; The method for penalizing the feature channel output after the non-sensitive convolution kernel in step 3 includes: Set the penalty ratio punish_ratio = insensitive_ratio; a batch of images passes through the Conv layer and then the BN layer and the ReLu layer in turn; according to the obtained insensitive convolution kernel, the [γ i , γ k , ..., γ n ] is finally determined as the channel to be punished during the i+1th Epoch training. According to the result of multiple similarity comparison statistics, the [γ i , γ k , ..., γ n ] to recalculate; According to the determined γ index [γ i , γ k , ..., γ n ] and the calculation formula of the BN layer, give the corresponding BN [γ i , γ k , ..., γ n ]l plus an L1 regularization, the formula is: l1_reg=l1_reg+l1([γ i ,c k ,...,c n ] l ) Among them, l1_reg represents the selected [γ i ,γk,...,γ n ] l After calculating L1, it is accumulated, l represents the layer index, and the accumulated l1_reg is added to the loss function, the formula is: loss=loss+l1_stren*l1_reg Among them, l1_stren is a hyperparameter, which indicates the penalty strength. According to the addition derivative formula and the chain derivative rule, the selected [γ i , γ k , ..., γ n ] l The formula for gradient update is: Among them, α represents the learning rate, sign(*) is a sign function, and in [γ i , γ k , ..., γ n ] l After adding the L1 penalty, the scaling factor corresponding to the insensitive convolution kernel is obtained [γ i , γ k , ..., γ n ] l The value of will be additionally subtracted from l1_stren*[sign(γ i ), sign(γ k ), ..., sign(γ n )] l , let the non-sensitive convolution kernel corresponding to [γ i , γ k , ..., γ n ] l Due to the additional penalty, it quickly approaches 0, thus achieving the purpose of non-sensitive convolution kernel channel penalty; The method of adding spatial attention to the output features of the sensitive convolution kernel in step 3 includes: According to the guidance of the algorithm for evaluating the sensitivity of convolution kernels to different types of diseases, spatial attention is added to the feature map defined as the output of the sensitive convolution kernel after a certain BN layer; The channel penalty based on the convolution kernel sensitivity directly affects the [γ i , γ k , ..., γ n ] l Punishment is carried out, and the [γ i , γ k ,... ， γ n ] l The process of calculating BN is as follows: [x1, x2,..., x c ]·([c w1 ,c w2 =1,...,γw c] ·[γ1,γ2,...,γ c ])→y l Among them, x i represents the feature output by the i-th convolution kernel of the convolution layer, c represents the number of channels of the feature layer, and γ i Indicates that the i-th feature map corresponds to the i-th γ in BN, γ wi It means that after the channel penalty of the non-sensitive convolution kernel, the original γi will be additionally scaled compared to the normal training update, where yw2=1 means that the channel corresponds to the sensitive convolution kernel and is not additionally penalized; The method for constructing the intermediate teacher model in step 4 includes: After over-structural pruning of the model, knowledge distillation is used to assist in fine-tuning the model; an intermediate teacher model with a similar structure is built through the structured pruning strategy in step 4; During the fine-tuning training process, on the one hand, the image data is passed through the small model after structured pruning to obtain the classification layer output S(x), and on the other hand, the image data is passed through the intermediate teacher model to obtain the classification layer output T(x); in the loss function, on the one hand, the cross entropy between the output of the small model and the true label y is calculated, and on the other hand, the KL divergence is used to calculate the difference between the student and teacher outputs. Finally, when the disease model after structured pruning is fine-tuned, the training loss function is: Loss=(1-α)·CrossEntropy(S(x),y)+a·KLDiv(T(x),S(x)) Among them, α is the balance parameter of the distillation loss, which is used to adjust the weight between the two parts of the loss term, S(x) represents the output of the student model classification layer, y represents the true label in the original classification image, CrossEntropy(.,.) represents the cross entropy loss function, and KLDiv(.,.) is used to calculate the two distributions. KL Divergence loss.

2. The lightweight rice disease identification method based on sensitivity under complex background according to claim 1 is characterized in that: The method in step 1 comprises: The dataset contains images of no less than 9 types of rice diseases and healthy rice, which were taken between 45 and 82 days of rice growth. The symptoms of different types of diseases are similar due to factors such as different developmental stages of rice, different disease sites, and disease severity, as well as complex backgrounds. Methods for preprocessing images include: rotation, scaling, translation, flipping, and adjusting to a uniform size.

3. The lightweight rice disease identification method based on sensitivity under complex background according to claim 1 is characterized in that: In step 2, after each round of training, the training set is divided according to the specified rules, and the divided data is input into the lightweight network model for reasoning. The specific method is: After each Epoch training, the sensitivity of each convolution kernel is calculated once, and the entire training set is divided into batches according to the rules. The division rules include: ① Each batch contains rice disease images of the same category and has the same number of images; ② The images in the i-th batch and the i+1-th batch belong to different categories, that is, the two adjacent batch images in the same Epoch belong to different categories; ③ The last batch of the i-th Epoch and the first batch of the i+1-th Epoch belong to different categories, that is, when different Epochs are connected, the images of the two batches belong to different categories respectively; After the lightweight network model completes the i-th Epoch training, the training data set is divided according to the rules, and the training data is input into the model in the order of batch0, batch1, and batch2 for inference.

4. The lightweight rice disease identification method based on sensitivity under complex background according to claim 3 is characterized in that: The evaluation algorithm of the sensitivity of different types of diseases according to the convolution kernel in step 2 is specifically: Batch i The image is input into the model and each convolution kernel batch is recorded i Image output i，l，j ; where i represents the input image of the i-th batch, l represents the convolution layer index, j represents the j-th convolution kernel in the i-th layer, and similarly, batch i+1 The diseased image is input into the model and each convolution kernel batch is recorded i+1 output i+1，l，j , and then use the squared Euclidean distance to calculate the feature map output i,l，j and output i+1，l，j similarity; The calculated similarity is divided by the width and height of each layer of feature map to get the final similarity. The formula is as follows: similar(output i，l，j ,output i+I，l，j )=sum((output i，l，j -output i+I，l，j ) 2 ) / h l / w l Among them, h l 、w l They are the output of the convolution kernel of the lth layer respectively. i，l，j The height and width of output i,l，j and output i+1，l，j They are the outputs of the same convolution kernel and therefore have the same height and width; In order to define sensitive convolution kernels and insensitive convolution kernels, set a ratio of insensitive convolution kernels in advance. Calculate the similarity based on the two batch output feature maps before and after the convolution kernel, and sort the similarities from high to low. After each similarity comparison of the output feature maps of two batches, the convolution kernels that appear within the insensitive_ratio are counted. After the similarity comparison of the entire training data set is completed, the convolution kernels that appear within the insensitive_ratio are ranked from high to low according to the number of times they appear. The convolution kernels with the highest insensitive_ratio are finally determined as the insensitive convolution kernels in the model after the end of this round of training.

5. The lightweight rice disease identification method based on sensitivity under complex background according to claim 1 is characterized in that: The method for performing structured pruning in step 4 includes: In the model obtained by channel penalty training in step 3, the γ vectors in each BN layer are collected and sorted from small to large according to the absolute value, and prune_ratio = punish_ratio = insensitive_ratio is set. [γ vectors] ranked within the top prrune_ratio are selected from the global i , γ k , ..., γ n ] channels and delete the convolution kernels corresponding to these channels from the model.

6. The lightweight rice disease identification method based on sensitivity under complex background according to claim 1, characterized in that: The method for determining the performance in step 5 includes: The precision, recall, accuracy, and F1 score indicators are used to evaluate the effectiveness of the model in identifying rice leaf diseases. The lightweight network model obtained after fine-tuning in step 4 is tested. As the Epoch increases, the test accuracy of the model increases until the test accuracy stabilizes.

7. A lightweight rice disease recognition system based on sensitivity under complex background, characterized in that: include: A data set preprocessing unit is used to obtain a data set containing real field complex background images, divide the data set into a training set and a test set, and preprocess the training set using image enhancement technology; The evaluation algorithm unit of the sensitivity of convolution kernels to different types of diseases is used to build a lightweight network model for inter-class similarity and complex backgrounds. The lightweight network model selects the VGG13 network and uses adaptive global average pooling to replace the fully connected layer. After each round of training, the training set is divided according to the specified rules, and the divided data is input into the lightweight network model for reasoning. The sensitive convolution kernel and the insensitive convolution kernel in this round of model are calculated according to the evaluation algorithm of the sensitivity of convolution kernels to different types of diseases. The channel penalty-linked spatial attention unit based on the sensitivity of the convolution kernel is used to penalize the feature channels output after all insensitive convolution kernels according to the output of the evaluation algorithm, and conversely, let the convolution kernels with high sensitivity automatically pay attention to the difference features between similar diseases of different categories during the model training process; on the other hand, after a certain feature layer, only add spatial attention to the output features of the sensitive convolution kernel, so that the convolution kernels with high sensitivity can focus on the diseased area while paying attention to the differences between diseases of similar categories; The structured pruning unit based on the channel penalty of the convolution kernel sensitivity is used to perform structured pruning according to the results of the non-sensitive convolution kernel channel penalty. At the same time, an intermediate teacher model is constructed based on the structured pruning strategy to assist in fine-tuning the lightweight network model with excessive structured pruning. The model testing and output unit is used to test the lightweight network model obtained after fine-tuning through the test set to determine whether the model performance and model size meet the requirements. If not, the structured pruning unit based on the channel penalty of the convolution kernel sensitivity is entered for retraining. If the conditions are met, a lightweight network model for class similarity and complex background is output for rice disease recognition. The method for penalizing the feature channel output after the insensitive convolution kernel in the channel penalty linkage spatial attention unit based on the convolution kernel sensitivity includes: Set the penalty ratio punish_ratio = insensitive_ratio; a batch image passes through the Cony layer and then the BN layer and the ReLu layer in turn; according to the obtained insensitive convolution kernel, the [γ i , γ k , ..., γ n ] is finally determined as the channel to be punished during the i+1th Epoch training. According to the result of multiple similarity comparison statistics, the [γ i , γ k , ..., γ n ] to recalculate; According to the determined γ index [γ i , γ k , ..., γ n ] and the calculation formula of the BN layer, give the corresponding BN [γ i , γ k , ..., γ n ] l Add an L1 regularization, the formula is: l1_reg=l1_reg+l1([γ i ,c k ,...,c n ] l ) Among them, l1_reg represents the selected [γ i , γ k , ..., γ n ]lCalculate L1 and then accumulate it, l represents the layer index, and add the accumulated l1_reg to the loss function, the formula is: loss=loss+l1_stren*l1_reg Among them, l1_stren is a hyperparameter, which indicates the penalty strength. According to the addition derivative formula and the chain derivative rule, the selected [γ i , γ k , ..., γ n ] l The formula for gradient update is: Among them, α represents the learning rate, sign(*) is a sign function, and in [γ i , γ k , ..., γ n ] l After adding the L1 penalty, the scaling factor corresponding to the insensitive convolution kernel is obtained [γ i , γ k , ..., γ n ] ι The value of l1_stren will be additionally subtracted when the gradient is updated * [sign(γ i ), sign(γ k ), ..., sign(γ n )] ι , let the non-sensitive convolution kernel corresponding to [γ i , γ k , ..., γ n ] ι Due to the additional penalty, it quickly approaches 0, thus achieving the purpose of non-sensitive convolution kernel channel penalty; Methods for adding spatial attention to sensitive convolution kernel output features include: According to the guidance of the algorithm for evaluating the sensitivity of convolution kernels to different types of diseases, spatial attention is added to the feature map defined as the output of the sensitive convolution kernel after a certain BN layer; The channel penalty based on the convolution kernel sensitivity directly affects the [γ i , γ k , ..., γ n ] ι Punishment is carried out, and the [γ i , γ k , ..., γ n ] ι The process of calculating BN is as follows: [x1, x2,…, x c ]·([c w1 ,c w2 =1,...,c wc ]·[γ1,γ2,...,γ c ])→y l Among them, x i represents the feature output by the i-th convolution kernel of the convolution layer, c represents the number of channels of the feature layer, and γ i Indicates that the i-th feature map corresponds to the i-th γ in BN, γ wi Represents the original γ after the channel penalty of the non-sensitive convolution kernel i Compared to normal training updates, an additional scaling is performed, where γ w2 =1 means that the channel corresponds to a sensitive convolution kernel and is not additionally penalized; The method for constructing an intermediate teacher model in the structured pruning unit based on the channel penalty of the convolution kernel sensitivity includes: After over-structural pruning of the model, knowledge distillation is used to assist in fine-tuning the model; an intermediate teacher model with a similar structure is built through the strategy of structured pruning; During the fine-tuning training process, on the one hand, the image data is passed through the small model after structured pruning to obtain the classification layer output S(x), and on the other hand, the image data is passed through the intermediate teacher model to obtain the classification layer output T(x); in the loss function, on the one hand, the cross entropy between the output of the small model and the true label y is calculated, and on the other hand, the KL divergence is used to calculate the difference between the student and teacher outputs. Finally, when the disease model after structured pruning is fine-tuned, the training loss function is: Loss=(1-α)·CrossEntropy(S(x),y)+a·KLDiv(T(x),S(x)) Among them, α is the balance parameter of the distillation loss, which is used to adjust the weight between the two parts of the loss terms, S(x) represents the output of the student model classification layer, y represents the true label in the original classification image, CrossEntropy(.,.) represents the cross entropy loss function, and KLDiv(.,.) is the KL divergence loss for calculating the two distributions.

Images