Rice disease identification method and system based on CNN-CBAM-BiGRU combined model

Abstract

The invention provides a rice disease identification method and system based on a CNN-CBAM-BiGRU combined model. According to the method, richer feature combinations are obtained by improving the CNN, the influence of dimension reduction on data is weakened by improving a CBAM attention mechanism, and information loss is avoided through BiGRU. The method can effectively overcome the defects that a traditional convolutional neural network lacks global attention to the context and a recurrent neural network lacks local attention. In an experiment, through comparison with classical models such as VGG16, LeNet, AlexNet, GoogleNet V1 and RestNet 50, a result shows that the recognition accuracy of the CNN-CBAM-BiGRU combined model provided by the invention on the four diseases is as high as 98.03%, and a reliable recognition method is provided for rice disease recognition research.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a rice disease recognition method and system based on a CNN-CBAM-BiGRU joint model. Background Art

[0002] Rice is one of the most important food crops in my country. Ensuring the stability of rice yield is related to my country's food security. Rice diseases can occur throughout the entire growth cycle of the crop, which not only causes losses to rice production, but also threatens food safety. Taking effective measures to timely discover and deal with this problem can ensure the stability of the crop's yield. The traditional way for farmers to identify diseases is through human observation or consulting agricultural technicians. In addition, it is difficult to consult experts due to geographical restrictions. These methods are not only inefficient, subjective, but also time-consuming and labor-intensive. For crop disease recognition, the research on machine learning technology in the field of image recognition is constantly developing. For example, support vector machine (SVM), K-means clustering algorithm and fuzzy C-means algorithm have been well verified in the field of image recognition and have a high accuracy rate for identifying targets. However, for objects with complex image features that are not obvious or with similar features, the recognition effect will be reduced. In recent years, image recognition based on deep learning has become the focus of many researchers. It has been proven to be more effective in various fields, and people have begun to use deep learning methods to deal with crop disease recognition.

[0003] Among deep learning methods, convolutional neural network (CNN) is one of the representative deep learning technologies and has gradually become the mainstream method in the field of computer vision. It can set multiple convolution kernels to obtain multi-angle local features and obtain global features by superimposing multiple layers of convolution. It has been widely used in fields such as recognition and classification and has achieved good results. M.Lin et al. proposed a new deep network structure of NIN to enhance the model discriminability of local patches in the receptive field and constructed a micro neural network with a more complex structure to abstract the data in the receptive field. It is easier to interpret than the traditional fully connected layer and is not easy to overfit. AGHoward et al. divided the standard convolution into two parts: deep convolution and pointwise convolution, which effectively reduced the number of parameters and improved the running speed of traditional CNN. Qiu Jing et al. used a model constructed by using a deep convolutional network with a convolution size of 9×9 and maximum pooling to establish a rice disease recognition model. After 5 iterations, the recognition accuracy of the model can reach more than 90%, with strong generalization ability, high accuracy, good robustness and small loss rate. Meng Liang et al. proposed a lightweight CNN crop disease recognition model with residual units as basic elements. It can simplify the model structure and expand the applicable scenarios of the model while ensuring the recognition accuracy of the model. Wang Guowei et al. used the LeNet model to identify five corn diseases, selected the Adam algorithm instead of the SGD algorithm to optimize the model, adjusted the learning rate by the exponential decay method, added the L2 regularization term to the cross entropy function, and selected the Dropout strategy and ReLU excitation function. Finally, a 10-layer CNN network structure was determined, and the average recognition rate reached 96%. Xu Jianpeng et al. proposed a convolutional neural network based on the RectifiedAdam optimizer to automatically identify the growth period of rice, and the comprehensive recognition rate reached 97.33%. Barbedo et al. discussed the application of deep learning and transfer learning in plant disease classification, and found that the combined use of CNN and transfer learning provides an effective method for plant disease recognition. In the field of text processing, S. Lai et al. found that RCNN uses a bidirectional recurrent neural network to obtain contextual information that affects the current text from the forward and reverse directions, and retains key features through the maximum pooling operation in CNN. Kamal et al. embedded the constructed deep separable convolutional architecture into the MobileNet network and compared the optimizers of SGD, Adam and Nadam. The experimental results show that the classification accuracy of MobileNet on the Plant Village dataset is reduced by 98.34%, among which the Adam optimizer has better performance and faster network convergence speed.Khan et al. proposed an apple disease detection system based on deep learning. First, they built an apple disease dataset annotated by experts, which contained about 9,000 high-quality RGB images. Then, they customized a lightweight classification model for detection. The detection stage actually detected and located each symptom from the diseased leaf image. Ahad et al. focused on the applicability comparison of CNN technology in rice disease detection and location and the fact that only a few diseases were considered in rice disease detection research. They used the nine most popular rice disease databases in Bangladesh to classify six CNN deep learning architectures, compared six individual CNN networks, transfer learning, and ensemble techniques, and the results showed that the ensemble framework had the highest accuracy of 98%.

[0004] Although these studies have achieved good results, the convolution feature extraction pooling method will lose some information. CNN focuses on local feature information and lacks attention to global feature information, which is one of the reasons for the high misrecognition rate. In addition, the model parameters are large and the cost is high. Therefore, it is limited to establish the recognition of image feature information through a single neural network. In recent years, many scholars have tried to fuse neural networks for joint training. GAOBO LIANG et al. introduced RNN, combined CNN and RNN, and proposed a CNN-RNN framework to deepen the understanding of image content. The accuracy rate in the study of blood cell image recognition reached 90.79%. In the study of abnormal working conditions of oil wells, Li Kewen et al. proposed an improved network structure CBiA-PSL model based on the CNN-BiGRU joint network. CNN was used to learn the deep features of the grayscale images of working condition samples. BiGRU effectively avoided information loss and strengthened the connection of CNN pooling layer features, thereby improving recognition accuracy. Zhao Y et al. proposed a deep learning network based on multi-model fusion and embedded improved attention. The classification accuracy of corn, potato and tomato reached 98.44%, 99.43% and 95.20% respectively. Chen et al. proposed a bearing fault diagnosis based on multi-scale CNN and LSTM, using two CNNs with different kernel sizes to extract different frequency signal features from the original data, and then using LSTM to identify the fault type. L. Guo et al. proposed a hybrid model CRAN based on CNN-RNN and attention, which fully utilized the advantages of multiple single models and effectively combined convolutional neural networks and recurrent neural networks with the help of attention mechanisms to achieve better results. Zhao et al. embedded the improved CBAM module into the ResNet network and discussed the impact of the embedding position of CBAM on plant disease classification. The improved model has a classification accuracy of 95.37% for tomato diseases and 97.59% for various diseases. He et al. added a BN layer to the convolution layer of Faster R-CNN to improve the convergence speed, used the center cost algorithm as the loss function to improve the accuracy of disease recognition, and used the convolution layer structure of VGG16 as a feature extraction network. The average accuracy of corn disease diagnosis can reach 97.23%. et al. integrated the proposed convolutional neural network model and four other common network models (MobileNetV3Small, EfficientNetV2L, InceptionV3, and MobileNetV2) into a CNNs model, then used the PSO algorithm for hyperparameter optimization and used the grid search algorithm to redistribute the weights. The verification accuracy on the tomato disease dataset was 99.6%. Borhani et al. proposed a lightweight deep learning method based on visual transformer (ViT) for real-time automatic plant disease classification. The classic convolutional neural network (CNN) method and the combination of CNN and ViT were also used for plant disease classification. The experiment concluded that although the attention block improved the accuracy, it reduced the prediction speed. Combining the attention block with the CNN block can make up for the lack of speed. Based on ViT, Wang et al. introduced the pyramid structure in the CNN architecture to improve ViT and proposed the PyramidVisionTransformer model, which overcomes the difficulty of transplanting Transformer to various dense prediction tasks. The model uses a progressively shrinking pyramid to reduce the computational complexity of large feature maps. The results show that it improves the performance of many downstream tasks, including object detection, instance and semantic segmentation.

[0005] The above research methods prove the feasibility of the neural network fusion model. In the rice disease recognition method, there are few studies on the recognition of diseases by fusion network. Therefore, through the mutual assistance of multiple models, joint training can effectively overcome the limitations of a single model, thereby improving recognition efficiency and accuracy. Based on the existing theoretical basis, the present invention proposes a CNN-CBAM-BIGRU network model, which is a method for studying the recognition of rice diseases by fusion network, and the relationship between features can be better understood by fusing and improving the model. Summary of the invention

[0006] The purpose of the present invention is to solve the problems in the prior art and propose a rice disease recognition method and system based on a CNN-CBAM-BiGRU joint model.

[0007] The present invention is implemented by the following technical scheme. The present invention proposes a rice disease recognition method based on the CNN-CBAM-BiGRU joint model, and the method comprises the following steps:

[0008] Step 1: Collect data and construct a data set based on the collected data;

[0009] Step 2: Construct a CNN-CBAM-BiGRU joint model, use CNN to extract features from input data, and extract rich multi-dimensional features from the image; then, input the features extracted by CNN into CBAM, which is used to enhance meaningful features and suppress unimportant features. The CBAM refines the features from two dimensions, channel and space, to improve the expressiveness of the features; finally, input the features processed by CBAM into BiGRU, which is used to capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition;

[0010] Step 3: Use the constructed dataset to train the CNN-CBAM-BiGRU joint model;

[0011] Step 4: Input the actual data into the trained model to complete the identification of rice diseases.

[0012] Furthermore, the CNN is improved based on the Inception v1 module by increasing the depth of the network. First, two 3×3 convolutions are used to replace the 5×5 convolution, which reduces the computational complexity of the model and increases the receptive field, making it more conducive to detecting larger lesions. Second, a 3×3 convolution and a 1×1 convolution are used to replace a branch. The 3×3 convolution is used to extract information about smaller lesions to avoid losing some key information. The convolution kernel of 1×1 is used for upsampling and downsampling, which can increase or decrease the number of channels without changing the size of the feature map. Finally, the output of the first part is combined with the output of the second part to obtain a richer feature combination.

[0013] Furthermore, the bidirectional gated recurrent unit BiGRU is composed of a forward GRU and a backward GRU; the forward GRU processes from the start position to the end position of the sequence to capture information from the past to the present; the backward GRU processes from the end position to the start position of the sequence to capture information from the future to the present; finally, the forward hidden state and the backward hidden state of the current time step are concatenated to obtain the output of the BiGRU at that time step.

[0014] Furthermore, the output is determined by two GRUs at the same time, and the specific formula is expressed as:

[0015]

[0016] in They are forward propagation and backward propagation respectively; and are the weights for forward propagation and backward propagation respectively.

[0017] Furthermore, the CBAM as a whole has two parts, CAM and SAM, which sequentially enhance the important information in the channel and spatial features; the input feature map F first enters the CAM, and is respectively compressed from different spatial directions by average pooling and maximum pooling to highlight the channel features, and is simultaneously passed to the shared network to extract the channel attention weight. The shared network includes two multi-layer perceptrons MLP and one hidden layer, which can be expressed by the formula:

[0018]

[0019] Among them, M c (F) is the weight of feature map F in channel C, and The channel features are obtained after the average pooling and maximum pooling operations, and σ represents the Sigmoid activation function. In SAM, the results of the CAM output are also aggregated by two pooling operations to obtain the channel information of the feature map to obtain the average and maximum features of the entire channel. The information of different spaces is combined through the splicing operation and convolution to generate the spatial attention weight. The generation process can be expressed as:

[0020]

[0021] Among them, f 7×7 represents a convolution operation with a convolution kernel size of 7×7. and The spatial features are obtained after average pooling and maximum pooling operations.

[0022] Furthermore, the improved structure of the channel attention module CAM is as follows: first, one-dimensional convolution is used to weaken the impact of dimensionality reduction on the data, and then the second fully connected layer in the MLP is replaced by a long short-term memory (LSTM) network based on the recurrent neural network (RNN) structure, and time series are added to the channel to more accurately locate disease characteristics.

[0023] Furthermore, LSTM introduces a three-gate design in RNN: the forget gate F(t) filters the memory information in the output features of the previous moment that is valuable to the time unit and subsequent time units, the input gate I(t) filters the important feature information at the current moment t, and the output gate O(t) extracts the information that has the greatest impact on the output of the current unit. The results are transmitted between gates by weight product.

[0024] The present invention also proposes a rice disease recognition system based on a CNN-CBAM-BiGRU joint model, the system comprising:

[0025] Collection module: collects data and constructs a data set based on the collected data;

[0026] Building modules: Building a CNN-CBAM-BiGRU joint model, using CNN to extract features from input data, and being able to extract rich multi-dimensional features from images; then, inputting the features extracted by CNN into CBAM, which is used to enhance meaningful features and suppress unimportant features, and the CBAM refines the features from two dimensions, channel and space, to improve the expressiveness of the features; finally, inputting the features processed by CBAM into BiGRU, which is used to capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition;

[0027] Training module: Use the constructed dataset to train the CNN-CBAM-BiGRU joint model;

[0028] Identification module: Input the actual data into the trained model to complete the identification of rice diseases.

[0029] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the rice disease identification method based on the CNN-CBAM-BiGRU joint model are implemented.

[0030] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the rice disease identification method based on the CNN-CBAM-BiGRU joint model.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] In view of the limitations of convolutional neural networks in rice disease identification and the high misrecognition rate, the present invention adopts a neural network fusion method in deep learning and proposes a joint neural network model of CNN-CBAM-BiGRU for rice disease identification. The model uses a convolutional neural network to extract sample features and transfers them to a special recurrent neural network for joint training. By improving CNN, a richer feature combination is obtained, the impact of dimensionality reduction on data is weakened by improving the CBAM attention mechanism, and information loss is avoided by BiGRU. This method can effectively overcome the lack of global attention to context in traditional convolutional neural networks and the lack of local attention to recurrent neural networks. In the experiment, by comparing with models such as VGG16, LeNet, AlexNet, GoogleNet V1, RestNet50, etc., the results show that the recognition accuracy of the CNN-CBAM-BiGRU joint model proposed in the present invention for four diseases is as high as 98.03%, providing a reliable recognition method for rice disease recognition research. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0034] Figure 1 These are images of rice leaf diseases, including (a) rice blast, (b) bacterial leaf blight, (c) sesame spot, (d) sheath blight, and (e) healthy leaves.

[0035] Figure 2 It is a schematic diagram of data enhancement for sesame spot disease images.

[0036] Figure 3 It is a schematic diagram of a convolutional neural network.

[0037] Figure 4 It is a schematic diagram of the improved CNN model.

[0038] Figure 5 It is a schematic diagram of a recurrent neural network.

[0039] Figure 6 It is a schematic diagram of the internal structure of the GRU unit.

[0040] Figure 7 It is a schematic diagram of the network structure of bidirectional GRU.

[0041] Figure 8 It is a schematic diagram of the CBAM module structure.

[0042] Fig. 9 This is a schematic diagram of the LSTM structure.

[0043] Fig.10 It is a schematic diagram of the improved CBAM structure.

[0044] Fig.11 This is a schematic diagram of the model structure of CNN-CBAM-BiGRU.

[0045] Fig.12 It is a schematic diagram of the loss curve of the CNN-CBAM-BiGRU model.

[0046] Fig.13 It is a schematic diagram of the accuracy curve of the CNN-CBAM-BiGRU model. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Combination Figure 1-Figure 13 The present invention proposes a rice disease recognition method based on a CNN-CBAM-BiGRU joint model, the method comprising the following steps:

[0049] Step 1: Collect data and construct a data set based on the collected data;

[0050] This experiment uses RGB images of four diseases (rice blast, bacterial leaf blight, sesame spot, and sheath blight). To ensure fairness in model training, healthy leaf image data is introduced to balance positive and negative samples. The open source dataset is used as the main training data for the model: the public dataset collected from the Kaggle website includes six types of rice images, including rice blast, bacterial leaf blight, sesame spot, sheath blight, and healthy leaves, including 634 images of rice blast, 635 images of bacterial leaf blight, 638 images of sesame spot, 632 images of sheath blight, and 648 images of healthy leaves. The images are saved in JPG format. Rice leaf disease images are as follows: Figure 1 The main characteristics of the disease are shown in Table 1.

[0051] Table 1 Characteristics of rice leaf disease images

[0052]

[0053] The images currently collected have problems such as inconsistent pixels, unreasonable distribution of the number of categories, and insufficient overall number. Inconsistent image pixels will increase the computational complexity of the model during training, resulting in longer training time; the large difference in the number of images of each category will bias the model training results, affecting the judgment of recognition accuracy; insufficient overall number will lead to insufficient model training, and there will not be enough information to find key disease features to fully fit the data.

[0054] To address the above issues, we first use image batch processing to calibrate the pixels of all images to 224×224, reducing the computational pressure of the model input layer and making it more compatible with the convolution kernel operations in the network; we normalize all images, that is, compress the pixel values ​​in the image to between [0,1] according to the overall distribution, thereby increasing the convergence speed of the network. Then, we use data augmentation methods to expand the data set, and by fine-tuning the pixel positions and pixel values ​​of the images, the model can learn more feature information. For example, operations such as rotation, translation, and shearing show the possibility of disease features appearing in other locations; operations such as adjusting contrast and brightness can enhance the difference between spots of similar colors; and operations such as adding salt and pepper noise and Gaussian noise will improve the model's ability to resist interference. The effect of applying data augmentation methods to sesame spot disease images is shown below. Figure 2 As shown in Table 2, the number of rice disease images was adjusted according to the research requirements to establish a rice disease dataset. At this time, the dataset has a total of 6374 images, and the training set and test set are divided into a ratio of 8:2. At this time, the number of images in the dataset changes as shown in Table 2.

[0055] Table 2 Number and division of datasets

[0056]

[0057] Step 2: Construct a CNN-CBAM-BiGRU joint model, use CNN to extract features from input data, and extract rich multi-dimensional features from the image; then, input the features extracted by CNN into CBAM, which is used to enhance meaningful features and suppress unimportant features. The CBAM refines the features from two dimensions, channel and space, to improve the expressiveness of the features; finally, input the features processed by CBAM into BiGRU, which is used to capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition;

[0058] Convolutional Neural Network (CNN) is a feedforward neural network with convolution operations. It is one of the representative algorithms of deep learning and has achieved success in the field of computer vision. Figure 3 As shown in the figure, it consists of five parts: input layer, convolution layer, pooling layer, fully connected layer and output layer: the probability is output using a logical function or other classification function, and the label with the highest probability is the prediction result. CNN is a classic and widely used structure. The local connection, weight sharing and pooling operation of the convolutional neural network can effectively reduce the complexity of the network and the number of training parameters, making the model invariant to translation, distortion and scaling to a certain extent, and having strong robustness and fault tolerance, and is also easy to train and optimize.

[0059] The most direct way to improve the performance of CNN is to increase the width and depth of the network, but this will also lead to a decrease in the performance of CNN and bring a greater amount of computation. The Inception module assembles multiple convolution and pooling operations into a network. When designing a network model, the network model is assembled in modules, which can not only increase the performance of the network, but also ensure computational efficiency. In order not to lose disease features and accurately extract disease features, the present invention improves CNN based on the Inception v1 module to increase its depth. First, two 3×3 convolutions are used to replace the 5×5 convolution, which reduces the amount of computation of the model, increases the receptive field, and is more conducive to detecting larger lesions. Secondly, a 3×3 convolution and a 1×1 convolution are used to replace a branch. The 3×3 convolution is used to extract smaller lesion information to avoid losing some key information. The convolution kernel is 1×1 for upsampling and downsampling, which can increase or decrease the number of channels without changing the size of the feature map. Finally, the output of the first part is combined with the output of the second part to obtain a richer feature combination. Improved CNN structure such as Figure 4 shown.

[0060] Recurrent Neural Network (RNN) Figure 5 As shown in the figure, it is usually widely used to process time series data, making it more adaptable in time series data analysis. In actual operation, the data needs to be converted into a specific form, namely (number of samples, time step, number of eigenvalues). During the processing of each time step in the RNN network, each feature in the sample will be multiplied by the corresponding weight, and then the bias parameter will be added. The weighted and bias-adjusted result will be handed over to the activation function for further processing. The activation function will perform nonlinear transformation on the input value in a specific mathematical way according to the input value, thereby introducing nonlinear characteristics into the model and enhancing the model's expressive power. This processing process is repeated continuously in each time step, allowing the RNN to gradually learn and capture the dynamic patterns and long-term dependencies in the time series data, providing effective solutions for various time series-related tasks, such as speech recognition, text generation in natural language processing, and sentiment analysis.

[0061] The Gated Recurrent Unit (GRU) is an improvement on LSTM. Similar to LSTM, it is proposed to solve the problems of gradient explosion, gradient disappearance and long-distance dependency in long-term memory and back propagation. Compared with LSTM, GRU has fewer parameters and simpler structure, but its performance is comparable to that of LSTM. In GRU, only two gates are used, namely the reset gate and the update gate. The reset gate determines how to combine new input information with previous memory, which helps to capture short-term dependencies in time series; the update gate defines the amount of previous memory saved to the current time step, which helps to capture long-term dependencies in time series. Its internal structure is as follows: Figure 6 As shown in the figure, x t is the input, u t represents the update gate, r t represents the reset gate, h t-1 is the hidden state at time t-1, h t is the output of the hidden layer, h t ' is the candidate hidden state, and ρ represents the activation function. The GRU gated update is publicized as:

[0062] r t =ρ(ω xr x t +ω hr h t-1 +b r )

[0063] u t =ρ(ω xu x t +ω hu h t-1 +b u )

[0064] h′ t =tanh(ω xh x t +(r t ·h t-1 )+b h )

[0065] h t =(1-u t )h t-1 +r t ·h′ t

[0066] where ω xr ,ω hr ,ω xu ,ω hu ,ω xh ,ω hh is the corresponding r t 、ut 、h t ', "·" represents the Hadamard product, b r , b u , b h is the corresponding r t 、u t 、h t ''s bias parameter. GRU uses a reset gate and an update gate to achieve selective forgetting of the original hidden information and selective memory of the current node information. The gate signal interval is [0,1]. The candidate hidden state uses the reset gate to control how much information in the past is forgotten, and the update gate controls how much new information the current state needs to receive from the historical state. This design can deal with the gradient decay problem in recurrent neural networks and better capture dependencies with larger time steps in time series.

[0067] The Bidirectional Gated Recurrent Unit (BiGRU) consists of a forward GRU and a backward GRU. The forward GRU processes from the start position to the end position of the sequence to capture information from the past to the present; the backward GRU processes from the end position to the start position of the sequence to capture information from the future to the present. Finally, the forward hidden state and the backward hidden state of the current time step are concatenated to obtain the output of the BiGRU at this time step. This output contains information from the past and the future and can more comprehensively represent the features of the current time step. Through this bidirectional processing method, the BiGRU can better utilize the contextual information of the sequence. Its structure is as follows: Figure 7 As shown. The output is determined by two GRUs at the same time, and the specific formula is expressed as:

[0068]

[0069] in They are forward propagation and backward propagation respectively; and are the weights of forward propagation and backward propagation respectively. In this way, the data feature information is superimposed through the forward and backward propagation BiGRU, so that the most important features are retained to the maximum extent.

[0070] The CBAM as a whole has two parts, CAM (Channel Attention Module) and SAM (Spatial Attention Module), which sequentially enhance the important information of the two features of channel and space; the input feature map F first enters the CAM, and is respectively compressed from different spatial directions by average pooling and maximum pooling to highlight the channel features, and is simultaneously passed to the shared network to extract the channel attention weight. The shared network contains two multi-layer perceptrons MLP and one hidden layer, which can be expressed by the formula:

[0071]

[0072] Among them, M c (F) is the weight of feature map F in channel C, and The channel features are obtained after the average pooling and maximum pooling operations, and σ represents the Sigmoid activation function. In SAM, the results of the CAM output are also aggregated by two pooling operations to obtain the channel information of the feature map to obtain the average and maximum features of the entire channel. The information of different spaces is combined through the splicing operation and convolution to generate the spatial attention weight. The generation process can be expressed as:

[0073]

[0074] Among them, f 7×7 represents a convolution operation with a convolution kernel size of 7×7. and The spatial features are obtained after average pooling and maximum pooling operations. The overall structure of CBAM is as follows Figure 8 shown.

[0075] First, global maximum pooling and global average pooling operations are performed on the feature map in the channel attention module (CAM) to obtain feature descriptions at different levels for the same feature map, and then the two processing results are transmitted to the improved multi-layer perceptron (MLP) layer respectively. Since the original MLP layer consists of only two simple fully connected layers, this structure limits its further extraction of the weights after pooling. In order to make the entire module obtain more accurate information, one-dimensional convolution is first used to weaken the impact of dimensionality reduction on the data, and then the second fully connected layer in the MLP is replaced with a long short-term memory (Convolutional long short-term memory, LSTM) network based on the recurrent neural network (RNN) structure, and time series are added to the channel to more accurately locate the disease characteristics. Fig. 9As shown in the figure, LSTM introduces a three-gate design in the traditional RNN. The forget gate F(t) filters the memory information that is valuable to the time unit and the subsequent time unit in the output features of the previous moment, the input gate I(t) filters the important feature information at the current moment t, and the output gate O(t) extracts the information that has the greatest impact on the output of the current unit. The results are transmitted between gates by weight product. The data after the pooling operation will first be sent to the fully connected layer to strengthen the correlation between the nodes, and then the three-gate structure of LSTM will filter the irrelevant feature information at the time level, strengthen the spatial relationship of the output features, and improve the efficiency of the subsequent SAM spatial feature extraction.

[0076] It should also be noted that by observing the CBAM structure, it can be found that there is a strong correlation between the CAM and SAM modules. The input information of the SAM module comes directly from the CAM module. The improved CAM module has undergone pooling operations and strengthened spatial feature dependencies, which is quite different from the initial input data. At this time, the optimized features of CBAM may cause performance degradation in the deep layer of the model, affecting the calculation of subsequent convolutional layers. According to the basic principles of the attention mechanism, a weighted operation is added to the CBAM as a whole to associate the original input data, reduce the impact of continuous complex operations between modules on the data, and correct the output results of CBAM, so as to obtain more noteworthy disease features to improve the model's ability to recognize rice diseases. The improved CBAM is as follows Fig.10 shown.

[0077] The CNN-CBAM-BiGRU model is as follows Fig.11 As shown in the figure. First, CNN is used to extract features from the input data. CNN is good at capturing local spatial features and can extract rich multi-dimensional features from images. Then, the features extracted by CNN are input into the CBAM module. CBAM can further enhance meaningful features and suppress unimportant features. It refines features from two dimensions, channel and space, to improve the expressiveness of features. Finally, the features processed by CBAM are input into BiGRU. BiGRU can capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition.

[0078] Step 3: Use the constructed dataset to train the CNN-CBAM-BiGRU joint model;

[0079] The experiments were conducted in Windows 11 operating system (64-bit), the CPU was 12th Gen Intel(R) Core(TM) i7-12700H 2.70GHz, the GPU was NVIDIA GeForce RTX3060 Laptop 6GB, the memory RAM was 32GB, all codes were written in Python, the neural network was built on keras2.5.1, and the deep learning platform was Tensorflow2.4.0. The model parameters were jointly tested, and the model parameters are shown in Table 3.

[0080] Table 3 Model parameters

[0081]

[0082] The present invention uses accuracy, precision, recall and F1 value (F1) to measure the performance of each model.

[0083] Accuracy refers to the ratio of correctly classified samples to the total number of samples. It intuitively reflects the overall classification accuracy of the model, but it can be misleading in the case of imbalanced samples. For example, when the number of samples in one class is much larger than that in other classes, the accuracy may be high even if the model always predicts the majority class, but in fact the classification effect on the minority class is very poor.

[0084]

[0085] Precision is also called precision, which refers to the proportion of true positive samples among all samples predicted as positive. It mainly measures the accuracy of the model's prediction of positive samples. High precision means that the model rarely mistakenly predicts negative samples as positive.

[0086]

[0087] Recall rate, also known as recall rate, refers to the proportion of samples that are correctly predicted as positive among all samples that are actually positive. It reflects the ability of the model to find all true positive samples. A high recall rate means that the model can find as many true positive samples as possible, but may mistakenly predict some negative samples as positive.

[0088]

[0089] The F1 value is the harmonic mean of precision and recall, which takes into account the accuracy and comprehensiveness of the model. When a balance between precision and recall is needed, the F1 value is a good indicator. It can avoid the one-sidedness caused by focusing only on precision or recall.

[0090]

[0091] Among them, TP (True Positive) is the true positive, which indicates the number of images correctly identified as this type of rice disease. TN (True Negative) is the true negative, which indicates the number of images correctly identified as not having this type of rice disease. FP (False Positive) false positive and FN (False Negative) false negative are the complement of the former two.

[0092] Step 4: Train the model module number and parameters, input the actual data into the trained model, and thus complete the identification of rice diseases.

[0093] Experimental results and analysis

[0094] The model is trained in two stages. The first stage is to find the optimal number of modules for the improved CNN and improved CBAM. The improved CNN and improved CBAM modules are superimposed for testing, and the improved CNN-CBAM modules are gradually superimposed. The improved CNN model is used independently as a control test, and the results of each test are recorded. The experimental results are shown in Table 4.

[0095] Table 4 Experimental results comparison table

[0096]

[0097] According to the results in Table 4, it is not difficult to see that the recognition accuracy of the model increases with the increase of the depth of the improved CNN-CBAM model. However, at the beginning, due to the insufficient network depth, the feature extraction ability of the CNN module is weak. As the model depth increases, the influence of the CBAM module on the accuracy gradually increases. When the improved CNN-CBAM module is stacked to the fourth one, the recognition accuracy of the model will decrease due to overfitting caused by excessive model training. Therefore, according to the current data, the improved CNN-CBAM module can be stacked 3 times to obtain the best performance.

[0098] The second stage is to find the optimal parameters and optimal information collection position of BiGRU based on the first stage. If the initial data set is directly transferred to BiGRU, it will increase a lot of computational costs. First, use the CNN module to extract important information from the data and remove some irrelevant information. Secondly, introduce the BiGRU module for independent testing, and convert the output of the improved CNN-CBAM module number from 1 to 4 into a one-dimensional vector form, and then input the converted value into the BiGRU module with different parameters for experiment. Finally, splice the output data and record the experimental results. The specific results are shown in Table 5.

[0099] Table 5 Experimental results comparison table

[0100]

[0101] According to the results in Table 5, it is not difficult to see that compared with the improved CNN-CBAM module stacked alone, when the BiGRU network is 2 layers and 80 nodes, the accuracy of the model reaches the highest, which is 4.48% higher than the improved CNN-CBAM model. Compared with increasing the scale by stacking models, the cost of BiGRU is relatively small, and significant improvements can still be achieved. From the comparison of the results in Table 5, it is obvious that the more layers and nodes there are, the higher the accuracy of the model will be, but when the number of layers and nodes reaches a certain number, the improvement of the model recognition performance will be less and less. Combining Tables 4 and 5, the influence of BiGRU on the model in the early stage due to the lack of feature extraction of CNN caused the accuracy of the model to fluctuate. With the continuous increase of CNN-CBAM modules, when the number of modules reaches 3, the model reaches the best, with the highest accuracy of 98.03%. As the number of modules increases, overfitting will lead to the performance of BiGRU to decrease. Taking all factors into consideration, the optimal parameter situation is 3 improved CNN-BiGRU modules, and the BiGRU network is 2 layers and 80 nodes.

[0102] Finally, the CNN-CBAM-BiGRU joint model is trained. The model converges quickly and has low loss. The accuracy curve and loss curve are as follows: Figure 12-13 As shown in the figure. The CNN-CBAM-BiGRU joint model is based on the Inception v1 module to improve CNN. The improved CNN module is then combined with the improved CBAM attention mechanism. The combined output is used to input the BiGRU model, which can effectively capture feature information. In order to reduce the limitations of CNN, the output features after passing through the CNN model are spliced ​​with the output features of the BiGRU module and transmitted to the classification layer for disease recognition. The recognition accuracy of the CNN-CBAM-BiGRU model after feature splicing reaches 98.03%.

[0103] The ablation experiment results of the three models of the combination of CNN and CBAM attention mechanism are shown in Table 6. The multilayer perceptron in the original CBAM attention extracts information by reducing the dimension, which will cause some information to be omitted when training the spot disease. The present invention improves this part and compares it with the original method. The experimental results are shown in Table 6.

[0104] Table 6 Improved CBAM ablation experiment

[0105]

[0106] The data shows that the CBAM attention mechanism helps the accuracy of disease recognition. Compared with the single improved CNN model, the accuracy of the original CBAM model increased by 2.45%, and the accuracy of the improved CBAM model increased by 3.29%. This is because the improved CBAM implemented by using one-dimensional convolution weakens the impact of dimensionality reduction on the data. This method effectively enhances the impact of CBAM on CNN, thereby improving performance.

[0107] Since CNN focuses on local feature information and lacks attention to global features and dependencies between features, the BiGRU module is introduced to extract this information and analyze the impact of BiGRU on the experiment. The present invention uses several classic CNN models and the present invention model to conduct ablation experiments, while maintaining the consistency of the number of nodes. The experimental results are shown in Table 7.

[0108] Table 7 CNN-CBAM-BiGRU ablation experiment

[0109]

[0110] As can be seen from Table 7, compared with the first 6 data, the improved CNN and CBAM have higher recognition accuracy than the other 5 models. This is because the improved CNN and CBAM models improve the feature extraction ability and can obtain richer feature combinations, which ultimately makes the CNN-CBAM-BiGRU model have the highest accuracy. VGG16 has a high fitting ability, but its parameter volume is too large and the parameter adjustment is difficult, which is 1.5% lower than the model of the present invention. LeNet has the lowest recognition accuracy due to its small input size, which is 3.72% lower than the model of the present invention. AlexNet has a recognition accuracy of 97.12% due to its excellent feature extraction ability, but it is 1.91% lower than the model of the present invention due to its prone to overfitting. GoogLeNet V1 increases the expression ability of the network by using the Inception module, but as the number of network layers increases, it will also make model optimization more difficult, and the recognition accuracy is 1.82% lower than the model of the present invention. RestNet50 avoids the gradient vanishing problem due to its residual structure, and the large amount of redundant data it generates will affect the experimental results, which is 2.8% lower than the model of the present invention. Compared with Table 6, it can be seen that BiGRU effectively improves the model recognition accuracy by extracting the dependency between features.

[0111] The following is an analysis of the number of BiGRU nodes: two parameters x and y are set to represent the magnification coefficients of CNN and BiGRU, and the effect of the number of nodes on the model is explored by adjusting the size of the magnification coefficient. The results are shown in Table 8. When x is 1, the accuracy of the model has stabilized. Considering the training cost, x is set to 1 and the effect of the number of nodes on BiGRU is analyzed. The recognition accuracy of the model increases with the increase in the number of nodes, but the continuous increase in the number of nodes will inevitably lead to longer model training time and larger model calculations. When x is 1 and y is 1.5, the highest accuracy is 98.66%. However, as the number of nodes increases, the accuracy of the model decreases. The balanced model cost is kept at x and y, that is, the number of nodes is 80.

[0112] Table 8 Node analysis

[0113]

[0114] Usually, in order to prevent overfitting, the larger the input data, the better the recognition performance of the model, but a larger input will inevitably lead to a larger amount of calculation. The following explores the impact of input size on the model by adjusting the size of the input image. The results are shown in Table 9.

[0115] Table 9 Input size analysis

[0116]

[0117] The data shows that in the experiment based on this dataset, the accuracy is proportional to the input size. However, as the input size increases, it takes longer to train, and the increase in size does not significantly improve the model accuracy. Taking all factors into consideration, an input size of 224×224 should be selected in this experiment.

[0118] The present invention also proposes a rice disease recognition system based on a CNN-CBAM-BiGRU joint model, the system comprising:

[0119] Collection module: collects data and constructs a data set based on the collected data;

[0120] Building modules: Building a CNN-CBAM-BiGRU joint model, using CNN to extract features from input data, and being able to extract rich multi-dimensional features from images; then, inputting the features extracted by CNN into CBAM, which is used to enhance meaningful features and suppress unimportant features, and the CBAM refines the features from two dimensions, channel and space, to improve the expressiveness of the features; finally, inputting the features processed by CBAM into BiGRU, which is used to capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition;

[0121] Training module: Use the constructed dataset to train the CNN-CBAM-BiGRU joint model;

[0122] Identification module: Input the actual data into the trained model to complete the identification of rice diseases.

[0123] The present invention extracts rice disease features by improving CNN and CBAM and combining BiGRU. The improved CNN extracts richer features by increasing the network depth, and the improved CBAM locates the disease features more accurately by adding one-dimensional convolution and replacing a fully connected layer, while the BiGRU model superimposes image feature information so that more feature combinations are retained. According to the experimental results, more feature combinations are obtained by stacking the improved CNN-CBAM modules, and finally the recognition effect after 3 modules is determined to be the best according to the experimental results, with a recognition accuracy of 93.55%. The features extracted by the improved CNN-CBAM module are input into the BiGRU module, and the understanding of the disease structure is deepened by analyzing the relationship between the features. Finally, the recognition accuracy of the CNN-CBAM-BIGRU model is 98.03%. After comparison experiments with classic network models such as VGG16, LeNet, and AlexNet, the recognition accuracy of the model of the present invention is still the highest, proving that the improved CNN and CBAM modules of the present invention extract richer features. Finally, the present invention verifies the parameters of the BiGRU module through experiments, and analyzes the impact of the number of nodes on the model performance by adjusting the amplification factor. Although the more nodes, the better, too many nodes will bring extremely high computational complexity, and after the number of nodes reaches a certain number, the help to the model performance will become smaller and smaller. Therefore, the present invention determines the number of nodes to be 80, and then compares the impact of different input sizes on the model. Obviously, the larger the size, the better, but the training time will also be longer. Considering lower cost and higher recognition accuracy, the present invention sets the input size to 224×224.

[0124] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the rice disease identification method based on the CNN-CBAM-BiGRU joint model are implemented.

[0125] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of the rice disease identification method based on the CNN-CBAM-BiGRU joint model.

[0126] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0127] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0128] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0129] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0130] The above is a detailed introduction to the rice disease recognition method based on the CNN-CBAM-BiGRU joint model proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A rice disease recognition method based on CNN-CBAM-BiGRU joint model, characterized in that: The method comprises the following steps: Step 1: Collect data and construct a data set based on the collected data; Step 2: Construct a CNN-CBAM-BiGRU joint model, use CNN to extract features from input data, and extract rich multi-dimensional features from the image; then, input the features extracted by CNN into CBAM, which is used to enhance meaningful features and suppress unimportant features. The CBAM refines the features from two dimensions, channel and space, to improve the expressiveness of the features; finally, input the features processed by CBAM into BiGRU, which is used to capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition; Step 3: Use the constructed dataset to train the CNN-CBAM-BiGRU joint model; Step 4: Input the actual data into the trained model to complete the identification of rice diseases.

2. The method according to claim 1, characterized in that The CNN is improved based on the Inception v1 module by increasing the depth of the network. First, two 3×3 convolutions are used to replace the 5×5 convolution, which reduces the computational complexity of the model and increases the receptive field, making it more conducive to detecting larger lesions. Second, a 3×3 convolution and a 1×1 convolution are used to replace a branch. The 3×3 convolution is used to extract information about smaller lesions to avoid losing some key information. The convolution kernel of 1×1 is used for upsampling and downsampling, which can increase or decrease the number of channels without changing the size of the feature map. Finally, the output of the first part is combined with the output of the second part to obtain a richer feature combination.

3. The method according to claim 1, characterized in that: The bidirectional gated recurrent unit BiGRU consists of a forward GRU and a backward GRU; the forward GRU processes from the start position of the sequence to the end position, capturing information from the past to the present; The backward GRU processes from the end position of the sequence to the starting position, capturing information from the future to the present; finally, the forward hidden state and the backward hidden state of the current time step are concatenated to obtain the output of the BiGRU at that time step.

4. The method according to claim 3, characterized in that The output is determined by two GRUs at the same time, and the specific formula is expressed as: in They are forward propagation and backward propagation respectively; and are the weights for forward propagation and backward propagation, respectively.

5. The method according to claim 1, characterized in that The CBAM as a whole has two parts, CAM and SAM, which sequentially enhance the important information in the channel and spatial features; the input feature map F first enters the CAM, and then compresses the spatial information from different spatial directions through average pooling and maximum pooling to highlight the channel features, and then enters the shared network to extract the channel attention weight. The shared network contains two multi-layer perceptrons MLP and one hidden layer, which can be expressed by the formula: Among them, M c (F) is the weight of feature map F in channel C, and The channel features are obtained after the average pooling and maximum pooling operations, and σ represents the Sigmoid activation function. In SAM, the results of the CAM output are also aggregated by two pooling operations to obtain the channel information of the feature map to obtain the average and maximum features of the entire channel. The information of different spaces is combined through the splicing operation and convolution to generate the spatial attention weight. The generation process can be expressed as: Among them, f 7×7 represents a convolution operation with a convolution kernel size of 7×7. and The spatial features are obtained after average pooling and maximum pooling operations.

6. The method according to claim 5, characterized in that The improved structure of the channel attention module CAM is as follows: first use one-dimensional convolution to weaken the impact of dimensionality reduction on the data, then replace the second fully connected layer in the MLP with a long short-term memory (LSTM) network based on the recurrent neural network (RNN) structure, and add a time series to the channel to more accurately locate the disease characteristics.

7. The method according to claim 6, characterized in that LSTM introduces a three-gate design in RNN: the forget gate F(t) filters the memory information in the output features of the previous moment that is valuable to the time unit and subsequent time units, the input gate I(t) filters the important feature information at the current moment t, and the output gate O(t) extracts the information that has the greatest impact on the output of the current unit. The results are transmitted between gates by weight product.

8. A rice disease recognition system based on CNN-CBAM-BiGRU joint model, characterized in that: The system comprises: Collection module: collects data and constructs a data set based on the collected data; Building modules: Building a CNN-CBAM-BiGRU joint model, using CNN to extract features from input data, and being able to extract rich multi-dimensional features from images; then, inputting the features extracted by CNN into CBAM, which is used to enhance meaningful features and suppress unimportant features, and the CBAM refines the features from two dimensions, channel and space, to improve the expressiveness of the features; finally, inputting the features processed by CBAM into BiGRU, which is used to capture long-term dependencies in sequence data, thereby improving the accuracy of disease recognition; Training module: Use the constructed dataset to train the CNN-CBAM-BiGRU joint model; Identification module: Input the actual data into the trained model to complete the identification of rice diseases.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images