Disease and pest identification method based on improved residual network of multi-head self-attention mechanism

Abstract

The invention discloses a multi-head self-attention mechanism-based disease and insect pest identification method for an improved residual network. The method comprises the following steps of: acquiring data; preprocessing the data; a multi-head self-attention mechanism is added behind the residual network, and an improved residual network model is formed; the trained model is evaluated; the trained weight file is transferred into the raspberry Pi, and the raspberry Pi is connected with a camera, so that the plant diseases and insect pests of the trees can be detected in real time. According to the method, a multi-head self-attention mechanism is added on the basis of a ResNet18 network structure, projection matrixes of different queries, keys and values are used each time to generate different attention sub-spaces, and finally outputs of all the sub-spaces are spliced together to form final output, so that the weight of key feature information is enhanced, and the recognition precision of the model is further improved. The method is used for preventing and treating forestry diseases and insect pests, improves forestry income and has wide research and market application prospects.

Description

Technical Field

[0001] The present invention belongs to the technical field of smart agriculture, and specifically relates to a method for identifying pests and diseases based on an improved residual network with a multi-head self-attention mechanism. Background Art

[0002] As an important part of my country's economy, forestry not only provides abundant timber resources for people's daily life, but also provides a basic guarantee for the protection of my country's ecological environment. Forestry pests and diseases are also called smokeless forest fires, and the economic losses caused by them are as high as tens of billions of yuan each year. Therefore, accurate identification of forestry pests and diseases and timely treatment of them play an important role in the protection of forest resources, timber production and the stability of the ecological environment.

[0003] There are many types of forestry pests and diseases. The traditional method of relying on forestry experts to observe and identify the types of forestry-related pests and diseases is slow, subjective, and difficult to obtain. It is impossible to quickly and accurately identify the types of pests and diseases and give corresponding countermeasures, which has become a considerable obstacle to the development of agriculture and forestry. The use of manual sorting of pests and diseases mainly uses the naked eye to observe the characteristics and screen them according to the different characteristics. This not only greatly increases the workload of manual work, but also has relatively low efficiency. With the development of modern science and technology, we have a new vision and new solutions for how to quickly identify pests and diseases. For example, the use of machine learning technology combined with computer vision technology to identify pests and diseases can not only quickly and accurately analyze and process a large amount of data, but also further improve the level of identification and prevention of forestry pests and diseases, and provide strong support for scientific research.

[0004] Carrying out research on the identification of forest pests and diseases can prevent and control forest pests and diseases from the root, protect the animal and plant environment, and maintain biodiversity. It is of great practical significance to improve the utilization efficiency and protection level of forest resources and promote the sustainable development of forestry. Therefore, it is very important to explore an intelligent diagnosis method for the identification of forest pests and diseases. Summary of the invention

[0005] In order to solve a series of problems such as low accuracy and insufficient objectivity of traditional forestry pest and disease image recognition algorithms, the present invention proposes a pest and disease recognition method based on an improved residual network with a multi-head self-attention mechanism. The recognition method is to add a multi-head self-attention mechanism to the traditional residual network (ResNet18), and use different query, key and value projection matrices to generate different attention subspaces each time, and finally splice the outputs of all subspaces together to form the final output, which can not only enhance the weight of key feature information, but also further improve the recognition accuracy of the model.

[0006] In order to achieve the above-mentioned invention object, the present invention adopts the following technical solutions: The present invention provides a method for identifying pests and diseases based on an improved residual network with a multi-head self-attention mechanism, which comprises the following steps: (1) Collect images of common pests and diseases to obtain a data set; (2) preprocessing the pest image in step (1); (3) adding a multi-head self-attention mechanism after the convolutional layer of the residual network, constructing an improved residual network based on the multi-head self-attention mechanism, and forming an improved residual network model to train the pest and disease images in step (2); (4) Evaluating the improved residual network model trained in step (3) and calculating various indicators of the model; (5) The trained data is stored in a weight file and then transferred to the Raspberry Pi. The Raspberry Pi is connected to a camera to achieve real-time detection of pests and diseases.

[0007] Furthermore, the pests and diseases include the larvae and adult forms of the scale insect, the hemp stink bug, the Japanese ridge borer, the starry longhorn beetle, the mulberry longhorn beetle, the pine sawtooth beetle, the willow blue leaf beetle, the yellow thorn moth, the brown-edged green moth, the frost moth, the poplar fan moth, the poplar small moth, the American white moth, the human striped moth, and the ribbon swallowtail butterfly.

[0008] Furthermore, the preprocessing includes image format adjustment and data enhancement.

[0009] Furthermore, the format adjustment specifically adjusts all images to jpg format, with pixels uniformly set to 96 96dpi.

[0010] Furthermore, 90% of the images in the data set are randomly selected as the training data set, and the remaining images are used as the test data set.

[0011] Furthermore, the data enhancement is to increase the training data set, specifically, to rotate or flip the images that are relatively concentrated in the pre-training data, and to increase random noise, so as to achieve the purpose of increasing the training data set.

[0012] Furthermore, in step (3), during training, a 50% inactivation rate is selected, the optimizer selects the Adam algorithm to optimize the loss function, the batch size is set to 16, the initial learning rate is 0.0005, and the number of iterations is 400.

[0013] Furthermore, in the step (3), the improved residual network model is a Self-Attiention+ResNet18 model, which is a multi-head self-attention mechanism added after the convolutional layer of the ResNet18 network to construct an improved residual network based on the multi-head self-attention mechanism; and then the network is formed by initializing the training parameters of the ResNet18 network with the ImageNet data set.

[0014] Furthermore, in step (3), during training, the hyperparameter epoch is pre-set to indicate the number of training times, specifically: (1) Use pre-trained weights to initialize the Self-Attention+ResNet18 model network; (2) Use the Self-Attention+ResNet18 model network for feature extraction; (3) Use the Softmax function on the output layer to classify and predict the input image; (4) Judge the output results according to the loss function and calculate the loss value; (5) Calculate the network gradient based on the loss value and use the Adam algorithm to dynamically modify the weight; (6) The model is modified by optimization algorithm; (7) Repeat steps (2) to (6), continuously modifying the network weights until all epoch iterations are completed and an effective recognition algorithm is finally obtained.

[0015] Furthermore, the indicators in step (4) include accuracy, precision, recall and F1 value.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: 1. High accuracy: The data set of the present invention comes from a real source and has high authenticity and reliability.

[0017] 2. Model structure improvement: The present invention adds a self-attention module to the classic network ResNet18, and on this basis improves the original model parameters to finally form a self-Attention+ResNet18 network structure, which can not only effectively increase the weight of effective information and reduce the weight of irrelevant information, but also further improve the accuracy of the entire model in identifying pests and diseases.

[0018] 3. Strong practicability: The present invention can identify forest pests and diseases in real time, which is helpful to improve my country's ecological security. It has low cost and high efficiency, and is easy to promote and apply. In addition, the method described in the present invention can also be combined with technologies such as the Internet of Things and cloud computing to establish a real-time monitoring and early warning system, and can be seamlessly integrated with intelligent devices such as robots and drones to automatically realize monitoring and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Figure 2 is a diagram of data preprocessing operations.

[0020] Figure 2 This is the training flow chart.

[0021] Figure 3 Block diagram of the improved residual network based on multi-head self-attention mechanism. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is further described in detail with reference to the following specific examples.

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention. Example 1

[0024] 1. Data Collection The data set used in the present invention is all from the China Software Cup, and there are a total of 1400 basic images of data, and the data set labels are 15 kinds of common pests, namely, paramecium scale, hemp stink bug, Japanese ridge borax, star longhorn beetle, mulberry longhorn beetle, pine monochaete, willow blue leaf beetle, yellow thorn moth, brown edge green thorn moth, frost moth, Yang fan boat moth, Yang Xiaozhou moth, American white moth, human pattern dirty lamp moth and ribbon swallowtail butterfly, including their larvae and adult forms, a total of 30 kinds of classification. These classifications are marked with corresponding labels, such as label 0 represents paramecium scale larvae, label 1 represents paramecium scale adults, label 2 represents hemp stink bug larvae, label 3 represents hemp stink bug adults, and so on, a total of 30 labels are marked. The data set is expanded to 4200 data through data enhancement, and the data set is divided according to the ratio of 9: 1, wherein 420 are used as test sets, and the rest are used as training sets.

[0025] 2. Data preprocessing In view of the differences in image format and size, all images in the dataset were adjusted to jpg format with a uniform pixel size of 96. 96 dpi. Because the number of parameters in the deep residual network is large, there is a risk of overfitting. When the number of samples is small, it will cause insufficient training samples, which will affect the accuracy of the network. Therefore, it is necessary to increase the data to train the samples. In order to improve the robustness of the network model and reduce the risk of overfitting, the method of data enhancement is used to further increase the training data set. The data preprocessing operation process is as follows Figure 1 As shown in the figure, 90% of the images in the dataset are randomly selected as the training set, and the remaining 10% of the images are used as the test set. Then, the relatively concentrated images in the pre-training data are rotated or flipped, and random noise is added to further expand the dataset to 4200 data.

[0026] 3. Model training During training, the hyperparameter epoch is pre-set to indicate the number of training times. The detailed steps are as follows: Figure 2 As shown, specifically: Use pre-trained weights to initialize the Self-Attention+ResNet18 model network; The maximum number of iterations is 400; Batch size is set to 16; The learning rate is 0.0005; The optimizer selects the Adam (Adaptive Moment Estimation) algorithm.

[0027] (2) Use the network to extract features; (3) Use the Softmax function on the output layer to classify and predict the input image; (4) Judge the output results according to the loss function and calculate the loss value; (5) Calculate the network gradient based on the loss value and use the Adam algorithm to dynamically modify the weight; (6) The model is modified by optimization algorithm; (7) Repeat steps (2) to (6) above, and continuously modify the network weights until all epoch iterations are completed, and finally an effective recognition algorithm is obtained.

[0028] The present invention adds a multi-head self-attention mechanism after the convolutional layer of the classic ResNet18 network to construct an improved residual network based on the multi-head self-attention mechanism, such as Figure 3As shown in the figure, the network built by initializing the training parameters of the ResNet18 network in the ImageNet dataset is formed to form a Self-Attiention+ResNet18 model.

[0029] This experiment was built on a Window 10 system. The processor was an Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz, the main graphics card was an Intel(R) UHD Graphics Family, the editor was Pycharm, the Python language was used, and the version was 3.11.5. The network was built using the Pytorch library, version 2.0.0.

[0030] 4. Model Evaluation For classification and recognition problems, the commonly used evaluation criteria are accuracy, precision, recall and F1-score. In the present invention, since the main consideration is the accuracy of forest pest recognition, the evaluation criteria used are the accuracy and F1-score that are commonly used and generally recognized in the field of computer vision.

[0031] Accuracy: It is an indicator used to measure the difference between the classifier's prediction result and the actual result. The closer it is to 1, the more accurate the classification result is. The calculation formula is as follows:

[0032] In the formula, Indicates that the predicted category is a positive sample, and the actual category is also the number of positive samples; Indicates the number of predicted categories that are positive samples and the actual categories that are negative samples; Indicates that the predicted category is a negative sample and the actual category is the number of negative samples; Indicates the number of samples whose predicted categories are negative samples and whose actual categories are positive samples.

[0033] F1 value: It is an indicator used in statistics to measure the accuracy of a binary classification model. It takes into account both the precision and recall of the classification model. The F1 value can be regarded as a harmonic average of the model's precision and recall. The calculation formula of F1 value is:

[0034] In the formula, It represents the accuracy rate, and its calculation formula is: ; Recall is the rate of recall, which is calculated as: .

[0035] In the above formulas for precision and recall, generally speaking, the higher the F1 value, the better the classification effect of the model.

[0036] 5. System deployment Store the trained data in a weight file, convert the weight file into a file format supported by the onnx Raspberry Pi Linux system, save the file in the Raspberry Pi, and connect the camera to the Raspberry Pi to perform real-time detection of tree pests and diseases.

[0037] After experimental verification, the present invention can effectively identify forestry disease images, and according to the identified results, the spraying and management of pesticides can be effectively carried out. In addition, the present invention can also identify images of various diseases by expanding the data set, and use different disease characteristics as the main difference. The present invention can also combine the Internet of Things, cloud computing and other technologies to establish a real-time monitoring and early warning system, and seamlessly integrate with robots, drones and other intelligent devices to automatically realize monitoring and processing.

[0038] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for a person skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to replace some of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions claimed to be protected by the present invention.

Claims

1. A pest and disease identification method based on an improved residual network with a multi-head self-attention mechanism, characterized in that: The following steps are involved: (1) Collect images of common pests and diseases to obtain a dataset; (2) preprocessing the pest image in step (1); (3) adding a multi-head self-attention mechanism after the convolutional layer of the residual network, constructing an improved residual network based on the multi-head self-attention mechanism, and forming an improved residual network model to train the pest and disease images in step (2); (4) Evaluating the improved residual network model trained in step (3) and calculating various indicators of the model; (5) The trained data is stored in a weight file and then transferred to the Raspberry Pi. The Raspberry Pi is connected to a camera to achieve real-time detection of pests and diseases.

2. The pest identification method according to claim 1, characterized in that: The pests and diseases include the larvae and adult forms of the scale insect, the hemp stink bug, the Japanese ridge borer, the starry longhorn beetle, the mulberry longhorn beetle, the pine sawtooth beetle, the willow blue leaf beetle, the yellow thorn moth, the brown-edged green moth, the frost moth, the poplar fan moth, the poplar small moth, the American white moth, the human striped moth, and the ribbon swallowtail butterfly.

3. The pest identification method according to claim 1, characterized in that: The preprocessing includes image format adjustment and data enhancement.

4. The pest identification method according to claim 3, characterized in that: The format adjustment specifically adjusts all images to jpg format, with a uniform pixel size of 96 96 dpi.

5. The pest identification method according to claim 3, characterized in that: 90% of the images in the dataset are randomly selected as the training dataset, and the remaining images are used as the testing dataset.

6. The pest identification method according to claim 5, characterized in that: The data enhancement is to increase the training data set, specifically, to rotate or flip the images that are relatively concentrated in the pre-training data, increase random noise, and achieve the purpose of increasing the training data set.

7. The pest identification method according to claim 1, characterized in that: In step (3), during training, a 50% inactivation rate is selected, the optimizer selects the Adam algorithm to optimize the loss function, the batch size is set to 16, and the initial learning rate is 0.0005.

8. The method for identifying pests and diseases according to claim 1, characterized in that: In the step (3), the improved residual network model is a Self-Attiention+ResNet18 model, which is a model in which a multi-head self-attention mechanism is added after the convolutional layer of the ResNet18 network to construct an improved residual network based on the multi-head self-attention mechanism; Then, the network is formed by initializing the training parameters of the ResNet18 network using the ImageNet dataset.

9. The pest identification method according to claim 8, characterized in that: In step (3), the hyperparameter epoch is pre-set during training to indicate the number of training times, specifically: (1) Use pre-trained weights to initialize the Self-Attention+ResNet18 model network; (2) Use the Self-Attention+ResNet18 model network for feature extraction; (3) Use the Softmax function on the output layer to classify and predict the input image; (4) Judge the output results according to the loss function and calculate the loss value; (5) Calculate the network gradient based on the loss value and use the Adam algorithm to dynamically modify the weight; (6) The model is modified by optimization algorithm; (7) Repeat steps (2) to (6), continuously modifying the network weights until all epoch iterations are completed and an effective recognition algorithm is finally obtained.

10. The pest identification method according to claim 1, characterized in that: The indicators in step (4) include accuracy, precision, recall and F1 value.

Images