A pest identification method and system based on multi-scale attention learning network

Abstract

The present invention provides a pest recognition method based on a multi-scale attention learning network, including obtaining an image containing pests; obtaining an image recognition result using a multi-scale attention learning network model according to the obtained image containing pests; wherein, after obtaining the image containing pests, the image containing pests is divided into a training set and a test set; and the multi-scale attention learning network model is trained by a decoupling learning strategy. The agricultural pest recognition algorithm of the present invention can simultaneously solve the problems caused by the complex and changeable background of pest recognition, the high similarity between pests, and the unbalanced data distribution; the TLM in the MS-ALN can effectively locate the target of the pest and cut it out, and the ADM and ARM can further encourage the network to learn the fine-grained features that distinguish the pest categories. The combination of the three modules can effectively improve the accuracy of pest classification.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a pest identification method and system based on a multi-scale attention learning network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Pest recognition plays an important role in pest control and agricultural management. Accurate classification of agricultural pests is crucial for pest control. Since some pests are very similar, some can only be accurately identified by professionals, which greatly reduces the efficiency of pest control and increases the difficulty of agricultural management. In recent years, with the development of computer vision, it has become possible to let computers classify pests instead of humans.

[0004] However, the classification of agricultural pests still faces some adjustments. First, the small size of pests and the complex environment increase the difficulty of pest identification, and the complex background information often affects the extraction of pest target features; second, different types of pests have highly similar morphologies, and traditional recognition algorithms are difficult to effectively extract the features of pest images and accurately classify them; finally, due to the geographical distribution, growth environment, and quantity characteristics of the pests themselves, some pest images are easier to obtain and some are more difficult. Therefore, the distribution of pest images has the characteristics of inter-class imbalance. If no special treatment is given, the recognition accuracy of the category with fewer images will be very poor. The above three problems have brought great challenges to the research on the identification of agricultural pests. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a pest identification method and system based on a multi-scale attention learning network, which can reduce the interference factors of the pest identification task and improve the overall stability and generalization ability of the model.

[0006] According to some embodiments, the present invention adopts the following technical solutions:

[0007] A pest recognition method based on a multi-scale attention learning network, comprising:

[0008] Acquire an image containing pests;

[0009] According to the acquired images containing pests, a multi-scale attention learning network model is used to obtain image recognition results;

[0010] After obtaining the image containing the pests, the image containing the pests is divided into a training set and a test set; and the multi-scale attention learning network model is trained by a decoupled learning strategy.

[0011] Furthermore, the training set is used to train the parameters of the multi-scale attention learning network model; and the test set is used to test the multi-scale attention learning network model.

[0012] Furthermore, dividing the images containing pests into a training set and a test set also includes expanding and standardizing the training set and the test set.

[0013] Furthermore, the multi-scale attention learning network model includes a target positioning module, an attention detection module and an attention deletion module.

[0014] Furthermore, the target positioning module is used to extract and aggregate feature values ​​of input image data to obtain a pest target map.

[0015] Furthermore, the attention detection module is used to extract and aggregate feature values ​​of the pest target graph to obtain a pest component graph.

[0016] Furthermore, the attention deletion module deletes the pest component map to obtain an attention deletion map.

[0017] A pest identification system based on a multi-scale attention learning network, comprising:

[0018] An image acquisition module is configured to acquire an image containing pests;

[0019] An image recognition module is configured to obtain an image recognition result using a multi-scale attention learning network model according to an acquired image containing pests;

[0020] After obtaining the image containing the pests, the image containing the pests is divided into a training set and a test set; and the multi-scale attention learning network model is trained by a decoupled learning strategy.

[0021] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device, a pest identification method based on a multi-scale attention learning network.

[0022] A terminal device includes a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded and executed by the processor for a pest identification method based on a multi-scale attention learning network.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. The agricultural pest recognition algorithm of the present invention can simultaneously solve the problems caused by the complex and changeable background of pest recognition, the high similarity between pests, and the unbalanced data distribution;

[0025] 2. The TLM in MS-ALN can effectively locate the target of pests and cut them out. The ADM and ARM can further encourage the network to learn fine-grained features to distinguish pest categories. The combination of the three modules can effectively improve the accuracy of pest classification;

[0026] 3. The feature extraction network and classifier parameters of multiple scale images in the present invention are shared, which can effectively reduce the number of parameters, reduce the memory occupied by the model, and facilitate industrial promotion;

[0027] 4. The multi-scale attention learning network constructed by the present invention was experimented on the large-scale pest dataset IP102 and achieved the highest accuracy of 74.61%. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings in the specification, which constitute a part of the present application, are used to provide further understanding of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0029] Figure 1 is a flow chart of the agricultural pest identification method in Example 1 of the present invention;

[0030] Figure 2 is a structural diagram of a multi-scale attention learning network in Example 1 of the present invention;

[0031] Figure 3 is a flow chart of the target positioning module in Example 1 of the present invention;

[0032] Figure 4 is a flow chart of the attention detection module in Embodiment 1 of the present invention;

[0033] Figure 5 It is a flow chart of the attention deletion module in embodiment 1 of the present invention. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0035] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0036] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0037] Example 1

[0038] like Figure 1 As shown, a pest recognition method based on a multi-scale attention learning network includes: obtaining an image containing pests;

[0039] According to the acquired images containing pests, a multi-scale attention learning network model is used to obtain image recognition results;

[0040] After obtaining the image containing the pests, the image containing the pests is divided into a training set and a test set; and the multi-scale attention learning network model is trained by a decoupled learning strategy.

[0041] Specifically,

[0042] like Figure 1 As shown, including:

[0043] S1. Obtain pictures of agricultural pests and classify the images according to the pest species. Divide the collected pictures into a training set and a test set in a ratio of 4:1. The training set is used to train the model parameters, and the test set is used to test the model accuracy.

[0044] S2, expand the data images in the training set after sampling;

[0045] Data augmentation was performed by random flipping, random mirroring, and random contrast and brightness changes. Specifically, the images were randomly flipped from -30° to 30°, each image was horizontally mirrored with a probability of 0.5, and the image contrast and brightness were randomly changed, with a floating ratio of 0.2. Finally, all images in the training set were resized to 448×448, and all image data were standardized.

[0046] S3. Construct a multi-scale attention learning network (MS-ALN), train the network model based on the expanded data using a decoupled learning strategy, and save the optimal parameters of the network;

[0047] The multi-scale attention learning network consists of three modules, namely TLM, ADM and ARM, and the feature extraction network uses ResNet50. When the image is input, first, the pest image passes through TLM to crop the main target of the image and filter part of the background of the image; then the image passes through ADM to detect the recognizable area of ​​the object and further amplify the small differences in the area. Finally, the network uses ARM to randomly erase a part of the recognizable area of ​​the image, thereby encouraging the network model to learn multiple high-response areas and further improve the overall stability of the network.

[0048] Figure 2 Network structure for agricultural pest identification based on multi-scale attention learning network;

[0049] In the target localization module (TLM), the original image (X r ) is passed into ResNet50 for feature extraction, and the output of the last layer of the network is extracted as the feature map At this time, the size of the feature map is C×H×W, where C is the number of channels, H is the height of the feature map, and W is the width of the feature map. The feature map is then summed along the channel dimension to obtain the response map R, and the size of the response map is 1×H×W. The global average pooling operation (Global Average Pooling, GAP) is then used to calculate the discrimination threshold θ for determining the target area of ​​the pest. The response data in the original response map R is further compared with the threshold to divide the high response area, that is, the target area of ​​the pest. Afterwards, a rectangular cropping frame is determined to cover all high response points with the smallest area. The area divided by the cropping frame is the area where the pest object is located. In order to further improve the positioning accuracy of the module and reduce the influence of accidental factors, this embodiment also extracts the output feature map of the penultimate layer in the feature extraction network. Used to calculate the second rectangular cropping frame. Finally, the areas covered by the two rectangles are intersected to obtain the final pest target area, and the area is cropped out to obtain the pest target map ( o ), and the size of the target image is enlarged to 448 × 448. This image not only retains most of the pest area, but also filters out part of the background information and reduces environmental interference factors.

[0050] Figure 3 is a flow chart of a target positioning module in an embodiment of the present invention;

[0051] In the attention detection module (ADM), the pest target map is first calculated according to the same steps as in the TLM to obtain the target response map R oThen, 11 window sizes for sliding window operations are pre-set, and the window sizes are set to (4,4), (3,5), (5,3), (6,6), (5,7), (7,5), (8,8), (6,10), (10,6), (7,9), (9,7), and 11 sliding window operations are constructed accordingly. The 13 sliding window operations are applied to the target response graph R in turn. o , and further obtain 11 groups of window response values. The size of this value represents the amount of information contained in the corresponding window, that is, the larger the value, the more the network attention is focused on the area. In order to solve the problem of regional information redundancy in different window areas, this embodiment reuses the non-maximum suppression operation (Non-Maximum Suppression, NMS) to calculate 4 output windows with high window response values ​​and small overlapping areas between windows, where the IOU parameter in the NMS operation is set to 0.25. Finally, the four windows are cropped out and uniformly enlarged to 224×224 size to obtain the pest parts map In this group of pictures, each picture represents the attention area of ​​a network. By magnifying the attention area and the tiny differences within it, the network's ability to represent the characteristics of different pests can be effectively enhanced.

[0052] Figure 4 is a flow chart of an attention detection module in an embodiment of the present invention;

[0053] In ARM, a pest component image obtained in ADM is randomly selected, and a deletion operation is performed on the corresponding position of the pest target image according to the attention image to obtain the attention deletion image (X d ). By deleting some of the areas where the network focuses its attention, the network can be encouraged to learn more response areas and focus on more parts of the pests, which improves the generalization ability and ability to cope with noise of the model to a certain extent, and is conducive to improving the overall stability of the network.

[0054] Figure 5 is a flow chart of an attention deletion module in an embodiment of the present invention;

[0055] During the training process, the original pest image X is input r , through TLM, ADM and ARM, we can get the pest target map X o , Pest Parts Diagram Attention Removal Graph X d The above four images are sequentially passed into the feature extraction network ResNet50 and the classifier to obtain four groups of pest prediction probabilities With P(X d), the four groups of predicted probabilities are calculated in turn with the true labels of the pests to obtain the cross entropy loss, and the global parameters of the network are optimized based on the loss. Considering that there are multiple component graphs, the loss value may be large, so the weighted loss of the component graph is used in the calculation of the final loss, and the weight is set to 0.5.

[0056] In the decoupled learning process, the sample average sampling strategy is first used to train the global parameters of the network, and train the network's feature extraction and pest classification capabilities. This sampling strategy has the same sampling probability for each sample, that is, sampling follows the distribution of the data set itself. In this way, sampling will not distort the distribution of the data, nor will it distort the learning of the features; then the parameters of the feature extraction network are frozen, and the class average sampling strategy is used to train the parameters of the classifier separately. This sampling method makes the probability of sampling from each class the same, that is, rebalancing the distribution of the data. This sampling is conducive to the adjustment of the classifier boundary and can effectively alleviate the problems caused by unbalanced data distribution.

[0057] S4. For the test data, first build a network structure consistent with the training model, load the network parameters trained in step 3 into the actual application model, and then identify the newly received agricultural pest images. First, pass the original image to the feature extraction network to extract features and the TLM module to locate the specific location of the pests, and obtain the pest target map X. o , and then X o The feature codes are passed into the same feature extraction network, and finally the feature codes are passed into the classifier to obtain the final pest number.

[0058] Ablation experiment

[0059] In order to demonstrate the effectiveness of the three modules constructed in the present invention and the decoupled learning strategy adopted, we conducted an ablation experiment, and the experimental data used the IP102 dataset.

[0060] IP102: It contains more than 75,000 images of 102 common agricultural pests, including 45,095 training images, 7,508 verification images, and 22,619 test images. Compared with other pest datasets, the IP102 dataset can better reflect the main problems of pest recognition tasks. There are three main challenges in this dataset. First, the background of the images in the dataset is complex and changeable, and pests have different morphological characteristics in different life cycles, with strong intra-class differences. Second, some pests have very similar characteristics, and the inter-class differences are small. Third, the data type distribution of pests is similar to the actual distribution. There is a large amount of common pest data and a small amount of uncommon pest data, so it has the characteristics of uneven distribution between classes. Due to these factors, classification on the IP102 dataset is more challenging. This dataset is one of the benchmark datasets for pest recognition tasks.

[0061] Analysis of ablation experiment results:

[0062] First, when only ResNet50 is used as the recognition network, the classification accuracy is only 66.08%. When TLM is introduced, the accuracy is improved by 3.05%. It can be seen that this module can effectively reduce the interference factors of the picture background by cutting out the pest target; on this basis, we introduce ADM to detect high-response areas and let the network better learn the fine-grained features of pests. The network test accuracy is 72.43%, and the test accuracy has increased by 3.30%. On this basis, when ARM is added, the test accuracy is improved by 1.13%. It can be seen that the ARM module can further improve the recognition accuracy of pests by improving the overall stability and generalization ability of the network. Finally, the decoupled learning strategy is applied to the training process of the network model, and the final test accuracy reaches 74.61%. This fully proves that when the learning of the classifier is separated from the learning of the feature extraction network, it can effectively alleviate the problems caused by the long-tail distribution of the data and further improve the test accuracy of the network model. The above experiments show that the network constructed in this paper and the training strategy used have made great contributions to the final accuracy, and the improvements made are effective.

[0063] Table 1 Ablation experiment results

[0064] Method Accuracy(%) Only ResNet50 66.08％ ResNet50+TLM 69.13％ ResNet50+TLM+ADM 72.43％ ResNet50+TLM+ADM+ARM 73.56％ ResNet50+TLM+ADM+ARM+DL 74.61％

[0065] The calculation method of this recognition accuracy is: in the test set, the number of pictures correctly recognized by the program is divided by the total number of pictures in the test set. The formula is:

[0066]

[0067] Where n is the total number of test set images. If the i-th image is successfully recognized, then m i =1, otherwise m i =0.

[0068] Example 2

[0069] A pest identification system based on a multi-scale attention learning network, comprising:

[0070] An image acquisition module is configured to acquire an image containing pests;

[0071] An image recognition module is configured to obtain an image recognition result using a multi-scale attention learning network model according to an acquired image containing pests;

[0072] After obtaining the image containing the pests, the image containing the pests is divided into a training set and a test set; and the multi-scale attention learning network model is trained by a decoupled learning strategy.

[0073] Example 3

[0074] A computer-readable storage medium storing a plurality of instructions, wherein the instructions are suitable for being loaded and executed by a processor of a terminal device. A pest identification method based on a multi-scale attention learning network provided in this embodiment 1.

[0075] Example 4

[0076] A terminal device includes a processor and a computer-readable storage medium, the processor is used to implement various instructions; the computer-readable storage medium is used to store multiple instructions, and the instructions are suitable for being loaded and executed by the processor. A pest identification method based on a multi-scale attention learning network provided in this embodiment 1.

[0077] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0078] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0079] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0080] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0081] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0082] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A pest identification method based on a multi-scale attention learning network, characterized in that: include: Acquire an image containing pests; According to the acquired images containing pests, a multi-scale attention learning network model is used to obtain image recognition results; After obtaining the image containing the pests, the image containing the pests is divided into a training set and a test set; the multi-scale attention learning network model is trained by a decoupled learning strategy; The multi-scale attention learning network model includes a target positioning module, an attention detection module and an attention deletion module; The target positioning module is used to extract and aggregate feature values ​​of input image data to obtain a pest target map; The attention detection module is used to extract and aggregate the feature values ​​of the pest target image, and uniformly amplify it to obtain a pest component image; The attention deletion module deletes the pest component map to obtain an attention deletion map; In the decoupled learning process, the sample averaging strategy is first used to train the global parameters of the network, and then the parameters of the feature extraction network are frozen, and the class average sampling strategy is used to train the parameters of the classifier separately.

2. A pest identification method based on a multi-scale attention learning network as claimed in claim 1, characterized in that: The training set is used to train the parameters of the multi-scale attention learning network model; the test set is used to test the multi-scale attention learning network model.

3. A pest identification method based on a multi-scale attention learning network as claimed in claim 2, characterized in that: The method of dividing the images containing pests into a training set and a test set also includes expanding and standardizing the training set and the test set.

4. A pest identification system based on a multi-scale attention learning network, characterized in that: include: An image acquisition module is configured to acquire an image containing pests; An image recognition module is configured to obtain an image recognition result using a multi-scale attention learning network model according to an acquired image containing pests; After obtaining the image containing the pests, the image containing the pests is divided into a training set and a test set; the multi-scale attention learning network model is trained by a decoupled learning strategy; The multi-scale attention learning network model includes a target positioning module, an attention detection module and an attention deletion module; The target positioning module is used to extract and aggregate feature values ​​of input image data to obtain a pest target map; The attention detection module is used to extract and aggregate the feature values ​​of the pest target image, and uniformly amplify it to obtain a pest component image; The attention deletion module deletes the pest component map to obtain an attention deletion map; In the decoupled learning process, the sample averaging strategy is first used to train the global parameters of the network, and then the parameters of the feature extraction network are frozen, and the class average sampling strategy is used to train the parameters of the classifier separately.

5. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executed by a pest identification method based on a multi-scale attention learning network according to any one of claims 1 to 3.

6. A terminal device, characterized in that: The invention comprises a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; and the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executed by a pest identification method based on a multi-scale attention learning network as described in any one of claims 1 to 3.

Images