Corn pest area detection method combining context awareness and multi-scale mixed attention

Abstract

The present invention relates to a corn pest area detection method combining context perception and multi-scale hybrid attention, comprising: establishing a training sample set; constructing a corn pest area detection model: constructing a basic pest area detection model, using a multi-scale hybrid attention module and a context perception module to improve a feature aggregation network in the basic pest area detection model, and obtaining a corn pest area detection model; training the corn pest area detection model; pest area detection: inputting a sample to be detected into the corn pest area detection model, generating positioning information of the pest area in the sample to be detected, and counting the number of pest areas. The present invention introduces multi-scale context features into a small target pest area through a context perception module to strengthen the representation capability of the target feature, enhance the network's ability to extract such related features, and solve the problem that the prior art cannot accurately identify the pest area due to its small scale.

Description

Technical Field

[0001] The present invention relates to the technical field of pest area detection, and in particular to a corn pest area detection method combining context perception and multi-scale mixed attention. Background Art

[0002] Corn is one of the most planted crops in the world and an important source of food in the world. It is estimated that by 2020, about one-third of the world's farms will grow corn. Ensuring its yield is of great significance to world food security. Corn yield is affected by various factors, among which the impact of pests is quite prominent. Usually pests will first eat the middle leaves and then damage the leaves during the growth of corn. After being eaten, the crop yield is severely reduced and the stems are easily broken when encountering wind. Detection of pest areas and evaluation of their development trends are important prerequisites for pest control, which provides a theoretical basis for the formulation of scientific prevention and control strategies. Therefore, corn pest area detection plays a vital role in corn yield.

[0003] At present, there are two main methods for detecting corn pest areas: manual survey and computer vision and image processing technology. These methods have the following limitations: the former is inefficient, highly subjective and prone to errors; although the latter has achieved good results in some agricultural tasks, it is not effective for detecting corn pest areas. The reason is that the size of the pest area accounts for a very small proportion of the image, and the complex environmental changes in natural fields cause complex backgrounds in the collected data. These factors lead to poor detection results of traditional machine learning methods in such tasks.

[0004] Therefore, solving the technical task of accurately detecting small target corn pest areas in complex environments has become a top priority. Summary of the invention

[0005] The purpose of the present invention is to provide a corn pest area detection method that combines context perception and multi-scale mixed attention, which can improve the performance of pest area detection, reduce the missed detection rate, is suitable for pest area detection in different growth cycles of corn, and has good robustness and generalization ability.

[0006] To achieve the above object, the present invention adopts the following technical solution: a method for detecting corn pest areas combining context perception and multi-scale hybrid attention, the method comprising the following steps in order:

[0007] (1) Establishing a training sample set: Collecting real field corn images and preprocessing them, marking the effective areas in the images, and establishing a training sample set;

[0008] (2) Constructing a corn pest area detection model: Constructing a basic pest area detection model YOLOv4, using a multi-scale hybrid attention module and a context-aware module to improve the feature aggregation network in the basic pest area detection model YOLOv4, and obtaining a corn pest area detection model;

[0009] (3) Training a corn pest area detection model: Input the training samples in the training sample set into the corn pest area detection model, and the corn pest area detection model outputs feature maps of different scales. The feature maps of different scales are decoded by the head module of the corn pest area detection model, and the predicted pest area coordinates and classification results are output, and the loss is calculated and the parameters are updated;

[0010] (4) Pest area detection: The sample to be detected is input into the corn pest area detection model to generate the location information of the pest area in the sample to be detected and count the number of pest areas.

[0011] The step (2) specifically comprises the following steps:

[0012] (2a) Constructing the basic pest area detection model YOLOv4: Constructing the backbone network, feature aggregation network, and head module to form the basic pest area detection model YOLOv4;

[0013] (2b) Construct a multi-scale hybrid attention module: Construct a multi-scale hybrid attention module by combining the position attention module and the multi-scale channel attention module;

[0014] (2c) constructing a context-aware module;

[0015] (2d) The multi-scale hybrid attention module and context-aware module are used to improve the feature aggregation network of the basic pest area detection model YOLOv4 to obtain the corn pest area detection model.

[0016] The step (3) specifically comprises the following steps:

[0017] (3a) The training samples are input into the corn pest area detection model, and the convolution operation in the backbone network outputs feature maps of different scales;

[0018] (3b) The feature maps of different scales output by the backbone network are input into the improved feature aggregation network, the multi-scale hybrid attention module extracts effective features, and the target feature representation is enhanced by the context-aware module to obtain an output feature map containing target positioning information and classification information;

[0019] (3c) decoding the output feature map obtained in step (3b) and outputting the prediction result, i.e., the coordinates of the pest area and the classification result, and calculating the loss;

[0020] (3d) Back propagation, updating relevant parameters in the corn pest area detection model.

[0021] The step (2b) specifically comprises the following steps:

[0022] (2b1) Construct a position attention module to calculate the spatial dependency between any two positions in the feature map. Given an intermediate feature map X∈R C×H×W As input, the feature map size is H×W, the channel dimension is C, and the specific calculation formula is as follows:

[0023]

[0024] From this we can deduce as follows:

[0025] Q(X)=σ(I(X) T M(X))N(X)

[0026] =σ((W I X) T W M X)N(X)

[0027] =σ(X T W I T W M X)N(X)

[0028] Among them, (i, j) represents the coordinates of the current position and other positions whose dependencies are to be calculated, q is the output signal, Q is the position attention mechanism, (k, j) is the coordinate of any point in the feature map, σ represents the softmax function, I(x i，j )=W I x i，j ,M(x k，l )=W M x k，l ,N(x k，l )=W N xk ，l ,I(x i，j ), M(x k，l ), N(x k，l ) are the outputs of the corresponding convolutional layers; W I , W M , W N are the parameters of the corresponding convolutional layers;

[0029] (2b2) ​​Construct a multi-scale channel attention module to extract features along the channel dimension at global and local scales through global average pooling operations and 1x1 convolution operations. The specific calculation formula is as follows:

[0030] Global scale:

[0031] G(X′)=BN(C 2 ((LR(BN(C 1 (g(X′)))))))

[0032] Local scale:

[0033] L(X′)=BN(C 2 (LR(BN(C 1 (X′)))))

[0034] Multi-scale channel attention:

[0035]

[0036] Among them, X′ is the output of the position attention module, CA is the multi-scale channel attention, G(X′)∈R C×1×1 is the global scale, L(X′)∈R C×H×W is the local scale, C 1 With C 2 They represent the convolution kernel parameters respectively. and The convolution operation, BN represents the Batch Normalization operation, LR represents the Leaky Relu activation function operation, and g represents the global average pooling operation. It is the element addition operation of broadcast mechanism;

[0037] (2b3) Combine the position attention module with the multi-scale channel attention module to construct a multi-scale hybrid attention module. The specific calculation formula is as follows:

[0038]

[0039]

[0040] Among them, X″ is the feature map of output optimization; It is the element-wise multiplication operation.

[0041] The step (2c) specifically comprises the following steps:

[0042] (2c1) extracting shallow feature maps and deep feature maps output by the backbone network;

[0043] (2c2) Changing the shape of shallow and deep feature maps and target features through upsampling and downsampling operations Figure 1 To;

[0044] (2c3) First, the shallow and deep feature maps are optimized through a multi-scale mixed attention module and the two are fused. Then, the features are refined through convolution operations and fused with the target feature map to enhance the feature information of the pest area.

[0045] The loss calculation in step (3c) consists of three parts: confidence loss, positioning loss, and classification loss. The calculation formula is as follows:

[0046] Loss=Loss (CIoU )+Loss (conf) +Loss (cls)

[0047]

[0048]

[0049]

[0050]

[0051]

[0052] Among them, Loss is the total loss, Loss (CIoU) 、Loss (conf) 、Loss (cls) They are positioning loss, confidence loss and classification loss respectively, d represents the Euclidean distance between the center points of the two bounding boxes, c represents the diagonal distance of the closure, IoU represents the intersection over union of the predicted bounding box and the true bounding box, and w gt With h gt The width and height of the real border are respectively, w and h are the width and height of the predicted bounding box respectively. The corn pest area detection model divides the input image into S×S grids, each grid has B aiming boxes, represents that there is a real object in the jth aiming box in the i-th grid, which is equal to 1, and On the contrary, it means that there is no real object in the j-th aiming box in the i-th grid, which is equal to 0, λ noobj is the weight coefficient.

[0053] It can be seen from the above technical scheme that the beneficial effects of the present invention are: first, the present invention introduces multi-scale contextual features for small target pest areas through a context perception module to enhance the representation ability of target features, enhance the network's ability to extract such related features, and solve the problem that the prior art cannot accurately identify the pest area due to its small scale; second, a multi-scale hybrid attention mechanism is used to solve the complex background problem in the real field environment, so that the network can learn the importance of different features and focus on effective features; third, by combining the basic pest area detection model YOLOv4 with the above two modules, the pest area detection performance is improved, the missed detection rate is reduced, and it is suitable for pest area detection in different growth cycles of corn, and has good robustness and generalization ability, overcoming the defects of the prior art in detecting corn pest areas under complex backgrounds. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a flow chart of the method of the present invention;

[0055] Figure 2 Schematic diagram of the structure of the multi-scale hybrid attention module of the present invention;

[0056] Figure 3 is a structural diagram of a context awareness module of the present invention;

[0057] Figure 4 This is a schematic diagram of the pest area detection obtained by the present invention. DETAILED DESCRIPTION

[0058] like Figure 1 As shown, a method for detecting corn pest areas combining context perception and multi-scale mixed attention includes the following steps in order:

[0059] (1) Establishing a training sample set: Collecting real field corn images and preprocessing them, marking the effective areas in the images, and establishing a training sample set;

[0060] (2) Constructing a corn pest area detection model: Constructing a basic pest area detection model YOLOv4, using a multi-scale hybrid attention module and a context-aware module to improve the feature aggregation network in the basic pest area detection model YOLOv4, and obtaining a corn pest area detection model;

[0061] (3) Training a corn pest area detection model: Input the training samples in the training sample set into the corn pest area detection model, and the corn pest area detection model outputs feature maps of different scales. The feature maps of different scales are decoded by the head module of the corn pest area detection model, and the predicted pest area coordinates and classification results are output, and the loss is calculated and the parameters are updated;

[0062] (4) Pest area detection: The sample to be detected is input into the corn pest area detection model to generate the location information of the pest area in the sample to be detected and count the number of pest areas.

[0063] The step (2) specifically comprises the following steps:

[0064] (2a) Constructing the basic pest area detection model YOLOv4: Constructing the backbone network, feature aggregation network, and head module to form the basic pest area detection model YOLOv4;

[0065] (2b) Construct a multi-scale hybrid attention module: Construct a multi-scale hybrid attention module by combining the position attention module and the multi-scale channel attention module;

[0066] (2c) constructing a context-aware module;

[0067] (2d) The multi-scale hybrid attention module and context-aware module are used to improve the feature aggregation network of the basic pest area detection model YOLOv4 to obtain the corn pest area detection model.

[0068] The step (3) specifically comprises the following steps:

[0069] (3a) The training samples are input into the corn pest area detection model, and the convolution operation in the backbone network outputs feature maps of different scales;

[0070] (3b) The feature maps of different scales output by the backbone network are input into the improved feature aggregation network, the multi-scale hybrid attention module extracts effective features, and the target feature representation is enhanced by the context-aware module to obtain an output feature map containing target positioning information and classification information;

[0071] (3c) decoding the output feature map obtained in step (3b) and outputting the prediction result, i.e., the coordinates of the pest area and the classification result, and calculating the loss;

[0072] (3d) Back propagation, updating relevant parameters in the corn pest area detection model.

[0073] The step (2b) specifically comprises the following steps:

[0074] (2b1) Construct a position attention module to calculate the spatial dependency between any two positions in the feature map. Given an intermediate feature map X∈R C×H×W As input, the feature map size is H×W, the channel dimension is C, and the specific calculation formula is as follows:

[0075]

[0076] From this we can deduce as follows:

[0077] Q(X)=σ(I(X) T M(X))N(X)

[0078] =σ((W I X) T W M X)N(X)

[0079] =σ(X T W I T W M X)N(X)

[0080] Among them, (i, j) represents the coordinates of the current position and other positions whose dependencies are to be calculated, q is the output signal, Q is the position attention mechanism, (k, j) is the coordinate of any point in the feature map, σ represents the softmax function, I(x i，j )=W I x i，j ,M(x k，l )=W M x k，l ,N(x k，l )=W N x k，l ,I(x i，j ), M(x k，l ), N(x k，l ) are the outputs of the corresponding convolutional layers; W I , W M , W N are the parameters of the corresponding convolutional layers;

[0081] (2b2) ​​Construct a multi-scale channel attention module to extract features along the channel dimension at global and local scales through global average pooling operations and 1x1 convolution operations. The specific calculation formula is as follows:

[0082] Global scale:

[0083] G(X′)=BN(C 2 ((LR(BN(C 1 (g(X′)))))))

[0084] Local scale:

[0085] L(X′)=BN(C 2 (LR(BN(C 1 (X′)))))

[0086] Multi-scale channel attention:

[0087]

[0088] Among them, X′ is the output of the position attention module, CA is the multi-scale channel attention, G(X′)∈R C×1×1 is the global scale, L(X′)∈R C×H×W is the local scale, C 1 With C 2 They represent the convolution kernel parameters respectively. and The convolution operation, BN represents the Batch Normalization operation, LR represents the Leaky Relu activation function operation, and g represents the global average pooling operation. It is the element addition operation of broadcast mechanism;

[0089] (2b3) Combine the position attention module with the multi-scale channel attention module to construct a multi-scale hybrid attention module. The specific calculation formula is as follows:

[0090]

[0091]

[0092] Among them, X″ is the feature map of output optimization; It is the element-wise multiplication operation.

[0093] The step (2c) specifically comprises the following steps:

[0094] (2c1) extracting shallow feature maps and deep feature maps output by the backbone network;

[0095] (2c2) Changing the shape of shallow and deep feature maps and target features through upsampling and downsampling operations Figure 1 To;

[0096] (2c3) First, the shallow and deep feature maps are optimized through a multi-scale mixed attention module and the two are fused. Then, the features are refined through convolution operations and fused with the target feature map to enhance the feature information of the pest area.

[0097] The loss calculation in step (3c) consists of three parts: confidence loss, positioning loss, and classification loss. The calculation formula is as follows:

[0098] Loss=Loss (CIoU) +Loss (conf) +Loss (cls)

[0099]

[0100]

[0101]

[0102]

[0103]

[0104] Among them, Loss is the total loss, Loss (CIoU) 、Loss (conf) 、Loss (cls) They are positioning loss, confidence loss and classification loss respectively, d represents the Euclidean distance between the center points of the two bounding boxes, c represents the diagonal distance of the closure, IoU represents the intersection over union of the predicted bounding box and the true bounding box, and w gt With hgt The width and height of the real border are respectively, w and h are the width and height of the predicted bounding box respectively. The corn pest area detection model divides the input image into S×S grids, each grid has B aiming boxes, represents that there is a real object in the jth aiming box in the i-th grid, which is equal to 1, and On the contrary, it means that there is no real object in the j-th aiming box in the i-th grid, which is equal to 0, λ noobj is the weight coefficient.

[0105] like Figure 2 As shown in the figure, the multi-scale hybrid attention module is composed of the position attention module and the multi-scale channel attention module. The position attention module is mainly composed of convolution operations and matrix operations. First, the input feature map is compressed by convolution operations, and the spatial dependency matrix is ​​obtained by combining matrix operations. Then, matrix multiplication is performed with the compressed input feature map to obtain the selection weight, and finally it is added to the input feature map. The multi-scale channel attention module is mainly composed of convolution operations and is divided into two branches, the global branch and the local branch. First, the global branch obtains a one-dimensional channel feature vector through a global average pooling operation, and then the local branch uses a convolution operation with a convolution kernel of 1x1. The calculation results of the two branches are combined through an addition operation with a broadcast mechanism, and finally combined with the input feature map to output the optimized feature map.

[0106] like Figure 3 As shown in the figure, the input of the context-aware module is the feature maps of different scales output by the backbone network of the pest area detection model. First, the shallow feature map and the deep feature map are scaled to the same size as the target feature map through upsampling and downsampling operations respectively. In order to prevent the information in the feature maps of different scales from overwhelming the information in the target feature map, a multi-scale mixed attention module is used for optimization. Then, the feature maps of different scales are fused, and the information contained in them is refined using convolution operations. Finally, the fused multi-scale feature map is fused with the target feature map to supplement the pest area information in the target feature map.

[0107] from Figure 4 It can be seen that the present invention can accurately locate and identify the corn pest area, and can also accurately locate the pest area with a small size and complex background in the image. It can also be seen that the present invention can accurately count the detected pest areas, which can be used to determine the degree of damage in different areas and evaluate their future development trends.

[0108] In summary, the present invention introduces multi-scale contextual features for small target pest areas through a context-aware module to enhance the representation capability of target features, and enhances the network's ability to extract such related features, thereby solving the problem that the prior art cannot accurately identify pest areas due to their small scale. The present invention uses a multi-scale hybrid attention mechanism to solve complex background problems in real field environments, so that the network can learn the importance of different features and focus on effective features.

Claims

1. A corn pest area detection method combining context perception and multi-scale mixed attention, characterized in that: The method comprises the following steps in order: (1) Establishing a training sample set: Collecting real field corn images and preprocessing them, marking the effective areas in the images, and establishing a training sample set; (2) Constructing a corn pest area detection model: Constructing a basic pest area detection model YOLOv4, using a multi-scale hybrid attention module and a context-aware module to improve the feature aggregation network in the basic pest area detection model YOLOv4, and obtaining a corn pest area detection model; (3) Training a corn pest area detection model: Input the training samples in the training sample set into the corn pest area detection model, and the corn pest area detection model outputs feature maps of different scales. The feature maps of different scales are decoded by the head module of the corn pest area detection model, and the predicted pest area coordinates and classification results are output, and the loss is calculated and the parameters are updated; (4) Pest area detection: The sample to be detected is input into the corn pest area detection model to generate the location information of the pest area in the sample to be detected, and the number of pest areas is counted; The step (2) specifically comprises the following steps: (2a) Constructing the basic pest area detection model YOLOv4: Constructing the backbone network, feature aggregation network, and head module to form the basic pest area detection model YOLOv4; (2b) Construct a multi-scale hybrid attention module: Construct a multi-scale hybrid attention module by combining the position attention module and the multi-scale channel attention module; (2c) constructing a context-aware module; (2d) A multi-scale hybrid attention module and a context-aware module are used to improve the feature aggregation network of the basic pest area detection model YOLOv4 to obtain a corn pest area detection model; The loss calculation consists of three parts: confidence loss, positioning loss, and classification loss. The calculation formula is as follows: Loss=Loss (CIoU) +Loss (conf) +Loss (cls) Among them, Loss is the total loss, Loss (CIoU) 、Loss (conf) 、Loss (cls) They are positioning loss, confidence loss and classification loss respectively, d represents the Euclidean distance between the center points of the two bounding boxes, c represents the diagonal distance of the closure, IoU represents the intersection over union of the predicted bounding box and the true bounding box, and w gt With h gt are the width and height of the real border, w and h are the width and height of the predicted bounding box, respectively. The corn pest area detection model divides the input image into S×S grids, each grid has B aiming boxes, represents that there is a real object in the j-th aiming box in the i-th grid, which is equal to 1, and On the contrary, it means that there is no real object in the j-th aiming box in the i-th grid, which is equal to 0, λ noobj is the weight coefficient.

2. The method for detecting corn pest areas by combining context perception and multi-scale hybrid attention according to claim 1, characterized in that: The step (3) specifically comprises the following steps: (3a) The training samples are input into the corn pest area detection model, and the convolution operation in the backbone network outputs feature maps of different scales; (3b) The feature maps of different scales output by the backbone network are input into the improved feature aggregation network, the multi-scale hybrid attention module extracts effective features, and the target feature representation is enhanced by the context-aware module to obtain an output feature map containing target positioning information and classification information; (3c) decoding the output feature map obtained in step (3b) and outputting the prediction result, i.e., the coordinates of the pest area and the classification result, and calculating the loss; (3d) Back propagation, updating relevant parameters in the corn pest area detection model.

3. The method for detecting corn pest areas by combining context perception and multi-scale hybrid attention according to claim 1, characterized in that: The step (2b) specifically comprises the following steps: (2b1) Construct a position attention module to calculate the spatial dependency between any two positions in the feature map. Given an intermediate feature map X∈R C×H×W As input, the feature map size is H×W, the channel dimension is C, and the specific calculation formula is as follows: From this we can deduce as follows: Among them, (i, j) represents the coordinates of the current position and other positions whose dependencies are to be calculated, q is the output signal, Q is the position attention mechanism, (k, j) is the coordinate of any point in the feature map, σ represents the softmax function, I(x i,j )=W I x i,j ,M(x k,l )=W M x k,l ,N(x k,l )=W N x k,l ,I(x i,j ),M(x k,l ), N(x k,l ) are the outputs of the corresponding convolutional layers; W I ,W M ,W N are the parameters of the corresponding convolutional layers; (2b2) ​​Construct a multi-scale channel attention module to extract features along the channel dimension at global and local scales through global average pooling operations and 1x1 convolution operations. The specific calculation formula is as follows: Global scale: G(X′)=BN(C2((LR(BN(C1(g(X′))))))) Local scale: L(X′)=BN(C2(LR(BN(C1(X′))))) Multi-scale channel attention: Where X′ is the output of the position attention module, CA is the multi-scale channel attention, G(X′)∈R C×1×1 is the global scale, L(X′)∈R C×H×W is the local scale, C1 and C2 represent the convolution kernel parameters respectively. and The convolution operation, BN represents the Batch Normalization operation, LR represents the Leaky Relu activation function operation, and g represents the global average pooling operation. It is the element addition operation of broadcast mechanism; (2b3) Combine the position attention module with the multi-scale channel attention module to construct a multi-scale hybrid attention module. The specific calculation formula is as follows: Among them, X″ is the feature map of output optimization; It is the element-wise multiplication operation.

4. The method for detecting corn pest areas by combining context perception and multi-scale hybrid attention according to claim 1, characterized in that: The step (2c) specifically comprises the following steps: (2c1) extracting shallow feature maps and deep feature maps output by the backbone network; (2c2) The shapes of shallow and deep feature maps are changed to be consistent with the target feature map through upsampling and downsampling operations; (2c3) First, the shallow and deep feature maps are optimized through a multi-scale mixed attention module and the two are fused. Then, the features are refined through convolution operations and fused with the target feature map to enhance the feature information of the pest area.

Images