Method for Detecting Tiny-Scale Pest Images Based on Attention Fusion Factor Feature Pyramid

Abstract

The present invention relates to a method for detecting extremely small-scale pest images based on an attention fusion factor feature pyramid, which solves the defect of difficult detection of extremely small-scale pest images compared with the prior art. The present invention includes the following steps: acquisition of extremely small-scale pest image data; construction of an attention fusion factor feature pyramid; construction of a pest target localization and classification network; joint training of the attention fusion factor feature pyramid and the pest target localization and classification network; acquisition of the to-be-detected extremely small-scale pest image; and localization and recognition of the extremely small-scale pest image. The present invention improves the ability to extract detailed features of small-scale pest images and enhances the feature expression ability of small-scale pests.

Description

Technical Field

[0001] The present invention relates to the technical field of pest image detection, and specifically to a method for detecting extremely small-scale pest images based on an attention fusion factor feature pyramid. Background Art

[0002] Throughout the entire growth cycle of crops, they are invaded by various pests, which seriously affect the yield and quality of crops. The identification and detection of agricultural pests are the key to crop pest monitoring and control. Due to the rapid development of deep learning technology, identification methods based on deep neural networks have been widely applied to the automatic detection and identification tasks of agricultural pests. This identification method has largely solved the problems of insufficient scientific and technological personnel and the difficulty of manual identification in completing a large number of pest identifications, and can timely and effectively prevent and control agricultural pests, reducing the losses caused by pests.

[0003] Although great progress has been made in the target detection and identification tasks of agricultural pests based on deep learning technology, the existing agricultural pest target detection and identification technologies pose great challenges to the accurate identification and detection of extremely small-scale pest targets, and problems such as missed detection and false detection are likely to occur. This is because the scale of the pest targets collected in the image is extremely small, making it difficult for the deep convolutional neural network to characterize their features. Current methods basically use a feature pyramid network to solve the detection and identification of small-scale pest targets. Low-level detail features are helpful for the identification of small pests, and high-level semantic features are helpful for the identification of larger-scale pests. However, low-level features also require the help of other high-level features when identifying small-scale pest targets. How much help comes from other layers? How to balance the information fusion between the high-level and low-level features in the feature pyramid network to improve the feature characterization ability of small-scale pests.

[0004] Therefore, how to further improve the feature expression of the existing network for extremely small-scale pests and achieve high accuracy in the detection and identification of extremely small-scale pest targets has become an urgent technical problem to be solved in the field of intelligent plant protection. Summary of the Invention

[0005] The purpose of the present invention is to solve the defect in the prior art that it is difficult to detect extremely small-scale pest images, and to provide a method for detecting extremely small-scale pest images based on an attention fusion factor feature pyramid to solve the above problems.

[0006] To achieve the above purpose, the technical solution of the present invention is as follows:

[0007] A method for detecting extremely small-scale pest images based on an attention fusion factor feature pyramid, comprising the following steps:

[0008] 11) Acquisition of extremely small-scale pest image data: Select extremely small-scale pest images from the acquired pest image data, and select the corresponding extremely small-scale pest target coordinate information and class labels for the selected pest images;

[0009] 12) Construction of a feature pyramid based on an attention fusion factor;

[0010] 13) Construction of a pest target localization and classification network: Use the detailed feature information of extremely small-scale pest images to construct a one-stage anchor-free pest target localization and classification network;

[0011] 14) Joint training of the feature pyramid based on an attention fusion factor and the pest target localization and classification network: Train the feature pyramid based on an attention fusion factor and the pest target localization and classification network through a soft weighted loss function;

[0012] 15) Acquisition of the extremely small-scale pest image to be measured: Acquire the extremely small-scale pest image to be detected;

[0013] 16) Localization and recognition of extremely small-scale pest images: Input the extremely small-scale pest image to be detected into the trained feature pyramid based on an attention fusion factor to obtain the detailed feature information of the extremely small-scale pest image; Input the extracted detailed feature information of the extremely small-scale pest image into the trained pest target localization and classification network to obtain the class and corresponding position coordinates of the pest target, and count the number of each type of pest.

[0014] The construction of the feature pyramid based on an attention fusion factor includes the following steps:

[0015] 21) Set the multi-scale pest feature extraction network as a deep residual network to enhance the feature expression ability of pest targets. Select the output feature maps of 4 residual blocks of the deep residual network, denoted as F1, F2, F3, and F4, as the input of the feature pyramid based on an attention fusion factor, and extract rich feature information of extremely small-scale pests from them;

[0016] 22) Perform channel transformation on the feature maps F1, F2, F3, and F4 input to the feature pyramid network based on an attention fusion factor. Use a 1x1x256 convolutional layer, and the channel number of the output pest feature maps is unified to 256. The output feature maps are F1', F2', F3', and F4' respectively;

[0017] 23) Perform scale transformation on the feature map F4' input to the feature pyramid network based on an attention fusion factor. Use the bilinear interpolation method to upsample the input pest feature map by 2 times, and the size of the output feature map becomes 2 times the original, which is F4”;

[0018] 24) Input the upsampled pest feature map F4” into the attention weight generation module,

[0019] The first layer of the attention weight generation module is a convolutional layer with a convolution kernel size of 1×1, the second layer is a ReLU non-linear activation layer, the third layer is a convolutional layer with a convolution kernel size of 3×3 to further extract the pest feature information of extremely small scales, and finally the sigmoid activation function is used to output the attention weight W;

[0020] 25) After multiplying the upsampled pest feature map F4” by the output attention weight W, the feature map is fused with the channel-transformed feature map F3' output in step 22) through the corresponding position point addition operation, and the pest feature map P1 with attention features is output, constituting the first level of the feature pyramid based on the attention fusion factor;

[0021] 26) Taking the generated first-layer pest feature map P1 of the feature pyramid based on the attention fusion factor as the input, performing step 23), and outputting the feature map P1'; taking the generated first-layer pest feature P1 of the feature pyramid as the input, performing step 24), and outputting the attention weight W', multiplying the feature map P1' by the output attention weight W', and then fusing the feature map with the channel-transformed feature map F2' output in step 22) through the corresponding position point addition operation, and outputting the pest feature map P2 with attention features, constituting the second level of the feature pyramid based on the attention fusion factor;

[0022] 27) Taking the generated first-layer pest feature map P2 of the feature pyramid as the input, performing step 23), and outputting the feature map P′2; taking the generated first-layer pest feature P2 of the feature pyramid as the input, performing step 24), and outputting the attention weight W”, multiplying the feature map P2' by the output attention weight W”, and then fusing the feature map with the channel-transformed feature map F1' output in step 22) through the corresponding position point addition operation, and outputting the pest feature map P3 with attention features, constituting the third level of the feature pyramid based on the attention fusion factor;

[0023] 28) Finally, through the above steps, the pest feature maps P1, P2, and P3 with attention features are obtained, constituting a feature pyramid network with three layers of fused attention feature maps.

[0024] The construction of the pest target location and classification network includes the following steps:

[0025] 31) Set the pest target location and classification network as a one-stage anchor-free multi-class pest target location and classification network;

[0026] 32) Set the multi-class pest target location and classification network to include an extremely small scale pest feature transformation layer, an extremely small scale pest location sub-network, and an extremely small scale pest classification sub-network;

[0027] 33) Set the input of the extremely small-scale pest feature transformation layer as the pest feature map fused with attention. It uses 256 convolutional layers of 3×3 to further extract features from the pest feature map, obtaining extremely small-scale pest feature information;

[0028] 34) Set the inputs of the extremely small-scale pest localization sub-network and the extremely small-scale pest classification sub-network as the extremely small-scale pest feature information;

[0029] Among them, the extremely small-scale pest localization sub-network uses a convolutional layer of 1×1×4C to output the position of the corresponding category of the extremely small-scale pest, where C is the number of pest categories, and 4 represents the position information (l, r, t, b) of the pest target bounding box. Among them, the variables l, r, t, and b are the distances from the center point of the pest to the left, right, upper, and lower boundaries of the bounding box respectively;

[0030] The extremely small-scale pest classification sub-network uses a convolutional layer of 1×1×C to output the category name corresponding to the extremely small-scale pest.

[0031] The joint training of the feature pyramid based on the attention fusion factor and the pest target localization and classification network includes the following steps:

[0032] 41) Determination of positive and negative training samples:

[0033] 411) Input the extremely small-scale pest images with coordinate information and category labels into the deep residual network and the feature pyramid based on the attention fusion factor,

[0034] Among them, the corresponding coordinate information and category labels are expressed as (x, y, w, h, L), and (x, y) and (w, h) are the coordinates and length and width of the center point of the pest bounding box respectively, and L is the category label of the small-scale pest;

[0035] After passing through the deep residual network and the feature pyramid based on the attention fusion factor respectively, an extremely small-scale pest feature map P k is obtained, and its corresponding stride is s k , where k = 1, 2, 3;

[0036] 412) Definition of the positive sample region. Define a valid positive sample region within the true annotation box of the pest target: For the region with the center point (x, y), its positive sample region is (x - Rs k , y - Rs k , x + Rs k , y + Rs k ), where R is the region hyperparameter, set to 1.5, and s k represents the feature map stride;

[0037] 413) Mapping of the extremely small-scale pest feature map:

[0038] Map each position (x k in the feature map P i , y i ) to the original pest image to obtain the position (x o , y o ), and the mapping relationship is:

[0039]

[0040] 414) Determination of positive and negative sample points:

[0041] Using the obtained positive sample region, determine whether the position of the sample point in the feature map of pests at the smallest scale after mapping falls within the positive sample region. If it falls within the positive sample region, the corresponding sample point is a positive sample, and its class label is the label L of the corresponding annotation box; otherwise, it is a negative sample point, and the corresponding label is 0.

[0042] 42) Use the obtained positive and negative training samples to train the pest target localization and classification network and calculate the network loss value;

[0043] 421) Calculation of the weight of the loss value:

[0044] When using the soft weighted loss function to train the pest target localization and classification network, all positive samples are given different attentions. The closer the position is to the center of the pest target, the more attention is given.

[0045] For the sample point p at the position (i, j) of the small-scale pest image ij The weight w ij Given is:

[0046]

[0047] Among them, the weight function f measures the distance from the sample point to the pest target bounding box B. The closer the distance, the greater the corresponding weight. The definition of this weight function f is as follows:

[0048]

[0049] Among them, the variables l, r, t, and b are the distances from the pest center point to the left, right, upper, and lower boundaries of the bounding box, respectively;

[0050] 422) Calculation of the total loss value of the pest target localization and classification network training:

[0051] Set the total loss value of the pest target localization and classification network to be equal to the sum of the loss values of the two sub-networks, namely the smallest-scale pest localization sub-network and the smallest-scale pest classification sub-network;

[0052] The extremely small-scale pest classification sub-network uses the Focal loss function to solve the problem of imbalance between positive and negative samples. The extremely small-scale pest localization sub-network uses the IoU loss function.

[0053] The total loss function of the pest target localization and classification network is:

[0054]

[0055] Among them, p represents a sample, p + and p- and respectively represent positive and negative samples. The variables c, d, and f respectively represent the category, bounding box, and center weight value of the pest predicted by the network. c * , d * and f * are the corresponding true annotation values; l FL and l IoU respectively represent the Focal loss and IoU loss functions;

[0056] 423) Training of the deep residual network, feature pyramid based on attention fusion factor, and pest target localization and classification network:

[0057] Input the extremely small-scale pest images with coordinate information and category labels selected into the deep residual network to extract pest features, then further input them into the feature pyramid based on attention fusion factor to output the extremely small-scale pest information with fused attention, and then input them into the pest target localization and classification network. By comparing the prediction results of the pest target localization and classification network with the annotated true coordinate information and category labels, use the method in step 422) to calculate the total loss value of the pest target localization and classification network, and use the gradient backpropagation algorithm to update the parameters of the deep residual network, feature pyramid based on attention fusion factor, and pest target localization and classification network. When the total loss of the deep residual network, feature pyramid based on attention fusion factor, and pest target localization and classification network is 0 or reaches the set value, the training is completed.

[0058] Beneficial effects

[0059] The method for detecting extremely small-scale pest images based on the feature pyramid with attention fusion factor in the present invention, compared with the prior art, improves the ability to extract detailed features of small-scale pest images and enhances the feature expression ability of small-scale pests; adopts an anchor-free pest position regression method to avoid complex anchor box settings and improve the efficiency of small-scale pest target detection; finally, by introducing a weight factor into the loss function, reduces the misdetection of small-scale pests, thereby improving the accuracy and speed of small-scale pest image detection.

[0060] The method described in the present invention can be applied to the recognition and localization tasks of multi-category small-scale pests, effectively improving the accuracy and efficiency of small-scale pest image detection.

[0061] In the present invention, by introducing an attention fusion factor into the feature pyramid network, the feature pyramid network can adaptively adjust the underlying feature layer to obtain help from the high-level feature map when performing bottom-up and top-down feature fusion, further enriching the features of extremely small-scale pests. On the other hand, for network optimization, it is found here that different samples should be given different attentions, that is, the closer the position is to the center of the pest target, the more attention is given. Therefore, a soft weighted loss function is used to optimize the parameters of the neural network, which can reduce the false detection rate of extremely small-scale pest target detection. Brief Description of the Drawings

[0062] Figure 1 is the sequence diagram of the method of the present invention;

[0063] Figure 2 is the effect diagram of detecting an extremely small-size pest image by using the method of the present invention. Detailed Embodiments

[0064] To further understand and recognize the structural features and achieved effects of the present invention, the following is a detailed description in conjunction with preferred embodiments and accompanying drawings:

[0065] As Figure 1 shown, an extremely small-scale pest image detection method based on an attention fusion factor feature pyramid according to the present invention includes the following steps:

[0066] The first step is to obtain extremely small-scale pest image data: select extremely small-scale pest images from the obtained pest image data, and select the corresponding extremely small-scale pest target coordinate information and category labels of the selected pest images.

[0067] The second step is to construct a feature pyramid based on an attention fusion factor.

[0068] Since the pest target scale in the actual scene is extremely small, and the deep neural network has limited ability to represent pest features, resulting in low detection and recognition accuracy of small-scale pest targets. The existing solution uses the underlying features of the feature pyramid network to detect small-scale pest targets, but does not consider the help of high-level features to low-level features in the process of detecting pest targets. Therefore, the present invention proposes a feature pyramid network based on an attention fusion factor. This multi-scale pest feature extraction network is a deep residual network, which can enhance the feature expression ability of pest targets. Construct a feature pyramid network based on an attention fusion factor, and input the extremely small-scale pest image into the feature pyramid network based on an attention fusion factor to extract rich feature information of the extremely small-scale pest.

[0069] The specific steps are as follows:

[0070] (1) Set the multi-scale pest feature extraction network as a deep residual network to enhance the feature expression ability of pest targets. Select the output feature maps of 4 residual blocks of the deep residual network, denoted as F1, F2, F3, and F4, as the input of the feature pyramid based on the attention fusion factor, and extract rich feature information of extremely small-scale pests from them.

[0071] (2) Perform channel transformation on the feature maps F1, F2, F3, and F4 input to the feature pyramid network based on the attention fusion factor. Use a convolutional layer with 1x1x256. The number of channels of the output pest feature maps is unified to 256, and the output feature maps are F1', F2', F3', and F4' respectively.

[0072] (3) Perform scale transformation on the feature map F4' input to the feature pyramid network based on the attention fusion factor. Use the bilinear interpolation method to upsample the input pest feature map by a factor of 2. The size of the output feature map becomes twice the original, which is F4".

[0073] (4) Input the upsampled pest feature map F4" into the attention weight generation module.

[0074] The first layer of the attention weight generation module is a convolutional layer with a kernel size of 1×1, the second layer is a ReLU non-linear activation layer, the third layer is a convolutional layer with a kernel size of 3×3, further extract the feature information of extremely small-scale pests, and finally use the sigmoid activation function to output the attention weight W.

[0075] (5) After multiplying the upsampled pest feature map F4" by the output attention weight W, perform element-wise addition with the channel-transformed feature map F3' output in step (2) at corresponding positions to fuse the feature maps, and output the pest feature map P1 with attention features, forming the first level of the feature pyramid based on the attention fusion factor.

[0076] (6) Use the generated first-layer pest feature map P1 of the feature pyramid based on the attention fusion factor as the input, execute step (), and output the feature map P1'; use the generated first-layer pest feature P1 of the feature pyramid as the input, execute step (4), and output the attention weight W'. After multiplying the feature map P1' by the output attention weight W', perform element-wise addition with the channel-transformed feature map F2' output in step (2) at corresponding positions to fuse the feature maps, and output the pest feature map P2 with attention features, forming the second level of the feature pyramid based on the attention fusion factor.

[0077] (7) Use the first-layer pest feature map P2 of the generated feature pyramid as the input, execute step (3), and output the feature map P'2; use the first-layer pest feature P2 of the generated feature pyramid as the input, execute step (4), and output the attention weight W". After performing element-wise multiplication on the feature map P2' and the output attention weight W", perform feature map fusion with the feature map F1' after channel transformation output in step (2) through element-wise addition at corresponding positions, and output the pest feature map P3 with attention features, constituting the third layer of the feature pyramid based on the attention fusion factor.

[0078] (8) Finally, through the above steps, the pest feature maps P1, P2, and P3 with attention features are obtained, constituting a feature pyramid network with 3 layers of fused attention feature maps.

[0079] The third step is to construct a pest target localization and classification network: Use the detailed feature information of the extremely small-scale pest images to construct a one-stage anchor-free pest target localization and classification network. Since the existing methods have low detection efficiency, and the size and number of manually set anchor boxes are unreasonable, it is difficult to quickly and accurately extract the regions of interest of small-scale pests. Therefore, the present invention proposes a one-stage anchor-free multi-class pest target localization and classification network, which realizes the accurate localization of small-scale pest targets by learning the distance from the center of the pest target to the bounding box. The specific steps are as follows:

[0080] (1) Set the pest target localization and classification network as a one-stage anchor-free multi-class pest target localization and classification network.

[0081] (2) Set that the multi-class pest target localization and classification network includes an extremely small-scale pest feature transformation layer, an extremely small-scale pest localization sub-network, and an extremely small-scale pest classification sub-network.

[0082] (3) Set the input of the extremely small-scale pest feature transformation layer as the pest feature map with fused attention. It uses 256 3×3 convolutional layers to perform further feature extraction on the pest feature map to obtain the extremely small-scale pest feature information.

[0083] (4) Set the inputs of the extremely small-scale pest localization sub-network and the extremely small-scale pest classification sub-network as the extremely small-scale pest feature information;

[0084] Among them, the extremely small-scale pest localization sub-network uses a 1×1×4C convolutional layer to output the positions of the corresponding classes of extremely small-scale pests, where C is the number of pest classes, and 4 represents the position information (l, r, t, b) of the pest target bounding box, where the variables l, r, t, b are the distances from the pest center point to the left, right, upper, and lower boundaries of the bounding box respectively;

[0085] The extremely small-scale pest classification sub-network uses a 1×1×C convolutional layer to output the class names of the extremely small-scale pests.

[0086] Step 4: Joint training of the attention fusion factor feature pyramid and the pest target localization and classification network: Train the attention fusion factor feature pyramid and the pest target localization and classification network through a soft weighted loss function.

[0087] The separate training of the attention fusion factor feature pyramid network and the pest target localization and classification network will lead to low training efficiency and high complexity of the network, which is not conducive to practical application scenarios. Therefore, in this invention, the attention fusion factor feature pyramid network and the pest target localization and classification network are jointly trained, and only one loss function is designed to optimize, which can improve the detection accuracy while enhancing the training efficiency.

[0088] The specific steps are as follows:

[0089] (1) Determination of positive and negative training samples. Different from the object detection method based on anchor boxes, the positive and negative training samples are determined by using the method based on the intersection over union. This invention adopts an anchor-free pest target detection method and uses a method based on the central circle falling method to determine positive and negative training samples.

[0090] A1) Input the extremely small-scale pest image with coordinate information and class label into the deep residual network and the attention fusion factor feature pyramid,

[0091] where the corresponding coordinate information and class label are expressed as (x, y, w, h, L), (x, y) and (w, h) are the coordinates of the center point of the pest bounding box and the length and width respectively, and L is the class label of the small-scale pest;

[0092] After passing through the deep residual network and the attention fusion factor feature pyramid respectively, the extremely small-scale pest feature map P k is obtained, and its corresponding stride is s k , where k = 1, 2, 3;

[0093] A2) Definition of the positive sample region. Define a valid positive sample region within the true annotation box of the pest target: For the region with the center point (x, y), its positive sample region is (x - Rs k , y - Rs k , x + Rs k , y + Rs k ), where R is the region hyperparameter, set to 1.5, and s k represents the feature map stride;

[0094] A3) Mapping of the extremely small-scale pest feature map:

[0095] Map each position (x k in the feature map P i , y i), mapped into the original pest image, to obtain the position (x o , y o ), and the mapping relationship is:

[0096]

[0097] A4) Determination of positive and negative sample points:

[0098] Using the obtained positive sample region, determine whether the position of the sample point in the pest feature map at the minimum scale after mapping falls within the positive sample region. If it falls within the positive sample region, the corresponding sample point is a positive sample, and its class label is the label L of the corresponding annotation box; otherwise, it is a negative sample point, and the corresponding label is 0.

[0099] (2) Using the obtained positive and negative training samples to train the pest target localization and classification network to calculate the network loss value. Using the usual method to train the network will lead to ineffective training, resulting in low detection accuracy for small-scale pests. The present invention designs a soft weighted loss function. During network training, different attentions are given to all positive samples, that is, the closer the position is to the center of the pest target, the more attention is given.

[0100] B1) Calculation of the weight of the loss value:

[0101] Using the soft weighted loss function, during the training of the pest target localization and classification network, different attentions are given to all positive samples. The closer the position is to the center of the pest target, the more attention is given;

[0102] For the sample point p at the position (i, j) of the small-scale pest image ij The weight w given ij is:

[0103]

[0104] where the weight function f measures the distance from the sample point to the pest target bounding box B. The closer the distance, the greater the corresponding weight. The definition of this weight function f is as follows:

[0105]

[0106] where the variables l, r, t, and b are the distances from the pest center point to the left, right, top, and bottom boundaries of the bounding box respectively;

[0107] B2) Calculation of the total loss value of the pest target localization and classification network training:

[0108] Set the total loss value of the pest target localization and classification network to be equal to the sum of the loss values of the two sub-networks, namely the minimum-scale pest localization sub-network and the minimum-scale pest classification sub-network;

[0109] The extremely small-scale pest classification sub-network uses the Focal loss function to solve the problem of imbalance between positive and negative samples. The extremely small-scale pest localization sub-network uses the IoU loss function.

[0110] The total loss function of the pest target localization and classification network is:

[0111]

[0112] where p represents a sample, p + and p - and respectively represent positive and negative samples. The variables c, d, and f respectively represent the category, bounding box, and center weight value of the pest predicted by the network. c * , d * and f * are the corresponding true annotation values; l FL and l IoU respectively represent the Focal loss and IoU loss functions.

[0113] B3) Training of the deep residual network, feature pyramid based on attention fusion factor, and pest target localization and classification network:

[0114] Input the extremely small-scale pest images with coordinate information and category labels selected into the deep residual network to extract pest features, and then further input them into the feature pyramid based on the attention fusion factor to output the extremely small-scale pest information with fused attention. Then input it into the pest target localization and classification network. By comparing the prediction results of the pest target localization and classification network with the annotated true coordinate information and category labels, use the total loss value of the pest target localization and classification network calculated in step B2), and adopt the gradient backpropagation algorithm to update the parameters of the deep residual network, the feature pyramid based on the attention fusion factor, and the pest target localization and classification network. When the total loss of the three, namely the deep residual network, the feature pyramid based on the attention fusion factor, and the pest target localization and classification network, is 0 or reaches the set value, the training is completed.

[0115] Step 5, acquisition of the extremely small-scale pest image to be measured: Obtain the extremely small-scale pest image to be detected.

[0116] Step 6, localization and recognition of the extremely small-scale pest image: Input the extremely small-scale pest image to be detected into the trained feature pyramid based on the attention fusion factor to obtain the detailed feature information of the extremely small-scale pest image; input the extracted detailed feature information of the extremely small-scale pest image into the trained pest target localization and classification network to obtain the category and corresponding position coordinates of the pest target, and count the number of each type of pest.

[0117] The present invention constructs a feature pyramid network based on an attention fusion factor, which enhances the network's ability to extract detailed features of small-scale pest images and strengthens the feature expression ability of small-scale pests; adopts an anchor-free pest position regression method to avoid complex anchor settings and improve the efficiency of small-scale pest target detection; finally, by introducing a weight factor into the loss function, the misdetection of small-scale pests is reduced, thereby improving the accuracy and speed of small-scale pest image detection. The small-scale pest image detection method based on the feature pyramid network with an attention fusion factor proposed by the present invention can be applied to the recognition and positioning tasks of multi-category small-scale pests, effectively improving the accuracy and efficiency of small-scale pest image detection.

[0118] Experiments on the small-scale pest image dataset show that, as shown in Table 1, the detection accuracy of small-scale pests by the method of the present invention is higher than that of the current relatively advanced detection methods. Figure 2 The following is the effect diagram of using the method of the present invention to detect multi-category pest images. Figure 2 It can be seen that although the pest scale is extremely small, the detection method of the present invention still performs well. This is because the construction of the feature pyramid network based on the attention fusion factor enhances the network's ability to extract detailed features of small-scale pest images and strengthens the feature expression ability of small-scale pests; at the same time, by introducing a weight factor into the loss function, the misdetection of small-scale pests is reduced, and finally the accurate detection of small-scale pest images is achieved.

[0119] Table 1 Detection results of the method of the present invention on the small-scale pest image dataset

[0120] Method AP <![CDATA[AP 0.5 > <![CDATA[AP 0.75 > <![CDATA[AP s > <![CDATA[AP m > <![CDATA[AP l > RetinaNet 41.2 64.7 48.4 25.7 47.3 45.0 ATSS 46.6 72.6 55.4 31.7 51.0 40.0 FSAF 45.2 71.3 52.7 30.5 49.5 35.1 FCOS 45.7 71.9 53.5 32.8 50.9 45.0 FreeAnchor 43.6 68.1 51.6 27.1 48.7 40.1 The method of the present invention 48.9 75.8 57.2 36.1 53.1 60.0

[0121] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and the principles described in the specification are only the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A very small scale pest image detection method based on attention fusion factor feature pyramid, characterized by: The following steps are involved: 11) Acquisition of extremely small-scale pest image data: selecting an extremely small-scale pest image from the acquired pest image data, and selecting extremely small-scale pest target coordinate information and category labels corresponding to the selected pest image; 12) Construction of feature pyramid based on attention fusion factor; 13) Construction of pest target localization and classification network: Utilizing the detailed feature information of extremely small-scale pest images, a one-stage anchor-free pest target localization and classification network is constructed; 14) Joint training of the attention fusion factor feature pyramid and the pest target location classification network: The attention fusion factor feature pyramid and the pest target location classification network are trained using a soft weighted loss function; The joint training of the attention fusion factor feature pyramid and the pest target positioning classification network includes the following steps: 141) Determination of positive and negative training samples: 1411) Inputting the extremely small scale pest image with coordinate information and category labels into the deep residual network and the feature pyramid based on the attention fusion factor, The corresponding coordinate information and category labels are expressed as (x, y, w, h, L), where (x, y) and (w, h) are the coordinates, length, and width of the center point of the pest bounding box, respectively, and L is the category label of the small-scale pest. The extremely small scale pest feature map P is obtained by using the deep residual network and the feature pyramid based on the attention fusion factor. k , the corresponding step size is s k , where k = 1, 2, 3; 1412) Definition of positive sample area: define a valid positive sample area in the real annotation box of the pest target: for the area with the center point (x, y), the positive sample area is (x-Rs k ,y-Rs k ,x+Rs k ,y+Rs k ), where R is the region hyperparameter, set to 1.5, s k Indicates the feature map step size; 1413) Extremely small scale pest feature mapping: The feature map P k Each position (x i ,y i ), mapped to the original pest image, and the position (x o ,y o ), the mapping relationship is: 1414) Determination of positive and negative sample points: Using the obtained positive sample area, determine whether the mapped position of the sample point in the extremely small-scale pest feature map falls within the positive sample area. If it falls within the positive sample area, the corresponding sample point is a positive sample, and its category label is the label L of the corresponding annotation box. Otherwise, it is a negative sample point, and the corresponding label is 0. 142) Using the obtained positive and negative training samples to train the pest target positioning classification network to calculate the network loss value; 1421) Weight calculation of loss value: When training the pest target location classification network using a soft weighted loss function, all positive samples are given different attention levels. The closer the location is to the center of the pest target, the more attention it is given. The sample point p at position (i, j) of the small-scale pest image ij The weight w given ij for: The weight function f measures the distance between the sample point and the pest target bounding box B. The closer the distance, the greater the corresponding weight. The weight function f is defined as follows: Among them, the variables l, r, t, and b are the distances from the center point of the pest to the left, right, top, and bottom boundaries of the bounding box, respectively; 1422) Calculation of total loss value of pest target location classification network training: The total loss value of the pest target positioning and classification network is set to be equal to the sum of the loss values of the two sub-networks: the very small scale pest positioning sub-network and the very small scale pest classification sub-network; The extremely small scale pest classification sub-network adopts the Focalloss loss function to solve the problem of imbalance between positive and negative samples, and the extremely small scale pest localization sub-network adopts the IoUloss loss function. The total loss function of the pest target positioning and classification network is: Among them, p represents the sample, p + and p - and represent positive samples and negative samples respectively. Variables c, d, and f represent the category, bounding box, and center weight value of the pest predicted by the network respectively. * d * and f * is the corresponding true annotation value; l FL and l IoU Represent Focal loss and IoU loss loss functions respectively; 1423) Training of deep residual networks, feature pyramids based on attention fusion factors, and pest target positioning and classification networks: The selected extremely small-scale pest image with coordinate information and category labels is input into the deep residual network to extract pest features, and then further input into the extremely small-scale pest information fused with attention based on the attention fusion factor feature pyramid output, and then input into the pest target positioning and classification network. By comparing the prediction result of the pest target positioning and classification network with the annotated real coordinate information and category labels, the total loss value of the pest target positioning and classification network is calculated using step 422), and the gradient back propagation algorithm is used to implement parameter updates of the deep residual network, the feature pyramid based on the attention fusion factor, and the pest target positioning and classification network. When the total loss of the deep residual network, the feature pyramid based on the attention fusion factor, and the pest target positioning and classification network is 0 or reaches a set value, the training is completed; 15) Acquiring images of extremely small-scale pests to be detected: Acquiring images of extremely small-scale pests to be detected; 16) Localization and Recognition of Extremely Small-Scale Pest Images: The extremely small-scale pest images to be detected are input into a trained feature pyramid based on attention fusion factors to obtain detailed feature information of the extremely small-scale pest images. The extracted detailed feature information of the extremely small-scale pest images is input into a trained pest target localization and classification network to obtain the pest target category and corresponding location coordinates, and the number of pests of each category is counted.

2. The method for detecting pests in very small scale images based on the attention fusion factor feature pyramid according to claim 1, characterized in that: The construction of the feature pyramid based on the attention fusion factor includes the following steps: 21) The multi-scale pest feature extraction network is set as a deep residual network to enhance the feature expression ability of pest targets. The output feature maps of the four residual blocks of the deep residual network, denoted as F1, F2, F3 and F4, are selected as the input of the feature pyramid based on the attention fusion factor to extract rich feature information of extremely small-scale pests; 22) The feature maps F1, F2, F3, and F4 input to the feature pyramid network based on the attention fusion factor are transformed into channels, and a 1x1x256 convolution layer is used. The number of channels of the output pest feature map is unified to 256, and the output feature maps are F1', F2', F3', and F4' respectively; 23) Performing a scale transformation on the feature map F4' input to the feature pyramid network based on the attention fusion factor, using a bilinear interpolation method to upsample the input pest feature map by a factor of 2, and the output feature map size becomes twice the original, i.e. F4"; 24) Input the upsampled pest feature map F4" into the attention weight generation module, The first layer of the attention weight generation module is a convolution layer with a convolution kernel size of 1×1, the second layer is a ReLU nonlinear activation layer, and the third layer is a convolution layer with a convolution kernel size of 3×3 to further extract extremely small-scale pest feature information. Finally, a sigmoid activation function is used to output the attention weight W. 25) After dot-multiplying the upsampled pest feature map F4″ with the output attention weight W, the feature map is fused with the channel-transformed feature map F3′ output in step 22) by adding corresponding positions, and the pest feature map P1 with attention features is output, forming the first level of the feature pyramid based on the attention fusion factor; 26) Using the generated first-layer pest feature map P1 based on the attention fusion factor feature pyramid as input, executing step 23), outputting a feature map P1'; Using the generated first-layer pest feature P1 of the feature pyramid as input, executing step 24), outputting an attention weight W', performing a dot multiplication between the feature map P1' and the output attention weight W', and then fusing the feature maps with the channel-transformed feature map F2' output in step 22) by adding corresponding positions, outputting a pest feature map P2 with attention features, and forming the second level of the feature pyramid based on the attention fusion factor; 27) Using the generated first-layer pest feature map P2 of the feature pyramid as input, executing step 23), outputting a feature map P'2; Using the generated first-layer pest feature P2 of the feature pyramid as input, executing step 24), outputting an attention weight W", performing a dot multiplication of the feature map P2' and the output attention weight W", and then fusing the feature maps with the channel-transformed feature map F1' outputted in step 22) by adding corresponding positions, outputting a pest feature map P3 with attention features, forming the third level of the feature pyramid based on the attention fusion factor; 28) Finally, after the above steps, the pest feature maps P1, P2 and P3 of the attention features are obtained, forming a feature pyramid network with three layers of fused attention feature maps.

3. The method for detecting pests in very small scale images based on the attention fusion factor feature pyramid according to claim 1, characterized in that: The construction of the pest target positioning classification network includes the following steps: 31) The pest target localization and classification network is set as a one-stage anchor-free multi-category pest target localization and classification network; 32) Setting a multi-category pest target positioning and classification network includes a very small scale pest feature transformation layer, a very small scale pest positioning sub-network and a very small scale pest classification sub-network; 33) The input of the very small scale pest feature transformation layer is set as the pest feature map fused with attention. It uses 256 3×3 convolution layers to further extract features from the pest feature map to obtain very small scale pest feature information; 34) Assume that the inputs of the extremely small-scale pest positioning sub-network and the extremely small-scale pest classification sub-network are both extremely small-scale pest feature information; The very small scale pest localization sub-network uses a 1×1×4C convolutional layer to output the location of the corresponding category of the very small scale pest, where C is the number of pest categories and 4 represents the location information of the pest target bounding box (l, r, t, b). The variables l, r, t, and b are the distances from the center point of the pest to the left, right, top, and bottom boundaries of the bounding box, respectively. The extremely small-scale pest classification subnetwork uses a 1×1×C convolutional layer to output the category name corresponding to the extremely small-scale pests.

Images