A Dense Tiny Pest Image Detection Method Based on a Multi-Core Attention Network

Abstract

The present invention relates to a method for detecting images of dense and tiny pests based on a multi-core attention network, which solves the defect of difficult detection of images of dense and tiny pests compared with the prior art. The present invention includes the following steps: generating a dataset of images of dense and tiny pests; extracting multi-scale features of pest images; constructing a small-scale pest dual-channel multi-core feature extraction model; training the small-scale pest dual-channel multi-core feature extraction model; and obtaining the detection results of images of dense and tiny pests. The present invention significantly improves the detection accuracy of images of dense and tiny pests, uses a dense pest area focusing extraction module to extract the dense pest area, effectively extracts multi-scale features of pests through a cross-stage local network, and combines the proposed small-scale pest dual-channel multi-core feature extraction model to more accurately identify tiny pests, accelerating the detection speed and accuracy of pest infestations.

Description

Technical Field

[0001] The present invention relates to the technical field of pest image recognition, and specifically, it is a method for detecting dense and tiny pest images based on a multi-core attention network. Background Art

[0002] In modern agriculture, pests such as aphids cause serious harm to crops. By sucking plant sap, spreading viruses, and secreting honeydew, etc., they cause the yellowing of crop leaves and even the death of the whole plant, seriously affecting the yield and quality of crops. The rise of convolutional neural networks has promoted the progress of object detection technology, resulting in two-stage and one-stage detection methods, but there are still many challenges in the application of agricultural pest detection.

[0003] Especially in the recognition of dense and tiny images, existing methods have many drawbacks. First, from the perspective of the characteristics of the images themselves, the resolution of dense and tiny images such as agricultural pests is usually low. With fewer pixels, a large amount of key details are lost. For example, the limb textures and morphological features of aphids are difficult to present clearly, which makes it difficult for the model to obtain sufficient effective features for accurate recognition. At the same time, in the images taken in the field, the scale range of pests varies greatly, and due to the dense distribution, the proportion between different pests is out of balance. The model has to process both extremely small-scale individuals and large-scale targets, which poses extremely high requirements for the feature capture ability. If not careful, it will miss one thing while attending to another. In addition, pests are densely arranged on plants, and occlusion occurs frequently. The information of the occluded part cannot be obtained by the model, further increasing the recognition difficulty. The receptive fields of some existing classic deep learning models are difficult to adapt to dense and tiny images such as agricultural pests. If the receptive field is too small, it cannot capture enough context information, resulting in inaccurate recognition of tiny pest individuals; if the receptive field is too large, it will introduce too much irrelevant background information, interfering with the model's judgment. In addition, in order to effectively extract the features of dense and tiny pest images, a relatively large-scale model may be required, but this will undoubtedly greatly increase the consumption of computing resources and reduce the computing speed. In agricultural scenarios where real-time pest detection is required, it is difficult to meet the actual application requirements. Summary of the Invention

[0004] The purpose of the present invention is to solve the defect in the prior art that it is difficult to detect dense and tiny pest images, and to provide a method for detecting dense and tiny pest images based on a multi-core attention network to solve the above problems.

[0005] To achieve the above purpose, the technical solution of the present invention is as follows:

[0006] A method for detecting dense and tiny pest images based on a multi-core attention network, comprising the following steps:

[0007] Generation of a Dense Tiny Pest Image Dataset: Using a dense pest area focusing and extraction module, focus on and extract the dense pest area to generate a dense tiny pest image dataset;

[0008] Extraction of Multi-scale Features of Pest Images: Extract multi-scale features by a Cross-Stage Partial Network;

[0009] Construct a small-scale pest two-channel multi-core feature extraction model;

[0010] Training of the Small-scale Pest Two-channel Multi-core Feature Extraction Model: Input the dense tiny pest image dataset into the small-scale pest two-channel multi-core feature extraction model for training;

[0011] Obtaining the Detection Results of Dense Tiny Pest Images: Obtain the pest image to be detected and input it into the Cross-Stage Partial Network to obtain the multi-scale feature map of the pest image, and input the multi-scale feature map of the pest image into the trained small-scale pest two-channel multi-core feature extraction model to obtain the detection results of the pest image.

[0012] The generation of the dense tiny pest image dataset includes the following steps:

[0013] Obtain wheat images damaged by pests, annotate and preprocess the images to construct a pest dataset;

[0014] Construct a dense pest area focusing and extraction module;

[0015] Construct a dense tiny pest image dataset: Input the pest dataset into the dense pest area focusing and extraction module to extract the dense pest area images, and mix the dense pest area images with the pest dataset to form a dense tiny pest image dataset.

[0016] The extraction of the multi-scale features of pest images includes the following steps:

[0017] Construct a Cross-Stage Partial Network, which includes a focusing structure, a convolutional normalization and activation structure, and a Cross-Stage Partial layer structure. The Cross-Stage Partial layer structure decomposes the residual block stack and introduces a Spatial Pyramid Pooling structure before the last Cross-Stage Partial layer structure to extract features;

[0018] The input image obtained from the dense tiny pest image dataset first enters the focusing structure for slicing operations to generate multiple low-resolution feature maps and stack them to quadruple the number of input channels. Then, the multi-channel feature maps are concatenated and stacked together, and are cross-processed through the convolutional normalization and activation structure and the Cross-Stage Partial layer structure to enhance the small-scale pest features, obtaining a small-scale pest feature map;

[0019] The focusing structure extracts pixel values from the input image every other pixel, and in this way, four independent feature layers are generated. Then these four feature layers are stacked together to integrate the width and height information of the image into the channel dimension, thereby quadrupling the number of input channels;

[0020] The convolution normalization activation structure adds a batch normalization layer and a SiLU activation function after the convolutional layer. The formula for the SiLU activation function f(x) is:

[0021] f(x) = x·sigmoid(x)

[0022] Among them, the formula for the sigmoid function is expressed as:

[0023]

[0024] Among them, x represents the input value, and e represents the natural constant;

[0025] The cross-stage local layer structure is divided into two parts: one part directly performs forward propagation, retaining relatively original feature information, including basic features at different scales; the other part undergoes multiple repeated 1×1 basic convolutions and 3×3 depthwise separable convolution operations to further extract and refine features, and mine more representative features at different scales. The information of these two parts is merged at the end of the network, enabling the fusion of features at different levels and scales, enriching the diversity of features, forming a feature map containing multi-scale information, and obtaining a multi-scale feature map of the pest image.

[0026] The construction of the small-scale pest two-channel multi-core feature extraction model includes the following steps:

[0027] It is set that the small-scale pest two-channel multi-core feature extraction model includes a two-channel feature pyramid module and a multi-core attention network module;

[0028] It is set that the two-channel feature pyramid module is composed of a top-down information flow and a bottom-up information flow;

[0029] Among them, the top-down information flow: starts from the higher layer of the network and gradually transmits information to the lower layer. First, the higher-level feature map is upsampled, and then it is fused with the feature map of the middle layer or the bottom layer to enhance the semantic information in the lower-level feature map. The formula is expressed as:

[0030] P5 = Conv(f3)

[0031] P4 = Conv(CSPLayer(UpSample(P5) ⊕ f2))

[0032] P3 = CSPLayer(UpScale(P4) ⊕ f1)

[0033] where, f i (i = 3, 2, 1) respectively represent three feature maps of the information flow input from top to bottom, P i (i = 5, 4, 3) respectively represent three feature maps of the information flow output from top to bottom, ⊕ represents the fusion operation, Conv represents the convolution operation, CSPLayer represents the cross-stage partial layer structure, and UpSample represents the upsampling operation;

[0034] Bottom-up information flow: Starting from the lower-level feature maps, the information is gradually passed to the higher-level feature maps. In the bottom-up information flow, downsampling operations are performed on the lower-level feature maps and fused with the higher-level feature maps. The formula is expressed as:

[0035]

[0036] where, P i (i = 5, 4, 3) respectively represent three feature maps of the information flow output from top to bottom, respectively represent the final output feature maps obtained from the bottom-up information flow, ⊕ represents the fusion operation, CSPLayer represents the cross-stage partial layer structure, and Downsample represents the downsampling operation;

[0037] Construct a multi-core attention network module. The multi-core attention network module includes an aggregation module, an extraction module, and a reconstruction module.

[0038] Set the aggregation module. The aggregation module uses a standard depthwise separable convolution with a kernel size of k×k, where k is the set kernel size;

[0039] Set the extraction module. The extraction module uses a multi-branch depthwise strip convolution layer. In each branch, a 1×j depthwise separable strip convolution and a j×1 depthwise separable strip convolution are used to simulate a j×j standard depthwise separable convolution, where the kernel sizes j in different branches are different;

[0040] Set the reconstruction module. The reconstruction module uses a 1×1 standard convolution to reconstruct the relationships between different channels;

[0041] First, the feature information from the dual-channel feature pyramid network outputs a local information feature map through the aggregation module. Secondly, the multi-scale features of the local information feature map are extracted through the extraction module to obtain multi-scale feature maps of different channels. Finally, the multi-scale feature maps reconstruct the relationships between different channels through the reconstruction module and output the final feature map;

[0042] The formula of the multi-core attention network module is expressed as:

[0043]

[0044] Among them, P refers to the input feature, Att and Out respectively represent the attention map and the output result, Conv 1×1 represents a 1×1 convolution operation, DW-Conv is the abbreviation of depthwise separable convolution, Scale i where i takes the values of 0, 1, 2, 3, Scale i when taking 1, 2, 3 represents the branch option, Scale0 represents the identity mapping.

[0045] The training of the small-scale pest dual-channel multi-core feature extraction model includes the following steps:

[0046] Set the bounding box regression loss function:

[0047] Define the bounding box regression loss function L of the small-scale pest dual-channel multi-core feature extraction model as:

[0048]

[0049] Among them, IoU represents the intersection over union of the areas of the predicted bounding box and the true bounding box, b and b gt respectively represent the center points of the predicted bounding box and the true bounding box, ρ 2 (b, b gt ) is the squared Euclidean distance between the center points of the predicted bounding box and the true bounding box, c is the diagonal length of the minimum enclosing rectangle of the predicted bounding box and the true bounding box, α is the weight coefficient used to balance the influence of the aspect ratio, v is a measure of the aspect ratio consistency between the predicted bounding box and the true bounding box, and the calculation formula is:

[0050]

[0051] w and h are the width and height of the predicted bounding box, w gt and h gt are the width and height of the true bounding box, and the formula for the weight coefficient α is:

[0052]

[0053] The formula for the intersection over union IoU is:

[0054]

[0055] bbox is the area size of the predicted bounding box, bbox gt is the area size of the true bounding box, ∩ represents the intersection operation, ∪ represents the union operation;

[0056] The multi-scale feature maps of pest images output from the cross-stage local layer structure are input into the dual-channel feature pyramid module for training.

[0057] According to the top-down information flow, first, the multi-scale feature maps of pest images in the top layer are processed by a two-dimensional convolution to obtain the feature map P5. P5 is then upsampled to expand the image size of the feature map. Subsequently, it is concatenated with the multi-scale feature maps of pest images in the middle layer and undergoes a two-dimensional convolution operation through the cross-stage local layer structure to obtain the feature map P4. P4 is then upsampled to expand the image size of the feature map, and then concatenated with the multi-scale feature maps of pest images in the lower layer to obtain the feature map P3 through the cross-stage local layer structure. According to the bottom-up information flow, first, the feature map P3 in the bottom layer is directly output to obtain the feature map At the same time, the feature map P3 is downsampled and concatenated with the feature map P4, and then passes through the cross-stage local layer structure to obtain the output feature map. Then, it is repeatedly downsampled and concatenated with the feature map P5, and passes through the cross-stage local layer structure to obtain the output feature map.

[0058] Each output feature map is then trained through the multi-core attention network module. First, it passes through a standard depthwise separable convolution, and then through three depthwise separable convolution branches with different convolution kernel sizes for multi-scale fusion with the original feature map to obtain the attention map Att. The attention map Att is then multiplied by the feature map P to obtain the final training output Out.

[0059] The output Out is used to calculate the center point of the pest target in the pest image through classification training, and the long and short axes of the pest target in the pest image are calculated through regression training. The pest target detection box is determined based on the center point, long and short axes of the pest target.

[0060] Through continuous training, according to the bounding box regression loss function L, the predicted bounding box is continuously made to approach the true bounding box, and the gradient backpropagation algorithm is used to adjust the weights of the model. Finally, the trained model learns to detect the pests in the pest image.

[0061] The construction of the dense pest area focusing extraction module includes the following steps:

[0062] Set the focusing factor focusfactor to determine the specified pixel size to be extracted.

[0063] According to the width w and height H of the input pest image, obtain the number of cutting row blocks and column blocks:

[0064]

[0065] where, num rows represents the number of row blocks, and num cols represents the number of column blocks.

[0066] Set the focusing extraction algorithm of the dense pest area focusing extraction module, and its expression is as follows:

[0067] x_start i = i × focusfactor (i = 1, 2, 3... focusfactor),

[0068] x_end i = min((i + 1) × focusfactor, H) (i = 1, 2, 3... focusfactor),

[0069] y_start j = j ×... focusfactor (j = 1, 2, 3... focusfactor),

[0070] y_end j = min((j + 1) × focusfactor, W) (j = 1, 2, 3... focusfactor)

[0071]

[0072] (i = 1, 2, 3... focusfactor, j = 1, 2, 3... focusfactor)

[0073] Among them, x_start i , x_end i , y_start j , y_enf j respectively represent the starting coordinate of the x-axis, the ending coordinate of the x-axis, the starting coordinate of the y-axis, and the ending coordinate of the y-axis. min represents taking the minimum value, and cropImage represents the focusing cropping operation according to the four coordinates of the image;

[0074] Judge whether there are pest instances in the focused area. If so, save the image and annotation information; if not, discard it.

[0075] The construction of the dense tiny pest image dataset includes the following steps:

[0076] Obtain pest images from the pest dataset;

[0077] Input the pest images into the dense pest area focusing extraction module to obtain several dense pest images according to the set focusing factor size;

[0078] Classify according to the pest density, and set the density level k i The formula is as follows:

[0079]

[0080] m represents the number of images, and n i represents the number of pests in the i-th image, and max(n1, n2…n i )(i = 1, 2, 3…m) represents the number of pests in the image with the largest number of pests among all images.

[0081] Set multiple intervals, and set a density level for each interval. If k i falls within a certain interval, the pest density level is the density level of that interval until the pest density level affects the setting of the focus factor focusfactor.

[0082] Mix the obtained dense pest images with the original pest dataset to form a dense tiny pest image dataset.

[0083] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a method for detecting dense tiny pest images based on a multi-core attention network can be implemented.

[0084] A computer device, characterized in that it includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, a method for detecting dense tiny pest images based on a multi-core attention network can be implemented.

[0085] Beneficial effects

[0086] The method for detecting dense tiny pest images based on a multi-core attention network of the present invention significantly improves the detection accuracy of dense tiny pest images compared with the prior art. The dense pest area focusing extraction module is used to extract the dense pest area, and the multi-scale features of pests are effectively extracted through the cross-stage local network. Combining the proposed small-scale pest two-channel multi-core feature extraction model can more accurately identify tiny pests, and accelerate the detection speed and accuracy of pest damage.

[0087] By constructing a dense pest area focusing extraction module, the present invention can accurately extract the dense pest area image from the pest images with complex backgrounds, generate a more targeted and adaptable dataset, and by grading the pest density level, the model can be optimized according to different pest density scenarios, effectively enhancing the detection ability in complex environments in actual agricultural production.

[0088] The present invention constructs a dual-channel multi-core feature extraction model for small-scale pests, which improves the efficiency and accuracy of pest detection. Through the top-down and bottom-up bidirectional information flow design of the dual-channel feature pyramid module, the efficient fusion of multi-scale features is achieved, reducing information loss and comprehensively enhancing the detection performance of the model for pests of different sizes. Through the multi-core attention network module, it can accurately capture richer and more detailed multi-scale features of pests, and at the same time optimize and integrate the extracted multi-scale features to output a more representative feature map, thereby more precisely identifying tiny pests. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 is the method sequence diagram of the present invention;

[0090] Figure 2 is the model structure diagram of the dual-channel multi-core feature extraction model for small-scale pests of the present invention;

[0091] Figure 3 is the model structure diagram of the multi-core attention network module of the present invention;

[0092] Figure 4 is the result diagram of pest disease images detected by the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0093] To further understand and recognize the structural features and achieved effects of the present invention, the following is a detailed description in conjunction with preferred embodiments and accompanying drawings:

[0094] As Figure 1 shown, a method for detecting dense tiny pest images based on a multi-core attention network according to the present invention includes the following steps:

[0095] The first step is to generate a dense tiny pest image dataset: Use the dense pest area focusing extraction module to focus on extracting the dense pest area to generate a dense tiny pest image dataset.

[0096] The collection process uses a specially designed collection device, which consists of a front macro lens camera, a mobile data transmission terminal, and a telescopic carbon fiber bracket. The pests collected include various pests, mostly common pests in the field. At the same time, to address the problem of the small number of rare pest species collected, data enhancement processing is performed on them, including data rotation and data flipping, to increase the proportion of rare pest species in the pest dataset. The annotation information includes the central point coordinates, width, height of the pest bounding box, and pest category. Pests with severe occlusion where the visible part is less than a certain ratio are not annotated, and the annotation files are saved in a special format to construct a pest dataset.

[0097] (1) Obtain wheat images damaged by pests, annotate and preprocess the images, and construct a pest dataset;

[0098] (2) Construct a dense pest area focusing extraction module.

[0099] By observing the pest dataset, it can be seen that the distribution of pests is often aggregated, possibly in clusters or bands, and scattered occurrences are less common. In addition, the background in the pest dataset often appears very complex, which brings many difficulties to pest detection. To reduce the influence of the cluttered background in the detection process and enable the model to focus more on the pests to be detected, thereby improving the detection accuracy. Therefore, it is necessary to construct a dense pest area focusing extraction module to solve such problems.

[0100] The construction of the dense pest area focusing extraction module includes the following steps:

[0101] A1) Set the focusing factor focusfactor to determine the specified pixel size to be extracted;

[0102] A2) According to the width W and height H of the input pest image, obtain the number of cutting row blocks and column blocks:

[0103]

[0104] where, num rows represents the number of row blocks, num cols represents the number of column blocks,

[0105] A3) Set the focusing extraction algorithm of the dense pest area focusing extraction module, and its expression is as follows:

[0106] x_start i = i × focusfactor (i = 1, 2, 3... focusfactor),

[0107] x_end i = min((i + 1) × focusfactor, H) (i = 1, 2, 3... focusfactor),

[0108] y_start j = j × focusfactor (j = 1, 2, 3... focusfactor),

[0109] y_end j = min((j + 1) × focusfactor, W) (j = 1, 2, 3... focusfactor)

[0110] image_patch = cropImage(x_starti , y_start j , x_end i , y_end j )(i

[0111] = 1, 2, 3…focusfactor, j = 1, 2, 3…focusfactor)

[0112] Among them, x_start i , x_end i , y_start j , y_enf j respectively represent the starting coordinate of the x-axis, the ending coordinate of the x-axis, the starting coordinate of the y-axis, and the ending coordinate of the y-axis. min represents taking the minimum value, and cropImage represents the focusing and cropping operation of the image according to the four coordinates.

[0113] A4) Determine whether there are pest instances in the focused area. If so, save the image and annotation information; if not, discard it.

[0114] (3) Construct a dense and tiny pest image dataset: Input the pest dataset into the dense pest area focusing and extraction module to extract the images of the dense pest area, and mix the images of the dense pest area with the pest dataset to form a dense and tiny pest image dataset. Since the denser the pest distribution and the smaller the pest scale, the greater the detection difficulty. To strengthen the solution to the problems of pest density and small scale, it is necessary to reconstruct this pest dataset. First, obtain pest images from the pest dataset. The pest images are input into the dense pest area focusing and extraction module to obtain a series of dense pest images according to the set focusing factor size. Mix the obtained dense pest images with the original pest dataset to form a dense and tiny pest image dataset. The reconstructed dense and tiny pest image dataset can well strengthen the detection of small-scale and dense pests.

[0115] The construction of the dense and tiny pest image dataset includes the following steps:

[0116] B1) Obtain pest images from the pest dataset;

[0117] B2) Input the pest images into the dense pest area focusing and extraction module to obtain several dense pest images according to the set focusing factor size;

[0118] B3) Classify according to the pest density level, and set the density level k i The formula is as follows:

[0119]

[0120] m represents the number of images, n irepresents the number of pests in the i-th image, and max(n1, n2…n i )(i = 1, 2, 3…m) represents the number of pests in the image with the largest number of pests among all images,

[0121] Set multiple intervals, set a density level for each interval. If k i falls within a certain interval, the pest density level is the density level of that interval until the pest density level affects the setting of the focus factor focusfactor;

[0122] B4) Mix the obtained dense pest images with the original pest dataset to form a dense tiny pest image dataset.

[0123] The second step is the extraction of multi-scale features of pest images: Extract multi-scale features by the Cross-Stage Partial Network. During the design process, it is necessary to ensure that the network can effectively extract multi-scale features and improve the detection accuracy, and at the same time control the computational amount to avoid excessive consumption of computing resources and low operating efficiency due to an overly complex structure. Therefore, a Cross-Stage Partial Network is constructed to extract multi-scale features.

[0124] (1) Construct a Cross-Stage Partial Network. The Cross-Stage Partial Network includes a focus structure, a convolution normalization activation structure, and a Cross-Stage Partial layer structure. The Cross-Stage Partial layer structure decomposes the residual block stack and introduces a Spatial Pyramid Pooling structure before the last Cross-Stage Partial layer structure to extract features;

[0125] The input image obtained from the dense tiny pest image dataset first enters the focus structure for slicing operations to generate multiple low-resolution feature maps and stack them to quadruple the number of input channels. Then, the multi-channel feature maps are concatenated and stacked together, and are cross-processed by the convolution normalization activation structure and the Cross-Stage Partial layer structure to enhance the small-scale pest features, obtaining a small-scale pest feature map.

[0126] (2) The focus structure extracts the pixel value every other pixel from the input image. In this way, four independent feature layers are generated, and then these four feature layers are stacked together to integrate the width and height information of the image into the channel dimension, thereby quadrupling the number of input channels.

[0127] (3) The convolution normalization activation structure adds a batch normalization layer and a SiLU activation function after the convolution layer. The formula of the SiLU activation function f(x) is:

[0128] f(x) = x·sigmoid(x)

[0129] Among them, the formula of the sigmoid function is expressed as:

[0130]

[0131] Among them, x represents the input value, and e represents the natural constant.

[0132] The batch normalization layer can ensure that when the network depth increases, the training speed is not affected, accelerating the model convergence. The SiLU activation function enables the network to better learn image features. In this process, different-scale features in the feature map are further extracted and strengthened. Some tiny detail features and features in larger regions are enhanced in expression, and the differences between different-scale features become more obvious.

[0133] (4) The cross-stage local layer structure is divided into two parts: one part directly performs forward propagation, retaining relatively original feature information, including basic features at different scales; the other part undergoes multiple repeated 1×1 basic convolutions and 3×3 depthwise separable convolution operations to further extract and refine features, mining more representative features at different scales. The information of these two parts is merged at the end of the network, enabling the fusion of features at different levels and scales, used to enrich the diversity of features, forming a feature map containing multi-scale information, and obtaining the multi-scale feature map of the pest image. This fusion not only integrates the original features and the processed features but also combines the features obtained under different receptive fields, thus more comprehensively covering the multi-scale information in the image.

[0134] The third step, as Figure 2 shown, construct a small-scale pest two-channel multi-core feature extraction model.

[0135] (1) Set that the small-scale pest two-channel multi-core feature extraction model includes a two-channel feature pyramid module and a multi-core attention network module.

[0136] (2) Set the two-channel feature pyramid module, which consists of a top-down information flow and a bottom-up information flow;

[0137] Among them, the top-down information flow: starts from the higher layer of the network and gradually transmits information to the lower layer. First, upsample the higher-level feature map, and then fuse it with the feature map of the middle layer or the bottom layer to enhance the semantic information in the lower-level feature map. This fusion helps to supplement fine details and improve the ability to capture small objects and fine features. The formula is expressed as:

[0138] P5 = Conv(f3)

[0139] P4 = Conv(CSPLayer(UpSample(P5) ⊕ f2))

[0140] P3 = CSPLayer(UpSample(P4) ⊕ f1)

[0141] Among them, f i (i = 3, 2, 1) respectively represent three feature maps of the information flow input from top to bottom, P i (i = 5, 4, 3) respectively represent three feature maps of the information flow output from top to bottom, ⊕ represents the fusion operation, Conv represents the convolution operation, CSPLayer represents the cross-stage partial layer structure, UpSample represents the upsampling operation;

[0142] Bottom-up information flow: Starting from the lower-level feature maps, gradually transfer the information to the higher-level feature maps. In the bottom-up information flow, perform downsampling operations on the lower-level feature maps and fuse them with the higher-level feature maps. The formula is expressed as:

[0143]

[0144] Among them, P i (i = 5, 4, 3) respectively represent three feature maps of the information flow output from top to bottom, respectively represent the final output feature maps obtained from the bottom-up information flow, ⊕ represents the fusion operation, CSPLayer represents the cross-stage partial layer structure, Downsample represents the downsampling operation.

[0145] (3) As Figure 3 shown, construct a multi-core attention network module. The multi-core attention network module includes an aggregation module, an extraction module, and a reconstruction module,

[0146] Set the aggregation module. The aggregation module uses a standard depthwise separable convolution with a kernel size of k×k. Among them, k is the set kernel size;

[0147] Set the extraction module. The extraction module uses a multi-branch depthwise strip convolution layer. In each branch, use a depthwise separable strip convolution of 1×j and a depthwise separable strip convolution of j×1 to simulate a standard depthwise separable convolution of j×j. Among them, the kernel size j in different branches is different;

[0148] Set the reconstruction module. The reconstruction module uses a standard convolution of 1×1 to reconstruct the relationship between different channels;

[0149] First, the feature information from the dual-channel feature pyramid network outputs a local information feature map through the aggregation module. Secondly, extract the multi-scale features of the local information feature map through the extraction module to obtain multi-scale feature maps of different channels. Finally, the multi-scale feature maps reconstruct the relationship between different channels through the reconstruction module and output the final feature map;

[0150] The formula of the multi-core attention network module is expressed as:

[0151]

[0152] Among them, P refers to the input feature, Att and Out represent the attention map and the output result respectively, and Conv 1×1 represents a 1×1 convolution operation, DW-Conv is the abbreviation of depthwise separable convolution, Scale i where i takes the values of 0, 1, 2, 3 in Scale i When Scale takes 1, 2, 3, it represents the branch option, and Scale0 represents the identity mapping.

[0153] Step 4, training of the small-scale pest two-channel multi-core feature extraction model: Input the dense tiny pest image dataset into the small-scale pest two-channel multi-core feature extraction model for training.

[0154] The training of the small-scale pest two-channel multi-core feature extraction model includes the following steps:

[0155] (1) Set the bounding box regression loss function:

[0156] Define the bounding box regression loss function L of the small-scale pest two-channel multi-core feature extraction model as:

[0157]

[0158] Among them, IoU represents the intersection over union of the predicted bounding box and the ground truth bounding box in terms of area size, b and b gt represent the center points of the predicted bounding box and the ground truth bounding box respectively, ρ 2 (b, b gt ) is the squared Euclidean distance between the center points of the predicted bounding box and the ground truth bounding box, c is the diagonal length of the smallest enclosing rectangle of the predicted bounding box and the ground truth bounding box, α is the weight coefficient used to balance the influence of the aspect ratio, v is the measure of the aspect ratio consistency between the predicted bounding box and the ground truth bounding box, and the calculation formula is:

[0159]

[0160] w and h are the width and height of the predicted bounding box, w gt and h gt are the width and height of the ground truth bounding box, and the formula for the weight coefficient α is:

[0161]

[0162] The formula for the intersection over union IoU is:

[0163]

[0164] bbox is the area size of the predicted bounding box, bbox gtis the area size of the real bounding box, ∩ represents the intersection operation, and ∪ represents the union operation.

[0165] (2) The multi-scale feature maps of pest images from the cross-stage local layer structure are input into the dual-channel feature pyramid module for training.

[0166] According to the top-down information flow, first, the multi-scale feature maps of pest images in the top layer are processed by a two-dimensional convolution to obtain the feature map P5. P5 is then upsampled to expand the image size of the feature map, and then concatenated with the multi-scale feature maps of pest images in the middle layer. Through the cross-stage local layer structure, a two-dimensional convolution operation is performed to obtain the feature map P4. P4 is then upsampled to expand the image size of the feature map, and then concatenated with the multi-scale feature maps of pest images in the lower layer. Through the cross-stage local layer structure, the feature map P3 is obtained. According to the bottom-up information flow, first, the feature map P3 in the bottom layer is directly output to obtain the feature map At the same time, the feature map P3 is downsampled and concatenated with the feature map P4, and through the cross-stage local layer structure, the output feature map is obtained. Repeat the process of downsampling and concatenating with the feature map P5, and through the cross-stage local layer structure, the output feature map is obtained.

[0167] (3) Each output feature map is further trained by the multi-core attention network module. First, through a standard depthwise separable convolution, and then through three depthwise separable convolution branches with different convolution kernel sizes, multi-scale fusion is performed with the original feature map to obtain the attention map Att. The attention map Att is then multiplied by the feature map P to obtain the final training output Out.

[0168] The output Out is used to calculate the center point of the pest target in the pest image through classification training, and the long and short axes of the pest target in the pest image are calculated through regression training. According to the center point, long and short axes of the pest target, the detection box of the pest target is determined.

[0169] (4) Through continuous training, according to the bounding box regression loss function L, the predicted bounding box is continuously made to approach the real bounding box, and the gradient backpropagation algorithm is used to adjust the weights of the model. Finally, the trained model learns to detect the pests in the pest image.

[0170] Step 5, obtaining the detection results of dense tiny pest images: Obtain the pest image to be detected and input it into the cross-stage local network to obtain the multi-scale feature maps of the pest image. The multi-scale feature maps are input into the trained small-scale pest dual-channel multi-core feature extraction model to obtain the pest image detection results.

[0171] Here, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting dense tiny pest images based on a multi-core attention network can be implemented. A computer device includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes, the method for detecting dense tiny pest images based on a multi-core attention network can be implemented.

[0172] As Figure 4 shown, the method of the present invention detects pests in 4 randomly selected pest images, and it can be seen that each pest can be accurately detected, indicating that the method of the present invention has good detection accuracy for dense small-scale pests in complex environments.

[0173] Table 1 Comparison table of the method of the present invention and the most advanced detection method on the dense tiny pest dataset

[0174]

[0175]

[0176] In Table 1, the best results are shown in bold. As can be seen from Table 1, compared with the detection results of other methods, the method of the present invention can greatly improve the detection accuracy, far exceeding other models in the average accuracy of pests. At the same time, the number of parameters of the model is also lightweight and compact, which proves that the present invention can be applied to actual agricultural scenarios.

[0177] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art of this industry should understand that the present invention is not limited by the above embodiments. The above embodiments and the descriptions in the specification are only the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection required by the present invention is defined by the appended claims and their equivalents.

Claims

1. A dense micro-pest image detection method based on a multi-core attention network, characterized in that: The following steps are involved: 11) Generation of a dense micro-pest image dataset: Using the dense pest region focus extraction module, focus extraction is performed on dense pest regions to generate a dense micro-pest image dataset; 12) Extraction of multi-scale features of pest images: Multi-scale features are extracted by a cross-stage local network; 13) Construct a dual-channel multi-core feature extraction model for small-scale pests; The construction of a small-scale pest dual-channel multi-core feature extraction model comprises the following steps: 131) Setting a small-scale pest dual-channel multi-core feature extraction model including a dual-channel feature pyramid module and a multi-core attention network module; 132) Setting a dual-channel feature pyramid module, the dual-channel feature pyramid module consists of a top-down information flow and a bottom-up information flow; Among them, the top-down information flow starts from the higher layers of the network and gradually transfers information to the lower layers. First, the higher-level feature maps are upsampled and then fused with the feature maps of the middle or bottom layers to enhance the semantic information in the lower-level feature maps. The formula is expressed as: P5=Conv(f3) Among them, f i Represents the three feature maps of top-down information flow input, P i Represent the three feature maps output by the top-down information flow, Represents a fusion operation, Conv represents a convolution operation, CSPLayer represents a cross-stage local layer structure, and UpSample represents an upsampling operation; Bottom-up information flow: Starting from the lower-level feature map, information is gradually passed to the higher-level feature map. In the bottom-up information flow, the lower-level feature map is downsampled and fused with the higher-level feature map. The formula is expressed as: Among them, P i Represent the three feature maps output by the top-down information flow, Represent the final output feature map obtained from the bottom-up information flow, represents the fusion operation, CSPLayer represents the cross-stage local layer structure, and Downsample represents the downsampling operation; 133) Construct a multi-core attention network module, which includes an aggregation module, an extraction module, and a reconstruction module. Set the aggregation module, which uses a standard depth-wise separable convolution with a kernel size of k×k, where k is the set kernel size; Set up the extraction module. The extraction module uses a multi-branch depthwise strip convolution layer. In each branch, a 1×j depthwise separable strip convolution and a j×1 depthwise separable strip convolution are used to simulate a j×j standard depthwise separable convolution. The kernel size j in different branches is different. Set up the reconstruction module, which uses a 1×1 standard convolution to reconstruct the relationship between different channels; First, the feature information from the dual-channel feature pyramid network is aggregated to output a local information feature map. Second, the extraction module extracts multi-scale features from the local information feature map to obtain multi-scale feature maps of different channels. Finally, the multi-scale feature map is reconstructed from the relationship between different channels through the reconstruction module to output the final feature map. The multi-core attention network module formula is expressed as: Among them, P refers to the input feature, Att and Out represent the attention map and output results respectively, Conv 1×1 Represents a 1×1 convolution operation, DW-Conv is the abbreviation of depth-separable convolution, Scale i Where i takes the value of 0, 1, 2, or 3, and Scale i When 1, 2, or 3 is selected, it indicates branch options, and Scale 0 indicates identity mapping; 14) Training of a small-scale pest dual-channel multi-core feature extraction model: A dense dataset of tiny pest images is fed into a small-scale pest dual-channel multi-core feature extraction model for training; 15) Obtaining dense micro-pest image detection results: Obtain the pest image to be detected and input it into the cross-stage local network to obtain a multi-scale feature map of the pest image. The multi-scale feature map of the pest image is input into the trained small-scale pest dual-channel multi-core feature extraction model to obtain the pest image detection results.

2. The method for detecting dense tiny pest images based on a multi-core attention network according to claim 1, characterized in that: The generation of the dense tiny pest image dataset includes the following steps: 21) Obtain images of wheat infested by pests, annotate and preprocess the images, and construct a pest dataset; 22) Construct a focused extraction module for dense pest areas; 23) Constructing a dense micro-pest image dataset: Input the pest dataset into the dense pest region focus extraction module to extract dense pest region images, and then mix the dense pest region images with the pest dataset to form a dense micro-pest image dataset.

3. The method for detecting dense tiny pest images based on a multi-core attention network according to claim 1, characterized in that: The extraction of multi-scale features of the pest image comprises the following steps: 31) Constructing a cross-stage local network, which includes a focusing structure, a convolutional normalized activation structure, and a cross-stage local layer structure. The cross-stage local layer structure decomposes the residual block stack and introduces a spatial pyramid pooling structure before the last cross-stage local layer structure to extract features; The input image obtained from a dense dataset of tiny pest images first enters a focusing structure for slicing operations to generate multiple low-resolution feature maps, which are then stacked to quadruple the number of input channels. The multi-channel feature maps are then spliced, combined, and stacked together. The convolutional normalization activation structure and the cross-stage local layer structure are then cross-processed to enhance the small-scale pest features, resulting in a small-scale feature map of the pests. 32) The focusing structure extracts the pixel value of every other pixel from the input image, generating four independent feature layers in this way. These four feature layers are then stacked together to integrate the width and height information of the image into the channel dimension, thereby quadrupling the number of input channels. 33) Convolutional normalization activation structure adds a batch normalization layer and SiLU activation function after the convolution layer. The formula of SiLU activation function f(x) is: f(x)=x·sigmoid(x) Among them, the formula of the sigmoid function is expressed as: Among them, x represents the input value, and e represents the natural constant; 34) The cross-stage local layer structure is divided into two parts: one part directly performs forward propagation to retain relatively original feature information, including basic features at different scales; the other part undergoes multiple repeated 1×1 basic convolution and 3×3 depthwise separable convolution operations to further extract and refine features and mine more representative features at different scales. These two parts of information are merged at the end of the network, allowing features at different levels and scales to be integrated to enrich the diversity of features, form a feature map containing multi-scale information, and obtain a multi-scale feature map of pest images.

4. The method for detecting dense tiny pest images based on a multi-core attention network according to claim 1, characterized in that: The training of the small-scale pest dual-channel multi-core feature extraction model includes the following steps: 41) Set the bounding box regression loss function: The bounding box regression loss function L of the small-scale pest dual-channel multi-kernel feature extraction model is defined as: Among them, IoU represents the intersection-over-union ratio of the predicted bounding box and the true bounding box area, b and b gt denote the center points of the predicted bounding box and the true bounding box, respectively, and ρ 2 (b,b gt ) is the squared Euclidean distance between the center points of the predicted bounding box and the true bounding box, c is the diagonal length of the minimum enclosing rectangle of the predicted bounding box and the true bounding box, α is the weight coefficient used to balance the influence of aspect ratio, and v is a measure of the consistency of aspect ratio between the predicted bounding box and the true bounding box, calculated as: w and h are the width and height of the predicted bounding box, w gt and h gt is the width and height of the true bounding box, and the formula for the weight coefficient α is: The formula for intersection over union (IoU) is: bbox is the area size of the predicted bounding box, bbox gt is the area of the true bounding box, ∩ represents the intersection operation, and ∪ represents the union operation; 42) The multi-scale feature map of the pest image from the cross-stage local layer structure is input into the dual-channel feature pyramid module for training. According to the top-down information flow, the multi-scale feature map of the pest image in the top layer is first processed by a two-dimensional convolution to obtain the feature map P5. P5 is then upsampled to expand the image size of the feature map, and then it is spliced with the multi-scale feature map of the pest image in the middle layer to obtain the feature map P4 through a two-dimensional convolution operation through a cross-stage local layer structure. P4 is then upsampled to expand the image size of the feature map, and then it is spliced with the multi-scale feature map of the pest image in the lower layer to obtain the feature map P3 through a cross-stage local layer structure. According to the bottom-up information flow, the feature map P3 at the bottom layer is first directly output to obtain the feature map At the same time, the feature map P3 is downsampled and spliced with the feature map P4 through the cross-stage local layer structure to obtain the output feature map Then repeatedly downsample and splice the feature map P5 across the local layer structure to obtain the output feature map 43) Each output feature map After training with the multi-core attention network module, it first passes through a standard depthwise separable convolution, and then performs multi-scale fusion with the original feature map through three depthwise separable convolution branches with different convolution kernel sizes to obtain the attention map Att. The attention map Att is then multiplied with the feature map P to obtain the final training output Out; Output Out is calculated through classification training to obtain the center point of the pest target in the pest image, and through regression training to calculate the major and minor axes of the pest target in the pest image. The pest target detection frame is determined based on the center point and major and minor axes of the pest target; 44) Through continuous training, the predicted bounding box is continuously made close to the true bounding box according to the bounding box regression loss function L, and the gradient back propagation algorithm is used to adjust the weight of the model. Finally, the trained model learns to detect pests in pest images.

5. The method for detecting dense tiny pest images based on a multi-core attention network according to claim 2, characterized in that: The construction of the concentrated pest area focus extraction module comprises the following steps: 51) Setting the focus factor to determine the size of the specified pixel to be extracted; 52) According to the width W and height H of the input pest image, obtain the number of cutting row blocks and column blocks: Among them, num rows Indicates the number of row blocks, num cols Indicates the number of column blocks, 53) Set the focus extraction algorithm of the dense pest area focus extraction module, and its expression is as follows: x_start i =i×focusfactor, x_end i =min((i+1)×focusfactor,H), y_start j =j×focusfactor, y endj =min((d+1)×focusfactor,W), image_patch=cropImage(x_start i ,y_start j ,x_end i ,y_end j ), Among them, x_start i 、x_end i 、y_start j 、y_end j They represent the x-axis starting point coordinate, x-axis end point coordinate, y-axis starting point coordinate, and y-axis end point coordinate respectively. min means taking the minimum number. cropImage means focusing and cropping the image based on the four coordinates. 54) Determine whether there is a pest instance in the focused area. If yes, save the image and annotation information; otherwise, discard it.

6. The method for detecting dense tiny pest images based on a multi-core attention network according to claim 2, characterized in that: The construction of a dense tiny pest image dataset comprises the following steps: 61) Obtain pest images from the pest dataset; 62) Inputting the pest image into the dense pest area focus extraction module to obtain a number of dense pest images according to the set focus factor size; 63) According to the density of pests, set the density level k i The formula is as follows: m represents the number of images, n i Indicates the number of pests in the i-th image, max(n1,n2…n i ) means taking the number of pests in the picture with the largest number of pests among all images, Set multiple intervals and set a density level for each interval. If k i If it falls within a certain interval, the pest density is the density level of the interval, until the pest density affects the setting of the focus factor; 64) The obtained dense pest images are mixed with the original pest dataset to form a dense tiny pest image dataset.

7. A computer-readable storage medium, characterized in that The storage medium stores a computer program. When the computer program is executed by the processor, the dense tiny pest image detection method based on the multi-core attention network described in any one of claims 1 to 6 can be implemented.

8. A computer device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, a dense tiny pest image detection method based on a multi-core attention network as described in any one of claims 1 to 6 can be implemented.

Images