Crop pest identification method and system based on BiFPN and multi-scale attention mechanism

Abstract

The invention relates to the technical field of target identification, in particular to a BiFPN and multi-scale attention mechanism-based crop pest identification method and system, and the method comprises the steps: obtaining an agricultural insect image data set covering multiple scenes, carrying out the division and multi-strategy data enhancement processing of the data set, and generating a training set, a verification set and a test set, through staged cooperation of a convolution unit and a self-attention unit, multi-scale initial features containing local details and global semantics are output, an improved space attention mechanism and a channel attention mechanism are applied to the fused multi-scale features, the enhanced multi-scale features are input into a fine-grained classification module, and a fine-grained classification result is obtained. Through discriminative feature selection and feature aggregation, the model is trained and optimized by adopting a triple collaborative loss function, and by screening high-discriminative feature points and aggregating feature association information through a graph convolutional network to strengthen inter-class differences, various types of related pests can be accurately distinguished.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of target recognition, in particular to a crop pest recognition method and system based on a BiFPN and a multi-scale attention mechanism. BACKGROUND

[0002] Agriculture is a core industry for ensuring food security and social and economic stability, and pest damage is a key factor leading to reduced yield and deteriorated quality of crops. Timely and accurate differentiation between ordinary insects and pests in farmland and targeted prevention and treatment have become an important link in ensuring agricultural production. Traditional pest identification relies on manual inspection and expert experience, and has problems such as low efficiency, high cost, scarcity of expert resources, and delayed response, which cannot meet the real-time monitoring needs of large-scale farmland. Therefore, intelligent pest identification technology based on deep learning has become a research focus.

[0003] In existing deep learning pest identification technology, various technical solutions have appeared, such as using a PestNAS model to fuse pest and plant context features through an adaptive feature fusion module and a feature pyramid mechanism. However, most of them use traditional FPN for multi-scale feature fusion. Such FPN only supports one-way feature transmission from top to bottom, and the interactive fusion of shallow features and deep features is insufficient, making it difficult for the model to simultaneously capture the details of small-scale pests and represent the semantics of large-scale pests. At the same time, traditional FPN lacks a dynamic weight adjustment mechanism, and uses fixed weights to fuse features of different scales, which cannot automatically allocate weights according to the amount of feature information, easily causing redundant feature accumulation, and thus reducing the model's ability to distinguish pests of different scales. In addition, traditional spatial attention mechanisms usually use single-scale pooling processing, which cannot effectively cover the spatial distribution range of pests of different sizes, and the spatial positioning accuracy of small-scale pests is insufficient. Channel attention mechanisms usually use fixed compression rates to filter channel features, which cannot model both fine-grained and coarse-grained channel information, resulting in inaccurate key channel feature filtering. In actual farmland environments, pest images have complex backgrounds, and it is difficult to accurately focus on the pest area, thus reducing the recognition accuracy. At present, there is a need for a crop pest recognition method and system based on BiFPN and multi-scale attention mechanism. SUMMARY

[0004] To solve the problems of low recognition accuracy and poor adaptability to complex backgrounds in traditional crop pest identification, the application provides a crop pest recognition method and system based on BiFPN and multi-scale attention mechanism.

[0005] In a first aspect, the application provides a crop pest recognition method based on BiFPN and multi-scale attention mechanism, which adopts the following technical solution: A crop pest recognition method based on BiFPN and multi-scale attention mechanism, comprising: S1. Obtain an image dataset of agricultural insects covering multiple scenarios, and perform data partitioning and multi-strategy data augmentation on the dataset to generate training, validation and test sets; S2. Using the obtained dataset as input, construct a CNN-Transformer hybrid backbone network. Through the phased collaboration of convolutional units and self-attention units, output multi-scale initial features containing local details and global semantics. S3. The initial multi-scale features are input into the BiFPN feature fusion module, and the fused multi-scale features are obtained through bidirectional feature transfer and dynamic weighted fusion. S4. Apply improved spatial attention mechanism and channel attention mechanism to the fused multi-scale features respectively to obtain enhanced multi-scale features; S5. Input the enhanced multi-scale features into the fine-grained classification module to complete the construction of the BiFPN-ConFormer model. Through discriminative feature selection and feature aggregation, output the pest category prediction results. S6. The BiFPN-ConFormer model is trained and optimized using a triple collaborative loss function. The model parameters are adjusted using the validation set, and the model performance is evaluated using the test set to complete the identification of crop pests.

[0006] Furthermore, the process of performing partitioning and multi-strategy data augmentation on the dataset specifically includes: S11. Randomly crop the obtained original image dataset, use bilinear interpolation to scale the cropped local image to a uniform pixel size, perform random horizontal flipping on the cropped image dataset, and generate random probability values ​​in the range of 0-1. S12. Perform Mixup enhancement on the flipped image dataset, and randomly select two insect images from the same batch. and Mixing coefficients are sampled from a uniform distribution in [0,1]. The pixel matrix and label vector of the two images are linearly weighted and mixed to generate a new image x and a new label y; S13. Finally, the image dataset is enhanced with CutMix, and two insect images are randomly selected. and Sampling mixing coefficient And calculate the bounding box parameters of the rectangular clipping region, construct the binary mask matrix M, and then... Replace cropped area pixels with corresponding region pixels, by The new samples are generated by proportionally mixing labels, and the expression for generating the new samples is: , in, It is a binary mask. To multiply pixel by pixel, The proportion of combinations sampled from a uniform distribution (0,1). and For the combined features and labels, For image Corresponding tags For image Corresponding tags.

[0007] Furthermore, the construction of the CNN-Transformer hybrid backbone network includes a phased serial fusion architecture. The first and second stages are local feature extraction stages, both constructed using depthwise separable convolutional units. Each depthwise separable convolutional unit consists of a depthwise convolutional layer and a pointwise convolutional layer. The depthwise convolutional layer independently performs spatial convolution on each channel of the input feature map to extract local features, while the pointwise convolutional layer fuses multi-channel features through 1×1 convolution. The third stage is the global semantic feature extraction stage, in which a Transformer module is deployed. The Transformer module uses a multi-head self-attention mechanism to flatten the feature map output from the second stage into a feature sequence. By calculating the association weights between feature points, a feature semantic association is established across the entire image.

[0008] Further, step S3 specifically includes the following sub-steps: S31. Take the multi-scale initial features as input. The multi-scale initial features contain at least three feature maps of different resolutions, each corresponding to shallow local detail features. Mid-layer transition characteristics and deep global semantic features ; S32. Perform top-down feature transfer and fusion, starting from the deepest high semantic features. Initially, its resolution is upsampled to match the intermediate transition features using a bilinear interpolation algorithm. Consistent, the upsampled deep features are compared with Input the dynamic weighted fusion unit, and fuse according to the weight ratio to obtain the first-stage fusion features. Similarly, After upsampling and The fusion yields the first stage of fusion characteristics. ; S33. Based on top-down fusion characteristics, max pooling is used to... Resolution downsampling to Features are consistent, and the downsampled shallow fusion features are combined with Input the dynamic weighted fusion unit to obtain the second-stage fusion features. Similarly, After downsampling fusion, get the second stage fusion feature ; S34, integration , and , get the fused multi-scale feature, the expression of the dynamic weighted fusion unit is: , wherein, is the output feature after fusion, is the i-th input feature, is the corresponding learnable weight, is the minimum value to avoid the denominator being zero.

[0009] Further, the improved spatial attention mechanism includes performing adaptive average pooling of multiple scales on the input feature , compressing the channel number from C to 1 through 1x1 convolution, generating multiple single-channel attention maps, performing upsampling operation on the single-channel attention map using bilinear interpolation algorithm, obtaining the spatially aligned multi-scale attention map, performing concatenation operation on the multi-scale attention map in the channel dimension, and performing 3x3 convolution operation on the concatenated attention map to generate a fusion attention map, applying Sigmoid activation function to the fusion attention map to constrain the attention weight value to the interval [0, 1], and obtaining the final spatial attention map, the expression of the spatial attention map is: , wherein, is the Sigmoid activation function, is the convolution operation, is the adaptive average pooling operation with scale , is the concatenation operation of the multi-scale attention map, is the upsampling operation, C is the channel number, H is the height of the feature map, and W is the width of the feature map.

[0010] Further, the channel attention mechanism comprises: obtaining single-level input features in the fused multi-scale features, setting a multi-dimension reduction ratio set and constructing corresponding parallel dimension reduction processing branches, for each branch, performing adaptive average pooling on the input features to extract global context, performing channel dimension reduction on the input features through convolution operation, applying layer normalization and ReLU activation function to the dimension-reduced feature map in sequence to enhance nonlinearity, then restoring the channel number through 1*1 convolution to generate single-granularity channel features, concatenating the single-granularity channel features of each branch in the channel dimension, performing fusion and dimension compression on the concatenated features through 1*1 convolution, and then applying a Sigmoid activation function to generate final channel attention weights.

[0011] Further, the input of the reinforced multi-scale features into the fine-grained classification module comprises: calculating the classification probability of each feature point in the reinforced multi-scale features through a fully connected layer and a softmax function, arranging the feature points in descending order of probability value, selecting a fixed number of high-probability points as discriminative feature points at different levels, regarding all selected discriminative feature points as nodes in a graph structure, constructing a graph , aggregating node features through a graph convolution network, aggregating node features output by the graph convolution network into a plurality of super nodes through a pooling operation, performing average pooling on the super nodes, and outputting a final classification result through a linear classifier, wherein the inter-layer propagation expression of the graph convolution network is: , wherein V is a node set, is an edge set, is the lth layer node feature, is a corresponding degree matrix, is a trainable weight of the lth layer graph convolution, is a graph adjacency matrix with self-connection.

[0012] Further, the training and optimization of the BiFPN-ConFormer model using the triple synergistic loss function comprises: S61, performing global average pooling operation on the multi-scale initial feature maps output by each level of the CNN-Transformer hybrid backbone network respectively to obtain the class prediction scores corresponding to the features of each level, calculating the classification loss of each level feature using a cross-entropy loss function, summing all level classification losses to obtain a feature extraction loss; S62, calculating the class prediction scores of the feature points that are not selected as discriminative feature points in the fine-grained classification module to obtain the discard region prediction scores, calculating the classification loss of each level and each class discard region using a cross-entropy loss function, summing all losses to obtain a weak selection loss; S63, performing an average pooling operation on the super node features output by the fine-grained classification module to obtain a global feature vector, inputting the global feature vector into a linear classifier to obtain a class prediction probability of each node, calculating the difference between the prediction probability and the true probability using a soft target cross-entropy loss function, and obtaining a graph combination loss by taking the average value; S64, assigning a weight coefficient to each loss, and obtaining a triple synergistic total loss by weighted summation.

[0013] Further, the model performance is evaluated by the test set, including inputting the training set into the BiFPN-ConFormer model, calculating the prediction result of the model and the triple synergistic total loss by forward propagation, calculating the gradient of the loss with respect to each model parameter by back propagation, updating the parameters using the AdamW optimizer, minimizing the total loss, after the training is completed, loading the saved temporary optimal model, inputting the test set into the model to obtain the prediction result, and using accuracy, precision, recall and weighted F1-score as evaluation indicators, and the evaluation expression of the F1-score is: , wherein Z is the number of categories, is the i-th F1-score, is the sample proportion.

[0014] In a second aspect, a crop pest identification system based on BiFPN and multi-scale attention mechanism includes: A data acquisition module configured to acquire an agricultural insect image dataset covering multiple scenes, and perform division and multi-strategy data augmentation processing on the dataset to generate a training set, a validation set and a test set; A model module configured to construct a CNN-Transformer hybrid backbone network using the acquired dataset as input, and output multi-scale initial features containing local details and global semantics through phased cooperation of convolution units and self-attention units; A conversion module configured to input the multi-scale initial features into a BiFPN feature fusion module to obtain fused multi-scale features through bidirectional feature transmission and dynamic weighted fusion; A reinforcement module configured to apply an improved spatial attention mechanism and a channel attention mechanism to the fused multi-scale features, respectively, to obtain reinforced multi-scale features; An output module configured to input the reinforced multi-scale features into a fine-grained classification module to complete the construction of a BiFPN-ConFormer model, and output pest class prediction results through discriminative feature selection and feature aggregation; The optimization module is configured to train and optimize the BiFPN-ConFormer model by using a triple synergy loss function, adjust model parameters by using a verification set, and evaluate model performance by using a test set to complete crop pest identification.

[0015] In summary, the present application has the following beneficial technical effects: 1. The multi-strategy data enhancement scheme composed of random cropping, random horizontal flipping, Mixup enhancement and CutMix enhancement is used to process the agricultural insect image dataset, wherein the random cropping simulates the different position distribution of pests in the image, the random horizontal flipping breaks the dependence of the model on the posture direction of the pests, the Mixup enhancement generates transitional new samples by linearly weighting the pixels and labels of two images to enhance the robustness of the model to subtle differences between samples, and the CutMix enhancement replaces the local area of one image with the corresponding area of another image, and mixes the labels according to the area proportion of the replaced area, thereby expanding the diversity of samples while preserving the integrity of the local features of insects.

[0016] 2. The CNN-Transformer hybrid backbone network of the present application can efficiently capture different dimensional features of pests and adapt to monitoring tasks in multiple scenes such as vegetable gardens, orchards and farmlands. The deep separable convolution units in the first two stages can accurately extract local details such as pest edges and textures, such as the surface features of aphids and the nodule structure of moth larvae, which are key to distinguishing closely related pests. The Transformer module in the third stage can capture the overall shape of the pest and its relative position with the crop background by establishing global semantic associations, thereby solving the recognition confusion problem of densely distributed pests. The multi-scale features output by this architecture take into account both details and global information, providing a high-quality foundation for subsequent fusion processing and meeting the dual needs of accurate counting and species identification in sex lure monitoring systems, thereby improving the reliability of the device in recognizing pests in different ecological scenes.

[0017] 3. The spatial and channel dual attention mechanism of the present application can effectively filter complex background noise in the field and improve recognition accuracy. In the farmland environment, the background such as crop leaves, soil and weeds is easily confused with pest features, leading to false positives in traditional models. The improved spatial attention can simultaneously locate small pests that are densely distributed and large pests that are dispersed by using multi-scale pooling, thereby solving the positioning bias problem of single-scale attention. The improved channel attention can accurately select and distinguish the key channel features of target pests and non-target insects by using multi-branch dimension reduction fusion, thereby improving the recognition accuracy of pests.

[0018] 4. The present application selects high discriminative feature points (such as the tail spine features of wasps and the shell texture features of snails), and then aggregates feature association information through a graph convolution network to strengthen the differences between classes. When applied to field monitoring, it can accurately distinguish between closely related pests. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is a crop pest identification method based on BiFPN and multi-scale attention mechanism according to an embodiment of the present application.

[0020] Figure 2 is a model structure diagram of a crop pest identification method based on BiFPN and multi-scale attention mechanism according to an embodiment of the present application.

[0021] Figure 3 is a loss function calculation flowchart of a crop pest identification method based on BiFPN and multi-scale attention mechanism according to an embodiment of the present application.

[0022] Figure 4 is a discriminative feature aggregation flowchart of a crop pest identification method based on BiFPN and multi-scale attention mechanism according to an embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application will be further described in detail below with reference to the accompanying drawings.

[0024] Embodiment 1 Referring to Figure 1 , the crop pest identification method based on BiFPN and multi-scale attention mechanism according to the embodiment includes: S1, obtaining an agricultural insect image dataset covering multiple scenes, and performing division and multi-strategy data enhancement processing on the dataset to generate a training set, a validation set and a test set; S2, constructing a CNN-Transformer hybrid backbone network with the obtained dataset as input, outputting multi-scale initial features containing local details and global semantics through phased cooperation of convolution units and self-attention units; S3, inputting the multi-scale initial features into a BiFPN feature fusion module to obtain fused multi-scale features through bidirectional feature transmission and dynamic weighted fusion; S4, applying an improved spatial attention mechanism and a channel attention mechanism to the fused multi-scale features, respectively, to obtain reinforced multi-scale features; S5, inputting the reinforced multi-scale features into a fine-grained classification module to complete BiFPN-ConFormer model construction, and outputting pest class prediction results through discriminative feature selection and feature aggregation; S6, training and optimizing the BiFPN-ConFormer model using a triple synergistic loss function, adjusting model parameters using the validation set, evaluating model performance through the test set, and completing crop pest identification.

[0025] Specifically, a crop pest identification method based on BiFPN and multi-scale attention mechanism includes the following steps: S1, obtain an agricultural insect image dataset covering multiple scenes, and perform division and multi-strategy data enhancement processing on the dataset to generate a training set, a validation set and a test set; As shown in Figure 1 , an agricultural insect image dataset covering multiple scenes is obtained by combining field collection and network public data screening, the multiple scenes include different farmland types (such as wheat fields, rice fields, and vegetable greenhouses), different weather conditions, and different light intensities, to ensure that the dataset can reflect the diversity distribution of insects in real farmland environments; the dataset contains a total of 5162 agricultural common insect images, covering 11 types of agricultural insects, namely ants, bees, beetles, caterpillars, earwigs, grasshoppers, moths, slugs, snails, wasps, and weevils, and the number of samples in each category is evenly distributed without long-tail distribution phenomenon, which can avoid classification bias caused by unbalanced class sample distribution during model training. The above-mentioned 5162 agricultural insect image dataset is divided in proportion, of which 80% of the images are used as the training set for parameter learning of the BiFPN-ConFormer model; the remaining 20% of the images are further divided into a validation set and a test set in a 1:1 ratio, the validation set is used for parameter adjustment and overfitting monitoring during model training, and the test set is used for performance evaluation after model training is completed, to ensure that the class distribution of the training set, the validation set and the test set is consistent with the original dataset, and to avoid the influence of class distribution deviation caused by data division on the generalization ability of the model.

[0026] To expand the number of dataset samples, increase sample diversity, and improve the model's adaptability to insect images in different scenes, multi-strategy data enhancement processing is performed on the divided training set, including random cropping and scaling, random horizontal flipping, Mixup enhancement, and CutMix enhancement. The execution order and specific operations of each enhancement strategy are as follows: The first step is random cropping and scaling and random horizontal flipping. For each original image in the training set, the resolution is , the upper left corner coordinates of the cropping area and the cropping area size are first determined by a random number generator, where , , and are in the range of and the minimum bounding rectangle of the insect target is determined by a pre-trained U-Net semantic segmentation model, so as to ensure that the cropped region completely contains the bounding rectangle and avoids incomplete insect targets; then, a bilinear interpolation algorithm is used to scale the cropped local image to a uniform pixel size, the algorithm obtains the gray value of a to-be-interpolated point by weighting the gray values of four pixel points around the to-be-interpolated point, and the calculation formula is: , wherein, is the coordinate of the to-be-interpolated point, is the coordinates of four pixel points around the to-be-interpolated point, is the gray value of the corresponding pixel point, is the gray value of the to-be-interpolated point, which can ensure smooth transition of the pixels of the scaled image and no loss of details, after scaling, a random probability value p in the interval [0, 1] is generated by using the numpy library of Python, if p≤0.5, the image pixel matrix is flipped along the vertical center line, and the pixel coordinates of the flipped image satisfy wherein, is the image pixel matrix before flipping, is the image pixel matrix after flipping, is the width of the target image, so as to simulate the characteristics of different orientations of insects and enhance the adaptability of the model to orientation changes, if p>0.5, the image remains unchanged.

[0027] The second step is Mixup enhancement, the training set images processed in the above manner are divided into batches, each batch contains a fixed number of images and corresponding label vectors, the labels adopt one-hot encoding, that is, each type of insect corresponds to a one-hot vector with a total number of categories, only the position corresponding to the type of insect is 1, and the other positions are 0; for each batch, a random index replacement matrix P of (B×B) dimensions is generated, wherein B is the batch size, the matrix is a unit replacement matrix, the images in the batch are randomly shuffled and then paired again through matrix operation, so as to ensure that each image is randomly combined with other images in the batch; then, a mixing coefficient is sampled from a uniform distribution [0, 1], that is, the pixel matrices and label vectors of each pair of samples are mixed in a linear weighted manner to generate new samples, and the calculation formula is wherein, , are the pixel matrices of the paired two images, , are the corresponding label vectors, , are the pixel matrix and label vector of the mixed new sample, through the linear weighted mixing, a transition sample between the two original images is generated, which helps the model to learn the feature transition rule between different insect categories and improves the anti-interference ability.

[0028] The third step is CutMix enhancement. The training set images are also divided into batches, generating two independent batches (source image batch and target image batch). One image is taken from each batch to form a paired sample. For each pair of samples , is the source image, is the target image, and the mixing coefficient is sampled from a uniform distribution [0, 1]. The bounding box parameters of the rectangular clipping region are calculated according to , , where and are the width and height of the target image, respectively), the width and height of the bounding box, and the ratio of the area of the clipping region to the total area of the image is ; then a binary mask matrix is constructed. If the pixel coordinates (x, y) are within the clipping region, M(x, y) = 0, otherwise M(x, y) = 1. Finally, the pixels in the source image with mask value 0 are replaced by the pixels in the target image corresponding region, and the label vectors of the two images are mixed in the ratio . The calculation formula of the new sample is: , where is the binary mask, is the pixel-wise multiplication, is the combined ratio sampled from the uniform distribution (0, 1), and are the mixed features and labels, is the label corresponding to the image , and is the label corresponding to the image . This operation can introduce background features from different images while preserving the integrity of local insect features, helping the model learn local discriminative features of insect targets and improve recognition accuracy in complex farmland backgrounds.

[0029] S2, the obtained dataset is used as input to build a CNN-Transformer hybrid backbone network. Through the phased cooperation of convolution units and self-attention units, multi-scale initial features containing local details and global semantics are output. As Figure 2 , Figure 4As shown, the overall architecture design of the CNN-Transformer hybrid backbone network in this embodiment adopts a three-stage serial fusion structure. The first and second stages focus on local feature extraction, capturing fine-grained features such as edges, textures, and local limb structures of insect images through depthwise separable convolutional units. The third stage focuses on global semantic feature extraction, establishing semantic associations of feature points across the entire image through the Transformer module, capturing global information such as the spatial relationship between the insect target and the background, and the overall morphology of the insect. The three stages are connected in sequence, with the output of the previous stage serving as the input of the next stage, forming a feature extraction link from local to global.

[0030] The first stage uses the uniform-sized image output from step S1 as input and constructs a feature extraction layer using depthwise separable convolutional units. This stage aims to extract basic local features such as edges and color gradients in the shallow layers of the image. The depthwise separable convolutional unit consists of depthwise convolutional layers and pointwise convolutional layers. It performs spatial convolution independently on each channel of the input feature map, with each channel corresponding to an independent convolutional kernel (kernel size set to 3×3, stride set to 2, padding set to 1). Spatial feature extraction is performed only within a single channel to avoid cross-channel information interference. Let the input feature map be... , Input the number of channels. Given the height and width of the input feature map, the output feature map of the depthwise convolutional layer is... The calculation formula is: , in, For batch indexing, For channel index, For spatial coordinates, The parameters i and j of the 3×3 depth convolution kernel corresponding to the c-th channel are the kernel offsets. This represents the height and width of the feature map after depthwise convolution. This operation enables independent extraction of local features for each channel while reducing computational cost. Compared to traditional convolution, the computational cost is reduced to [value missing]. .

[0031] For the output of deep convolutional layers Perform a 1×1 convolution operation to achieve multi-channel feature fusion and channel number adjustment. The number of convolution kernels in the point convolutional layer is set to... By using 1×1 convolutional kernels to weight and combine local features from different channels along the channel dimension, a fused shallow local feature map is generated. This operation does not change the size of the feature map space. , With the premise of keeping unchanged, the fusion and dimension promotion of multi-channel local features are completed, and the first stage is implemented by stacking the above deep separable convolution units. In this embodiment, four deep separable convolution units are stacked, and BatchNorm normalization and ReLU activation functions are applied after each unit to enhance the non-linear expression and training stability, and finally the shallow local feature map is output.

[0032] The second stage takes the shallow local feature map output by the first stage as input As input, the deep separable convolution unit is also used for construction, but the convolution kernel parameters and the number of unit stacks are adjusted to realize the extraction of the middle layer local features such as insect limb joints and wing textures. The input feature map The size of the convolution kernel of the deep convolution layer is still set to 3*3, but the step is set to 2 and the padding is set to 1. Each channel corresponds to an independent convolution kernel. The output feature map The calculation formula is consistent with that of the first stage deep convolution, and only the input feature and the size parameter are different. This operation further extracts the middle layer local features of each channel, and the receptive field is expanded to 7*7. The number of convolution kernels of the point convolution layer is set to The number of input channels is increased from 64 to 128 through 1*1 convolution, realizing the fusion of multi-channel middle layer features. The output feature map The calculation formula is consistent with that of the first stage point convolution, and only the input channel number (64) and the output channel number (128) are different. This operation increases the channel number to improve the feature expression ability and fuses the middle layer local features of different channels.

[0033] The second stage adopts six deep separable convolution unit stacks (BatchNorm and ReLU are also applied after each unit), and residual connection is introduced after the third unit (the input feature and the output feature of the unit are added element by element) to alleviate the gradient disappearance problem of the deep network. Finally, the middle layer local feature map is output, which contains the middle layer structure details of the insect.

[0034] The third stage takes the middle layer local feature map output by the second stage as input As input, the Transformer module is deployed to establish the semantic association of feature points in the full graph range relying on the multi-head self-attention mechanism, so as to extract the global semantic features of the insect target such as the global morphology, the target and the background space relationship, etc. First, the middle layer local feature map is flattened, and the spatial dimension of the feature map is arranged in row priority order to generate a one-dimensional sequence , then the feature sequence A linear embedding layer is applied to increase the dimension of each feature point from 128 to 256 by a 1x1 convolution only acting on the channel dimension (without spatial dimension transformation), obtaining an embedding feature sequence The core of the linear embedding process is to realize linear transformation of the feature dimension through a weight matrix, and the calculation formula is: , wherein, is a batch index, is a feature point sequence index, is an embedding feature dimension index, is a weight parameter of the linear embedding layer, used to establish the association between the embedding feature dimension and the original feature dimension.

[0035] The Transformer module adopts a multi-head self-attention mechanism. The embedding feature sequence is first linearly transformed to generate a query matrix Q, a key matrix K, and a value matrix V, and the dimensions of the three matrices are consistent with . Then, according to the number of attention heads h (which is 8 in this embodiment), Q, K, and V are each split into h parallel sub-matrices, i.e. , and the splitting process is realized through the linear transformation matrix of the corresponding attention head, i.e. is obtained by multiplying Q and the query linear transformation matrix of the i-th attention head, is obtained by multiplying K and the key linear transformation matrix of the i-th attention head, is obtained by multiplying V and the value linear transformation matrix of the i-th attention head.

[0036] For each attention head, the self-attention weight and the output are independently calculated. First, the product of and the transpose matrix of is calculated to obtain an association score matrix between feature points. Then, the association score matrix is divided by , wherein, is a sub-matrix dimension, used to avoid the gradient disappearance of the Softmax function caused by too large association scores), and then the association scores are normalized to attention weights by the Softmax function (after normalization, the sum of the weights in each row is 1), and finally the attention weights are multiplied by to obtain the output of the i-th attention head, and the calculation formula is: , wherein, is an association score matrix between feature points, To activate the function, the association score is normalized to the attention weight. After the single attention head calculation is completed, the outputs of the h attention heads are spliced in the feature dimension to obtain a spliced feature with a dimension of Then the spliced feature is multiplied by a linear transformation matrix to realize the fusion of multi-attention head features, and finally generate multi-head self-attention output This process captures different types of semantic associations between feature points in parallel through multiple attention heads.

[0037] The third stage of the Transformer module is stacked by multiple Transformer encoder layers (11 encoder layers are stacked in this embodiment), each of which contains a multi-head self-attention sublayer and a feedforward neural network sublayer, and both sublayers use residual connection and LayerNorm normalization to ensure the training stability and feature expression ability of the deep network. The input feature of the sublayer is added to the output feature of the sublayer element by element to form the feature after residual connection This operation can directly pass the input feature to the subsequent layer, which can alleviate the gradient vanishing problem in the training process of the deep network.

[0038] The normalized feature is processed, the feature distribution is adjusted to a stable distribution with a mean close to 0 and a variance close to 1, and a minimum value is introduced, and the calculation formula is: , wherein, is the mean of the feature , var is the variance of the feature , and is the minimum value to avoid zero denominator.

[0039] Finally, the feedforward neural network sublayer is processed, which is composed of two fully connected layers and a ReLU activation function. The first fully connected layer increases the dimension of the input feature from to 1024, and the second fully connected layer restores the feature dimension to 256; between the two fully connected layers, a nonlinear transformation is introduced through the ReLU activation function to enhance the nonlinear expression ability of the feature.

[0040] After the third stage is processed by the above Transformer module, the output feature sequence needs to be remapped to the feature map format: first, the feature sequence output by the Transformer module is reshaped according to the spatial arrangement order before flattening to obtain a feature with a dimension of a preliminary global semantic feature map; and applying a 1x1 convolution to the preliminary feature map to adjust the number of channels to 256 and finally output a deep global semantic feature map .

[0041] S3, inputting the multi-scale initial features into the BiFPN feature fusion module to obtain fused multi-scale features through bidirectional feature transmission and dynamic weighted fusion; The input of this module is three multi-scale initial features of different resolutions output by the CNN-Transformer hybrid backbone network in step S2, which are: shallow local detail features , middle transition features , and deep global semantic features The three features cover different resolutions and feature levels, providing diversified information input for bidirectional fusion. Starting from the deep high semantic features, the global semantic information is gradually transmitted to the shallow features through resolution up-sampling and dynamic weighted fusion, making up for the lack of semantic features in the shallow features. The deepest global semantic features are used as the processing object, and a bilinear interpolation algorithm is used to perform a resolution up-sampling operation on it to increase the resolution of the from to the same as the middle transition features , which is mainly used for subsequent process adaptation of different resolution feature fusion. If the resolution difference is caused by the expansion of the feature level, the up-sampling multiple is calculated according to the target resolution / original resolution. The bilinear interpolation algorithm obtains the pixel value of the interpolation point by weighting the gray values of the four pixel points around the interpolation point, ensuring the smooth transition of the spatial details of the feature map after up-sampling without obvious jaggies or blurring. The up-sampled and the original are input into the dynamic weighted fusion unit, which assigns different importance weights to the two input features through learnable weights and generates the first stage fusion features The core of the dynamic weighted fusion unit is to realize adaptive weighted fusion of features through formula calculation, and the expression is: , wherein, is the output feature after fusion, is the i-th input feature, is the corresponding learnable weight, is to avoid the extreme value of the denominator being zero.

[0042] The same bilinear interpolation algorithm as described above is used to perform a resolution up-sampling operation on the first stage fusion features to increase the resolution from ​, to the shallow layer local detail feature consistent The channel number is kept unchanged (still 128) in the up-sampling process. The up-sampled and the original The input dynamic weighted fusion unit calculates according to the same fusion formula above to generate the first stage fusion feature At this time While retaining On the basis of retaining high-resolution local details, the middle-layer structure from and the global semantic information from are integrated to achieve the enhancement of the semantic of the shallow features.

[0043] After that, the bottom-up transmission takes the top-down fused shallow features as the starting point, and through resolution down-sampling and dynamic weighted fusion, the shallow high-resolution detail information is gradually transmitted to the deep features to make up for the loss of details in the deep features. The top-down fused shallow fusion features are taken as the processing object, and the maximum pooling algorithm is used to perform the resolution down-sampling operation, with the pooling kernel size set to 2x2 and the step set to 2, so that the resolution of is reduced to be consistent with The maximum pooling selects the maximum value in the pooling window as the output pixel value, which can retain the key local details in the feature map (such as the insect edge inflection point), while reducing the feature map dimension and the calculation amount. The down-sampled is input into the dynamic weighted fusion unit together with the first stage fusion feature , and the fusion formula is calculated to generate the second stage fusion feature The resolution down-sampling operation is performed on the second stage fusion feature using 2x2 maximum pooling (step 2), because the original resolution has been consistent with the resolution of the deep features , and the down-sampling multiple is set to 1, which is mainly used to maintain the uniformity of the process. The down-sampled is input into the dynamic weighted fusion unit together with the original deep global semantic feature , and the fusion formula is calculated to generate the second stage fusion feature .

[0044] Finally, the key fusion features generated in the top-down and bottom-up fusion processes are integrated to obtain the fused multi-scale features covering different resolutions. The first stage fusion feature , the second stage fusion feature , and the second stage fusion feature are integrated to form the fused multi-scale feature set.

[0045] S4, respectively applying improved spatial attention mechanism and channel attention mechanism to the fused multi-scale features to obtain reinforced multi-scale features; In this embodiment, the fused multi-scale features output in step S3 are taken as input, and improved spatial attention mechanism and channel attention mechanism are respectively deployed. Through the synergistic reinforcement of spatial dimension key region positioning and channel dimension effective feature screening, the discriminative features of pest targets are highlighted and the background interference features are suppressed, and finally the reinforced multi-scale features are obtained. The input of this mechanism is the fused multi-scale feature set output by the BiFPN feature fusion module in step S3, including shallow layer fusion features with a channel number of 64 , middle layer fusion features with a channel number of 128, middle layer fusion features , and deep layer fusion features with a channel number of 256 .

[0046] For the single-level input features X in the fused multi-scale features, X can be , and , the dimension satisfies , C is the channel number, H is the feature map height, and W is the feature map width. A multi-scale sampling set is set. In this embodiment, three scales are adopted, which are 3, 5, and 7. Based on the determination of the size distribution of pests in the data set in step S1, adaptive average pooling operations with scales of , are performed on X. The adaptive average pooling operation dynamically adjusts the pooling window size, so that the pooling results under different scales can cover the key region of the pest target, and extract the spatial attention prior information under this scale. Then, a 1x1 convolution operation is applied to the pooling features of each scale to compress the channel number of each pooling feature from C to 1, generating three single-channel preliminary attention maps, denoted as . The single-channel design can reduce parameter redundancy while retaining the attention weight distribution of the spatial dimension.

[0047] Since the resolution of the preliminary attention map after multi-scale pooling is inconsistent with the input feature X, a bilinear interpolation algorithm is used to perform up-sampling operation on , the interpolation kernel size is set to 3x3 and the step is set to 1, and the resolution of each preliminary attention map is unified to the original size HxW of the input feature X, obtaining the spatially aligned multi-scale attention map, denoted as , This operation ensures that the spatial position of the multi-scale attention map corresponds to the input feature X one by one, avoiding the attention weight allocation deviation caused by spatial misalignment in subsequent fusion.

[0048] In the channel dimension, the spatially aligned multi-scale , Perform the concatenation operation to obtain the dimension as splicing attention map ,right A 3×3 convolution operation is applied, with the kernel size set to 1, stride set to 1, and padding set to 1. This convolution operation enables semantic interaction and information fusion of multi-scale attention weights, generating a fused attention map. Finally, for Applying the Sigmoid activation function constrains the attention weights to the [0,1] interval, resulting in the final spatial attention map: , in, It is the Sigmoid activation function. For convolution operations, For the scale Adaptive average pooling operation, This is for the splicing operation of multi-scale attention maps. For the upsampling operation, C is the number of channels, H is the feature map height, and W is the feature map width, resulting in the final spatial attention map. The feature X is multiplied element-wise along the spatial dimension to enhance the features of the target spatial region of the pest (regions with weights close to 1) and suppress the features of the background interference region (regions with weights close to 0), thus obtaining the features enhanced by spatial attention.

[0049] Next, channel attention mechanism deployment and feature enhancement are performed: Obtain single-level input features from the fused multi-scale features, i.e., features enhanced with spatial attention. Where B is the batch size, and multiple dimensionality reduction ratio sets are defined. In this embodiment Based on the importance analysis of channel features of 11 types of pests in the dataset from step S1, it was determined that the channel feature extraction requirements can be covered from fine-grained to coarse-grained. Correspondingly, three parallel dimensionality reduction processing branches were constructed. First, for... Perform adaptive average pooling operation to transfer the feature map from Compress to The size is determined by calculating the global mean of each channel to extract its global context information; subsequently, a 1×1 convolution is used to compress the number of channels from C to [the desired size]. The dimensionality-reduced features are obtained; layer normalization and ReLU activation functions are then applied sequentially to the dimensionality-reduced features. The expression for layer normalization is as follows: ,in, The characteristic mean, For characteristic variance, To avoid the denominator zero of the minimum value, the expression of the ReLU activation function is used to stabilize the feature distribution , which is used to enhance the nonlinear expression ability of features and suppress the propagation of invalid features. Finally, the channel number is restored to C through 1x1 convolution, generating single-granularity channel features under the corresponding dimension reduction ratio, and the three branches output . .

[0050] The 3 single-granularity channel features are spliced in the channel dimension to obtain multi-granularity spliced features with a dimension of , denoted as . The is subjected to a 1x1 convolution operation to compress the channel number from 3C to C, realizing adaptive fusion of multi-granularity channel features, and obtaining the fused features, denoted as . The is subjected to a Sigmoid activation function to constrain the feature values to the interval [0, 1], generating the final channel attention weight . The and are multiplied element by element in the channel dimension to realize the strengthening of pest key channel features (channels with weights close to 1) and the suppression of invalid channel features (channels with weights close to 0), obtaining the channel attention enhanced features: , where is the input feature, is the adaptive average pooling operation, is the 1x1 convolution operation, is the layer normalization operation, is the ReLU activation function, is the feature splicing operation, is the Sigmoid activation function, and the above spatial attention enhancement-channel attention enhancement process is performed on in the fused multi-scale features to obtain the corresponding enhanced features: shallow layer enhanced features, middle layer enhanced features and deep layer enhanced features.

[0051] The improved channel attention mechanism has the following characteristics: Granularity decomposition mechanism: different compression ratio branches capture multi-granularity features.

[0052] Dynamic weight fusion: learnable convolution adaptively fuses multi-granularity features, replacing artificial weighted fusion.

[0053] In this experiment, three dimension reduction ratios, .

[0054] S5, input the reinforced multi-scale features into a fine-grained classification module to complete BiFPN-ConFormer model construction, and output pest category prediction results through discriminative feature selection and feature aggregation; For each level feature map in the reinforced multi-scale features Corresponding to the shallow, middle and deep layers, for each level feature map Calculate the feature point classification probability, and flatten the spatial dimension of to a feature point sequence, and the dimension of each feature point is the channel number of the level feature map (such as the shallow The feature point dimension of the deep layer is 64); then the feature point sequence is input into two fully connected layers, the first fully connected layer maps the feature point dimension to 256, and after BatchNorm normalization and ReLU activation function enhancement of non-linear expression, the second fully connected layer is input, and the dimension is further mapped to the total number of pest categories (11 categories in this embodiment); finally, the output of the second fully connected layer is converted into the class probability distribution of each feature point through the Softmax function, and the calculation formula is: , Among them, is the probability distribution matrix of the kth layer feature point, , are the learnable weight matrices of the two fully connected layers respectively, is the BatchNorm normalization operation, is the ReLU activation function, is the Softmax function, which ensures that the class probability sum of each feature point is 1, according to the calculated feature point classification probability, the feature points of each level are arranged in descending order of probability value (taking the maximum class probability value corresponding to each feature point as the basis for sorting), and a fixed number of high-probability feature points are selected as discriminative feature points according to the level: in this embodiment, the shallow has the highest resolution and the most feature points, 128 high-probability feature points are selected, the middle selects 64 high-probability feature points, and the deep selects 32 high-probability feature points, which are determined based on the total number of feature points of each level and the discriminative requirement, which not only ensures that the selected feature points cover the key parts of pests, but also avoids excessive number of subsequent aggregation calculation.

[0055] All levels of discriminative feature points selected are regarded as node set V in graph structure , each node corresponds to a discriminative feature point, and the node feature is the channel feature of the feature point in the original reinforced feature map, and the edge set Represented by the adjacency matrix A, the feature similarity (using cosine similarity) between any two nodes i and j is calculated. If the similarity is greater than a preset threshold (set to 0.6 in this embodiment), the two nodes are considered to have a semantic relationship, and the corresponding adjacency matrix... ,otherwise To enhance the influence of the node's own characteristics, an identity matrix is ​​superimposed on the adjacency matrix A. (Total number of nodes), resulting in the graph adjacency matrix containing self-connections. To ensure that each node is connected to itself, a two-layer graph convolutional network is constructed to aggregate node features. The propagation process of each GCN layer follows the inter-layer propagation rules of graph convolution, and the calculation formula is as follows: , Where V is the set of nodes, Let be the set of edges. For the features of the l-th layer nodes, for The corresponding degree matrix, Let be the trainable weights of the l-th layer graph convolution. As a graph adjacency matrix containing self-connections, this propagation formula allows each node's features to aggregate the feature information of its neighboring nodes, thereby enabling the discrimination of semantic interactions and global associations between feature points.

[0056] The node feature matrix output by the second-layer GCN Perform supernode aggregation operation: Based on the original hierarchical affiliation of the feature points (shallow, middle, deep), aggregate the node features of the same level into a supernode, that is, aggregate the 128 shallow nodes into a supernode. The 64 nodes in the middle layer are aggregated into a super node. The 32 deep nodes are aggregated into a super node. The aggregation method involves taking the average of the node features at each level column-wise to obtain the features of each super node; then, global average pooling is performed on the features of the three super nodes to obtain the global aggregated features, which integrate the global semantic association information of the discriminative feature points at different levels.

[0057] The aforementioned global aggregated features are input into a linear classifier. The linear classifier maps the 128-dimensional global aggregated features to an 11-dimensional class score vector using a learnable weight matrix. Then, it transforms the features into a final class probability distribution using a Softmax function. The class with the highest probability value is taken as the pest category prediction result, as shown in the formula: , in, For the final predicted pest category, The global average pooling feature represents the three supernodes. learnable weight matrix of a linear classifier.

[0058] So far, by integrating the CNN-Transformer hybrid backbone network of step S2, the BiFPN feature fusion module of step S3, the improved attention mechanism of step S4, and the fine-grained classification module of the present step S5, the overall construction of the BiFPN-ConFormer model is completed. The model can receive the agricultural insect image data preprocessed by step S1 and output the corresponding pest category prediction results, realizing multi-scale and high-precision crop pest identification.

[0059] S6, a triple synergistic loss function is used to train and optimize the BiFPN-ConFormer model, the model parameters are adjusted using the validation set, and the model performance is evaluated through the test set, completing the crop pest identification.

[0060] As shown in Figure 3 , a global average pooling operation is performed on the multi-scale initial feature maps output by step S2, the spatial dimensions of the feature maps at each level are compressed to 1x1 size, and the global context vectors corresponding to the feature maps at each level are obtained. The global context vectors at each level are input into a single-layer fully connected layer and mapped to a dimension consistent with the number of pest categories (11 categories), and the category prediction scores corresponding to the feature maps at each level are obtained. The cross-entropy loss function is used to calculate the classification loss at each level. The cross-entropy loss can effectively measure the difference between the prediction scores and the true labels, and the calculation formula is: , wherein, is the feature extraction loss, K represents the set of levels of the multi-scale feature maps, is the prediction score of the true category corresponding to the kth feature, is the sum of the classification losses of the three levels, ensuring that the features at each level can effectively participate in category differentiation.

[0061] The weak selection loss aims to suppress redundant / interfering feature points that are not selected as discriminative feature points in the fine-grained classification module, and guide the model to focus on the key discriminative region of the pest. The specific calculation process is as follows: for the feature points that are not selected in step S5 (i.e., the low-probability feature points that do not enter the fixed number after sorting by classification probability at each level), the corresponding category prediction scores (taken from the probability distribution output by the two fully connected layers in step S5) are extracted as the discard region prediction scores . The cross-entropy loss function is used to calculate the classification loss of the discard region of each level and each category. By penalizing the prediction scores of the discard region, the expression of invalid feature points is suppressed, and the calculation formula is: , wherein, For weak selection loss, To sum the loss of the discarded regions over all levels and all categories, The probability of discarding a region is considered ineffective. This probability is converted into a loss through logarithmic operations to ensure that the impact of the discarded region's feature points on the model's classification decision is minimized.

[0062] The graph combination loss aims to optimize the aggregation effect of the graph convolutional network (GCN) on discriminative feature points in the fine-grained classification module, ensuring the classification accuracy after global feature fusion. The specific calculation process involves: processing the supernode features output by the graph convolutional network in step S5... The average pooling operation is performed to obtain the global feature vector. This global feature vector is then input into a linear classifier to obtain the predicted class probability for each node (discriminator feature point). (i is the node index, t is the class index); the predicted probability and the true probability are calculated using the soft-target cross-entropy loss function. The difference between the probability distributions corresponding to the true labels (where the true category is 1 and the rest are 0) is used as the average of the losses of all nodes as the graph composite loss, calculated using the following formula: , in, The graph combination loss is given by N, where N is the total number of discriminative feature points. The total number of categories, To average the loss across all nodes and ensure that the loss calculation is not affected by the number of nodes, this method accurately reflects the difference between the GCN aggregation effect and the classification target.

[0063] To balance the contributions of the three types of losses to model optimization, weight coefficients are assigned to each loss: feature extraction loss weight. Weak selection loss weights Auxiliary loss, avoiding over-suppression of feature diversity, graph combination loss weights. The total loss of the triple collaboration is obtained by weighted summation, and the calculation formula is as follows: , Where L is the total loss of the triple collaboration, which guides the updating of model parameters.

[0064] The model training optimization process adopts an iterative process of forward propagation-backward propagation-parameter update: the training set samples are input into the BiFPN-ConFormer model in batches, the model prediction result and the triple synergistic total loss L are calculated by forward propagation; the total loss is calculated by the backward propagation algorithm (based on the chain rule) to calculate the gradient of all learnable parameters of the model (including the backbone network convolution kernel, the Transformer attention weight, the BiFPN dynamic weight, the attention mechanism parameter, and the classification module full connection layer weight); the model parameters are updated by using the AdamW optimizer (the weight decay coefficient is set to 0.05 to avoid parameter overfitting) to minimize the total loss L; during the training process, the total loss and the classification accuracy of the model are calculated on the validation set every 5 epochs, if the validation set accuracy does not improve for 3 consecutive epochs, the learning rate is reduced by using the learning rate cosine annealing strategy (warm-up time is 5 epochs, and the base learning rate is 0.0005), and the model parameters of the current validation set accuracy optimal (temporary optimal model) are saved to avoid model overfitting.

[0065] After the model training is completed (iterating to the preset 30 epochs or the validation set accuracy converges), the saved temporary optimal model parameters are loaded, the test set samples are input into the model, and the class prediction results of all samples in the test set are obtained by forward propagation; the accuracy (Acc), precision (Pre), recall (Re), and weighted F1 score are used as evaluation indicators to comprehensively measure the model performance, and the calculation formulas of the indicators are as follows: Accuracy (Acc): reflects the overall classification correctness of the model, that is, the proportion of correctly predicted samples in the test set to the total number of samples: Precision (Pre): reflects the proportion of actual positive samples in the predicted positive samples (a certain type of pest), and measures the prediction accuracy: Recall reflects the probability of correctly predicting the insect class, and is calculated as F1 score is the weighted harmonic mean of precision and recall, commonly used to comprehensively evaluate the classification performance of the model. Among them, the weighted F1 score is calculated by weighted average of the F1 scores of each class, and the weight is the proportion of the corresponding class samples in the data set, and the specific formula is: ,

[0066] In the above formulas, respectively represent the number of true positives, false positives, true negatives, and false negatives; i is the class index, and Z is the total number of classes, The weight value of the ith sample in the data set is evaluated by the above test set. If the model reaches the preset threshold (in this embodiment, the accuracy is ≥ 94%, and the weighted F1 score is ≥ 0.94) in the core indicators such as accuracy and weighted F1 score, it is determined that the model training is qualified and can be put into practical crop pest identification application.

[0067] Embodiment 2 The difference between this embodiment and embodiment 1 is that this embodiment provides specific comparative experiments: A pest identification architecture that fuses multi-scale feature enhancement and attention optimization is proposed. To verify its effectiveness, we conducted a system evaluation on the dataset, selected six representative comparative models, including mainstream CNN architectures (VGG16, ResNet50, DenseNet121) and cutting-edge models (ConvNeXt-tiny, MobileViT-s), and used the original model Pest-ConFormer as a direct comparison benchmark; Table 1 is a comparison of experimental results;

[0068] The experimental results show that the model in this paper has greatly improved accuracy and F1-score compared to mainstream CNN classification models (VGG16 / ResNet50 / DenseNet121) and new architectures (ConvNeXt-tiny and MobileViT-s), and the accuracy and F1-score of the model in this work exceed the original model Pest-ConFormer by 0.78% and 0.85%, which verifies the effectiveness of the dual-path feature aggregation and improved attention mechanism in improving model recognition.

[0069] Embodiment 3 provides a crop pest identification system based on BiFPN and multi-scale attention mechanism, comprising: The data acquisition module is configured to acquire an agricultural insect image dataset covering multiple scenarios, and perform division and multi-strategy data augmentation processing on the dataset to generate a training set, a validation set, and a test set; The model module is configured to construct a CNN-Transformer hybrid backbone network with the acquired dataset as input, and output multi-scale initial features containing local details and global semantics through the phased cooperation of convolution units and self-attention units; The conversion module is configured to input the multi-scale initial features into the BiFPN feature fusion module, and obtain fused multi-scale features through bidirectional feature transmission and dynamic weighted fusion; The reinforcement module is configured to apply improved spatial attention mechanisms and channel attention mechanisms to the fused multi-scale features, respectively, to obtain reinforced multi-scale features. The output module is configured to input the reinforced multi-scale features into a fine-grained classification module, complete BiFPN-ConFormer model construction, output a pest category prediction result through discriminative feature selection and feature aggregation, and output the pest category prediction result. The optimization module is configured to train and optimize the BiFPN-ConFormer model by using a triple synergistic loss function, adjust model parameters by using a verification set, evaluate model performance by using a test set, and complete crop pest identification.

[0070] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, any equivalent changes made in the structure, shape, and principle of the present application should be covered within the protection scope of the present application.

Claims

1. A crop pest recognition method based on BiFPN and multi-scale attention mechanism, characterized in that, The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model.

2. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. S12, perform Mixup augmentation on the flipped image dataset, randomly select two insect images from the same batch With , uniformly sample the mixing coefficient from [0, 1] , linearly weight mix the pixel matrix and label vector of the two images to generate a new image x and a new label y; S13. Finally, the image dataset is enhanced with CutMix, and two insect images are randomly selected. and Sampling mixing coefficient And calculate the bounding box parameters of the rectangular clipping region, construct the binary mask matrix M, and then... Replace cropped area pixels with corresponding region pixels, by The new samples are generated by proportionally mixing labels, and the expression for generating the new samples is: ， where, is a binary mask, is a pixel-wise multiplication, is a combined proportion sampled from a uniform distribution (0, 1), and is the mixed feature and label, is an image corresponding label, is an image corresponding label.

3. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The application relates to a crop pest recognition method based on a BiFPN-ConFormer model.

4. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. S31, inputting the multi-scale initial features, the multi-scale initial features including at least three feature maps of different resolutions, respectively corresponding to shallow local detail features , middle transition features , and deep global semantic features ; S32, perform top-down feature transmission and fusion, from the deepest high semantic features Initially, its resolution is up-sampled to the middle layer transition feature Consistently, the up-sampled deep layer feature is fused with Input dynamic weighted fusion unit, fuse by weight ratio to get the first stage fusion feature , similarly, the up-sampled is fused with to get the first stage fusion feature ; S33. Based on top-down fusion characteristics, max pooling is used to... Resolution downsampling to Features are consistent, and the downsampled shallow fusion features are combined with Input the dynamic weighted fusion unit to obtain the second-stage fusion features. Similarly, After downsampling and The fusion yields the characteristics of the second stage of fusion. ; S34, integrating , and , to obtain the fused multi-scale features, and an expression of the dynamic weighted fusion unit is: ， wherein, is the output feature after fusion, is the i-th input feature, is corresponding learnable weight, is to avoid the minimum value of the denominator being zero.

5. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The improved spatial attention mechanism includes input features Adaptive average pooling of multiple scales is performed, the number of channels is compressed from C to 1 through a 1*1 convolution, a plurality of single-channel attention maps are generated, bilinear interpolation algorithm is used on the single-channel attention map to perform upsampling operation, and a plurality of multi-scale attention maps after spatial alignment are obtained. The multi-scale attention maps are spliced in the channel dimension, and a 3*3 convolution operation is performed on the spliced attention maps to generate a fusion attention map. The Sigmoid activation function is applied to the fusion attention map to constrain the attention weight value to the interval [0, 1], and the final spatial attention map is obtained. The expression of the spatial attention map is: ， wherein, is a Sigmoid activation function, is a convolution operation, is an adaptive average pooling operation with a scale of is an adaptive average pooling operation with a scale of is a concatenation operation of multi-scale attention maps, is an up-sampling operation, C is the number of channels, H is the height of the feature map, and W is the width of the feature map.

6. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN-ConFormer model. The application relates to a crop pest recognition method based on a BiFPN 7. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The input of the reinforced multi-scale feature into the fine-grained classification module includes: calculating the classification probability of each feature point through a full connection layer and a softmax function for each hierarchical feature map in the reinforced multi-scale feature, arranging the feature points in descending order according to the probability value, selecting a fixed number of high-probability points as discriminative feature points at different levels, regarding all selected discriminative feature points as nodes in a graph structure, and constructing a graph The node features are aggregated by using a graph convolution network, the node features output by the graph convolution network are aggregated into several super nodes through a pooling operation, average pooling is performed on the super nodes, and a final classification result is output through a linear classifier, and the inter-layer propagation expression of the graph convolution network is as follows: ， where V is a set of nodes, E is a set of edges, is a node feature of the l-th layer, is a is a degree matrix, is a trainable weight of the l-th layer graph convolution, is a graph adjacency matrix with self-connections.

8. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The training and optimization of the BiFPN-ConFormer model by using the triple synergistic loss function comprises: S61, performing a global average pooling operation on the multi-scale initial feature maps output by each level of the CNN-Transformer hybrid backbone network to obtain class prediction scores corresponding to the features of each level, calculating the classification loss of each level of feature by using a cross-entropy loss function, summing all the classification losses to obtain a feature extraction loss; S62, calculating the class prediction scores of the feature points that are not selected as discriminative feature points in the fine-grained classification module to obtain discard region prediction scores, calculating the classification loss of each level and each class of discard region by using a cross-entropy loss function, summing all the losses to obtain a weak selection loss; S63, performing an average pooling operation on the super node features output by the graph convolution network in the fine-grained classification module to obtain a global feature vector, inputting the global feature vector into a linear classifier to obtain the class prediction probability of each node, calculating the difference between the prediction probability and the true probability by using a soft target cross-entropy loss function, and taking the average to obtain a graph combination loss; S64, assigning weight coefficients to each loss, and obtaining a triple synergistic total loss by weighted summation.

9. The crop pest recognition method based on BiFPN and multi-scale attention mechanism according to claim 1, characterized in that, The model performance is evaluated by a test set, which comprises inputting the training set into the BiFPN-ConFormer model, calculating the prediction result of the model and the triple synergistic total loss by forward propagation, calculating the gradient of the loss with respect to each model parameter by back propagation, updating the parameters using the AdamW optimizer, minimizing the total loss, loading the saved temporary optimal model after training is completed, inputting the test set into the model to obtain the prediction result, and using accuracy, precision, recall and weighted F1-score as evaluation indicators, and the evaluation expression of the F1-score is: ， wherein Z is the number of classes, is the F1-score of the i-th class, is the sample proportion.

10. A crop pest recognition system based on BiFPN and multi-scale attention mechanism, executing the method of claim 1, characterized in that, Comprise: The data acquisition module is configured to obtain an agricultural insect image dataset covering multiple scenes, and perform division and multi-strategy data augmentation processing on the dataset to generate a training set, a validation set and a test set; The model module is configured to construct a CNN-Transformer hybrid backbone network with the obtained dataset as input, output multi-scale initial features containing local details and global semantics through the phased cooperation of convolution units and self-attention units; The conversion module is configured to input the multi-scale initial features into a BiFPN feature fusion module to obtain fused multi-scale features through bidirectional feature transmission and dynamic weighted fusion; The reinforcement module is configured to apply improved spatial attention mechanisms and channel attention mechanisms to the fused multi-scale features to obtain reinforced multi-scale features; The output module is configured to input the reinforced multi-scale features into a fine-grained classification module to complete the construction of the BiFPN-ConFormer model, and output pest class prediction results through discriminative feature selection and feature aggregation. The optimization module is configured to train and optimize the BiFPN-ConFormer model by using a triple synergistic loss function, adjust model parameters by using a verification set, evaluate model performance by using a test set, and complete crop pest identification.

Images