Pest detection method, device and equipment based on improved RTDETR model

Abstract

The invention discloses a pest detection method, device and equipment based on an improved RTDETR model, and relates to the technical field of target detection, and the method comprises the steps: obtaining a to-be-detected farmland image; the to-be-detected farmland image is input to a preset pest detection model, a pest detection result is obtained, the preset pest detection model comprises a preset backbone network, a preset encoder and a preset decoder, the preset backbone network comprises an integrated attention Basic Block module replacing a preset number of Basic Block modules in an original RTDETR model, and the preset encoder and the preset decoder are connected with the integrated attention Basic Block module; therefore, the attention capability of the model to key features is enhanced, and the preset encoder comprises an adaptive sparse self-attention module replacing an original AIFI module, so that the feature processing efficiency and precision are improved. Compared with an existing mode, the pest detection model can better capture pest characteristics, and the reliability of pest detection results is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of target detection, and particularly relates to a pest detection method, device and equipment based on an improved RTDETR model. BACKGROUND

[0002] Agricultural pest detection, as a key link of the intelligent plant protection system, is of great significance to ensure the healthy growth of crops and agricultural yield. With the advancement of agricultural modernization, the traditional pest monitoring method relying on manual patrol has been unable to meet the efficient needs of current agricultural production, and has many defects such as low efficiency, high misjudgment rate, etc. The automatic recognition technology based on computer vision has gradually become the mainstream direction to solve this problem, especially in complex farmland scenes.

[0003] Among numerous computer vision algorithms, the DETR series algorithm based on the Transformer architecture has shown better modeling ability than the traditional convolutional neural network (CNN) due to its global attention mechanism, which brings new opportunities for agricultural pest detection. However, the existing agricultural pest detection technology based on the real-time target detector (RTDETR) has insufficient feature extraction capability for dense small targets, resulting in a high missing detection rate in the early stage of pest infestation, which affects the timeliness and accuracy of pest detection, and it is difficult to fully meet the core requirements of real-time, lightweight and cross-scene robustness of agricultural intelligent transformation, which limits its wide application and in-depth development in the field of agricultural pest detection. SUMMARY

[0004] The main purpose of the present application is to provide a pest detection method, device and equipment based on an improved RTDETR model, which aims to solve the technical problem of insufficient feature extraction capability for dense small targets in the existing computer vision-based agricultural pest detection technology, resulting in low accuracy of pest detection.

[0005] To achieve the above-mentioned purpose, the present application provides a pest detection method of an improved RTDETR model, which comprises:

[0006] obtaining a farmland image to be detected;

[0007] input the farmland image to be detected into a preset pest detection model to obtain a pest detection result, the preset pest detection model comprising: a preset backbone network, a preset encoder, and a preset decoder, the preset backbone network comprising integrated attention BasicBlock modules that replace a preset number of BasicBlock modules in an original RTDETR model, the preset encoder comprising an adaptive sparse self-attention module that replaces an original AIFI module, the preset encoder being configured to perform feature fusion on multi-scale feature maps obtained through the preset backbone network and input the obtained fused feature maps to the preset decoder for target detection.

[0008] In an embodiment, the step of inputting the farmland image to be detected into a preset pest detection model to obtain a pest detection result comprises:

[0009] The preset backbone network is configured to perform feature extraction on the farmland image to be detected to obtain the multi-scale feature maps, the multi-scale feature maps comprising: a P2 feature map, a P3 feature map, a P4 feature map, and a P5 feature map.

[0010] The preset encoder is configured to perform feature fusion on the multi-scale feature maps to obtain a fused feature map.

[0011] The preset decoder is configured to perform target detection on the fused feature map to obtain a pest detection result.

[0012] In an embodiment, the preset backbone network further comprises: a convolution and normalization layer, a max-pooling layer, and two BasicBlock modules connected in sequence, the second BasicBlock module being further connected to the first integrated attention BasicBlock module.

[0013] The step of performing feature extraction on the farmland image to be detected through the preset backbone network to obtain the multi-scale feature maps comprises:

[0014] The convolution and normalization layer is configured to perform initial feature extraction on the farmland image to be detected to obtain image basic features.

[0015] The max-pooling layer is configured to down-sample the image basic features to obtain a P2 feature map.

[0016] The first BasicBlock module is configured to perform feature extraction on the P2 feature map to obtain a P3 feature map.

[0017] The second BasicBlock module is configured to perform detailed feature extraction on the P3 feature map to obtain a detailed feature map.

[0018] The first feature of the detail feature map is extracted based on an integrated channel and spatial attention mechanism through a first integrated attention BasicBlock module, to obtain a P4 feature map;

[0019] The first feature of the detail feature map is extracted based on an integrated channel and spatial attention mechanism through a first integrated attention BasicBlock module, to obtain a P5 feature map.

[0020] In an embodiment, the integrated attention BasicBlock module comprises a main path and an attention branch;

[0021] The main path is composed of two convolution groups connected in series, and each convolution group comprises a preset convolution layer, a batch normalization layer and a ReLU activation function layer connected in series; the attention branch comprises a global average pooling layer, a fully connected layer and a Sigmoid activation function layer connected in series;

[0022] The ReLU activation function layer in the first convolution group is further connected with the global average pooling layer and the preset convolution layer in the second convolution group.

[0023] In an embodiment, the preset encoder further comprises an input projection module and a preset CCFM module, and the adaptive sparse self-attention module is connected with the input projection module and the preset CCFM module respectively;

[0024] The step of obtaining a fused feature map by fusing the multi-scale feature maps through the preset encoder comprises:

[0025] The multi-scale feature maps are unified in channels through the input projection module, to obtain the multi-scale feature maps unified in channels;

[0026] The multi-scale feature maps unified in channels are enhanced in features through the adaptive sparse self-attention module, to obtain a target multi-scale feature map;

[0027] The target multi-scale feature map is fused in features through the preset CCFM module, to obtain a fused feature map.

[0028] In an embodiment, the adaptive sparse self-attention module comprises a multi-head self-attention mechanism unit, a sparse enhancement unit, an FFN unit and a layer normalization unit;

[0029] The step of obtaining a target multi-scale feature map by enhancing the multi-scale feature maps unified in channels in features through the adaptive sparse self-attention module comprises:

[0030] The multi-head self-attention mechanism unit performs position-dependent relationship extraction on the channel-unified multi-scale feature map, and generates attention weights according to the extraction result;

[0031] The sparse enhancement unit adjusts the attention weights according to the sparse mask, and obtains sparse-enhanced attention weights;

[0032] The FFN unit performs nonlinear transformation on the sparse-enhanced attention weights, and obtains target feature representation;

[0033] The target feature representation is normalized, and the normalized target feature is added to the channel-unified multi-scale feature map to obtain a target multi-scale feature map.

[0034] In an embodiment, the preset decoder includes an IoU-aware query selection mechanism module, an original RETER decoding module, and a preset detection head.

[0035] The step of performing target detection on the fusion feature map by the preset decoder to obtain a pest detection result includes:

[0036] The IoU-aware query selection mechanism module performs initial query on the fusion feature map to obtain an initial object.

[0037] The preset RETER decoding module combines the initial object to perform accurate query on the fusion feature map to obtain a multi-scale query vector.

[0038] The preset detection head maps the multi-scale query vector to obtain a pest detection result.

[0039] In an embodiment, before the step of inputting the to-be-detected farmland image into a preset pest detection model to obtain a pest detection result, the method includes:

[0040] Obtain a multi-scene agricultural pest dataset, and preprocess the multi-scene agricultural pest dataset to obtain a target dataset.

[0041] Train an initialized pest detection model using the target dataset to obtain a preset pest detection model.

[0042] In addition, to achieve the above-mentioned purpose, the present application also provides a pest detection device based on an improved RTDETR model, which includes:

[0043] An image acquisition module is configured to acquire a to-be-detected farmland image.

[0044] The pest detection module is configured to input the farmland image to be detected into a preset pest detection model to obtain a pest detection result, the preset pest detection model comprising: a preset backbone network, a preset encoder and a preset decoder, the preset backbone network comprising integrated attention BasicBlock modules that replace a preset number of BasicBlock modules in an original RTDETR model, the preset encoder comprising an adaptive sparse self-attention module that replaces an original AIFI module, the preset encoder being configured to perform feature fusion on multi-scale feature maps obtained through the preset backbone network, and the obtained fused feature maps being input into the preset decoder for target detection.

[0045] In addition, to achieve the above-mentioned purpose, the application further provides a pest detection device based on an improved RTDETR model, which comprises a memory, a processor, and a pest detection program based on an improved RTDETR model stored in the memory and executable on the processor, the pest detection program based on an improved RTDETR model being configured to implement the steps of the pest detection method based on an improved RTDETR model as described above.

[0046] The application discloses a pest detection method based on an improved RTDETR model, which comprises: obtaining a farmland image to be detected; inputting the farmland image to be detected into a preset pest detection model to obtain a pest detection result, the preset pest detection model comprising: a preset backbone network, a preset encoder and a preset decoder, the preset backbone network comprising integrated attention BasicBlock modules that replace a preset number of BasicBlock modules in an original RTDETR model, the preset encoder comprising an adaptive sparse self-attention module that replaces an original AIFI module, the preset encoder being configured to perform feature fusion on multi-scale feature maps obtained through the preset backbone network, and the obtained fused feature maps being input into the preset decoder for target detection.

[0047] Due to the introduction of the integrated attention mechanism BasicBlock module in the backbone network, the model's attention ability to key features is enhanced, effectively improving the detection accuracy of small pest targets. At the same time, the adaptive sparse self-attention module optimizes the feature fusion process, enabling the model to better capture pest features and reducing the miss rate and false detection rate. In addition, the improved model structure significantly reduces the computational load and parameter amount while maintaining high detection accuracy, improving the model's running efficiency and making it more suitable for real-time farmland pest detection scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application together with the specification.

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, for those of ordinary skill in the art, the other drawings can also be obtained based on these drawings without any creative effort.

[0050] Figure 1 Flowchart of the first embodiment of the pest detection method based on the improved RTDETR model of the present application;

[0051] Figure 2 Flowchart of the construction process of the preset pest detection model of the present application;

[0052] Figure 3 Flowchart of the second embodiment of the pest detection method based on the improved RTDETR model of the present application;

[0053] Figure 4 Model structure diagram of the preset pest detection model of the present application;

[0054] Figure 5 Module structure diagram of the integrated attention BasicBlock module;

[0055] Figure 6 Flowchart of the third embodiment of the pest detection method based on the improved RTDETR model of the present application;

[0056] Figure 7 Module structure diagram of the ASSA module;

[0057] Figure 8 Module structure diagram of the pest detection device based on the improved RTDETR model of the present application;

[0058] Figure 9 Structure diagram of the pest detection device based on the improved RTDETR model of the present application.

[0059] The object implementation, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application, and are not used to limit the present application.

[0061] In order to better understand the technical solutions of the present application, the present application will be described in detail with reference to the accompanying drawings and specific embodiments.

[0062] The embodiment of the application provides a pest detection method based on an improved RTDETR model Figure 1 , Figure 1 The embodiment of the application provides a pest detection method based on an improved RTDETR model

[0063] Step S10: acquiring a to-be-detected farmland image.

[0064] It should be noted that the execution subject of the embodiment can be a computing service device that needs to have image acquisition, data processing, model calling, network communication and program running functions, for example, a mobile phone, a personal computer, a pest detection instrument, an agricultural machine device and the like, and can also be other electronic devices that can access the pest detection system. The embodiment is not limited in this regard. In this regard, a pest detection device based on an improved RTDETR model (referred to as a detection device) is selected as an example to describe the embodiment and each of the following embodiments.

[0065] It should be understood that the to-be-detected farmland image can be a real-time photographed farmland image, or a historical photographed image that needs to be detected from the network or a historical database. The embodiment is not limited in this regard.

[0066] In a specific implementation, in an actual farmland scene, an image acquisition device such as a camera can be used to photograph the area in the farmland scene that needs to be detected in real time to obtain the to-be-detected farmland image.

[0067] Step S20: inputting the to-be-detected farmland image into a preset pest detection model to obtain a pest detection result, wherein the preset pest detection model comprises a preset backbone network, a preset encoder and a preset decoder, the preset backbone network comprises integrated attention BasicBlock modules that replace a preset number of BasicBlock modules in an original RTDETR model, the preset encoder comprises an adaptive sparse self-attention module that replaces an original AIFI module, the preset encoder is used for performing feature fusion on a multi-scale feature map obtained through the preset backbone network, and the obtained fused feature map is input into the preset decoder for target detection.

[0068] It should be noted that the preset pest detection model can be constructed based on an original RTDETR model (for example, RTDETR-R18). The RTDETR is improved on the basis of the DETR framework, and the anchor box and the non-maximum suppression (NMS) mechanism in the traditional target detection are abandoned. The target position and the category are directly predicted through the Transformer encoder-decoder, and the post-processing complexity is effectively reduced.

[0069] The original RTDETR model can be composed of a backbone network, an efficient hybrid encoder, and a transformer decoder.

[0070] The backbone network can perform multi-scale feature extraction on the input image by using a residual network (e.g., ResNet18). The efficient hybrid encoder can decompose complex multi-scale feature interaction into two relatively independent steps: intra-scale feature interaction and cross-scale feature fusion, thereby efficiently processing and fusing multi-scale features from the backbone network. The transformer decoder can decode the encoded features into target instances based on the standard transformer decoder structure, combined with an IoU-aware query selection mechanism, and then output the corresponding predicted target class and bounding box.

[0071] It should be understood that, in order to address the problem that the original RTDETR-R18-based agricultural pest detection method is difficult to balance detection accuracy, speed, and lightweight requirements in complex farmland environments, the preset pest detection model is improved as follows:

[0072] In the preset backbone network, a certain number of BasicBlock modules are replaced with integrated attention BasicBlock modules (BasicBlock_Attention modules) compared to the original RTDETR model to enhance the attention ability to key features of the image and improve the small target detection capability.

[0073] In the preset encoder, the AIFI module in the original RTDETR model is replaced with an adaptive sparse self-attention module (TransformerEncoderLayer_ASSA module, referred to as ASSA module). This allows the encoder to more effectively fuse multi-scale feature maps, focus on key feature information through the adaptive sparse self-attention (ASSA) mechanism, and improve the quality and efficiency of feature expression.

[0074] In addition, compared to the original RTDETR model, the preset pest detection model can also add a 1 / 4 resolution P2 branch as a small target dedicated detection path (achieved through downsampling and channel alignment) to effectively enhance the detection capability of small target objects.

[0075] Compared with the original RTDETR model, the preset pest detection model can also use a RepC3 unit to replace a standard C3 unit to realize structure reparameterization.

[0076] In a specific implementation, the original RTDETR model is improved based on the above improvement process to obtain an improved model, and then the improved model can be trained and optimized through an agricultural pest detection data set, and finally a preset pest detection model for pest detection is obtained.

[0077] Further, in order to specifically illustrate the model training and verification process, before step S20, steps S01-S02 are further included.

[0078] Step S01: Obtain a multi-scene agricultural pest data set, and pre-process the multi-scene agricultural pest data set to obtain a target data set.

[0079] It should be understood that first, agricultural pest image data from different scenes can be collected, which can cover various crop planting areas, different climate conditions, shooting angles, lighting environments, and different growth periods of pests, etc., to ensure the diversity and representativeness of the data set. The collected original data can come from various channels such as field shooting, data sharing of research institutions, or historical pest monitoring records.

[0080] Then, the collected agricultural pest data set can be pre-processed using image enhancement techniques, including rotation, scaling, translation, brightness adjustment, etc., to improve data diversity and enhance model generalization ability.

[0081] It should be noted that after the above pre-processing step, the original multi-scene agricultural pest data set is converted into a target data set that has been sorted, cleaned, labeled, and possibly enhanced. This target data set will be used as input data for model training, and is directly related to the performance and accuracy of the preset pest detection model.

[0082] Step S02: Train the initialized pest detection model through the target data set to obtain a preset pest detection model.

[0083] It should be noted that in order to facilitate subsequent model training and verification, the obtained target data set can also be divided according to a proportion, for example, divided into a training set and a test set according to a 7:3 proportion.

[0084] It should be understood that the initialized pest detection model is the model obtained by improving the original RTDETR model as described above. The initial values of various parameters in the improved model (such as the weights of the convolutional layer, the weights of the fully connected layer, and the parameters in the attention module) can be set first, usually using random initialization or using the parameters of a pre-trained model as the initialization parameters to provide a starting point for model training.

[0085] Then, the hardware environment (such as GPU acceleration devices) and software environment (such as deep learning frameworks, programming languages, etc.) required for model training can be set, and the hyperparameters in the training process are determined, such as the learning rate (which controls the size of the model parameter update step), the batch size (the number of image samples used for each training iteration), the number of training rounds (the number of times the model is trained on the entire dataset), etc.

[0086] Then, the training set is input into the initialized pest detection model, so that the model gradually learns how to accurately detect pest targets from images and classify them. During the training process, a loss function (such as a classification loss or a positioning loss) can be used to measure the gap between the model's prediction results and the true labels, and the model parameters are updated by minimizing the loss function. As the training continues, the detection accuracy and performance of the model will gradually improve, and eventually a trained pest detection model is obtained.

[0087] Finally, the trained pest detection model is verified using the validation set to evaluate its performance on unseen data, such as accuracy, recall, mean average precision (mAP), etc. to determine whether the trained pest detection model meets the pre-set performance conditions. When the model training is completed and the performance indicators are satisfactory, the trained model parameters and structure are saved to form a pre-set pest detection model.

[0088] For example, if the original RTDETR model is RTDETR-R18, the steps to obtain the pre-set pest detection model of the present application by improving the original model can be as shown in Figure 2 . Figure 2 The construction process of the pre-set pest detection model of the present application is shown in the flowchart.

[0089] As can be seen from Figure 2 , first, a multi-scenario agricultural pest dataset can be obtained, and the multi-scenario agricultural pest dataset can be preprocessed to obtain a target dataset. Then, the target dataset is preprocessed and divided into a training set and a validation set. The original RTDETR-R18 model is structurally improved according to the aforementioned improvement process to obtain an improved model, which is an initialized pest detection model. Finally, the improved model is trained and verified using the training set and the validation set, and a pre-set pest detection model is finally obtained.

[0090] This example introduces the BasicBlock_Attention module into the backbone network, enhancing the original RTDETR model's ability to focus on key features and effectively improving the detection accuracy of tiny pest targets. The ASSA module also optimizes the feature fusion process, enabling the model to better capture pest characteristics and reducing missed detection and false detection rates. Furthermore, while maintaining high detection accuracy, the improved model structure significantly reduces computational complexity and parameter requirements, improving model efficiency and making it more suitable for real-time agricultural pest detection scenarios.

[0091] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 3 , Figure 3 This is a flow chart of the second embodiment of the pest detection method based on the improved RTDETR model of this application.

[0092] In this embodiment, in order to specifically illustrate the specific processing process of the preset pest detection model on the input image, step S20 includes: steps S201 to S203:

[0093] Step S201: extracting features of the farmland image to be detected through the preset backbone network to obtain the multi-scale feature map, which includes: a P2 feature map, a P3 feature map, a P4 feature map, and a P5 feature map.

[0094] It should be noted that the preset skeleton network can perform layer-by-layer processing and feature extraction of the farmland image to be detected through operations such as convolutional layers, pooling layers, and improved integrated attention BasicBlock modules to obtain feature maps of multiple scales: P2 feature map, P3 feature map, P4 feature map, and P5 feature map.

[0095] Compared with the original RTDETR-R18 model, the multi-scale feature map output by this preset pest detection model also includes the P2 feature map, which effectively enhances the detection capability of small target objects: the P2 feature map has a higher resolution and can capture smaller-scale detail information in the image, such as the local features of tiny pests.

[0096] As the scale of the feature map decreases from P2 to P5, the corresponding semantic level gradually increases, reflecting a wider range of contextual information and more abstract object features in the image. For example, although the P5 feature map has a lower resolution, it contains the overall structure and semantic information of large-scale objects in the image, which helps the model understand the distribution of pests throughout the image.

[0097] Furthermore, this can be combined with Figure 4 The process of extracting multi-scale feature maps using the preset backbone network is described in detail. Figure 4 This is a schematic diagram of the model structure of the pest detection model preset in this application. Step S201 specifically includes: Steps S2011 to S2016:

[0098] Step S2011: performing initial feature extraction on the farmland image to be detected through the convolution and normalization layer to obtain basic image features.

[0099] Depend on Figure 3 It can be seen that the preset backbone network also includes: three convolution and normalization layers (ConvNormLayer), a maximum pooling layer (Maxpool) and two BasicBlock modules connected in sequence. The second BasicBlock module is also connected to the first BasicBlock_Attention module, and the second BasicBlock_Attention module is respectively connected to the first BasicBlock_Attention module and the preset encoder.

[0100] Among them, the convolution and normalization layer (ConvNormLayer) is obtained by combining the convolution layer (Conv) and the normalization layer (Norm). The model can first perform initial feature extraction on the input farmland image to be detected through multiple (for example, 3 in the figure) ConvNormLayers to obtain the basic features of the image, such as edges, textures, etc.

[0101] Step S2012: down-sample the basic features of the image through the maximum pooling layer to obtain a P2 feature map.

[0102] It should be understood that after the convolution operation, the maximum pooling layer (Maxpool) is used to downsample the feature map to reduce the spatial size of the feature map while retaining the most important feature information, which helps to reduce the computational complexity and improve the robustness of the model to image scale changes.

[0103] Step S2013: extract features from the P2 feature map through the first BasicBlock module to obtain a P3 feature map.

[0104] Step S2014: extract detail features from the P3 feature map through the second BasicBlock module to obtain a detail feature map.

[0105] It can be understood that inputting the P2 feature map into the first BasicBlock module for further feature extraction can learn higher-level feature representations, thereby obtaining more abstract P3 feature maps highlighting important structural information in the image.

[0106] Subsequently, the second BasicBlock module receives the P3 feature map and continues to perform feature extraction to mine deeper-level detailed features, generate a detailed feature map, and enhance the ability to capture complex patterns in the image.

[0107] Step S2015: performing first feature extraction on the detailed feature map based on the integrated channel and spatial attention mechanism through the first integrated attention BasicBlock module to obtain a P4 feature map.

[0108] Step S2016: performing first feature extraction on the detailed feature map based on the integrated channel and spatial attention mechanism through the first integrated attention BasicBlock module to obtain a P5 feature map.

[0109] Finally, the detailed feature map is passed to the first BasicBlock_Attention module, which performs feature enhancement on the detailed feature map through the integrated channel and spatial attention mechanism. The attention mechanism enables the network to focus on more discriminative regions in the image. Thus, the first BasicBlock_Attention module obtains a P4 feature map, and the second BasicBlock_Attention module obtains a P5 feature map.

[0110] The P2-P5 feature maps obtained by the above preset backbone network in turn are beneficial to further improve the expression ability of the features and the detection accuracy of the pest target.

[0111] Further, reference can also be made to Figure 5 The processing process of the input image by the integrated attention BasicBlock module in the preset pest detection model will be described in detail, Figure 5 which is a module structure diagram of the integrated attention BasicBlock module.

[0112] As can be seen from Figure 5 The integrated attention BasicBlock module (BasicBlock_Attention module) includes a main path and an attention branch.

[0113] The main path is composed of two convolution groups connected in series, and each convolution group includes a preset convolution layer (3x3 Conv), a batch normalization layer (BN), and a ReLU activation function layer connected in series.

[0114] The attention branch includes, in sequence, a global average pooling layer (GAP), a fully connected layer (FC1 and FC2 connected through a ReLU activation function layer), and a Sigmoid activation function layer; the ReLU activation function layer in the first convolution group on the main path is also connected with the global average pooling layer (GAP) on the attention branch, and the preset convolution layer (3x3Conv) in the second convolution group on the main path.

[0115] It should be noted that, similar to the original BasicBlock module, in the BasicBlock_Attention module, the main path can also include two 3x3Conv, each of which is connected with a batch normalization (BN) and a ReLU activation function in sequence. The main function of this path is to extract the basic features of the image.

[0116] The convolution operation formula of the main path is as follows:

[0117] Fmain=BN(ReLU(BN(Conv3×3(X)))

[0118] In the formula, X represents the input feature map, Conv3x3 represents the 3x3 convolution operation, BN represents the batch normalization operation, and ReLU represents the activation function. In order to simplify the representation, the second 3x3 convolution layer and the subsequent batch normalization operation are omitted here.

[0119] In addition, in the BasicBlock_Attention module, the attention branch first performs global average pooling (GAP) on the output of the second convolution layer in the main path, compressing the three-dimensional feature map (CxHxW) into a one-dimensional vector (C). Subsequently, a dimension reduction operation is performed through a fully connected layer (FC1), followed by a nonlinear transformation using a ReLU activation function, and then another fully connected layer (FC2) is used to restore the original channel number C. Finally, the Sigmoid activation function is used to generate the attention weight of each channel.

[0120] The formula of the attention branch is as follows:

[0121] z=GAP(Fmain)

[0122] A=σ(W2δ(W1z))

[0123] In the formula, GAP represents the global average pooling operation, W1 and W2 are the weight matrices of the two fully connected layers, δ represents the ReLU activation function, and σ is the Sigmoid activation function.

[0124] Finally, feature fusion is performed in the BasicBlock_Attention module: the attention weight is multiplied with the output of the main path channel by channel to obtain the weighted feature map. The weighted feature is added to the input feature through residual connection to generate the final output feature Y.

[0125] wherein the feature fusion formula can be expressed as follows:

[0126] Y = A O Fmain + X

[0127] In the formula, O represents a channel-by-channel multiplication operation, and Y represents the final output feature map.

[0128] After feature fusion, the features of important channels are amplified, and the features of unimportant channels are suppressed, so that the model can focus on more meaningful feature information; the residual connection can alleviate the gradient vanishing problem in the deep network, facilitate the training and optimization of the model, and at the same time retain the original information in the input feature, avoid information loss, and enhance the feature expression ability of the model.

[0129] The BasicBlock_Attention module effectively improves the attention ability of the model to key features by combining the basic feature extraction of the main path and the channel attention enhancement of the attention branch, especially when dealing with small pest targets and complex farmland backgrounds, it can significantly improve the discrimination and expression ability of the features, and thus improve the precision and performance of pest detection.

[0130] In specific implementation, the farmland image to be detected is input into a preset backbone network for multi-scale feature extraction and conversion into a series of feature maps (Feature Maps) with different spatial resolutions and semantic levels: in the process, the P2 feature map corresponds to the feature map after three ConvNormLayer and one MaxPool. The feature map is located after the MaxPool layer, and the size is reduced to 1 / 4 of the original image; the P3 feature map is the feature map after two times of BasicBlock processing, located between two Blocks layers, and the feature map size is further reduced to 1 / 8 of the original image; P4 is the feature map after two times of BasicBlock_Attention module processing, located in the subsequent Blocks layer, and the feature map size is reduced to 1 / 16 of the original image; P5 is also the feature map after two times of BasicBlock_Attention processing, but located in a deeper network layer, and the feature map size is finally reduced to 1 / 32 of the original image. These feature maps (P2-P5 feature maps) capture the information of the image at different scales, providing rich input for the subsequent detection module.

[0131] Step S202: performing feature fusion on the multi-scale feature maps through the preset encoder to obtain a fused feature map.

[0132] It should be understood that the preset encoder plays a key role in the model in feature fusion of multi-scale feature maps. Since different scale feature maps each contain unique information, but at the same time there may be problems of incomplete information or large semantic gap, the multi-scale feature maps (P2-P5 feature maps) can be effectively fused by the preset encoder to generate a more rich, complete and strong expression ability of the fusion feature map.

[0133] It should be further pointed out that the preset encoder can be an efficient hybrid encoder that replaces the AIFI module in the original RTDETR model with the ASSA module.

[0134] In specific implementation, after the preset encoder receives the P2-P5 feature maps extracted from the backbone network, it can use key components such as the ASSA module to interact and fuse the feature maps across scales. The ASSA module can dynamically adjust the correlation weights between features of different positions and scales, focus more on the feature information related to the pest detection task, and suppress some irrelevant or potentially interfering features.

[0135] In the above manner, the preset encoder can integrate the complementary information in the multi-scale feature maps to generate a fusion feature map that not only retains the key details of each scale in the image, but also enhances the semantic information of the features, so that the subsequent target detection can more accurately identify the pest target.

[0136] Step S203: performing target detection on the fusion feature map by the preset decoder to obtain a pest detection result.

[0137] It should be understood that the preset decoder can determine whether there is a pest in the image, the specific position of the pest (usually represented in the form of a bounding box), and the category of the pest, etc. detailed information based on the rich feature information contained in the fusion feature map through a series of prediction and regression operations, so as to realize accurate pest target detection.

[0138] In a specific implementation, after the preset decoder receives the fusion feature map generated by the encoder, the fusion feature map can be further processed and converted first to adapt to the requirements of the target detection task. Then, through the prediction network and other structures inside the decoder, each position and possible scale in the fusion feature map is predicted to generate a series of candidate pest detection bounding boxes and corresponding class probabilities. These bounding boxes and class probabilities will be matched and optimized with the features in the fusion feature map according to the parameters and feature patterns learned by the model, and finally the position and class information most likely to represent the real pest target are selected to form the final pest detection result. These detection results will be output in a structured manner, facilitating users or subsequent decision systems to analyze and process the pest situation in the farmland.

[0139] In this embodiment, through the feature extraction of the preset backbone network, the feature fusion of the preset encoder, and the target detection of the preset decoder, the entire preset pest detection model can efficiently and accurately detect the pest target from the image of the farmland to be detected and output detailed detection results, providing strong support for the monitoring and prevention of agricultural pests.

[0140] Based on the first and second embodiments of the present application, in the third embodiment of the pest detection method based on the improved RTDETR model of the present application, the same or similar contents as the above-mentioned first and second embodiments can be referred to the above introduction, and will not be repeated hereinafter. On this basis, please refer to Figure 6 , Figure 6 is a flowchart of the third embodiment of the pest detection method based on the improved RTDETR model of the present application.

[0141] In this embodiment, in order to specifically illustrate the image processing process of the preset encoder in the preset pest detection model, step S202 specifically includes steps S2021-S2023:

[0142] Step S2021: uniform the channels of the multi-scale feature maps through the input projection module to obtain the multi-scale feature maps after channel unification.

[0143] It should be noted that reference can also be made to the above Figure 3 The process of realizing multi-scale feature fusion by the preset encoder is described.

[0144] As Figure 4 can be seen, the preset encoder further includes an input projection module and a preset CCFM module, and the input projection module can be represented as a series of parallel convolution layers (Conv). Each parallel convolution layer (Conv) corresponds to a feature map (P2, P3, P4, or P5 feature map) of one scale output by the aforementioned preset backbone network.

[0145] In a specific implementation, after projection, the channel numbers of the P2, P3, P4 and P5 feature maps are unified (for example, unified to 256), thereby obtaining the P2, P3, P4 and P5 feature maps after channel unification.

[0146] Step S2022: performing feature enhancement on the multi-scale feature map after channel unification by the adaptive sparse self-attention module, to obtain a target multi-scale feature map.

[0147] It should be noted that, similar to the original AIFI module, the TransformEncoderLayer_ASSA module (i.e., the ASA module) can only act on the highest layer semantic feature P5 feature map after input projection. Therefore, the target multi-scale feature map can be the P5 feature map after feature enhancement.

[0148] In addition, since the core function of the ASSA enhancement mechanism is to perform sparse enhancement processing on the output of the self-attention mechanism unit, thereby improving the representation ability of the key features. Here, reference can also be made to Figure 6 The enhancement mechanism of the ASSA module is described, Figure 7 The structure of the ASSA module is shown in FIG. 6.

[0149] As can be seen from Figure 7 It can be seen that the ASSA module includes: a multi-head self-attention mechanism unit (Self-Attention Mechanism), a sparse enhancement unit (ASSA Enhancement), an FFN unit and a layer normalization unit (Layer Normalization) connected in sequence.

[0150] After the P5 feature map is obtained from the second BasicBlock_Attention module in the preset backbone network and is subjected to channel unification by a convolution layer (Conv), the P5 feature map after channel unification can be input to the ASSA module for feature enhancement.

[0151] Specifically, in the ASSA module, the position-dependent relationship of the multi-scale feature map after channel unification (here, the P5 feature map after channel unification is taken as the input feature) can be extracted through a multi-head self-attention mechanism unit (Self-Attention Mechanism), and attention weights can be generated according to the extraction result; then, the attention weights can be adjusted according to the sparse mask through a sparse enhancement unit (ASSA Enhancement) to obtain sparse-enhanced attention weights; then, the sparse-enhanced attention weights can be nonlinearly transformed through an FFN unit to obtain target feature representation; finally, the target feature representation can be normalized, and the normalized target feature can be added to the multi-scale feature map after channel unification to obtain a target multi-scale feature map as the output feature.

[0152] In the above process, an additional network structure (such as a convolutional layer or a fully connected layer) can be used to generate a sparse mask M for screening important attention connections. Therefore, the sparse mask M can be applied to the attention weights to generate sparse-enhanced attention weights, which can be expressed as follows:

[0153] Adjusted Attention=Attention⊙M

[0154] In the formula, represents an element-level multiplication operation.

[0155] In a specific implementation, after obtaining the P5 feature map enhanced by the above feature, the P2, P3, and P4 feature maps after channel unification mentioned above can be added to the P5 feature map to obtain a target multi-scale feature map as the input of the preset CCFM module. Figure 1

[0156] Step S2023: performing feature fusion on the target multi-scale feature map through the preset CCFM module to obtain a fused feature map.

[0157] It should be noted that the fused feature map obtained by the preset CCFM module is also the output of the ASSA module. Figure 4 It can be known that the preset CCFM module can replace the original CCFM module of the RepC3 unit instead of the standard C3 unit. The CCFM module adopts a fusion strategy based on a convolutional neural network (CNN), and its structure is similar to that of an FPN or a PANet, including a series of convolutional layers (Conv), up-sampling layers (Upsample), and feature connection layers (Contact), so as to realize multi-scale feature information fusion.

[0158] ​The output of the preset CCFM module is a set of new feature maps fused with multi-scale information, i.e., the above-mentioned fusion feature maps are P2, P3, P4 and P5 feature maps after fusion. Compared with the original P2, P3, P4 and P5 feature maps, the fusion feature maps have stronger comprehensive representation capability and provide high-quality input for the subsequent Transformer decoder.

[0159] Further, the preset decoder also includes Figure 4 It can be seen that the preset adder can include an IoU-aware query selection mechanism module (IOU-aware query selection), a preset RETETR decoder (RETETR Decoder) and a preset detection head (omitted in the figure). In order to illustrate the image processing process of the preset decoder in the preset pest detection model, step S203, specifically includes steps S2031-S2033:

[0160] Step S2031: The initial object is obtained by performing initial query on the fusion feature map through the IoU-aware query selection mechanism module.

[0161] It should be noted that the core idea of the IoU-aware query selection mechanism is that not only the classification score is relied on to select the query, but also a specific constraint is imposed on the model during the training process, so that the model can learn to assign a higher classification confidence to the features with a higher Intersection over Union (IoU) between the predicted bounding box and the real target. In this way, when selecting the Top-K features from the feature sequence (P2, P3, P4 and P5 feature maps after fusion) output by the encoder as the initial object query, these queries not only have a higher classification score, but also their corresponding initial position prediction can be closer to the real target.

[0162] Step S2032: The multi-scale query vector is obtained by performing accurate query on the fusion feature map through the preset RETETR decoder in combination with the initial object.

[0163] It should be noted that the decoder of the RTDETR adopts a standard Transformer decoder structure, and its main task is to receive the initial object query from the IoU-aware query selection mechanism and combine the P2, P3, P4 and P5 feature maps after fusion output by the aforementioned preset encoder, and finally output a query vector capable of accurately describing the target position and category through multi-layer iterative optimization.

[0164] Step S2033: The pest detection result is obtained by mapping the multi-scale query vector through the preset detection head.

[0165] It should be understood that after the preset RETER decoding module iteratively optimizes the multi-layer Transformer decoder, the output multi-scale query vector already contains rich information about the potential target in the image. The detection head (Prediction Heads) can "translate" these high-dimensional query vectors into the final, human-understandable detection results: map the query vector to a class probability distribution vector, and map the query vector to parameters describing the bounding boxes of the object to describe the location of the pest target.

[0166] In a specific implementation, the preset decoder is based on the RTDETR architecture, and uses the fusion features generated by the aforementioned encoder and the selected query to predict the position (bounding boxes), class, and corresponding confidence score (confidence scores) of the pest target. Finally, the pest detection result is output by the detection head, including the position, class, and confidence of the pest, and the like.

[0167] The embodiment embeds the TransformerEncoderLayer_ASSA structure in the backbone feature extraction process, further improving the efficiency and accuracy of feature processing. And for the small target detection task, a P2 branch is specially designed and added, and combined with the RTDETR Decoder containing the P2 branch, the detection ability of small target objects is effectively enhanced, and more reliable pest detection results are obtained, thereby better meeting the needs of real-time detection tasks in agricultural machinery devices.

[0168] Further, in order to compare the performance of the preset pest detection model improved by the original RTDETR model and the original RTDETR model, a comparative experiment can be used to verify the performance of the two models.

[0169] For example, in the experiment, the RTDETR model can be an RTDETR-R18 model, i.e. the original R18 model, and the preset pest detection model of the present application can be represented as an improved R18 model. The original experiment can be performed using an NVIDIA 4070ti 16G GPU, with 150 rounds of training, a batch size of 16, and an input image resolution of 1024x1024. The computational complexity (Flops), recall (Recall, R), parameter quantity (Param), mean average precision (Mean Average Precision, MAP), and precision (Precision, P) can be selected as performance evaluation parameters.

[0170] Based on the experimental data, the table obtained is shown in Table 1 below, which is a model performance evaluation table.

[0171] Table 1 model performance evaluation table

[0172] Network Model Flops R Params MAP (50)% Original R18 model 57G 0.92 19.8M 87.7 Improved R18 model 47.7G 0.92 19.7M 88.4

[0173] Based on the data in Table 1, the improved R18 model proposed in this embodiment has significant advantages in key performance indicators such as calculation efficiency, parameter size and detection accuracy. Specifically, the calculation amount of the model is optimized from 57G to 47.7G, the parameter amount is reduced from 19.8M to 19.7M, and the average precision mean (MAP50) is improved from 87.7% to 88.4%.

[0174] Experimental data fully confirm that the improved R18 model proposed in this embodiment significantly improves the detection performance while achieving dual reduction of calculation amount and parameter amount, especially in the small object detection task. Therefore, applying the improved R18 model to the agricultural pest detection task can effectively solve the pest detection problems in complex farmland environments such as leaf occlusion and uneven illumination.

[0175] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the pest detection device method based on the improved RTDETR model of the present application. More forms of simple transformation based on this technical concept are within the protection scope of the present application.

[0176] The present application also provides a pest detection system based on an improved RTDETR model, please refer to Figure 8 , Figure 8 The present application based on the module structure diagram of the pest detection device of the improved RTDETR model, the device comprises:

[0177] The image acquisition module 801 is used for acquiring the farmland image to be detected;

[0178] The pest detection module 802 is used for inputting the farmland image to be detected into a preset pest detection model to obtain a pest detection result, the preset pest detection model comprising: a preset backbone network, a preset encoder and a preset decoder, the preset backbone network comprising integrated attention BasicBlock modules replacing a preset number of BasicBlock modules in the original RTDETR model, the preset encoder comprising an adaptive sparse self-attention module replacing the original AIFI module, the preset encoder being used for feature fusion on the multi-scale feature maps obtained through the preset backbone network, and the obtained fusion feature maps being input into the preset decoder for target detection.

[0179] The embodiment introduces a BasicBlock module with an integrated attention mechanism into the backbone network, enhances the model's attention to key features, effectively improves the detection accuracy of small pest targets, and optimizes the feature fusion process with the adaptive sparse self-attention module, so that the model can better capture pest features and reduce the missed detection rate and false detection rate. In addition, the improved model structure significantly reduces the computational complexity and parameter amount while maintaining high detection accuracy, improves the running efficiency of the model, and is more suitable for real-time pest detection scenarios in farmland.

[0180] In addition, the present application also provides a pest detection device based on an improved RTDETR model. The pest detection device based on the improved RTDETR model comprises at least one processor and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the pest detection method based on the improved RTDETR model in the above-mentioned embodiment one.

[0181] Reference will be made to the following Figure 9 , Figure 9 FIG. 1 is a structural schematic diagram of a pest detection device based on an improved RTDETR model according to an embodiment of the present application. The pest detection device based on the improved RTDETR model in the embodiment of the present application can include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. Figure 9 The pest detection device based on the improved RTDETR model shown is only an example and should not impose any limitation on the functions and use range of the embodiments of the present application.

[0182] As Figure 9As shown, the pest detection device based on the improved RTDETR model can include a processor 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1002 or loaded from a storage device 1003 into a random access memory (RAM) 1004. In the RAM 1004, various programs and data required for the operation of the pest detection device based on the improved RTDETR model are also stored. The processor 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. In general, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; the storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the pest detection device based on the improved RTDETR model to communicate with other devices wirelessly or by wire to exchange data. Although the pest detection device based on the improved RTDETR model with various systems is shown in the figure, it should be understood that all the systems shown are not required to be implemented or possessed. More or fewer systems can be alternatively implemented or possessed.

[0183] The above-mentioned embodiment numbers of the application are only for description, not representing the advantages and disadvantages of the embodiments, and are only part of the embodiments of the application, and do not limit the patent scope of the application. Any equivalent structural transformation based on the technical concept of the application, or direct / indirect application in other related technical fields is included in the patent protection scope of the application.

Claims

1. A pest detection method based on an improved RTDETR model, characterized in that: The method comprises: Obtain the farmland image to be detected; The farmland image to be detected is input into a preset pest detection model to obtain a pest detection result. The preset pest detection model includes: a preset backbone network, a preset encoder and a preset decoder. The preset backbone network includes an integrated attention BasicBlock module that replaces a preset number of BasicBlock modules in the original RTDETR model. The preset encoder includes an adaptive sparse self-attention module that replaces the original AIFI module. The preset encoder is used to perform feature fusion on the multi-scale feature map obtained through the preset backbone network, and input the obtained fused feature map into the preset decoder for target detection.

2. The method according to claim 1, wherein The step of inputting the farmland image to be detected into a preset pest detection model to obtain the pest detection result includes: Performing feature extraction on the farmland image to be detected through the preset backbone network to obtain the multi-scale feature map, wherein the multi-scale feature map includes: a P2 feature map, a P3 feature map, a P4 feature map, and a P5 feature map; Performing feature fusion on the multi-scale feature map through the preset encoder to obtain a fused feature map; The preset decoder is used to perform target detection on the fused feature map to obtain a pest detection result.

3. The method according to claim 2, wherein The preset backbone network further includes: a convolution and normalization layer, a maximum pooling layer, and two BasicBlock modules connected in sequence, the second BasicBlock module being further connected to the first integrated attention BasicBlock module; The step of extracting features from the farmland image to be detected through the preset backbone network to obtain the multi-scale feature map includes: Performing initial feature extraction on the farmland image to be detected through the convolution and normalization layers to obtain basic image features; Downsampling the basic features of the image through the maximum pooling layer to obtain a P2 feature map; Perform feature extraction on the P2 feature map through the first BasicBlock module to obtain a P3 feature map; Performing detail feature extraction on the P3 feature map through the second BasicBlock module to obtain a detail feature map; Performing a first feature extraction based on an integrated channel and spatial attention mechanism on the detail feature map through the first integrated attention BasicBlock module to obtain a P4 feature map; The first integrated attention BasicBlock module is used to extract the first feature of the detail feature map based on the integrated channel and spatial attention mechanism to obtain the P5 feature map.

4. The method according to claim 3, wherein The integrated attention BasicBlock module includes a main path and an attention branch; The main path consists of two connected convolution groups, each of which includes: a preset convolution layer, a batch normalization layer, and a ReLU activation function layer connected in sequence; the attention branch includes a global average pooling layer, a fully connected layer, and a Sigmoid activation function layer connected in sequence; The ReLU activation function layer in the first convolution group is also connected to the global average pooling layer and the preset convolution layer in the second convolution group.

5. The method according to claim 2, wherein The preset encoder further includes: an input projection module and a preset CCFM module, and the adaptive sparse self-attention module is connected to the input projection module and the preset CCFM module respectively; The step of performing feature fusion on the multi-scale feature map by the preset encoder to obtain a fused feature map includes: Performing channel unification on the multi-scale feature map by the input projection module to obtain the multi-scale feature map after channel unification; Performing feature enhancement on the multi-scale feature map after channel unification through the adaptive sparse self-attention module to obtain a target multi-scale feature map; The preset CCFM module is used to perform feature fusion on the target multi-scale feature map to obtain a fused feature map.

6. The method according to claim 5, wherein The adaptive sparse self-attention module includes: a multi-head self-attention mechanism unit, a sparse enhancement unit, a FFN unit and a layer normalization unit; The step of performing feature enhancement on the multi-scale feature map after channel unification by the adaptive sparse self-attention module to obtain a target multi-scale feature map includes: Extracting position dependencies on the multi-scale feature map after channel unification through the multi-head self-attention mechanism unit, and generating attention weights according to the extraction results; The sparse enhancement unit adjusts the attention weight according to the sparse mask to obtain the sparse enhanced attention weight; Performing a nonlinear transformation on the sparse enhanced attention weights through the FFN unit to obtain a target feature representation; A target multi-scale feature map is obtained by normalizing the target feature representation and adding the normalized target feature to the multi-scale feature map after channel unification.

7. The method according to claim 2, wherein The preset decoder includes: an IoU-aware query selection mechanism module, an original RETER decoding module, and a preset detection head; The step of performing target detection on the fused feature map by the preset decoder to obtain a pest detection result includes: Performing an initial query on the fused feature map through the IoU-aware query selection mechanism module to obtain an initial object; Performing precise query on the fused feature map in combination with the initial object through the preset RETER decoding module to obtain a multi-scale query vector; The multi-scale query vector is mapped by the preset detection head to obtain a pest detection result.

8. The method according to claim 1, wherein Before the step of inputting the farmland image to be detected into a preset pest detection model to obtain the pest detection result, the method includes: Acquire a multi-scenario agricultural pest dataset, and preprocess the multi-scenario agricultural pest dataset to obtain a target dataset; The initialized pest detection model is trained using the target data set to obtain a preset pest detection model.

9. A pest detection device based on an improved RTDETR model, characterized in that: The device comprises: An image acquisition module is used to acquire images of farmland to be detected; A pest detection module is used to input the farmland image to be detected into a preset pest detection model to obtain a pest detection result. The preset pest detection model includes: a preset backbone network, a preset encoder and a preset decoder. The preset backbone network includes an integrated attention BasicBlock module that replaces a preset number of BasicBlock modules in the original RTDETR model. The preset encoder includes an adaptive sparse self-attention module that replaces the original AIFI module. The preset encoder is used to perform feature fusion on the multi-scale feature map obtained through the preset backbone network, and input the obtained fused feature map into the preset decoder for target detection.

10. A pest detection device based on an improved RTDETR model, characterized in that: The device includes: a memory, a processor, and a pest detection program based on an improved RTDETR model stored in the memory and executable on the processor, wherein the pest detection program based on the improved RTDETR model is configured to implement the steps of the pest detection method based on the improved RTDETR model as described in any one of claims 1 to 8.

Images