Multi-band and multi-scale end-to-end pest detection method and electronic equipment

Abstract

The invention provides a multi-band and multi-scale end-to-end pest detection method and electronic equipment, and belongs to the technical field of computer vision. The method comprises the following steps: in an encoder, firstly performing feature fusion and dynamic multi-scale feature enhancement DMFE operation on multi-scale features from a high-dimensional direction to a low-dimensional direction to realize full fusion of high-dimensional features and low-dimensional features so as to enhance the multi-scale feature extraction capability of a model; meanwhile, in an encoder, wavelet transform is utilized to perform frequency band decomposition on features after preliminary fusion, and dynamic cross-scale convolution attention noise reduction DCCD calculation is performed for high-frequency sub-bands with relatively concentrated noise content, so that the multi-scale feature extraction capability of the model is further improved while noise interference is suppressed. According to the method, the background noise interference resistance and the multi-scale target detection capability of the pest detection model are effectively improved, so that the detection precision and generalization capability of the model are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a multi-band and multi-scale end-to-end pest detection method and electronic equipment. Background Art

[0002] In the field of plant protection, pest detection is a crucial task that directly affects the yield and quality of crops. With the rapid development of computer vision and deep learning technology, pest detection methods based on image processing have gradually become a research hotspot. Among them, convolutional neural networks (CNNs) have shown great potential in pest target detection due to their powerful feature extraction capabilities.

[0003] Currently, mainstream pest detection schemes usually follow the following process: first, CNNs are used to extract features from the input image containing pests to generate a series of candidate regions containing potential pest targets; then, the features of these candidate regions are classified and the borders are adjusted through classifiers and bounding box regressors to distinguish the pest categories and accurately locate their positions; finally, non-maximum suppression (NMS) technology is used to process overlapping candidate boxes to optimize the final detection results.

[0004] However, although this CNN-based approach has made some progress in pest detection, it still faces several challenges. Specifically, existing solutions often rely on general residual network structures in the feature extraction stage. Although these networks can effectively extract image features, they lack effective mechanisms to suppress background noise when processing pest images, and there is also the problem of insufficient multi-scale feature extraction. These problems have jointly affected the existing pest target detection model, resulting in its weak noise resistance, low detection accuracy, and limited generalization ability. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention proposes a multi-band and multi-scale end-to-end pest detection method to solve the technical problems existing in the prior art.

[0006] The technical solution adopted by the present invention is:

[0007] In a first aspect, a multi-band and multi-scale end-to-end pest detection method is provided, comprising:

[0008] The images containing insect pests are input into the residual network ResNet50 to extract the multi-scale features of the images;

[0009] The multi-scale features of the image are input into the multi-band multi-scale denoising encoder, and the multi-scale features are firstly fused and dynamically enhanced from high dimension to low dimension to obtain multi-dimensional fused features;

[0010] Performing wavelet transformation on the multi-dimensional fusion features to obtain low-frequency sub-bands and high-frequency sub-bands;

[0011] Input the high frequency sub-band into the dynamic cross-scale convolution attention denoising DCCD module to perform dynamic cross-scale convolution attention denoising calculation;

[0012] The output after the DCCD module processing is subjected to feature fusion and feature re-enhancement processing with the low-frequency sub-band to complete the processing of the multi-band multi-scale dimension reduction sampling MMD module;

[0013] The output of the MMD module is subjected to dynamic multi-scale feature fusion and enhancement with high-dimensional features, and the enhanced features are flattened into sequences and concatenated, and then input into a decoding preprocessing module;

[0014] The output of the decoding preprocessing module is decoded by the decoder and used as the input of the prediction head to obtain the target detection result.

[0015] Furthermore, the multi-scale features are subjected to feature fusion and dynamic multi-scale feature enhancement from high dimension to low dimension to obtain multi-dimensional fusion features, specifically:

[0016] In the multi-band multi-scale denoising encoder, the high-dimensional features output by the last stage of ResNet50, Stage n, are input into a single-layer Transformer encoder for processing to obtain multi-scale feature enhancement within the high-dimensional features, and the enhanced features are scaled down.

[0017] The scale-restored features are upsampled and concatenated with the features output by the Stage n-1 of ResNet50, and input into the dynamic multi-scale feature fusion enhancement DMFE module for dynamic multi-scale feature fusion enhancement;

[0018] The fused and enhanced features are upsampled and concatenated with the features output from the Stage n-2 of ResNet50, input into the DMFE module, and then dynamic multi-scale feature fusion enhancement is performed again to obtain multi-dimensional fused features.

[0019] Furthermore, the high-dimensional features are input into a single-layer Transformer encoder for processing, including: position embedding of the features, multi-head self-attention mechanism calculation, and feedforward neural network processing.

[0020] Furthermore, the DMFE module automatically selects a corresponding multi-scale convolution kernel combination according to the height or width of the input feature, performs group convolution calculation and noise reduction processing, and performs multi-scale enhancement on the processed features to achieve dynamic multi-scale feature fusion enhancement of the input features.

[0021] Furthermore, the obtained low-frequency sub-band is a low-frequency sub-band with a scale reduced by half.

[0022] Furthermore, the DCCD module dynamically adjusts the combination of multi-scale convolution kernels according to the scale of the input features, generates weights using cross-channel linear transformation, multiplies the weights with the output of the grouped convolution, fully extracts the multi-scale target features in the input features, and combines the average and maximum values ​​to suppress noise, adds the output after multi-scale transformation and the output after noise reduction processing, and realizes dynamic cross-scale feature fusion and noise reduction.

[0023] Furthermore, the wavelet transform is a Haar wavelet transform.

[0024] Furthermore, the target detection result includes: pest category and location information.

[0025] In a second aspect, an electronic device is provided, including:

[0026] one or more processors;

[0027] A storage device for storing one or more programs;

[0028] When one or more programs are executed by one or more processors, the one or more processors implement a multi-band and multi-scale end-to-end pest detection method provided in the first aspect.

[0029] In a third aspect, a computer-readable storage medium storing a computer program is provided. When the computer program is executed by a processor, a multi-band and multi-scale end-to-end pest detection method provided in the first aspect is implemented.

[0030] It can be seen from the above technical solution that the beneficial technical effects of the present invention are as follows:

[0031] 1. When there is strong background noise interference in the input image, the present invention can effectively identify the pest category and its specific location in the image, showing excellent anti-noise interference ability and multi-scale feature extraction ability, thereby significantly improving the model's detection performance for multi-scale pest targets in complex environments.

[0032] 2. The present invention performs bidirectional feature fusion and dynamic multi-scale feature enhancement (DMFE) operations in the encoder in order to achieve full fusion of high-dimensional features and low-dimensional features, thereby enhancing the multi-scale feature extraction capability of the model. The wavelet transform is used to decompose the features after preliminary fusion into frequency bands, and dynamic cross-scale convolutional attention denoising (DCCD) calculation is performed on high-frequency sub-bands with relatively concentrated noise content, aiming to suppress noise interference while further improving the multi-scale feature extraction capability of the model.

[0033] 3. The present invention introduces the dynamic multi-scale feature fusion enhancement (DMFE) technology in the encoder to improve the quality of feature fusion. In order to achieve effective feature decomposition, Haar wavelet transform is used. Subsequently, dynamic cross-scale convolution attention noise reduction (DCCD) calculation is performed for the high-frequency sub-bands in the decomposition results to reduce the noise content and enhance the multi-scale feature extraction capability of the model. This series of operations ensures that the multi-scale features output by the encoder have high quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the prior art description. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0035] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;

[0036] Figure 2 This is a block diagram of a multi-band and multi-scale end-to-end pest detection network according to an embodiment of the invention.

[0037] Figure 3 This is a network block diagram of a multi-band and multi-scale downsampling module according to an embodiment of the invention. DETAILED DESCRIPTION

[0038] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0039] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0040] Example

[0041] This embodiment provides a multi-band multi-scale end-to-end pest detection method. Figure 1 As shown, including:

[0042] The image containing pests is input into the multi-band multi-scale end-to-end pest detection network M2DETR. The network block diagram is as follows Figure 2 As shown;

[0043] In this embodiment, the multi-band multi-scale end-to-end pest detection network includes:

[0044] Feature extraction module, which extracts key pest feature information from the input image, including the edge and texture of the pest;

[0045] The multi-band multi-scale noise reduction coding module receives the feature map from the feature extraction module, performs multi-scale feature fusion and denoising on it, and flattens and concatenates the feature maps after feature fusion and enhancement to generate coding output;

[0046] The decoding preprocessing module provides the decoder with high-quality pest feature input suitable for decoding, so that the decoder can more effectively recover the pest target information from the encoder;

[0047] The decoder module is responsible for decoding the specific content of the pest image from the encoded output;

[0048] The prediction head module receives the pest feature map output by the decoder and generates the final pest category and its location information.

[0049] In the multi-band multi-scale end-to-end pest detection network, the pest-containing image is input into the feature extraction module to extract the multi-scale features of the image and then sent to the multi-band multi-scale noise reduction encoder for encoding;

[0050] In this embodiment, the feature extraction module is the residual network ResNet50.

[0051] In this embodiment, during the encoding process, feature fusion and dynamic multi-scale feature enhancement are first performed on the multi-scale features from high dimension to low dimension to obtain multi-dimensional fusion features, specifically including:

[0052] The outputs of the last three stages (Stage 2, Stage 3, Stage 4) of the residual network 50 (ResNet50) are selected. First, the feature output of Stage 4 is sent to a single-layer Transformer encoder for processing to obtain multi-scale feature enhancement within the high-dimensional features, and the scale of the enhanced features is restored to match the scale before enhancement, so as to facilitate subsequent fusion with low-dimensional features; secondly, the scale-restored features are upsampled and concatenated with the features output from the Stage 3 stage of ResNet50, and input into the dynamic multi-scale feature fusion enhancement DMFE module for dynamic multi-scale feature fusion enhancement, thereby realizing dynamic multi-scale fusion enhancement of the input features; finally, the fused and enhanced features are upsampled and concatenated with the features output from the Stage 2 stage of ResNet50, and input into the DMFE module together, and dynamic multi-scale feature fusion enhancement is performed again to achieve further fusion with the low-dimensional output of the residual network 50 to obtain multi-dimensional fusion features.

[0053] In this embodiment, a single-layer Transformer encoder first embeds the input features of Stage 4 by position, then applies a multi-head self-attention mechanism for calculation, and finally processes it through a feedforward neural network to obtain multi-scale feature enhancement within the high-dimensional features.

[0054] In this embodiment, within the DMFE module, the module automatically selects the corresponding multi-scale convolution kernel combination according to the height or width of the input feature, performs group convolution calculation and noise reduction processing, and then performs multi-scale enhancement on the processed features to achieve dynamic multi-scale fusion enhancement of the input features.

[0055] The features enhanced by dynamic multi-scale fusion are input into the multi-band multi-scale dimension reduction sampling MMD module to achieve feature scale reduction, multi-scale feature enhancement and noise reduction processing. The network block diagram is as follows Figure 3 shown.

[0056] In this embodiment, inside the MMD module, the module performs wavelet transformation on the fused features to obtain a low-frequency subband LL and three high-frequency subbands HL, LH and HH with a half scale. In this embodiment, the wavelet transform is a Haar wavelet transform.

[0057] In order to effectively suppress image noise interference and further extract multi-scale features in the image, this embodiment uses the high-frequency sub-bands (HL, LH, HH) with higher noise content to input the dynamic cross-scale convolution attention denoising DCCD module to perform dynamic cross-scale convolution attention denoising (DCCD) calculation;

[0058] In this embodiment, the DCCD module dynamically adjusts the combination of multi-scale convolution kernels according to the scale of the input feature, generates weights using cross-channel linear transformation, and suppresses noise by combining the average and maximum calculations to achieve dynamic cross-scale feature fusion and noise reduction, specifically including: the DCCD module dynamically adjusts the combination of multi-scale convolution kernels according to the scale of the input feature map, and the multi-scale convolution kernel candidate set includes {1x1, 3x3, 5x5, 7x7}. After the dynamic multi-scale grouped convolution, DCCD uses a 1x1 convolution kernel to perform a cross-channel linear transformation on the features after grouped convolution to achieve effective fusion between features of each scale, and converts it into a weight between 0 and 1. Subsequently, the weight is multiplied by the output of the grouped convolution, so that the model can re-weight and sum between multiple scale feature maps according to the learned multi-scale weights, thereby obtaining a multi-scale feature output. In order to suppress noise interference, DCCD also calculates the average and maximum values ​​of the features after dynamic multi-scale grouped convolution in the channel dimension, so as to suppress the influence of noise points on the expression of effective features. Next, the output after the average and maximum value calculation is rebalanced through a 3x3 convolution kernel, and then the sigmoid function is applied to the original input features, so that the model can focus on the detailed features of the global range while paying attention to the expression of significant features, thereby effectively suppressing the noise interference in the input image. Finally, the output after multi-scale transformation is added to the output after denoising, ensuring that the model can obtain low-noise, high-quality feature output while obtaining multi-scale feature output.

[0059] The output after DCCD processing is fused with the low-frequency sub-band LL, and then the fused features are multi-scale enhanced. This process not only completes the operation of halving the input feature scale, but also improves the quality of multi-scale features, thereby completing the multi-band multi-scale downsampling (MMD) operation.

[0060] The output after MMD processing is combined with high-dimensional features for dynamic multi-scale feature fusion enhancement (DMFE), and then the enhanced features are flattened into sequences and concatenated, and finally sent to the decoding preprocessing module.

[0061] After decoding preprocessing and decoding operations, the decoded output is sent to the prediction head to obtain the pest category and location information, thereby completing the pest target detection.

[0062] In one embodiment, an electronic device is provided, including:

[0063] one or more processors;

[0064] A storage device for storing one or more programs;

[0065] When one or more programs are executed by one or more processors, the one or more processors implement a multi-band and multi-scale end-to-end pest detection method provided by the above embodiment.

[0066] In one embodiment, a computer-readable storage medium storing a computer program is provided. When the computer program is executed by a processor, a multi-band and multi-scale end-to-end pest detection method provided in the above embodiment is implemented.

[0067] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A multi-band and multi-scale end-to-end pest detection method, characterized in that: include: The images containing insect pests are input into the residual network ResNet50 to extract the multi-scale features of the images; The multi-scale features of the image are input into the multi-band multi-scale denoising encoder, and the multi-scale features are firstly fused and dynamically enhanced from high dimension to low dimension to obtain multi-dimensional fused features; Performing wavelet transformation on the multi-dimensional fusion features to obtain low-frequency sub-bands and high-frequency sub-bands; Input the high frequency sub-band into the dynamic cross-scale convolution attention denoising DCCD module to perform dynamic cross-scale convolution attention denoising calculation; The output after the DCCD module processing is subjected to feature fusion and feature re-enhancement processing with the low-frequency sub-band to complete the processing of the multi-band multi-scale dimension reduction sampling MMD module; The output of the MMD module is subjected to dynamic multi-scale feature fusion and enhancement with high-dimensional features, and the enhanced features are flattened into sequences and concatenated, and then input into a decoding preprocessing module; The output of the decoding preprocessing module is decoded by the decoder and used as the input of the prediction head to obtain the target detection result.

2. The multi-band multi-scale end-to-end pest detection method according to claim 1, characterized in that: The multi-scale features are fused and dynamically enhanced from high dimension to low dimension to obtain multi-dimensional fusion features, specifically: In the multi-band multi-scale denoising encoder, the high-dimensional features output by the last stage of ResNet50, Stage n, are input into a single-layer Transformer encoder for processing to obtain multi-scale feature enhancement within the high-dimensional features, and the enhanced features are scaled down. The scale-restored features are upsampled and concatenated with the features output by the Stage n-1 of ResNet50, and input into the dynamic multi-scale feature fusion enhancement DMFE module for dynamic multi-scale feature fusion enhancement; The fused and enhanced features are upsampled and concatenated with the features output from the Stage n-2 of ResNet50, input into the DMFE module, and then dynamic multi-scale feature fusion enhancement is performed again to obtain multi-dimensional fused features.

3. The multi-band multi-scale end-to-end pest detection method according to claim 2, characterized in that: The high-dimensional features are input into a single-layer Transformer encoder for processing, including: position embedding of the features, multi-head self-attention mechanism calculation, and feedforward neural network processing.

4. The method of end-to-end pest detection based on multi-band and multi-scale according to claim 2, characterized in that: The DMFE module automatically selects the corresponding multi-scale convolution kernel combination according to the height or width of the input feature, performs group convolution calculation and noise reduction processing, and performs multi-scale enhancement on the processed features to achieve dynamic multi-scale feature fusion enhancement of the input features.

5. The method of end-to-end pest detection based on multi-band and multi-scale according to claim 1, characterized in that: The obtained low-frequency sub-band is a low-frequency sub-band with a scale reduced by half.

6. The method of end-to-end pest detection based on multi-band and multi-scale according to claim 1, characterized in that: The DCCD module dynamically adjusts the combination of multi-scale convolution kernels according to the scale of the input features, generates weights using cross-channel linear transformation, multiplies the weights with the output of the grouped convolution, fully extracts the multi-scale target features in the input features, and suppresses noise by combining the average and maximum calculations. The output after multi-scale transformation is added to the output after noise reduction processing to achieve dynamic cross-scale feature fusion and noise reduction.

7. The method of end-to-end pest detection based on multi-band and multi-scale according to claim 1, characterized in that: The wavelet transform is Haar wavelet transform.

8. The multi-band multi-scale end-to-end pest detection method according to claim 1, characterized in that: The target detection result includes: pest category and location information.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the multi-band and multi-scale end-to-end pest detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the computer program implements the multi-band and multi-scale end-to-end pest detection method according to any one of claims 1 to 8.

Images