Unmanned aerial vehicle image target detection method and system

Abstract

The invention provides an unmanned aerial vehicle image target detection method and system, and the method comprises the steps: carrying out the filtering of a feature map through a Gaussian filter and a Laplacian filter through the filtering and fine-grained processing of a feature map based on a frequency domain attention mechanism, and highlighting the edge features of a target; processing an input image by using improved ResNet50, introducing spatial convolution, reducing spatial redundant information through reconstruction and separation operations, and generating a feature map with expressive force; filtering low-level information through high-level semantic information by using the features subjected to selective fusion network SFCA fusion processing, and fusing the features; performing classification and regression by using a DETR-based detection network to realize target detection; and model training is completed, the test set is input into the model, and unmanned aerial vehicle image target detection is completed according to a prediction output result. The method solves the technical problems that the target detection efficiency and accuracy are low, the feature pattern quality and expression ability are poor due to insufficient fusion of high-level features and low-level features, and gradient explosion is likely to occur.

Description

Technical Field

[0001] The present invention relates to the fields of machine vision and artificial intelligence, and in particular to a method and system for detecting targets in unmanned aerial vehicle images. Background Art

[0002] With the rapid development of drone technology, drones have shown great application potential in many fields, such as agriculture, environmental monitoring, traffic monitoring, disaster relief, and military reconnaissance. Drones can be quickly deployed to the target area and capture a large amount of image and video data through the cameras and other sensor devices on board. However, effectively identifying and locating specific targets from massive amounts of data is an extremely challenging task. Traditional image processing and pattern recognition methods often fail to achieve ideal results when faced with complex backgrounds, lighting changes, target diversity, and other problems. Therefore, it has become an urgent need to study and develop drone image target detection algorithms.

[0003] In addition, with the continuous advancement of artificial intelligence and machine learning technology, especially the rapid development of deep learning technology, new solutions are provided for drone image target detection. Deep learning algorithms can automatically extract effective features from large amounts of data and recognize complex patterns through multi-layer neural network structures. This provides new opportunities for the research of drone image target detection algorithms.

[0004] At present, the main challenges of drone image target detection algorithms lie in the diversity of targets, the complexity of backgrounds, and the timeliness of target detection.

[0005] The existing invention patent application document "A method and system for fusion of visible light and infrared images with low resource consumption" with publication number CN116958001A, the existing method includes: firstly, capturing small-resolution visible light and infrared images through low-power, low-resolution optical and infrared sensors mounted on drones; then using local implicit image functions to learn the continuous representation of source images to obtain high-resolution source image features; secondly, using a fusion strategy based on spatial and channel attention models to retain high-value feature information of the two source images; finally, sending the fusion features obtained by weighted fusion to the decoder to reconstruct a high-quality, high-resolution fused image. The above-mentioned prior art obtains results for the fusion of two different types of images, and has certain requirements for the equipment for collecting data, and it is necessary to collect visible light pictures and infrared pictures of the same scene. It is only necessary to use low-power equipment to collect pictures to achieve high-resolution fusion results. However, the model used in the above-mentioned prior art requires model training, and the cost of training operations is relatively high.

[0006] The existing invention patent application document with publication number CN117197475A, "A target detection method for large-scale multi-interference source scenes", includes: extracting rain and fog removal features and low-light enhancement features through interference image input pre-training of rain and fog removal network and low-light enhancement network, and generating visual enhancement features through bidirectional adaptive dynamic fusion; extracting rigid object skeleton features, rigid object skeleton graph, non-rigid object skeleton features, and non-rigid object skeleton graph through interference image input pre-training, and generating optimized skeleton graph and skeleton features through selective integration; inputting visual enhancement features into the target detector backbone network to generate three different scale features, and performing skeleton feature enhancement on each feature; inputting enhanced features into feature pyramid and detection head to generate target detection results. The above-mentioned prior art mainly focuses on feature prominence in rain and fog and low-light scenes, and uses yolov5 in the final detection stage, which has a non-maximum suppression process, resulting in a long prediction time. At the same time, the aforementioned existing technology uses a pre-trained model to enhance the features of rain, fog and low light in images. However, this enhancement method requires training, and the time and process of training lead to reduced target detection efficiency.

[0007] The existing invention patent application document "A feature fusion method and target detection network based on gating mechanism" with publication number CN113537396A, the existing method includes: obtaining low-level detail features and high-level semantic features of the detection target image; using the low-level detail features and high-level semantic features as input features of the gated fusion module; splicing the low-level detail features and the high-level semantic features, performing a convolution operation, and using a first activation function to obtain an update gating signal and a reset gating signal; using the reset gating signal to filter the low-level detail features to obtain the filtered low-level detail features; splicing the filtered low-level detail features with the high-level semantic features, performing a convolution operation, and using a second activation function to obtain the current candidate features; using the update gating signal to selectively fuse the low-level detail features and the current candidate features to obtain fused features, and the fused features are used for target detection. From the specific implementation of the prior art, it can be seen that the method of extracting features in the aforementioned prior art is to extract low-level features through the first layer of convolution, input the low-level features into the second layer of convolution, obtain high-level features, and then use the high-level features and low-level features as two inputs, through the gated fusion module, select high-level features through low-level details for feature fusion. However, it is difficult for the aforementioned prior art to achieve selective fusion and obtain a feature map with prominent features. It is impossible to combine the rich semantic information of high-level features and the detailed position information of low-level features, and the high-level and low-level features are not fully integrated, and the quality and expression ability of the feature map are low.

[0008] The existing invention patent application document "A method and device for detecting small targets for drones based on improved DETR" with publication number CN116958687A, the existing method includes: constructing a data set for detecting small targets of drones, and dividing the data set into a training set and a test set; constructing a small target detection network based on improved DETR; using ShuffleNet-d as the feature extraction network of DETR, and introducing a 1×1 convolution module to extract features along the channel dimension; wherein ShuffleNet-d deletes the original global pooling and fully connected layers of ShuffleNetv2; replaces the self-attention in the encoder in DETR with FlashAttention-2; the Neck layer in DETR uses a deformable cross-scale feature fusion module deformable-CCFM; uses Smooth-L1 and DIOULoss as the loss function of the lightweight feature extraction network of DETR improved based on FlashAttention-2; and uses the data set to train and evaluate the small target detection network based on improved DETR. The above-mentioned prior art uses ShuffleNet-d as the feature extraction network, and there is a gradient explosion. At the same time, the prior art simply fuses the extracted multi-scale feature maps in the fusion stage without performing any processing on the feature maps.

[0009] In summary, the existing technologies have technical problems such as low target detection efficiency and accuracy, insufficient fusion of high-level features and low-level features resulting in poor feature map quality and expression ability, and easy occurrence of gradient explosion. Summary of the invention

[0010] The technical problem to be solved by the present invention is: how to solve the technical problems in the prior art of low target detection efficiency and accuracy, insufficient fusion of high-level features and low-level features resulting in poor feature map quality and expression ability, and easy occurrence of gradient explosion.

[0011] The present invention adopts the following technical solutions to solve the above technical problems: a method for detecting targets in drone images comprises:

[0012] S1. Based on the frequency domain attention mechanism, the input feature map is processed using Gaussian filter and Laplacian filter to obtain a fine-grained feature map to highlight the edge features of the target;

[0013] S2, set up spatial convolution Sconv and residual network, build feature extraction model ResNet50, use spatial convolution Sconv to perform reconstruction and separation operations to reduce spatial redundant information, use feature extraction model ResNet50 to perform multi-scale feature extraction operations on input feature maps, and generate suitable expressive feature maps, where the residual network includes: 2-level residual units;

[0014] S3, using the adaptive feature selection fusion network SFCA, taking the preset high-level features as channel weights, performing feature selection operations to filter the embedded semantic information of the preset low-level features, and performing fusion operations to fuse and process the applicable expressive feature maps to obtain enhanced model features;

[0015] S4. Use the DETR-based detection network to perform classification and regression, perform modeling, obtain a suitable detection model, input the preset test set into the suitable detection model, obtain the predicted output result, and perform target detection on the drone image. The DETR-based detection network includes: an encoder and a decoder.

[0016] The present invention first extracts features, then performs edge enhancement on the extracted features, fuses the enhanced features, and finally inputs the fused feature map into the detection network to obtain the detection result. The present invention utilizes the image information collected by the camera or other sensor equipment carried by the drone, and realizes the automatic recognition and positioning of the target object through algorithm analysis. It solves the problem of drones quickly and accurately identifying specific targets in complex environments, and provides technical support for the application of drones in agriculture, environmental protection, military and other fields. The present application can better complete the extraction of multi-scale features and target detection of drone inspection images.

[0017] In a more specific technical solution, S1 includes:

[0018] S11. Using a Gaussian filter, according to the following logic, a noise suppression operation is performed on the fine-grained feature map to obtain a Gaussian filter feature map:

[0019] f L =Gaussian(f i ) (5)

[0020] Where Gaussian represents a 3×3 Gaussian kernel filter, C, H, and W represent f i The channel, height and width size, f i ∈R C×H×W Represents the input feature map;

[0021] S12, using discrete cosine transform DCT and global average pooling GAP, according to the following logic, the Gaussian filter feature map is subjected to noise reduction processing:

[0022] A S =Softmax(GAP(DCT(f L )) (6)

[0023] In the formula, DCT and GAP represent channel-by-channel discrete cosine transform and global average pooling respectively;

[0024] S13. Use the Softmax function to enhance relevant information features and suppress irrelevant features to obtain the attention score A. S ∈R C×1×1 ;

[0025] S14. Use a high-pass filter and a Laplace filter to extract enhanced detail information from the Gaussian filter feature map according to the attention score using the following logic:

[0026]

[0027] Where Laplacian represents the 3×3 Laplacian kernel filter, and represents channel multiplication;

[0028] S15. Use the residual path to process and enhance the detail information to obtain a fine-grained feature map:

[0029]

[0030] In the formula, Represents pixel-by-pixel addition.

[0031] Aiming at the problem of target recognition in drone images, the present invention innovatively introduces an edge enhancement module based on the frequency domain attention mechanism. The application of this module makes the key features of the feature map more prominent, providing support for subsequent feature extraction.

[0032] In order to refine the feature maps of different layers and obtain feature maps with more prominent features, the present invention introduces a frequency domain attention mechanism module, which has no parameters and does not require training. First, a low-pass filter is used to filter out high-frequency noise in the feature map; then, the feature map is re-weighted in the transform domain to enhance information that is more relevant to the target; finally, a high-pass filter is used to reduce the loss of details. This module mainly utilizes the attention mechanism based on parameter-free transformation and filter, which can achieve better fine-grained feature representation while maintaining a low computational cost.

[0033] In a more specific technical solution, S2 includes:

[0034] S21, spatial convolution Sconv performs a separation operation to separate the corresponding informative feature map and non-informative feature map in the spatial content to determine the spatial redundant information, wherein the information content of the input feature map is evaluated by using the scale factor in the group normalization layer in the feature extraction model ResNet50;

[0035] S22, perform reconstruction operation to remove spatial redundant information, perform splicing operation through cross reconstruction operation, add the information feature map and the non-information feature map to obtain the spatial fine feature map X w, to obtain a suitable expressive feature map.

[0036] The present invention introduces spatial convolution technology in the feature extraction process. Through spatial convolution, the redundant parts in the spatial information can be efficiently eliminated, thereby highlighting the key information, laying the foundation for subsequent target detection.

[0037] In a more specific technical solution, S21 includes:

[0038] S211. According to the input feature map, an intermediate feature map is given:

[0039] X∈R N×C×H×W

[0040] Where N is the batch axis, C is the channel axis, H and W are the spatial height and width axes;

[0041] S212, using the following logic, subtract the mean μ of the intermediate feature map X and divide it by the standard deviation σ to standardize the intermediate feature map X in the input feature map:

[0042]

[0043] Where ε is a small positive constant added for division stability, and γ and β are trainable affine transformation parameters.

[0044] S213, using the trainable affine transformation parameters γ∈R in the group normalization layer C As a measure of the spatial pixel variance of each batch and channel, the correlation weight W is normalized using the following logic: γ ∈R C , get the importance of different feature maps:

[0045]

[0046] S214, according to the relevant rights γ Re-weight the importance of different feature maps, map the importance of different feature maps to a preset range through the sigmoid function, and perform gating processing through a preset threshold to obtain information weight W1 and non-information weight W2;

[0047] S215, multiply the input feature map X by the information weight W1 and the non-information weight W2 respectively to obtain the information weighted feature X w1 , non-informative weighted features X w2 , the non-information weighted feature X w2 As spatial redundant information.

[0048] In a more specific technical solution, in S214, according to the preset threshold, the information weight W1 and the non-information weight W2 are processed using the following logic:

[0049] W=Gate(Sigmoid(W γ (GN(X)))) (3)

[0050] In a more specific technical solution, in S22, the following logic is used to reconstruct the information feature graph and the non-information feature graph:

[0051]

[0052] In the formula, is element-wise multiplication, is element wise summation and ∪ is concatenation.

[0053] In order to effectively reduce the redundant parts in the spatial information and generate feature maps that can highlight the key information, thereby reducing the computational complexity, the present invention proposes a Resnet50 network based on spatial convolution optimization. In the model, the original last layer of 3*3 convolution is replaced by Sconv. Sconv includes two major operations: separation and reconstruction. In the separation operation, first, the feature map is processed by group normalization, and then the corresponding weight is calculated according to the mean and variance of each feature map; in the reconstruction operation, the information-rich feature map is added to the information feature map by pixel, and the two feature maps are connected in series to form a more expressive feature map. This innovative design enables our network to reduce the computational burden and improve processing efficiency while maintaining high performance.

[0054] In more specific technical solutions, S3 includes:

[0055] S31, perform feature selection operation, use the preset CA module to perform average pooling and maximum pooling on the feature map, calculate the channel weight through the sigmoid function, and filter the low-level features to embed semantic information through the channel weight;

[0056] S32, perform a fusion operation. In order to unify the dimensions of high-level features and low-level features, use bilinear interpolation to upsample and downsample the preset high-level features, unify the dimensions of the preset high-level features and the preset low-level features, and obtain unified dimensional features:

[0057] S33, filtering the low-level features according to the channel weights to obtain low-level filtered features;

[0058] S34, fusion processing low-level features and preset high-level features to obtain enhanced model features

[0059] S35. Combine transposed convolution and bilinear interpolation to restore the scale of preset high-level features.

[0060] In a more specific technical solution, in S34, the following logic is used to fuse the low-level features and the preset high-level features:

[0061] f att =BL(T-Conv(f high )) (9)

[0062] f out =f low *CA(f att )+(f att ) (10).

[0063] In the feature fusion stage, the present invention uses the semantic information of the highest-level features to carefully screen the low-level features, thereby improving the overall quality and expression ability of the feature map.

[0064] Specifically, in order to combine the rich semantic information of high-level features and the detailed location information of low-level features. The present invention introduces a selective feature fusion module. First, the size of the high-level feature map is adjusted by deconvolution and bilinear interpolation to match the underlying feature map; then, the low-level feature map is screened and processed using the weights determined by the high-level semantic information; finally, the processed low-level feature map is fused with the high-level feature map to generate a more expressive and information-rich feature map. This strategy not only achieves the deep fusion of high-level and low-level features, but also significantly improves the quality and expressiveness of the feature map.

[0065] In a more specific technical solution, in S4, the decoder Transformer decoder includes: no less than 2 multi-head attention modules Multi-Head Attention and Feedforward network. The Multi-Head Attention module is used to perform information exchange between the encoder and the decoder, and the Feedforward network is used to perform feature extraction operations and information transformation operations.

[0066] In a more specific technical solution, a drone image target detection system includes:

[0067] The filtering module is used to process the input feature map using Gaussian filter and Laplace filter based on the frequency domain attention mechanism to obtain a fine-grained feature map to highlight the edge features of the target;

[0068] The reconstruction and separation module is used to set the spatial convolution Sconv and the residual network, construct the feature extraction model ResNet50, use the spatial convolution Sconv to perform reconstruction and separation operations to reduce spatial redundant information, and use the feature extraction model ResNet50 to perform multi-scale feature extraction operations on the input feature map to generate a suitable expressive feature map, wherein the residual network includes: a 2-level residual unit, and the reconstruction and separation module is connected with the filtering module;

[0069] A fusion module is selected to utilize the SFCA-based adaptive feature selection fusion network, use the preset high-level features as channel weights, perform feature selection operations to filter the embedded semantic information of the preset low-level features, perform fusion operations to fuse and process the applicable expressive feature maps, obtain enhanced model features, and select the fusion module to be connected with the reconstruction separation module;

[0070] The target detection module is used to perform classification and regression using the DETR-based detection network, to perform modeling, to obtain a suitable detection model, to input a preset test set into the suitable detection model, to obtain a predicted output result, so as to perform target detection on the drone image, wherein the DETR-based detection network includes: an encoder, a decoder, and the target detection module is connected with the selection fusion module.

[0071] Compared with the prior art, the present invention has the following advantages:

[0072] The present invention first performs edge enhancement on the original image, then performs feature extraction on the enhanced image, then selectively fuses the features, and finally inputs the fused feature map into the detection network to obtain the detection result. The present invention utilizes the image information collected by the camera or other sensor equipment carried by the drone, and realizes automatic recognition and positioning of the target object through algorithm analysis. It solves the problem of drones quickly and accurately identifying specific targets in complex environments, and provides technical support for the application of drones in agriculture, environmental protection, military and other fields. The present application can better complete the extraction of multi-scale features and target detection of drone inspection images.

[0073] Aiming at the problem of target recognition in drone images, the present invention innovatively introduces an edge enhancement module based on the frequency domain attention mechanism. The application of this module makes the key features of the feature map more prominent, providing support for subsequent feature extraction.

[0074] In order to refine the feature maps of different layers and obtain feature maps with more prominent features, the present invention introduces a frequency domain attention mechanism module, which has no parameters and does not require training. First, a low-pass filter is used to filter out high-frequency noise in the feature map; then, the feature map is re-weighted in the transform domain to enhance information that is more relevant to the target; finally, a high-pass filter is used to reduce the loss of details. This module mainly utilizes the attention mechanism based on parameter-free transformation and filter, which can achieve better fine-grained feature representation while maintaining a low computational cost.

[0075] The present invention introduces spatial convolution technology in the feature extraction process. Through spatial convolution, the redundant parts in the spatial information can be efficiently eliminated, thereby highlighting the key information, laying the foundation for subsequent target detection.

[0076] In order to effectively reduce the redundant parts in the spatial information and generate feature maps that can highlight the key information, thereby reducing the computational complexity, the present invention proposes a Resnet50 network based on spatial convolution optimization. In the model, the original last layer of 3*3 convolution is replaced by Sconv. Sconv includes two major operations: separation and reconstruction. In the separation operation, first, the feature map is processed by group normalization, and then the corresponding weight is calculated according to the mean and variance of each feature map; in the reconstruction operation, the information-rich feature map is added to the information feature map by pixel, and the two feature maps are connected in series to form a more expressive feature map. This innovative design enables our network to reduce the computational burden and improve processing efficiency while maintaining high performance.

[0077] In the feature fusion stage, the present invention uses the semantic information of the highest-level features to carefully screen the low-level features, thereby improving the overall quality and expression ability of the feature map.

[0078] Specifically, in order to combine the rich semantic information of high-level features and the detailed location information of low-level features. The present invention introduces a selective feature fusion module. First, the size of the high-level feature map is adjusted by deconvolution and bilinear interpolation to match the underlying feature map; then, the low-level feature map is screened and processed using the weights determined by the high-level semantic information; finally, the processed low-level feature map is fused with the high-level feature map to generate a more expressive and information-rich feature map. This strategy not only achieves the deep fusion of high-level and low-level features, but also significantly improves the quality and expressiveness of the feature map.

[0079] The present invention solves the technical problems existing in the prior art of low target detection efficiency and accuracy, insufficient fusion of high-level features and low-level features resulting in poor feature map quality and expression ability, and easy occurrence of gradient explosion. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 A schematic diagram of basic steps of a method for detecting targets in drone images according to Embodiment 1 of the present invention;

[0081] Figure 2 This is a schematic diagram of the IDFBA structure of Example 1 of the present invention;

[0082] Figure 3 It is a schematic diagram of the spatial convolution Sconv structure and the upper layer separation operation and lower layer reconstruction operation principle of Example 1 of the present invention;

[0083] Figure 4 This is a schematic diagram of the SFCA module structure of Example 1 of the present invention;

[0084] Figure 5 This is a schematic diagram of the encoding and decoding structure of Example 1 of the present invention. DETAILED DESCRIPTION

[0085] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0086] Example 1

[0087] like Figure 1 As shown, a method for detecting a target in an unmanned aerial vehicle image provided by the present invention comprises the following basic steps:

[0088] S1. Use filtering based on the frequency domain attention mechanism to perform fine-grained processing on the obtained feature map, and filter the feature map through Gaussian filter and Laplacian filter to highlight the edge features of the target;

[0089] like Figure 2As shown, the image edge feature enhancement module based on the frequency domain attention mechanism used in this embodiment labels the target image through labelimg, and divides the labeled data into a training set and a test set according to a ratio of 8:2, for example; a Gaussian low-pass filter is used to filter out the noise in the feature map, because the noise is usually in the high-frequency spectrum. The Gaussian filter can better maintain the detail information of the image when processing the edge part, because the Gaussian filter determines the weight according to the distance between the pixel point and the center pixel, and the farther the distance, the lower the weight of the pixel. This makes the Gaussian filter have better smoothing effect and detail retention ability in image processing; the feature map is re-weighted in the transform domain to enhance the relevant information of the target extraction; because some detail information is weakened by low-pass filtering, this embodiment uses high-pass filtering to avoid the loss of details. In addition, the work of IDFAB does not require training parameters, and multi-scale feature mapping can be achieved at a low cost.

[0090] In this embodiment, the input feature map is assumed to be f i ∈R C×H×W , where C, H and W represent f i The channel, height and width size of the input feature. In order to reduce the high-frequency noise of the input feature, a low-pass filter, a Gaussian filter, is used to process the input feature map. Unlike other widely used low-pass filters, the Gaussian filter can retain more detailed information due to its structure. For fine-grained feature representation, this embodiment uses a Gaussian filter of size, for example, 3×3 to suppress noise, which can be expressed as:

[0091] f L =Gaussian(f i ) (5)

[0092] Here, Gaussian represents a 3×3 Gaussian kernel filter.

[0093] In this embodiment, discrete cosine transform DCT and global average pooling GAP are used to filter feature maps with rich information and further reduce the influence of noise, which is expressed as follows:

[0094] A S =Softmax(GAP(DCT(f L )) (6)

[0095] Among them, DCT and GAP represent channel-by-channel discrete cosine transform and global average pooling, respectively.

[0096] In this embodiment, the Softmax function is used to further emphasize relevant information features and suppress irrelevant features, which is represented by Softmax. Through these processes, the attention score A is obtained. s ∈R C×1×1Considering that the low-pass filter inevitably destroys the high-frequency information of the input features, this embodiment uses a high-pass filter-Laplacian filter to extract and enhance detail information. The extraction logic can be expressed as:

[0097]

[0098] Among them, Laplacian represents a 3×3 Laplacian kernel filter, and Represents channel multiplication.

[0099] This embodiment uses the residual path to make the output more stable and avoid gradient disappearance, which can be expressed as follows:

[0100]

[0101] in, Represents pixel-by-pixel addition.

[0102] S2. Use the improved ResNet50 to process the input image. By introducing spatial convolution, replace the original 3*3 convolution, reduce the redundant information in the space through reconstruction and separation operations, and generate an expressive feature map;

[0103] This embodiment adopts a feature extraction model based on the improved ResNet50, performs multi-scale feature extraction through the enhanced ResNet50, and uses the enhanced version of ResNet-50 as the feature extraction network; in this embodiment, ResNet-50 uses a residual network to alleviate the gradient vanishing problem, thereby promoting convergence to solve the degradation problem commonly found in deep neural networks. Considering the excessive background interference and lack of features in drone images, this embodiment enhances the feature extraction network by changing the last layer of convolutional blocks in the original ResNet-50 model. This module is designed to extract deeper semantic information and remove redundant information, thereby improving the detection efficiency of the model. ResNet-50 has two levels of residual units.

[0104] like Figure 3 As shown, in this embodiment, the 3*3 convolution of the last layer of ResNet-50 is improved and replaced with a new convolution - spatial convolution Sconv, whose main function is to eliminate spatial information redundancy, generate expressive feature maps and reduce the amount of calculation. The operations performed using spatial convolution Sconv include but are not limited to two stages: separation and reconstruction operations.

[0105] In this embodiment, the purpose of the separation operation is to separate the feature maps with rich information corresponding to the spatial content from the feature maps with less information. In this embodiment, the scaling factor in the group normalization layer is used to evaluate the information content of different feature maps. The normalization layer is an improved algorithm proposed to address the weakness of the batch normalization algorithm that is highly dependent on the batch size. Specifically, given an intermediate feature map X∈R N×C×H×W , where N is the batch axis, C is the channel axis, and H and W are the spatial height and width axes. This example first normalizes the input feature X by subtracting the mean μ and dividing by the standard deviation σ, as shown below:

[0106]

[0107] where μ and σ are the mean and standard deviation of X, ε is a small positive constant added for division stability, and γ and β are trainable affine transformations.

[0108] In this embodiment, the trainable parameter γ∈R in the normalization layer is used C As a way to measure the spatial pixel variance of each batch and channel. Richer spatial information reflects more variations in spatial pixels, leading to larger γ. The normalized correlation weight W γ ∈R C It is obtained by the formula, which represents the importance of different feature maps.

[0109]

[0110] In this embodiment, W γ The weights of the re-weighted feature map are mapped to the range (0,1) by the sigmoid function and gated by the threshold. The weights above the threshold are set to 1 to obtain the information weight W1, and set to 0 to obtain the non-information weight W2 (the threshold is set to 0.5 in the experiment). The whole process of obtaining W can be expressed as:

[0111] W=Gate(Sigmoid(W γ (GN(X)))) (3)

[0112] In this embodiment, the input feature X is multiplied by W1 and W2 respectively to obtain two weighted features: X with large information content w1 and X with small amount of information w2 Therefore, this embodiment divides the input features into two parts: X w1 Informative and expressive spatial content, while X w2 There is little or no information, which is considered redundant.

[0113] In this embodiment, in order to reduce spatial redundancy, a reconstruction operation is further proposed to add features with rich information to features with less information to generate features with richer information, thereby saving space. Instead of directly adding these two parts, a cross reconstruction operation is used to fully combine the two weighted different information features to strengthen the information flow between different information features; the cross-reconstructed feature X w1 and X w2 Splice to get the spatial fine feature map X w The entire construction operation of the reconstruction can be expressed as:

[0114]

[0115] in, is element-wise multiplication, is the element-wise summation and ∪ is the concatenation. After applying the above module to the intermediate input feature X, not only the informative features are separated from the less informative features, but also they are reconstructed to enhance the representative features and suppress the redundant features in the spatial dimension.

[0116] S3, using the selective fusion network SFCA to fuse the processed features, filtering low-level information through high-level semantic information, and thus strategically fusing features;

[0117] like Figure 4 As shown, in the adaptive feature selection fusion network adopted in this embodiment, the multi-scale feature map generated by the backbone network has high-level features and rich semantic information, but the target positioning is relatively rough. On the contrary, low-level features provide precise target locations but contain limited semantic information. A common solution to this problem is to directly sum the upsampled high-level features and low-level features by pixel, enriching each layer with semantic information. However, this technology does not perform feature selection, but simply adds the pixel values ​​of multiple feature layers. In this embodiment, an SFCA module is designed. This module strategically fuses features by using high-level features as weights to filter important semantic information embedded in low-level features.

[0118] In this embodiment, in the feature selection stage, the CA module first performs average pooling and maximum pooling on the feature map, and then calculates the channel weights through the sigmoid function and screens through the weights. The basic purpose of pooling is to downsample and reduce the dimension of the feature map; eliminate redundant data, compress functions, and reduce parameter counts; and achieve translation, rotation and scale invariance. The CA module uses global average pooling and global maximum pooling to calculate the average and maximum values ​​of each channel respectively. The main purpose of maximum pooling is to extract the most relevant data from each channel, while the purpose of average pooling is to evenly obtain all data from the feature map to minimize excessive losses. Therefore, the combination of these pooling methods in the CA module is conducive to extracting the most representative information from each channel while minimizing information loss. The weight information is then multiplied by the feature map of the corresponding scale to generate a filtered feature map.

[0119] In the fusion stage of this embodiment, given an input high-level feature f high ∈R C×H×W and an input low-level feature The high-level space is initially enlarged using transposed convolutions with stride size 2 and a 3×3 kernel, resulting in a feature size of f high ∈R C×2H×2W ; In order to unify the dimensions of high-level features and low-level features, bilinear interpolation is used to upsample or downsample the high-level features to obtain The CA module converts high-level features into corresponding attention weights, obtains features with consistent dimensions, and then filters low-level features. The filtered low-level features are fused with high-level features to enhance the feature representation of the model, and the The following formulas (9) and (10) represent the fusion process of feature selection:

[0120] f att =BL(T-Conv(f high )) (9)

[0121] f out =f low *CA(f att )+(f att ) (10)

[0122] In the image sampling process of this embodiment, a method combining transposed convolution and bilinear interpolation is used to restore the scale of high-order features. Bilinear interpolation is simple and fast, and realizes image scaling by directly operating pixels. Transposed convolution adapts to the data through learnable parameters, so that the output not only expands the feature map, but also reconstructs the input in the form of convolution. This is achieved by filling zeros to realize the convolution operation after the feature map is expanded. At the same time, by sampling different areas of the input image at different output image positions, the problem of non-uniform sampling can be handled.

[0123] S4. Use the DETR-based detection network for classification and regression to detect the target;

[0124] In this embodiment, encoding and decoding are performed based on DETE. DETR uses a structure composed of 6 stacked Transformerdecoders. The first encoder inputs the features of the picture, and the input of the subsequent encoder is the output of the encoder in the previous layer. Each Transformer decoder is composed of several Multi-Head Attention and Feedforward networks, each of which is a basic building block in a Transformerdecoder. Multi-Head Attention is used to interact with encoder and decoder information, and the Feedforward network is used to extract features and information transformation. The decoder of DETR consists of two main parts: Transformer decoder and position embedding. Position embedding is a special embedding method used to convert target position information into a form that can be learned by neural networks. In DETR, the dimension of the position embedding is the same as the feature dimension of the encoder output, so it can be spliced ​​with the feature vector as the input of the decoder.

[0125] like Figure 5 As shown, in this embodiment, each Transformer decoder is responsible for processing the interaction between the encoder output features and the target position information. Specifically, each decoder includes but is not limited to: Multi-Head Attention, Feedforward network and LayerNorm. In Multi-Head Attention, the features and target position information output by the encoder are used as query vectors and key vectors, respectively, to calculate the attention weights. The calculated attention weights are weighted and summed with the encoder output features to obtain the input features of the decoder. In the Feedforward network, the input features of the decoder are transformed and nonlinearly mapped to obtain a higher-level feature representation. Finally, normalization is performed in LayerNorm to obtain the final decoder output features.

[0126] When the model training is completed, the test set is input into the model, and the target detection of the drone image is completed based on the predicted output results.

[0127] The present invention first performs edge enhancement on the original image, then performs feature extraction on the enhanced image, then selectively fuses the features, and finally inputs the fused feature map into the detection network to obtain the detection result. The present invention utilizes the image information collected by the camera or other sensor equipment carried by the drone, and realizes automatic recognition and positioning of the target object through algorithm analysis. It solves the problem of drones quickly and accurately identifying specific targets in complex environments, and provides technical support for the application of drones in agriculture, environmental protection, military and other fields. The present application can better complete the extraction of multi-scale features and target detection of drone inspection images.

[0128] Aiming at the problem of target recognition in drone images, the present invention innovatively introduces an edge enhancement module based on the frequency domain attention mechanism. The application of this module makes the key features of the feature map more prominent, providing support for subsequent feature extraction.

[0129] In order to refine the feature maps of different layers and obtain feature maps with more prominent features, the present invention introduces a frequency domain attention mechanism module, which has no parameters and does not require training. First, a low-pass filter is used to filter out high-frequency noise in the feature map; then, the feature map is re-weighted in the transform domain to enhance information that is more relevant to the target; finally, a high-pass filter is used to reduce the loss of details. This module mainly utilizes the attention mechanism based on parameter-free transformation and filter, which can achieve better fine-grained feature representation while maintaining a low computational cost.

[0130] The present invention introduces spatial convolution technology in the feature extraction process. Through spatial convolution, the redundant parts in the spatial information can be efficiently eliminated, thereby highlighting the key information, laying the foundation for subsequent target detection.

[0131] In order to effectively reduce the redundant parts in the spatial information and generate feature maps that can highlight the key information, thereby reducing the computational complexity, the present invention proposes a Resnet50 network based on spatial convolution optimization. In the model, the original last layer of 3*3 convolution is replaced by Sconv. Sconv includes two major operations: separation and reconstruction. In the separation operation, first, the feature map is processed by group normalization, and then the corresponding weight is calculated according to the mean and variance of each feature map; in the reconstruction operation, the information-rich feature map is added to the information feature map by pixel, and the two feature maps are connected in series to form a more expressive feature map. This innovative design enables our network to reduce the computational burden and improve processing efficiency while maintaining high performance.

[0132] In the feature fusion stage, the present invention uses the semantic information of the highest-level features to carefully screen the low-level features, thereby improving the overall quality and expression ability of the feature map.

[0133] Specifically, in order to combine the rich semantic information of high-level features and the detailed location information of low-level features. The present invention introduces a selective feature fusion module. First, the size of the high-level feature map is adjusted by deconvolution and bilinear interpolation to match the underlying feature map; then, the low-level feature map is screened and processed using the weights determined by the high-level semantic information; finally, the processed low-level feature map is fused with the high-level feature map to generate a more expressive and information-rich feature map. This strategy not only achieves the deep fusion of high-level and low-level features, but also significantly improves the quality and expressiveness of the feature map.

[0134] The present invention solves the technical problems existing in the prior art of low target detection efficiency and accuracy, insufficient fusion of high-level features and low-level features resulting in poor feature map quality and expression ability, and easy occurrence of gradient explosion.

[0135] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting targets in drone images, characterized in that: The method comprises: S1. Based on the frequency domain attention mechanism, the input feature map is processed using Gaussian filter and Laplacian filter to obtain a fine-grained feature map to highlight the edge features of the target; S2. Set spatial convolution Scon v And residual network, construct feature extraction model ResNet50, using the spatial convolution Scon v Reconstructing and separating operations to reduce spatial redundant information, using the feature extraction model ResNet50, performing multi-scale feature extraction operations on the input feature map to generate a suitable expressive feature map, wherein the residual network includes: a 2-level residual unit; S3, using the adaptive feature selection fusion network SFCA, taking the preset high-level features as channel weights, performing feature selection operations to filter the embedded semantic information of the preset low-level features, and performing fusion operations to fuse and process the applicable expressive feature map to obtain enhanced model features; S4. Use the DETR-based detection network to perform classification and regression, perform modeling, obtain an applicable detection model, input a preset test set into the applicable detection model, obtain a predicted output result, and perform target detection on the drone image, wherein the DETR-based detection network includes: an encoder and a decoder.

2. The method for detecting a target in an unmanned aerial vehicle image according to claim 1, characterized in that: The S1 includes: S11, using the Gaussian filter, according to the following logic, a noise suppression operation is performed on the fine-grained feature map to obtain a Gaussian filter feature map: f L =Gaussian(f i ) (5) Where Gaussian represents a 3×3 Gaussian kernel filter, C, H, and W represent f i The channel, height and width size, f i ∈R C×H×W represents the input feature map; S12, using discrete cosine transform DCT and global average pooling GAP, according to the following logic, the Gaussian filter feature map is subjected to noise reduction processing: A S =Softmax(GAP(DCT(f L )) (6) In the formula, DCT and GAP represent channel-by-channel discrete cosine transform and global average pooling respectively; S13. Use the Softmax function to enhance relevant information features and suppress irrelevant features to obtain the attention score A. s ∈R C ×1×1 ; S14, using a high-pass filter and the Laplace filter, according to the attention score, from the Gaussian filter feature map, using the following logic to extract enhanced detail information: Where Laplacian represents the 3×3 Laplacian kernel filter, and represents channel multiplication; S15. Using the residual path, process the enhanced detail information to obtain the fine-grained feature map: In the formula, Represents pixel-by-pixel addition.

3. The method for detecting target in drone image according to claim 1, characterized in that: The S2 includes: S21, the spatial convolution Scon v Performing the separation operation to separate the corresponding informative feature graph and the non-informative feature graph in the spatial content to determine the spatial redundant information, wherein the information content of the input feature graph is evaluated using a scaling factor in a group normalization layer in the feature extraction model ResNet50; S22, perform a reconstruction operation to remove the spatial redundant information, perform a splicing operation through a cross-reconstruction operation, add the information feature map to the non-information feature map, and obtain a spatial fine feature map X w , to obtain the applicable expressive feature map.

4. The method for detecting a target in an unmanned aerial vehicle image according to claim 3, characterized in that: The S21 includes: S211. According to the input feature map, an intermediate feature map is given: X∈R N×C×H×W Where N is the batch axis, C is the channel axis, H and W are the spatial height and width axes; S212, using the following logic, subtract the average value μ of the intermediate feature map X and divide it by the standard deviation σ to standardize the intermediate feature map X in the input feature map: Where ε is a small positive constant added for division stability, and γ and β are trainable affine transformation parameters. S213, using the trainable affine transformation parameter γ∈R in the group normalization layer C As a measure of the spatial pixel variance of each batch and channel, the correlation weight W is normalized using the following logic: γ ∈R C , get the importance of different feature maps: S214, according to the relevant rights γ Re-weight the importance of the different feature maps, map the importance of the different feature maps to a preset range through a sigmoid function, and perform gating processing through a preset threshold to obtain an information weight W1 and a non-information weight W2; S215, multiply the input feature map X by the information weight W1 and the non-information weight W2 respectively to obtain the information weighted feature X w1 , non-informative weighted features X w2 , the non-information weighted feature X w2 As the spatial redundant information.

5. The method for detecting a target in an unmanned aerial vehicle image according to claim 4, characterized in that: In S214, according to the preset threshold, the information weight W1 and the non-information weight W2 are obtained by processing using the following logic: W=Gate(Sigmoid(W γ (GN(X)))) (3) 6. The method for detecting target in drone image according to claim 3, characterized in that: In S22, the reconstruction operation is performed on the information feature graph and the non-information feature graph using the following logic: In the formula, is element-wise multiplication, is element wise summation and ∪ is concatenation.

7. The method for detecting a target in an unmanned aerial vehicle image according to claim 1, characterized in that: The S3 includes: S31, performing the feature selection operation, using a preset CA module to perform average pooling and maximum pooling on the feature map, calculating the channel weight through a sigmoid function, and filtering the low-level feature embedding semantic information through the channel weight; S32, performing the fusion operation, in order to unify the dimensions of the high-level features and the low-level features, using bilinear interpolation to upsample and downsample the preset high-level features, unifying the dimensions of the preset high-level features and the preset low-level features, and obtaining unified dimensional features: S33, filtering the low-level features according to the channel weights to obtain low-level filtered features; S34: Fusing the low-level features with the preset high-level features to obtain the enhanced model features S35. Restore the scale of the preset high-level features by combining transposed convolution and bilinear interpolation.

8. The method for detecting a target in an unmanned aerial vehicle image according to claim 7, characterized in that: In S34, the following logic is used to fuse the low-level features and the preset high-level features: f att =BL(T-Conv(f high )) (9) f out =f low *CA(f att )+(f att ) (10)。 9. The method for detecting a target in an unmanned aerial vehicle image according to claim 1, characterized in that: In S4, the decoder Transformer decoder includes: no less than 2 multi-head attention modules Multi-HeadAttention and Feedforward network, the multi-head attention module Multi-Head Attention is used to perform encoder and decoder information exchange, and the Feedforward network is used to perform feature extraction operations and information transformation operations.

10. A drone image target detection system, characterized in that: The system comprises: The filtering module is used to process the input feature map using Gaussian filter and Laplace filter based on the frequency domain attention mechanism to obtain a fine-grained feature map to highlight the edge features of the target; A reconstruction and separation module is used to set a spatial convolution Sconv and a residual network, construct a feature extraction model ResNet50, use the spatial convolution Sconv to perform reconstruction and separation operations to reduce spatial redundant information, and use the feature extraction model ResNet50 to perform multi-scale feature extraction operations on the input feature map to generate a suitable expressive feature map, wherein the residual network includes: a 2-level residual unit, and the reconstruction and separation module is connected to the filtering module; A selection fusion module is used to utilize the preset high-level features as channel weights based on the adaptive feature selection fusion network SFCA to perform feature selection operations to filter the embedded semantic information of the preset low-level features, and perform fusion operations to fuse and process the applicable expressive feature map to obtain enhanced model features, and the selection fusion module is connected to the reconstruction separation module; The target detection module is used to perform classification and regression using the DETR-based detection network, to perform modeling, to obtain an applicable detection model, to input a preset test set into the applicable detection model, to obtain a predicted output result, so as to perform target detection on the drone image, wherein the DETR-based detection network includes: an encoder, a decoder, and the target detection module is connected to the selection fusion module.

Images