A target detection method for drone perspective

Abstract

The present invention belongs to the field of target recognition and detection technology, and in particular relates to a target detection method for drones, comprising the following steps: S1, constructing a target detection network CT-RODN; the CT-RODN comprises a backbone network, a feature fusion module, and an attention prediction head; the backbone network comprises a frequency-domain-based block composite attention module FBAM and a frequency-domain-based CNN-Transformer module FCTB; the FBAM module is used to enhance the understanding of small aerial targets by combining frequency-domain analysis; the FCTB module is used to process and aggregate feature maps acquired by convolution to obtain global information about the image; the feature fusion module is used to fuse features extracted at each level; the attention prediction head is used to extract key areas of the fused feature map and perform target detection and recognition based on the extracted key areas. This method can ensure the effectiveness of small target detection in highly dynamic environments, thereby ensuring the reliability of target detection results for drones in complex backgrounds and multi-scale target scenarios.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition and detection, and in particular relates to a target detection method oriented to the perspective of an unmanned aerial vehicle (UAV). Background Art

[0002] With the rapid development of modern drone technology, object detection has become a key research area, particularly for real-time, accurate target identification and localization in complex environments such as urban landscapes or crowded places. Object detection aims to detect the location and size of target objects in a given image or video, and perform related tasks such as classification or identification.

[0003] However, current target detection systems, due to their application areas (primarily surveillance systems or smart car driving), mostly operate from a static or ground-based perspective. The characteristics of the objects they detect differ significantly from those of drones. Consequently, traditional target detection systems face numerous challenges, including small target size, easy background fusion, and occlusion, which severely impact detection accuracy and practicality. While a variety of deep learning-based target detection algorithms, such as SSD, YOLO, and Faster R-CNN, have improved detection accuracy and speed to a certain extent, they still face numerous deficiencies when dealing with small target detection in highly dynamic environments. Especially when observed from a drone's perspective, the small size and dynamic changes of the target make detection even more difficult.

[0004] Therefore, how to ensure the effectiveness of small target detection in highly dynamic environments, thereby ensuring the reliability of target detection results of UAVs in complex backgrounds and multi-scale target scenes, has become an urgent problem to be solved. Summary of the Invention

[0005] In response to the above-mentioned shortcomings of the existing technology, the present invention provides a target detection method for the perspective of drones, which can ensure the effectiveness of small target detection in high-dynamic environments, thereby ensuring the reliability of target detection results of drones in complex backgrounds and multi-scale target scenes.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] A method for detecting an object from a drone's perspective includes the following steps:

[0008] S1. Construct a target detection network (CT-RODN). CT-RODN consists of a backbone network, a feature fusion module, and an attention prediction head. The backbone network is used to extract features from the input image. The feature fusion module is used to fuse the extracted features at each level. The attention prediction head is used to extract the key areas of the fused feature map and perform target detection and recognition based on the extracted key areas.

[0009] The backbone network includes a frequency-domain-based block-based composite attention module (FBAM) and a frequency-domain-based CNN-Transformer module (FCTB). The FBAM module enhances the understanding of small aerial targets by combining frequency-domain analysis. The FCTB module processes and aggregates the feature maps obtained by convolution to obtain global information about the image.

[0010] The backbone network works as follows: First, the input image is downsampled fourfold through the convolution modules Conv1 and Conv2. Then, the FBAM module and the Conv module are used alternately to extract features and downsample the image. Next, the FCTB module processes and aggregates the feature map information captured by the convolution to obtain the global information of the image. At the same time, the fast spatial pyramid pooling (SPPF) is introduced to construct feature maps of different scales through multi-branch pooling layers and fuse them to enhance the expressiveness of features.

[0011] S2. Obtain and process images captured from the drone's perspective to obtain training data; and use the training data to train the target detection network CT-RODN.

[0012] S3. Use the trained target detection network CT-RODN to perform actual drone target detection tasks.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] 1. When using this method for target recognition, the FBAM module enhances its understanding of small aerial targets by combining frequency domain analysis. In this way, by emphasizing the important frequency components in the image, the network can focus more on important local features, thereby improving the recognition accuracy of small and low-contrast targets. The FCTB module is used to process and aggregate the feature maps obtained by convolution to obtain global information of the image. In this way, by combining the contextual environment around the target and analyzing the overall spectral information, the accuracy of target detection is improved, especially when the target is partially obscured. This context-aware approach significantly enhances the system's recognition capabilities in highly dynamic and complex environments, ensuring the performance and reliability of drones when performing critical tasks.

[0015] 2. In this method, the feature fusion module (FFM) can inject more high-level semantic information into low-level features and provide richer details for high-level features, thereby enhancing the cross-scale detection performance of the detector for small targets.

[0016] 3. The setting of the attention prediction head can extract the key areas of the fused feature map. By introducing the attention mechanism, the ability to detect targets in complex backgrounds can be improved, and targets of different scales can be detected from feature maps of different resolutions.

[0017] The target detection network CT-RODN constructed by this method can take into account both model complexity and detection accuracy. It not only improves the speed and accuracy of target detection, but also significantly enhances the application potential of drones in actual operating environments, meeting the urgent needs of modern drone technology development for real-time and accurate target detection.

[0018] In summary, the use of this method can ensure the effectiveness of small target detection in highly dynamic environments, thereby ensuring the reliability of target detection results of UAVs in complex backgrounds and multi-scale target scenes.

[0019] Preferably, the working process of the FBAM module includes:

[0020] First, the input feature map F∈R C×H×W Evenly divided into multiple local blocks F along the spatial dimension P ; Then further divided into multiple local sub-blocks C in equal proportion along the channel dimension i , i∈{1,2,…,n}; where C i ∈R (C / n)×P×P ; P represents the size of the local block; n is the number of equal divisions;

[0021] Then, for each local sub-block C i , extract its frequency component and compare it with the corresponding local sub-block C i Perform element-by-element multiplication to obtain the corresponding frequency component features; and divide each local sub-block C i The frequency component features of the local block are spliced ​​and combined to form the channel frequency attention vector Freq of the local block;

[0022] Next, the frequency attention vector Freq and the corresponding local block feature F P Perform element-by-element multiplication to obtain the block frequency channel attention feature combination; and restore the block frequency channel attention feature combination to its original size to obtain the complete aggregation vector F c ; F c Concat the original feature vector F to obtain an intermediate feature map with a channel dimension of 2C, and use a convolution layer with a convolution kernel size of 1×1 to reduce its channel number to C to obtain the feature map Fs ;

[0023] Then, for the feature map F s , perform maximum pooling and average pooling along the channel direction to generate feature maps and Will and Splicing is performed along the channel, and the spliced ​​features are reduced in dimension using 7×7 convolution, and then the Sigmoid activation function is used to generate feature E s ∈R 1×H×w ;

[0024] Finally, E s With F S Multiply them together to get the attention feature F after the channel and spatial information are aggregated o .

[0025] With this setting, the frequency domain-based block composite attention module (FBAM) enhances the understanding of small aerial targets by combining frequency domain analysis, greatly enriches the diversity of different frequency features, and optimizes the utilization of frequency information.

[0026] Preferably, a two-dimensional discrete cosine transform 2D-DCT is used to extract the local sub-block C i frequency components.

[0027] In this way, 2D-DCT is introduced into the channel attention mechanism to adaptively obtain more robust frequency domain features on different frequency components.

[0028] Preferably, the FCTB module is configured to:

[0029] First, the feature map is subjected to a 1×1 convolution operation with a stride of 1, which reduces the number of channels of the feature map by half while keeping the spatial size of the feature map unchanged.

[0030] Afterwards, the feature map is processed through two branches respectively; one branch enters the SpectralBlock operation to capture the frequency components in the feature map; the other branch enters the Convolutional Multi-Head Self-Attention (CMHSA) mechanism, which processes the feature map through CMHSA to capture the dependencies and global information between different positions in the feature map;

[0031] Then, the output feature maps of the two branches are spliced ​​together, and the spliced ​​feature map is subjected to another 1×1 convolution operation with a stride of 1 to obtain the final feature map.

[0032] With this setup, the FCTB module, through the combination of spectral blocks and the convolutional multi-head self-attention mechanism, can simultaneously capture the frequency components and global dependencies in the feature map. This multi-scale, multi-angle feature fusion helps enhance the model's representational capabilities, enabling it to more comprehensively understand the input data. Furthermore, for tasks that require simultaneous consideration of local details and global information (such as image recognition and video analysis), the FCTB module may bring performance improvements. By combining the advantages of spectral blocks and the convolutional multi-head self-attention mechanism, the model can more accurately capture and understand the key information in the input data.

[0033] Preferably, the SpectralBlock is composed of a spectrum gating network; the spectrum gating network includes a fast Fourier transform FFT layer, a weighted gating and an inverse fast Fourier transform IFFT layer.

[0034] Preferably, the working process of the Spectral Block includes:

[0035] First, use FFT to convert the image space into spectrum space;

[0036] Afterwards, weighted gating applies a specific learnable weight parameter w c To determine the weight of each frequency component and learn through back-propagation technology to capture the lines and edges of the image;

[0037] Then, the weighted spectrum data is converted back to the image space through IFFT, normalized by layer normalization, and channel mixing is achieved through the multi-layer perceptron (MLP) block to obtain the frequency domain feature extraction result;

[0038] Finally, the frequency domain feature extraction results are element-wise added to the original input in a residual connection manner.

[0039] This setting, through the Spectral Block, can capture the different frequency components in the image to understand the local frequency.

[0040] The image is converted from the spatial domain to the frequency domain through FFT, so that the high-frequency components such as lines and edges in the image and the low-frequency components such as background and smooth areas can be separated in the spectrum. The weighted gating mechanism uses the learnable weight parameter w c Weighting different frequency components helps the model learn which frequency components are more important for specific tasks during training. This frequency domain feature extraction method can more directly focus on texture, structure, and other characteristics of the image compared to performing convolution operations directly in the spatial domain.

[0041] The weighted gating mechanism not only helps the model capture important frequency components but also potentially suppresses noise or unnecessary frequency components to a certain extent. Because noise tends to be distributed in high-frequency regions, weighted gating can reduce the contribution of these regions through learned weight parameters. Furthermore, converting the weighted spectral data back to image space through IFFT and further processing with layer normalization and MLP blocks can further enhance the model's ability to extract important features and reduce noise interference.

[0042] The MLP block performs channel mixing within the spectral block, fusing features from different channels to enhance the model's representational capabilities. This channel mixing helps the model understand images from multiple perspectives and capture richer feature information. Finally, a residual connection is used to perform element-wise addition of the frequency domain feature extraction results to the original input. This helps maintain the integrity of the input information and mitigates the problem of vanishing or exploding gradients in deep networks. Residual connections facilitate model convergence during training and improve generalization.

[0043] Preferably, the process of processing the feature map by the convolutional multi-head self-attention CMHSA mechanism includes:

[0044] Flatten the projected markers into a one-dimensional vector:

[0045]

[0046] in, is the labeled input of the Q / K / V matrix of the i-th layer; Flatten() represents the flattening operation; Conv is the standard convolution operation; s=1 represents the size of the convolution kernel; x i is the mark before convolution projection;

[0047] The attention function between Q, K and V is:

[0048]

[0049] Where c is the number of channels of the input token; k = 4 represents the number of heads of CMHSA; Conv represents a standard 1×1 convolution operation; the dot product matrix is ​​calculated by instance normalization IN after Softmax;

[0050] The output values ​​of each head are concatenated through linear projection and adjusted to a two-dimensional feature map as the final output.

[0051] Thus, compared with linear projection, each convolutional projection of CMHSA only requires s 2 C 2 Parameters and O(s 2 C 2T) FLOPs, where T is the number of tags processed and C is the channel dimension of the tags. These parameters are negligible compared to the model's FLOPs and total parameter count. For vehicle target detection from a drone's perspective, the traditional multi-head self-attention mechanism (MSHA) suffers from high computational complexity, a large number of parameters, and inferior processing efficiency of local features compared to convolutional operations. In contrast, the convolutional multi-head self-attention (CMMHSA) designed in this invention has the advantages of low computational complexity and a small number of parameters.

[0052] Preferably, the attention prediction head APH is set based on the normalized attention module NAM attention mechanism; the attention prediction head APH infers the attention maps of the input image in two independent dimensions, channel and space, respectively, and uses these attention maps to weight the input feature map, thereby adaptively capturing the salient areas of the image.

[0053] Such a setting can extract key areas, make the model focus on useful objects, and improve the detection model's ability to resist confusing information.

[0054] Preferably, the attention prediction head APH includes a channel attention submodule and a spatial attention submodule; the channel attention submodule uses a batch normalization scaling factor to reflect the importance of each channel; the spatial attention submodule uses the same normalization method to process each pixel in the space to suppress irrelevant features;

[0055] Wherein, the scaling factor B out The calculation formula is as follows:

[0056]

[0057] Where BN represents the batch normalization operation, μ B and σ B are the mean and standard deviation of the small batch samples B, respectively, while γ and β are trainable affine transformation parameters; ε is a small constant, usually 10-5 or 10-6, to prevent zero division and improve the numerical stability of the calculation; B in Input feature map to the channel attention submodule;

[0058] The expression of the channel attention submodule is:

[0059] M C =sigmoid(w γ (BN(F1)));

[0060] Among them, M c represents the output feature, γ is the scaling factor for each channel; weight w γ =γ i / ∑ j＝0 γ j .

[0061] The expression of the spatial attention submodule is:

[0062] M s =sigmoid(W λ (BN(F2)));

[0063] Among them, M s represents the output feature, λ is the scaling factor for each spatial dimension, and the weight w λ =λ i / Σ j＝0 λ j .

[0064] In this way, APH, combining the channel attention submodule and the spatial attention submodule, can adaptively weight the input feature map from both the channel and spatial dimensions, thereby more accurately capturing salient areas and key features in the image. This mechanism helps improve the overall performance of the model, including classification accuracy, detection precision, and robustness. Furthermore, because both submodules utilize normalization methods for calculation, the overall computational efficiency is also high, making it suitable for deployment in real-world applications.

[0065] Preferably, the feature fusion module is a feature fusion module based on a bidirectional feature pyramid network BiFPN. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to make the purpose, technical solutions and advantages of the invention more clear, the present invention will be further described in detail below with reference to the accompanying drawings, in which:

[0067] Figure 1 Schematic diagram of the target detection network CT-RODN in the embodiment;

[0068] Figure 2 Schematic diagram of the FBAM module in the embodiment;

[0069] Figure 3 Schematic diagram of the FCTB module in the embodiment;

[0070] Figure 4 Schematic diagram of the feature fusion module FFM in the embodiment;

[0071] Figure 5 Schematic diagram of a weighted fusion module in the embodiment;

[0072] Figure 6 Schematic diagram of the attention prediction head APH in the embodiment. DETAILED DESCRIPTION

[0073] The following is a further detailed description through specific implementation methods:

[0074] Example:

[0075] This embodiment discloses a target detection method for a drone perspective, including the following steps:

[0076] S1. Construct the target detection network CT-RODN. The structure of the target detection network CT-RODN is as follows: Figure 1 As shown in the figure, CT-RODN consists of a backbone network, a feature fusion module, and an attention prediction head. The backbone network is used to extract features from the input image. The feature fusion module is used to fuse the extracted features at each level. The attention prediction head is used to extract the key areas of the fused feature map and perform target detection and recognition based on the extracted key areas.

[0077] Among them, the backbone network includes a frequency-domain-based block composite attention module FBAM and a frequency-domain-based CNN-Transformer module FCTB; the FBAM module is used to enhance the understanding of small aerial targets by combining frequency-domain analysis; the FCTB module is used to process and aggregate the feature maps obtained by convolution to obtain the global information of the image; the working process of the backbone network includes: first, for the input image, four times downsampling is performed through the convolution modules Conv1 and Conv2; then, the FBAM module and the Conv module are used alternately for feature extraction and image downsampling; then, the FCTB module is used to process and aggregate the feature map information captured by convolution to obtain the global information of the image, and at the same time, fast spatial pyramid pooling SPPF is introduced to construct feature maps of different scales through multi-branch pooling layers, and fuse them to enhance the expressive ability of features.

[0078] This paper takes Convolutional Neural Network (CNN) and Transformer as the basic structure, introduces the attention mechanism and frequency domain feature extraction technology, and proposes a CNN and Transformer based Real-time Object Detection Network (CT-RODN), which can achieve accurate recognition of large, medium and small targets from the perspective of drones.

[0079] CT-RODN consists of four main components: a frequency-domain based block-and-compound attention module (FBAM), a frequency-domain based CNN-Transformer block (FCTB), a feature fusion module (FFM), and an attention prediction head (APH). The backbone network, consisting of the FBAM and FCTB, is used to efficiently extract features from the input image.

[0080] This network is a single-stage detection network, requiring only a single forward network to directly predict the location and category of the object. Non-maximum suppression (NMS) is used to remove redundant, low-quality detection boxes to generate the final result. To balance detection accuracy and speed, the present invention resizes all input images to 640×640 resolution. After the image is input to the CT-RODN network, it is first downsampled by a factor of four using the convolutional modules Conv1 and Conv2 to reduce network parameters and computational overhead. Feature extraction and image downsampling are then performed alternately using the FBAM module and the Conv module. Next, the FCTB module is used to process and aggregate the feature map information captured by the convolution to obtain global image information. Fast Spatial Pyramid Pooling (SPPF) is introduced to construct feature maps of different scales through multi-branch pooling layers and fuse them to enhance feature representation. These modules constitute the model's backbone network. Building on this backbone network, the present invention designs an FFM module to fuse features from different layers within the network backbone. FFM injects more high-level semantic information into low-level features while providing richer details for high-level features, thereby enhancing the detector's cross-scale detection performance for small targets. By fusing multi-scale features, the network's cross-scale target detection capability is improved. Finally, the present invention designs an APH detection head to output the model's final detection results. APH introduces an attention mechanism to improve the ability to detect targets in complex backgrounds, detecting targets of different scales from feature maps of different resolutions.

[0081] Frequency domain-based block composite attention module FBAM

[0082] The frequency-domain-based block-based composite attention module (FBAM) enhances understanding of small aerial targets by combining frequency-domain analysis. This significantly enriches the diversity of different frequency features and optimizes the utilization of frequency information. This module first spatially partitions the input features, extracts frequency-domain information from them, then combines the frequency component features of different blocks to form local channel features, which are then concatenated with the original features that have not been extracted in the frequency domain. Finally, a 1×1 convolutional layer outputs a feature map of the same size as the original, thereby improving target recognition and enhancing the overall performance of the model.

[0083] The structure of the FBAM module is as follows Figure 2 In specific implementation, the working process of the FBAM module includes:

[0084] First, the input feature map F∈R C×H×W Evenly divided into multiple local blocks F along the spatial dimension P ; Then further divided into multiple local sub-blocks C in equal proportion along the channel dimension i , i∈{1,2,…,n}; where C i ∈R (C / n)×P×P ; P represents the size of the local block; n is the number of equal divisions.

[0085] In specific implementation, the selection of the local block size P should follow the following principles:

[0086] In order to adapt to the characteristics of the network using multi-scale feature maps for detection, the local block size on each feature map should be consistent;

[0087] The width and height of the local block should be able to divide the width and height of the feature map to avoid the use of padding operations and thus prevent the introduction of irrelevant information.

[0088] Then, for each local sub-block C i , extract its frequency component and compare it with the corresponding local sub-block C i Perform element-by-element multiplication to obtain the corresponding frequency component features; and divide each local sub-block C i The frequency component features of the local block are spliced ​​and combined to form the channel frequency attention vector Freq of the local block.

[0089] Each local sub-block C i A corresponding 2D-DCT frequency component is assigned for element-wise multiplication with the corresponding block. Each DCT frequency component can be expressed by the following formula:

[0090]

[0091] where ui and v i Represents the corresponding X i The two-dimensional index of the frequency components, and It is a multidimensional vector obtained by aggregation. After frequency domain feature extraction, the generated vector contains the feature information of different frequency components in the frequency domain. These vectors are spliced ​​together to form a channel frequency attention vector Freq of a local block. The formula is as follows:

[0092] Freq=cat([Freq1,Freq2,…,Freq n ]).

[0093] Next, the frequency attention vector Freq and the corresponding local block feature F P Perform element-by-element multiplication to obtain the block frequency channel attention feature combination; and restore the block frequency channel attention feature combination to its original size to obtain the complete aggregation vector F c ; F c Concat the original feature vector F to obtain an intermediate feature map with a channel dimension of 2C, and use a convolution layer with a convolution kernel size of 1×1 to reduce its channel number to C to obtain the feature map F s The calculation formula is as follows:

[0094] F s =Conv 1×1 (F c ; F).

[0095] The purpose of this operation is to ensure the integrity of spatial information by combining features that have not been extracted in the frequency domain, while reducing misunderstandings or biases that may be caused by a single source of information.

[0096] Then, for the feature map F s , perform maximum pooling and average pooling along the channel direction to generate feature maps and Will and Splicing is performed along the channel, and the spliced ​​features are reduced in dimension using 7×7 convolution, and then the Sigmoid activation function is used to generate feature E s ∈R 1×H×w ;

[0097] Finally, E s With F S Multiply them together to get the attention feature F after the channel and spatial information are aggregated o The calculation formula is as follows:

[0098] F o =E s ·Fs =τ(Conv 7×7 (MaxPool(F s );AvgPool(F s )))·F s .

[0099] With this setting, the frequency domain-based block composite attention module (FBAM) enhances the understanding of small aerial targets by combining frequency domain analysis, greatly enriches the diversity of different frequency features, and optimizes the utilization of frequency information.

[0100] In specific implementation, two-dimensional discrete cosine transform 2D-DCT is used to extract the local sub-block C i frequency components.

[0101] In order to retain and introduce more frequency domain information, the present invention uses a two-dimensional discrete cosine transform (2D-DCT) to map the input features to the frequency domain. The result of the 2D-DCT transform includes a direct current component (DC coefficient) and an alternating current component (AC coefficient), where the DC coefficient represents the average component of the image and reflects low-frequency information; the AC coefficient captures high-frequency information, such as line boundaries and textures. 2D-DCT converts a two-dimensional image into the frequency domain, calculates which two-dimensional cosine waves the image consists of, and identifies areas with a higher degree of change in the image (for example, the target boundary area is rich in high-frequency information). Let represents a two-dimensional image, then The pixel value corresponding to the input position (i, j). Here i represents the row index and j represents the column index, i∈{0,1,…,H-1}, j∈{0,1,…,W-1}, so the two-dimensional discrete cosine transform can be expressed by the following formula:

[0102]

[0103] in It is a 2D-DCT spectrum graph, h and w represent the position of the spectrum graph. In the frequency spectrum, h∈{0,1,…,H-1}, w∈{0,1,…,W-1}. is defined as the basis function of the two-dimensional discrete cosine transform. The basis function is expressed as The formula is abbreviated as follows:

[0104]

[0105] When h and w in the above formula are both zero, the result of the formula is equivalent to global average pooling (GAP), which is expressed as That is, the calculated GAP result is proportional to the lowest frequency of the 2D-DCT, indicating that GAP is a special case of the 2D-DCT. The use of global average pooling (GAP) in the channel attention mechanism tends to retain low-frequency information while discarding components at other frequencies. Therefore, the present invention introduces the 2D-DCT into the channel attention mechanism to adaptively obtain more robust frequency domain features across different frequency components.

[0106] Frequency Domain-Based CNN-Transformer Module (FCTB)

[0107] The structure of the FCTB module is as follows Figure 3 The work of the FCTB module includes:

[0108] First, the feature map undergoes a 1×1 convolution operation with a stride of 1, which halves the number of channels of the feature map from c1 to c2 / 2, while keeping the spatial size of the feature map unchanged.

[0109] Afterwards, the feature map is processed through two branches respectively; one branch enters the SpectralBlock operation to capture the frequency components in the feature map; the other branch enters the Convolutional Multi-Head Self-Attention (CMHSA) mechanism, which processes the feature map through CMHSA to capture the dependencies and global information between different positions in the feature map;

[0110] Then, the output feature maps of the two branches are concatenated, and the concatenated feature map is subjected to another 1×1 convolution operation with a stride of 1 to obtain the final feature map. The number of channels of the final output feature map is c2.

[0111] With this setup, the FCTB module, through the combination of spectral blocks and the convolutional multi-head self-attention mechanism, can simultaneously capture the frequency components and global dependencies in the feature map. This multi-scale, multi-angle feature fusion helps enhance the model's representational capabilities, enabling it to more comprehensively understand the input data. Furthermore, for tasks that require simultaneous consideration of local details and global information (such as image recognition and video analysis), the FCTB module may bring performance improvements. By combining the advantages of spectral blocks and the convolutional multi-head self-attention mechanism, the model can more accurately capture and understand the key information in the input data.

[0112] 1) About Spectral Block

[0113] like Figure 3 As shown on the left side of the diagram, the Spectral Block consists of a spectrum gating network, which includes a Fast Fourier Transform (FFT) layer, weighted gating, and an Inverse Fast Fourier Transform (IFFT) layer. The Spectral Block's working process includes:

[0114] First, use FFT to convert the image space into spectrum space;

[0115] Afterwards, weighted gating applies a specific learnable weight parameter w c To determine the weight of each frequency component and learn through back-propagation technology to capture the lines and edges of the image;

[0116] Then, the weighted spectrum data is converted back to the image space through IFFT, normalized by layer normalization, and channel mixing is achieved through the multi-layer perceptron (MLP) block to obtain the frequency domain feature extraction result;

[0117] Finally, the frequency domain feature extraction results are element-wise added to the original input in a residual connection manner.

[0118] This setting, through the Spectral Block, can capture the different frequency components in the image to understand the local frequency. By converting the image from the spatial domain to the frequency domain through FFT, high-frequency components such as lines and edges in the image and low-frequency components such as background and smooth areas can be separated in the spectrum. The weighted gating mechanism uses a learnable weight parameter w c Weighting different frequency components helps the model learn which frequency components are more important for specific tasks during training. This frequency-domain feature extraction method can more directly focus on image characteristics such as texture and structure, compared to direct convolution operations in the spatial domain. The weighted gating mechanism not only helps the model capture important frequency components but also suppresses noise or unnecessary frequency components to a certain extent. Because noise is often distributed in high-frequency regions, weighted gating can reduce the contribution of these regions through learned weight parameters. Furthermore, converting the weighted spectral data back to image space through IFFT and further processing it with layer normalization and MLP blocks can further enhance the model's ability to extract important features and reduce noise interference. The MLP block performs channel mixing within the spectral block, fusing features from different channels to enhance the model's representational capabilities. This channel mixing method helps the model understand the image from multiple perspectives and capture richer feature information. Finally, using residual connections to perform element-wise addition of the frequency-domain feature extraction results to the original input helps maintain the integrity of the input information and mitigate the problems of vanishing and exploding gradients in deep networks. Residual connections make it easier for the model to converge during training and also improve the generalization ability of the model.

[0119] 2) About Convolutional Multi-Head Self-Attention CMHSA

[0120] In vehicle target detection from the perspective of drones, the use of traditional multi-head self-attention mechanism (MSHA) has problems such as high computational complexity, large number of parameters, and less effective processing of local features than convolution operations. In contrast, the convolutional multi-head self-attention (CMHSA) designed in the present invention has the advantages of low computational complexity and small number of parameters. Specifically, CMHSA reduces computational complexity and number of parameters and improves computational efficiency by introducing convolutional projection to replace the position linear projection in the original MHSA. In addition, the introduction of convolutional layers enables CMHSA to better capture local features, which is particularly important for vehicle target detection from the perspective of drones.

[0121] Figure 3 The right part shows how the input 2-D Token map is projected through the convolutional projection layer.

[0122] The process of processing the feature map by the convolutional multi-head self-attention CMHSA mechanism includes:

[0123] The projected markers are flattened into one-dimensional vectors for subsequent processing, which can be expressed as:

[0124]

[0125] in, is the labeled input of the Q / K / V matrix of the i-th layer; Flatten() represents the flattening operation; Conv is the standard convolution operation; s=1 represents the size of the convolution kernel; x i is the mark before convolution projection;

[0126] The attention function between Q, K and V is:

[0127]

[0128] Where c is the number of channels of the input token; k = 4 represents the number of heads of CMHSA; Conv represents a standard 1×1 convolution operation; the dot product matrix is ​​calculated by instance normalization IN after Softmax;

[0129] The output values ​​of each head are concatenated through linear projection and adjusted to a two-dimensional feature map as the final output.

[0130] Thus, compared with linear projection, each convolutional projection of CMHSA only requires s 2 C 2 Parameters and O(s 2 C 2T) FLOPs, where T is the number of tags processed and C is the channel dimension of the tags. These parameters are negligible compared to the model's FLOPs and total parameter count. For vehicle target detection from a drone's perspective, the traditional multi-head self-attention mechanism (MSHA) suffers from high computational complexity, a large number of parameters, and inferior processing efficiency of local features compared to convolutional operations. In contrast, the convolutional multi-head self-attention (CMMHSA) designed in this invention has the advantages of low computational complexity and a small number of parameters.

[0131] Feature Fusion Module (FFM)

[0132] Since drones fly at high altitudes, the targets within their field of view are small and lack visual details, which means that it is very difficult for the detection model to detect small targets well. Affected by various emergencies, the flight altitude of the drone in the work scene may change dramatically, and the target scale in its field of view also fluctuates, which requires the detection model to have a strong cross-scale target detection capability. Based on this, the present invention introduces a feature fusion module (FFM) based on a bidirectional feature pyramid network (BiFPN) to fuse features of different levels, so as to predict targets of different scales using feature maps of different levels.

[0133] BiFPN is optimized based on the PANet network of Yolov5. Figure 4 (a) shows that when the network has only one input edge, the node does not perform feature fusion and contributes little to the fusion between different features. Therefore, the present invention removes these nodes to simplify the network structure. Figure 4 As shown in (b), BiFPN removes the and The present invention also adds skip connections between sibling input and output nodes so that more features can be fused without significantly increasing the computational cost, such as Figure 4 In addition, the contributions of input features of different resolutions are different. The present invention adds additional weights to each input and proposes an efficient weighted fusion module (Weighted Fusion Module), as shown in Figure 5 The calculation formula is as follows:

[0134]

[0135] in, represents the input feature, ω i is a learnable weight, and ωi ≥0. In order to improve the stability of training, the present invention normalizes the weights to [0,1] and sets ε to a smaller value to improve the stability of calculation. Figure 4 Taking the third layer in (b) as an example, the calculation process of the two feature fusion is as follows:

[0136]

[0137] Among them, Resize is a downsampling or upsampling operation used to match the resolution of different fusion features, and Conv represents a standard 1×1 convolution. is the intermediate feature of the third layer in the top-down pathway, and is the output feature of layer 3 in the bottom-up path. Features at other positions are fused in a similar way.

[0138] Attention Prediction Head (APH)

[0139] In drone images, the wide field of view always contains many confusing environmental factors. In order to improve the detection model's ability to resist confusing information, the present invention applies the Normalization-based Attention Module (NAM) attention mechanism to form Attention Prediction Heads (APH) before the detection head, thereby extracting key areas and allowing the model to focus on useful objects. NAM is a lightweight and efficient attention mechanism that adopts the integration method of Convolutional Block Attention Module (CBAM) and redesigns the channel and spatial attention submodules. For the channel attention submodule, it uses the scaling factor of batch normalization to reflect the importance of each channel.

[0140] The attention prediction head (APH) is based on the normalized attention module (NAM) attention mechanism. It infers attention maps of the input image in two independent dimensions, channel and spatial, and uses these attention maps to weight the input feature map, adaptively capturing salient areas of the image. This setup can extract key areas, allowing the model to focus on useful objects and improve the detection model's ability to resist confusing information.

[0141] The attention prediction head APH includes a channel attention submodule and a spatial attention submodule; the channel attention submodule uses a batch normalization scaling factor to reflect the importance of each channel; the spatial attention submodule uses the same normalization method to process each pixel in the space to suppress irrelevant features;

[0142] Wherein, the scaling factor Bout The calculation formula is as follows:

[0143]

[0144] Where BN represents the batch normalization operation, μ B and σ B are the mean and standard deviation of the mini-batch samples B, respectively, while γ and β are trainable affine transformation parameters; ε is a small constant, usually 10 -5 or 10 -6 , to prevent division by zero and improve the numerical stability of the calculation; B in Input feature map to the channel attention submodule;

[0145] The channel attention submodule is as follows Figure 6 (a) is shown, and its expression is:

[0146] M C =sigmoid(w γ (BN(F1)));

[0147] Among them, M C represents the output feature, γ is the scaling factor for each channel; weight w γ =γ i / ∑ j＝0 γ j .

[0148] The spatial attention submodule is as follows Figure 6 (b) is shown, and its expression is:

[0149] M s =sigmoid(W λ (BN(F2)));

[0150] Among them, M s represents the output feature, λ is the scaling factor for each spatial dimension, and the weight w λ =λ i / ∑ j＝0 λ i .

[0151] In this way, APH, combining the channel attention submodule and the spatial attention submodule, can adaptively weight the input feature map from both the channel and spatial dimensions, thereby more accurately capturing salient areas and key features in the image. This mechanism helps improve the overall performance of the model, including classification accuracy, detection precision, and robustness. Furthermore, because both submodules utilize normalization methods for calculation, the overall computational efficiency is also high, making it suitable for deployment in real-world applications.

[0152] S2. Obtain and process images captured from the drone's perspective to obtain training data; and use the training data to train the target detection network CT-RODN.

[0153] S3. Use the trained target detection network CT-RODN to perform actual drone target detection tasks.

[0154] When using this method for target recognition, the FBAM module incorporates frequency domain analysis to enhance understanding of small aerial targets. This emphasizes significant frequency components in the image, allowing the network to focus more closely on important local features, thereby improving the recognition accuracy of small, low-contrast targets. The FCTB module processes and aggregates the feature maps acquired by convolution to capture global image information. By incorporating the target's surrounding context and analyzing the overall spectral information, this improves target detection accuracy, particularly when the target is partially occluded. This context-aware approach significantly enhances the system's recognition capabilities in highly dynamic and complex environments, ensuring the performance and reliability of drones performing critical missions. Furthermore, the feature fusion module (FFM) in this method infuses lower-level features with more high-level semantic information and provides richer details for higher-level features, thereby enhancing the detector's cross-scale detection performance for small targets. Furthermore, the attention prediction head extracts key regions of the fused feature map. By introducing an attention mechanism, the ability to detect targets in complex backgrounds can be improved, allowing detection of targets of varying scales from feature maps of varying resolutions. Furthermore, a novel real-time target detection network framework based on CNN and Transformer was designed to balance model complexity and detection accuracy. This invention is expected to not only improve the speed and accuracy of target detection, but also significantly enhance the application potential of drones in real-world operating environments, meeting the urgent need for real-time and accurate target detection in the development of modern drone technology.

[0155] Using this method, the effectiveness of small target detection in highly dynamic environments can be guaranteed, thereby ensuring the reliability of target detection results of UAVs in complex backgrounds and multi-scale target scenes.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the technical solutions. Those skilled in the art should understand that modifications or equivalent replacements of the technical solutions of the present invention that do not depart from the purpose and scope of the technical solutions of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A target detection method for drone perspective, characterized in that: The following steps are involved: S1. Construct the object detection network CT-RODN. CT-RODN includes a backbone network, a feature fusion module, and an attention prediction head. The backbone network is used to extract features from the input image. The feature fusion module is used to fuse the extracted features at each level; the attention prediction head is used to extract the key areas of the fused feature map and perform target detection and recognition based on the extracted key areas; The backbone network includes a frequency-domain-based block-based composite attention module (FBAM) and a frequency-domain-based CNN-Transformer module (FCTB). The FBAM module enhances the understanding of small aerial targets by combining frequency-domain analysis. The FCTB module processes and aggregates the feature maps obtained by convolution to obtain global information about the image. The backbone network works as follows: First, the input image is downsampled fourfold through the convolution modules Conv1 and Conv2. Then, the FBAM module and the Conv module are used alternately to extract features and downsample the image. Next, the FCTB module processes and aggregates the feature map information captured by the convolution to obtain the global information of the image. At the same time, the fast spatial pyramid pooling (SPPF) is introduced to construct feature maps of different scales through multi-branch pooling layers and fuse them to enhance the expressiveness of features. S2. Obtain and process images captured from the drone's perspective to obtain training data; and use the training data to train the target detection network CT-RODN. S3. Use the trained target detection network CT-RODN to perform actual drone target detection tasks. The work of the FCTB module includes: First, the feature map is subjected to a 1×1 convolution operation with a stride of 1, which reduces the number of channels of the feature map by half while keeping the spatial size of the feature map unchanged. Afterwards, the feature map is processed through two branches respectively; one branch enters the SpectralBlock operation to capture the frequency components in the feature map; the other branch enters the Convolutional Multi-Head Self-Attention (CMHSA) mechanism, which processes the feature map through CMHSA to capture the dependencies and global information between different positions in the feature map; Then, the output feature maps of the two branches are spliced ​​together, and the spliced ​​feature map is subjected to another 1×1 convolution operation with a stride of 1 to obtain the final feature map.

2. The target detection method for drones according to claim 1, wherein: The working process of the FBAM module includes: First, the input feature map F∈R C×H×W Evenly divided into multiple local blocks F along the spatial dimension P ; Then further divided into multiple local sub-blocks C in equal proportion along the channel dimension i , i∈{1,2,…,n}; where C i ∈R (C / n)×P×P ; P represents the size of the local block; n is the number of equal divisions; Then, for each local sub-block C i , extract its frequency component and compare it with the corresponding local sub-block C i Perform element-by-element multiplication to obtain the corresponding frequency component features; and divide each local sub-block C i The frequency component features of the local block are spliced ​​and combined to form the channel frequency attention vector Freq of the local block; Next, the frequency attention vector Freq and the corresponding local block feature F P Perform element-by-element multiplication to obtain the block frequency channel attention feature combination; and restore the block frequency channel attention feature combination to its original size to obtain the complete aggregation vector F c ; F c Concat the original feature vector F to obtain an intermediate feature map with a channel dimension of 2C, and use a convolution layer with a convolution kernel size of 1×1 to reduce its channel number to C to obtain the feature map F s ; Then, for the feature map F s , perform maximum pooling and average pooling along the channel direction to generate feature maps and Will and Splicing is performed along the channel, and the spliced ​​features are reduced in dimension using 7×7 convolution, and then the Sigmoid activation function is used to generate feature E s ∈R 1×H×w ; Finally, E s With F s Multiply them together to get the attention feature F after the channel and spatial information are aggregated o .

3. The target detection method for drones according to claim 2, wherein: Use two-dimensional discrete cosine transform 2D-DCT to extract the local sub-block C i frequency components.

4. The target detection method for drones according to claim 1, wherein: The SpectralBlock is composed of a spectrum gating network; the spectrum gating network includes a fast Fourier transform (FFT) layer, a weighted gating layer, and an inverse fast Fourier transform (IFFT) layer.

5. The target detection method for drone perspective according to claim 4, characterized in that: The working process of SpectralBlock includes: First, use FFT to convert the image space into spectrum space; Afterwards, weighted gating applies a specific learnable weight parameter w c To determine the weight of each frequency component and learn through back-propagation technology to capture the lines and edges of the image; Then, the weighted spectrum data is converted back to the image space through IFFT, normalized by layer normalization, and channel mixing is achieved through the multi-layer perceptron (MLP) block to obtain the frequency domain feature extraction result; Finally, the frequency domain feature extraction results are element-wise added to the original input in a residual connection manner.

6. The target detection method for drone perspective according to claim 1, wherein: The process of processing the feature map by the convolutional multi-head self-attention CMHSA mechanism includes: Flatten the projected markers into a one-dimensional vector: in, is the labeled input of the Q / K / V matrix of the i-th layer; Flatten() represents the flattening operation; Conv is the standard convolution operation; s=1 represents the size of the convolution kernel; x i is the mark before convolution projection; The attention function between Q, K and V is: Where c is the number of channels of the input token; k = 4 represents the number of heads of CMHSA; Conv represents a standard 1×1 convolution operation; the dot product matrix is ​​calculated by instance normalization IN after Softmax; The output values ​​of each head are concatenated through linear projection and adjusted to a two-dimensional feature map as the final output.

7. The target detection method for drone perspective according to claim 1, wherein: The attention prediction head APH is based on the normalized attention module NAM attention mechanism setting; The attention prediction head (APH) infers the attention maps of the input image in two independent dimensions, channel and space, and uses these attention maps to weight the input feature map to adaptively capture the salient areas of the image.

8. The target detection method for drone perspective according to claim 7, characterized in that: The attention prediction head APH includes a channel attention submodule and a spatial attention submodule; the channel attention submodule uses a batch normalization scaling factor to reflect the importance of each channel; The spatial attention submodule uses the same normalization method to process each pixel in space to suppress irrelevant features; Wherein, the scaling factor B out The calculation formula is as follows: Where BN represents the batch normalization operation, μ B and σ B are the mean and standard deviation of the mini-batch samples B, respectively, while γ and β are trainable affine transformation parameters; ε is a constant; B in Input feature map to the channel attention submodule; The expression of the channel attention submodule is: M c =sigmoid(w γ (BN(F1))); Among them, M c represents the output feature, γ is the scaling factor for each channel; weight w γ =γ i / Σ j＝0 γ j ; The expression of the spatial attention submodule is: M s =sigmoid(W λ (BN(F2))); Among them, M s represents the output feature, λ is the scaling factor for each spatial dimension, and the weight W λ =λ i / ∑ i＝0 λ j .

9. The target detection method for drone perspective according to claim 1, wherein: The feature fusion module is a feature fusion module based on the bidirectional feature pyramid network BiFPN.

Images