Target detection method

Abstract

The invention provides a target detection method in order to solve the technical problem that real-time detection of targets under complex backgrounds such as aerial images and remote sensing images is difficult to achieve in an existing method. According to the method, a double-domain cooperation mechanism of spatial domain channel selection and frequency domain component selection is adopted; the spatial domain realizes channel dimension adaptive screening through multi-scale selectivity and dynamic selection of local features extracted by different convolution kernels; the frequency domain operation takes adaptive selection and dynamic modulation as the core, the features are converted to the frequency domain through FFT, then the frequency domain features are modulated through grouping convolution, a selection mechanism is introduced to realize adaptive screening of frequency domain components, the design emphasizes the selective integration of the frequency domain features, and the frequency domain features are converted into the frequency domain. Key global information can be flexibly reserved according to specific semantic requirements of different targets, and target recognition and positioning in aerial images and remote sensing images with complex backgrounds are achieved.

Description

Technical Field

[0001] This invention relates to the field of computer vision and target detection technology, and in particular to a target detection method. This method is applicable to multiple application scenarios such as aerial surveillance, disaster relief, traffic management, and precision agriculture, and can achieve real-time and accurate detection of targets with multi-scale, densely distributed small targets and complex backgrounds in aerial images, remote sensing images, and images with similar characteristics to aerial / remote sensing images. Background Technology

[0002] Drone aerial images exhibit characteristics significantly different from conventional natural images due to their unique imaging perspective and flexible acquisition methods. First, aerial images often cover large-scale scenes, including various targets such as roads, buildings, vehicles, pedestrians, and farmland, with complex and textured backgrounds. Targets are easily obscured by ground textures, shadows, and changes in lighting. Second, due to the uncertainty of drone flight altitude and shooting angle, the same type of target appears with significant size differences in the image, especially with a large number of small and densely packed targets, increasing the difficulty of detection. Simultaneously, both the drone and the target may be in motion during aerial photography, leading to blurring, occlusion, and distortion in the image, making target boundaries unclear. Furthermore, aerial images typically have high resolution, which, while rich in detail, significantly increases the computational cost of feature extraction and detection, placing higher demands on real-time performance.

[0003] Drone aerial target detection has significant practical applications. It can support public safety and emergency response, such as quickly locating trapped individuals during disaster relief and identifying unusual behavior or vehicles during security patrols. In smart cities and traffic management, aerial target detection enables real-time analysis of road congestion, traffic flow, and illegal activities, providing data support for decision-making. In agriculture and forestry, detecting crop growth status and identifying pests, diseases, or illegal logging can significantly improve management efficiency and timeliness. In the inspection of energy, power, and oil infrastructure, automated defect detection using aerial imagery can reduce manual inspection costs and enhance safety.

[0004] For the task of target detection in drone aerial photography, existing methods can be divided into the following two categories:

[0005] 1. Traditional Target Detection Methods: Traditional target detection methods, represented by the YOLO and DETR series, were originally designed for natural scenes. In drone aerial photography scenarios, they lack adaptive optimization and often suffer from poor detection results, high computational resource consumption, and poor real-time performance.

[0006] 2. Target Detection Methods Customized for UAV Aerial Photography Scenarios: In response to the aforementioned challenges in aerial images, several methods specifically designed for aerial photography scenarios have been proposed in recent years, improving target detection performance through feature alignment and feature fusion. However, the inventors of this application have found in practice that current target detection methods customized for UAV aerial photography scenarios still have the following technical problems:

[0007] (1) Poor detection performance in complex scenarios, and difficulty in achieving effective target recognition and localization with local information;

[0008] Aerial images are characterized by complex backgrounds, multi-scale structures, and densely distributed small targets. In such cases, local features may not provide sufficient information for accurate target identification. Combining the relationship between objects and their surrounding environment is an effective way to enhance detection accuracy. Previous methods relied solely on spatial modeling, which struggled to effectively capture information about the target's surroundings. Although some recent studies (such as...) While attempts have been made to incorporate frequency domain information to capture global dependencies, these methods typically employ globally homogeneous frequency domain modulation strategies. They merely transform features to the frequency domain, perform frequency domain modulation through a convolution operation, and then transform them back to the spatial domain for global feature enhancement or to preserve high-frequency edge features. However, they lack discriminative selection of key frequency components and struggle to dynamically adjust based on the specific semantic requirements of different targets. Therefore, existing methods exhibit poor detection performance for aerial images with complex backgrounds, making it difficult to identify and locate targets in aerial images.

[0009] (2) Key information is lost during feature fusion, resulting in poor detection performance for small targets;

[0010] In urban scenes, common small targets such as traffic signs and pedestrians are tiny and easily occluded in aerial images. Existing algorithms, when fusing multi-scale features, struggle to accurately locate these small targets because deep features focus on semantic information and lack spatial details. For example, for dense pedestrians on a road, while deep features can determine the high-dimensional semantics of the target, they cannot accurately pinpoint its specific coordinates in the image; while shallow features can capture local details of the target's location, they lack sufficient semantic information to accurately identify its meaning. Therefore, multi-scale feature fusion is crucial for small target detection in aerial images. However, because deep features are semantically rich and shallow features provide detailed spatial information, misalignment can easily occur during fusion, severely impacting the accuracy requirements for small target detection in aerial images and affecting actual detection results. Furthermore, existing algorithms, when fusing multi-scale features at different levels, have failed to address and effectively resolve the semantic gap and spatial misalignment issues.

[0011] The same technical problems exist for target detection in other images with similar characteristics to aerial images, such as target detection in remote sensing images. Summary of the Invention

[0012] To address the technical challenge of real-time target detection in complex backgrounds such as aerial and remote sensing images using existing methods, this invention proposes a target detection method.

[0013] Furthermore, the present invention also solves the technical problem that existing methods have poor detection performance for small targets in aerial images and remote sensing images with high positioning accuracy requirements.

[0014] The technical solution of this invention is:

[0015] A target detection method, characterized by the following steps:

[0016] Step 1: Obtain the datasets used to build the training and test sets;

[0017] The dataset includes remote sensing / aerial images and their labels; the labels are the location, pixel size, and category of each target in the aerial / remote sensing images.

[0018] Step 2: Image preprocessing and dataset partitioning;

[0019] If the resolution of the remote sensing / aerial images in the dataset is greater than 3072... If the dataset contains 1728 pixels, it is then sliced, scaled proportionally, and padded with pixels in sequence before the dataset is divided into training and testing sets.

[0020] Step 3: Construct the target detection model;

[0021] The target detection model consists of a serially connected multi-domain feature extraction network, a feature fusion network, and a detection head;

[0022] The multi-domain feature extraction network comprises a serially connected preliminary feature extraction module and n multi-domain feature extraction modules; n=3 or 4. The preliminary feature extraction module is used to downsample the image and extract preliminary features, which serve as the input to the top-level multi-domain feature extraction module. Each multi-domain feature extraction module includes a spatial-frequency selection module, a channel attention mechanism, and a convolution module. During spatial processing, the spatial-frequency selection module uses square convolution and horizontal and vertical strip convolution to extract dense local features, horizontal local features, and vertical local features from the received features. Then, it uses channel selection weights to adaptively converge the local features to obtain the extracted spatial features. During frequency processing, the spatial-frequency selection module adaptively filters and modulates the frequency feature components in the received features to capture the contextual information around different targets, thus obtaining the extracted frequency features. The channel attention mechanism is used to recalibrate the extracted spatial and frequency features. The convolution module is used to refine the recalibrated features and rearrange the channels to obtain a feature map that integrates local and global features. This feature map is the output of the multi-domain feature extraction module.

[0023] The feature fusion network is used to fuse the shallow feature map output by at least one multi-domain feature extraction module located in the shallow layer and the deep feature map output by at least one multi-domain feature extraction module located in the deep layer of the multi-domain feature extraction network, and output a fused feature map.

[0024] The detection head is used to extract the location, pixel size, and category of the target from the fused feature map;

[0025] Step 4, Model Training and Testing;

[0026] The target detection model is trained and tested using training and testing sets;

[0027] Step 5, target detection;

[0028] After the remote sensing / aerial images are processed by the preprocessing method in step 2 to reduce their pixel count, they are input into the trained target detection model. The target detection model outputs the target detection results, including the location and size of the target and the category to which the target belongs.

[0029] Furthermore, the method for a single multi-domain feature extraction module to extract features from the preliminary features is as follows:

[0030] Step 3A.1, input the features of the multi-domain feature extraction module Downsampling is performed through downsampling convolution to obtain features. ;

[0031] Step 3A.2: For features First, channel expansion is performed, then the data is partitioned along the channel dimension to obtain the original input features for subsequent spatial and frequency domain processing. ;

[0032] Step 3A.3, spatial processing;

[0033] By using stacked square convolutions and strip convolutions, features from the original input are... Extract dense multi-scale local features and then perform analysis on the original input features. Perform average pooling, channel expansion, dimension transformation, and... After calculation, channel selection weights are generated. These weights are then used to effectively select from the extracted dense multi-scale local features, resulting in local detail features. ;

[0034] Step 3A.4, frequency domain processing;

[0035] First, the original input features Frequency domain features are obtained by mapping to the frequency domain using Fast Fourier Transform. Subsequently, the frequency domain features Divide the data into m parts along the channel dimension to obtain frequency domain sub-features. Next, for each frequency domain sub-feature Independent modulation is performed using 1×1 convolution / filter; and each frequency domain sub-feature is modulated. Simultaneously with modulation (before / after), frequency domain features The number of channels is compressed to m, and then the frequency domain characteristics after channel compression are analyzed. Perform in the channel dimension The calculations yielded the sub-features in each frequency domain. The selection weights are then used; then, the frequency domain sub-features are utilized. The selection weights for independently modulated frequency domain sub-features Frequency domain sub-bands at various spatial locations are filtered to obtain adaptive frequency domain features. Finally, the frequency domain features The global semantic features are obtained by returning to the spatial domain through the inverse fast Fourier transform. ;

[0036] Steps 3A.3 and 3A.4 above can be performed in a different order or in parallel;

[0037] Step 3A.5, extract local detail features With global semantic features By splicing along the channel dimension, the features are obtained. ;

[0038] Step 3A.6, feature Recalibration is performed using a channel attention mechanism to highlight information-rich channels, resulting in calibrated features. ;

[0039] Step 3A.7, for features Channel compression is performed to obtain feature maps. ;

[0040] Step 3A.8, feature With feature map Perform residual connections to obtain features And output it.

[0041] Furthermore, the feature fusion network in the target detection model constructed in step 3 is an improved feature fusion network. This improved fusion network has feature alignment + fusion functions. For the shallow feature map and deep feature map that are input, the semantic preliminary alignment of the channel dimension and the fine alignment of the spatial position are performed first, and then adaptive channel selection fusion is performed to obtain the fused feature map.

[0042] Furthermore, the improved feature fusion network includes 1×1 convolutions for channel dimension transformation, upsampling convolutions and downsampling convolutions for scale transformation, and s feature alignment and fusion modules; s is determined based on the number of feature maps to be aligned and fused and their layer level: if the feature map input to the improved feature fusion network is from the adjacent front end... A multi-domain feature extraction module, then If the feature maps input to the improved feature fusion network are from non-adjacent front-end features... A multi-domain feature extraction module, then The feature fusion network first performs stepwise alignment and fusion processing from deep to shallow layers, and then from shallow to deep layers. Along the path from deep to shallow layers, 1×1 convolutions and upsampling convolutions are used to convert the two features in each feature alignment and fusion module along this path into features with consistent channel dimensions and scale. Along the path from shallow to deep layers, downsampling convolutions and 1×1 convolutions are used to convert the two features in each feature alignment and fusion module along this path into features with consistent channel dimensions and scale. The fused features output by the feature alignment and fusion modules along the path from shallow to deep layers are the output of the entire improved feature fusion network. Each individual feature alignment and fusion module processes the input features... and The process of feature alignment and fusion is as follows:

[0043] Step 1) Align the channel dimensions;

[0044] Align features using independent 1×1 convolutions and The dimension is used to obtain the features. ;

[0045] Step 2) Semantic and spatial alignment;

[0046] Step 2.1) Channel semantic alignment;

[0047] Features Features are formed by splicing along the channel dimension ;

[0048] Features Simultaneously perform global max pooling and global average pooling operations, and then sum the features obtained from the global max pooling and global average pooling operations element by element to obtain the feature. ;

[0049] Features are paired using 1×1 convolution. After channel compression, the result is Then, the features are processed by 1×1 convolution. Channel restoration was performed to obtain ;

[0050] Features Channel alignment weights are generated after processing by the sigmoid layer. ;

[0051] Features formed by splicing Alignment weights with channels Element-wise multiplication is performed to achieve preliminary channel semantic alignment of the multi-scale input features, resulting in preliminarily aligned features. ;

[0052] Step 2.2) Perform fine-grained alignment in the spatial dimension;

[0053] feature Semantic associations across spatial locations are performed using depthwise separable convolutions, generating fine-grained spatial alignment weights. ;

[0054] Fine-grained spatial alignment weights With features Perform element-wise positional multiplication to obtain the aligned features. ;

[0055] Step 3) Feature fusion;

[0056] Features after feature alignment Uniformly divided along the channel dimension, resulting in And introduce learnable channel selection parameters. Balancing features at different scales The contributions between them are analyzed to achieve optimal feature integration and obtain the fused features. .

[0057] Furthermore, the feature fusion network in the target detection model constructed in step 3 also includes a feature enhancement network. The feature enhancement network is set between the feature fusion network and the decoding detection module. It is used to perform fine-grained enhancement, local enhancement, and global enhancement on the fusion feature map output by the fusion network before decoding detection, and then perform residual connection with the original fusion feature map to obtain the enhanced multi-scale fusion feature map and output it.

[0058] Furthermore, the feature enhancement network includes multiple parallel multi-branch feature enhancement modules; each multi-branch parallel enhancement module enhances the input features. The process of comprehensive enhancement is as follows: First, the features Divide into enhancement branches evenly along the channel dimension and identity branches The enhancement branch performs subsequent multi-branch parallel enhancement, while the identity branch retains some input channels, promoting feature reuse and feature fusion; secondly, for the enhancement branch, the input features... pass Perform feature activation to obtain Subsequently The process is performed through four parallel branches: a residual branch, a fine-grained enhancement branch, a local enhancement branch, and a global contextual information enhancement branch. Finally, the identity branch is... Enhanced output features with four parallel branches After concatenation, channel blending is performed using 1×1 convolution to obtain the enhanced multi-scale feature map. .

[0059] Furthermore, in step 3, the detection head adopts a classification-regression-confidence decoupled architecture design.

[0060] Furthermore, the dataset in step 1 uses the public datasets VisDrone, UAVDT, CODrone, or UAVVaste.

[0061] The present invention also proposes a target detection model, which is special in that it is obtained by using steps 1-4 of the target detection method described above.

[0062] The beneficial effects of this invention are:

[0063] 1. This invention adopts a dual-domain collaborative mechanism of "spatial channel selection + frequency component selection": In the spatial domain, local features extracted by different convolution kernels are selectively and dynamically selected at multiple scales to achieve adaptive filtering of channel dimensions; the frequency domain operation is based on "adaptive selection and dynamic modulation". Features are transformed to the frequency domain through FFT, and then the frequency domain features are modulated through grouped convolution. A selection mechanism is introduced to achieve adaptive filtering of frequency domain components. This design emphasizes the "selective integration" of frequency domain features, which can flexibly retain key global information according to the specific semantic needs of different targets, and realize target recognition and localization in aerial images and remote sensing images with complex backgrounds.

[0064] 2. Aerial images cover a wide area, and complex backgrounds can severely interfere with the detection and recognition of foreground targets. For example, in urban aerial photography, building shadows may be misidentified as vehicles or other targets, leading to detection errors. Introducing surrounding contextual semantic information into targets is beneficial for distinguishing between foreground and background and helps the model achieve accurate localization. However, different targets require different contextual information. For example, denser, smaller targets need local details and the surrounding environment to help them be accurately located, while larger targets or background areas rely more on overall structural semantics. Therefore, it is necessary to provide adaptive contextual information for different targets to meet diverse needs. This invention introduces a multi-domain learning and selection mechanism in the feature extraction process, using channel selection to fuse multi-scale local features and discriminatively selecting frequency domain components to meet the different contextual needs of diverse targets in complex aerial photography scenarios, providing discriminative features for subsequent target detection and recognition.

[0065] 3. Some existing methods directly concatenate channels during feature aggregation using PAN after feature extraction, which can lead to the obfuscation (dilution) of key features of small targets during the fusion process. This invention adopts a semantically guided feature alignment fusion strategy, which effectively alleviates the problem of obfuscation (dilution) of key features of small targets during multi-scale feature fusion, thereby improving the detection effect of small targets in aerial photography.

[0066] 4. This invention first achieves a rough overall semantic alignment between different feature maps through channel alignment, and then further refines it through spatial position alignment to achieve a more precise alignment. This lays the foundation for effectively fusing deep and shallow features and improves the detection accuracy of small targets in aerial and remote sensing images.

[0067] 5. This invention also introduces a multi-branch parallel enhancement module to further enhance the fused multi-scale features, improve the discriminability between small target features and the background, and thus further improve the detection performance. Attached Figure Description

[0068] Figure 1 This is a flowchart of the target detection method proposed in Embodiment 1 of the present invention.

[0069] Figure 2 This is a schematic diagram of the structure and principle of the multi-domain feature extraction module, the core part of the multi-domain feature extraction network in Embodiment 1 of the present invention.

[0070] Figure 3 This is a schematic diagram of the structural principle of the basic fusion module, the core part of the feature fusion network in Embodiment 1 of the present invention.

[0071] Figure 4 This is a flowchart of the target detection method proposed in Embodiment 2 of the present invention.

[0072] Figure 5 This is a schematic diagram of the structural principle of the feature alignment and fusion module, the core part of the feature fusion network in Embodiment 2 of the present invention.

[0073] Figure 6 This is a flowchart of the target detection method proposed in Embodiment 3 of the present invention.

[0074] Figure 7 This is a schematic diagram illustrating the structural principle of the first implementation method of the multi-branch feature enhancement module in Embodiment 3 of the present invention.

[0075] Figure 8 This is a schematic diagram illustrating the structural principle of the second implementation method of the multi-branch feature enhancement module in Embodiment 3 of the present invention.

[0076] Figure 9 This is a schematic diagram illustrating the structural principle of the third implementation method of the multi-branch feature enhancement module in Embodiment 3 of the present invention.

[0077] Figure 10 This is a flowchart of the target detection method proposed in Embodiment 4 of the present invention.

[0078] Figure 11 This is a flowchart of the target detection method proposed in Embodiment 5 of the present invention.

[0079] Figure 12 The figure shows the experimental test results of the target detection method proposed in Embodiment 3 of the present invention.

[0080] Figure 13 This is a comparison of the experimental test results of Embodiment 1 of the present invention with those of existing target detection methods.

[0081] Figure 14 This is a comparison of the experimental test results of Examples 1-3 of the present invention.

[0082] In the diagram: Mul&Add means multiplying each element at its position and then adding them together; c means splicing along the channel dimension. Detailed Implementation

[0083] The present invention will be further described in detail below with reference to the accompanying drawings.

[0084] Reference Figure 1 , 3 5. The target detection method proposed in this invention has the following specific steps:

[0085] Step 1: Obtain the datasets used to build the training and test sets;

[0086] Obtain a public dataset that includes drone aerial images and their labels, such as VisDrone, UAVDT, CODrone, or UAVVaste; or obtain a public dataset that includes remote sensing images and their labels, such as SIMD; where the labels are the location, pixel size, and category of each target in the aerial / remote sensing image.

[0087] Step 2: Image preprocessing and dataset partitioning;

[0088] Step 2.1: If the resolution of the aerial images / remote sensing images in the public dataset is greater than 3072... If the resolution is 1728 pixels, then the original aerial / remote sensing image needs to be sliced ​​to obtain several images of uniform size and lower resolution (e.g., 1280 pixels). Images with a resolution of 960 pixels or less provide standardized input for subsequent feature extraction and improve actual detection results; if the resolution of aerial / remote sensing images in the public dataset is less than or equal to 3072 pixels... If the image is 1728 pixels, there is no need to perform image slicing; proceed directly to step 2.2.

[0089] Step 2.2: To balance detection accuracy and computational cost, the aerial images / remote sensing images / sliced ​​images in the public dataset are scaled proportionally and pixel-filled, while maintaining consistent image size to further reduce image resolution; typically, scaling to a resolution of 640 is used. 640 pixels or 800 800 pixels.

[0090] Step 2.3: Divide the public dataset after preprocessing in Step 2.2 into a training set and a test set.

[0091] Step 3: Construct the object detection model;

[0092] The target detection model consists of a serially connected multi-domain feature extraction network, a feature fusion network, and a detection head.

[0093] Multi-domain feature extraction networks are used to extract multi-domain features from images in both the spatial and frequency domains to adaptively capture both local details and global semantics of the target. The multi-domain feature extraction network consists of a pre-extraction module connected in series and n multi-domain feature extraction modules; n=3 or 4. The pre-extraction module downsamples the image to extract preliminary features, which serve as input to the top-level multi-domain feature extraction module. A single multi-domain feature extraction module includes a spatial-frequency selection module, a channel attention mechanism, and a convolution module. Specifically: During spatial processing, the spatial-frequency selection module uses square convolutions and horizontal and vertical strip convolutions to extract dense local features, horizontal local features, and vertical local features from the received features. Then, it uses channel selection weights to adaptively converge these local features, resulting in the extracted spatial features. During frequency processing, the spatial-frequency selection module adaptively filters and modulates the frequency-domain feature components in the received features to accurately capture the contextual information surrounding different targets, resulting in extracted frequency-domain features. The channel attention mechanism is used to redirect (also known as recalibrate) the extracted spatial and frequency-domain features. The convolution module refines the redirected features and rearranges the channels, resulting in a feature map that effectively integrates local and global features. This feature map is the output of the multi-domain feature extraction module. The feature maps output by different multi-domain feature modules have different scales, thus the entire multi-domain feature extraction network outputs multi-scale feature maps.

[0094] The input to the multi-domain feature extraction network is the image preprocessed in step 2, and the output is a multi-scale feature map.

[0095] The feature fusion network is used to fuse shallow and deep features output by a multi-domain feature extraction network. Its input consists of shallow feature maps from at least one shallow multi-domain feature extraction module and deep feature maps from at least one deep multi-domain feature extraction module. The output is a fused feature map that integrates shallow detail information and deep semantic information.

[0096] The detection head is used to extract the target's location, pixel size, and category from the fused feature map. The detection head is an existing unit, preferably designed using a common classification-regression-confidence decoupled architecture. It is specifically adapted to the characteristics of multi-scale targets, densely distributed small targets, and complex backgrounds in aerial and remote sensing images. It works in conjunction with the front-end multi-domain feature extraction network and feature fusion network to achieve accurate target classification, localization, and confidence assessment.

[0097] As a further preferred option, the feature fusion network in the object detection model is an improved feature fusion network. This improved fusion network has feature alignment and fusion functions. For the input shallow and deep feature maps, it first performs preliminary semantic alignment at the channel dimension and fine alignment at the spatial position, and then performs adaptive channel selection fusion to obtain a fused feature map. The advantages of this preferred option are: preliminary semantic alignment at the channel dimension enables features from different channels to achieve initial semantic consistency, strengthening key semantic channels; fine alignment at the spatial position corrects the spatial offset of different features, solving the semantic misalignment problem between multi-scale features; and the use of adaptive channel selection fusion avoids the problem of dilution of key features of small targets during feature fusion.

[0098] As a further preferred option, the target detection model may also include a feature enhancement network. This feature enhancement network is positioned between the feature fusion network and the decoding detection module. Before decoding detection, it performs fine-grained, local, and global enhancements on the fused feature map output by the fusion network, then performs a residual connection with the original fused feature map to obtain an enhanced multi-scale fused feature map, which is then output. By comprehensively enhancing the fused features through the feature enhancement network, the difference between the target and the background in the feature map can be increased, improving the target detection model's ability to distinguish small targets with high localization accuracy requirements. Correspondingly, the input to the detection head module at this time is the enhanced multi-scale fused feature map output by the feature enhancement network.

[0099] Step 4: Model training and testing;

[0100] Using the training and test sets obtained in step 2, the target detection model is trained and tested using existing conventional methods until a target detection model that meets the recognition requirements is obtained.

[0101] Step 5: Target detection;

[0102] After the aerial images are processed by the preprocessing method in step 2 to reduce their pixel count, they are input into the trained target detection model. The target detection model outputs the target detection results, including the location and size of the target and the category to which the target belongs.

[0103] To make the technical solution of the present invention clearer and easier to understand, the present invention will be described in more detail below through several embodiments.

[0104] Example 1:

[0105] Reference Figure 1-3 This embodiment includes the following steps:

[0106] Step 1: Obtain the datasets used to build the training and test sets;

[0107] The publicly available dataset VisDrone was selected to construct the training and test sets.

[0108] Step 2: Image preprocessing;

[0109] The original aerial images in the publicly available VisDrone dataset have a resolution of less than 3072. 1728 pixels, without image slicing, the original aerial images are directly converted to a resolution of 640 by proportional scaling and pixel filling. 640-pixel image;

[0110] The VisDrone public dataset, after image preprocessing, is divided into training and testing sets.

[0111] Step 3: Construct the object detection model;

[0112] The target detection model consists of a serially connected feature extraction network, a feature fusion network, and a decoding detection module.

[0113] A. Multi-domain feature extraction network;

[0114] Feature extraction is fundamental for accurate target recognition and localization, and sufficient and effective feature extraction improves the actual detection performance of target detection models. Current feature extraction networks often rely on spatial domain modeling, which struggles to fully meet the diverse contextual information requirements of different targets, a problem particularly pronounced in complex aerial photography scenarios. For example, densely packed small targets require local details and the surrounding environment to aid in their accurate localization, while larger targets or background areas rely more heavily on overall structural semantics.

[0115] To address the aforementioned issues, this embodiment introduces a multi-domain feature extraction network based on multi-domain selection learning, such as... Figure 1 As shown, the multi-domain feature extraction network includes a pre-feature extraction module connected in series and four multi-domain feature extraction modules.

[0116] The preliminary feature extraction module is a convolutional layer with a kernel size of 3×3 and strid = 2. This convolutional layer downsamples the image input to the multi-domain feature extraction network to obtain the preliminary features. These preliminary features are the input to the top-level multi-domain feature extraction module.

[0117] like Figure 2 As shown, a single multi-domain feature extraction module includes a downsampling module, a convolution module, and a spatial frequency selection module. The single multi-domain feature extraction module processes the input features... The specific process of feature extraction is as follows:

[0118] Step 3A.1: Downsampling;

[0119] For the features of the input multi-domain feature extraction module First, the input features are processed by a 3×3 downsampled convolution with a stride of 2 (in other embodiments, larger convolutions such as 5×5 or 7×7 can also be used, but the computational cost will be greater) Perform downsampling to obtain features :

[0120]

[0121] in, This is a Batch Normalization layer used to constrain the numerical range of the weights of a 3×3 downsampled convolution with a stride of 2, thereby accelerating the convergence of the 3×3 downsampled convolution with a stride of 2. The activation function introduces nonlinearity into the 3×3 convolution with a stride of 2, enhancing its representational power.

[0122] Step 3A.2: Process the features obtained after downsampling First, channel expansion is performed, and then the data is divided along the channel dimension to obtain the original input features for subsequent spatial and frequency domain processing.

[0123] For the features obtained after downsampling First, channel expansion is performed using a 1×1 convolution (in other embodiments, larger convolutions can also be used, such as a 3×3 convolution with padding of 1; or a 5×5 convolution with padding of 2). Then, the channel-expanded features are divided into two sub-features along the channel dimension. and These are used as the original input features for subsequent spatial and frequency domain processing, respectively.

[0124] .

[0125] Step 3A.3: Spatial processing;

[0126] In the spatial processing workflow, to address the multi-scale target problem in aerial images, this embodiment utilizes stacked square convolutions and strip convolutions to extract features from the original input. Extract dense multi-scale local features and then perform analysis on the original input features. Perform average pooling, channel expansion, dimension transformation, and... After calculation, channel selection weights are generated, and these weights are used to effectively select the extracted dense multi-scale local features.

[0127] Specifically:

[0128] For the original input features The original input features are extracted by sequentially applying 3×3 standard convolution (square convolution), 1×3 depthwise convolution (strip convolution in the horizontal direction), and 3×1 depthwise convolution (strip convolution in the vertical direction). The system extracts dense local features, where the order of 1×3 depthwise convolutions and 3×1 depthwise convolutions can be interchanged. Furthermore, the convolution size can be adjusted to a larger size; for example, the original input features can be extracted by sequentially passing through a standard 5×5 convolution, a 1×5 depthwise convolution, and a 5×1 depthwise convolution. It produces dense local features. However, the computational cost increases when using larger convolution sizes.

[0129] Because a serial architecture is used, the outputs of the three convolutions... With different receptive fields, and the introduction of strip convolution, features in both horizontal and vertical directions can be captured, thus effectively capturing multi-scale local features:

[0130]

[0131]

[0132]

[0133] In obtaining multi-scale local features Simultaneously with the back / before / after, the original input features First, perform global average pooling, then expand and transform the channel dimensions using 1×1 convolutions (similarly, larger-sized convolutions can also be used for channel dimension expansion and transformation), and then... The operator obtains the selection weights of the channel dimension. This allows us to utilize the selection weights of the channel dimension. Local features at multiple scales Adaptive enhancement and suppression are performed. The relevant calculation process is as follows:

[0134]

[0135] in, For global average pooling, the processed dimension is The channel dimension is increased using 1×1 convolution, resulting in a new channel dimension of... In the process Before the operation, the channel dimensions are adjusted and transformed to... Then apply it on the 0th dimension. The function is used for computation to obtain local features at multiple scales. The sum of the selection weights on the same channel is 1, thus obtaining local features for multiple scales. Channel selection weight .

[0136] The calculated channel selection weights With multi-scale local features ( ) Perform element-wise positional multiplication and then sum them to obtain local detail features. :

[0137]

[0138] in, Local features The corresponding channel selection weight.

[0139] Step 3A.4: Frequency domain processing;

[0140] To capture the contextual dependencies of different targets, this embodiment introduces frequency domain component selection in frequency domain processing, selectively enhancing and suppressing different frequency band features, thereby providing adaptive contextual information for different targets.

[0141] First, for the original input features of the frequency domain processing branch The frequency domain features are obtained by mapping them to the frequency domain using a fast Fourier transform. :

[0142]

[0143] Subsequently, frequency domain features The target is divided into four parts along the channel dimension (in other embodiments, the number of groups can also be other values, such as 1, 2, 3, 5, 6, etc.; ablation experiments show that the target detection effect is significantly better when the number of groups is greater than 1 than when the number of groups is 1, and the target detection effect is best when the number of groups is 4), resulting in four frequency domain sub-features. .

[0144] Next, for each frequency domain sub-feature Independent modulation is achieved through 1×1 convolution. Frequency domain features are divided into low-frequency, mid-frequency, and high-frequency components. Low-frequency components determine the overall contour and are rich in semantic information, high-frequency components concentrate edge and detail features, and mid-frequency components carry the transition details between low-frequency and high-frequency components. The 1×1 convolution, with its linear combination capability, can selectively enhance or suppress these frequency components. In other embodiments, filters can also be used for independent modulation.

[0145] In terms of each frequency domain sub-feature Simultaneously with / before / after modulation, frequency domain features First, use a 1×1 convolution to compress its channel number to 4 (the compressed channel number corresponds to the previous frequency domain features). (Number of groups), and then the frequency domain characteristics after compression of the channel. Perform in the channel dimension The frequency domain sub-features can be obtained through computation. ( Selection weights .

[0146] Then, by selecting weights Frequency domain sub-features that have been independently modulated Frequency domain sub-bands at various spatial locations are filtered to obtain adaptive frequency domain features. The specific calculations are as follows:

[0147]

[0148]

[0149]

[0150] in, For a 1×1 convolution with C input channels and 4 output channels; For frequency domain sub-features The corresponding selection weight, each selection weight For size is The weighted selection graphs have a sum of 1 for all elements located at the same spatial position. To analyze the sub-features of each frequency domain Modulated 1×1 convolution; This is for element-wise multiplication.

[0151] Finally, the adaptive frequency domain features obtained after filtering are... The global semantic features are obtained by returning to the spatial domain through the inverse fast Fourier transform. :

[0152]

[0153] The order of steps 3A.3 and 3A.4 above can be interchanged or performed in parallel.

[0154] Step 3A.5: Concatenate the features obtained from spatial domain processing and frequency domain processing;

[0155] Multi-scale local detail features With global semantic features Concatenating along the channel dimension, for example, assuming that the multi-scale local detail features and the global semantic features have the same shape, are all... Then, after splicing along the channel dimension, we get The features. After splicing, the features are obtained. :

[0156] .

[0157] Step 3A.6: Attention mechanism calibration;

[0158] The features obtained by concatenating along the channel dimension are recalibrated using a channel attention mechanism to highlight the information-rich channels, resulting in calibrated features. :

[0159] .

[0160] Step 3A.7: Feature refinement;

[0161] The recalibrated features are processed using 1×1 convolutions (larger convolutions may also be used in other embodiments). Channel compression is performed to obtain feature maps. :

[0162] .

[0163] Step 3A.8: Residual connection;

[0164] A residual connection is introduced from the downsampled original features to the output of the space frequency selection module to ensure gradient flow, avoid network degradation, and obtain the features. And output:

[0165] .

[0166] feature This is the output of the multi-domain feature extraction module.

[0167] In summary, through the synergistic effect of spatial channel selection and frequency component selection, this embodiment achieves adaptive convergence and fusion of local multi-scale features and global semantic features in the feature extraction stage, thereby providing diverse contextual information for subsequent target detection tasks.

[0168] B. Feature fusion network;

[0169] The feature fusion network first fuses some / all features from the multi-scale features output by the multi-domain feature extraction network from deep layers to shallow layers, and then fuses them from shallow layers to deep layers.

[0170] Specifically, such as Figure 1 As shown, the feature fusion network in this embodiment includes 4 basic fusion modules, 2 1×1 convolutions, 2 upsampling modules, and 2 downsampling convolutions.

[0171] The process of gradual fusion from deep to shallow layers is as follows:

[0172] First, the features output by the multi-domain feature extraction module 4 in the multi-domain feature extraction network. First, a 1×1 convolution is used to transform the channel dimensions to obtain the features. Then, an upsampling module is used to process the features. Upsampling is performed to obtain features ;feature Features output by multi-domain feature extraction module 3 Same scale; features with the same scale With features It is sent to the first basic fusion module for fusion processing, and the fused features are output. .

[0173] Then, the fusion features output by the first basic fusion module First, a 1×1 convolution is used to transform the channel dimensions to obtain the features. Then, an upsampling module is used to process the features. Upsampling is performed to obtain features ;feature Features output by multi-domain feature extraction module 2 Same scale; features with the same scale With features It is sent to the second basic fusion module for fusion processing, and the fused features are output. .

[0174] The process of gradually aligning and blending from shallow to deep layers is as follows:

[0175] First, a 3×3 downsampling convolution with a stride of 2 is used to process the fused features output by the second basic fusion module (in other embodiments, larger convolutions such as 5×5 or 7×7 can also be used, but the computational cost will be greater). Perform downsampling to obtain features ;feature Features from the output of the second 1×1 convolution Same scale; features with the same scale and It is fed into the third basic fusion module for fusion processing, and the fused features are output. .

[0176] Then, a 3×3 downsampling convolution with a stride of 2 is used to process the fused features output by the third basic fusion module (in other embodiments, a larger convolution such as 5×5 or 7×7 can also be used, but the computational cost will be greater). Perform downsampling to obtain features ;feature Features output by the first 1×1 convolution Same scale; features with the same scale and It is sent to the fourth basic fusion module for fusion processing, and the fused features are output. .

[0177] The fusion features obtained after the above processing , and The output of the entire feature fusion network is fed into the backend detection head.

[0178] The core—basic fusion module—of the feature fusion network in this embodiment will be described in detail below.

[0179] For ease of explanation, the two features in the input of a single basic fusion module will be denoted as features. and . Reference Figure 3 A single basic fusion module inputs features and The feature fusion process is as follows:

[0180] Step 1) Align the channel dimensions;

[0181] Align features using independent 1×1 convolutions and The dimension is used to obtain the features. :

[0182]

[0183]

[0184] Among them, features and The sizes are consistent, and the alignment of the channel dimensions is achieved through 1×1 convolution.

[0185] Step 2) Channel splicing;

[0186] The features obtained after aligning the channel dimensions { Features are formed by splicing along the channel dimension .

[0187] Step 3) Feature fusion;

[0188] Feature fusion is achieved using 3×3 convolution to obtain fused features. .

[0189] Step 4) Channel compression;

[0190] Use 1×1 convolution pairs to fuse features The channel dimension is compressed to obtain a multi-scale fused feature map. And output it.

[0191] C. Detection head;

[0192] The detection head employs a commonly used classification-regression-confidence decoupling design, separating the target classification task and the coordinate regression task into two independent parallel branches to avoid optimization conflicts between classification and regression tasks in a single branch. The confidence branch predicts the probability of target presence, and the three work together to achieve accurate detection. The input to the detection head is a multi-scale fused feature map output by the feature fusion network, and the output is the target's location, pixel size, and category.

[0193] Step 4: Model training and testing;

[0194] Using the training and test sets obtained in step 2, the target detection model constructed in step 3 is trained and tested using existing conventional methods until a target detection model that meets the recognition requirements is obtained.

[0195] Step 5: Target detection;

[0196] After the aerial images are processed by the preprocessing method in step 2 to reduce their pixel count, they are input into the target detection model trained in step 4. The target detection model outputs the target detection results, including the location and size of the target and the category to which the target belongs.

[0197] Example 2:

[0198] Multi-scale feature fusion is of great significance in target detection, especially for small target detection tasks in aerial images where high localization accuracy is required. Since small targets often occupy only a few pixels in the feature map, single-scale feature representations are insufficient to provide sufficient discriminative information. Therefore, it is necessary to integrate multi-scale features from multiple multi-domain feature extraction modules at different levels to enhance the feature representation capability of the target.

[0199] For multi-domain feature extraction networks at the front end, deep features extracted by deep feature extraction modules contain rich semantic information but have low spatial resolution, while shallow features extracted by shallow feature extraction modules retain detailed spatial location information but have relatively scarce semantic information. Effectively fusing these complementary features is crucial for improving target detection performance. However, there are significant semantic gaps and spatial misalignments between features at different scales. This misalignment can have a more severe negative impact on the detection of small aerial targets, which requires high positioning accuracy.

[0200] To overcome the aforementioned negative impacts and improve the detection effect of small targets in aerial photography, this embodiment improves the target detection model constructed based on Embodiment 1: the feature fusion network is improved to an improved feature fusion network with feature alignment and fusion functions.

[0201] The improved feature fusion network first performs stepwise alignment and fusion processing on some / all features in the multi-scale features output by the multi-domain feature extraction network from deep to shallow layers, and then performs stepwise alignment and fusion processing on shallow to deep layers.

[0202] Specifically, such as Figure 4 As shown, the improved feature fusion network in this embodiment includes four feature alignment and fusion modules, two 1×1 convolutions, two upsampling convolutions, and two downsampling convolutions.

[0203] The process of gradually aligning and blending from deep to shallow layers is as follows:

[0204] First, the features output by the multi-domain feature extraction module 4 in the multi-domain feature extraction network. First, a 1×1 convolution is used to transform the channel dimensions to obtain the features. Then, use one upsampling convolution to apply the features. Upsampling is performed to obtain features ;feature Features output by multi-domain feature extraction module 3 Same scale; features with the same scale With features The features are fed into the first feature alignment and fusion module for alignment and fusion processing, and the fused features are output. .

[0205] Then, the fused features output by the first feature alignment and fusion module First, a 1×1 convolution is used to transform the channel dimensions to obtain the features. Then, use one upsampling convolution to apply the features. Upsampling is performed to obtain features ;feature Features output by multi-domain feature extraction module 2 Same scale; features with the same scale With features The features are fed into the second feature alignment and fusion module for alignment and fusion processing, and the fused features are output. .

[0206] The process of gradually aligning and blending from shallow to deep layers is as follows:

[0207] First, a 3×3 downsampling convolution with a stride of 2 is used to apply the fused features output by the second feature alignment and fusion module (in other embodiments, a larger convolution such as 5×5 or 7×7 can also be used, but the computational cost will be higher) to the fused features. Perform downsampling to obtain features ;feature Features from the output of the second 1×1 convolution Same scale; features with the same scale and The data is fed into the third feature alignment and fusion module for alignment and fusion processing, and the fused features are output. .

[0208] Then, a 3×3 downsampling convolution with a stride of 2 is used to fused the output of the third feature alignment and fusion module (in other embodiments, a larger convolution such as 5×5 or 7×7 can also be used, but the computational cost will be greater) to fused the features. Perform downsampling to obtain features ;feature Features output by the first 1×1 convolution Same scale; features with the same scale and The features are fed into the fourth feature alignment and fusion module for alignment and fusion processing, and the fused features are output. .

[0209] The fusion features obtained after the above processing , and The output of the entire improved feature fusion network is fed into the backend detection head.

[0210] The core of the improved feature fusion network in this embodiment—the feature alignment and fusion module—is described in detail below.

[0211] For ease of explanation, the two features in the input single feature alignment and fusion module will be denoted as features. and . Reference Figure 5 The single feature alignment and fusion module aligns the input features. and The process of feature alignment and fusion is as follows:

[0212] Step 1) Align the channel dimensions;

[0213] Align features using independent 1×1 convolutions and The dimension is used to obtain the features. :

[0214]

[0215]

[0216] Among them, features and The sizes are consistent, and the alignment of the channel dimensions is achieved through 1×1 convolution.

[0217] Step 2) Semantic and spatial alignment;

[0218] A progressive alignment strategy is adopted to eliminate semantic and spatial differences between multi-scale feature maps.

[0219] Step 2.1) Channel semantic alignment;

[0220] Multi-scale input features aligned with channel dimensions { Features are formed by splicing along the channel dimension .

[0221] To extract global context information for each channel, the features are... Simultaneously perform global max pooling and global average pooling operations, and then sum the features obtained from the global max pooling and global average pooling operations element by element to obtain the feature. .

[0222] Subsequently, the features were processed by 1×1 convolution. After channel compression, the result is Then, the features are processed by 1×1 convolution. Channel restoration was performed to obtain .

[0223] The above features To features The process realizes the features and Semantic interaction.

[0224] Then, the features Channel alignment weights are generated after processing by the sigmoid layer. .

[0225] Next, the features formed by splicing will be... Alignment weights with channels Element-wise multiplication is performed to achieve preliminary channel semantic alignment of the multi-scale input features, resulting in preliminarily aligned features. .

[0226] The calculation process is as follows:

[0227]

[0228]

[0229] in, For global max pooling, For global average pooling; For channel-compressed convolution and channel-restoring convolution A 1×1 convolutional block; It is a sigmoid layer whose output value is in the range [0,1] and is used as the weight for channel alignment.

[0230] Step 2.2) Perform fine-grained alignment in the spatial dimension;

[0231] Features after channel alignment Semantic associations across spatial locations are generated using depthwise separable convolutions (consisting of a large-size depthwise convolution (such as a 3×3 or 5×5 depthwise convolution) and a 1×1 convolution), resulting in fine-grained spatial alignment weights. .

[0232] Then, fine-grained spatial alignment weights are applied. Features after initial alignment Element-wise multiplication is performed to achieve fine-grained alignment, resulting in aligned features. .

[0233] The calculation process is as follows:

[0234]

[0235]

[0236] in, The depthwise convolution with a kernel size of 3×3 is designed to reduce computational resource consumption while enabling interaction across spatial locations.

[0237] Step 3) Feature fusion;

[0238] Features after feature alignment Uniformly divided along the channel dimension, resulting in Furthermore, a learnable channel selection parameter is introduced to balance features at different scales. The contributions between them are analyzed to achieve optimal feature integration and obtain the fused features. :

[0239]

[0240] in, Select parameters to initialize the learnable channels to 1 / 2. This indicates element-wise multiplication.

[0241] To ensure the training stability of the object detection model, this embodiment further refines the selection parameters for the learnable channels. Apply absolute value constraints:

[0242]

[0243] In summary, this embodiment effectively alleviates the semantic misalignment problem between multi-scale features by progressive channel semantic alignment, fine spatial alignment, and adaptive channel selection fusion, and avoids the dilution of small target features during the fusion process. This effect is particularly crucial in aerial photography scenarios.

[0244] Example 3:

[0245] To further improve the detection performance of aerial images, such as Figure 6 As shown, this embodiment is based on embodiment 2, but introduces a feature enhancement network between the improved feature fusion network and the decoding detection module, and the rest is the same as embodiment 2.

[0246] The feature enhancement network includes multiple parallel multi-branch feature enhancement modules to improve the discriminative ability of the fused multi-scale features. The number of multi-branch feature enhancement modules is consistent with the number of fused features output by the front-end feature fusion network. In this embodiment, there are three multi-branch feature enhancement modules.

[0247] The core of the feature enhancement network in this embodiment—the multi-branch feature enhancement module—will be described in detail below.

[0248] For ease of explanation, the features input into a single multi-branch feature enhancement module will be denoted as features. . Reference Figure 7 A single multi-branch parallel enhancement module for features The process of comprehensive enhancement is as follows:

[0249] Step 1) Features Divide the channel evenly into two parts: enhancement branch and identity branches The enhancement branch performs subsequent multi-branch parallel enhancement, while the identity branch retains some input channels, promoting feature reuse and feature fusion.

[0250] Step 2) For the augmentation branch, input features pass Perform feature activation to obtain Subsequently The process is performed through four parallel branches: a residual branch, a fine-grained enhancement branch, a local enhancement branch, and a global contextual information enhancement branch.

[0251] ;

[0252]

[0253] in, This represents a depthwise convolution with a kernel size of 1×1, used to capture fine-grained local detail features; This indicates a depthwise convolution with a kernel size of 3×3, used to extract multi-scale local features; For the global branch used for global feature enhancement, key global features are captured by modulation in the frequency domain; For residual connections, ensure the complete transmission of feature information.

[0254] Step 3) Branch the identity Enhanced output features with four parallel branches After concatenation, channel blending is performed using 1×1 convolution to obtain the enhanced multi-scale feature map. :

[0255]

[0256] It can provide a discriminative representation for subsequent detection heads.

[0257] It should be noted that in other embodiments, the multi-branch feature enhancement module can be modified and replaced in the following ways: for example, the local branches of the multi-branch feature enhancement module can use a larger-size depthwise convolution, such as a 5×5 depthwise convolution to extract multi-scale local features, such as... Figure 8 As shown; the global branch of the multi-branch feature enhancement module can be implemented using a Transformer layer, such as... Figure 9 As shown.

[0258] Example 4:

[0259] like Figure 10 As shown, the only difference between this embodiment and Embodiment 2 is that the multi-domain feature extraction network in the target detection model includes only 3 serial multi-domain feature extraction modules; and the fused features output by the two deeper multi-domain feature extraction modules in these 3 serial multi-domain feature extraction modules are all sent to the improved feature fusion network in the back end.

[0260] Example 5:

[0261] like Figure 11As shown, the difference between this embodiment and embodiment 2 is only that: of the four multi-domain feature extraction modules in the multi-domain feature extraction network of the target detection model, only the fused features output by multi-domain feature extraction module 2 and multi-domain feature extraction module 4 are sent to the improved feature fusion network in the back end; correspondingly, the structure of the improved feature fusion network is also different from that of embodiment 2 and has been adapted, which will be explained in detail below.

[0262] like Figure 11 As shown, the improved feature fusion network in this embodiment includes three feature alignment and fusion modules, two 1×1 convolutions, two upsampling convolutions, and two downsampling convolutions.

[0263] The process of gradually aligning and blending from deep to shallow layers is as follows:

[0264] The features output by the multi-domain feature extraction module 4 in the multi-domain feature extraction network First, the channel dimension is transformed and upsampled by one 1×1 convolution and one upsampling convolution, respectively, to obtain the features. Then, the features are processed sequentially using one 1×1 convolution and one upsampling convolution. After performing channel dimension transformation and upsampling, the features are obtained. ;feature Features output by multi-domain feature extraction module 2 Same scale; features with the same scale With features The features are fed into the first feature alignment and fusion module for alignment and fusion processing, and the fused features are output. .

[0265] The process of gradually aligning and blending from shallow to deep layers is as follows:

[0266] First, a 3×3 downsampling convolution with a stride of 2 is used to apply the fused features output by the first feature alignment and fusion module (in other embodiments, a larger convolution such as 5×5 or 7×7 can also be used, but the computational cost will be greater) to the fused features. Perform downsampling to obtain features ;feature Features with the same scale as the output of the second 1×1 convolution; features with the same scale The features output from the second 1×1 convolution are fed into the second feature alignment and fusion module for alignment and fusion processing, outputting fused features. .

[0267] Then, a 3×3 downsampling convolution with a stride of 2 is used to apply the fused features output by the second feature alignment and fusion module (in other embodiments, a larger convolution such as 5×5 or 7×7 can also be used, but the computational cost will be greater) to the fused features. Perform downsampling to obtain features ;feature The feature scale is the same as that of the output of the first 1×1 convolution; The features output from the first 1×1 convolution are fed into the third feature alignment and fusion module for alignment and fusion processing, outputting fused features. .

[0268] The fusion features obtained after the above processing , and The output of the entire improved feature fusion network is fed into the backend detection head.

[0269] Technical effectiveness verification:

[0270] Select and As an evaluation indicator for aerial target detection.

[0271] Average accuracy ( ) is the most core and widely recognized evaluation indicator in the field of target detection. The index comprehensively considers the classification accuracy and localization accuracy of the target detection model, and its calculation is based on the accuracy rate (…). ) and recall rate ( The curve formed by () Specifically, first, an intersection-union ratio (IU) is set. , The threshold is used to compare the predicted bounding box with the ground truth bounding box. Predictions exceeding this threshold are considered true cases. ), otherwise it is a false positive ( Subsequently, by calculating the corresponding precision and recall rates at different confidence thresholds, a graph can be plotted. curve. That is to say The area enclosed by the curve and the coordinate axes can be calculated using the following formula:

[0272] ;

[0273] in, For recall rate Accuracy of time To facilitate practical calculations, an approximate method of interpolating and summing the recall intervals is usually adopted.

[0274] That is A specific form of indicator that will The threshold is fixed at 0.5. This means that as long as the overlap area between the predicted bounding box and the ground truth bounding box exceeds 50%, the prediction is considered a correct detection. Because it has a high tolerance for localization errors in bounding boxes, it better reflects the model's ability to classify objects and is often used for preliminary and rapid comparison of model performance.

[0275] Figure 12 The experimental test results of the target detection method proposed in Embodiment 3 of the present invention are presented. Observation Figure 12 It can be seen that the present invention has achieved good detection results, and can achieve accurate positioning and detection even for dense small targets and occluded targets. Figure 13 This is a comparison of the experimental test results of Embodiment 1 of the present invention with those of existing target detection methods. Figure 14 The experimental test results of Embodiments 1-3 of the present invention are compared. The comparison of the experimental test results shows that Embodiment 1 of the present invention is superior to existing target detection algorithms, Embodiment 2 is superior to Embodiment 1, and Embodiment 3 is superior to Embodiment 2.

Claims

1. A target detection method, characterized in that, Including the following steps: Step 1: Obtain the datasets used to build the training and test sets; The dataset includes remote sensing / aerial images and their labels; the labels are the location, pixel size, and category of each target in the aerial / remote sensing images. Step 2: Image preprocessing and dataset partitioning; If the resolution of the remote sensing / aerial images in the dataset is greater than 3072... If the dataset contains 1728 pixels, it is then sliced, scaled proportionally, and padded with pixels in sequence before the dataset is divided into training and testing sets. Step 3: Construct the target detection model; The target detection model consists of a serially connected multi-domain feature extraction network, a feature fusion network, and a detection head; The multi-domain feature extraction network comprises a serially connected preliminary feature extraction module and n multi-domain feature extraction modules; n=3 or 4. The preliminary feature extraction module is used to downsample the image and extract preliminary features, which serve as the input to the top-level multi-domain feature extraction module. Each multi-domain feature extraction module includes a spatial-frequency selection module, a channel attention mechanism, and a convolution module. During spatial processing, the spatial-frequency selection module uses square convolution and horizontal and vertical strip convolution to extract dense local features, horizontal local features, and vertical local features from the received features. Then, it uses channel selection weights to adaptively converge the local features to obtain the extracted spatial features. During frequency processing, the spatial-frequency selection module adaptively filters and modulates the frequency feature components in the received features to capture the contextual information around different targets, thus obtaining the extracted frequency features. The channel attention mechanism is used to recalibrate the extracted spatial and frequency features. The convolution module is used to refine the recalibrated features and rearrange the channels to obtain a feature map that integrates local and global features. This feature map is the output of the multi-domain feature extraction module. The feature fusion network is used to fuse the shallow feature map output by at least one multi-domain feature extraction module located in the shallow layer and the deep feature map output by at least one multi-domain feature extraction module located in the deep layer of the multi-domain feature extraction network, and output a fused feature map. The detection head is used to extract the location, pixel size, and category of the target from the fused feature map; Step 4, Model Training and Testing; The target detection model is trained and tested using training and testing sets; Step 5, target detection; After the remote sensing / aerial images are processed by the preprocessing method in step 2 to reduce their pixel count, they are input into the trained target detection model. The target detection model outputs the target detection results, including the location and size of the target and the category to which the target belongs.

2. The target detection method according to claim 1, characterized in that: The method for a single multi-domain feature extraction module to extract features from the preliminary features is as follows: Step 3A.1, input the features of the multi-domain feature extraction module Downsampling is performed through downsampling convolution to obtain features. ; Step 3A.2: For features First, channel expansion is performed, then the data is partitioned along the channel dimension to obtain the original input features for subsequent spatial and frequency domain processing. ; Step 3A.3, spatial processing; By using stacked square convolutions and strip convolutions, features from the original input are... Extract dense multi-scale local features and then perform analysis on the original input features. Perform average pooling, channel expansion, dimension transformation, and... After calculation, channel selection weights are generated. These weights are then used to effectively select from the extracted dense multi-scale local features, resulting in local detail features. ; Step 3A.4, Frequency Domain Processing; First, the original input features Frequency domain features are obtained by mapping to the frequency domain using Fast Fourier Transform. ; Subsequently, frequency domain features Dividing the channel along the channel dimension into m parts, we obtain the frequency domain sub-features. ; Next, for each frequency domain sub-feature Independent modulation is performed using 1×1 convolution / filter; and each frequency domain sub-feature is modulated. Simultaneously with modulation (before / after), frequency domain features The number of channels is compressed to m, and then the frequency domain characteristics after channel compression are analyzed. Perform in the channel dimension The calculations yielded the sub-features in each frequency domain. The selection weight; Then, using each frequency domain sub-feature The selection weights for independently modulated frequency domain sub-features Frequency domain sub-bands at various spatial locations are filtered to obtain adaptive frequency domain features. ; Finally, frequency domain features The global semantic features are obtained by returning to the spatial domain through the inverse fast Fourier transform. ; Steps 3A.3 and 3A.4 above can be performed in a different order or in parallel; Step 3A.5, extract local detail features With global semantic features By splicing along the channel dimension, the features are obtained. ; Step 3A.6, feature Recalibration is performed using a channel attention mechanism to highlight information-rich channels, resulting in calibrated features. ; Step 3A.7, for features Channel compression is performed to obtain feature maps. ; Step 3A.8, feature With feature map Perform residual connections to obtain features And output it.

3. The target detection method according to claim 1 or 2, characterized in that: The feature fusion network in the target detection model constructed in step 3 is an improved feature fusion network. This improved fusion network has feature alignment and fusion functions. For the shallow feature map and deep feature map as input, the semantic preliminary alignment of the channel dimension and the fine alignment of the spatial position are performed first, and then adaptive channel selection fusion is performed to obtain the fused feature map.

4. The target detection method according to claim 3, characterized in that: The improved feature fusion network includes a 1×1 convolution for channel dimension transformation, an upsampling convolution and a downsampling convolution for scale transformation, and s feature alignment fusion modules. s is determined based on the number and level of the feature maps to be aligned and fused: if the feature maps input to the improved feature fusion network are from neighboring front-end features... A multi-domain feature extraction module, then If the feature maps input to the improved feature fusion network are from non-adjacent front-end features... A multi-domain feature extraction module, then ; The feature fusion network first performs stepwise alignment and fusion processing from deep to shallow layers, and then performs stepwise alignment and fusion processing from shallow to deep layers. Along the path from deep to shallow processing, 1×1 convolution and upsampling convolution are used to transform the two features in the feature alignment and fusion module into features with consistent channel dimensions and scale. Along the path from shallow to deep processing, downsampling convolution and 1×1 convolution are used to transform the two features in the feature alignment and fusion module into features with consistent channel dimensions and scale. The fused features output by the feature alignment and fusion module along the path from shallow to deep processing are the output of the entire improved feature fusion network; A single feature alignment and fusion module aligns the input features. and The process of feature alignment and fusion is as follows: Step 1) Align the channel dimensions; Align features using independent 1×1 convolutions and The dimension is used to obtain the features. ; Step 2) Semantic and spatial alignment; Step 2.1) Channel semantic alignment; Features Features are formed by splicing along the channel dimension ; Features Simultaneously perform global max pooling and global average pooling operations, and then sum the features obtained from the global max pooling and global average pooling operations element by element to obtain the feature. ; Features are paired using 1×1 convolution. After channel compression, the result is Then, the features are processed by 1×1 convolution. Channel restoration was performed to obtain ; Features Channel alignment weights are generated after processing by the sigmoid layer. ; Features formed by splicing Alignment weights with channels Element-wise multiplication is performed to achieve preliminary channel semantic alignment of the multi-scale input features, resulting in preliminarily aligned features. ; Step 2.2) Perform fine-grained alignment in the spatial dimension; feature Semantic associations across spatial locations are performed using depthwise separable convolutions, generating fine-grained spatial alignment weights. ; Fine-grained spatial alignment weights With features Perform element-wise positional multiplication to obtain the aligned features. ; Step 3) Feature fusion; Features after feature alignment Uniformly divided along the channel dimension, resulting in And introduce learnable channel selection parameters. Balancing features at different scales The contributions between them are analyzed to achieve optimal feature integration and obtain the fused features. .

5. The target detection method according to claim 4, characterized in that: The feature fusion network in the object detection model constructed in step 3 also includes a feature enhancement network; The feature enhancement network is set between the feature fusion network and the decoding and detection module. Before decoding and detection, it performs fine-grained enhancement, local enhancement, and global enhancement on the fusion feature map output by the fusion network, and then performs residual connection with the original fusion feature map to obtain the enhanced multi-scale fusion feature map and output it.

6. The target detection method according to claim 5, characterized in that: The feature enhancement network includes multiple parallel multi-branch feature enhancement modules; each multi-branch parallel enhancement module enhances the input features. The process of comprehensive enhancement is as follows: First, the features Divide into enhancement branches evenly along the channel dimension and identity branches The enhancement branch performs subsequent multi-branch parallel enhancement, while the identity branch retains some input channels to promote feature reuse and feature fusion. Secondly, for the augmentation branch, the input features pass Perform feature activation to obtain Subsequently The process is performed through four parallel branches: residual branch, fine-grained enhancement branch, local enhancement branch, and global context information enhancement branch. Finally, the identity branch Enhanced output features with four parallel branches After concatenation, channel blending is performed using 1×1 convolution to obtain the enhanced multi-scale feature map. .

7. The target detection method according to claim 6, characterized in that: In step 3, the detection head adopts a classification-regression-confidence decoupled architecture design.

8. The target detection method according to claim 7, characterized in that: The dataset used in step 1 is a public dataset such as VisDrone, UAVDT, CODrone, or UAVVaste.

9. A target detection model, characterized in that, The target detection method is obtained by using steps 1-4 of any one of claims 1-8.

Images