Enhanced object detection method, device, and medium based on feature fusion

Abstract

This invention relates to computer vision and target detection technology, and particularly to an enhanced target detection method, device, and medium based on feature fusion. The method includes: acquiring flight parameter data, environmental scene data, and aerial image data from an aerial photography device; extracting multi-level initial feature maps from the aerial image data using an initial feature extraction network; for each level of the initial feature map, extracting local and global features respectively using a parallel-deployed local feature extraction network and a global feature extraction network; inputting the local and global features corresponding to each initial feature map, the flight parameter data, and the environmental scene data into a gating fusion module to generate a corresponding fused feature map; fusing the fused feature maps from multiple levels at multiple scales to obtain multi-scale fused features; and outputting the target detection result. This invention effectively improves the detection accuracy and robustness of multi-scale targets in aerial images.

Description

[0001] This invention is a divisional application of Chinese Patent Application No. CN2025117367976, filed with the Chinese Patent Office on November 25, 2025, entitled "Enhanced Target Detection Method, Device and Medium Based on Feature Fusion". The entire contents of this application are included in the parent application. Technical Field

[0002] This invention relates to computer vision and target detection technology, and in particular to an enhanced target detection method, device and medium based on feature fusion. Background Technology

[0003] With the popularization of drone technology, using drones for low-altitude road monitoring, traffic management, and emergency response has become an important means. However, target detection in low-altitude aerial images faces many unique challenges: First, the shooting perspective (overhead or oblique view) leads to varied target shapes and huge differences in scale, and often there is severe occlusion; second, changes in flight altitude and angle, as well as complex ground environments (such as different lighting, shadows, and road types) can significantly affect the appearance quality of the image, posing a severe test to the robustness of the model.

[0004] Existing mainstream object detection algorithms, such as YOLO and Faster R-CNN, while performing excellently on general datasets, often experience performance degradation when directly applied to low-altitude aerial photography scenarios. This is because these algorithms typically rely on a single type of feature extractor (such as CNN), making them insensitive to small target features or edge features in aerial data, easily leading to missed or false detections. Secondly, in multi-scale feature fusion, they usually employ simple upsampling and concatenation operations. Simple concatenation is insufficient to effectively fuse global and local features. Gating mechanisms constrain local and global features, significantly mitigating numerical differences and avoiding the problem of linear fusion failing due to significant differences between them. This allows for the dynamic and adaptive fusion of the two types of features.

[0005] Some improvements have emerged in the industry. For example, patent document (CN120580610A) discloses a method and apparatus for UAV image target detection based on multi-scale feature fusion. This method simply fuses (e.g., stitches or adds) features from different levels within the network to improve multi-scale detection capabilities. However, this method mainly focuses on the scale differences of feature maps, ignoring the semantic gaps and spatial geometric mismatches between features at different levels, making it difficult to effectively establish accurate spatial correspondences between cross-scale features. Therefore, existing technologies still have significant shortcomings: either they fail to fully utilize the multi-source prior information available in aerial photography missions, or their underlying mechanisms for feature extraction and fusion are not deep or comprehensive enough. There is an urgent need in this field for an enhanced target detection method that can deeply integrate multi-source information, collaboratively optimize local and global feature extraction, and accurately embed spatial and scale perception capabilities in multi-scale fusion, so as to fundamentally improve detection accuracy and robustness in complex low-altitude scenarios. Summary of the Invention

[0006] To address the aforementioned technical problems, the technical solution adopted by this invention is as follows: According to a first aspect of the present invention, an enhanced target detection method based on feature fusion is provided, the method comprising: S100 acquires flight parameter data, environmental scene data, and aerial image data from aerial photography equipment.

[0007] S200, extract multi-level initial feature maps of the aerial image data through an initial feature extraction network.

[0008] S300 extracts local and global features respectively for each level's initial feature map through parallel deployment of local and global feature extraction networks.

[0009] S400, the local and global features corresponding to each initial feature map, the flight parameter data and the environmental scene data are input together into a gating fusion module to generate the corresponding fused feature map.

[0010] S500 performs multi-scale fusion of feature maps from multiple levels to obtain multi-scale fused features.

[0011] S600 outputs target detection results based on multi-scale fusion features.

[0012] According to a second aspect of the present invention, an electronic device is provided, including a processor and a memory; the processor executes the steps of the method described in the first aspect of the present invention by invoking a program or instructions stored in the memory.

[0013] According to a third aspect of the present invention, a computer-readable storage medium is provided that stores a program or instructions that cause a computer to perform the steps of the method described in the first aspect of the present invention.

[0014] The present invention has at least the following beneficial effects: 1. Upgrade to multimodal perception technology: Based on pure visual analysis, flight parameter data and environmental scene data are innovatively introduced to complete the technological upgrade from single-modal to multimodal perception and expand the dimensions of data perception.

[0015] 2. Imparting physical perception capability to feature fusion: Through an enhanced gating fusion mechanism, while processing local and global visual features, flight status and environmental context information are incorporated, enabling the feature fusion process to have physical perception capability. The fusion strategy can be adaptively adjusted for different flight conditions (such as altitude and attitude) and environmental conditions (such as lighting and road structure).

[0016] 3. Enhance the ability to handle complex scenarios: By leveraging the synergistic effect of multi-source data, it can better address the complex challenges of aerial photography scenarios. For example, it can adaptively adjust the feature extraction strategy when the flight altitude changes, and optimize the feature representation under different lighting conditions.

[0017] 4. Enhance model performance stability: While maintaining detection accuracy, significantly improve the model's robustness and generalization ability, making it more adaptable to diverse environmental disturbances and flight state changes in aerial photography scenarios.

[0018] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart of an enhanced target detection method based on feature fusion provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0023] It should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. A process can be terminated when its operation is complete, but it may also have additional steps not included in the figures. A process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0024] (Example 1) This invention provides an enhanced target detection method based on feature fusion for low-altitude aerial photography scenarios, such as... Figure 1 As shown, the method includes the following steps: S10, acquire aerial image data.

[0025] In this embodiment of the invention, aerial image data is collected by a visible light camera or optoelectronic pod mounted on the aerial photography equipment and input in the form of an aerial video stream. The aerial photography equipment can be a drone.

[0026] S20, the aerial image data is extracted by the initial feature extraction network and input into the initial feature extraction network to obtain multiple multi-level initial feature maps of different levels.

[0027] In this embodiment of the invention, the initial feature extraction network (also known as the backbone network) is responsible for extracting rich and representative multi-level features from the original aerial images. This network is a deep convolutional neural network, and its structural design follows the principle of gradually decreasing spatial resolution and gradually increasing semantic information from shallow to deep layers.

[0028] Specifically, the aerial image data (usually preprocessed, such as size normalization) is input into the initial feature extraction network. During the forward propagation process, the network outputs feature maps with different spatial sizes and receptive fields at different depth stages, constituting the "multiple initial feature maps at different levels".

[0029] As a preferred embodiment, the initial feature extraction network adopts an architecture based on the CSPNet (Cross-Stage Local Network) concept, such as CSPDarknet. This network contains multiple stages, each consisting of a set of convolutional layers, normalization layers (such as BN layers), and activation functions (such as SiLU). It gradually reduces the spatial size of the feature map through downsampling operations (such as convolutions with a stride of 2) while increasing the number of channels.

[0030] The specific definitions of feature maps at each level are as follows: Shallow feature maps: These originate from earlier stages of the network. These feature maps have high spatial resolution (e.g., downsampled by 4 or 8 times relative to the input image) and retain rich details such as edges, corners, and textures, which are crucial for locating small targets. However, they have weaker semantic information and more noise.

[0031] Mid-level feature maps: These originate from the intermediate stages of the network. These feature maps have moderate spatial resolution (e.g., a downsampling factor of 16x), balancing detailed and semantic information, and typically contain part and contour information of the target.

[0032] Deep feature maps: These originate from the final stage of the network. These feature maps have the lowest spatial resolution (e.g., a downsampling factor of 32x), the largest receptive field, and contain highly abstracted, strong semantic information. They are very effective for recognizing large targets and understanding scene categories, but a large amount of spatial detail information is lost.

[0033] Finally, the initial feature extraction network outputs a set (e.g., three or four) of feature maps from different layers, denoted as {P1, P2, ..., Pn}, where P1 represents the shallowest / highest resolution feature map and Pn represents the deepest / lowest resolution feature map. This set of multi-scale initial feature maps together forms the basis for subsequent feature enhancement and fusion, ensuring that the model can capture the complete information spectrum from pixel-level details to global semantics, laying a solid foundation for addressing the core challenge of varying target scales in low-altitude scenes.

[0034] S30: For the initial feature map of each level, local features and global features are extracted respectively through parallel deployment of local feature extraction network and global feature extraction network.

[0035] In this embodiment of the invention, this step aims to extract complementary feature representations from the same initial feature map through two heterogeneous parallel processing paths, thereby overcoming the limitations of a single type of network structure.

[0036] Specifically, for any initial feature map X∈R from step S20 C×H×W (C is the number of channels in the initial feature map, H is the height of the initial feature map, and W is the width of the initial feature map), perform the following parallel processing: Local feature extraction network (CNN branch): Design objective: Inheriting the inductive bias of convolutional neural networks, this branch efficiently extracts local detail features from images, such as edges, corners, textures, and basic shapes. It possesses strong capabilities in capturing local patterns and boasts high computational efficiency.

[0037] Core Structure: Depthwise separable convolutions are preferred for construction to significantly reduce computational complexity while maintaining performance. A basic building block consists of the following: A depthwise convolution is used to perform spatial filtering on each input channel individually.

[0038] A pointwise convolution is used to combine the output channels of a depthwise convolution.

[0039] Each convolution is optionally followed by a batch normalization layer and an activation function.

[0040] Alternative solution: Standard convolutional layers can also be used in series, but the computational load is relatively high.

[0041] Processing flow: After the initial feature map X is input into this branch, it undergoes a series of convolution operations as described above to perform nonlinear transformations and feature transformations, finally outputting local features F. cnn ∈R C×H×W This local feature preserves the original spatial resolution and enhances the representation of the local region.

[0042] Here, the number of channels represents the depth of the feature map. Each channel can be viewed as a response map of a "feature detector" or "filter" that focuses on extracting a specific feature (such as edge, color, texture, semantic concept). The height represents the number of pixels (or feature points) in the vertical direction of the feature map, and the width represents the number of pixels (or feature points) in the horizontal direction of the feature map.

[0043] 2. Global Feature Extraction Network (Sliding Window Transformer Branch): Design objective: To leverage the self-attention mechanism of Transformer to establish long-range dependencies of features and global contextual information, thereby understanding the relationships between different parts of an image. This is crucial for handling occlusion, complex backgrounds, and large targets with complex structures.

[0044] Core Structure: A simplified version of the sliding window Transformer Block is used. This block is the basic unit for building modern visual Transformers, and its core operations include: Window-based multi-head self-attention: The feature map is divided into non-overlapping windows, and self-attention is computed only within each window, which greatly reduces computational complexity.

[0045] Shifted window multi-head self-attention: At a deeper level, window configurations are shifted, enabling information exchange between different windows, thereby achieving global modeling across windows.

[0046] Multilayer perceptron: performs a nonlinear transformation on the output of the self-attention layer.

[0047] Each core operation is preceded by layer normalization and includes residual connections to ensure training stability.

[0048] Processing flow: The initial feature map X may first need to have its dimensions adjusted through a linear embedding layer to fit the Transformer structure. Subsequently, it is fed into one or more simplified SwimTransformer modules. Through a self-attention mechanism, the model computes the globally contextualized feature representation, ultimately outputting the global feature F. swin ∈R C×H×W .

[0049] The two branches described above process the same initial feature map in parallel and independently. They are designed to have similar computational loads and output dimensions to ensure the effectiveness of subsequent fusion. Finally, for each input feature map, this step outputs a pair of complementary feature maps in the feature space: detailed local features and context-rich global features. This pair of features provides an ideal foundation for adaptive fusion in the subsequent S40 step.

[0050] Through this parallel heterogeneous architecture design, the model can simultaneously leverage the efficiency of CNN in local feature extraction and the powerful ability of Transformer in global relationship modeling, laying a solid technical foundation for improving detection accuracy and robustness in complex low-altitude scenarios.

[0051] S40, through the gating fusion module, the local features and global features are adaptively fused to generate a fused feature map.

[0052] In this embodiment of the invention, the gated fusion module aims to implement an intelligent, data-driven feature fusion strategy. Its core idea is to dynamically and adaptively evaluate and weight the relative importance of local detail features and global context features at each spatial location and channel, thereby achieving the optimal feature combination.

[0053] Furthermore, the gating fusion module is configured to execute a local-global feature competition mechanism for aerial photography scenarios, the mechanism including: S41, Calculate the feature competition graph of local features and global features in spatial location, wherein the feature competition graph represents the degree of dependence of each spatial location on local features and global features.

[0054] The calculation of the feature competition map between local and global features in spatial location includes: The similarity or distance metric between local and global features is calculated positionally, and the results are mapped to a feature competition graph representing the degree of dependency. The similarity metric is preferably cosine similarity. Cosine similarity focuses on measuring the directional consistency between local and global features. Lower similarity indicates a greater difference between the feature's direction and the global context, and a greater reliance on unique local details. The distance metric is preferably L1 distance or Euclidean distance. L1 distance or Euclidean distance focuses on measuring the absolute numerical difference between local and global features. A larger distance indicates a more significant difference between the feature value at that location and the global background, and a greater reliance on local information.

[0055] In this embodiment of the invention, the feature competition map is generated in the following manner: Calculate the cosine similarity, L1 distance, or Euclidean distance between local and global features in spatial location to generate an initial response map; The initial response map is normalized to map its numerical range to the interval [0, 1], thus obtaining the feature competition map.

[0056] When using cosine similarity, its range is [-1, 1], and it can be linearly normalized using (similarity + 1) / 2. When using L1 distance or Euclidean distance, it can be non-linearly normalized using the Sigmoid function.

[0057] The feature competition graph generated in the above manner can transform the abstract competition relationship between local and global features into a quantifiable spatial graph with clear physical meaning.

[0058] S42, Based on the feature competition graph, generate a spatial dimension gating weight graph.

[0059] The generation of the spatial dimension gated weight graph includes: S4201, The feature competition map is concatenated with local features and global features along the channel dimension to obtain a comprehensive feature map; S4202, Input the comprehensive feature map into the weight generation network; S4203, The weight generation network generates a gated weight graph for the spatial dimension.

[0060] In a preferred embodiment of the present invention, the weight generation network consists of 1×1 convolutional layers and a sigmoid activation function, wherein a spatially dimensional gated weight map is generated through the convolutional layers and the sigmoid activation function in the weight generation network.

[0061] S43, using the gated weight map, perform spatially weighted fusion of local and global features to generate a fused feature map.

[0062] In this embodiment of the invention, the space-wise weighted fusion is achieved through the following formula: The fused feature map = G⊙local features + (1-G)⊙global features, where G represents the gated weight map of the spatial dimension and ⊙ represents the position-by-position multiplication operation.

[0063] Accordingly, in edge and textured areas, the G value approaches 1, and the fusion result is biased towards local features; in flat and background areas, the G value approaches 0, and the fusion result is biased towards global features.

[0064] S50 performs multi-scale fusion of feature maps from multiple levels to obtain multi-scale fused features.

[0065] In this embodiment of the invention, this step aims to intelligently fuse feature maps from different levels with different spatial resolutions and semantic information through a multi-scale coordinate attention mechanism. The core innovation of this mechanism lies in co-encoding "scale" as a key dimension that can be parallel to "spatial coordinates" and "channels," thereby generating attention weights that can perceive the global multi-scale context and achieve accurate alignment and adaptive enhancement of cross-scale features.

[0066] S50 specifically includes the following sub-steps: S51 aligns the resolution of the fused feature maps from multiple levels.

[0067] The multiple fused feature maps from step S40 are spatially aligned. Specifically, for feature maps with smaller spatial dimensions (usually from deeper network layers), they are enlarged by upsampling operations (such as bilinear interpolation or transposed convolution); for feature maps with larger spatial dimensions (usually from shallower network layers), they are reduced by downsampling operations (such as strided convolution or max pooling), ultimately unifying all feature maps to the same intermediate spatial resolution (Hmid×Wmid).

[0068] S52 stitches the aligned feature maps together along the channel dimension to form multi-scale intermediate features.

[0069] All the aligned feature maps in S51 will be stitched together along the channel dimension to form a comprehensive multi-scale intermediate feature tensor F. mid ∈R Hmid×Wmid×Ctotal Here, Ctotal is the sum of the number of channels in all feature maps. This tensor simultaneously preserves spatial details and semantic information from different scales.

[0070] S53, perform global pooling on the multi-scale intermediate features along the three dimensions of height, width and channel to obtain multi-dimensional pooling results.

[0071] The multi-scale intermediate feature tensor is subjected to three-way cooperative coordinate encoding to capture long-range spatial dependencies across scales. Specifically, this is implemented as follows: First, perform global average pooling along the height direction to generate a feature vector Vh∈R with a width of Wmid. 1 ×Wmid×Ctotal This vector encodes global contextual information for each vertical stripe across all scales.

[0072] Secondly, global average pooling is performed along the width direction to generate a feature vector Vw∈R with height Hmid. Hid ×1×Ctotal This vector encodes the global contextual information for each horizontal stripe across all scales.

[0073] Next, global average pooling is performed along the channel direction to generate a feature map Vc∈R with spatial dimensions of Hmid×Wmid. (Hmid×Wmid×1) This feature map encodes global statistics for each spatial location across all channels as a result of multi-dimensional pooling.

[0074] S54 inputs the multi-dimensional pooling results into the shared transformation network to predict unified multi-scale perceptual attention weights.

[0075] Furthermore, S54 may specifically include: S5401 performs dimensional transformation and alignment on the three-dimensional pooling results obtained in S53 to form a unified comprehensive perceptual feature: The feature vectors obtained by high-dimensional pooling are copied and expanded along the high-dimensional dimension to generate a high-perception feature map. The feature vectors obtained by pooling along the width dimension are copied and expanded along the width dimension to generate a width-aware feature map. The feature map obtained by pooling along the channel dimension is copied and expanded along the channel dimension to generate a channel-aware feature map. The three perceptual feature maps are added element by element to form a comprehensive perceptual feature, which integrates global contextual information from the three dimensions of height, width, and channel.

[0076] S5402, the integrated perceptual features are input into the shared transform network to predict unified multi-scale perceptual attention weights. The shared transform network is a lightweight convolutional module, whose structure includes: The first convolutional layer uses a 1×1 convolutional kernel for feature compression and dimensionality unification, reducing the number of input channels Ctotal to Ctotal / r. Here, r is a compression ratio greater than 1 (e.g., r=16).

[0077] Nonlinear activation layer: The ReLU activation function is used to introduce nonlinear transformation capability.

[0078] The second convolutional layer uses a 1×1 convolutional kernel to map features into a single-channel output.

[0079] Output activation layer: The Sigmoid activation function is used to normalize the output values ​​to the [0, 1] interval, ultimately generating a multi-scale perceptual attention weight map M. attention ∈R Hmid×Wmid×1 Through the aforementioned multi-scale perceptual attention mechanism, this invention achieves the following beneficial effects: Collaborative global perception: By fusing global information from three orthogonal dimensions—height, width, and channel—the generated spatial attention weight map has a more comprehensive view and can more accurately identify important regions.

[0080] Scale adaptation: This mechanism can sense and weigh feature responses from different levels, thus effectively addressing the problem of target scale diversity caused by changes in flight altitude in aerial images.

[0081] Lightweight and efficient structure: The shared transform network uses parameter sharing and small convolutional kernels, resulting in low computational overhead, making it ideal for deployment on embedded UAV platforms with limited computing resources.

[0082] Enhanced feature discrimination: Through dynamic feature enhancement, the distinguishability between foreground targets and complex backgrounds is effectively improved, thereby improving the overall accuracy and robustness of the target detection model.

[0083] S55, the multi-scale perceptual attention weights are applied to the multi-scale intermediate features for enhancement, resulting in multi-scale fused features.

[0084] The multi-scale perceptual attention weight map M generated by S54 attention The multi-scale intermediate feature Fmid obtained in S52 is applied to the feature enhancement through element-wise multiplication, thus obtaining the final multi-scale fused feature F. enhanced The calculation process is as follows: F enhanced =M attention ⊙F mid .

[0085] This step enables adaptive feature filtering based on global contextual information. Each value in the multi-scale perceptual attention weight map integrates the responses of features at all scales in the same spatial location. This is achieved through the multiplication operation described above: Positions with high weight values ​​indicate that the spatial point is identified as important at all scales, and its features are enhanced; Locations with low weight values ​​indicate that the spatial point may be background or interfering information, and its features are suppressed.

[0086] The final output is a multi-scale fusion feature enhanced by multi-scale contextual information. It retains the discriminative information related to the target to the greatest extent and is sent to the subsequent detection head to complete the final target classification and localization.

[0087] S60 outputs target detection results based on multi-scale fusion features.

[0088] In this embodiment of the invention, this step is responsible for converting the feature map, which has been optimized by all the aforementioned steps and is rich in multi-scale context and adaptive enhancement information, into a final detection result with clear physical meaning.

[0089] Specifically, the multi-scale fused features are input into the detection head, and the target detection result is output. The detection head receives the multi-scale fused features from step S50. This detection head typically employs an anchor-based or anchor-free prediction mechanism, and its core task is to perform the following two sub-tasks in parallel at each spatial location: Target classification: Predict the probability that a target exists at this location and determine its category (e.g., vehicle, pedestrian, non-motorized vehicle, etc.).

[0090] Bounding box regression: Predicts the precise geometric coordinates of the target bounding box at this location, typically the center point coordinates, width, and height, denoted as x, y, w, and h.

[0091] In a preferred embodiment, the specific structure and operation of the detection head are as follows: Feature input: The multi-scale fused features (usually a set of feature maps from different levels that have been intelligently fused) are input into the detection head.

[0092] Convolutional processing: The detector head consists of several consecutive convolutional layers, which are used to further refine the features and transform them to the final prediction space. Each convolutional layer is usually followed by a batch normalization layer and an activation function (such as SiLU or Leaky ReLU) to maintain the stability of training and introduce non-linearity.

[0093] Parallel prediction: After the final convolutional processing, the network performs parallel prediction through two independent convolutional layers (or one output layer with two branches): Classification branch: Use a 1×1 convolution with K output channels (where K is the total number of target categories to be detected) and output K scores for each spatial location. Usually, it is then processed by the Sigmoid function to represent the probability of existence of each category.

[0094] Regression branch: Using another 1×1 convolution with 4 output channels (corresponding to the four parameters x, y, w, and h of the bounding box), it outputs a raw prediction of the bounding box for each spatial location.

[0095] Output Decoding and Post-processing: The raw prediction tensor output by the detection head needs to be decoded and post-processed to form the final detection result. This typically includes: Decoding: For anchor-box-based methods, the offset of the regression branch output is applied to the preset anchor box to calculate the absolute coordinates in the image space; for anchor-box-less methods, the output may be directly scaled.

[0096] Non-maximum suppression: This is a crucial post-processing step. Since the detection head generates a large number of overlapping bounding boxes in space, the NMS algorithm filters them based on the classification score and the overlap between the bounding boxes (such as the intersection-union ratio, IoU), removing redundant bounding boxes that point to the same target, and finally retaining a concise set of high-quality detection results.

[0097] Finally, the method outputs a list of target detection results, where each result includes at least: a target category label, a confidence score for that category, and a bounding box locating the target in the original aerial image coordinate system. These results can be directly used in high-level applications such as traffic flow statistics, event detection, and target tracking in low-altitude monitoring systems.

[0098] By inputting deeply optimized multi-scale fusion features into the detection head, this invention ensures that the model can make final decisions based on the richest and most robust feature representations, thereby achieving high-precision and high-recall real-time target detection in complex low-altitude aerial photography scenarios.

[0099] In practical applications, the enhanced target detection methods S10 to S60 are implemented and carried out by a pre-trained enhanced target detection model. This trained model is the parameterized and fixed form of the method flow, which can be directly deployed on a computing device to perform inference tasks.

[0100] 1. Model training phase Before being put into practical use, the enhanced object detection model requires an offline model training phase to learn and optimize all its parameters. This training phase includes: Training data preparation: Collect and construct a large-scale low-altitude aerial photography dataset. This dataset contains aerial images, corresponding flight parameter data, and precise target bounding boxes and category labels.

[0101] Training objective: Define a multi-task loss function that includes classification loss, bounding box regression loss, etc., and use the backpropagation algorithm and gradient descent optimizer (such as AdamW) to iteratively update the model parameters, including the initial feature extraction network, parallel local feature extraction networks and global feature extraction networks, gated fusion modules, multi-scale fusion networks, and all trainable weights in the detection head.

[0102] The training process is unique in that it forces the model to learn how to effectively utilize flight parameters and environmental scene data. For example, the fully connected layers in the gated fusion module need to learn to generate meaningful fusion weights α and β based on the input conditional vector; the shared transformation network in the multi-scale fusion network needs to learn to associate information such as flight altitude with multi-scale features to generate more accurate attention maps.

[0103] Training Results: After training, a model weight file with the optimal parameter set is obtained. This file encapsulates all the knowledge the model has learned from the data, including but not limited to: how to extract local and global features, how to dynamically fuse multi-source information, how to generate multi-scale perceptual attention, and how to ultimately identify and locate the target.

[0104] 2. Model Deployment and Inference Phase Once training is complete, the model can be deployed in a real computing environment to execute the aforementioned S100 to S600 processes: Deployment methods: The trained model can be packaged into a software library, embedded program or cloud API service and deployed on the UAV's onboard computing unit, ground station or edge server or cloud platform.

[0105] Workflow: In actual operation, the system: Initialization: Load the trained model weights into the memory of the computing device and initialize the model's computation graph.

[0106] Real-time inference: Receive real-time aerial video stream (S10).

[0107] The data is input into the loaded model. The model then executes a fixed process from S20 to S60 in sequence: feature extraction, parallel processing, gated fusion, and multi-scale fusion, finally outputting the target detection result through the detection head.

[0108] Output and Application: The structured detection results (target category, confidence level, location box) output by the model can be displayed on the monitoring screen in real time, or directly provided to subsequent advanced application modules such as traffic flow analysis, violation determination, and route planning.

[0109] 3. Advantages Through this "offline training, online inference" model, the core advantages of the method described in this invention are fully demonstrated in practical applications: High efficiency: When performing inference, the trained model only needs forward propagation calculation and does not need to be trained again, which meets the stringent real-time requirements of low-altitude monitoring.

[0110] Consistency: Ensures that models deployed at different times and locations maintain consistent detection performance and behavior as long as they are based on the same training results.

[0111] Automation and intelligence: The entire complex process from S10 to S60 is completed automatically within the model. Users or upper-level systems do not need to understand the internal mechanism. They only need to input data to obtain accurate detection results, realizing end-to-end intelligent processing from raw data to semantic understanding.

[0112] In summary, the methods described in S10 to S60 can be applied efficiently and reliably in reality through a well-trained enhanced target detection model, providing a powerful technical core for various visual perception tasks in the low-altitude economy.

[0113] Furthermore, the method also includes: Receive monitoring task instructions.

[0114] Monitoring task instructions originate from ground control stations or cloud platforms and can be received via a human-machine interface. Each instruction contains at least one identifier indicating the current task mode or performance requirements. For example: Fast patrol mode: requires extremely high real-time performance (high frame rate) and low power consumption, with relatively relaxed requirements for detection accuracy.

[0115] Precision inspection mode: requires the highest detection accuracy, but has relatively relaxed requirements for computing resources and real-time performance.

[0116] Conventional monitoring mode: achieving a balance between accuracy and efficiency.

[0117] According to the monitoring task instructions, the gating fusion module can be selectively enabled or bypassed.

[0118] The received monitoring task instruction is parsed, and a control signal for controlling the gating fusion module is generated according to a preset strategy. Based on the control signal, the forward inference path is dynamically reconstructed. When the instruction requires high precision (such as "precise inspection mode"), the gated fusion module is activated. At this time, the model will fully execute steps S30 and S40, that is, extract local and global features through parallel branches and use the gated fusion module for refined adaptive fusion to achieve the best performance of the model.

[0119] When instructions demand high efficiency and low power consumption (such as "fast patrol mode"), the gated fusion module is bypassed. In this case, the system skips the complex calculations in S30 and S40. Optionally, the initial feature map output from step S20 is directly passed to step S50 for multi-scale fusion, or a simpler default fusion strategy is adopted (e.g., directly using the output of the local feature branch as the input to S50). By bypassing the computationally intensive sliding window Transformer branch and gated weight calculations, the system can significantly improve inference speed and reduce power consumption.

[0120] Furthermore, in practical deployment, the target detection method provided in this embodiment of the invention can work in conjunction with a task adaptive control system. This system receives monitoring task instructions (such as "event detection" or "traffic flow statistics") and generates corresponding acquisition and transmission strategies based on these instructions.

[0121] The acquisition strategy controls one or more of the following: flight altitude of the aircraft, gimbal pitch angle, and acquisition frequency. Flight altitude controls the relative distance between the aircraft and the ground, directly affecting the ground resolution of the image and the field of view per frame. Gimbal pitch angle determines the shooting angle (e.g., orthographic or tilted). Acquisition frequency determines the temporal resolution of the video data.

[0122] The transmission strategy controls one or more of the following: the transmission interval of image data, the compression algorithm and parameters, and the transmission resolution. The transmission interval determines the period at which data is transmitted to the ground station or cloud (e.g., real-time streaming, one frame every 2 seconds). The compression algorithm and parameters balance data volume and information integrity by selecting lossy or lossless encoding methods and their strengths (e.g., bitrate, quantization parameters). The transmission resolution determines whether the original image is downsampled before transmission.

[0123] The data collection strategy and the data transmission strategy are configured collaboratively according to the task type, as shown in the following specific embodiment: When the monitoring task is "Event Detection": Acquisition Strategy: A "low-altitude hovering acquisition strategy" is adopted. This involves controlling the aircraft to descend to a relatively low altitude and hovering, thereby obtaining higher ground resolution images to facilitate the identification of accident details, violations, etc.; at the same time, high frame rate shooting is used to capture fast-moving processes.

[0124] Transmission Strategy: A "high-frequency image transmission strategy" is adopted. High-definition images or video streams captured are transmitted to the command center in real time or near real time with the highest priority and minimal delay to ensure that information on emergencies can be perceived and responded to in a timely manner.

[0125] When the monitoring task is "traffic flow statistics": Data collection strategy: A "fixed-altitude cruise data collection strategy" is adopted. The aircraft is controlled to cruise at a fixed, high altitude to obtain a wide and stable bird's-eye view, covering a longer road segment, thereby ensuring the spatial continuity and accuracy of traffic statistics.

[0126] Sending strategy: A "timed batch sending strategy" is adopted. Instead of transmitting the entire video stream, structured statistical data (such as traffic flow and average speed) can be packaged and sent periodically (e.g., every minute) after traffic statistics are completed on the device; alternatively, only low-resolution images can be extracted and transmitted periodically for verification. This significantly saves communication bandwidth and storage costs.

[0127] (Example 2) This embodiment provides an enhanced target detection method based on feature fusion, the method comprising: S100 acquires flight parameter data, environmental scene data, and aerial image data from aerial photography equipment.

[0128] In this embodiment of the invention, the aerial photography equipment can be a drone. Environmental scene data is acquired through onboard environmental perception sensors or an external traffic information service platform, and may include, for example, light intensity, shadow distribution, road structure, and dynamic traffic participant density. Flight parameter data originates from the flight control system, navigation, and attitude sensors of the aerial photography equipment (such as a drone), and specifically includes flight altitude, speed, real-time latitude and longitude, pitch angle, roll angle, and yaw angle. Aerial image data is collected by a visible light camera or optoelectronic pod mounted on the aerial photography equipment and input in the form of an aerial video stream.

[0129] S200, extract multi-level initial feature maps of the aerial image data through an initial feature extraction network.

[0130] S300 extracts local and global features respectively for each level's initial feature map through parallel deployment of local and global feature extraction networks.

[0131] S400, the local and global features corresponding to each initial feature map, the flight parameter data and the environmental scene data are input together into a gating fusion module to generate the corresponding fused feature map.

[0132] The gating fusion module executes the following processing flow: S410, Multi-Source Data Encoding and Alignment: S4101 encodes flight parameter data into flight feature vectors through a fully connected network.

[0133] Flight parameter data (including altitude, speed, pitch angle, roll angle, and yaw angle) are encoded into fixed-dimensional flight feature vectors using a specially designed fully connected network. Specific implementation includes: Step 1: Standardize the acquired raw flight parameter data (including flight altitude, speed, pitch angle, roll angle, and yaw angle) to make its mean 0 and variance 1, thus forming a stable input vector V. fligh t∈R 6 ; Step 2: Convert the preprocessed input vector V fligh The t-feed-in encoding network undergoes a nonlinear transformation in a multilayer perceptron to obtain a nonlinear feature vector, specifically including: First, the 6-dimensional input vector is linearly projected into a higher-dimensional hidden space (e.g., 64-dimensional) through a first fully connected layer. The transformation formula is as follows: H1=W1·V flight +b1 where W1 is the weight matrix and b1 is the bias vector.

[0134] Subsequently, the ReLU activation function is applied to H1 to introduce nonlinearity, resulting in the nonlinear eigenvector HC1, which is formulated as: HC1=max(0,H1).

[0135] This step aims to learn the complex coupling relationships between flight parameters, such as the impact of coordinated changes in altitude and velocity on the imaging scale.

[0136] Step 3: Feature Compression and Refinement The nonlinear feature vector HC1 is input into the second fully connected layer for feature compression and refinement, mapping it to the target dimension Df (e.g., 32-dimensional) to obtain the flight feature vector. The transformation formula is: F flight =W2·HC1+b2, where W2 and b2 are the weight matrix and bias vector of this layer, respectively. This step refines the learned high-dimensional nonlinear features into a compact and information-rich flight feature vector F. flight ∈R Df .

[0137] The final output flight feature vector no longer has original, explicit physical units (such as meters or degrees), but represents a comprehensive flight state embedding. The different dimensions of this vector jointly encode the current dynamic behavior pattern of the flight platform and its deep abstract features of potential impact on aerial imaging, providing an information-dense and dimensionally unified mathematical representation for subsequent cross-modal fusion.

[0138] Through the above steps, this invention achieves the intelligent transformation of low-level flight parameters with different physical meanings and dimensions into a high-level semantic feature that can be used for neural network reasoning. This is a key prerequisite for realizing the deep integration of physical and visual information.

[0139] S4102 converts environmental scene data into environmental feature vectors through an embedding layer.

[0140] Environmental scene data (including light intensity, shadow distribution, road structure complexity, and traffic participant density) is transformed into environmental feature vectors through an embedding layer: Continuous value processing: Normalize continuous variables such as light intensity and then directly input them into the fully connected layer; Categorical value processing: Categorical variables such as road structure type are embedded into dense vectors using a lookup table; Multimodal fusion: The processed environmental features are concatenated and uniformly mapped to an environmental feature vector F through a fully connected layer. env ∈R Dc Dc represents the dimension of the environmental feature vector.

[0141] S4103 fuses the encoded flight feature vector and environmental feature vector, and aligns them with the dimensions of visual features through spatial expansion, specifically including: The flight feature vector F flight and environmental feature vector F env By concatenating along the channel dimension, we obtain the multi-source feature vector F. multi ; The multi-source feature vector F is broadcast through a broadcast operation. multi The feature map is copied and expanded in the spatial dimension to generate a flight environment feature M with the same spatial size as the initial feature map of the corresponding level. multi ∈R HM×WM×（Df+Dc） , where HM and WM are the height and width of the multi-source feature map, respectively.

[0142] S420, Multimodal Feature Fusion: S4201 splices the spatially expanded flight environment features with local and global features along the channel dimension to form a basic multimodal feature representation.

[0143] S4202 inputs flight environment features into a lightweight guidance network to generate a spatially dimensional fused guidance signal S∈R. H×W×1 .

[0144] The specific implementation of the lightweight bootstrapping network includes: Feature extraction layer: A 3×3 convolutional layer is used to capture the local spatial context of flight environment features; Feature mapping layer: Channel dimension compression is performed using a 1×1 convolutional layer; Output activation layer: The Sigmoid activation function is used to normalize the output value to the [0,1] interval to generate the fusion guidance signal S2.

[0145] The value of each pixel position in the fusion guidance signal is a weight between 0 and 1, representing a priori suggestion from external physical information for the fusion strategy at that position: (1) Guidance with clear physical meaning: When S approaches 1, it indicates that based on flight status and environmental conditions, local detailed features should be prioritized for this region; when S approaches 0, it indicates that this region is more suitable for relying on global context features. (2) Deep integration of multi-source information: The fused guidance signal transforms abstract flight parameters and environmental data into specific space guidance information.

[0146] (3) Advantages of adaptive fusion: It can dynamically adjust the fusion strategy based on external factors such as flight altitude and lighting conditions; It provides stable and reliable fusion guidance in complex aerial photography scenarios, significantly improving the robustness of target detection.

[0147] The introduction of this fusion guidance signal enables deep collaboration between external physical information and internal visual features, providing a more intelligent and adaptive feature fusion scheme for aerial target detection.

[0148] S430, Adaptive Gated Fusion: Based on the multimodal feature representation and fused guiding signal, a spatially adaptive gated weight map is generated through a gating mechanism, specifically including: S4301, the multimodal feature representation is input into a bottleneck structure convolutional network (e.g., a 1×1 convolution to compress the number of channels, followed by a 3×3 convolution for spatial context aggregation, and then a 1×1 convolution to restore the number of channels) to generate a preliminary feature energy map.

[0149] S4302 adds or multiplies the preliminary feature energy map with the fusion guidance signal element by element to modulate the feature energy map, so that the generation process of the gating weight is clearly guided by external multi-source information.

[0150] S4303 passes the modulated feature energy map through an output layer consisting of a 1×1 convolution and a sigmoid activation function to generate the final spatial dimension gated weight map G.

[0151] S440, weighted fusion output: Using the generated gated weight map G, local and global features are weighted and fused using the following formula: The fused feature map = G⊙local features + (1-G)⊙global features. Here, ⊙ represents position-by-position multiplication.

[0152] S500 performs multi-scale fusion of feature maps from multiple levels to obtain multi-scale fused features.

[0153] The S500 specifically includes: S510, resolution alignment: unifies the fused feature maps of multiple levels to the same intermediate spatial resolution; S520, Channel stitching: The aligned feature maps are stitched together along the channel dimension to form multi-scale intermediate features; S530, Multi-dimensional pooling: Global pooling is performed on the multi-scale intermediate features along the height, width and channel dimensions respectively to capture long-range dependencies across scales and obtain multi-dimensional pooling results; S540 inputs the multi-dimensional pooling results into the shared transformation network to predict a unified multi-scale perceptual attention weight map. S550, the multi-scale perceptual attention weight map is applied to the multi-scale intermediate features for enhancement, resulting in multi-scale fused features.

[0154] S600 outputs target detection results based on multi-scale fusion features.

[0155] The differences between this embodiment and Embodiment 1 are explained as follows: Data source expansion: Based on the pure visual analysis in Example 1, this embodiment innovatively introduces two important types of multi-source information: flight parameter data and environmental scene data, realizing a technological upgrade from single-modal to multi-modal perception.

[0156] Enhanced fusion mechanism: The gated fusion module in the S400 step not only processes local and global visual features, but also incorporates flight status and environmental context information, giving the feature fusion process physical perception capabilities and enabling adaptive adjustments for different flight conditions (such as altitude and attitude) and environmental conditions (such as lighting and road structure).

[0157] Improved technical performance: Through the synergistic effect of multi-source data, this embodiment can better cope with the complex challenges in aerial photography scenarios, such as adaptively adjusting the feature extraction strategy when the flight altitude changes and optimizing feature representation under different lighting conditions, thereby significantly improving the robustness and generalization ability of the model while maintaining detection accuracy.

[0158] This embodiment provides a more intelligent and robust solution for aerial target detection by combining multi-source data fusion with an adaptive gating mechanism, and is particularly suitable for practical application scenarios such as drone inspection and security monitoring.

[0159] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being configured to perform the method described in this invention.

[0160] This invention also provides a computer-readable storage medium storing computer-executable instructions for performing the methods described in this invention.

[0161] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0162] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An enhanced target detection method based on feature fusion, characterized in that, The method includes: S100 acquires flight parameter data, environmental scene data, and aerial image data from aerial photography equipment; S200, extract multi-level initial feature maps of the aerial image data through an initial feature extraction network; S300 extracts local and global features respectively through parallel deployment of local and global feature extraction networks for each level of the initial feature map. S400, the local and global features corresponding to each initial feature map, the flight parameter data and the environmental scene data are input into a gating fusion module to generate the corresponding fused feature map; S500 performs multi-scale fusion of feature maps from multiple levels to obtain multi-scale fused features; S600 outputs target detection results based on multi-scale fusion features.

2. The method according to claim 1, characterized in that, in, The gating fusion module is configured to execute the following processing flow: S410, multi-source data encoding and alignment, includes: S4101 encodes flight parameter data into flight feature vectors through a fully connected network; S4102, converts environmental scene data into environmental feature vectors through an embedding layer; S4103 fuses the encoded flight feature vector and environmental feature vector, and achieves dimensional alignment with visual features through spatial expansion; S420, multimodal feature fusion, includes: S4201 combines the spatially expanded flight environment features with local and global features along the channel dimension to form a basic multimodal feature representation; S4202 inputs flight environment characteristics into a lightweight guidance network to generate a spatial dimension fused guidance signal; S430, Based on the multimodal feature representation and fused guidance signal, a spatially adaptive gated weight map is generated through a gating mechanism; S440 uses the generated gated weight map to perform weighted fusion of local and global features to generate the corresponding fused feature map.

3. The method according to claim 2, characterized in that, S4103 specifically includes: The flight feature vector and the environment feature vector are concatenated along the channel dimension to obtain a multi-source feature vector; The multi-source feature vectors are copied and expanded in spatial dimension through broadcasting operations to generate flight environment features with the same spatial size as the initial feature map of the corresponding level.

4. The method according to claim 2, characterized in that, The value of each pixel position in the fusion guidance signal is a weight between 0 and 1, representing a priori suggestion from external physical information for the fusion strategy at that pixel position.

5. The method according to claim 2, characterized in that, The S430 specifically includes: S4301, The multimodal feature representation is input into a bottleneck structure convolutional network to generate a preliminary feature energy map; S4302 modulates the feature energy map by adding or multiplying the preliminary feature energy map with the fusion guidance signal element by element. S4303 passes the modulated feature energy map through an output layer consisting of a 1×1 convolution and a sigmoid activation function to generate the final spatial dimension gated weight map.

6. The method according to claim 2, characterized in that, The local and global features are weighted and fused using the following formula: The fused feature map = G⊙local features + (1-G)⊙global features, where G represents the gated weight map and ⊙ represents the position-by-position multiplication operation.

7. The method according to claim 2, characterized in that, The S500 specifically includes: S510, resolution alignment: unifies the fused feature maps of multiple levels to the same intermediate spatial resolution; S520, Channel stitching: The aligned feature maps are stitched together along the channel dimension to form multi-scale intermediate features; S530, Multi-dimensional pooling: Global pooling is performed on the multi-scale intermediate features along the height, width and channel dimensions respectively to capture long-range dependencies across scales and obtain multi-dimensional pooling results; S540 inputs the multi-dimensional pooling results into the shared transformation network to predict a unified multi-scale perceptual attention weight map. S550, the multi-scale perceptual attention weight map is applied to the multi-scale intermediate features for enhancement, resulting in multi-scale fused features.

8. An electronic device, characterized in that, Including processor and memory; The processor executes the steps of the method as described in any one of claims 1 to 7 by invoking programs or instructions stored in the memory.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a program or instructions that cause a computer to perform the steps of the method as described in any one of claims 1 to 7.

Images