A target detection method based on adaptive feature enhancement and dynamic balance learning

Abstract

The application discloses a target detection method based on adaptive feature enhancement and dynamic balance learning, and belongs to the technical field of computer vision and pattern recognition. The method comprises the following steps: constructing an adaptive double-path backbone network, which comprises a main path and an enhancement path, wherein an adaptive expansion-collaborative attention module is arranged in the enhancement path and is used for dynamically enhancing features; constructing a feature pyramid network based on visual complexity; generating spatially-varying fusion weights according to local image content to adaptively fuse multi-scale features; and constructing a scene-aware dynamic balance detection head, and adopting an EIG-Focal loss function to jointly optimize long-tail distribution and difficult-easy sample imbalance problems. Under the condition that the model parameter quantity is controlled below 1M, the application achieves performance significantly superior to that of the prior art in key indicators such as small target detection and long-tail category recognition, and is particularly suitable for edge intelligent application scenarios such as unmanned aerial vehicles and mobile terminals in which resources are limited.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and pattern recognition technology, specifically relating to a lightweight, high-precision, and highly robust target detection method for resource-constrained platforms (such as drones, mobile terminals, embedded devices, and IoT nodes). Background Technology

[0002] Object detection, a key technology for endowing machines with "vision," is a microcosm of the progress made in computer vision driven by deep learning. From two-stage detectors driven by region proposal (such as R-CNN, Fast R-CNN, Faster R-CNN) to single-stage detectors driven by single-shot regression (such as the YOLO series, SSD, RetinaNet), and then to Transformer architecture detectors based on set prediction (such as DETR), algorithms have continuously approached or even surpassed human-level accuracy. However, these models, which shine on high-performance GPU clusters in data centers, immediately reveal their fatal weakness of being "unsuited" when deployed to edge devices with extremely limited computing power, memory, and power consumption. This "landing gap" from the cloud to the edge has created an urgent need for ultra-lightweight object detection models.

[0003] Although deep learning has driven its performance to new heights, mainstream high-performance models (such as Faster R-CNN, DETR) are difficult to deploy on edge devices due to their huge computational overhead. Existing ultra-lightweight solutions (such as YOLOv5n / v8n) have achieved efficient inference through lightweight backbones such as MobileNet, but still have significant bottlenecks: (1) Insufficient feature representation: Lightweight operations such as depthwise separable convolution weaken the interaction between channels, resulting in weak response to small targets (<32² pixels) in remote sensing images. (2) Static feature fusion: Standard Feature Pyramid Network (FPN) adopts a fixed element-wise addition fusion rule. (3) Long-tail learning bias: Faced with the inherent long-tail data distribution in the real world (a few head category samples are abundant, and a large number of tail category samples are scarce), general loss functions (such as Focal Loss) fail to effectively balance the learning weights of each category, resulting in a serious degradation of the model's ability to identify rare categories. Therefore, a cohesive, collaborative, end-to-end solution is urgently needed to balance accuracy and fairness while achieving extreme lightweightness. Summary of the Invention

[0004] The purpose of this invention is to provide a target detection method based on adaptive feature enhancement and dynamic equilibrium learning. To achieve this purpose, this invention constructs a three-layer cascaded, cohesive target detection framework. This framework deeply integrates all technical means into a unified, jointly optimizable network.

[0005] The present invention provides a target detection method based on adaptive feature enhancement and dynamic equilibrium learning, comprising the following steps: An adaptive dual-path backbone network is constructed to extract multi-scale features from the input image, thus obtaining multi-scale features. A feature pyramid network based on visual complexity is constructed, and the multi-scale features are adaptively fused to obtain multiple fused features. A scene-aware dynamic balance detection head is constructed to classify and regress the multiple fused features, and output the category with the highest probability.

[0006] Furthermore, the adaptive dual-path backbone network includes an initial downsampling layer and five consecutive processing stages, denoted as S1, S2, S3, S4, and S5, respectively; the k-th processing stage outputs features. The multi-scale features are obtained, where k represents the index of the processing stage; Both the S1 and S2 phases use two cascaded standard lightweight building blocks; The S3, S4, and S5 stages have the same structure, all adopting an adaptive dual-path architecture. The adaptive dual-path architecture includes a main path and an enhanced path, and the features output by the main path and the enhanced path are fused to obtain the features output by this processing stage. The main path includes two cascaded standard lightweight building blocks, namely the first standard lightweight building block and the second standard lightweight building block. The enhancement path embeds an adaptive extended-cooperative attention module (A-ECAM). The A-ECAM performs adaptive feature enhancement on the features output from the first standard lightweight building block.

[0007] Furthermore, the standard lightweight building block is defined as follows: ; in, This represents a 1×1 pointwise convolution operation. It uses a 3×3 depthwise convolution, BN for batch normalization, and SiLU for activation. This represents the input data for the standard lightweight building block in the k-th processing stage. This indicates that the standard lightweight building blocks are used for input data. The processed output features.

[0008] Furthermore, the adaptive extended-cooperative attention module performs adaptive feature enhancement on the features output from the first standard lightweight building block, including the following steps: Step 1: Calculate the features output by the first standard lightweight building block. Visual complexity ; Step 2, based on visual complexity Activation factors are generated through dynamic gating functions. ; Step 3, when the activating factor If the value is less than the set threshold, the Adaptive Extended-Collaborative Attention Module (A-ECAM) is skipped, which is equivalent to the output of A-ECAM being 0. The feature output of the k-th processing stage is... Features of the main path output; When activator When the threshold is greater than or equal to the set threshold, the features are processed by the Adaptive Extended-Collaborative Attention Module (A-ECAM). Perform feature enhancement to obtain enhanced features. Then the features output by the k-th processing stage Represented as: ; in, This represents the output of the second standard lightweight building block in the k-th processing stage, where k = 3, 4, 5.

[0009] Furthermore, the Adaptive Extended-Collaborative Attention Module (A-ECAM) for features Perform feature enhancement to obtain enhanced features. The specific steps are as follows: Step 31, for features Perform channel expansion to obtain expanded features. ; Step 32, based on the expanded features Calculate the channel attention weight vectors separately Spatial attention weight map ; Step 33: Perform an outer product operation on the channel attention weight vector. Spatial attention weight map Merge into a collaborative attention weight tensor ; Step 34, utilize the collaborative attention weight tensor For the extended features Recalibrate and project it back to the features using a 1×1 convolution. Enhanced features obtained with the same channel dimensions ; as the output feature of the Adaptive Extended-Collaborative Attention Module (A-ECAM).

[0010] Furthermore, the visual complexity The calculation formula is: ; in, Representation of features Middle position The value at position i represents the feature's index in the height direction, j represents the feature's index in the width direction, H represents the feature's height, and W represents the feature's width. Describing the L2 norm, This is the gradient operator.

[0011] Furthermore, the dynamic gating function is expressed as follows: ; in, It is the Sigmoid activation function. Scaling factor This is a preset threshold.

[0012] Furthermore, the feature pyramid network based on visual complexity includes five fusion layers, which correspond one-to-one with the processing stages in the adaptive dual-path backbone network; when When =1, 2, 3 and 4, the first The fusion process in each fusion layer includes: No. The input to each fusion layer is the feature and characteristics , is represented as: ; ; in, This represents the feature of the output of the k-th processing stage. This represents the feature output in the (k+1)th processing stage. Indicates the feature Upsampled features Indicates upsampling; when When =5, =0, meaning the input to the 5th fusion layer is only the first one. 5 Output features of each processing stage ; Calculation features Visual complexity graph , is represented as: ; Where C represents the number of channels, Representation of features height, Representation of features width, Representation of features The c-th channel is at position The gradient magnitude at point c = 1, 2, …, C; Calculate the dynamic fusion weight graph , is represented as: ; in, The Sigmoid activation function is used. This represents a 3×3 convolution operation; Finally, the first The fusion features output by each fusion layer The calculation is as follows: ; Here, ⊙ represents element-wise multiplication.

[0013] Furthermore, the scene-aware dynamic balance detection head SA-DB Head includes regression and classification branches; The classification branch is optimized using the scene-aware EIG-Focal loss function, and the classification loss is... Defined as: ; in, For real labels, Let α represent the probability predicted by the model, and α≥0 represent the focusing parameter of the focus loss. Indicates dynamic weights; Indicates the difficulty of the scenario. Indicates the weight of basic effective information. The modulation intensity parameter represents the scene difficulty. Index representing the category, This represents the total number of categories, and tanh() is the hyperbolic tangent function.

[0014] Furthermore, the total loss of the object detection network Represented as: ; in, It is the total number of all positive samples in a batch. It is the classification loss for all positive samples. It is the regression loss, where Pos is the set of all positive samples. The index represents the positive sample; the target detection network includes the adaptive dual-path backbone network, the feature pyramid network VC-FPN, and the scene-aware dynamic balance detection head SA-DB Head.

[0015] Beneficial Effects: This invention discloses a target detection method based on adaptive feature enhancement and dynamic balance learning, belonging to the field of computer vision and pattern recognition technology. The method constructs a lightweight target detection framework, which includes an adaptive dual-path backbone network, a visual complexity-based feature pyramid network, and a scene-aware dynamic balance detection head. The adaptive dual-path backbone network dynamically enhances features by introducing an adaptive extension-cooperative attention module in the enhancement path, improving feature representation capabilities; the visual complexity-based feature pyramid network generates dynamic fusion weights based on the local visual complexity of the image, achieving adaptive fusion of multi-scale features. The method of this invention effectively solves key technical problems commonly found in complex application scenarios such as remote sensing images and video streams, such as the contradiction between model efficiency and accuracy, long-tail data distribution, and difficulty in small target detection, by integrating lightweight network architecture design, adaptive feature pyramid, and dynamic class balance loss function, thus providing strong technical support for edge intelligent computing. The present invention includes an adaptive dual-path backbone network, a visual complexity feature pyramid network (VC-FPN), and an execution scene-aware dynamic balance detection head (SA-DBHead); the total number of parameters of the dual-path backbone network is less than 1,000,000, and the computational cost is less than 6 GFLOPs when the input resolution is 640x640. This invention achieves three interrelated and highly challenging objectives simultaneously, while strictly controlling the total number of model parameters to below 1,500,000: High-precision multi-scale detection: Especially in scenarios with dense small targets, such as remote sensing and aerial photography, the recall and precision for detecting tiny targets (area <32² pixels) reach industry-leading levels. Intrinsic class fairness: Through a novel dynamic learning mechanism, it effectively mitigates the bias caused by long-tail data distribution, ensuring that the model's ability to identify rare tail categories remains within a reasonable and acceptable range compared to common head categories, rather than completely failing. Deployment friendliness: The entire model architecture is simple, has no external dependencies, is end-to-end trainable, has fast inference speed, and low memory consumption. It can be seamlessly deployed on various resource-constrained edge computing hardware platforms without complex post-processing or additional security modules. Attached Figure Description

[0016] Figure 1 This is a flowchart of the image processing of an adaptive dual-path backbone network.

[0017] Figure 2 This is a flowchart of the feature processing of the Adaptive Extended-Collaborative Attention Module (A-ECAM) of the present invention.

[0018] Figure 3 This is a flowchart of the scene-aware dynamic balance detection head SA-DB Head classification process, where (a) is the regression branch diagram for locating the target and (b) is the classification branch diagram for identifying the target.

[0019] Figure 4 This is the overall flowchart of the method of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and through a complete, non-limiting embodiment.

[0021] Example 1 The present invention provides a lightweight and robust object detection method based on adaptive feature enhancement and dynamic equilibrium learning, such as... Figure 1 As shown, it includes the following steps: Step 1: Construct an adaptive dual-path backbone network to extract multi-scale features from the input image. The adaptive dual-path backbone network uses depthwise separable convolutions as its basic building blocks and adopts a dual-path architecture in at least one stage, with one path being the main path and the other being the enhancement path. The enhancement path contains an adaptive expansion-co-attention module (A-ECAM). An adaptive dual-path backbone network is used as the feature extraction front-end in the method of this invention to extract features from the input image. Extract a series of multi-scale features {P1, P2, P3, P4, P5}, where the features... The spatial resolution is the input. k represents the index of the processing stage. This represents the feature output by the k-th processing stage. express 3D real space, express The real space of dimension .

[0022] The Adaptive Dual-path Backbone consists of an initial downsampling layer (Stem Layer) and five consecutive processing stages (Stage S1-S5), denoted as S1 stage, S2 stage, S3 stage, S4 stage, and S5 stage, respectively.

[0023] Phases S1 and S2 use standard lightweight building blocks, which are defined as follows: ; Where DWConv represents depthwise convolution, Conv represents pointwise convolution, BN is batch normalization layer, and SiLU is activation function. This represents the input data for the standard lightweight building block in stage k; In the S3, S4, and S5 processing stages, this specific implementation employs an innovative adaptive dual-path architecture. This architecture comprises a main path and an enhancement path. The main path, similar to the preceding S1 and S2 processing stages, consists of two cascaded lightweight building blocks (BBs): a first standard lightweight building block (BB) and a second standard lightweight building block (BB). The enhancement path includes the Adaptive Extended-Collaborative Attention Module (A-ECAM).

[0024] like Figure 2 As shown, the workflow of the Adaptive Extended-Collaborative Attention Module (A-ECAM) includes: channel extension of the input features from the main path; calculation of the spatial attention weight map and channel attention weight vector of the extended features; fusion of the spatial attention weight map and channel attention weight vector through an outer product operation to generate a complete spatial-channel collaborative attention weight tensor; recalibration of the extended features using the attention weight tensor, and fusion of it with the output features of the main path, as detailed below: (a) Receive the output features from the first standard lightweight building block BB of the main path. As input, express 3D real space; visual complexity of computing input features Visual complexity Defined as the average gradient magnitude of the feature: ; in, Representation of features Middle position The value at position i represents the feature's index in the height direction, j represents the feature's index in the width direction, H represents the feature's height, and W represents the feature's width. This represents the L2 norm.

[0025] (b) Based on visual complexity An activation factor between 0 and 1 is generated through a dynamic gating function. The dynamic gating function is represented as follows: ; in, It is the Sigmoid activation function. Scaling factor The preset threshold; (c) When the activator After being calculated, when the activation factor When the value is less than 0.5, the calculation of the Adaptive Extended-Collaborative Attention Module (A-ECAM) is skipped, and only the main path is retained; If activator If the value is greater than or equal to 0.5, then the following complete enhancement process will be executed: (c1) Output characteristics of the first standard lightweight building block BB Perform channel expansion: , where rC is the length of the vector, is a product dimension, indicating that there are a total of r×C dimensions, r>1 is the expansion factor, and C is the channel dimension; (c2) Based on features after channel expansion Calculate the channel attention weight vectors respectively. Spatial attention weight map ; (c3) The two are fused through an outer product operation to generate a three-dimensional collaborative attention weight tensor. ,in ; (c4) Utilizing the 3D Collaborative Attention Weight Tensor Features after channel expansion Recalibrate it and project it back to the C-dimensional channel through a 1×1 convolution; (d) Add the output of step (c4) to the output of the second standard lightweight building block of the main path to obtain the final output feature of this stage. Activation factor When the value is less than 0.5, this step will not be executed, and the output feature of the main path building block will be the final output feature of this stage.

[0026] Step 2: Construct a Visual Complexity-Based Feature Pyramid Network (VC-FPN) to fuse multi-scale features. The VC-FPN transmits information through top-down and bottom-up paths to generate a dynamic fusion weight for each spatial location, thereby achieving spatially adaptive feature fusion. Visual Complexity-Based Feature Pyramid Network (VC-FPN) is used for intelligent fusion of multi-scale features from the backbone network. For the first... layer( =1,2,3,4), and its input is the feature of this layer. ( ) and features after top-down upsampling ; The core of the Visual Complexity Feature Pyramid Network (VC-FPN) lies in calculating the features related to the input features. Corresponding visual complexity diagram .

[0027] Calculate the input to the first Visual complexity diagram of a layer feature pyramid network. Let the input be from the first to the second layer. The features of the layer feature pyramid network are ,in, Representation of features height, Representation of features The width.

[0028] First, the gradient magnitude is calculated for the feature in the spatial dimension. For the c-th channel (c=1,2,…,C), the gradient magnitude is calculated at the position... The gradient magnitude at a given point is defined as: ; in, Represents the gradient in the horizontal direction. Represents the gradient in the vertical direction. Representation of features The c-th channel is at position The characteristics at that location. Then, averaging is performed along the channel dimension to obtain... × Visual complexity graph ,in: ; Where C represents the number of channels, Representation of features height, Representation of features width, Representation of features The c-th channel is at position The gradient magnitude at point c = 1, 2, …, C. The visual complexity map is used to characterize the degree of structural change at different spatial locations in an image; edges and textured regions have higher complexity values, while flat regions have lower complexity values.

[0029] The Visual Complexity-Based Feature Pyramid Network (VC-FPN) uses a lightweight transformer (consisting of a 3x3 convolutional layer and a sigmoid activation function) to transform... Mapped to a dynamic fusion weight graph : ; Finally, a spatially adaptive weighted fusion operation is performed to obtain the fused features. : ; Here, ⊙ represents element-wise multiplication, and this operation achieves content-aware pixel-level feature fusion. Specifically, for the features fused at the 5th layer... In other words, because there is no 6th layer feature Therefore, there are no features after top-down upsampling. ,therefore .

[0030] Step 3: Construct a Scene-Aware Dynamic Balanced Head (SA-DB Head) to process the fused features. The system performs target classification and localization, and outputs target detection results, including the final target bounding box, category label, and confidence score. The scene-aware dynamic balance detection head (SA-DB Head) uses an EIG-Focal loss function when calculating the classification loss. This loss function pre-calculates the effective information weights based on the number of samples in each category in the training set, and combines the Focal Loss mechanism to focus on hard-to-classify samples.

[0031] For each scale of the second layer and above, the fusion features Connect a detection head that contains a regression branch (for predicting bounding box coordinates) and a classification branch (for predicting class probabilities).

[0032] The innovation of SA-DB Head lies in the scene-aware EIG-Focal loss function (SA-EIG-Focal Loss) used in its classification branch. This loss function is used to simultaneously address the problems of class imbalance and imbalance between easy and difficult samples.

[0033] For any positive sample i_sample in the training set (i.e., the anchor point that matches a real target box), its dynamic effective information weights The calculation method is as follows: (a) Pre-statistics for each category Total number of training samples And calculate the global maximum number of samples. ; (b) Calculation Category Basic effective information weights: ,in It is a smoothing constant. ∈(0,1] represents the equilibrium strength parameter; (c) Assess the local scene difficulty at the location of sample i_sample ; (d) Weighting the basic effective information and scene difficulty Combining these, we obtain the final dynamic weights: ; Where η≥0 is the scene difficulty modulation intensity parameter, and tanh() is the hyperbolic tangent function.

[0034] The final classification loss of sample i_sample Defined as: ; in, For real labels (one-hot encoded) Let α represent the probability predicted by the model, and α≥0 be the focusing parameter of Focal Loss. This invention represents the total loss of the object detection network. The target detection network comprises the adaptive dual-path backbone network, the feature pyramid network VC-FPN, and the scene-aware dynamic balance detection head SA-DB Head, which is the sum of the classification loss and GIoU (Generalized Intersection over Union) regression loss for all positive samples.

[0035] Step 4: Correct the weights of some parameters in each layer of the network through backpropagation of errors, such as... Figure 4 As shown.

[0036] After calculating the final classification loss Then, the error gradient is backpropagated to each layer of the network using the chain rule: Detection Head Layer: First, correct the convolutional layer weights in the detection head, updating the formula as follows: ; Where t is an integer representing the current update round number, This represents the weight of the detector head after the (t+1)th training step (or iteration). This represents the current weight of the detector head at the t-th training step. Controlling the step size of each parameter update determines the speed and stability of model convergence. Represents the total loss function The gradient (partial derivative) of the current weights of the detector head. It indicates the direction of weight updates—that is, adjusting the weights along the direction of the fastest decrease in loss.

[0037] Feature Pyramid Layer (VC-FPN): The gradient continues to propagate backward, correcting the fusion weights in the VC-FPN. Since the fusion weights in the VC-FPN are dynamically generated from the visual complexity map, backpropagation adjusts the parameters of the convolutional layers that generate these weights. The update formula is similar to the update formula for the weights of the detection head convolutional layers, enabling the network to learn to automatically reduce noise weights in complex background regions and enhance feature weights in target regions.

[0038] Backbone (Adaptive Dual-path Backbone): Gradients are ultimately backpropagated to the backbone network. In the standard lightweight building block (BB), backpropagation corrects the weights of the convolutional layers. In the A-ECAM module, backpropagation corrects the parameters for channel expansion and attention calculation. The update formula is similar to the update formula for the weights of the detection head convolutional layers. This means that the network not only determines "what is important" (attention mechanism) through forward propagation, but also learns "how to adjust parameters to make important features more prominent" through backpropagation. This invention can be deployed on resource-constrained edge computing devices, including drones, mobile terminals, smart cameras, onboard computing units of autonomous vehicles, industrial robot controllers, or embedded AI chips.

[0039] This embodiment takes the construction and deployment of an ultra-lightweight vehicle and pedestrian detection system for urban traffic monitoring scenarios as an example, and elaborates on the entire process from data preparation, model building, training to edge device deployment.

[0040] Example 2 1. Application Scenarios and System Objectives Application scenarios: Ultra-lightweight vehicle and pedestrian detection system for urban traffic monitoring scenarios, deployed in smart cameras at urban intersections, analyzes 1080p video streams in real time, and detects five common traffic participants: pedestrians, bicycles, motorcycles, cars, and buses. Core objectives: Total model parameters ≤ 1,000,000; End-to-end inference speed ≥ 30 FPS on the NVIDIA Jetson AGX Orin edge computing platform; Maintain high recall in scenarios containing small objects (such as pedestrians in the distance) and occlusion.

[0041] 2. Hardware and software environment configuration Training server: CPU: Intel Xeon Gold 6248R; GPU: 4×NVIDIA V100 32GB; RAM: 256 GB Deep learning framework: PyTorch 1.12.1 + CUDA 11.6 + cuDNN 8.4.0

[0042] 3. Dataset Preparation and Preprocessing Dataset: A subset of the BDD100K (Berkeley DeepDrive) dataset was used. This dataset contains 100,000 1280×720 street view images, labeled with 10 categories. In this example, 5 relevant categories were selected: pedestrian, rider, car, bus, and motorcycle. Data partitioning: The BDD100K (Berkeley DeepDrive) dataset was partitioned into a training set (70k images) and a validation set (10k images).

[0043] 4. Model Building Overall Architecture: Instantiating the "Adaptive PICO-Det++" model proposed in this invention. First layer: Adaptive dual-path backbone network The adaptive dual-path backbone network is the cornerstone of this invention, designed to maximize feature representation within an extremely limited computational budget. Instead of simply stacking depthwise separable convolutions, the adaptive dual-path backbone network introduces a dual-path architecture and a co-attention mechanism. Lightweight Building Blocks: The input image in the adaptive dual-path backbone network undergoes five consecutive processing stages (S1-S5), each outputting corresponding features, P1, P2, P3, P4, and P5. Stages S1-S2 employ two standard lightweight building blocks, both based on depthwise separable convolution, supplemented by batch normalization (BN) and the SiLU activation function. The adaptive dual-path backbone network ultimately outputs five features at different scales: {P1, P2, P3, P4, P5}. These features {P1, P2, P3, P4, P5} are then fed into a feature pyramid network.

[0044] Dual-path Architecture: In order to enrich the feature representation without significantly increasing the parameters, a dual-path design was adopted in the S3, S4 and S5 processing stages, including the main path and the augmentation path. The input features were divided into two independent branches for processing. Main Path: Contains two standard lightweight building blocks responsible for routine feature extraction; Enhancement Path: This is a short path specifically designed for feature enhancement, specifically the Adaptive Extended-Collaborative Attention Module (A-ECAM), whose core is the Extended Attention Module (EAM).

[0045] Adaptive Extended-Collaborative Attention Module (A-ECAM): This paper proposes an extended-collaborative attention module (ECAM) to upgrade the original attention module (EAM). A-ECAM not only operates on a single path but also facilitates information interaction between the main path and the enhanced paths.

[0046] Step 1: Channel Expansion: The input to the enhanced path is the output feature of the first standard lightweight building block of the main path. , This represents the output feature of the first standard lightweight building block in the main path during the k-th processing stage. First, its channel count is expanded from C to rC (r=2) using a 1×1 convolution: ; in, Represents the features in the k-th processing stage. Features after channel expansion; Step 2: Spatial-channel collaborative attention, which is the key innovation of this invention. Instead of simply calculating channel attention, it models spatial attention and channel attention simultaneously and makes them work together.

[0047] Channel attention branch: Perform global average pooling (GAP) to obtain a vector. Channel attention weight vectors are generated using a multilayer perceptron (MLP) with a dimensionality reduction ratio of k_ratio = 4. ; ; in, This represents the weight matrix of the first fully connected layer (dimensionality reduction), used to reduce and compress global information to decrease computation and model inter-channel relationships. Its dimension is typically [dimensionality value missing]. , This represents the weight matrix of the second fully connected layer (dimensionality increase after dimensionality reduction), used to increase the dimensionality of intermediate features and restore them to the original number of channels. It typically has dimensions of [missing information]. (r is the dimensionality reduction ratio). This represents a nonlinear activation function (usually ReLU or SiLU), which increases the expressive power of the model by introducing nonlinearity.

[0048] Spatial attention branch: Max pooling and average pooling are performed along the channel dimension, the results are concatenated, and a spatial attention weight map is generated by a 7×7 convolution. : ; in, This represents a 7×7 convolution operation; Synergistic Integration: Channel Weighting and spatial weights Performing the outer product yields a complete attention weight tensor. : ; in, This represents the outer product operation. This means that each channel c at each position (i,j) has an independent weight. This enables fine-grained feature recalibration.

[0049] Step 3: Feature Fusion and Projection: Apply attention weight tensors With extended features Element-wise multiplication is performed, and then the result is projected back to the C-dimensional dimension through a 1×1 convolution to obtain the enhanced features. ; Finally, the enhanced features will be obtained. Output of the second standard lightweight building block of the main path Add them together to obtain the features of the output of the k-th processing stage. Represented as: ; Key parameters of the Adaptive Expanded-Collaborative Attention Module (A-ECAM): Channel expansion factor r = 2, MLP dimensionality reduction ratio k_ratio = 4, and gating function parameters. The Extended-Collaborative Attention Module (ECAM) achieves precise control over features in both spatial and channel dimensions with minimal increase in computation through a collaborative attention mechanism, greatly improving feature quality, especially beneficial for the activation of small targets.

[0050] Second layer: Feature pyramid network based on visual complexity The fusion formula for the standard feature pyramid (FPN) is as follows: .in, In this formula, the "+" sign is fixed, meaning that regardless of the input, the low-level features... and high-level characteristics The input is always mixed in a 1:1 ratio. This clearly ignores the diversity of the input content; The core idea of ​​this invention is that the fusion weights should be a function of the input content. Therefore, for each fusion layer... ( A lightweight Content-Aware Weight Predictor (CAWP) was designed for (e.g., 1, 2, 3, 4).

[0051] Input: For the first The layer fusion layer, where the Content-Aware Weight Predictor (CAWP) takes two features as input: the first feature and the second feature. k Output features of each processing stage and the first k+ Output features of one processing stage Upsampled features .

[0052] Calculate the dynamic fusion weight graph , is represented as: ; in, This represents a 3×3 convolution operation; Dynamic fusion weight graph It is spatially variable, meaning different regions of the image can have different fusion weights. For example, in the region where the target is located, if the lower-level details are clearer, The value will be close to 1; in the background region, if the higher-level semantics are more reliable, The value will be close to 0.

[0053] Adaptive Fusion: The Final First The fusion features output by each fusion layer The calculation is as follows: ; Here, ⊙ represents element-wise multiplication. This spatially adaptive fusion method enables the feature pyramid based on visual complexity to dynamically determine the information source according to the characteristics of local image content, thus achieving unprecedented flexibility and intelligence.

[0054] Training details: All parameters of the Content-Aware Weight Predictor (CAWP), including the encoder and 1×1 convolutions, are jointly optimized end-to-end with the backbone network and the detector head using the standard backpropagation algorithm. During training, the network automatically learns the optimal content-weight mapping.

[0055] Third layer: Dynamic balance detection head guided by effective information This layer is the "decision center" of the model, and its design directly determines whether the model's "attitude" towards different categories is fair and reasonable.

[0056] Basic structure of the detection head: Fusion features at each scale ( Each of the sequences (2, 3, 4, 5) is connected to an independent detector head. Each detector head contains two parallel 3×3 convolutional branches: Regression branches: such as Figure 3 As shown in (a) above, this is the regression branch plot for locating the target, outputting four values ​​representing the offset of the predicted bounding box. The GIoU Loss function is used for supervision because it handles non-overlapping boxes better and outperforms the traditional Smooth L1 Loss. Classification branches: such as Figure 3 As shown in (b) above, this is the classification branch diagram for target identification, and the output... Values ​​( (where is the number of foreground categories), after Sigmoid activation, the independent probability of each category is obtained. This is the key innovative area of ​​this invention.

[0057] The formal definition of an effective information metric is as follows: Before training begins, the entire training set is traversed, and statistics are collected for each category. Number of positive samples Then, calculate the maximum number of samples. The weight of the basic effective information of category c_class. Defined as: ; in, It is a category The number of training samples, It is the maximum number of samples across all categories. It is a very small constant. It is a hyperparameter used to control the equilibrium strength. ∈(0,1], when hyperparameter When = 1, it degenerates into a strictly inverse frequency-weighted system; when the hyperparameters... When the hyperparameters approach 0, it degenerates into unweighted weighting. This can be achieved by adjusting the hyperparameters. It can flexibly find the optimal balance between "complete equilibrium" and "preserving the original distribution". In experiments, it was found that the hyperparameter... =0.5 is a robust default value that performs well across multiple datasets.

[0058] Construction of the EIG-Focal loss function: To address the intertwined problems of class imbalance and imbalance between easy and difficult samples, the EIG-Focal loss function is proposed by deeply integrating the concept of effective information measurement with Focal Loss. Focal Loss Review: Standard Focal Loss is defined as... ,in It is the predicted probability of the true category. It is the balance factor, and α is the focusing parameter; EIG-Focal's innovation: In object detection, independent binary classification is typically used for each foreground category, thus balancing the factors... This is usually omitted. The innovation of this invention lies in incorporating dynamic weights. As a dynamic amplification factor in Focal Loss; ; in, It is a true tag with one-hot encoding. It is the predicted category The probability. α is the focusing parameter of Focal Loss, which is 2 in this embodiment.

[0059] The total loss of the entire model is the weighted sum of the classification loss and regression loss for all positive samples: ; in, It is the total number of all positive samples in a batch. It is the classification loss for all positive samples. This is the GIoU regression loss, where Pos is the set of all positive samples. 80% of the dataset was used for training and 20% for testing; the results are shown in Table 1.

[0060] Table 1 Performance Evaluation and Validation

[0061] This invention improves the mean average precision (mAP) by 2.7 percentage points and the average precision for small objects (AP_s) by 7.7 percentage points, while reducing the model size to 30.6% of the baseline. This fully meets the comprehensive requirements of urban traffic monitoring for accuracy, efficiency and small object detection capabilities.

[0062] Example 3 1. Application Scenarios and Task Definitions Application scenarios: A remote sensing image arbitrary direction target detection system based on DOTA-v1.5, deployed on military / civilian UAVs or satellite ground stations, performs automatic target recognition (ATR) on the acquired high-resolution optical remote sensing images based on the target detection method in Specific Implementation 1, for use in maritime surveillance, airport activity analysis, battlefield situational awareness, etc.

[0063] Core task: Detect targets such as aircraft, ships, vehicles, and ports on the DOTA-v1.5 dataset, which contains 1,882,893 instances across 16 categories. All targets are labeled with rotated bounding boxes (RBoxes) in the following format: ,in, This represents the x-coordinate of the starting position of the target detection box in the image. This represents the ordinate of the starting position of the target detection box in the image. This represents the width of the target detection bounding box. The height of the target detection box is represented by θ, which is the rotation angle (in radians).

[0064] Technical challenges: Extremely wide target scale: from distant vehicles with a few pixels to port facilities with thousands of pixels. High proportion of small targets: targets with an area of ​​less than 1024 pixels account for 45% of the total. Dense arrangement: such as airplanes on the tarmac and ships in the port, they are very easy to obscure each other.

[0065] 2. Hardware and Software Environment Training server: Same as in Example 1 (4×NVIDIA V100, PyTorch 1.12). Deployment platform: NVIDIA Jetson AGX Xavier (for onboard real-time processing).

[0066] 3. DOTA-v1.5 Dataset Processing Dataset composition: Training set: 1,411 large images (sizes ranging from 800×800 to 20,000×20,000). Validation set: 458 large images. Test set: Unlabeled images submitted to the official server for evaluation. Image tiling (Crucial Step): Due to the large size of the original image, an overlapping sliding window tiling strategy is used. Tile size: 1024×1024. Overlap step: 512 pixels (i.e., 50% overlap) to reduce truncation of objects at the boundaries. Approximately 180,000 sub-images are generated after tiling.

[0067] 4. Model Architecture Input resolution: 1024×1024 (consistent with slice size); Adaptive dual-path backbone network: The adaptive extension-cooperative attention module A-ECAM of this invention is embedded in stages S2-S5 (because the overall information density of remote sensing images is high, adaptive enhancement needs to be enabled in more stages). Adaptive Extended-Collaborative Attention Module (A-ECAM) parameters: r=2, k_ratio =4, (Remote sensing images have rich textures and generally high visual complexity values.)

[0068] Modification of the detection head: Regression branch: Outputs 5 values Rotated IoU (RIoU) Loss is used instead of standard GIoU Loss; where, This indicates the distance or proportion by which the center point of the prediction box needs to be moved horizontally (x-axis) to align with the center of the target. This indicates the distance or proportion by which the center point of the prediction box needs to be moved in the vertical direction (y-axis). This indicates the amount by which the width of the prediction box needs to be adjusted to wrap around the width of the target. This indicates the amount by which the height of the prediction box needs to be adjusted to include the height of the target. This indicates the angle by which the predicted bounding box needs to be rotated. By adjusting this angle, the detection box can fit a tilted target (such as an object placed at an angle), instead of simply using a horizontal rectangle; Classification branch: SA-EIG-Focal Loss will still be used; Hyperparameters: γ=0.8 (medium-long tail in DOTA category distribution), η=0.7 (emphasizing small objectives in difficult scenarios).

[0069] 5. Model Training Details We employ stochastic gradient descent with momentum (SGD with Momentum), setting the learning rate (lr) to 0.01, momentum to 0.9, and weight decay to 5e-4. This is because SGD is typically more stable than Adam for remote sensing tasks. The learning rate scheduling strategy uses multi-step decay (MultiStepLR), reducing the learning rate to one-tenth of its original value at the 200th and 250th training epochs, respectively.

[0070] Total training epochs: 300. Batch size: 24 (limited by the large input size of 1024×1024). Label assignment: A rotation-adaptive training sample selection (R-ATSS) strategy is used to dynamically select positive samples for each ground truth rotated bounding box.

[0071] 6. Performance Evaluation and Results The results are shown in Table 2. Significant improvement was achieved for small objects: AP_s increased by 7.4 percentage points, demonstrating the strong detection capabilities of the Adaptive Extended-Collaborative Attention (A-ECAM) and VC-FPN modules for small ships and vehicles. The model is extremely lightweight: the number of parameters is only 23.7% of ReDet-tiny, achieving a real-time processing speed of 18 FPS on Jetson AGX Xavier. Here, AP_s refers to the average precision on small objects, AP_m refers to the average precision on medium objects, and AP_l refers to the average precision on large objects.

[0072] Table 2 Performance evaluation and results in Example 3

[0073] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

[0074] Based on the experimental results, the method of the present invention has the following advantages: 1. Extremely Lightweight: Thanks to depthwise separable convolutions, a dual-path design, and a streamlined content-aware weight predictor (CAWP), the complete model of this invention (excluding any external modules) has a total parameter count of only 0.92 million, a computational cost of 5.8 GFLOPs at 640×640 input, and a model file size of only 2.1 MB. This allows it to run at 28 FPS on a Raspberry Pi 4B and 42 FPS on an NVIDIA Jetson Nano, fully meeting the needs of most edge real-time applications.

[0075] 2. Superior Small Target Detection Capability: On the DOTA-v1.5 remote sensing dataset, the present invention achieves an mAP50 of 43.1%. Particularly noteworthy is its AP_s metric of 28.7% for small targets (area < 32²), which is 12.5 percentage points higher than the best-performing baseline model YOLOv8n (AP_s = 16.2%). Ablation experiments demonstrate that the Extended-Collaborative Attention Module (ECAM) and the visual complexity-based feature pyramid contribute significantly to this, resulting in AP_s gains of 4.2% and 3.8%, respectively.

[0076] 3. Excellent long-tail learning performance: On the highly challenging long-tail instance segmentation / detection dataset LVIS v1.0, this invention achieved the following results: overall AP: 21.5% (1.4% higher than the baseline using standard Focal Loss), tail class AP: 17.8% (9.1% higher than the baseline), and extremely rare class AP (Few-shot, 1-10 samples): 10.3% (7.2% higher than the baseline). These data eloquently demonstrate the powerful capabilities of EIG-Focal loss in handling long-tail problems.

[0077] 4. Efficient Training and Convergence: Since the introduction of the Content-Aware Weight Predictor (CAWP) and the Extended Collaborative Attention Module (ECAM) does not significantly increase the optimization difficulty, the convergence speed of the model in this invention is comparable to that of the standard YOLOv8n, typically reaching optimal performance within 200 epochs. This ensures its deployability in real-world projects.

Claims

1. A target detection method based on adaptive feature enhancement and dynamic equilibrium learning, characterized in that, Includes the following steps: An adaptive dual-path backbone network is constructed to extract multi-scale features from the input image, thus obtaining multi-scale features. A feature pyramid network based on visual complexity is constructed, and the multi-scale features are adaptively fused to obtain multiple fused features. A scene-aware dynamic balance detection head is constructed to classify and regress the multiple fused features, and output the category with the highest probability.

2. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 1, characterized in that, The adaptive dual-path backbone network includes an initial downsampling layer and five consecutive processing stages, denoted as S1, S2, S3, S4, and S5. The k-th processing stage outputs features. The multi-scale features are obtained, where k represents the index of the processing stage; Both the S1 and S2 phases use two cascaded standard lightweight building blocks; The S3, S4, and S5 stages have the same structure, all adopting an adaptive dual-path architecture. The adaptive dual-path architecture includes a main path and an enhanced path, and the features output by the main path and the enhanced path are fused to obtain the features output by this processing stage. The main path includes two cascaded standard lightweight building blocks, namely the first standard lightweight building block and the second standard lightweight building block. The enhancement path embeds an adaptive extended-cooperative attention module (A-ECAM). The A-ECAM performs adaptive feature enhancement on the features output from the first standard lightweight building block.

3. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 2, characterized in that, The standard lightweight building block is defined as follows: ； in, This represents a 1×1 pointwise convolution operation. It uses a 3×3 depthwise convolution, BN for batch normalization, and SiLU for activation. This represents the input data for the standard lightweight building block in the k-th processing stage. This indicates that the standard lightweight building blocks are used for input data. The processed output features.

4. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 2, characterized in that, The adaptive extension-cooperative attention module performs adaptive feature enhancement on the features output from the first standard lightweight building block, including the following steps: Step 1: Calculate the features output by the first standard lightweight building block. Visual complexity ; Step 2, based on visual complexity Activation factors are generated through dynamic gating functions. ; Step 3, when the activating factor If the value is less than the set threshold, the Adaptive Extended-Collaborative Attention Module (A-ECAM) is skipped, which is equivalent to the output of A-ECAM being 0. The feature output of the k-th processing stage is... Features of the main path output; When activator When the threshold is greater than or equal to the set threshold, the features are processed by the Adaptive Extended-Collaborative Attention Module (A-ECAM). Perform feature enhancement to obtain enhanced features. Then the features output by the k-th processing stage Represented as: ； in, This represents the output of the second standard lightweight building block in the k-th processing stage, where k = 3, 4, 5.

5. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 4, characterized in that, Adaptive Extended-Collaborative Attention Module (A-ECAM) for Features Perform feature enhancement to obtain enhanced features. The specific steps are as follows: Step 31, for features Perform channel expansion to obtain expanded features. ; Step 32, based on the expanded features Calculate the channel attention weight vectors separately Spatial attention weight map ; Step 33: Perform an outer product operation on the channel attention weight vector. Spatial attention weight map Merge into a collaborative attention weight tensor ; Step 34, utilize the collaborative attention weight tensor For the extended features Recalibrate and project it back to the features using a 1×1 convolution. Enhanced features obtained with the same channel dimensions ; As the output feature of the Adaptive Extended-Collaborative Attention Module (A-ECAM).

6. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 4, characterized in that, The visual complexity The calculation formula is: ； in, Representation of features Middle position The value at position i represents the feature's index in the height direction, j represents the feature's index in the width direction, H represents the feature's height, and W represents the feature's width. Describing the L2 norm, This is the gradient operator.

7. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 4, characterized in that, The dynamic gating function is expressed as follows: ； in, It is the Sigmoid activation function. Scaling factor This is a preset threshold.

8. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 2, characterized in that, The feature pyramid network based on visual complexity includes five fusion layers, which correspond one-to-one with the processing stages in the adaptive dual-path backbone network. when When =1, 2, 3 and 4, the first The fusion process in each fusion layer includes: No. The input to each fusion layer is the feature and characteristics , is represented as: ； ； in, This represents the feature of the output of the k-th processing stage. This represents the feature output in the (k+1)th processing stage. Indicates the feature Upsampled features Indicates upsampling; when When =5, =0, meaning the input to the 5th fusion layer is only the first one. 5 Output features of each processing stage ; Calculation features Visual complexity graph , is represented as: ； Where C represents the number of channels, Representation of features height, Representation of features width, Representation of features The c-th channel is at position The gradient magnitude at point c = 1, 2, …, C; Calculate the dynamic fusion weight graph , is represented as: ； in, The Sigmoid activation function is used. This represents a 3×3 convolution operation; Finally, the first The fusion features output by each fusion layer The calculation is as follows: ； Here, ⊙ represents element-wise multiplication.

9. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 1, characterized in that, The scene-aware dynamic balance detection head SA-DB Head includes regression and classification branches; The classification branch is optimized using the scene-aware EIG-Focal loss function, and the classification loss is... Defined as: ； in, For real labels, Let α represent the probability predicted by the model, and α≥0 represent the focusing parameter of the focus loss. Indicates dynamic weights; Indicates the difficulty of the scenario. Indicates the weight of basic effective information. The modulation intensity parameter represents the scene difficulty. Index representing the category, This represents the total number of categories, and tanh() is the hyperbolic tangent function.

10. The target detection method based on adaptive feature enhancement and dynamic equilibrium learning according to claim 1, characterized in that, Total loss of object detection network Represented as: ； in, It is the total number of all positive samples in a batch. It is the classification loss for all positive samples. It is the regression loss, where Pos is the set of all positive samples. The index represents the positive sample; the target detection network includes the adaptive dual-path backbone network, the feature pyramid network VC-FPN, and the scene-aware dynamic balance detection head SA-DB Head.

Images