Method and device for detecting aerial unmanned vehicle group target based on YOLOv7-tiny

Abstract

The application provides a YOLOv7-tiny-based aerial unmanned aerial vehicle group target detection method and device, and relates to the technical field of target detection. The method comprises the following steps: constructing a data set and preprocessing; constructing a network based on YOLOv7-tiny, introducing a multi-modal image level domain self-adaptive module into the backbone network, realizing cross-modal feature calibration through FiLM modulation, and aligning the domain distribution by combining an adaptive gradient inversion strategy; introducing a multi-scale focusing modulation module between the backbone network and the neck network, improving the multi-scale target perception ability through adaptive multi-scale context modeling and geometric enhancement convolution; adopting a lightweight multi-constraint intersection over union positioning loss function, and through segmented linear enhancement, shape consistency constraint and center offset constraint, combining dynamic weight scheduling to optimize the positioning accuracy. The application effectively solves the problems of feature deviation caused by environmental heterogeneity, multi-scale target detection difficulty and real-time and precision balance.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and more specifically, to a method and apparatus for target detection of aerial unmanned aerial vehicle swarms based on YOLOv7-tiny. Background Technology

[0002] With the rapid development and widespread application of UAV technology, aerial UAV swarm target detection has significant application value in fields such as remote sensing image analysis, low-altitude airspace security, and military reconnaissance and early warning. UAV swarm targets are typically characterized by small size, high speed, and high maneuverability, and often appear in dense formations, posing a severe challenge to target detection. Furthermore, real-world application scenarios often involve complex environmental changes, such as variations in lighting conditions, weather effects, and background interference, further increasing the difficulty of detection.

[0003] Traditional object detection methods mainly rely on hand-designed feature extractors and classifiers, such as Haar features, Histogram of Oriented Gradients (HOG) combined with Support Vector Machines (SVM). These methods are effective in simple scenarios, but their expressive power is limited when facing complex scenarios with large differences in target scale, obvious directionality, and strong environmental heterogeneity in UAV swarms, making it difficult to meet the requirements of high-precision detection.

[0004] In recent years, deep learning-based object detection technology has made significant progress. Among them, the YOLO (You Only Look Once) series of algorithms, with its end-to-end detection framework and excellent real-time performance, has become one of the mainstream methods in the field of object detection. YOLOv7-tiny, as a lightweight version of the YOLO series, significantly reduces the number of model parameters and computational complexity while maintaining high detection accuracy, making it suitable for resource-constrained edge deployment scenarios. However, target detection in aerial UAV swarms still faces the following technical challenges:

[0005] First, there is the issue of feature distribution shift caused by environmental heterogeneity. Target detection in aerial UAV swarms often faces factors such as changes in lighting, weather conditions, and background interference, which cause significant differences in the feature distribution of the same target under different environments, leading to a decrease in the model's generalization ability.

[0006] Secondly, multi-scale target detection is difficult. Targets in UAV swarms vary greatly in size and often appear in dense formations. Small targets are easily missed, and the boundaries of directional targets are blurred. Traditional detection methods struggle to simultaneously detect targets of different scales.

[0007] Third, it is difficult to balance real-time performance and accuracy. Edge deployment scenarios have high requirements for detection speed, but existing high-precision detection methods are often computationally too complex to meet real-time requirements; while lightweight models are fast, their detection accuracy is insufficient in complex scenarios.

[0008] Therefore, there is an urgent need for an aerial UAV swarm target detection method that can achieve high-precision and robust detection in complex environments while meeting the requirements for real-time deployment at the edge. Summary of the Invention

[0009] The purpose of this application is to provide a method and apparatus for target detection of aerial unmanned aerial vehicle (UAV) swarms based on YOLOv7-tiny, which can solve at least one of the aforementioned technical problems. The specific solution is as follows:

[0010] According to a specific embodiment of this application, this application provides a method for target detection of aerial unmanned aerial vehicle swarms based on YOLOv7-tiny, including:

[0011] Build the dataset and preprocess it;

[0012] A YOLOv7-tiny-based network is constructed. A multimodal image-level domain adaptive module is introduced into the backbone network of the network. A multi-scale focusing modulation module is introduced between the backbone network and the neck network. A lightweight multi-constraint cross-union ratio localization loss function is used as the localization loss function.

[0013] The network is trained to obtain an aerial drone swarm target detection model;

[0014] The image to be tested is input into the aerial drone swarm target detection model, and the detection result is output.

[0015] Furthermore, the multimodal image-level domain adaptive module includes:

[0016] Independent modal encoders are used to extract infrared image features and visible light image features separately;

[0017] A modulation mechanism based on feature linear modulation is used to project the visible light image features into scaling parameters and bias parameters, and then apply them to the infrared image features to obtain fused features.

[0018] The adaptive gradient reversal strategy is used to dynamically adjust the gradient reversal coefficients based on the magnitude of the domain classification loss, aligning source domain features with target domain features.

[0019] Furthermore, the modulation mechanism based on characteristic linear modulation includes:

[0020] Global average pooling is performed on the visible light image features to obtain a global feature vector;

[0021] The global feature vector is input into a multilayer perceptron and projected as scaling and bias parameters.

[0022] The scaling parameter is multiplied channel by channel with the infrared image features and then added to the bias parameter to generate a fused feature.

[0023] Furthermore, the adaptive gradient reversal strategy includes:

[0024] Maintain the identity mapping during forward propagation;

[0025] During backpropagation, the gradient is reversed and a scaling factor is applied, which is adaptively adjusted according to the domain classification loss.

[0026] When the domain classification loss is less than a preset threshold, the scaling factor is increased;

[0027] When the domain classification loss is greater than or equal to the preset threshold, the initial scaling factor is maintained.

[0028] Furthermore, the multi-scale focusing modulation module includes:

[0029] The multi-scale context modeling submodule is used to generate multi-scale aggregation weights based on input features and to perform weighted fusion of context features at different scales to generate multi-scale context features.

[0030] The geometrically enhanced convolution branch is used to generate sampling offsets through convolution and to perform geometrically enhanced convolution on the multi-scale context features to output directional enhanced features;

[0031] The cross-scale adaptive fusion submodule is used to integrate features from adjacent levels, generate fusion weights based on global statistical information, and perform weighted fusion on the features from adjacent levels.

[0032] Furthermore, the multi-scale context modeling submodule includes:

[0033] The input features are subjected to global average pooling to obtain a global context vector;

[0034] The global context vector is input into a multilayer perceptron and processed by a normalized exponential function to generate weights for each scale.

[0035] Based on a preset set of scale radii, the input features are aggregated at multiple scales to obtain context features at each scale.

[0036] Multi-scale context features are generated by multiplying the context features at each scale with their corresponding weights and summing the results.

[0037] Furthermore, the geometrically enhanced convolutional branch includes:

[0038] Perform convolution operations on the multi-scale context features to generate sampling offsets;

[0039] Based on the sampling offset, the multi-scale context features are subjected to depthwise separable convolution to output directional enhancement features.

[0040] Furthermore, the lightweight multi-constraint intersection-union ratio (IUU) localization loss function includes:

[0041] Calculate the intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box;

[0042] The cross-union ratio is piecewise linearly enhanced to obtain the enhanced cross-union ratio;

[0043] Calculate the shape consistency constraint term between the predicted bounding box and the true bounding box;

[0044] Calculate the center offset constraint term between the predicted bounding box and the true bounding box;

[0045] Based on the enhanced crossover ratio, the shape consistency constraint, and the center offset constraint, and combined with the dynamic weight scheduling mechanism, the positioning loss value is calculated.

[0046] Furthermore, the dynamic weight scheduling mechanism includes:

[0047] Based on the relationship between the intersection-union ratio and the preset threshold, the weight coefficients of the shape consistency constraint term and the center offset constraint term are dynamically calculated.

[0048] In the early stages of training, increase the weight coefficient of the center offset constraint term to prioritize the optimization of center point alignment;

[0049] In the later stages of training, the weight coefficient of the shape consistency constraint term is increased to prioritize the optimization of shape consistency.

[0050] This application also provides an aerial UAV swarm target detection device based on YOLOv7-tiny, including:

[0051] The data preprocessing module is used to construct the dataset and perform preprocessing.

[0052] The network construction module is used to build a YOLOv7-tiny-based network, introduces a multimodal image-level domain adaptive module into the backbone network, and introduces a multi-scale focusing modulation module between the backbone network and the neck network.

[0053] The model training module is used to train the network using a lightweight multi-constraint intersection-union-ratio localization loss function to obtain an aerial UAV swarm target detection model.

[0054] The target detection module is used to input the image to be tested into the target detection model of the aerial UAV swarm and output the detection results.

[0055] Compared with the prior art, the above-described solutions of this application have at least the following beneficial effects:

[0056] 1. This application discloses a method and apparatus for aerial UAV swarm target detection based on YOLOv7-tiny. By introducing a multimodal image-level domain adaptive module, the method utilizes FiLM feature modulation technology to achieve channel-level adaptive calibration of visible light to infrared features, and combines an adaptive gradient inversion strategy to align the feature distributions of the source and target domains. This effectively solves the feature shift problem caused by environmental heterogeneity such as changes in illumination and temperature, and significantly improves the model's generalization ability and detection robustness in complex environments.

[0057] 2. This application discloses a method and apparatus for aerial UAV swarm target detection based on YOLOv7-tiny. Through adaptive context modeling and geometrically enhanced convolution of the multi-scale focusing modulation module, it achieves accurate perception of multi-scale targets and directional features, effectively overcoming the detection difficulties caused by scale differences and boundary ambiguity, and especially improving the recall rate and positioning accuracy of small targets and fuzzy targets.

[0058] 3. This application discloses a method and apparatus for target detection of aerial UAV swarms based on YOLOv7-tiny. It adopts a lightweight multi-constraint intersection-union ratio (IUU) localization loss function. While completely abandoning high-overhead operators such as square root and square root, it introduces dynamic shape consistency constraints and center offset constraints. The regression process is optimized through a dynamic weight scheduling mechanism. Under the premise of ensuring high-precision localization, the computational complexity is greatly reduced, which is perfectly adapted to the real-time detection requirements of UAV edge or embedded platforms. Attached Figure Description

[0059] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0060] Figure 1 This is a flowchart illustrating an algorithm for a method of aerial unmanned aerial vehicle (UAV) swarm target detection based on YOLOv7-tiny, as shown in an embodiment of this application.

[0061] Figure 2 This is a schematic diagram of the structure of a YOLOv7-tiny network as shown in an embodiment of this application.

[0062] Figure 3 This is a schematic diagram of the structure of the multimodal image-level domain adaptive module shown in an embodiment of this application.

[0063] Figure 4 This is a schematic diagram of the structure of a multi-scale focusing modulation module as shown in an embodiment of this application. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0065] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms "a," "the," and "the" as used in the embodiments of this application and the appended claims are also intended to include the plural forms, and "multiple" generally includes at least two unless the context clearly indicates otherwise.

[0066] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0067] It should be understood that although the terms first, second, third, etc., may be used in the embodiments of this application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, first may also be referred to as second without departing from the scope of the embodiments of this application, and similarly, second may also be referred to as first.

[0068] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or device. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or device that includes that element.

[0069] The optional embodiments of this application are described in detail below with reference to the accompanying drawings.

[0070] like Figure 1 As shown, a method for target detection in aerial UAV swarms based on YOLOv7-tiny includes:

[0071] S1. Construct the dataset and perform preprocessing.

[0072] The dataset constructed in this embodiment is a self-built dataset. Target images are extracted from this dataset, and the samples are divided into training, validation, and test sets in an 8:1:1 ratio. The self-built dataset includes infrared and visible light images for detecting targets in aerial drone swarms. The images are adaptively resized to ensure they are converted to a square format for subsequent processing. Then, an N×N grid system is used to uniformly divide the resized images into local regions, facilitating subsequent target detection and classification.

[0073] S2. Construct a network based on YOLOv7-tiny, introduce a multimodal image-level domain adaptive module into the backbone network, introduce a multi-scale focusing modulation module between the backbone network and the neck network, and use a lightweight multi-constraint cross-union ratio localization loss function as the localization loss function.

[0074] This application's embodiments use YOLOv7-tiny as the basic network framework, such as... Figure 2 As shown, the framework includes a standard backbone network, a neck network, and a YOLO head. The technical improvement in this application's embodiments lies in the modular enhancement of the basic framework, specifically including: introducing a multimodal image-level domain adaptive module (MM-IDA) into the backbone network to achieve cross-modal feature complementarity and domain adaptive alignment; retaining a fast spatial pyramid pooling module (SPPCSPC) at the end of the backbone network to enhance the global receptive field; introducing a multi-scale focus modulation module (MSFM_Block) at the feature concatenation point of the neck network to enhance multi-scale feature perception and directional modeling; and using a lightweight multi-constraint intersection-union localization loss function (LiCIoU) as the localization loss function during model training.

[0075] The backbone network consists of multiple stacked ELAN blocks and convolutional layers (Conv2D) for extracting multi-scale features. The neck network employs a path aggregation network (PANet) structure to achieve multi-scale fusion of the feature pyramid. The detection head is a YOLO Head with three scales, outputting detection results at different resolutions. Each convolutional layer (Conv2D) is followed by batch normalization (BN) and an activation function (LeakyReLU) for extracting non-linear features.

[0076] This application provides a preferred embodiment in which a multimodal image-level domain adaptive module is used to fully utilize the complementary information between infrared and visible light images and reduce inter-domain differences through adversarial learning methods, thereby extracting more robust image features. Figure 2In the middle, an infrared image is input above the Multimodal Image-Level Domain Adaptive Module (MM-IDA), and a visible light image is input below it.

[0077] The backbone network starts with the input image and sequentially passes through multiple convolutional layers and ELAN blocks for downsampling, resulting in three feature maps of different scales, denoted as layers P3, P4, and P5. Layer P3 corresponds to a feature map of 1 / 8 the size of the input image, with a resolution of 52×52, and is used to detect small targets; layer P4 corresponds to a feature map of 1 / 16 the size of the input image, with a resolution of 26×26, and is used to detect medium-sized targets; layer P5 corresponds to a feature map of 1 / 32 the size of the input image, with a resolution of 13×13, and is used to detect large targets.

[0078] At the output of each scale feature map in the backbone network, i.e. after the ELAN Block in layers P3, P4, and P5, a multimodal image-level domain adaptive module (MM-IDA) is introduced. The MM-IDA module is used to fuse features from infrared and visible light images and perform domain adaptive alignment.

[0079] Specifically, MM-IDA is connected after layer P3 (52×52×256), layer P4 (26×26×512), and layer P5 (13×13×1024). Each MM-IDA module receives corresponding scale features from the infrared and visible light branches, achieves cross-modal modulation through a feature-linear modulation (FiLM) mechanism, and performs domain alignment using an adaptive gradient inversion (AdvGRL) strategy. The MM-IDA module is only enabled during the training phase and does not participate in forward computation during the inference phase, thus incurring no additional computational overhead.

[0080] Following the ELAN Block in layer P5, the original Fast Spatial Pyramid Pooling (SPPCSPC) module from YOLOv7-tiny is retained. The SPPCSPC module contains three parallel max pooling branches: 5×5 MaxPooling, 9×9 MaxPooling, and 13×13 MaxPooling. Each branch is preceded by a convolutional normalized activation layer. The outputs of the three branches are concatenated with the original input and then output through a convolutional layer to enhance the global receptive field and contextual information.

[0081] like Figure 3 As shown, the multimodal image-level domain adaptive module specifically includes:

[0082] In this embodiment of the application, infrared image features and visible light image features are first extracted using independent modal encoders, denoted as follows: and .

[0083] in, Indicates infrared image features; An independent encoder representing the infrared mode is used to extract features from infrared images; This represents the input infrared image; Represents visible light image features; An independent encoder representing visible light modes, used to extract features from visible light images; This represents the input visible light image.

[0084] A modulation mechanism based on feature linear modulation is used to project visible light image features into scaling and bias parameters, which are then applied to infrared image features to obtain fused features.

[0085] Subsequently, the visible light image features are projected into scaling and bias parameters using a modulation mechanism based on feature linear modulation, expressed as:

[0086] γ,β=MLP(GAP( ))

[0087] Where γ and β represent scaling and bias parameters, respectively, generated from visible light features by a multilayer perceptron (MLP) and used to modulate infrared features; MLP stands for Multilayer Perceptron; GAP stands for Global Average Pooling.

[0088] Applying scaling and bias parameters to the infrared features yields the fused features, expressed as:

[0089]

[0090] in, Indicates fusion characteristics; This indicates multiplication by channel.

[0091] The technical solution of this application embodiment enables the visible light mode to directly modulate the channel response of the infrared mode, thereby achieving cross-modal complementarity and information enhancement. The fused features are introduced at the output of the Efficient Layer Aggregation Module (ELAN Block) in the backbone network of the YOLOv7-tiny network, and subsequent domain alignment operations are performed at three scales. In this application embodiment, the three scales include: P3 layer, P4 layer, and P5 layer. P3 layer corresponds to a feature map of 1 / 8 the size of the input image, with a resolution of 52×52, used for detecting small targets; P4 layer corresponds to a feature map of 1 / 16 the size, with a resolution of 26×26, used for detecting medium-sized targets; and P5 layer corresponds to a feature map of 1 / 32 the size, with a resolution of 13×13, used for detecting large targets.

[0092] The adaptive gradient reversal strategy is used to dynamically adjust the gradient reversal coefficients based on the magnitude of the domain classification loss, aligning source domain features with target domain features.

[0093] In the domain adaptation part of the technical solution of this application embodiment, such as... Figure 4 As shown, the Multimodal Image-Level Domain Adaptive Module (MM-IDA) continues the principles of Image-Level Domain Adaptive (IDA), primarily consisting of an improved gradient inversion layer (AdvGRL) and a domain classifier. The basic principle is to maintain the identity mapping during forward propagation, while inverting the gradient and applying scaling during backward propagation. The expressions for the identity mapping and scaling are as follows:

[0094]

[0095]

[0096] Here, GRL stands for Gradient Reversal Layer; Represents the input tensor and feature vector; λ represents the identity matrix; λ represents the scaling factor of the gradient inversion layer.

[0097] Unlike gradient inversion with a fixed value, the gradient inversion layer (AdvGRL) uses a domain classification loss. Lc Size adaptive adjustment λ When the classification loss is small, it means that the domain classifier can easily distinguish the source of the sample; in this case, increasing the classification loss is appropriate. λ This forces the feature extractor to work harder to learn domain-invariant features; when the loss is large, the initial scaling factor is maintained. The specific expression is:

[0098]

[0099] in, This represents the scaling factor after adaptive adjustment; This represents the sample difficulty threshold, used to determine whether gradient inversion needs to be enhanced. This indicates the upper limit of the gradient, used to avoid excessive gradient reversal; This represents the initial scaling factor, used when the domain classification loss is large. Lc The domain classification loss value.

[0100] In the technical solution of this application embodiment, the feature extractor and the domain classifier have opposite objectives: the domain classifier tries to minimize the loss to distinguish between the source domain and the target domain, while the feature extractor attempts to confuse the classifier under the drive of the backward gradient, thereby learning a domain-invariant feature representation.

[0101] For fusion features at any scale The classification loss is defined using binary cross-entropy, and its expression is:

[0102]

[0103] Where x represents a fusion feature of arbitrary scale with dimensions C×H×W; C, H, and W represent the channel dimension, spatial height, and spatial width of the feature map, respectively. Represents the real number field, i.e., the values ​​that the elements can take; This represents the domain classification loss of the multimodal image-level domain adaptive module; D (⋅) represents a domain classifier used to determine whether a feature comes from the source domain or the target domain; Let represent the feature vector at the j-th spatial location; y represents the domain label, y=0 indicates that the feature comes from the source domain, and y=1 indicates that it comes from the target domain.

[0104] The domain classification loss of the technical solution in this application embodiment is calculated and accumulated at three scales respectively to ensure that features at different resolutions can obtain domain invariance.

[0105] Ultimately, the total loss function during training consists of the detection loss, the domain classification loss at each scale, and an optional modality consistency regularization term, expressed as:

[0106]

[0107] in, Represents the total loss function; This indicates losses from routine testing; This represents a modal consistency regularization term, used to constrain the consistency between infrared and visible light modes; express Weighting coefficients; l Indicates the first l Domain classification loss at various scales.

[0108] The technical solution of this application embodiment achieves deep fusion and adaptive calibration of cross-modal features in the Multimodal Image-Level Domain Adaptive Module (MM-IDA) through a modulation mechanism based on feature linear modulation, namely, FiLM feature modulation technology. Specifically, FiLM feature modulation technology first performs global average pooling on visible light image features to extract global context information; then, it projects the global context information into scaling and bias parameters through a multilayer perceptron; finally, it multiplies the scaling parameters with the infrared image features channel by channel and adds them to the bias parameters to obtain the fused features. This allows the visible light modality to directly modulate the channel response of the infrared modality in a parameterized manner, thereby achieving complementarity and enhancement of cross-modal information at the feature level.

[0109] The multimodal image-level domain adaptive module (MM-IDA) in this application embodiment is only enabled during the training phase. Therefore, it does not introduce additional computation during the forward propagation of the test image and output of detection results after model training is completed, i.e., it does not introduce additional computation during the inference phase. The MM-IDA addresses the feature distribution shift caused by environmental heterogeneity, such as temperature and illumination changes. Its core feature lies in constructing a cross-modal feature complementarity and adversarial domain alignment mechanism: utilizing FiLM-based feature modulation technology, visible light modes are mapped as parameters to adaptively calibrate infrared features, enhancing the anti-interference capability of the features; simultaneously, combined with an improved gradient inversion strategy (AdvGRL), source and target domain features are adaptively aligned to learn domain-invariant representations. The significant advantage of the MM-IDA is that it achieves robust feature enhancement, and this process is implemented only during the training phase, with zero computational increment during the inference phase, effectively balancing the model's generalization ability and real-time detection efficiency.

[0110] In target detection tasks involving aerial UAV swarms, two main problems are frequently encountered: first, different targets exhibit significant differences in spatial scale, and scale mismatch leads to decreased detection accuracy; second, some targets possess obvious directional characteristics and ambiguous boundary features. Based on this, the technical solution of this application introduces a lightweight multi-scale focusing modulation module (MSFM_Block) between the backbone network and the neck network. Through adaptive multi-scale context modeling, geometrically enhanced convolution, and cross-scale feature fusion, it effectively improves the model's target detection accuracy and robustness.

[0111] The multi-scale feature maps output from the backbone network are fed into the neck network. The neck network adopts a path aggregation network (PANet) structure, which includes top-down upsampling paths and bottom-up downsampling paths. Multi-scale feature fusion is achieved through feature concatenation and convolution operations.

[0112] Following the concatenation (Concat) operation of multiple features in the neck network, a multi-scale focusing modulation module (MSFM_Block) is introduced. Specifically, the MSFM_Block is inserted after the concatenation of the P3, P4, and P5 layers. The MSFM_Block enhances the multi-scale perception and directional modeling capabilities of features through adaptive multi-scale context modeling, geometrically enhanced convolution, and cross-scale adaptive fusion.

[0113] The technical solutions of the embodiments of this application, such as Figure 4As shown, the multi-scale focusing modulation module includes a multi-scale context modeling submodule, a geometrically enhanced convolutional branch, and a cross-scale adaptive fusion submodule, implemented using a complementary feature aggregation module with a coordinate attention mechanism. First, the input features are averaged to extract global context information; then, modulation parameters are generated through a multilayer perceptron; finally, fused features are output through 3×3 convolution and aggregation operations, achieving channel-level adaptive calibration of the visible light mode to the infrared mode.

[0114] The input features are Where B represents the batch size, which is the number of samples processed simultaneously in one forward propagation or training iteration.

[0115] The multi-scale context modeling submodule is used to generate multi-scale aggregation weights based on input features and to perform weighted fusion of context features at different scales to generate multi-scale context features. Specifically, the multi-scale context modeling submodule performs the following operations:

[0116] Global average pooling is performed on the input feature X to obtain the global context vector.

[0117] Define the set of scale radii The global context vector is input into the multilayer perceptron, and after processing by a normalized exponential function, the weights for each scale are generated, expressed as:

[0118]

[0119] in, Represents the set of scale radii; Each represents a radius of multiple scales; This represents the adaptive weights at the k-th scale; Indicates in k The softmax normalization operation performed on the dimension; K represents the total number of scale radii, i.e. the number of different scale radii used in multi-scale context modeling.

[0120] Based on a preset set of scale radii, the input feature X is aggregated at multiple scales to obtain contextual features at each scale, expressed as follows:

[0121]

[0122] in, Indicated by radius Multi-scale contextual features obtained through aggregation; Indicated by radius The aggregation operation performed.

[0123] Multiply the context features at each scale by their corresponding scale weights and sum them to generate multi-scale context features, expressed as:

[0124]

[0125] in, The final multi-scale contextual features are represented by a weighted sum of features at each scale.

[0126] Subsequently, modulation parameters are generated through a channel attention mechanism, expressed as:

[0127]

[0128] Where Y represents the modulated feature; This indicates a two-layer MLP; These represent the scaling parameters and bias parameters generated through the channel attention mechanism, respectively.

[0129] The technical solution of this application generates scaling parameters and bias parameters through a channel attention mechanism, enabling the network to automatically select appropriate context ranges for different regions, thereby taking into account the edge features of targets of different sizes.

[0130] To enhance the model's ability to model directional targets and blurred boundaries, the Multi-Scale Focusing Modulation (MSFM_Block) module introduces a lightweight geometrically enhanced convolution branch within its convolutional branch. This branch generates sampling offsets through convolution and performs geometrically enhanced convolution on the modulated feature Y, outputting directionally enhanced features. Specifically, the geometrically enhanced convolution branch performs the following operations:

[0131] Perform a convolution operation on the modulated feature Y to generate a sampling offset, expressed as:

[0132]

[0133] Where Ψ represents a 3×3 convolution, used to generate sparse sampling offsets and applied to depthwise separable convolution branches; This indicates the sampling offset.

[0134] The technical solution of this application embodiment performs depthwise separable convolution on the modulated feature Y according to the sampling offset, and outputs directional enhancement features. While maintaining low computational overhead, the convolution can be aligned with the directionality of the target, thereby enhancing the model's ability to perceive the target's direction and irregular boundaries.

[0135] The multi-scale focusing modulation module (MSFM_Block) introduces a cross-scale adaptive fusion module to integrate features from adjacent layers. It generates fusion weights based on global statistical information and performs weighted fusion on features from adjacent layers. Let the features of two adjacent layers be... The expression for the fusion process is:

[0136]

[0137] Where s represents the cross-layer fusion weight, which is output by the Sigmoid function; σ represents the Sigmoid function; These represent the features of two adjacent levels; U represents the features after cross-scale fusion.

[0138] The technical solution of this application integrates features from adjacent levels through a cross-scale adaptive fusion module, which can adaptively allocate weights between different scales, highlighting the scale features with the most discriminative power for detection and suppressing redundant information, thereby improving the recall rate of small and blurred targets. The multi-scale focusing modulation module aligns directional features through adaptive context modeling and geometrically enhanced convolution, effectively overcoming scale differences and boundary ambiguity, and significantly improving the model's performance in multi-scale target detection.

[0139] In multimodal fusion tasks, the accuracy of multi-source bounding box regression directly determines the recognition quality of targets in visible light, infrared, and other applications. However, multimodal detection scenarios are typically complex: on the one hand, targets often exhibit heterogeneous cross-domain morphologies, and the aspect ratio deviation between the multi-source bounding box and the labeled bounding box significantly reduces the evaluation effect of IoU; on the other hand, specific targets and their associated regions in visible light or infrared detection images are often small in scale, amplifying the impact of center point deviation on the fusion result, which traditional IoU loss is not sensitive enough to. Furthermore, common IoU improvement methods (such as GIoU, DIoU, CIoU) usually include high-cost operators such as square root, square root, logarithm, and power, which have excessive computational overhead when deployed on UAV edge devices or embedded systems, making it difficult to meet the requirements of high frame rate and low power consumption.

[0140] To address this, the technical solution of this application proposes introducing a multi-scale focusing modulation module between the backbone network and the neck network, and using a lightweight multi-constraint intersection-union-ratio (IoU) localization loss function as the localization loss function. By performing piecewise linear enhancement on the IoU and introducing shape consistency constraints and center offset constraints, the aspect ratio mismatch and small target offset problems in target detection are effectively solved. Simultaneously, LiCIoU avoids using complex operations such as logarithms and square roots at the formula level, retaining only lightweight operators such as addition, subtraction, multiplication, division, absolute value, max, and clip, thus significantly reducing computational complexity while ensuring detection accuracy. More importantly, LiCIoU's dynamic weight scheduling mechanism emphasizes center point alignment in the early stages of regression and highlights shape consistency in the later stages, thereby adaptively optimizing different stages of target detection in aerial UAV swarms.

[0141] The neck network outputs enhanced feature maps at three scales, which are then input into the corresponding YOLO Head detectors. The YOLO Head has three parallel branches, corresponding to scales P3, P4, and P5, with each branch outputting a predicted bounding box, object class, and confidence score. During model training, a lightweight multi-constraint intersection-union (LiCIoU) localization loss function is used.

[0142] In the technical solution of this application embodiment, the lightweight multi-constraint intersection-union ratio (LiCIoU) localization loss function includes:

[0143] Let the predicted bounding box be The true bounding box is The intersection-union ratio (IUR) between the predicted bounding box and the ground truth bounding box is calculated using the following expression:

[0144]

[0145] in, Indicates the predicted bounding box; Represents the true bounding box; These represent the coordinates of the center point of the predicted bounding box; These represent the coordinates of the center point of the actual bounding box; These represent the width and height of the predicted bounding box, respectively. These represent the width and height of the actual bounding box, respectively; This indicates intersection, union, and ratio.

[0146] Piecewise linear enhancement of the cross-union ratio (CUN) yields the enhanced CUN, expressed as:

[0147]

[0148] in, Indicates an enhanced crossover-union ratio; Represents the linear gain coefficient; This indicates the segment threshold.

[0149] The shape consistency constraint between the predicted bounding box and the true bounding box is calculated using the following expression:

[0150]

[0151] Wherein, SC represents the shape consistency constraint.

[0152] The center offset constraint term between the predicted bounding box and the true bounding box is calculated using the following expression:

[0153]

[0154] Where CO represents the center offset constraint term.

[0155] Based on the enhanced crossover ratio, shape consistency constraint, and center offset constraint, and combined with the dynamic weight scheduling mechanism, the positioning loss value is calculated.

[0156] Based on the relationship between the intersection-union ratio and a preset threshold, the weight coefficients of the shape consistency constraint term are dynamically calculated. Weighting coefficients of the center offset constraint term In the early stages of training, increase the weight coefficient of the center offset constraint term to prioritize optimizing center point alignment. In the later stages of training, increase the weight coefficient of the shape consistency constraint term to prioritize optimizing shape consistency. Specifically, the weight coefficients... The dynamic expression is:

[0157]

[0158]

[0159] in, These represent the weighting coefficients of the shape consistency constraint and the center offset constraint, respectively. , These represent the minimum and maximum values ​​of the weight coefficients for the shape consistency constraint, respectively. , These represent the minimum and maximum values ​​of the weight coefficients for the center offset constraint term, respectively. This represents the truncation function.

[0160] Finally, the expression for the lightweight multi-constraint intersection-union ratio (IUU) localization loss is:

[0161]

[0162] in, This represents the lightweight multi-constraint intersection-union ratio (IUU) localization loss value.

[0163] The technical solution of this application embodiment introduces a lightweight multi-constraint intersection-union ratio (IU) localization loss value. While ensuring the effectiveness of geometric constraints, this module avoids complex operators such as square roots, square roots, and logarithms, achieving both lightweight design and high efficiency. Through a dynamic weight scheduling mechanism, the model prioritizes center alignment in the early stages of training and focuses on shape refinement in the later stages, thus achieving an optimal balance between high detection accuracy and real-time inference efficiency in complex detection scenarios.

[0164] S3. Train the network to obtain an aerial drone swarm target detection model.

[0165] The training set images constructed in step S1 are input into the network constructed in step S2, and the prediction results are obtained through forward propagation. The localization loss is calculated using a lightweight multi-constraint intersection-union localization loss function, and multi-task joint optimization is performed by combining the classification loss and confidence loss. The network weights are updated through the backpropagation algorithm, and the optimization is iteratively performed until convergence to obtain the target detection model for aerial UAV swarms.

[0166] S4. Input the image to be tested into the trained aerial drone swarm target detection model, and output the detection results, including:

[0167] The input image to be tested undergoes adaptive resizing and normalization to ensure it is converted to a square format for subsequent processing. Then, an N×N grid system is used to uniformly divide the resized image into blocks, separating complex image scenes into local regions to facilitate subsequent object detection and classification.

[0168] For each N×N grid, if the grid contains the center point of any target, then the grid is activated to perform the following task:

[0169] Generate prediction bounding boxes: The activated grid generates B prediction bounding boxes based on B preset anchor points to cover and detect potential target objects. Each prediction box contains five key parameters: [x, y, w, h, confidence].

[0170] Where (x,y) indicates the relative position of the target center within the grid; (w,h) indicates the width and height dimensions of the predicted bounding box; confidence indicates the degree of overlap between the predicted box and the actual target bounding box, i.e., the intersection-over-union ratio (IoU) and the probability of the target existing within the grid.

[0171] Confidence Calculation: The confidence score (CS) of each predicted bounding box is calculated by combining the probability of the object's presence (Pr(Object)) with the intersection-over-union (IoU) ratio between the predicted and ground truth bounding boxes, i.e., CS = Pr(Object) * IoU. Pr(Object) represents a binary variable indicating whether the grid contains the object's center point. IoU quantifies the degree of matching between the predicted and ground truth bounding boxes.

[0172] To ensure unified processing and transmission, the detection results of each grid are organized into vectors. .in, These represent the coordinates of the target center; ...

[0173] After completing the prediction for all N×N grids, the prediction vectors of all grids are... The results are then aggregated to form the detection results for the entire image. Finally, the location, size, and confidence level information of all detected target objects in the entire image are output.

[0174] The technical solution of this application provides a method for aerial UAV swarm target detection based on YOLOv7-tiny. By introducing a multimodal image-level domain adaptive module and utilizing FiLM modulation and adaptive gradient inversion strategies, it achieves cross-modal feature complementarity and domain adaptive alignment, effectively solving the feature distribution shift problem caused by environmental heterogeneity and enhancing the robustness of the model in complex environments. At the same time, through adaptive context modeling and geometrically enhanced convolution of the multi-scale focusing modulation module, it strengthens the perception capability of multi-scale and directional targets, overcoming the detection difficulties caused by scale differences and boundary ambiguity. In addition, the lightweight multi-constraint intersection-union ratio (IoU) localization loss function abandons high-overhead operators and introduces dynamic geometric constraints, which significantly reduces computational complexity while ensuring high-precision localization, achieving the best balance between detection accuracy and real-time performance.

[0175] This application also provides an aerial drone swarm target detection device based on YOLOv7-tiny, including:

[0176] The data preprocessing module is used to build and preprocess the dataset.

[0177] In this embodiment, the data preprocessing module extracts infrared and visible light images from a self-built dataset and divides the samples into training, validation, and test sets in an 8:1:1 ratio. The data preprocessing module adaptively resizes the input images to ensure they are converted to a square format; it uses an N×N grid system to uniformly divide the adjusted images into local regions, separating complex image scenes; it normalizes pixel values, mapping them from [0,255] to the [0,1] range, and standardizes the color space.

[0178] The network construction module is used to build a YOLOv7-tiny-based network. It introduces a multimodal image-level domain adaptive module into the backbone network and a multi-scale focusing modulation module between the backbone network and the neck network.

[0179] In this embodiment, the multimodal image-level domain adaptive module includes an independent modal encoder, a feature-based linear modulation modulation mechanism, and an adaptive gradient inversion strategy. The independent modal encoder is used to extract infrared image features and visible light image features respectively. The feature-based linear modulation modulation mechanism projects the visible light image features into scaling and bias parameters, which are then applied to the infrared image features to obtain fused features. The adaptive gradient inversion strategy dynamically adjusts the gradient inversion coefficients according to the magnitude of the domain classification loss, aligning the source domain features with the target domain features. The multi-scale focusing modulation module includes a multi-scale context modeling submodule, a geometrically enhanced convolution branch, and a cross-scale adaptive fusion submodule. The multi-scale context modeling submodule generates multi-scale aggregation weights based on the input features and performs weighted fusion of context features at different scales to generate multi-scale context features. The geometrically enhanced convolution branch generates sampling offsets through convolution and performs geometrically enhanced convolution on the multi-scale context features to output directional enhanced features. The cross-scale adaptive fusion submodule integrates features from adjacent levels, generates fusion weights based on global statistical information, and performs weighted fusion of features from adjacent levels.

[0180] The model training module is used to train the network using a lightweight multi-constraint intersection-union-ratio localization loss function to obtain an aerial drone swarm target detection model.

[0181] In this embodiment, the model training module inputs images from the training set into the network and obtains prediction results through forward propagation; it calculates the localization loss using a lightweight multi-constraint intersection-union ratio (IU / R) localization loss function and performs multi-task joint optimization by combining classification loss and confidence loss; it updates the network weights through backpropagation algorithm and iteratively optimizes until convergence. The lightweight multi-constraint IU / R localization loss function includes calculating the IU / R between the predicted bounding box and the ground truth bounding box, performing piecewise linear enhancement on the IU / R, calculating the shape consistency constraint term, calculating the center offset constraint term, and calculating the localization loss value based on the enhanced IU / R, shape consistency constraint term, and center offset constraint term, combined with a dynamic weight scheduling mechanism; the dynamic weight scheduling mechanism dynamically calculates the weight coefficients of the shape consistency constraint term and the center offset constraint term based on the relationship between the IU / R and a preset threshold. In the early stage of training, the weight coefficient of the center offset constraint term is increased to prioritize optimizing center point alignment, and in the later stage of training, the weight coefficient of the shape consistency constraint term is increased to prioritize optimizing shape consistency.

[0182] The target detection module is used to input the image to be tested into the target detection model of the aerial UAV swarm and output the detection results.

[0183] In this embodiment, the target detection module performs adaptive size adjustment and normalization on the image to be tested, converting it into a square format; the converted image is evenly divided into N×N grids; for each grid containing the target center point, B predicted bounding boxes are generated, each predicted bounding box containing position parameters, size parameters, and confidence scores; the prediction results of all grids are summarized, and the position, size, and confidence scores of all detected targets in the image are output.

[0184] This application discloses an aerial UAV swarm target detection device based on YOLOv7-tiny. It adopts a modular design, with data preprocessing, network construction, model training, and target detection modules working collaboratively to achieve an end-to-end UAV swarm target detection process. The multimodal fusion and feature enhancement modules introduced in the network construction module are only enabled during the training phase, with zero computational increment during the inference phase, perfectly adapting to the computing power limitations of embedded platforms on the UAV side. While maintaining the lightweight advantages of YOLOv7-tiny, the overall system significantly improves detection performance in complex environments such as fog and changing lighting conditions. It features high precision, high robustness, low power consumption, and easy deployment, making it suitable for various application scenarios such as remote sensing image analysis, intelligent security monitoring, and unmanned driving environmental perception.

[0185] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0186] The units described in the embodiments of this application can be implemented in software or hardware. The names of the units are not, in some cases, limiting the scope of the unit itself.

Claims

1. A method for detecting aerial unmanned vehicle group targets based on YOLOv7-tiny, characterized in that, include: Build the dataset and perform preprocessing; A YOLOv7-tiny-based network is constructed. A multimodal image-level domain adaptive module is introduced into the backbone network of the network. A multi-scale focusing modulation module is introduced between the backbone network and the neck network. A lightweight multi-constraint cross-union ratio localization loss function is used as the localization loss function. The network is trained to obtain an aerial drone swarm target detection model; The image to be tested is input into the aerial drone swarm target detection model, and the detection result is output. The multimodal image-level domain adaptive module includes: Independent modal encoders are used to extract infrared image features and visible light image features separately; A modulation mechanism based on feature linear modulation is used to project the visible light image features into scaling parameters and bias parameters, and then apply them to the infrared image features to obtain fused features. An adaptive gradient reversal strategy is used to dynamically adjust the gradient reversal coefficients based on the magnitude of the domain classification loss, aligning source domain features with target domain features. The adaptive gradient reversal strategy includes: Maintain identity mapping during forward propagation; During backpropagation, the gradient is reversed and a scaling factor is applied, which is adaptively adjusted according to the domain classification loss. When the domain classification loss is less than a preset threshold, the scaling factor is increased; When the domain classification loss is greater than or equal to the preset threshold, the initial scaling factor is maintained.

2. The method of claim 1, wherein, The modulation mechanism based on characteristic linear modulation includes: Global average pooling is performed on the visible light image features to obtain a global feature vector; The global feature vector is input into a multilayer perceptron and projected as scaling and bias parameters. The scaling parameter is multiplied channel by channel with the infrared image features and then added to the bias parameter to generate a fused feature.

3. The method of claim 1, wherein, The multi-scale focusing modulation module includes: The multi-scale context modeling submodule is used to generate multi-scale aggregation weights based on input features and to perform weighted fusion of context features at different scales to generate multi-scale context features. The geometrically enhanced convolution branch is used to generate sampling offsets through convolution and to perform geometrically enhanced convolution on the multi-scale context features to output directional enhanced features; The cross-scale adaptive fusion submodule is used to integrate features from adjacent levels, generate fusion weights based on global statistical information, and perform weighted fusion on the features from adjacent levels.

4. The method of claim 3, wherein, The multi-scale context modeling submodule includes: The input features are subjected to global average pooling to obtain a global context vector; The global context vector is input into a multilayer perceptron and processed by a normalized exponential function to generate weights for each scale. Based on a preset set of scale radii, the input features are aggregated at multiple scales to obtain context features at each scale. Multi-scale context features are generated by multiplying the context features at each scale with their corresponding weights and summing the results.

5. The method of claim 3, wherein, The geometrically enhanced convolutional branch includes: Perform convolution operations on the multi-scale context features to generate sampling offsets; Based on the sampling offset, the multi-scale context features are subjected to depthwise separable convolution to output directional enhancement features.

6. The method of claim 1, wherein, The lightweight multi-constraint intersection-union ratio (IoU) localization loss function includes: Calculate the intersection-union ratio (IUU) between the predicted bounding box and the ground truth bounding box; The cross-union ratio is piecewise linearly enhanced to obtain the enhanced cross-union ratio; Calculate the shape consistency constraint term between the predicted bounding box and the true bounding box; Calculate the center offset constraint term between the predicted bounding box and the true bounding box; Based on the enhanced crossover ratio, the shape consistency constraint, and the center offset constraint, and combined with the dynamic weight scheduling mechanism, the positioning loss value is calculated.

7. The method of claim 6, wherein, The dynamic weight scheduling mechanism includes: Based on the relationship between the intersection-union ratio and the preset threshold, the weight coefficients of the shape consistency constraint term and the center offset constraint term are dynamically calculated. In the early stages of training, increase the weight coefficient of the center offset constraint term to prioritize the optimization of center point alignment; In the later stages of training, the weight coefficient of the shape consistency constraint term is increased to prioritize the optimization of shape consistency.

8. An aerial unmanned aerial vehicle swarm target detection device based on YOLOv7-tiny for implementing the method as described in any one of claims 1-7, characterized in that, include: The data preprocessing module is used to construct the dataset and perform preprocessing. The network construction module is used to build a YOLOv7-tiny-based network, introduces a multimodal image-level domain adaptive module into the backbone network, and introduces a multi-scale focusing modulation module between the backbone network and the neck network. The model training module is used to train the network using a lightweight multi-constraint intersection-union-ratio localization loss function to obtain an aerial UAV swarm target detection model. The target detection module is used to input the image to be tested into the target detection model of the aerial UAV swarm and output the detection results.

Images