Unmanned aerial vehicle small target detection method based on multi-channel domain generalization network

Abstract

The invention discloses an unmanned aerial vehicle small target detection method based on a multi-channel-domain generalization network, and relates to the technical field of computer vision, image processing and target detection, and the method comprises the steps: constructing and training the multi-channel-domain generalization network; inputting a low-resolution image containing the small target of the unmanned aerial vehicle into the trained multi-channel-domain generalization network for detection and identification; the multi-channel domain generalization network comprises a high-resolution target detection network, a low-resolution target detection network, a low-resolution domain generalization network and a discriminator, and the high-resolution target detection network is connected with the low-resolution target detection network through strong constraint; and the low-resolution target detection network and the low-resolution domain generalization network are connected through confrontation constraint. According to the method, the small target can be detected more accurately under different environments and change conditions, and high performance can be ensured to be kept under any background or condition.

Description

Technical Field

[0001] The present invention relates to the technical fields of computer vision, image processing and target detection, and in particular to a method for detecting small targets of unmanned aerial vehicles based on a multi-channel domain generalization network. Background Art

[0002] Drone target detection is a key branch in the field of computer vision. With the rapid development of drone technology, its application areas are also expanding, including military, civil and commercial purposes. As a result, potential security threats from drones have also begun to emerge, such as illegal intrusion and malicious attacks. The main purpose of drone target detection is to identify, locate and classify drones in images or video materials. At present, this technology has been widely used in many scenarios in daily life, such as airport security, sensitive area protection and video surveillance systems.

[0003] Early object detection techniques mainly relied on hand-crafted features and machine learning classifiers. Classic methods, such as the Viola-Jones algorithm using Haar features and methods based on HOG (Histogram of Oriented Gradients) features, performed poorly when dealing with complex backgrounds, multi-target recognition, and size changes. With the advancement of computer vision, the introduction of deep learning technology marked the emergence of new algorithms. R-CNN (Regions with CNN features) is a pioneering work that locates objects in images through region proposals and uses convolutional neural networks (CNNs) to extract features and classify them. Its subsequent improved version, Fast R-CNN, significantly improved speed and efficiency by integrating the detection process into a single network and introducing RoI (Region of Interest) pooling layers to extract features from shared feature maps. On this basis, one-stage object detectors such as YOLO (You Only Look Once) and SSD (Single Shot MultiBox Detector) further revolutionized the field. They directly predicted the category and location of the target through a single forward propagation process, greatly accelerating the detection speed. The latest YOLO-v8 algorithm is further developed in this series, which not only maintains the high efficiency of the YOLO series, but also significantly improves its accuracy and generalization ability, making it perform better in complex real-time detection scenarios.

[0004] In object detection, small object detection is a particularly challenging task, mainly because small objects occupy fewer pixels in the image and lack feature information. To complicate matters further, when the data for these small objects comes from different environments, there will be domain differences between the datasets, that is, different distribution characteristics. This domain difference may be caused by different acquisition conditions (such as lighting, background complexity), different data acquisition equipment, or different image processing techniques. These differences cause an object detection model that performs well in one environment to perform poorly in another environment, because the model may not be able to effectively extract universal features from the image that are applicable to all environments. This performance degradation caused by domain differences is particularly evident when dealing with small objects, because the recognizable features of these objects are limited. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a UAV small target detection method based on a multi-channel domain generalization network to improve the detection accuracy and robustness of small targets.

[0006] The objective of the present invention is achieved through the following technical solutions: A method for detecting small targets of unmanned aerial vehicles based on a multi-channel domain generalization network, comprising: Build and train a multi-channel domain generalization network; Input the low-resolution image containing the small drone target into the trained multi-channel domain generalization network for detection and recognition; The construction and training of a multi-channel domain generalization network comprises the following steps: Collect high-resolution images of drones as the first training image set, downsample the high-resolution images of drones by ten times to obtain images as the second training image set, and use low-resolution images as the third training image set; Constructing a multi-channel domain generalization network, the multi-channel domain generalization network includes a high-resolution target detection network, a low-resolution target detection network, a low-resolution domain generalization network, and a discriminator, the high-resolution target detection network is connected to the low-resolution target detection network through a strong constraint, and the low-resolution target detection network is connected to the low-resolution domain generalization network through an adversarial constraint; The high-resolution target detection network adopts the YOLO-v8 network. During model training, the first training image set is input into the high-resolution target detection network, and the first intermediate feature is obtained through the intermediate layer of the high-resolution target detection network. The low-resolution object detection network adopts the YOLO-v8 network. During model training, the second training image set is input into the low-resolution object detection network, and the second intermediate feature is obtained through the intermediate layer of the low-resolution object detection network. The first intermediate feature and the second intermediate feature are strongly constrained, the loss used for the strong constraint is feature alignment loss, and mean square error loss is used as the feature alignment loss; The low-resolution domain generalization network adopts the YOLO-v8 network. During model training, the third training image set is input into the low-resolution domain generalization network, and the third intermediate feature is obtained through the intermediate layer of the low-resolution domain generalization network; the low-resolution domain generalization network performs adversarial learning based on the third intermediate feature and the second intermediate feature, and then the intermediate layer of the low-resolution target detection network and the intermediate layer of the low-resolution domain generalization network are connected to the discriminator, the discriminator evaluates whether the low-resolution target detection network can effectively learn the domain features of the low-resolution domain generalization network, and outputs the discrimination result; the loss used in the training of the discriminator is the domain generalization loss.

[0007] Furthermore, the mean square error loss is used as the feature alignment loss, which is specifically expressed as: in Indicates the actual value, represents the predicted value, N is the sample size.

[0008] Furthermore, the calculation formula of the domain generalization loss is specifically expressed as: Where D is the discriminator, θ is the low-resolution target detection network parameter, It is an adversarial loss function used to measure the ability of the discriminator D to distinguish the feature source domain. Denotes the network parameter θ in the discriminator D to maximize the adversarial loss function In the case of minimizing the adversarial loss function process; x Represents the image features extracted from the low-resolution domain generalization network, representing the distribution of the data; is the output of the low-resolution object detection network, and They represent the data distribution of the low-resolution domain generalization network and the low-resolution target detection network parameters respectively, and E represents the mathematical expectation operation of the original multi-domain training data distribution.

[0009] Furthermore, the YOLO-v8 network includes three parts: Backbone, Neck and Head. The Backbone is used for feature extraction, the Neck is used for multi-scale feature fusion, the Head is used for target detection and classification tasks, and the Head part includes a detection head and a classification head. The classification loss function of the YOLO-v8 network uses binary cross entropy loss: Where N is the number of samples, is the true label of the i-th sample, is the probability that the i-th sample predicted by the model is a positive class; The regression loss function of the YOLO-v8 network uses DFL loss: in S i and S i+1 Model output or data representing two different states or points in time; k true is the true value of the regression target, k i and k i+1 For continuous values k true After discretization, the adjacent interval endpoints satisfy k i ≤ k true ≤ k i+1 .

[0010] Furthermore, the low-resolution image containing the small target of the drone is input into the trained multi-channel domain generalization network for detection and recognition. The specific process is as follows: The low-resolution image to be identified is input into the low-resolution target detection network. The Backbone part of the trained YOLO-v8 network extracts large-scale features. The Neck part fuses the feature maps from different stages of the Backbone. The Head part outputs the predicted bounding box of the small target of the drone.

[0011] Furthermore, the total loss of the multi-channel domain generalization network is: in, They are feature alignment losses , domain generalization loss , classification loss and regression loss The weight factor of .

[0012] The beneficial effects of the present invention are: The present invention uses a high-resolution network to strongly constrain a low-resolution network, thereby providing the low-resolution network with more high-resolution information features and improving the low-resolution network's ability to perceive small targets. At the same time, the network's domain generalization ability for small target features is enhanced by adversarial learning with a low-resolution domain generalization network that inputs low-resolution data sets of different scenes. In this way, the network can detect small targets more accurately under different environments and changing conditions, ensuring that high performance can be maintained regardless of any background or conditions, thereby solving common problems in small target detection, such as challenges caused by insufficient feature information and changes between domains. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 A diagram of a multi-channel domain generalized network architecture provided by an embodiment of the present invention; Figure 2 This is the YOLO-v8 network structure diagram; Figure 3 This is the structure diagram of the discriminator network. DETAILED DESCRIPTION

[0014] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0015] See also Figure 1-Figure 3 , the present invention provides a technical solution: A method for detecting small targets of unmanned aerial vehicles based on a multi-channel domain generalization network, comprising: Step S1: construct and train a multi-channel domain generalization network; Step S2: Input the low-resolution image containing the small target of the drone into the trained multi-channel domain generalization network for detection and recognition; The step S1 constructs and trains a multi-channel domain generalization network, including the following steps: Collect high-resolution images of drones as the first training image set, downsample the high-resolution images of drones by ten times to obtain images as the second training image set, and use low-resolution images as the third training image set; Construct a multi-channel domain generalization network, such as Figure 1 As shown, the multi-channel domain generalization network includes a high-resolution target detection network, a low-resolution target detection network, a low-resolution domain generalization network, and a discriminator. The high-resolution target detection network is connected to the low-resolution target detection network through a strong constraint, and the low-resolution target detection network is connected to the low-resolution domain generalization network through an adversarial constraint. The high-resolution target detection network adopts the YOLO-v8 network. During model training, the first training image set is input into the high-resolution target detection network, and the first intermediate feature is obtained through the intermediate layer of the high-resolution target detection network. The low-resolution object detection network adopts the YOLO-v8 network. During model training, the second training image set is input into the low-resolution object detection network, and the second intermediate feature is obtained through the intermediate layer of the low-resolution object detection network. The first intermediate feature and the second intermediate feature are strongly constrained. The loss used for the strong constraint is the feature alignment loss. The mean square error loss is used as the feature alignment loss. Specifically, it is expressed as: in Indicates the actual value, represents the predicted value, N is the number of samples. The mean squared error loss quantifies the error of the model prediction by taking the average of the sum of squares of the differences between the predicted values ​​and the actual values.

[0016] The low-resolution object detection network, the object detection channel uses ten times downsampled images from the same dataset as the high-resolution object detection network, and strongly constrains the features obtained from the intermediate layers of the high-resolution object detection network, so that the intermediate layers that are more sensitive to small object detection can learn the high-resolution features of the high-resolution object detection network, thereby improving the accuracy of small object detection.

[0017] These three channels are connected through a specific mechanism - strong constraints and adversarial constraints. This setting enables each channel to optimize the overall feature extraction and information fusion process through mutual learning while maintaining its own independent processing features.

[0018] The low-resolution domain generalization network adopts the YOLO-v8 network. During model training, the third training image set is input into the low-resolution domain generalization network, and the third intermediate feature is obtained through the intermediate layer of the low-resolution domain generalization network; the low-resolution domain generalization network performs adversarial learning based on the third intermediate feature and the second intermediate feature, and then the intermediate layer of the low-resolution target detection network and the intermediate layer of the low-resolution domain generalization network are connected to the discriminator. The discriminator evaluates whether the low-resolution target detection network can effectively learn the domain features of the low-resolution domain generalization network, and outputs the discrimination result. The discriminator network structure diagram is shown in FIG. Figure 3 As shown in Figure 2, through the feedback of the discriminator, the low-resolution object detection network can continuously adjust and optimize its parameters to more accurately adapt to and simulate the performance of the feature domain of the low-resolution domain generalization network.

[0019] The loss used in discriminator training is domain generalization loss, and the calculation formula is specifically expressed as: Where D is the discriminator, θ is the low-resolution target detection network parameter, It is an adversarial loss function used to measure the ability of the discriminator D to distinguish the feature source domain. Denotes the network parameter θ in the discriminator D to maximize the adversarial loss function In the case of minimizing the adversarial loss function process; x Represents the image features extracted from the low-resolution domain generalization network, representing the distribution of the data; is the output of the low-resolution object detection network, and They represent the data distribution of the low-resolution domain generalization network and the low-resolution target detection network parameters respectively, and E represents the mathematical expectation operation of the original multi-domain training data distribution.

[0020] In this configuration, the discriminator D is trained to maximize its ability to distinguish between the two types of samples, while the parameters of the low-resolution object detection network By minimizing the recognition ability of the discriminator's output, the purpose is to make the domain of the output feature image of the low-resolution object detection network better imitate the domain of the feature image of the low-resolution domain generalization network. Through adversarial learning, the domain adaptability of the low-resolution object detection network to the low-resolution domain generalization network is continuously improved, thereby improving the domain generalization ability of the object detection network.

[0021] like Figure 2 , the YOLO-v8 network includes three parts: Backbone, Neck and Head. The Backbone is used for feature extraction, the Neck is used for multi-scale feature fusion, the Head is used for target detection and classification tasks, and the Head part includes a detection head and a classification head; The classification loss function of the YOLO-v8 network uses binary cross entropy loss: Where N is the number of samples, is the true label of the i-th sample, is the probability that the i-th sample predicted by the model is a positive class; The regression loss function of the YOLO-v8 network uses DFL loss: in S i and S i+1 Model output or data representing two different states or points in time; k true is the true value of the regression target,k i and k i+1 For continuous values k true After discretization, the adjacent interval endpoints satisfy k i ≤ k true ≤ k i+1 .

[0022] The high-resolution object detection network mainly processes high-resolution images to extract large-scale features. In this channel, the model can identify objects in a large range, including some smaller targets, through convolutional layers and multi-scale feature extraction techniques, because high-resolution images provide enough details to support complex feature recognition. The low-resolution object detection network uses the same dataset as the high-resolution object detection network but performs a significant downsampling process. This design aims to enhance the network's sensitivity to small target detection through lower-resolution images. The feature alignment loss is used to make the features of this channel close to the features of the high-resolution object detection network, achieving effective fusion of the two features, reducing the performance degradation caused by information loss between high and low resolutions, and improving the recognition ability of small targets.

[0023] Furthermore, the total loss of the multi-channel domain generalization network is: in, They are feature alignment losses , domain generalization loss , classification loss and regression loss In this embodiment, under the default configuration, the value range of each weight factor is 0.1≤ α,β,η,μ ≤2.0, for the drone small target detection scenario, the initial value is set to α =1.0, β =0.5, η =0.8, μ =1.2 to enhance feature alignment and regression accuracy.

[0024] The output of the entire network is calculated by combining the YOLO-v8 target detection result loss, feature alignment loss, and domain generalization loss in the above proportions. This loss combination method not only improves the accuracy of target detection, but also strengthens the feature fusion and domain generalization capabilities between different resolutions, ensuring the high performance and robustness of the network in different environments.

[0025] The step S2 inputs the low-resolution image containing the small drone target into the trained multi-channel domain generalization network for detection and recognition. The specific process is as follows: The low-resolution image to be identified is input into the low-resolution target detection network. The Backbone part of the trained YOLO-v8 network extracts large-scale features. The Neck part fuses the feature maps from different stages of the Backbone. The Head part outputs the predicted bounding box of the small target of the drone.

[0026] The core advantage of the present invention is that it uses a high-resolution network to strongly constrain a low-resolution network, thereby providing the low-resolution network with more high-resolution information features and improving the low-resolution network's ability to perceive small targets. At the same time, the network's domain generalization ability for small target features is enhanced by adversarial learning with a low-resolution domain generalization network that inputs low-resolution data sets of different scenes. In this way, the network can detect small targets more accurately under different environments and changing conditions, ensuring that high performance can be maintained regardless of any background or conditions, thereby solving common problems in small target detection, such as challenges caused by insufficient feature information and changes between domains.

[0027] The above is only a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the form disclosed herein, and should not be regarded as excluding other embodiments, but can be used in various other combinations, modifications and environments, and can be modified within the scope of the concept described herein through the above teachings or the technology or knowledge of the relevant field. The changes and modifications made by those skilled in the art shall not deviate from the spirit and scope of the present invention, and shall be within the scope of protection of the claims attached to the present invention.

Claims

1. A method for detecting small drone targets based on a multi-channel domain generalization network, characterized in that: include: Build and train a multi-channel domain generalization network; Input the low-resolution image containing the small drone target into the trained multi-channel domain generalization network for detection and recognition; The construction and training of a multi-channel domain generalization network comprises the following steps: Collect high-resolution images of drones as the first training image set, downsample the high-resolution images of drones by ten times to obtain images as the second training image set, and use low-resolution images as the third training image set; Constructing a multi-channel domain generalization network, the multi-channel domain generalization network includes a high-resolution target detection network, a low-resolution target detection network, a low-resolution domain generalization network, and a discriminator, the high-resolution target detection network is connected to the low-resolution target detection network through a strong constraint, and the low-resolution target detection network is connected to the low-resolution domain generalization network through an adversarial constraint; The high-resolution target detection network adopts the YOLO-v8 network. During model training, the first training image set is input into the high-resolution target detection network, and the first intermediate feature is obtained through the intermediate layer of the high-resolution target detection network. The low-resolution object detection network adopts the YOLO-v8 network. During model training, the second training image set is input into the low-resolution object detection network, and the second intermediate feature is obtained through the intermediate layer of the low-resolution object detection network. The first intermediate feature and the second intermediate feature are strongly constrained, the loss used for the strong constraint is feature alignment loss, and mean square error loss is used as the feature alignment loss; The low-resolution domain generalization network adopts a YOLO-v8 network. During model training, the third training image set is input into the low-resolution domain generalization network, and a third intermediate feature is obtained through an intermediate layer of the low-resolution domain generalization network. The low-resolution domain generalization network performs adversarial learning based on the third intermediate feature and the second intermediate feature, and then connects the intermediate layer of the low-resolution object detection network and the intermediate layer of the low-resolution domain generalization network to the discriminator. The discriminator evaluates whether the low-resolution object detection network can effectively learn the domain features of the low-resolution domain generalization network and outputs a discrimination result. The loss used in training the discriminator is the domain generalization loss.

2. According to claim 1, a method for detecting small targets of unmanned aerial vehicles based on a multi-channel domain generalization network is characterized in that: The mean square error loss is used as the feature alignment loss, which is specifically expressed as: in Indicates the actual value, represents the predicted value, N is the sample size.

3. The method for detecting small drone targets based on a multi-channel domain generalization network according to claim 1 is characterized in that: The calculation formula of the domain generalization loss is specifically expressed as: Where D is the discriminator, θ is the low-resolution target detection network parameter, It is an adversarial loss function used to measure the ability of the discriminator D to distinguish the feature source domain. Denotes the network parameter θ in the discriminator D to maximize the adversarial loss function In the case of minimizing the adversarial loss function process; x Represents the image features extracted from the low-resolution domain generalization network, representing the distribution of the data; is the output of the low-resolution object detection network, and They represent the data distribution of the low-resolution domain generalization network and the low-resolution target detection network parameters respectively, and E represents the mathematical expectation operation of the original multi-domain training data distribution.

4. The method for detecting small drone targets based on a multi-channel domain generalization network according to claim 1 is characterized in that: The YOLO-v8 network includes three parts: Backbone, Neck and Head. The Backbone is used for feature extraction, the Neck is used for multi-scale feature fusion, and the Head is used for target detection and classification tasks. The Head part includes a detection head and a classification head. The classification loss function of the YOLO-v8 network uses binary cross entropy loss: Where N is the number of samples, is the true label of the i-th sample, is the probability that the i-th sample predicted by the model is a positive class; The regression loss function of the YOLO-v8 network uses DFL loss: in S i and S i+1 Model output or data representing two different states or points in time; k true is the true value of the regression target, k i and k i+1 For continuous values k true After discretization, the adjacent interval endpoints satisfy k i ≤ k true ≤ k i+1 .

5. The method for detecting small targets of unmanned aerial vehicles based on a multi-channel domain generalization network according to claim 4 is characterized in that: The low-resolution image containing the small target of the drone is input into the trained multi-channel domain generalization network for detection and recognition. The specific process is as follows: The low-resolution image to be identified is input into the low-resolution target detection network. The Backbone part of the trained YOLO-v8 network extracts large-scale features. The Neck part fuses the feature maps from different stages of the Backbone. The Head part outputs the predicted bounding box of the small target of the drone.

6. The method for detecting small drone targets based on a multi-channel domain generalization network according to claim 1, characterized in that: The total loss of the multi-channel domain generalization network is: in, They are feature alignment losses , domain generalization loss , classification loss and regression loss The weight factor of .

Images