A method for detecting and positioning unmanned aerial vehicle targets

By building jungle camouflage data sets and enhancing the cross-attention of identification application scenarios, and combining deep learning structures to detect and locate drones, the problem of drone target detection in complex environments is solved, and the accuracy and real-time efficiency of camouflage target detection are improved. .

CN117011724BActive Publication Date: 2024-12-03NAT UNIV OF DEFENSE TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202310579091.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-12-03
Estimated Expiration
2043-05-22

AI Technical Summary

Technical Problem

In complex environments, the drone target detection method has limited effect, especially when the ambient light illumination is weak and the target texture is close, it is difficult for the target to be accurately detected and positioned.

Method used

A drone target detection and positioning method is adopted to build a jungle camouflage data set, enhance the cross-discovery attention of identifying application scenarios and data, use deep learning structure to detect target image accuracy, and in the drone target detection and positioning system Solve the target spatial location and avoid using pre-trained models to maintain consistency in the comparison experiment.

Benefits of technology

Improve the drone camouflage target detection performance in complex environments, achieve high real-time efficiency, and the jungle camouflage target detection accuracy reaches 70.2%, which is 5.7% higher than the native method, and the drone flight altitude is ≥12 The error of the target positioning at meters is less than 2 meters, and the average operating efficiency is 29.4fps.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117011724B_ABST
    Figure CN117011724B_ABST
Patent Text Reader

Abstract

A method for detecting and positioning a drone target belongs to the technical field of drone target detection, including S1 setting an application scenario and constructing a jungle camouflage data set; S2 enhancing the cross-attention between identifying application scenarios and acquiring data; S3 acquiring the target image position and solving its spatial position; S4 verifying the result; wherein, S2 is embedded in a deep learning structure for target image accuracy detection; S3 solves the target spatial position according to the construction of a drone target detection and positioning system; S4 is set to not input a pre-trained model to maintain the consistency of the comparative experiment. The method solves the problem of drone visual detection and positioning in complex environments, improves the detection performance of drone camouflage targets, and has high real-time efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of unmanned aerial vehicle target detection and relates to a method for detecting and positioning unmanned aerial vehicle targets. Background Art

[0002] With the continuous development of artificial intelligence technology, the autonomous capabilities of drones have broad research potential, among which the precise perception capabilities of drones in complex environments have received extensive attention. In order to achieve precise environmental perception, drones usually carry payloads such as visible light, infrared, and laser ranging to collect data and detect and locate environmental targets; in some scenes with weak ambient illumination, close environment / target textures, and low target signal-to-noise ratio, targets are often hidden, camouflaged, and difficult to distinguish, which puts higher requirements on drone target detection algorithms.

[0003] At present, the target detection method of UAV directly adopts the general scene target detection and recognition method, and the target detection effect is very limited in complex environments. In the existing technology, for the positioning of UAV targets, the ground target positioning combined with visual tracking and laser ranging or the single-machine ground target positioning based on pure vision is mainly adopted. The ground target positioning combined with visual tracking and laser ranging has high accuracy, but the process is complicated and has high requirements for the unmanned platform carrier, synchronous control, etc.; the single-machine ground target positioning based on pure vision is simple to operate, but the accuracy is limited. The ground target positioning of a single UAV based on pure vision still needs to be verified in practice. Summary of the invention

[0004] To achieve the above objectives, the present invention provides a method for detecting and positioning drone targets, which solves the problem of visual detection and positioning of drones in complex environments, improves the detection performance of camouflaged drone targets, and has high real-time efficiency.

[0005] The technical solution adopted by the present invention is:

[0006] A first aspect of an embodiment of the present invention provides a method for detecting and positioning a target of a drone, comprising: S1 setting an application scenario and constructing a jungle camouflage data set; S2 enhancing the cross-attention between identifying the application scenario and acquiring data; S3 acquiring the position of a target image and solving its spatial position; S4 verifying the result; wherein S2 is embedded in a deep learning structure for detecting the accuracy of the target image; S3 solves the spatial position of the target based on building a drone target detection and positioning system; S4 is set not to input a pre-trained model to maintain the consistency of the comparative experiment.

[0007] In one of the embodiments, S1 includes: constructing target detection data and constructing target positioning data; constructing target detection data includes setting a jungle background and placing the target to be measured, and shooting at different heights and angles through a drone; constructing target positioning data includes the attitude data of the drone and the position data of the target, and a positioning module with a wireless transmission function is installed on the target end. The position data of the target collected by the drone will be transmitted to the PC end in real time by the installed module for recording, and finally the data recorded by the drone and the real data of the target are aligned through the timestamp.

[0008] In one embodiment, S2 includes setting an attention module for cross-enhancing convolutional features of different scales and extracting boundary data of weakly textured targets; the attention module is used for channel attention enhancement according to the attention mechanism and multi-scale fusion method, calculates channel spatial attention, and cross-enhances channel branch data, and after obtaining new features, calculates the channels in sequence and then outputs them; and outputs new features according to the deep learning structure.

[0009] In one of the embodiments, S2 also includes deep learning of the residual module, which is combined with the attention mechanism and the multi-scale fusion method for target feature saliency detection under weak contrast conditions.

[0010] In one embodiment, S2 also includes: the attention module includes a channel attention module and a spatial attention module; wherein the channel attention module downsamples the feature dimension from [b, c, w, h] to [b, c, 1, 1] based on average pooling in the channel dimension, and then transforms the channel dimension through at least one of downsampling, upsampling and activation function; the spatial attention module downsamples the feature dimension from [b, c, w, h] to two independent convolution features of dimension [b, 1, w, h] based on maximum pooling and average pooling in the spatial dimension, and then concatenates the two independent convolution features in the channel dimension to generate a convolution feature of [b, 2, w, h], and then transforms the dimension through two-dimensional convolution, activation function, etc. to generate a convolution feature of [b, 1, w, h].

[0011] In one embodiment, S2 further includes: an instance segmentation algorithm as a basic network for drone target detection; wherein the instance segmentation algorithm includes any one or more of a backbone network, an FPN fusion network, and a Header structure. The backbone network is used to extract image features; the FPN fusion network is used to fuse multiple different scale outputs of the backbone network; and the Header structure is used to output masks and category predictions related to a preset task.

[0012] In one of the embodiments, S3 includes visual detection of ground targets by a UAV, and the visual detection of ground targets by a UAV includes ground target detection and ground target positioning; ground target detection is used to obtain image coordinates of the target, and ground target positioning is used to convert the image coordinates into camera coordinates.

[0013] In one embodiment, S4 includes verifying the validity of the attention module and setting different module embedding methods.

[0014] The beneficial effects of the present invention are: setting application scenarios through S1 and constructing a jungle camouflage data set; S2 enhancing the cross-attention between identifying application scenarios and acquiring data; S3 acquiring the target image position and solving its spatial position; S4 verifying the results. Among them, S2 is embedded in the deep learning structure for target image accuracy detection; S3 solves the target spatial position based on the construction of the drone target detection and positioning system; S4 is set to not input the pre-training model to maintain the consistency of the comparative experiment. It solves the problem of drone visual detection and positioning in complex environments, improves the drone camouflage target detection performance, and has high real-time efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A flowchart of a method for detecting and positioning a drone target is provided according to an embodiment of the present invention;

[0017] Figure 2 This is a flow chart of a UAV ground camouflage target detection platform provided by an embodiment of the present invention;

[0018] Figure 3 An embodiment of the present invention provides a flow chart of UAV ground target positioning;

[0019] Figure 4 It is a ground camouflage target data set diagram provided by an embodiment of the present invention;

[0020] Figure 5 It is a statistical graph of the number of various types of targets in a data set provided by an embodiment of the present invention;

[0021] Figure 6 is a multi-scale fusion module diagram provided by an embodiment of the present invention;

[0022] Figure 7is a flow chart of a multi-channel cross-attention module provided by an embodiment of the present invention;

[0023] Figure 8 is a flow chart of a channel attention module provided by an embodiment of the present invention;

[0024] Fig. 9 is a flow chart of a spatial attention module provided by an embodiment of the present invention;

[0025] Fig.10 It is a Solo network diagram of multi-channel cross-attention enhancement provided by an embodiment of the present invention;

[0026] Fig.11 This is a flow chart of UAV ground target positioning provided by an embodiment of the present invention;

[0027] Fig.12 It is a cross-attention different embedding position map provided by an embodiment of the present invention;

[0028] Fig.13 is a comparison diagram of different network loss functions provided by an embodiment of the present invention;

[0029] Fig.14 It is an AP.5 output diagram of different Epoch stages provided by an embodiment of the present invention;

[0030] Fig.15 is a diagram of different attention module embedding methods provided by an embodiment of the present invention;

[0031] Fig.16 It is a detailed comparison index diagram of different methods provided by an embodiment of the present invention;

[0032] Fig.17 This is a diagram of the experimental UAV and camouflage target settings provided by an embodiment of the present invention;

[0033] Fig.18 This is a graph of the real-time positioning results of a single UAV on a ground target provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1 , Figure 1A flowchart of a method for detecting and positioning a target of a UAV is provided in accordance with an embodiment of the present invention; a first aspect of an embodiment of the present invention provides a method for detecting and positioning a target of a UAV, including: S1 setting an application scenario and constructing a jungle camouflage data set; S2 enhancing the cross-attention between identifying application scenarios and acquiring data; S3 acquiring the position of a target image and solving its spatial position; S4 verifying the result; wherein, S2 is embedded in a deep learning structure for detecting the accuracy of a target image; S3 solves the spatial position of the target based on building a UAV target detection and positioning system; S4 is set not to input a pre-trained model to maintain the consistency of the comparative experiment.

[0036] In this embodiment, the steps S1 to S4 of the UAV target detection and positioning method are executed. A multi-scale cross-attention enhancement module can be designed and embedded into the FPN (Feature Pyramid Networks) deep learning structure to improve the detection accuracy of camouflaged targets in complex scenarios. Secondly, a single UAV ground target positioning system is constructed to solve the target spatial position by integrating the UAV onboard positioning data, inertial navigation data and target detection data. Finally, a jungle camouflage data set is constructed by placing camouflage nets, camouflage vehicle models and other targets against a jungle background, and the full process verification of single UAV ground target visual detection and positioning is performed.

[0037] Furthermore, the results of the verification experiment show that the AP.5 detection accuracy of the method provided in the embodiment for jungle camouflage targets reaches 70.2%, which is 5.7% higher than the native method. When the UAV flight altitude is ≥12 meters, the ground target positioning error is less than 2 meters, and the average operating efficiency is 29.4fps, which preliminarily verifies the feasibility of single-machine ground camouflage target detection and positioning in complex scenarios.

[0038] It should be noted that when the above-mentioned steps S1 to S4 of the method for detecting and positioning a target unmanned aerial vehicle are executed multiple times, the order of the corresponding execution steps can be adjusted according to experimental needs.

[0039] In this embodiment, the target detection method represented by the Yolo series and the instance segmentation method represented by Solo can be used to obtain the image coordinates and category of the target.

[0040] Specifically, the detection method represents the target position in the form of a rectangular box; the instance segmentation represents the target position in the form of a contour mask, while the pixel-level segmentation method can describe the target position more accurately.

[0041] Furthermore, when Solo instance segmentation is used as the base network, the advantages of multi-scale and attention can be combined to set up a multi-scale cross-attention module to perform cross-attention weighted fusion on the FPN multi-scale features of the original network, so as to improve the feature preservation and semantic representation capabilities of weakly textured targets, thereby realizing real-time detection of camouflaged targets.

[0042] See also Figure 2 , Figure 2 It is a flow chart of a UAV ground camouflage target detection platform provided by an embodiment of the present invention; UAV ground target visual detection includes two parts: ground target detection and ground target positioning, wherein detection is to obtain the image coordinates of the target, and positioning is actually to convert the image coordinates into camera coordinates.

[0043] Preferably, an open source quad-rotor UAV ground target detection and positioning system is selected, and the UAV is equipped with a G2 optoelectronic pod; wherein the optoelectronic pod includes at least one of a vision, a gyro accelerometer, an encoder, and a magnetic orientation sensor.

[0044] Furthermore, the drone is also equipped with GPS module, BMP388 altimeter sensor, accelerometer and other sensors as well as an open source flight control and image and data transmission module (Homer).

[0045] Furthermore, the airborne end is mainly responsible for collecting data from the optoelectronic pod, positioning module, inertial navigation sensor, etc., and then transmits it to the ground processing end based on the image transmission module for real-time target detection and position solution.

[0046] Preferably, it is selected to receive airborne data in real time on the ground side to perform online target detection and positioning; alternatively, real-time processing can also be performed based on an airborne platform.

[0047] It should be noted that Figure 2 The sky-end image data transmission link transmits the Rtsp video stream, aircraft altitude information sky-end, and aircraft longitude and latitude information ground-end to the ground-end image data transmission link; the sky-end image data transmission link receives the flight trajectory control data and pod attitude control data transmitted by the ground-end image data transmission link.

[0048] See also Figure 3 , Figure 3 An embodiment of the present invention provides a flow chart for positioning a ground target by a UAV; wherein the network port data and serial port control of the G2 optoelectronic pod are directly connected to the network port and serial port 1 of the Homer module, and other sensor data are connected to the serial port 2 of the Homer module through the onboard flight control.

[0049] Furthermore, after receiving the airborne data, the ground terminal performs real-time target detection, recognition and position calculation.

[0050] See also Figure 4 , Figure 4 It is a ground camouflage target data set diagram provided by an embodiment of the present invention; in the constructed camouflage target data set, the camouflaged vehicle can be any one or more of a camouflaged trailer, a camouflaged missile vehicle and a camouflaged tank.

[0051] It should be noted that the dataset construction includes two parts: one is target detection data, and the other is target positioning data. We randomly place camouflaged vehicles and camouflage nets that are highly similar to the jungle background, and then take off the drone to shoot at different heights and angles.

[0052] Furthermore, to construct the target positioning data, the target end is equipped with a positioning module with wireless transmission function. While the drone is collecting data, the target's location data will be transmitted to the PC end in real time by the onboard module for recording. Finally, the data recorded by the drone and the real data of the target are aligned through the timestamp.

[0053] See also Figure 5 , Figure 5 It is a statistical chart of the number of various types of targets in a data set provided by an embodiment of the present invention; it shows the number of samples in each category, and trailers, missile vehicles and tanks can all be regarded as jungle camouflage coatings, and the samples corresponding to desert nets, jungle nets and military green nets.

[0054] See also Figure 6 , Figure 6 It is a multi-scale fusion module diagram provided by an embodiment of the present invention; the camouflaged target has a texture that is highly similar to the environment. In order to effectively segment the target, an attention module is set to cross-enhance convolution features of different scales to improve the boundary extraction capability of weak texture targets.

[0055] Figure 6 The multi-scale cross-attention structure shown in the figure replaces the weighted operation of FPN with the cross-attention module, and applies the attention mechanism and multi-scale fusion ideas. i+1 Perform channel attention enhancement and then calculate C i The spatial attention of C i+1 The branch data is cross-enhanced to obtain a new feature P' i+1 , then calculate {C2, C3, C4, C5} in turn, and output {P2', P3', P4', P5'}, and then output the new {P2, P3, P4, C5} features according to the FPN structure.

[0056] Furthermore, the implementation process of multi-scale cross-attention is shown in Table 1. Given the different scale features output by the two Block layers of the backbone network, the spatial attention weights obtained by the deep convolution are multiplied by the shallow convolution features after channel attention enhancement, and finally the output features are obtained.

[0057] Table 1 Multi-scale cross-attention implementation process

[0058]

[0059] See also Figure 7 , Figure 7 is a flow chart of a multi-channel cross-attention module provided by an embodiment of the present invention; assuming that C i The convolution feature dimension is [b, c, w, h], the channel attention weight is calculated, the dimension is [b, c, 1, 1], and Ci is enhanced channel by channel to obtain the enhanced feature, the dimension is [b, c, w, h]; record C i+1 The convolution feature dimension is [b1, c1, w1, h1], the spatial attention weight is calculated, the dimension is [b1, 1, w1, h1], and then the dimension is transformed to [b, 1, w, h], and then C i+1 The spatial attention of the Ci branch performs cross-pixel spatial enhancement on the enhanced features of the Ci branch, with a dimension of [b, c, w, h]. The cross-enhanced features are residually weighted with the Ci branch enhanced features to obtain the final output of the cross-attention module.

[0060] See also Figure 8 and Fig. 9 , Figure 8 is a flow chart of a channel attention module and a spatial attention module provided by an embodiment of the present invention; Fig. 9 This is a flow chart of the spatial attention module provided by one embodiment of the present invention; wherein the cross-attention module can combine deep learning methods such as attention, multi-scale and residual modules to improve the significance of target features under weak contrast conditions.

[0061] Furthermore, the cross-attention module includes a channel attention module and a spatial attention module. The channel attention module downsamples the feature dimension from [b, c, w, h] to [b, c, 1, 1] based on the average pooling of the channel dimension, and then transforms the channel dimension through downsampling, upsampling and activation functions. The spatial attention module downsamples the feature dimension from [b, c, w, h] to two independent convolutional features of dimension [b, 1, w, h] based on the maximum pooling and average pooling of the spatial dimension, and then concatenates the two independent convolutional features in the channel dimension to generate a convolutional feature of [b, 2, w, h], and then transforms the dimension through two-dimensional convolution, activation function, etc., and finally generates a convolutional feature of [b, 1, w, h]. It should be noted that b is the batch size of the network, c, w and h are the channel dimension and the width and height of the convolutional feature, respectively.

[0062] See also Fig.10 , Fig.10It is a Solo network diagram with multi-channel cross-attention enhancement provided by an embodiment of the present invention; the Solo instance segmentation algorithm is selected as the basic network for target detection.

[0063] Specifically, the Solo network includes a backbone network, an FPN fusion network, and a Header structure. The backbone network uses the ResNet18 module to extract image features; the FPN structure mainly fuses multiple different-scale outputs of the backbone network; and the Header structure mainly outputs task-related masks and category predictions.

[0064] Furthermore, the Solo network performs cross-attention enhancement on adjacent channels of the output of the 2nd to 5th layer of the backbone network. The cross-attention embedding positions are divided into: the 2nd and 3rd blocks of the backbone network, the 3rd and 4th blocks, and the 4th and 5th blocks. Since the cross-attention module does not change the data dimension, the Solo Header structure is used to process the fused features.

[0065] It should be noted that loss is a criterion for evaluating the degree of deviation between the algorithm's predicted value and the true value. The loss function of the instance segmentation algorithm consists of two parts: classification loss and mask loss. The calculation method is shown in the formula:

[0066] L=L conf +λL mask (1)

[0067] Among them, L conf is the loss function of the classification branch, Lmask is the loss function of the mask branch, which is a constant used to adjust the weights of the two parts and is set to 3.

[0068] Furthermore, L conf The Focal Loss function is used for calculation, and its definition is as shown in the formula:

[0069] L conf = -α(1-p cls ) τ log(p cls ) (2)

[0070] Among them, p cls is the category probability, α and τ take values ​​of 0.25 and 2, which respectively solve the problem of imbalance in the ratio of positive and negative samples and the ratio of difficult and easy samples.

[0071] L mask The calculation is shown in the formula:

[0072]

[0073] Where N pos is the number of positive samples, is the indicator function, when p cls >0, function is 1, otherwise it is 0. d(:,:) uses the Dice Loss function, where and are the predicted mask and the true mask on the kth block, respectively.

[0074] See also Fig.11 , Fig.11 This is a flow chart of ground target positioning by a UAV provided by an embodiment of the present invention. Ground target positioning is the conversion from image coordinates to world coordinates. Ground target positioning by a single UAV can make a preliminary estimate of the target position by using the elevation difference between the two as prior knowledge.

[0075] Furthermore, after obtaining the image position of the target based on the deep learning method, the absolute position of the ground camouflage target can be accurately calculated by integrating the UAV's own attitude position information such as latitude, longitude and altitude, the azimuth and pitch angle of the optoelectronic pod, and the slant distance between the UAV and the ground target, thereby realizing the positioning of the UAV on the ground target.

[0076] The specific process includes: first, the ground-side processing platform identifies the target from the image obtained by the airborne optoelectronic pod and obtains the image position, and solves the camera coordinates of the target according to the conversion method from image coordinates to camera coordinates: [x cam ,x cam ,f] T On this basis, the world coordinates of the target are solved according to the following formula:

[0077]

[0078] in, is the transformation matrix from the image coordinate system to the body coordinate system, is the conversion matrix from the body coordinate system to the geographic coordinate system. The roll angle of the optoelectronic pod is set to 0°, and (α, β) is the azimuth and pitch angle of the optoelectronic pod. Then the conversion matrix for:

[0079]

[0080] in, is the attitude and position information of the drone, which respectively represents the roll angle, pitch angle and heading angle, and the conversion matrix It can be expressed as:

[0081]

[0082] The latitude and longitude position of the drone is taken as the origin, that is, (x p ,y p )=(0,0), in solving the coordinates of the target relative to the drone (xp ,y p ), (x p ,y p ) coordinate conversion formula to longitude and latitude coordinates to calculate the longitude and latitude position of the target point.

[0083]

[0084]

[0085] Among them, R earth is the radius of the earth, and its value is 6378245. Before the specific implementation, the pod internal parameters need to be calibrated to correct errors such as pixel distortion; the calibration work can choose the common black and white grid plate, based on any one or more calibration toolboxes such as Matlab, OpenCV, and ROS, and the calibration parameters are finally used to solve the image coordinates to the camera coordinates.

[0086] See also Fig.12 , Fig.12 It is a cross-attention different embedding position map provided by an embodiment of the present invention; Fig.12 In order to maintain consistency in the comparative experiments, all networks were set to not input the pre-trained model.

[0087] Specifically, the network training parameter epoch is set to 200; Batch size is set according to experimental needs; the PC environment used for training is: To prove the effectiveness of the cross-attention module, design the module embedding method. According to the cross-attention structure embedded in different layers of the FPN structure, a total of 3 different embedding structures are formed. For the sake of distinction, the cross-attention enhancement only for the 2nd and 3rd layers is called improvement 1, the cross-attention enhancement for the 2nd, 3rd, and 4th layers is called improvement 2, and the cross-attention enhancement for the 2nd, 3rd, 4th, and 5th layers is called improvement 3.

[0088] It should be noted that the evaluation indicators include accuracy evaluation and efficiency evaluation. Among them, the segmentation accuracy adopts the COCO mAP standard indicator system, and selects indicators such as mAP@0.5, mAP@0.75 and mAP@0.5:0.95 respectively. The t in mAP@t represents the IoU threshold for correct prediction; the segmentation efficiency is measured by fps. IoU is the intersection-over-union ratio of the predicted mask to the true mask, and the specific calculation method is:

[0089]

[0090] Among them, R pre Table R true They represent the predicted mask and the true mask area respectively. If the IoU is greater than the threshold t(t∈[0,1]), the prediction result is considered correct.

[0091] See also Fig.13 , Fig.13 It is a comparison chart of different network loss functions provided by an embodiment of the present invention; it lists the model convergence of each network under 200 epoch iterations, and sets the initial state of training to the same parameters.

[0092] Furthermore, the improved method has higher training efficiency: on the one hand, the loss of the improved model with cross-attention converges faster, with the loss dropping below 0.2 at about 10 epochs, while the loss of the original Solo only starts to drop after 25 epochs; on the other hand, the loss of the improved model converges stably with less loss.

[0093] Furthermore, the loss convergence of Improvement 2 and Improvement 3 is relatively consistent, which shows that the improvement of multi-scale cross-attention enhancement of deep convolution is more obvious. The actual test results will be combined for analysis later.

[0094] Table 2 lists the index evaluation results of different improved methods, selecting AP.5, AP.75, AP.5:.95 and AR.5:.95 values. As can be seen from Table 2, the model detection accuracy of the improved method is improved compared with the native Solo, with AP.5 increased by 5.7% at most, AP.75 increased by 18.7% at most, and AP.5:.95 increased by 9.8% at most. The results fully prove that cross-attention has a significant effect on the detection accuracy of the native network. The improvement of AP.75 and AP.5:.95 is significantly greater than that of AP.5, indicating that the cross-attention improvement method has a higher target detection confidence.

[0095] Table 2 adds the comparative experiments in different positions, as follows:

[0096]

[0097] Comparing the performance of cross-attention embedding in different positions of the network, it is found that improvement 3 (2nd to 5th layer embedding) has the best performance, improvement 2 (2nd to 4th layer embedding) has the second best performance, and improvement 2 (only 2nd and 3rd layer embedding) has the smallest improvement. The experimental results are in line with expectations. That is, making fuller use of deep convolutions has a more significant effect on improving the algorithm's detection performance. By performing simple cross-weighting on the original FPN structure, the interaction between receptive fields of different scales can be effectively improved, that is, the network structure has a more significant improvement effect.

[0098] See also Fig.14 , Fig.14 It is an AP.5 output diagram of different Epoch stages provided by an embodiment of the present invention; the segmentation results of three improved strategies are listed, and only Improvement 3 effectively segments the camouflaged missile vehicle model in the figure, while the other two methods miss this type of target.

[0099] Furthermore, the multi-scale target detection results were compared according to the COCO evaluation criteria. As shown in Table 3, the improved method has improved the detection performance at the Small, Medium and Large scales. The Medium scale has the best improvement (10.6%), followed by the Large scale (9.2%), and the Small scale has the smallest improvement (7.1%). On the one hand, this illustrates the enhancement effect of cross-attention on multi-scale target features. On the other hand, it is also because the cross-attention structure of this paper is embedded in the latter layers of the backbone network, which has a stronger feature description capability for medium and large-scale targets. In addition, the limited accuracy of small target detection is mainly due to the weak feature description capability of the shallow convolutional network of the Solo backbone network itself that represents small-scale targets.

[0100] Table 3 compares the results of different scale detection experiments as follows:

[0101]

[0102]

[0103] See also Fig.15 , Fig.15 It is a diagram of different attention module embedding methods provided by one embodiment of the present invention; the cross-attention uses channel attention and spatial attention modules. In order to further analyze the performance of the cross-attention, the structure of only adding channel attention and only adding spatial attention is compared.

[0104] Specifically, there are two structural embedding methods, SENet and SPNet modules as channel and spatial attention respectively. Method a only performs channel attention on a certain branch, and does not perform cross-attention between multi-scale branches; method b uses the spatial attention of shallow branches to perform cross-enhancement on deep branches.

[0105] Table 4 lists the quantitative results of the detection performance of the three attention structures. It can be seen from the table that the detection results of method a, which only performs channel attention enhancement on a certain branch, have no significant improvement compared to the detection performance of the native Solo network, and are even slightly lower; method b uses the spatial attention of the shallow branch to cross-enhance the deep branch, and the improvement is more significant compared to the native Solo network, among which AP.5 is increased by 4.6%, AP.75 is increased by 16.3%, and AP.5:.95 is increased by 8.9%; and the improvement effect of the method in this paper is the most significant. It can be seen from the results that the difference between the cross-attention module and the cross-attention module is the greatest. After the introduction of the cross-attention module, the improvement of the model performance is the most significant. This is because the shallow layer Ci is compared with the deep layer C i+1 The feature map resolution is higher and the detailed description of the target features is better. The Ci layer is used to enhance the Ci+1 The layer is equivalent to giving higher-resolution detail guidance on the low-resolution feature map, which to a certain extent proves that the interaction of convolutional features of different scales can achieve better experimental performance.

[0106] Table 4 Comparison of different attention modules is as follows:

[0107]

[0108] See also Fig.16 , Fig.16 It is a detailed comparison indicator diagram of different methods provided in an embodiment of the present invention; it lists the detection results of different methods in several scenarios.

[0109] Specifically, in the first row, the boundary segmentation of the camouflaged tank by the Solo+SE method is greatly distorted; in the second and third rows, the Solo+SE method fails to detect the camouflaged trailer. The results show that both Solo+SP and the proposed method introduce cross-attention, and the network detection effect is significantly improved.

[0110] See 17, Fig.17 This is a diagram of the experimental drone and camouflaged target settings provided by an embodiment of the present invention; based on visual detection, a preliminary verification of single-machine positioning is carried out. This is an open source drone ground target detection and positioning prototype system. Fig.17 (a) is an open source drone, according to Fig.11 Positioning process construction; Fig.17 (b) The "positioning + data transmission" module carried by the target is used to transmit the target position to the PC in real time for record preservation; Fig.17 (c) is the camouflage net and camouflage uniform used for target camouflage. The experiment mainly simulates the detection and positioning of camouflaged people on the ground by UAV when the flight altitude is ≥10 meters.

[0111] As shown in Table 5, several sets of positioning data are randomly listed. It can be seen that when the UAV is ≥ 10 meters, it has a certain positioning effect on the camouflaged person on the ground, and the positioning error is less than 2 meters. This has a certain effect in single UAV visual positioning.

[0112] Table 5 UAV positioning results analysis, as follows:

[0113]

[0114] Fig.18 The figure shows the real-time display of continuous detection and positioning results. It has a good detection effect on camouflaged people against the background of the jungle at dusk, and can output real-time positioning estimates. In addition, the actual detection results show that the average running frame rate is 29.4fps, which meets the real-time requirements.

[0115] In one of the embodiments, S1 includes: constructing target detection data and constructing target positioning data; constructing target detection data includes setting a jungle background and placing the target to be measured, and shooting at different heights and angles through a drone; constructing target positioning data includes the attitude data of the drone and the position data of the target, and a positioning module with a wireless transmission function is installed on the target end. The position data of the target collected by the drone will be transmitted to the PC end in real time by the installed module for recording, and finally the data recorded by the drone and the real data of the target are aligned through the timestamp.

[0116] In this embodiment, the above description and Figure 4 Implementation, no further details are given.

[0117] In one embodiment, S2 includes setting an attention module for cross-enhancing convolutional features of different scales and extracting boundary data of weakly textured targets; the attention module is used for channel attention enhancement according to the attention mechanism and multi-scale fusion method, calculates channel spatial attention, and cross-enhances channel branch data, and after obtaining new features, calculates the channels in sequence and then outputs them; and outputs new features according to the deep learning structure.

[0118] In this embodiment, the attention module can be a convolutional block attention module, which is a simple and effective attention module for feedforward convolutional neural networks. Specifically, given an intermediate feature map, the attention map can be inferred along two independent dimensions (channel and space) in sequence, and then the attention map can be multiplied by the input feature map for adaptive feature refinement; because CBAM is a lightweight general module, it can be seamlessly integrated into any CNN architecture.

[0119] In one of the embodiments, S2 also includes deep learning of the residual module, which is combined with the attention mechanism and the multi-scale fusion method for target feature saliency detection under weak contrast conditions.

[0120] In this embodiment, the deep learning of the residual module can be achieved by adding an identity function to the fitted function. Assuming that the optimal function in a certain layer is denoted as H(x), then the objective function F(x) we fit is defined as F(x): = H(x)-x, and the function F(x) is called the "residual function".

[0121] Specifically, the function consists of two parts: the identity function and the residual function. The existence of the identity function avoids the "negative optimization" problem, while the residual function plays the role of "icing on the cake". Based on the assumption that the optimal function has a high similarity with the linear function, in the extreme case, if the optimal function is a linear function, then our optimization will become extremely easy, because the parameters of F(x) are close to 0 when initialized, so it will not obtain a large gradient during the training process, and thus will not affect the performance of the identity function.

[0122] In one embodiment, S2 also includes: the attention module includes a channel attention module and a spatial attention module; wherein the channel attention module downsamples the feature dimension from [b, c, w, h] to [b, c, 1, 1] based on average pooling in the channel dimension, and then transforms the channel dimension through at least one of downsampling, upsampling and activation function; the spatial attention module downsamples the feature dimension from [b, c, w, h] to two independent convolution features of dimension [b, 1, w, h] based on maximum pooling and average pooling in the spatial dimension, and then concatenates the two independent convolution features in the channel dimension to generate a convolution feature of [b, 2, w, h], and then transforms the dimension through two-dimensional convolution, activation function, etc., and finally generates a convolution feature of [b, 1, w, h].

[0123] In this embodiment, the attention mechanism is widely used in semantic segmentation and image description generation. When using attention to process tasks, the importance of different information is reflected by weights. The attention mechanism distinguishes the attention of different information in the weight allocation. The attention mechanism can be regarded as a multi-layer perception mechanism consisting of query matrix, key and weighted average.

[0124] In this embodiment, the channel attention mechanism can be SENet (Squeeze-and-Excitation Networks); or a network result similar to the experimental requirements. Exemplarily, SENet is divided into two parts: compression and excitation. The purpose of the compression part is to compress the global spatial information, and then perform feature learning in the channel dimension to form the importance of each channel to the channel. Finally, the excitation part assigns different weights to each channel.

[0125] In one of the embodiments, S2 also includes: an instance segmentation algorithm as a basic network for drone target detection; wherein the instance segmentation algorithm includes any one or more of a backbone network, an FPN fusion network, and a Header structure; the backbone network is used to extract image features; the FPN fusion network is used to fuse multiple different scale outputs of the backbone network; the Header structure is used to output masks and category predictions related to preset tasks.

[0126] In this embodiment, instance segmentation is a further refinement based on semantic detection, separating the foreground and background of the object, and achieving pixel-level object separation. It should be noted that semantic segmentation of an image and instance segmentation of an image are two different concepts. Semantic segmentation only distinguishes and segments objects of different categories, while instance segmentation further segments objects of different instances in the same class.

[0127] In one of the embodiments, S3 includes visual detection of ground targets by a UAV, and the visual detection of ground targets by a UAV includes ground target detection and ground target positioning; ground target detection is used to obtain image coordinates of the target, and ground target positioning is used to convert the image coordinates into camera coordinates.

[0128] In this embodiment, the relevant implementation process has been described in detail above and will not be repeated here.

[0129] In one embodiment, S4 includes verifying the validity of the attention module and setting different module embedding methods.

[0130] In this embodiment, verification can be performed based on relevant data of the result, and no further details will be given.

[0131] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements shall be within the scope of protection of the present application without causing the essence of the corresponding technical solutions to deviate from the spirit of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting and positioning an unmanned aerial vehicle target, characterized in that: include: S1 sets the application scenario and builds a jungle camouflage dataset; S2 enhances the cross-attention between identifying application scenarios and acquiring data; S3 obtains the target image position and solves its spatial position; S4 Validation of results; Among them, the attention module in S2 is embedded in the deep learning structure for target image accuracy detection; the deep learning structure is an instance segmentation algorithm; S3 solves the target spatial position according to the construction of the drone target detection and positioning system; S4 is set to not input the pre-training model to maintain the consistency of the comparative experiment; S2 includes setting an attention module for cross-enhancing convolutional features of different scales and extracting boundary data of weak texture targets; The attention module is used for channel attention enhancement according to the attention mechanism and multi-scale fusion method, calculates channel spatial attention, and cross-enhances channel branch data. After obtaining new features, the channels are calculated in sequence and then output; and the new features are output according to the deep learning structure; Specifically: Let C i The convolution feature dimension is [b, c, w, h], the channel attention weight is calculated, the dimension is [b, c, 1, 1], and Ci is enhanced channel by channel to obtain the enhanced feature, the dimension is [b, c, w, h]; record C i+1 The convolution feature dimension is [b1, c1, w1, h1], the spatial attention weight is calculated, the dimension is [b1, 1, w1, h1], and then the dimension is transformed to [b, 1, w, h], and then C i+1 The spatial attention of the Ci branch performs cross-pixel spatial enhancement on the enhanced features of the Ci branch, with a dimension of [b, c, w, h]. The cross-enhanced features are residually weighted with the Ci branch enhanced features to obtain the final output of the cross-attention module.

2. The method for detecting and positioning an unmanned aerial vehicle target according to claim 1, characterized in that: S1 includes: Constructing target detection data and constructing target positioning data; The target detection data construction includes setting a jungle background and placing the target to be detected, and shooting at different heights and angles by using a drone; The constructed target positioning data includes the attitude data of the drone and the position data of the target. A positioning module with a wireless transmission function is installed on the target end. The position data of the target collected by the drone will be transmitted to the PC end in real time by the installed module for recording. Finally, the data recorded by the drone and the real data of the target are aligned through the timestamp.

3. The method for detecting and positioning an unmanned aerial vehicle target according to claim 1, wherein: S2 also includes deep learning of the residual module, which combines the attention mechanism and the multi-scale fusion method for target feature saliency detection under weak contrast conditions.

4. The method for detecting and positioning an unmanned aerial vehicle target according to claim 1, wherein: S2 also includes: The attention module includes a channel attention module and a spatial attention module; The channel attention module downsamples the feature dimension from [b, c, w, h] to [b, c, 1, 1] based on channel dimension average pooling, and then transforms the channel dimension by at least one of downsampling, upsampling and activation function; The spatial attention module downsamples the feature dimension from [b, c, w, h] to two independent convolutional features of dimension [b, 1, w, h] based on the maximum pooling and average pooling of the spatial dimension, and then concatenates the two independent convolutional features in the channel dimension to generate a convolutional feature of [b, 2, w, h], and then transforms the dimension through two-dimensional convolution and activation function to finally generate a convolutional feature of [b, 1, w, h].

5. The method for detecting and positioning an unmanned aerial vehicle target according to claim 1, wherein: S2 also includes: The instance segmentation algorithm is used as the basic network for the drone target detection; wherein the instance segmentation algorithm includes a backbone network, an FPN fusion network and a Header structure; the backbone network is used to extract image features; the FPN fusion network is used to fuse multiple different scale outputs of the backbone network; the Header structure is used to output masks and category predictions related to preset tasks.

6. The method for detecting and positioning an unmanned aerial vehicle target according to claim 1, wherein: S3 includes The UAV ground target visual detection includes ground target detection and ground target positioning; the ground target detection is used to obtain the image coordinates of the target, and the ground target positioning is used to convert the image coordinates into camera coordinates.

7. The method for detecting and positioning an unmanned aerial vehicle target according to claim 1, wherein: S4 includes verifying the effectiveness of the attention module and setting different module embedding methods.

Citation Information

Patent Citations

  • Face recognition detection method based on hybrid attention mechanism

    CN112200161A

  • Unmanned aerial vehicle image target detection method based on large-kernel equivalent convolution attention mechanism

    CN116109947A