Unmanned aerial vehicle image traffic target detection method based on attention mechanism and reparameterization

Abstract

The application discloses a kind of unmanned aerial vehicle image traffic target detection methods based on attention mechanism and reparameterization, comprising:1, the image of the image acquisition equipment carried by unmanned aerial vehicle is used to construct data set;2, establish unmanned aerial vehicle image traffic target detection model, increase bypass convolution branch in the convolution layer of main network to improve feature extraction capability, information expression of traffic target is enhanced using attention mechanism, the position of target bounding box and the target class are more accurately positioned using the double-branch detection head of channel offset;3, construct loss function, until the parameter convergence of network after training iteration;4, reparameterization technology is used to fuse bypass convolution branch of main network during model inference, while retaining model performance, improve inference speed.The application can identify the traffic target position and category information in unmanned aerial vehicle image, and has important significance for traffic flow monitoring and constructing intelligent traffic system.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent transportation, in particular to a target detection network improvement method based on deep learning, and more particularly to a traffic target detection model training and target detection method. BACKGROUND

[0002] Unmanned aerial vehicles have broad application prospects in civilian fields. For example, in the field of agricultural plant protection, unmanned aerial vehicles can spray pesticides by installing spraying devices, and can realize the functions of patrol monitoring and pest monitoring in combination with cameras. As a carrier for aerial photography, unmanned aerial vehicles can receive less interference and have a wide view angle when flying in the air, so that the images or videos captured by the unmanned aerial vehicles using high-definition cameras have high quality and high resolution. By deploying unmanned aerial vehicles on urban traffic roads and performing real-time target detection on the images of the unmanned aerial vehicles, the traffic flow on the roads can be analyzed, which can provide effective data support for subsequent traffic supervision and traffic scheduling. Therefore, traffic target detection in images of unmanned aerial vehicles is an important part of building a smart traffic system and has high research value.

[0003] Target detection is an important basic research field in computer vision, which mainly locates the position of objects of interest in images and classifies the categories to which the objects belong. In recent years, the field of computer vision has made great progress, and traditional methods of manually identifying targets in images have become less and less suitable and have been gradually replaced by intelligent analysis methods. Target detection is the basis and premise for completing more refined tasks. Target detection refers to using computer and other hardware devices to obtain the position coordinates and classification information of targets in image or video media through a series of algorithms. Traditional research methods manually extract feature information required for target object detection in images, including HOG+SVM, DPM, and other methods. After obtaining the features of the target using feature extraction methods, the features are put into a classifier to obtain the information of the target. This manually designed target feature extractor does not have good generalization, and it does not have good performance for target occlusion and small target detection. With the great success of deep learning in image classification, deep learning has been introduced into the field of target detection.

[0004] Since the unmanned aerial vehicle image has rich target information, the existing detection model backbone network still has insufficient ability to extract features from the image, which affects the detection accuracy of the subsequent model; the traditional feature fusion network lacks sufficient utilization of feature map information and does not enhance the expression of effective messages and suppress redundant information; for small targets and easily confused targets in the unmanned aerial vehicle image, the existing target detection head directly uses a fully connected layer for classification and regression, ignoring the conflict between the classification task and the regression task; the existing target detection model cannot achieve a good balance between precision and speed, and is difficult to be applied in the unmanned aerial vehicle scene. SUMMARY

[0005] The present application is to solve the above-mentioned deficiencies in the prior art, and proposes a kind of unmanned aerial vehicle image traffic target detection method based on attention mechanism and reparameterization, to improve the feature extraction capability of model backbone network, improve the detection effect of small scale traffic target, thereby improve the accuracy of model positioning and classification effect of traffic target, provide support for subsequent traffic flow analysis, it has important significance to build intelligent transportation system.

[0006] To achieve the above-mentioned application purposes, the following technical solutions are adopted:

[0007] The unmanned aerial vehicle image traffic target detection method based on attention mechanism and reparameterization has the following steps:

[0008] Step 1, obtain a data set of N unmanned aerial vehicle aerial images with label information; N represents the total number of images in the data set; the label information includes the position information of the target detection frame and the target detection category;

[0009] Step 2, construct a target detection model, including: input end, backbone network, feature fusion network, output end; wherein, the input end includes: Mosaic data enhancement layer, image scaling padding layer, the backbone network includes: a plurality of parallel branches composed of backbone branch and bypass branch, each branch is a convolution layer; the feature fusion network includes: k-1 Level image super path, k-1 Level embedded GCAM attention mechanism path enhancement network; the output end includes: target pre-frame calculation module, channel offset double-branch target detection head;

[0010] Any m unmanned aerial vehicle aerial images with label information are input into the target detection network as current batch training data; m≤N;

[0011] Step 2.1, the Mosaic data enhancement layer of the input end splices the current batch training data after random cropping to obtain an image x;

[0012] The image scaling and padding layer at the input end adjusts the size of image x to the input size of the backbone network to obtain the preprocessed image X. in ;

[0013] Step 2.2, the preprocessed image X in The input is fed into the backbone network for feature extraction, resulting in extracted multi-scale feature maps {B1, B2, ..., B}. j ,...,B k}; where B j Let B represent the j-th multi-scale feature map. k Represents the minimum scale feature map;

[0014] Step 2.3, the minimum scale feature map B k The input is fed into the feature fusion network and processed sequentially through the (k-1)th level image super-resolution path to obtain the maximum-scale traffic target feature map P1 to be detected. Here, the output of the (j-1)th level image super-resolution path and the (kj)th multi-scale feature map B are combined. k-j Perform channel stitching and then input it into the super-resolution path of the j-th level image;

[0015] The traffic target feature map P1 is then processed sequentially through a path enhancement network with a k-1 level embedded GCAM attention mechanism to obtain the multi-scale traffic target feature map {P1, P2, ..., P} to be detected. k}; where P k This represents the feature map of the traffic target to be detected at the k-th scale;

[0016] Step 2.4: The target preselection box calculation module at the output end performs k-means clustering on the position information of the target detection boxes in the current batch of training data, and outputs nc×k size anchor values ​​of target preselection boxes; where nc represents the number of target preselection boxes for each scale feature map;

[0017] The dual-branch target detection head with channel offset at the output end detects the multi-scale traffic target feature map {P1, P2, ..., P} to be detected. k The system is processed to output a total of H×W×nc predicted target bounding boxes, along with their category information, location information, and confidence scores; where H is the height of the feature map of the traffic target to be detected, and W is the width.

[0018] The location information of the traffic target bounding box is the relative offset of the size anchor value between each traffic target bounding box and the corresponding target pre-selection box;

[0019] Step 3, the target detection network is trained by using a random gradient descent method, and a CIOU loss is used to calculate the loss of a predicted target bounding box, and a BCE loss is used to calculate the confidence loss and the classification loss, until the loss converges or reaches the maximum number of iterations, and then the training is stopped, thereby generating an optimal target detection network and its optimal parameters;

[0020] Step 4, the parallel branches in the backbone network are reparameterized into single branches by using a reparameterization method, thereby obtaining a final target detection network for detecting images collected by a UAV terminal in real time or static pictures saved locally to obtain a target detection result.

[0021] The UAV image traffic target detection method based on the attention mechanism and reparameterization has the characteristics that each image super-resolution path in step 2.3 is sequentially composed of a convolution layer, an up-sampling layer and a plurality of C3 modules;

[0022] The C3 module is sequentially composed of two parallel upper and lower branches and a convolution layer, wherein the upper branch includes a convolution layer and a bottleneck block; the lower branch includes a convolution layer, and the output results of the upper and lower branches are spliced in the channel and then input into the convolution layer of the C3 module of the lower branch;

[0023] Each path enhancement network is sequentially composed of a convolution down-sampling layer, a plurality of C3 modules and a GCAM attention mechanism module;

[0024] The GCAM attention mechanism module is sequentially composed of a global attention module and a channel attention module, wherein the global attention module processes the input data by using formula (1);

[0025]

[0026] In formula (1), F l represents the input of the lth global attention mechanism module, W l 1 , W l 2 , W l 3 , W l respectively represent the convolution kernel parameters of the 1st, 2nd and 3rd convolution layers of the lth global attention mechanism module, ReLU represents an activation function, LN represents layer normalization, the symbol represents matrix operation, U l represents the output of the lth global attention mechanism module.

[0027] The channel attention mechanism module processes the input data by using formula (2);

[0028]

[0029] In formula (2), Avgpool represents an average pooling operation, W q and W k represent two parameter-shared fully connected layers, Z l represents the output of the lth channel attention mechanism module.

[0030] The double-branch target detection head of the channel offset in step 2.4 is sequentially composed of a shift operation layer, a convolution layer, a double-branch calculation layer and a Concat layer.

[0031] The double-branch calculation layer includes a classification branch and a regression branch, and both branches are sequentially composed of a plurality of convolution layers and shift operation layers.

[0032] The classification branch outputs the category information of the traffic target by using a convolution layer.

[0033] The regression branch respectively obtains the relative offset of the target pre-box of the traffic target bounding box and the confidence of the traffic target bounding box by using two parameter-shared fully connected layers.

[0034] The reparameterization method in step 4 includes the following steps:

[0035] Step 4.1, updating the convolution kernel parameters in each parallel branch of the backbone network by using formula (4) to obtain updated convolution kernel parameters:

[0036]

[0037] In formula (4), W is the rth group of weight parameters of the reparameterized convolution kernel in the convolution layer of one branch; is the rth bias value of the reparameterized convolution kernel in the convolution layer of one branch, is the rth group of weight parameters of the original convolution kernel in the convolution layer of one branch; γ r , β r are the rth scaling factor and bias factor of the BN layer in the convolution layer, σ r is the rth standard deviation of the BN layer, μ r is the rth mean value of the BN layer;

[0038] Step 4.2, obtaining the fused convolution kernel weight parameters W fuse and the fused convolution kernel bias parameters B fuse by using formula (5);

[0039]

[0040] In formula (5), and respectively refer to the convolution kernel weight parameters of the original convolution layer of the backbone network and the rth group of convolution kernels of the bypass branch, and respectively refer to the rth bias value of the original convolution layer and the bypass branch convolution layer, and padding refers to padding the convolution kernel weight with zero around to align the convolution kernel size of the original convolution layer.

[0041] The electronic device comprises a memory and a processor, and the memory is used to store a program supporting the processor to execute any of the unmanned aerial vehicle image traffic target detection methods, and the processor is configured to execute the program stored in the memory.

[0042] The computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the unmanned aerial vehicle image traffic target detection methods are executed.

[0043] Compared with the prior art, the beneficial effects of the present application are reflected in:

[0044] 1、The present application increases additional bypass convolution branches in the corresponding convolution layer of the backbone network to improve the feature extraction capability of the model, and the model with multiple branches has better performance during training, and the model adopts a multi-branch structure during training, and adopts reparameterization during inference to equivalently convert the model parameters during training into another group of parameters.

[0045] 2、The present application constructs a GCAM attention mechanism module in the feature fusion network, embeds the attention mechanism module after each path enhancement network to obtain a traffic target feature map to be detected, enhances the feature information expression of the target in the unmanned aerial vehicle image from two dimensions of space and channel, makes the model focus on the target itself, reduces the interference of the complex background, the global attention module of the GCAM can capture the global information of the feature map, has stronger robustness to the target with large detection scale change, and thus improves the detection accuracy of multiple scale traffic targets.

[0046] 3、The traffic target detection model according to the embodiment of the application decouples the classification and regression tasks of traffic target detection by using a double-branch detection head module containing a shift operation, uses a full connection layer to complete the regression task which is more sensitive to object edge information, uses a convolution layer to complete the classification task which pays more attention to object overall information, improves the accuracy of the model in classifying easily confused traffic categories, and especially improves the accuracy of the bounding box regression of fine-grained traffic targets. BRIEF DESCRIPTION OF DRAWINGS

[0047] Figure 1 is a flowchart of a training method of a traffic target detection model according to an embodiment of the application;

[0048] Figure 2 is a structure diagram of a traffic target detection model according to an embodiment of the application;

[0049] Figure 3 is a structure diagram of a GCAM attention mechanism module according to an embodiment of the application;

[0050] Figure 4 is a schematic diagram of a channel offset double-branch detection head according to an embodiment of the application. DETAILED DESCRIPTION

[0051] In the embodiment, as shown in the figure, Figure 1 a traffic target detection method based on an attention mechanism and reparameterization includes the following steps:

[0052] Step 1, obtaining a data set of N unmanned aerial vehicle aerial images with label information; N represents the total number of images in the data set; the label information includes position information of a target detection box and a target detection category; the position information of the target is labeled by a rectangular box, and the category of the target is recorded; the processed data set includes 10 traffic target categories to be detected.

[0053] Step 2, constructing a target detection model, as shown in the figure, Figure 2 including an input end, a backbone network, a feature fusion network, and an output end; the input end includes a Mosaic data enhancement layer and an image scaling padding layer; the backbone network includes a plurality of parallel branches composed of a backbone branch and a bypass branch, and each branch is a convolution layer; the feature fusion network includes k-1 level image super-resolution paths and k-1 level path enhancement networks embedded with a GCAM (Global Context Attention Module) attention mechanism; the output end includes a target pre-box calculation module and a channel offset double-branch target detection head (SDPH);

[0054] Input any m labeled UAV aerial images as the current batch training data into the target detection network; m≤N; in this embodiment, m=4;

[0055] Step 2.1, the Mosaic data enhancement layer at the input end splices the current batch training data after random cropping to obtain an image x;

[0056] The image zoom padding layer at the input end adjusts the size of the image x to the input size of the backbone network to obtain a preprocessed image X in ; the input size of the backbone network is 640×640.

[0057] Step 2.2, the preprocessed image X in is input into the backbone network for feature extraction to obtain extracted multi-scale feature maps {B1, B2,..., B j ,...,B k}; wherein B j represents the jth multi-scale feature map, and B k represents the smallest scale feature map; in a specific embodiment, the scale k of the feature map is 4, and the feature extraction includes four multi-scale feature maps {B1, B2, B3, B4}. The backbone network uses a modified CSPDarknet for feature extraction, and additional convolution branches with a convolution kernel size of 1×1 are added to the convolution layers in the CSPDarknet. In this embodiment, additional convolution branches with a convolution kernel size of 1×1 are added to the 2nd-9th layers in the CSPDarknet to obtain RepBlock and CSPRepBottleneck modules, as shown in Figure 2 .

[0058] Step 2.3, the smallest scale feature map B k is input into the feature fusion network and sequentially processed by k-1 levels of image super-resolution paths to obtain a maximum scale traffic target feature map P1 to be detected, wherein the output of the j-1th level image super-resolution path is concatenated with the k-jth multi-scale feature map B k-j and then input into the jth level image super-resolution path;

[0059] The traffic target feature map P1 is sequentially processed by k-1 levels of path enhancement networks embedded with GCAM attention mechanisms to obtain multi-scale traffic target feature maps {P1, P2,..., P k}; wherein P k represents the kth scale traffic target feature map to be detected; in this embodiment, k=4;

[0060] The smallest scale feature map B4 extracted by the backbone network is input into the feature fusion network and sequentially processed by three levels of image super-resolution paths; a low-level feature map reuse strategy is adopted, as shown inFigure 2 As shown in the middle solid rectangle frame, an up-sampling module is added on the feature extraction network, and the feature fusion is performed by splicing the shallow feature maps. Finally, the C3 structure and the GCAM module are used to obtain the P1 detection layer required by the small target detection head. Finally, a down-sampling module is added on the bottom-up path enhancement network to align with the original network structure. B4 passes through the output of the third-level image super-resolution path to enter the first GCAM attention mechanism module to obtain P1. P1 enters the path enhancement network embedded with GCAM attention mechanism for a total of 3 levels in sequence to obtain the detection layers {P1, P2, P3, P4};

[0061] Each level of the image super-resolution path is composed of a convolution layer, an up-sampling layer and a plurality of C3 modules. The convolution kernel size of the convolution layer is 3x3, and the stride is 1. Each level of the path enhancement network is composed of a convolution down-sampling layer, a plurality of C3 modules and a GCAM attention mechanism module in sequence. The convolution kernel size of the convolution down-sampling layer is 3, and the stride is 2. The GCAM attention mechanism module is composed of a global attention module and a channel attention module in sequence, as shown in Figure 3 .

[0062] The GCAM attention mechanism module is composed of a global attention module and a channel attention module in sequence. The global attention module processes the input data using formula (1);

[0063]

[0064] In formula (1), F l represents the input of the lth global attention mechanism module, W l 1 , W l 2 , and W l 3 represent the convolution kernel parameters of the 1st, 2nd and 3rd convolution layers of the lth global attention mechanism module, respectively, all of which are 1x1 convolution kernels, ReLU represents the activation function, and LN represents the layer normalization. The symbol represents matrix operation, and U l represents the output of the lth global attention mechanism module.

[0065] The channel attention mechanism module processes the input data using formula (2);

[0066]

[0067] In formula (2), Avgpool represents the average pooling operation, W q and W k represent two parameter-shared fully connected layers, and Z lAn output of the lth channel attention mechanism module.

[0068] In step 2.4, the target pre-selection frame calculation module of the output end performs k-means clustering on the position information of the target detection frame of the current batch of training data, and outputs the size anchor value of nc x k target pre-selection frames; wherein nc represents the number of target pre-selection frames of each scale feature map.

[0069] The channel offset dual-branch target detection head of the output end processes the multi-scale traffic target feature map {P1, P2, …, P k} to be detected, and outputs the class information, position information and confidence of a total of H x W x nc predicted target bounding boxes; wherein H is the height of the traffic target feature map to be detected, and W is the width;

[0070] The position information of the traffic target bounding box is the relative offset of each traffic target bounding box and the size anchor value of the corresponding target pre-selection frame.

[0071] The channel offset dual-branch target detection head is sequentially composed of a shift operation layer, a convolution layer, a dual-branch calculation layer and a Concat layer.

[0072] The dual-branch calculation layer includes a classification branch and a regression branch, and both branches are sequentially composed of a plurality of convolution layers and shift operation layers.

[0073] The classification branch outputs the class information of the traffic target by using a convolution layer.

[0074] The regression branch uses two parameter-unshared fully connected layers to respectively obtain the relative offset of the target pre-selection frame of the traffic target bounding box and the confidence of the traffic target bounding box.

[0075] The position information of the traffic target bounding box is the relative offset of each traffic target bounding box and the size anchor value of the corresponding target pre-selection frame.

[0076] The structure of the channel offset dual-branch detection head is shown in Figure 4 , wherein nc represents the number of anchor frames, and ncls represents the number of traffic target classes to be detected; the original YOLO detection head uses a 1 x 1 convolution to map the channel number of the multi-scale feature map to (anchor frame number x (total number of classes + 5)) channel number, and then calculates the classification and regression loss. Directly using a 1 x 1 convolution as a detection head is not conducive to directly regressing the target bounding box coordinates and predicting the object class, because the classification and regression tasks are tasks with conflicting properties. Using a smaller convolution kernel is more sensitive to the edge information of the object, and is not robust enough to perform the regression task; using a larger convolution kernel has a larger receptive field, and has the characteristics of smaller spatial sensitivity to perform the classification task. The implementation is as follows:

[0077] Firstly, a 1x1 convolution layer is used to reduce the number of feature map channels, reducing the parameter quantity and calculation quantity of subsequent convolution layers; then parallel branch structures are used to independently perform classification and regression tasks on two branches; finally, the classification or regression results of the two independent branches are spliced to calculate the loss. For the classification branch, 2 3x3 convolution layers are used for feature extraction, and finally a 1x1 convolution layer is used to obtain the calculation result of classification; for the regression branch, 2 fully connected layers are used for feature extraction, and then 2 independent fully connected layers are used to obtain the confidence score and boundary box offset. The shift operation is used to exchange the channel information in different spaces to change the direction to increase the receptive field and aggregate spatial information. The shift operation is added before the 1x1 convolution dimension reduction layer and after the 3x3 convolution layer of the parallel branch. Experimental results show that the operation can improve the performance of the detection head without increasing any parameter quantity and calculation quantity, and slightly increases the delay during inference. To avoid excessive loss of feature information, the shift operation in the application only shifts 1 pixel along a specific dimension.

[0078] Step 3, use CIOU loss to calculate rectangular frame loss, and use BCE loss to calculate confidence loss and classification loss. Random gradient descent method is used to train the target detection network on the server, and after multiple iterations, the optimal target detection network parameters are generated;

[0079] Step 4, use the reparameterization method to reparameterize several parallel branches in the backbone network into several single branches, thereby obtaining the final target detection network for detecting images collected by the unmanned aerial terminal in real time or static pictures saved locally to obtain target detection results. The model with multiple branches has better performance during training, but increases memory resource consumption and also reduces the speed of the model during inference. For the 1x1 convolution bypass branch added to the 2nd-9th layer of the backbone network, the two branches are fused during inference using the structure reparameterization technology.

[0080] Step 4.1, update the convolution kernel parameters in each parallel branch of the backbone network using formula (4) to obtain the updated convolution kernel parameters:

[0081]

[0082] In formula (4), is the rth group of weight parameters of the reparameterized convolution kernel in the convolution layer of a branch; is the rth bias value of the reparameterized convolution kernel in the convolution layer of a branch, is the rth group of weight parameters of the original convolution kernel in the convolution layer of a branch; γ r, β r are the rth scaling factor and bias factor of the BN layer in the convolution layer, respectively, σ r is the rth standard deviation of the BN layer, μ r is the rth mean value of the BN layer.

[0083] Step 4.2, obtaining the fused convolution kernel weight parameter W fuse and the fused convolution kernel bias parameter B fuse by using formula (5), so as to fuse the main branch and the bypass branch of each parallel branch.

[0084]

[0085] In formula (5), and respectively refer to the convolution kernel weight parameter of the original convolution layer of the main network and the rth group of the convolution kernel of the bypass branch, and respectively refer to the rth bias value of the original convolution layer and the bypass branch convolution layer, and padding refers to padding the convolution kernel weight with zero to align the convolution kernel size of the original convolution layer. Using the fused model for inference does not introduce additional parameter quantity and calculation quantity, and can further improve the performance of the model. The fused 1x1 convolution kernel needs to be padded to convert to a 3x3 convolution kernel, and the padding value is 0. Finally, the convolution kernel parameters and bias parameters of the two branches are added to obtain the final required 3x3 convolution kernel parameters. The structure reparameterization technology makes the model use the multi-branch structure during training, and uses reparameterization to equivalently convert the model parameters during training to another group of parameters during inference. Compared with the model during training, the model during inference improves the inference speed without performance loss.

[0086] In this embodiment, an electronic device includes a memory for storing a program supporting a processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0087] In this embodiment, a computer readable storage medium has a computer program stored thereon, and the computer program is run by a processor to execute the steps of the above method.

[0088] The experiment is based on a 64-bit operating system Ubuntu 18.04LTS, and the deep learning framework is Pytorch v1.8.0. The CPU is intel(R)Core(TM)i9-10900K, and the GPU is NVIDIAGeForce RTX3080. The SGD optimizer is used during training, the learning rate is set to 0.04, the batch size is set to 8, and the learning epoch number is set to 140. In the inference stage, the NMS algorithm is used to screen the candidate target frame generated by the model inference to obtain the final result.

[0089] In the experiment, the evaluation index of the COCO data set is used, AP (Average Precision) is the area under the precision-recall curve, and map (mean Average Precision) is the average value of multiple class APs. The main indicators are map@0.5 and map@0.5:0.95. Among them, map@0.5 refers to the average accuracy of all target classes calculated under the IOU threshold of 0.5. Map@0.5:0.95 refers to the average accuracy under the interval of 10 threshold values from 0.5 to 0.95. FPS (Frames Per Second) is used as the evaluation index of the model detection speed;

[0090] The trained network is detected on the Visdrone data set, and the improved network AP reaches 44.6%, which is 10.7% higher than the original YOLOv5s network. Compared with other mainstream algorithms, the improved network has the characteristics of fast detection speed and high detection accuracy, as shown in Table 1.

[0091] Table 1 Comparison of experimental results on Visdrone data set

[0092]

[0093] In order to verify the influence of various improvement strategies on the detection effect of the model, an ablation experiment is carried out on the Visdrone data set, and the results are shown in Table 2. Among them, “√” represents the introduction of the proposed structure, and “ / ” represents that the Shift operation is not used in SDPH.

[0094] Table 2 Comparison of ablation experiment results

[0095]

[0096] The results show that the effect of the traffic target detection of the application is better than that of the mainstream target detection model, the target detection model proposed by the application achieves the best effect in the evaluation precision index, and the detection speed is balanced, which shows that the application has good application space and prospect, and provides a method and idea for the practical application of unmanned aerial vehicles in traffic flow monitoring in the future.

Claims

1. A UAV image traffic target detection method based on attention mechanism and reparameterization, characterized in that, The method comprises the following steps: Step 1, obtaining a dataset of N unmanned aerial vehicle (UAV) aerial images with label information; N represents the total number of images in the dataset; The label information comprises position information of a target detection frame and a target detection category; Step 2, constructing a target detection model, comprising an input end, a backbone network, a feature fusion network and an output end; wherein the input end comprises a Mosaic data enhancement layer and an image scaling padding layer; the backbone network comprises a plurality of parallel branches composed of a main branch and a bypass branch, each branch being a convolutional layer; the feature fusion network comprises k-1 level image super-resolution paths and k-1 level path enhancement networks embedded with GCAM attention mechanisms; and the output end comprises a target pre-frame calculation module and a channel offset double-branch target detection head; k represents the number of feature scales; Each level of image super-resolution path comprises a convolutional layer, an up-sampling layer and a plurality of C3 modules in sequence; The C3 module comprises two parallel upper and lower branches and a convolutional layer in sequence, wherein the upper branch comprises a convolutional layer and a bottleneck block; the lower branch comprises a convolutional layer, and the output results of the upper and lower branches are concatenated in the channel and then input into the convolutional layer of the C3 module; Each level of path enhancement network comprises a convolutional down-sampling layer, a plurality of C3 modules and a GCAM attention mechanism module in sequence; The GCAM attention mechanism module comprises a global attention module and a channel attention mechanism module in sequence; The channel offset double-branch target detection head comprises a shift operation layer, a convolutional layer, a double-branch calculation layer and a Concat layer in sequence; The double-branch calculation layer comprises a classification branch and a regression branch, and both branches comprise a plurality of convolutional layers and shift operation layers in sequence; The classification branch outputs the category information of the traffic target by using a convolutional layer; The regression branch uses two parameter-unshared fully connected layers to respectively obtain the relative offset of the target pre-frame of the traffic target bounding box and the confidence of the traffic target bounding box; Step 3, training the target detection model by using the stochastic gradient descent method, calculating the prediction target bounding box loss by using the CIOU loss, calculating the confidence loss and the classification loss by using the BCE loss, and stopping the training until the loss converges or the maximum number of iterations is reached, thereby generating an optimal target detection network and its optimal parameters; Step 4, reparameterizing the plurality of parallel branches in the backbone network into a plurality of single branches by using the reparameterization method, thereby obtaining a final target detection network for detecting images collected in real time by a UAV terminal or static pictures saved locally to obtain a target detection result. 2.The UAV image traffic target detection method based on attention mechanism and reparameterization according to claim 1, characterized in that, Any m UAV aerial images with label information are input into the target detection model as current batch training data; m≤N; Step 2.1, the input end Mosaic data enhancement layer randomly crops and splices the current batch of training data to obtain an image ; The image zoom padding layer of the input end adjusts the size of the image x to the input size of the backbone network to obtain a preprocessed image ; Step 2.2, the preprocessed image input into the backbone network for feature extraction to obtain the extracted multi-scale feature map ; wherein, denotes the jth multi-scale feature map, B k denotes the minimum scale feature map; Step 2.3, the minimum scale feature map B k Input into the feature fusion network and sequentially processed by k-1 image super-resolution paths to obtain a maximum scale traffic target feature map to be detected , wherein the output of the j-1 image super-resolution path is concatenated with the k-j multi-scale feature map B k-j is concatenated and input into the j image super-resolution path. Traffic target feature map After being processed by the path enhancement network of the k-1 level embedded GCAM attention mechanism in sequence, a multi-scale traffic target feature map {P1, P2, …, Pk} of the to-be-detected traffic target is obtained. k}; wherein, P k represents the to-be-detected traffic target feature map of the kth scale. In step 2.4, the target pre-selection frame calculation module of the output end performs k-means clustering on the position information of the target detection frame of the current batch of training data, and outputs the size anchor value of nc x k target pre-selection frames; wherein nc represents the number of target pre-selection frames of each scale feature map. The double-branch target detection head of the channel offset of the output end processes the multi-scale traffic target feature maps {P1, P2, …, P k} to be detected, and outputs class information, position information, and confidence of a total of HxWxnc predicted target bounding boxes; wherein H is the height of the traffic target feature map to be detected, and W is the width. The position information of the traffic target bounding box is the relative offset of each traffic target bounding box and the size anchor value of the corresponding target pre-selection frame. 3.The UAV image traffic target detection method based on attention mechanism and reparameterization according to claim 2, characterized in that, The global attention module in step 2.3 processes the input data using formula (1); (1) In formula (1), denotes the input of the lth global attention mechanism module, , , denote the convolution kernel parameters of the 1st, 2nd and 3rd convolution layers of the lth global attention mechanism module respectively, ReLU denotes an activation function, and LN denotes layer normalization, the symbol denotes matrix operation, denotes the output of the lth global attention mechanism module; The channel attention mechanism module processes the input data using formula (2); (2) in formula (2), denotes a max-pooling operation, and denotes a fully connected layer with two parameters not shared, denotes the output of the l-th channel attention mechanism module. 4.The UAV image traffic target detection method based on attention mechanism and reparameterization according to claim 3, characterized in that, The reparameterization method in step 4 includes the following steps: Step 4.1, updating the convolution kernel parameters in each parallel branch of the backbone network using formula (3) to obtain updated convolution kernel parameters: (3) In formula (3), is the rth group of weight parameters of the reparameterized convolution kernel in the convolutional layer of one branch; is the rth bias value of the reparameterized convolution kernel in the convolutional layer of one branch, is the rth group of weight parameters of the original convolution kernel in the convolutional layer of one branch; , are the rth scaling factor and bias factor of the BN layer in the convolutional layer, respectively, is the rth standard deviation of the BN layer, is the rth mean value in the BN layer. Step 4.2, obtaining fused convolution kernel weight parameters using formula (4) and fused convolution kernel bias parameters ; (4) In formula (4), and respectively refer to the convolution kernel weight parameters of the original convolution layer of the main network and the rth group of the bypass branch, and respectively refer to the rth bias value of the original convolution layer and the bypass branch convolution layer, and padding refers to padding the convolution kernel weight with zero around to align the convolution kernel size of the original convolution layer.

5. An electronic device comprising a memory and a processor, characterized in that The memory is used to store a program supporting the processor to execute the unmanned aerial vehicle image traffic target detection method of any one of claims 1-4, and the processor is configured to execute the program stored in the memory.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to perform the steps of the unmanned aerial vehicle image traffic target detection method of any one of claims 1-4.

Images