A method and system for UAV traffic target detection based on deep learning and shallow feature enhancement

Abstract

This invention relates to the field of computer vision technology and discloses a method and system for UAV traffic target detection based on deep learning and shallow feature enhancement, comprising the following steps: S1, the UAV collects traffic target images to obtain a dataset; S2, the dataset is fed into the model for training, and a DCSP backbone network is constructed to extract image features; S3, the features are further enhanced; S4, the SPAN network fuses information from feature maps at various scales; S5, the prediction network outputs target type and location information. The Wise-IOU method is introduced, and a dynamic non-monotonic focusing mechanism is used to calculate the bounding box loss, which reduces the impact of low-quality dataset annotation on training and enhances the generalization ability of the detection method. The fused feature map is input into the prediction network to obtain the final target type and location information, which can overcome the problems of target deformation, blurring, and small scale in UAV images and improve the detection accuracy of UAVs for traffic targets.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a method and system for detecting traffic targets on unmanned aerial vehicles (UAVs) based on deep learning and shallow feature enhancement. Background Technology

[0002] With the accelerating pace of urbanization and the continuous improvement of transportation systems, there is an increasing need for intelligent transportation systems to address issues such as traffic congestion, traffic safety, and environmental protection. Among these systems, the application of unmanned aerial vehicle (UAV) technology is receiving increasing attention and research. UAV applications in intelligent transportation include traffic monitoring, road inspection, emergency rescue, and intelligent delivery, among others. UAVs can be equipped with various sensors such as high-definition cameras, LiDAR, and GPS to perform tasks such as traffic flow monitoring, road accident image acquisition, and traffic construction planning, providing crucial data support for urban traffic management and planning.

[0003] However, the application of drones in the field of intelligent transportation also faces some challenges. One important issue is the inadequacy of drone target detection technology. In intelligent transportation scenarios, drones often need to detect targets in large areas, such as pedestrians, vehicles, and obstacles. However, while drone-captured images have high overall resolution, the number of small-scale target detection feature points, which account for a significant proportion, is sparse, easily leading to missed detections or misclassifying multiple targets as the same. Furthermore, the movement of the drone itself causes variations in image scale and perspective, posing considerable difficulties for target detection. Therefore, this paper proposes a drone traffic target detection method and system based on deep learning and shallow feature enhancement. Summary of the Invention

[0004] The purpose of this invention is to provide a method and system for detecting traffic targets by unmanned aerial vehicles (UAVs) based on deep learning and shallow feature enhancement, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a UAV traffic target detection method based on deep learning and shallow feature enhancement, comprising the following steps:

[0006] S1. The drone collects images of traffic targets to obtain a dataset;

[0007] S2. The dataset is fed into the model for training, and the DCSP backbone network is constructed to extract image features.

[0008] S3, further enhance features;

[0009] S4 and SPAN networks fuse information from feature maps at various scales;

[0010] S5. Predict the target type and location information output by the network.

[0011] Preferably, in step S2 above, the dataset is fed into the model for training. By constructing a DCSP backbone network to extract image features, five sets of feature maps of different sizes are obtained, including the following steps:

[0012] S201. Unify the size of the input image, normalize the image pixels, and arrange them in the order of CHW. Obtain the initial feature map through a Conv block containing 2D convolution, Batch Norm normalization, and SiLU activation function.

[0013] S202. The initial feature map is downsampled by multiple Conv blocks to obtain four new feature maps with decreasing size. The higher the layer, the smaller the size, which contains richer semantic information but cannot well express spatial information.

[0014] Preferably, in S3 above, the deformable convolutional DC2 module is incorporated to enhance the features, including the following steps:

[0015] S301: Sampling is performed by a convolutional module or a deformable convolutional module, increasing the receptive field while reducing computational parameters;

[0016] S302: Split into two branches based on the number of channels;

[0017] S303: Branch 1 is sent to the fusion module CAcat without processing;

[0018] CAcat is an attention-based feature fusion module that further optimizes the feature fusion effect by using the CA attention mechanism on top of concat. concat adds two input feature maps along the channel number dimension, resulting in a new feature map with the same size and the same number of channels as the sum of the input feature maps. The CA attention mechanism performs average pooling and max pooling simultaneously along both channel and spatial dimensions. After regularization and activation, it is then combined with the residual channels and the weights are redistributed.

[0019] S304: Branch 2 is sent to the fusion module CAcat through a residual channel composed of multiple DCN-Bottlenecks;

[0020] S305: In branch two, a new branch is led out after each DCN-Bottleneck and sent to the fusion module CAcat;

[0021] S306: Use CAcat to merge all branches;

[0022] S307: The number of output channels is adjusted to be the same as the number of input channels by a convolutional module or deformable convolutional module.

[0023] Preferably, in S4 above, the SPAN network is used, which is divided into two parts: top-down and bottom-up, to fuse information from feature maps at various scales. CAcat is used to fuse pairwise feature maps, including the following steps:

[0024] S401: Upsample the input feature map to obtain a feature map with the same size but different number of channels as the corresponding layer feature map in the backbone; use CAcat to fuse the obtained feature map with the corresponding feature map in the backbone; use the fused feature map to perform feature enhancement.

[0025] S402: Repeat S401 twice to obtain two sets of larger feature maps, which are then output to the medium and small target detection heads respectively;

[0026] S403: The feature map obtained from S401 is fused with the SPPF output feature map using CAcat and then sent to the high-level semantic information detection head.

[0027] As a preferred embodiment, in S5, Wise-IoU is used to optimize the bounding box loss calculation. Three decoupled detection heads are used to calculate the feature maps of three different scales output by SPAN to obtain the regression loss and classification loss. The final prediction result is then calculated, including the target location, bounding box, target category, and confidence score.

[0028] Preferably, in S2, the DC2 module introduces deformable convolution, which increases the effective receptive field and limits the sampling area by learning the position and weight of the sampling points, thereby enhancing the ability to handle geometric deformations and blur. The dynamic deformable convolution process is shown below:

[0029]

[0030] in Indicates the position of the output feature map eigenvalues ​​at that location Indicates the location of the input feature map eigenvalues ​​at that location This indicates the convolution weight at that point. Represents the convolutional spatial location matrix exist The value at that location, This indicates the offset parameter of the point's position. This indicates the offset weight of that point.

[0031] As a preferred embodiment, in S3, the DC2 module alleviates the gradient vanishing problem caused by increasing depth in deep neural networks by introducing a residual structure, thus enriching the branch during feature backpropagation.

[0032] As a preferred approach, in S4, a novel feature fusion network, SPAN, is proposed. This network enhances the utilization of shallow information, improves the model's performance in detecting small targets, reduces the number of model parameters, and optimizes the fusion between feature maps by using the CoordinateAttention mechanism.

[0033] This invention also provides a UAV traffic target detection system based on deep learning and shallow feature enhancement, including a dataset acquisition and production mechanism, a feature extraction mechanism, a feature enhancement mechanism, a feature fusion mechanism, and a target detection mechanism;

[0034] The dataset collection and production organization is used to obtain images of the front or rear of traffic targets, and to annotate and preprocess the images to obtain the dataset.

[0035] The feature extraction mechanism is used to train the dataset to obtain feature maps;

[0036] The feature enhancement mechanism is used to enhance the feature map;

[0037] The feature fusion mechanism is used to fuse information from feature maps at various scales in two parts: top-down and bottom-up. CAcat is used to fuse feature maps pairwise.

[0038] The target detection mechanism is used to optimize the bounding box loss calculation, obtain the regression loss and classification loss, and calculate the final prediction result.

[0039] Compared with existing technologies, the present invention employs the above technical solutions and has the following technical effects: Firstly, the present invention designs a DC2 (Deformable CSP Bottleneck with 2 convolutions) module composed of deformable convolutions, residual structures, and DBottleneck to enhance network features. Secondly, it increases the effective receptive field through adaptive deformable convolutions, enhancing the detection capability for small and blurred targets and reducing the false detection rate. Thirdly, it proposes a SPAN shallow feature fusion network, which, based on the PAN network, emphasizes the utilization of shallow features through attention-based feature fusion and fusion channel adjustment. Finally, it introduces the Wise-IOU method, using a dynamic non-monotonic focusing mechanism to calculate the bounding box loss, reducing the impact of low-quality dataset annotations on training and enhancing the generalization ability of the detection method. The fused feature map is input into the prediction network to obtain the final target type and location information, overcoming the problems of target deformation, blurring, and small scale in UAV photography, and improving the detection accuracy of UAVs for traffic targets. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the process of the present invention;

[0042] Figure 2 This is a schematic diagram of the S-UAV network structure of the present invention;

[0043] Figure 3 This is a schematic diagram of the DC2 module structure of the present invention;

[0044] Figure 4 This is a schematic diagram of the SPAN module structure of the present invention;

[0045] Figure 5 This is a schematic diagram of the first detection result of the present invention;

[0046] Figure 6 This is a schematic diagram of the second detection result of the present invention;

[0047] Figure 7 This is a schematic diagram of the system mechanism connection of the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] It should be noted that the structures, proportions, sizes, etc., shown in the accompanying drawings of this specification are only for the purpose of assisting those skilled in the art in understanding and reading the content disclosed in the specification, and are not intended to limit the conditions under which this application can be implemented. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in the proportions, or adjustments to the size should still fall within the scope of the technical content disclosed in this application, provided that they do not affect the effects and purposes that this application can produce.

[0050] Example

[0051] Please see Figure 1-7 This invention provides a technical solution: a method for detecting traffic targets using unmanned aerial vehicles (UAVs) based on deep learning and shallow feature enhancement, comprising the following steps:

[0052] S1. The drone collects traffic target images to obtain a dataset. The drone is used to collect traffic target images on urban roads and highways. The specific targets are pedestrians, bicycles, motorcycles, cars, buses, trucks and vans. The specific collection method is that the drone flies along the lane line, and the camera on it shoots in the same direction as the flight direction to obtain images of the front or rear of the traffic target, which can determine the direction of vehicle travel.

[0053] S2. The dataset is fed into the model for training, and the DCSP backbone network is constructed to extract image features. In S2, the dataset is fed into the model for training, and the DCSP backbone network is constructed to extract image features, obtaining 5 sets of feature maps of different sizes. This includes the following steps:

[0054] S201. Unify the size of the input image, normalize the image pixels, and arrange them in the order of CHW. Obtain the initial feature map through a Conv block containing 2D convolution, Batch Norm normalization, and SiLU activation function.

[0055] S202. The initial feature map is downsampled by multiple Conv blocks to obtain four new feature maps with decreasing size. The higher the layer, the smaller the size, which contains richer semantic information but cannot well express spatial information.

[0056] In S2, the DC2 module introduces deformable convolution, which increases the effective receptive field and limits the sampling area by learning the location and weight of sampling points, thereby enhancing the ability to handle geometric deformations and blur. The dynamic deformable convolution process is shown below:

[0057]

[0058] in Indicates the position of the output feature map eigenvalues ​​at that location Indicates the location of the input feature map eigenvalues ​​at that location This indicates the convolution weight at that point. Represents the convolutional spatial location matrix exist The value at that location, This indicates the offset parameter of the point's position. This indicates the offset weight of that point.

[0059] S3. Further Enhance Features; In S3, the DC2 module, which incorporates deformable convolutions, is used to enhance the features, including the following steps:

[0060] S301: Sampling is performed by a convolutional module or a deformable convolutional module, increasing the receptive field while reducing computational parameters;

[0061] S302: Split into two branches based on the number of channels;

[0062] S303: Branch 1 is sent to the fusion module CAcat without processing;

[0063] CAcat is an attention-based feature fusion module that further optimizes the feature fusion effect by using the CA attention mechanism on top of concat. concat adds two input feature maps along the channel number dimension, resulting in a new feature map with the same size and the same number of channels as the sum of the input feature maps. The CA attention mechanism performs average pooling and max pooling simultaneously along both channel and spatial dimensions. After regularization and activation, it is then combined with the residual channels and the weights are redistributed.

[0064] S304: Branch 2 is sent to the fusion module CAcat through a residual channel composed of multiple DCN-Bottlenecks;

[0065] S305: In branch two, a new branch is led out after each DCN-Bottleneck and sent to the fusion module CAcat;

[0066] S306: Use CAcat to merge all branches;

[0067] S307: The number of output channels is adjusted to be the same as the number of input channels by a convolutional module or a deformable convolutional module;

[0068] In S3, the DC2 module alleviates the gradient vanishing problem caused by increasing depth in deep neural networks by introducing residual structures, enriches the branches during feature backpropagation, and designs a DC2 (Deformable CSP Bottleneck with 2 convolutions) module consisting of deformable convolutions, residual structures and DBottleneck to achieve network feature enhancement.

[0069] S4 and SPAN networks fuse information from feature maps at various scales. In S4, the SPAN network is used, which is divided into two parts: top-down and bottom-up, to fuse information from feature maps at various scales. CAcat is used to fuse pairwise feature maps, and the process includes the following steps:

[0070] S401: Upsample the input feature map to obtain a feature map with the same size but different number of channels as the corresponding layer feature map in the backbone; use CAcat to fuse the obtained feature map with the corresponding feature map in the backbone; use the fused feature map to perform feature enhancement.

[0071] S402: Repeat S401 twice to obtain two sets of larger feature maps, which are then output to the medium and small target detection heads respectively;

[0072] S403: Use CAcat to fuse the feature map obtained from S401 with the SPPF output feature map, and send it to the high-level semantic information detection head;

[0073] In S4, a novel feature fusion network, SPAN, is proposed to enhance the utilization of shallow information. This improves the model's performance in detecting small targets while reducing the number of model parameters. Furthermore, by using the Coordinate Attention mechanism, the fusion between feature maps is optimized. Adaptive deformable convolutions increase the effective receptive field, enhancing the detection capability for small and blurred targets and reducing the false detection rate. Based on the PAN network, attention-based feature fusion and fusion channel adjustment are used to emphasize the utilization of shallow features.

[0074] S5 predicts the target type and location information output by the network. In S5, Wise-IoU is used to optimize the bounding box loss calculation. Three decoupled detection heads are used to calculate the regression loss and classification loss on the feature maps of three different scales output by SPAN. The final prediction result is calculated, including the target location, bounding box, target category, and confidence score. The Wise-IOU method is introduced, and a dynamic non-monotonic focusing mechanism is used to calculate the bounding box loss, which reduces the impact of low-quality dataset annotation on training and enhances the generalization ability of the detection method.

[0075] This invention also provides a UAV traffic target detection system based on deep learning and shallow feature enhancement, including a dataset acquisition and production mechanism, a feature extraction mechanism, a feature enhancement mechanism, a feature fusion mechanism, and a target detection mechanism. The actual detection effect is as follows: Figure 5 , Figure 6 As shown;

[0076] The dataset collection and production organization is used to obtain images of the front or rear of traffic targets, and then to calibrate and augment the collected images to obtain a dataset; finally, the dataset is divided into a training set and a validation set for training the detection model.

[0077] The model, composed of the feature extraction mechanism, feature enhancement mechanism, feature fusion mechanism, and target detection mechanism, has the following overall structure: Figure 2 As shown;

[0078] The feature extraction mechanism is used to train the dataset to obtain feature maps, and extracts image features by constructing a DCSP backbone network to obtain 5 sets of feature maps of different sizes.

[0079] The feature enhancement mechanism is used to enhance the feature map. It utilizes deformable convolution and residuals to enhance the detection capability of deformable targets and better adapt to changes in target scale. The specific structure is as follows: Figure 3 As shown;

[0080] The feature fusion mechanism is used to fuse information from feature maps at various scales in two parts: top-down and bottom-up. It uses CAcat to fuse pairwise feature maps, employs deeper upsampling, and fuses with feature maps of the same size in the backbone network to obtain richer shallow information, thus better enabling the detection of small targets. Simultaneously, considering the impact of deeper upsampling on model size, the subsequent downsampling fusion part is reduced to only one sampling layer, resulting in a certain optimization of the total number of model parameters compared to the original model. Furthermore, the attention-based fusion module CAcat further optimizes the fusion between feature maps, with the specific structure as follows: Figure 4 As shown;

[0081] The target detection mechanism employs a decoupled design to optimize bounding box loss calculation, obtaining regression loss and classification loss, and calculating the final prediction result. Wise-IoU is used to optimize the bounding box loss calculation. A total of three detection heads calculate the target location, bounding box, target category, and confidence score from the three feature maps output by SPAN. The fused feature map is then input into the prediction network to obtain the final target type and location information. This approach overcomes the problems of target deformation, blurring, and small scale in UAV-captured images, improving the detection accuracy of UAVs for traffic targets.

[0082] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

[0083] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting traffic targets using unmanned aerial vehicles (UAVs) based on deep learning and shallow feature enhancement, characterized in that: Includes the following steps: S1. The drone collects images of traffic targets to obtain a dataset; S2. The dataset is fed into the model for training, and the DCSP backbone network is constructed to extract image features. S3, further enhance features; S4 and SPAN networks fuse information from feature maps at various scales; S5. Predict the target type and location information output by the network; In S2, the dataset is fed into the model for training. Image features are extracted by constructing a DCSP backbone network, resulting in 5 sets of feature maps of different sizes. The process includes the following steps: S201. Unify the size of the input image, normalize the image pixels, and arrange them in the order of CHW. Obtain the initial feature map through a Conv block containing 2D convolution, Batch Norm normalization, and SiLU activation function. S202. The initial feature map is downsampled by multiple Conv blocks to obtain four new feature maps with decreasing size. The higher the layer, the smaller the size, which contains richer semantic information, but cannot well express spatial information. In S3, the DC2 module, which incorporates deformable convolutions, is used to enhance features, and includes the following steps: S301: Sampling is performed by a convolutional module or a deformable convolutional module, increasing the receptive field while reducing computational parameters; S302: Split into two branches based on the number of channels; S303: Branch 1 is sent to the fusion module CAcat without processing; S304: Branch 2 is sent to the fusion module CAcat through a residual channel composed of multiple DCN-Bottlenecks; S305: In branch two, a new branch is led out after each DCN-Bottleneck and sent to the fusion module CAcat; S306: Use CAcat to merge all branches; S307: The number of output channels is adjusted to be the same as the number of input channels by a convolutional module or a deformable convolutional module; In S4, the SPAN network is used, which is divided into two parts: top-down and bottom-up, to fuse information from feature maps at various scales. CAcat is used to fuse pairwise feature maps, and the steps include: S401: Upsample the input feature map to obtain a feature map with the same size as the corresponding layer feature map in the backbone but a different number of channels; The obtained feature map is fused with the corresponding feature map of the backbone using CAcat; feature enhancement is then performed on the fused feature map. S402: Repeat S401 twice to obtain two sets of larger feature maps, which are then output to the medium and small target detection heads respectively; S403: The feature map obtained from S401 is fused with the SPPF output feature map using CAcat and then sent to the high-level semantic information detection head.

2. The UAV traffic target detection method based on deep learning and shallow feature enhancement according to claim 1, characterized in that: In S5, Wise-IoU is used to optimize the bounding box loss calculation. Three decoupled detection heads are used to calculate the feature maps of three different scales output by SPAN to obtain the regression loss and classification loss. The final prediction result is then calculated, including the target location, bounding box, target category, and confidence score.

3. The UAV traffic target detection method based on deep learning and shallow feature enhancement according to claim 2, characterized in that: In S2, the DC2 module introduces deformable convolution, which increases the effective receptive field and limits the sampling area by learning the location and weight of sampling points, thereby enhancing the ability to handle geometric deformations and blur. The dynamic deformable convolution process is shown below: Where y(p) represents the feature value at position p in the output feature map, x(p) represents the feature value at position p in the input feature map, wi represents the convolution weight at that point, and ki represents the value of the convolution spatial position matrix K at position i.

4. The UAV traffic target detection method based on deep learning and shallow feature enhancement according to claim 3, characterized in that: In S3, the DC2 module alleviates the gradient vanishing problem caused by increasing depth in deep neural networks by introducing residual structures, and enriches the branches during feature backpropagation.

5. The UAV traffic target detection method based on deep learning and shallow feature enhancement according to claim 4, characterized in that: In S4, a novel feature fusion network, SPAN, is proposed, which can enhance the utilization of shallow information, improve the model's performance in detecting small targets, reduce the number of model parameters, and optimize the fusion between feature maps by using the CoordinateAttention mechanism.

6. A UAV traffic target detection system based on deep learning and shallow feature enhancement, as described in any one of claims 1-5, is characterized in that: This includes dataset collection and production organizations, feature extraction organizations, feature enhancement organizations, feature fusion organizations, and object detection organizations; The dataset collection and production organization is used to obtain images of the front or rear of traffic targets, and to annotate and preprocess the images to obtain the dataset. The feature extraction mechanism is used to train the dataset to obtain feature maps; The feature enhancement mechanism is used to enhance the feature map; The feature fusion mechanism is used to fuse information from feature maps at various scales in two parts: top-down and bottom-up. CAcat is used to fuse feature maps pairwise. The target detection mechanism is used to optimize the bounding box loss calculation, obtain the regression loss and classification loss, and calculate the final prediction result.

Images