An aerial image rotating target detection method based on annular smooth label
By using circular smooth labels and global context blocks to process target angles in aerial images, the YOLOv5s network is improved, solving the angle boundary problem of target detection in aerial images and improving recognition accuracy and feature extraction capabilities.
Patent Information
- Application Number
- CN202310316045.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-28
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2043-03-28
AI Technical Summary
Target detection in aerial images faces challenges such as difficulty in extracting angular features and boundary issues caused by target rotation. Existing algorithms do not achieve ideal accuracy in UAV aerial images.
A rotating target detection algorithm is constructed by using circular smooth labels (CSL) to process the target angle, combining global context blocks (GCBlock) to extract features, and using lightweight convolution GSConv in the model to improve the YOLOv5s network model.
It improves the recognition rate of target detection in aerial images, reduces computational costs and model parameters, solves the angle boundary problem, and enhances the accuracy of feature extraction and recognition.
Smart Images
Figure CN116597324B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of image processing and relates to a kind of aerial image rotating target detection method based on annular smooth label. BACKGROUND
[0002] For city management, traffic road vehicles are the focus, and vehicle detection based on unmanned aerial vehicle aerial image greatly increases the efficiency of traffic management departments. However, in actual application scenarios, aerial images usually have angles, which makes the accuracy of traditional target detection and recognition not very ideal.
[0003] Although the current one-stage target detection algorithm based on YOLO series has good performance in natural scenes, the characteristics of targets in unmanned aerial vehicle aerial images are quite different from those in natural scene pictures. The images collected by high-altitude unmanned aerial vehicles generally have the characteristics of difficult data set production, special target scale, and rotating targets. At the same time, the targets in aerial images are small and may be occluded, making feature extraction difficult. In some angle regression-based rotating target detection algorithms, the ideal prediction result may exceed the defined range, resulting in boundary problems and large losses. SUMMARY
[0004] Therefore, the purpose of the present application is to provide a kind of aerial image rotating target detection method based on annular smooth label. A kind of aerial image rotating target detection algorithm is proposed based on YOLOv5s network model, so as to solve the boundary problem of angle and improve the recognition rate. This method mainly uses circular smooth label (CSL) to process the target angle, uses the periodicity of the real box label according to the periodicity of the angle θ by using the Gaussian window function, divides the angle range into [-90, 90) by using the classification method, and uses the global context block (GCBlock) method to obtain the global context information, fully extracts the features and fuses them into the channel to improve the performance, captures the long-range dependency, and reduces the context feature redundancy. In order to avoid the increase of calculation cost and the influence on the convergence of the model caused by too many model parameters, a lightweight convolution GSConv is used in the neck network to replace the standard convolution, which reduces the operation cost, but its contribution to the learning ability of the model is not inferior to that of the standard convolution.
[0005] To achieve the above purpose, the present application provides the following technical scheme:
[0006] A kind of aerial image rotating target detection method based on annular smooth label, the method comprises the following steps:
[0007] S1: data preprocessing is performed on the images in the data set, and the training image label is processed into a format suitable for rotating target detection;
[0008] S2: Data augmentation is performed on the image processed by the label to enhance the robustness of the network;
[0009] S3: The backbone network part of the improved YOLOv5s model is optimized, which first passes through two Conv modules with a kernel size of 3x3 and a step of 2, and the feature map is reduced by 2 times; then passes through the first C3_GC module, wherein the C3 module is composed of two branch Concat, branch one is a Conv module, and branch two is composed of a Conv module and n BottleNeck modules, and the GC module is added to branch one of C3; the third to seventh layers are continuous Conv modules and C3 modules, the eighth layer is a C3_GC module, and the ninth layer is a SPPF module, which realizes the fusion of local features and global features at the feature Map level;
[0010] S4: The FPN structure of the neck network of the model is constructed, which is composed of GSConv module, up-sampling unit, Concat operation and C3 module, and after two times of up-sampling, P3 and P4 level feature maps are obtained, then Concat operation is performed with C3 and C4 level feature maps, the high layer feature is fused with the low layer feature through up-sampling, and the semantic feature of the high layer is transmitted; wherein GSConv uses shuffle to penetrate the information generated by standard convolution into each part of the information generated by depth separable convolution, so that the output of calculation is close to standard convolution;
[0011] S5: The PANet structure of the neck network of the model is constructed, which is a bottom-up feature pyramid structure, composed of GSConv module, Concat operation and C3 module, and is fused with the feature maps of the tenth layer and the fourteenth layer, the positioning information of the shallow layer is transmitted to the deep layer, and three scale feature maps of 108x108, 54x54 and 27x27 are obtained;
[0012] S6: The Head part of the model is constructed, which is composed of 3 Detect detection layers, only one rotation angle prediction channel θ needs to be added in the Head structure to realize the prediction of the rotating frame; in the forward operation, the network receives three feature maps of different scales, and grid prediction is performed on the three feature maps in turn, and the feature output is obtained by using convolution operation, then the feature map after the detector convolution is divided into grids, the size of the grid is the same as the input size, such as 54x54 feature map will become 54x54 grid, so one grid corresponds to 32x32 pixels in the original image, and so on; then x and y are predicted, and then w and h are predicted;
[0013] S7: The loss function part of the model is constructed; Loss=αL cls +βL obj +ηLloc + λL theta , where L cls is BCELoss, only calculate the classification loss of positive samples; L obj is still BCE Loss, note that this time the obj is the CIOU of the network predicted target bounding box and the real bounding box, calculate the obj loss of all samples; L loc uses CIOU Loss, only calculates the positioning loss of positive samples; L theta still uses BCE Loss;
[0014] S8: input the picture into the target detection network for training and output the detection result;
[0015] S9: use the trained target detection network in the actual scene.
[0016] Optionally, in the S1, the data preprocessing of the training image specifically includes: converting the data label from the poly format {(x1, y1), (x2, y2), (x3, y3), (x4, y4)} into the (x c , y c , w, h, θ) format, wherein w represents the longest side in the envelope box, h represents the other side corresponding to w, and θ represents the included angle between w and the x-axis counterclockwise; and then using the CSL to process the label in the data set, and transmitting the angle information in the label into a Gaussian function to obtain a Gaussian function smoothing label.
[0017] Optionally, in the S2, the data augmentation of the image specifically includes: modifying the three data augmentation methods of mosaic, random_perspective and ordinary data augmentation by adding θ dimension, and setting the perspective in the random_perspective to 0 in the hyperparameter.
[0018] Optionally, in the S3, the GC module includes three processes:
[0019] (a) using 1x1 convolution and softmax function to obtain attention weight, and then performing attention pooling to obtain global context feature;
[0020] (b) performing feature transformation through a bottleneck transform module;
[0021] (c) using addition to aggregate the global context feature to the feature of each position; this abstract definition is where ∑ j α j x jdenotes a global context modeling module that combines features of all positions together by a weighted average with weight j , and the detailed formula of the global context (GC) block is where i is the index of the query position, j enumerates all possible positions, W K and W v denote linear transformation matrices, is the weight of the global attention pool, and v2 ReLU(LN(W v1 (·))) denotes a bottleneck transform.
[0022] The present application has the following advantages:
[0023] (1) Since the aerial image target has a rotation angle, and the target with a large aspect ratio is very sensitive to the change of the angle, the current popular angle prediction method based on regression has more or less boundary problems, and the ideal prediction result exceeds the defined range, resulting in a large loss value. Therefore, the angle regression method is converted into a classification form, because the classification result is limited, and it will not appear outside the defined range. Inspired by this, the present application encodes the angle information into the CSL format, because of the setting of its window function, the model can measure the angle distance between the predicted label and the real box label, and the periodicity problem is solved by introducing periodicity.
[0024] (2) Due to the particularity of the high-altitude perspective of the aerial image, the image size is generally large, and the target in an image accounts for a small proportion of the whole image, and the feature extraction is difficult due to occlusion, which is prone to cause missed detection and false detection. By introducing a global context module (GCBlock) in the feature extraction part, global context information is obtained, and it is fused into the channel to improve the performance, and the long-range dependency relationship is captured, and the GC module adopts a bottleneck transform to reduce the redundancy in the global context features, thereby further reducing the number of parameters and FLOPs. Further use GSConv instead of Conv module in the neck network, while the contribution of model learning ability is not superior to that of standard convolution, the parameter amount is reduced.
[0025] Other advantages, objects, and features of the present application will be apparent to those skilled in the art from the following specification, and it is intended to be covered by the following claims. The objects and other advantages of the present application can be realized and obtained by the following specification. Attached Figure Description
[0026] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:
[0027] Figure 1 This is a flowchart of the present invention;
[0028] Figure 2 Here is a diagram of the C3_GC structure;
[0029] Figure 3 Here is a diagram of the GC Block structure;
[0030] Figure 4 This is a diagram of the improved algorithm structure. Detailed Implementation
[0031] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.
[0032] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0033] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.
[0034] The application is a kind of aerial image rotating target detection method, which mainly improves the rotating target detection precision and detection speed in the current aerial scene, and the flow chart is as shown in Figure 1 The specific steps include the following steps:
[0035] S1: data preprocessing is performed on the images in the data set, and the training images are processed into a format suitable for rotating target detection.
[0036] Further, in the step S1, the data preprocessing of the training images specifically includes: converting the data label from the poly format {(x1, y1), (x2, y2), (x3, y3), (x4, y4)} to the (x c ,y c ,w,h,θ) format, wherein w represents the longest side in the envelope box, h represents the other side corresponding to w, and θ represents the included angle between w and the x-axis counterclockwise; and then using CSL to process the label in the data set, and transmitting the angle information in the label into a Gaussian function to obtain a Gaussian function smoothing label. The mathematical expression of CSL is:
[0037]
[0038] In the above formula, g(x) is a window function, and the specific expression is r is the window radius, θ is the angle of the envelope box, x is the horizontal coordinate within the window range, and g(x) satisfies periodicity, symmetry, monotonicity and maximum g(x) = 1.
[0039] S2: data augmentation is performed on the images after label processing to enhance the robustness of the network.
[0040] Further, in the step S2, the data augmentation of the images specifically includes: modifying the three data augmentation methods of mosaic, random_perspective and ordinary data augmentation by adding θ dimension, and the perspective in random_perspective will cause the change of the target angle, so it is set to 0 in the hyperparameter.
[0041] S3: the training images are input into a feature extraction module, wherein the first two layers are two Conv modules with a kernel size of 3x3 and a step size of 2, and the feature map is reduced by 2 times; then passing through a first C3_GC module, the structure is as shown in Figure 2 The GC module can obtain global context information and fuse it into the channel to improve the performance and capture long-range dependencies, and the GC module adopts bottleneck transform to reduce the redundancy in the global context features, thereby further reducing the number of parameters and FLOPs, and the structure is as shown in Figure 3The C3 module is composed of two branches, branch one is a Conv module, and branch two is composed of a Conv module and n BottleNeck modules, and the GC module is added in branch one of C3. The third to seventh layers are continuous Conv modules and C3 modules, the eighth layer is a C3_GC module, and the ninth layer is an SPPF module, realizing the fusion of local features and global features at the feature Map level.
[0042] Further, the GC module in step S3 can be abstracted into three processes: (a) using a 1x1 convolution and a softmax function to obtain attention weights, and then performing attention pooling to obtain global context features; (b) performing feature transformation through a bottleneck transform module; (c) using addition to aggregate global context features to the features of each position. This abstract definition is where ∑ j α j x j represents a global context modeling module, which combines the features of all positions together to obtain global context features through weighted average of weights α j , and δ(·) represents a feature transformation that captures channel relationships, and F(,·,) represents a feature fusion function that aggregates global context features to the features of each position. The detailed formula of the global context (GC) block is where i is the index of the query position, j enumerates all possible positions, W K and W v are linear transformation matrices, is the weight of the global attention pool, δ(·) = W v2 ReLU(LN(W v1 (·))) represents a bottleneck transform.
[0043] S4: Construct the FPN structure of the model neck network, and perform upsampling from top to bottom, which is composed of a GSConv module, an upsampling unit, a Concat operation, and a C3 module. The input of this part is the feature after the SPPF module processing, which is Concat operated with the C3 and C4 level feature maps after two times of upsampling, fuses the high-level features through upsampling and low-level features, and transmits the semantic features of the high-level.
[0044] S5: PANet structure of the model neck network is a bottom-up feature pyramid structure that transmits the positioning information of the shallow layer to the deep layer, which is composed of GSConv module, Concat operation and C3 module. The input of this part is a feature map with a size of 108x108 level, which is processed by GSConv module, then Concat operation is performed with the feature map output by the fourteenth layer, and C3 module is entered to obtain a feature map with a size of 54x54 level. The same processing is performed on the feature map to obtain a feature map with a size of 27x27 level.
[0045] S6: The Head part of the model is composed of 3 Detect detection layers, and only one rotation angle prediction channel θ needs to be added at the Head structure to realize the prediction of the rotated frame. In the forward operation, the network receives three feature maps of different scales, and grid prediction is performed on the three feature maps in turn. The feature output is obtained by using convolution operation, and then the feature map after detector convolution is divided into grids, and the size of the grid is the same as the input size, such as 54x54 feature map will become 54x54 grid, then one grid corresponds to 32x32 pixels in the original image, and so on; then x and y are predicted, and then w and h are predicted.
[0046] S7: The loss function part of the model is constructed. After matching the positive and negative samples, the loss is calculated for back propagation to update the gradient, and the mathematical expression of the loss function is as follows: Loss = aL cls + βL obj + ηL loc + λL theta , wherein L cls is BCELoss, which only calculates the classification loss of the positive sample; L obj still uses BCE Loss, note that obj is the CIOU of the target bounding box predicted by the network and the real bounding box, and the obj loss of all samples is calculated at this time; L loc uses CIOU Loss, which only calculates the positioning loss of the positive sample; L theta still uses BCE Loss, which also only calculates the angle loss of the positive sample.
[0047] S8: The picture is input into the target detection network for training and output of the detection result, including object category, coordinate position and confidence. The overall framework of the network is shown in Figure 4 .
[0048] S9: The trained target detection network is used in the actual scene.
[0049] The embodiments are performed on the Drone Vehicle dataset. In addition, the radius of the window function in the embodiments is set as csl_radius = 2.0, and the size of all pictures is adjusted to 864x864 pixels.
[0050] In the present experiment, the accuracy of the detection algorithm is tested in three actual situations selected from the Drone Vehicle dataset, namely, dim situation, occlusion situation, and crowded arrangement situation.
[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions, which should be covered in the scope of the claims of the present application.
Claims
1. A method for detecting rotating targets in aerial images based on circular smooth labels, characterized in that: The method includes the following steps: S1: Perform data preprocessing on the images in the dataset, converting the training image labels into a format suitable for rotated object detection. Specifically, the data preprocessing includes converting the data labels from the poly format {(x1,y1),(x2,y2),(x3,y3),(x4,y4)} to (x...). c ,y c The format is (w,h,θ), where w represents the longest side in the envelope, h represents the other side corresponding to w, and θ represents the angle between w and the x-axis counterclockwise. Then, CSL is used to process the labels in the dataset, and the angle information in the labels is passed into a Gaussian function to obtain Gaussian smoothed labels. S2: Perform data augmentation on the labeled images to enhance the robustness of the network; S3: The backbone network of the YOLOv5s model is optimized and improved. First, it passes through two Conv modules with a kernel size of 3×3 and a stride of 2, reducing the feature map size by a factor of 2. Then, it passes through the first C3 global attention (GC) module, which consists of two branches: one is a Conv module, and the other is composed of a Conv module and n BottleNeck modules. The GC module is added to the first branch of C3. The third to seventh layers are continuous Conv and C3 modules, the eighth layer is the C3_GC module, and the ninth layer is the SPPF module, which realizes the fusion of local and global features at the feature map level. S4: Construct the FPN structure of the model's neck network, consisting of the GSConv module, upsampling unit, Concat operation, and C3 module. After two upsampling operations, feature maps at the P3 and P4 levels are obtained respectively. Then, the Concat operation is performed with the C3 and C4 level feature maps respectively to fuse high-level features with low-level features through upsampling, thus passing on the semantic features of the high-level features. Among them, GSConv uses shuffle to permeate the information generated by standard convolution into each part of the information generated by depthwise separable convolution, so that its calculated output is close to that of standard convolution. S5: The PANet structure for constructing the model's neck network is a bottom-up feature pyramid structure, consisting of the GSConv module, the Concat operation, and the C3 module. It is fused with the feature maps of the tenth and fourteenth layers to pass the localization information from the shallow layers to the deep layers, resulting in feature maps of three scales: 108×108, 54×54, and 27×27. S6: The head part of the model is constructed, consisting of 3 detection layers. A rotation angle prediction channel θ is added to the head structure to predict the rotation box. In the forward operation, the network receives 3 feature maps of different scales and performs gridded prediction on these 3 feature maps in sequence. The feature output is obtained by using convolution operation. Then, the feature map after convolution by the detector is divided into grids with the same size as the input size. Then, x and y are predicted, and w and h are predicted. S7: The loss function part of constructing the model; Loss = αL cls +βL obj +ηL loc +λL theta L cls For BCE Loss, only the classification loss for positive samples is calculated; L obj We still use BCE Loss, note that here obj is the CIOU between the network-predicted target bounding box and the ground truth bounding box, and calculate the obj loss for all samples; L loc CIOU Loss is used, and only the localization loss of positive samples is calculated; L theta It still uses BCE Loss; S8: Input the image into the object detection network for training and output the detection results; S9: Apply the trained object detection network to real-world scenarios.
2. The method for detecting rotating targets in aerial images based on circular smooth labels according to claim 1, characterized in that: In S2, image data augmentation specifically includes adding a θ dimension to three data augmentation methods: mosaic, random_perspective, and ordinary data augmentation. The perspective in random_perspective will cause a change in the target angle, and it is set to 0 in the hyperparameter.
3. The method for detecting rotating targets in aerial images based on circular smooth labels according to claim 2, characterized in that: In S3, the GC module includes three processes: (a) Attention weights are obtained by using 1×1 convolution and softmax function, and then attention pooling is performed to obtain global context features; (b) Perform feature transformation using the bottleneck transform module; (c) Use addition to aggregate global context features into features at each location; define this abstraction as... Where ∑ j α j x j This represents the global context modeling module, which uses weights α. j The weighted average combines the features of all positions to obtain global context features, δ(·) represents the feature transformation that captures channel relationships, and F(,·,) represents the feature fusion function that aggregates global context features to each position; The detailed formula for the Global Context (GC) block is as follows: Where i is the index of the query position, j enumerates all possible positions, and W... K and W v Represents a linear transformation matrix. These are the weights of the global attention pool, δ(·)=W v2 ReLU(LN(W v1 (·))) represents bottleneck transform.