A real-time vehicle detection method using feature processing and IoU strategy

Abstract

The present invention discloses a real-time vehicle detection method using feature processing and IoU strategy, comprising: obtaining an image to be detected including vehicle information; inputting the image to be detected into a vehicle detection model generated using feature processing and IoU strategy, and outputting a vehicle detection result. The method has higher accuracy in vehicle detection in traffic scenes, significantly improves the detection effect of the model on vehicles that are difficult to identify and easily confused to classify, and improves the problem of small targets and vehicle occlusion to a certain extent.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a real-time vehicle detection method using feature processing and IoU strategy. Background Art

[0002] With the development of the Internet of Things and big data, the demand for intelligent transportation systems in the information age is becoming more and more urgent. The intelligent transportation system (ITS) is composed of such algorithm systems that integrate informatization, intelligence, and socialization. As an important part of the intelligent transportation system, vehicle detection is of great significance to urban planning, traffic safety, accident analysis, etc., and promotes the development of smart cars and smart travel in the future. Creating a comprehensive, efficient and robust ITS and putting it into use is one of the core issues in the field of modern transportation and urban development.

[0003] However, in modern cities with growing vehicles, technology and population, real-time monitoring of traffic intersections is a challenging problem. In scenarios with large numbers of vehicles and frequent traffic jams, traditional methods such as radar and laser ranging will become unreliable. These methods not only have major defects in real-time and effectiveness, but they may also require human intervention to complete the operation. Nowadays, these decisions that rely on human professional experience can be digitized by big data systems and then judged by computers. Expensive, cumbersome and inefficient steps are eliminated. These image processing-based methods are now combined with emerging artificial intelligence (AI) technologies, especially image processing methods based on deep learning. This type of technology that makes decisions based on computer data has led to great changes in many fields, including transportation systems, and has made current transportation methods safer, smarter and more efficient. In recent years, with the development of deep learning technology, convolutional neural networks (CNNs) have become the research focus in the field of computer vision (CV) with their powerful adaptive learning capabilities. Its efficient feature extraction has changed the fundamental idea of ​​target detection. In the field of transportation, image processing based on deep learning provides strong support for vehicle detection and has become the mainstream algorithm in this field. This type of detection task mainly revolves around two subtasks: location positioning and vehicle classification. Classic algorithms such as the two-stage algorithm RCNN() series usually have high accuracy in detection tasks; lightweight and fast one-stage algorithms YOLO series, SSD, etc. Due to the need for real-time vehicle detection in traffic scenarios, and considering the efficiency and versatility of the algorithm, the YOLO series of algorithms can better meet the task requirements. It can complete the prediction task of multiple target locations and their categories through a single regression operation. In addition, this series of algorithms has been gradually improved by the community over the past few years and has been widely used in detection tasks in various fields.

[0004] Although deep learning-based methods have achieved remarkable results in the field of vehicle detection, even relatively complete neural network models will face some challenges considering the complex and changeable traffic scene environment. First, in scenes with dense traffic, the overlap and occlusion of vehicles are serious, resulting in reduced detection accuracy. Secondly, for small vehicles at a long distance, detection is difficult and they are easily ignored by the model. Furthermore, large vehicles have variable shapes, and the appearance characteristics of special vehicles for different tasks vary greatly. In addition, environmental factors such as lighting, shadows, similar backgrounds, etc. will interfere with the detector to a certain extent. In practical applications, the diversity of scenes, the distribution of vehicle groups, traffic congestion, etc. are more likely to amplify these negative effects. Therefore, accurate, efficient and robust detectors are particularly important in ITS systems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems in the above-mentioned technologies to a certain extent. To this end, the purpose of the present invention is to propose a real-time vehicle detection method using feature processing and IoU strategy, which has higher accuracy in vehicle detection in traffic scenes, significantly improves the detection effect of the model on vehicles that are difficult to identify and easily confused to classify, and improves the problem of small targets and vehicle occlusion to a certain extent.

[0006] To achieve the above object, an embodiment of the present invention proposes a real-time vehicle detection method using feature processing and IoU strategy, including:

[0007] Acquire an image to be detected including vehicle information;

[0008] The image to be detected is input into the vehicle detection model generated by feature processing and IoU strategy, and the vehicle detection result is output.

[0009] According to some embodiments of the present invention, the method of acquiring the image to be detected including the vehicle information includes: a direct acquisition method and an indirect acquisition method;

[0010] The direct acquisition method is to obtain images from a preset image database or to shoot images on site;

[0011] The indirect acquisition method is to obtain a video from a preset video database or shoot a live video, perform image frame processing on the obtained video, and obtain the image to be detected.

[0012] According to some embodiments of the present invention, a method for obtaining a vehicle detection model includes:

[0013] Set up the feature extraction module to perform multi-scale extraction and fusion of vehicle features based on the Sim_DFC module;

[0014] Set up a bounding box optimization module to optimize the bounding box regression of the vehicle by combining Shape-IoU and Inner-IoU;

[0015] Set up a feature fusion module to perform multi-scale fusion and transfer of vehicle features based on the improved BiFPN structure;

[0016] Determine an initial vehicle detection model based on a feature extraction module, a bounding box optimization module, and a feature fusion module;

[0017] A sample data set is obtained, and an initial vehicle detection model is trained based on the sample data set to obtain a vehicle detection model.

[0018] According to some embodiments of the present invention, a feature extraction module is provided, including:

[0019] The feature extraction module includes three parallel convolutional layers, which are identified as the first convolutional layer, the second convolutional layer and the third convolutional layer from left to right; the third convolutional layer is 3×3 in size, and the first convolutional layer and the second convolutional layer are 1×1 in size;

[0020] Based on SimAM attention, the features determined by the first convolutional layer are weighted, and the information focus is found from both spatial and channel aspects to highlight important features;

[0021] The important features are output by the sigmoid function, and the importance of the important features is distributed in the weighted feature map with a weight of 0-1 and passed to the right;

[0022] In the feature fusion part on the right, the features extracted by Conv, Ghost, and DSConv are fused with the weighted feature map on the left and then sent to the EMA attention mechanism again to balance the learning of channel and spatial attention and enhance the ability to capture key features;

[0023] Obtain multiple processed features and concatenate them, combine the multi-dimensionality of the attention mechanism with the multi-scale of the features, and generate the output through the fourth convolutional layer.

[0024] According to some embodiments of the present invention, setting a bounding box optimization module includes:

[0025] Introducing Shape-IoU, which calculates the loss by focusing on the shape and scale of the bounding box itself;

[0026] Inner-IoU is introduced to speed up the regression process by using auxiliary bounding boxes and scale factor ratios;

[0027] Combining the two, the auxiliary mechanism of Inner-IoU is used to enhance the learning ability of Shape-IoU.

[0028] According to some embodiments of the present invention, a formula combining Shape-IoU and Inner-IoU includes:

[0029]

[0030]

[0031]

[0032]

[0033]

[0034] Loss Shape-IoU =1-IoU+distance shape +0.5×Ω shape

[0035]

[0036] Among them, the GT box and bounding box are represented as b gt and b; IoU represents the intersection-over-union ratio of the two; c represents the length of the minimum hypotenuse of the detection box between the bounding box and the GT box; w gt 、h gt represents the width and height of the GT box; ww and hh are the weight coefficients of the introduced width and height, which are affected by the shape and size of the bounding box; scale is the scale factor, which is set manually; Ω shape Indicates scale penalty; distance shape represents shape loss; and is the center point coordinate of the GT box, and x c and c represents the center coordinates of the anchor box; w and h represent the width and height of the anchor box; substituting w and h into w t t in the equation, we get w w and w h , w w and w h is the ratio of the difference between the width and height weights; Loss Shape-IoU Represents Shape-IoU loss;

[0037]

[0038]

[0039]

[0040]

[0041]

[0042] union=(w gt xh gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter

[0043]

[0044] in, Respectively represent the first calculation parameter, the second calculation parameter, the third calculation parameter and the fourth calculation parameter of the GT frame; b l 、b r 、b t 、b b They represent the first, second, third and fourth calculation parameters of the bounding box respectively; ratio corresponds to the scale transformation factor; inter represents the screening parameters of the GT box and the bounding box; union represents the difference parameters between the GT box and the bounding box; IoU inner Represents Inner-IoU loss;

[0045] Combine Shape-IoU with Inner-IoU to get the combined loss function Loss Inner-Shape IoU ;

[0046] Loss Inner-Shape IoU =Loss Shape-IoU +IoU-IoU inner

[0047] Determine the loss function included in the bounding box optimization module.

[0048] According to some embodiments of the present invention, the feature fusion module includes:

[0049] The BiFPN module replaces the original ELAN-T module. The BiFPN module removes the module nodes whose fusion contribution is less than the preset threshold on the lateral edge side of the original ELAN-T module. The BiFPN module creates a diagonal path to realize a simplified bidirectional fusion network. For the non-edge intermediate layer, the BiFPN module adds an edge from the original input node to the output node, and simultaneously combines the multi-semantic information of the output node in the transmission.

[0050] According to some embodiments of the present invention, the sample datasets are the UA-DERTAC dataset and the Visdrone2019 dataset.

[0051] The present invention proposes a real-time vehicle detection method using feature processing and IoU strategy, which has higher accuracy in vehicle detection in traffic scenes, significantly improves the detection effect of the model on vehicles that are difficult to identify and easily confused for classification, and improves the problem of small targets and vehicle occlusion to a certain extent.

[0052] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0053] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0055] Figure 1 is a flow chart of a vehicle real-time detection method using feature processing and IoU strategy according to an embodiment of the present invention;

[0056] Figure 2 is a network structure diagram of a vehicle detection model according to an embodiment of the present invention;

[0057] Figure 3 is a structural diagram of a feature extraction module according to an embodiment of the present invention;

[0058] Figure 4 is a structural diagram of the SimAM attention mechanism according to an embodiment of the present invention;

[0059] Figure 5 is a structural diagram of an EMA attention mechanism according to an embodiment of the present invention;

[0060] Figure 6 is a schematic diagram of the Shape-IoU principle according to an embodiment of the present invention;

[0061] Figure 7 is a comparison diagram between BiFPN and PANet according to an embodiment of the present invention;

[0062] Figure 8(a)-Figure 8(d) is a comparison diagram of detection effects of captured images according to an embodiment of the present invention;

[0063] Fig. 9 This is the detection effect on Visdrone2019 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] like Figure 1 As shown, the embodiment of the present invention proposes a real-time vehicle detection method using feature processing and IoU strategy, including steps S1-S2:

[0066] S1. Acquire an image to be detected including vehicle information;

[0067] S2. Input the image to be detected into the vehicle detection model generated by feature processing and IoU strategy, and output the vehicle detection result.

[0068] The beneficial effects of the above technical solution are: higher accuracy in vehicle detection in traffic scenes, significantly improving the model's detection effect on vehicles that are difficult to identify and easily confused for classification, and to a certain extent improving the problem of small targets and vehicle occlusion.

[0069] According to some embodiments of the present invention, the method of acquiring the image to be detected including the vehicle information includes: a direct acquisition method and an indirect acquisition method;

[0070] The direct acquisition method is to obtain images from a preset image database or to shoot images on site;

[0071] The indirect acquisition method is to obtain a video from a preset video database or shoot a live video, perform image frame processing on the obtained video, and obtain the image to be detected.

[0072] According to some embodiments of the present invention, a method for obtaining a vehicle detection model includes:

[0073] Set up the feature extraction module to perform multi-scale extraction and fusion of vehicle features based on the Sim_DFC module;

[0074] Set up a bounding box optimization module to optimize the bounding box regression of the vehicle by combining Shape-IoU and Inner-IoU;

[0075] Set up a feature fusion module to perform multi-scale fusion and transfer of vehicle features based on the improved BiFPN structure;

[0076] Determine an initial vehicle detection model based on a feature extraction module, a bounding box optimization module, and a feature fusion module;

[0077] A sample data set is obtained, and an initial vehicle detection model is trained based on the sample data set to obtain a vehicle detection model.

[0078] The working principle and beneficial effects of the above technical solution: The feature extraction module deeply extracts image information and pays extra attention to vehicle information, deepening the focus on vehicles and their traffic characteristics. The efficient feature fusion structure of the feature fusion module makes a more perfect splicing of these information-rich feature maps, realizes the diversification of vehicle feature information, and further improves the expression of vehicle features; based on the bounding box optimization module, the bounding box regression of the vehicle is optimized through the strategy of combining Shape-IoU and Inner-IoU; it enables better allocation of detection frames and accelerates the convergence speed of model training; the improved IoU loss strategy makes the prediction frame more stable and reduces the impact caused by overlapping occlusion of vehicles or small targets. The initial vehicle detection model is trained based on the sample data set to obtain a vehicle detection model. After multiple trainings, the best model weight of the training is retained after the loss value stops decreasing, and the best weight is selected as the final high-precision vehicle detection model. Based on the vehicle detection model, the problem of difficult to identify vehicles and overlapping occlusion of vehicles is solved to a certain extent, and the detection of vehicles is more accurate.

[0079] According to some embodiments of the present invention, a feature extraction module is provided, including:

[0080] The feature extraction module includes three parallel convolutional layers, which are identified as the first convolutional layer, the second convolutional layer and the third convolutional layer from left to right; the third convolutional layer is 3×3 in size, and the first convolutional layer and the second convolutional layer are 1×1 in size;

[0081] Based on SimAM attention, the features determined by the first convolutional layer are weighted, and the information focus is found from both spatial and channel aspects to highlight important features;

[0082] The important features are output by the sigmoid function, and the importance of the important features is distributed in the weighted feature map with a weight of 0-1 and passed to the right;

[0083] In the feature fusion part on the right, the features extracted by Conv, Ghost, and DSConv are fused with the weighted feature map on the left and then sent to the EMA attention mechanism again to balance the learning of channel and spatial attention and enhance the ability to capture key features;

[0084] Obtain multiple processed features and concatenate them, combine the multi-dimensionality of the attention mechanism with the multi-scale of the features, and generate the output through the fourth convolutional layer.

[0085] The working principle and beneficial effects of the above technical solution are as follows: three parallel convolutional layers are set, which are determined as the first convolutional layer, the second convolutional layer and the third convolutional layer from left to right; among them, the third convolutional layer is 3×3 in size, and the first convolutional layer and the second convolutional layer are 1×1 in size; this structure is used to capture different levels and detail information of the input features, and the SimAM attention is used to weight the leftmost features, and the information focus is found from the two aspects of spatial channel to highlight important features and suppress redundant information. These features will be output through the sigmoid function, and the importance of the features will be distributed in the feature map with a weight of 0-1 and passed to the right. In the feature fusion part on the right, the features extracted by Conv, Ghost, and DSConv are fused with the weighted feature map on the left and then sent to the EMA attention mechanism again to balance the learning of channel and spatial attention to enhance the ability to capture key features. Multiple processed features are obtained and spliced, and the output is generated through the fourth convolutional layer by combining the multi-dimensionality of the attention mechanism and the multi-scale of the features. Figure 2 As shown in the figure, the input image of the backbone network is 640×640, 3 channels, and the feature extraction part is implemented by the Sim_DFC module. After MP downsampling and SPP-T, it is input into the neck network. Figure 2 In the figure, CBS is the convolution-batch normalization-activation function step, which is a simple feature extraction of the network or a change in the number of channels, some of which are accompanied by downsampling functions; MP is the pooling step, which mainly plays a role in downsampling; SPP-T is a faster spatial pyramid pooling, which is used to increase the receptive field and adapt the algorithm to images of different resolutions; Upsample is upsampling; Cat is the splicing feature map operation; ELAN-T is the feature extraction module of the original YOLOv7-tiny algorithm, and it is not replaced by Sim_DFC in the neck network because it avoids the increase of unnecessary parameters; Head is the detection head output by the model.

[0086] Among them, Figure 3-5 As shown in the figure, the module emphasizes multi-scale features by channel grouping. SimAM, EMA attention and Sigmoid function are used to better adjust important features from multiple dimensions, thus enhancing the effect of convolution. The addition of Ghost convolution and depthwise separable convolution further enhances the representation of features while greatly reducing the amount of computation. Finally, multiple features are concatenated and fused to complete the output. The efficient feature fusion of the Sim_DFC module improves the stability and robustness of the model and keeps the computational overhead as low as possible. Figure 3 CBS is the convolution-batch normalization-activation function step; Concatenate is the feature map concatenation operation; SimAM and EMA are the corresponding attention mechanisms. Figure 4 The process in represents that it generates attention weights by calculating the similarity between each pixel in the feature map and its neighboring pixels. Figure 5 In the figure, Groups represents the grouping operation, X / YAveragepool represents the pooling operation in different directions; after the concatenation operation and the output of the 1*1 convolution, Sigmoid represents the Sigmoid function; Reweight and GroupNorm represent the weighting of the spatial information encoding and the regularization of the data in this group; Softmax represents the Softmax function, and Matmul represents the matrix dot product operation.

[0087] The SimAM module directly calculates the attention parameters from the input feature map. The model complexity is low and there are no additional learning parameters. For vehicle detection, the module will adaptively assign weights based on the similarity of blocks in the image to find effective vehicle features. For the feature map of the left branch, the SimAM module can effectively reduce redundant information and facilitate the subsequent feature fusion between branches.

[0088] The two pooled branches of the EMA module are responsible for encoding channel information, and the features are fused by 1×1 convolution. The channel attention map is generated by the Sigmoid function, and then average pooled again, and combined with the information in the pooled 3×3 branch, the spatial information is encoded and shaped. The two spatial attention maps obtained in this way can be combined with image information and enhance the representation of features through cross-space cross-learning.

[0089] like Figure 6 As shown, according to some embodiments of the present invention, a bounding box optimization module is set, including:

[0090] Introducing Shape-IoU, which calculates the loss by focusing on the shape and scale of the bounding box itself;

[0091] Inner-IoU is introduced to speed up the regression process by using auxiliary bounding boxes and scale factor ratios;

[0092] Combining the two, the auxiliary mechanism of Inner-IoU is used to enhance the learning ability of Shape-IoU.

[0093] Working principle and beneficial effects of the above technical solution: In the optimization strategy combining Shape-IoU and Inner-IoU, IoU is used to reflect the difference between the auxiliary bounding box and the actual bounding box. By introducing Inner-Shape IoU, the training effect will be better during sample regression, thereby improving detection accuracy and convergence speed. Shape-IoU is introduced to calculate the loss by focusing on the shape and scale of the bounding box itself, making the bounding box regression more accurate. Introducing Inner-IoU. This method accelerates the regression process by using auxiliary bounding boxes and scale factor ratios. Combining the two, the auxiliary mechanism of Inner-IoU is used to enhance the learning ability of Shape-IoU while keeping the model with good generalization performance. Figure 6 The IoU in represents the intersection over union ratio between the bounding box and the GT box.

[0094] According to some embodiments of the present invention, a formula combining Shape-IoU and Inner-IoU includes:

[0095]

[0096]

[0097]

[0098]

[0099]

[0100] Loss Shape-IoU =1-IoU+distance shape +0.5×Ω shape

[0101]

[0102] Among them, the GT box and bounding box are represented as b gt and b; IoU represents the intersection-over-union ratio of the two; c represents the length of the minimum hypotenuse of the detection box between the bounding box and the GT box; w gt 、h gt represents the width and height of the GT box; ww and hh are the weight coefficients of the introduced width and height, which are affected by the shape and size of the bounding box; scale is the scale factor, which is set manually; Ω shape Indicates scale penalty; distance shape represents shape loss; and is the center point coordinate of the GT box, and x c and crepresents the center coordinates of the anchor box; w and h represent the width and height of the anchor box; substituting w and h into w t t in the equation, we get w w and w h , w w and w h is the ratio of the difference between the width and height weights; Loss Shape-IoU Represents Shape-IoU loss;

[0103]

[0104]

[0105]

[0106]

[0107]

[0108] union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter

[0109]

[0110] in, Respectively represent the first calculation parameter, the second calculation parameter, the third calculation parameter and the fourth calculation parameter of the GT frame; b l 、b r 、b t 、b b They represent the first, second, third and fourth calculation parameters of the bounding box respectively; ratio corresponds to the scale transformation factor; inter represents the screening parameters of the GT box and the bounding box; union represents the difference parameters between the GT box and the bounding box; IoU inner Represents Inner-IoU loss;

[0111] Combine Shape-IoU with Inner-IoU to get the combined loss function Loss Inner-Shape IoU ;

[0112] Loss Inner-Shape IoU =Loss Shape-IoU +IoU-IoU inner

[0113] Determine the loss function included in the bounding box optimization module.

[0114] In a specific embodiment, the complete UA-DETRAC dataset is used, and the experimental platform is as follows: CPU: 15vCPU Intel(R) Xeon(R) Platinum 8358P CPU@2.60GHz, GPU: NVIDIA RTX 3090, etc. The software platform uses ubuntu20.04 as the operating system, the deep learning framework is pytorch 1.11.0 version, and the acceleration environment is Cuda 11.7, etc. At the input end of the model, the algorithm will perform image preprocessing according to the input batch size, including random flipping, splicing and other operations. To reduce overfitting and improve model generalization. The batch size during training is set to 16. Due to the large amount of data, the epoch is set to 40. According to the experimental results, basically all models can reach convergence before this.

[0115] Calculate the average accuracy of the algorithm for the images in the first data set, and determine whether it is greater than 0.5; when it is determined that the average accuracy is greater than 0.5, it indicates that the training is successful and an improved algorithm is obtained;

[0116] Calculate the average precision mAP, including:

[0117]

[0118]

[0119]

[0120] Among them, P is precision, R is recall; TP is the number of correctly predicted positive samples, FP is the number of incorrectly predicted positive samples, FN is the number of incorrectly predicted negative samples, and P(R) is the curve formed by the change of P value with respect to R.

[0121] After feeding the data set for training according to the above conditions, the training results of the improved algorithm are shown in Table 1:

[0122] Table 1

[0123]

[0124]

[0125] The comparison results of various categories in the data set are shown in Table 2:

[0126] Table 2

[0127]

[0128] In summary, compared with the YOLOv7-tiny benchmark algorithm, the initial Sim_DFC module has a certain improvement in accuracy, but the number of parameters has reached an astonishing 12.08M, 24.5Gflops. After replacing the convolution, the accuracy of the module has slightly decreased, but the number of parameters has gradually decreased to 7.56M, 15.3Gflops. This proves that the replacement of the convolution at the corresponding position is correct and effective. Secondly, Shape-IoU and Inner-IoU were tried respectively. As can be seen from the table, the combination of the two Inner-Shape IoU has greatly improved the accuracy of the model. It makes the overall model increase the mAP50 value to 67.9% while the number of parameters remains basically unchanged. Finally, combined with the BiFPN of the improved channel, the number of parameters has been slightly reduced again, and the FPS value has been slightly improved. The model accuracy has reached a high 70.0%, significantly surpassing the detection accuracy of the original algorithm. Although the improved algorithm has a certain reduction in FPS, for traffic monitoring, a frame rate of 100 is fully sufficient to adapt to the real-time nature of the detection task. While meeting the real-time requirements, the accuracy improvement of nearly 10% is sufficient to prove the effectiveness of the improved strategy.

[0129] Figure 8(a)-Figure 8(d) The comparison results are in the actual traffic scene. In each group, the top one is the baseline (YOLOv7-tiny) and the bottom one is the improved one. (a) and (c) are the experimental results of the video. (b) and (d) are the images in the test set.

[0130] Comparing the captured videos, (a) shows that for the truck of the others category, the algorithm before the improvement mistakenly detected another vehicle in the others detection frame, and the car in front was not detected because it was blocked by the truck. In Figure (c), the original algorithm detected one less truck, which was also partially blocked. In Figure (b) of the test set, the baseline detected a car into three parts. In Figure (d), the original model classified the van as car, and the two slightly overlapping white vehicles in the back did not distinguish it. These problems still belong to classification and vehicle occlusion problems. For large trucks that are difficult to identify, small vehicles for special operations, and vans that are difficult to distinguish from cars, the improved feature extraction module can accurately identify and classify them. At the same time, for smaller targets, the improved Inner-Shape IoU mechanism can also enable the model to distinguish targets from occluders.

[0131] At the same time, the effectiveness of the algorithm was tested on Visdrone2019, and the results are shown in Table 3:

[0132] Table 3

[0133]

[0134] It can be seen that the vehicle detection model of the present invention still has advantages on a more challenging dataset such as Visdrone2019. After specific improvements, it is more excellent in the expressiveness of anchor boxes and the detection of small and medium-sized targets. This is also due to the high generalization performance of Sim_DFC and the optimization of Inner-Shape IoU for detection box regression. Compared with the original YOLOv7-tiny algorithm, there is a 1.8% improvement in mAP.50 accuracy.

[0135] Fig. 9 This is the actual effect of the vehicle detection model on the Visdrone2019 test set. It can be seen that even if the drone camera angle has many changes, the model can still stably detect vehicle targets. At night or in scenes with strong light, the effect of vehicle detection is still considerable. This shows that the vehicle detection model in the present invention is not limited to a single data set when used in traffic situations. Depending on the application scenario, the model will exert its due detection performance.

[0136] like Figure 7 As shown, according to some embodiments of the present invention, the feature fusion module includes:

[0137] The BiFPN module replaces the original ELAN-T module. The BiFPN module removes the module nodes whose fusion contribution is less than the preset threshold on the lateral edge side of the original ELAN-T module. The BiFPN module creates a diagonal path to realize a simplified bidirectional fusion network. For the non-edge intermediate layer, the BiFPN module adds an edge from the original input node to the output node, and simultaneously combines the multi-semantic information of the output node in the transmission.

[0138] The working principle and beneficial effects of the above technical solution: BiFPN, compared with PANet, can make the feature layers at different resolutions make the same contribution to the network output, and information fusion is more flexible. The improved BiFPN structure reduces the number of channels in the middle layer, while maintaining the richness of information, reducing computational redundancy, thereby optimizing the efficiency of the model in the feature fusion process. Figure 7 As shown in the figure, the module nodes whose fusion contribution is less than the preset threshold are removed from the horizontal edge side of the original ELAN-T module; the BiFPN module creates a path diagonally to realize a simplified bidirectional fusion network; for the non-edge intermediate layer, the BiFPN module adds an edge from the original input node to the output node, and combines the multi-semantic information of the output node in the transmission at the same time. The feature extraction capability is greatly improved. In addition, in order to prevent information duplication and reduce the number of parameters, the intermediate layer channel is optimized. Figure 7The Lower channels means that the number of channels in this part has been adjusted to be slightly lower in order to reduce the number of parameters.

[0139] According to some embodiments of the present invention, the sample datasets are the UA-DERTAC dataset and the Visdrone2019 dataset.

[0140] In one embodiment, the Sim_DFC module is applied to the backbone network to replace the original ELAN-T module, and the network structure is modified to the proposed BiFPN structure. Finally, the IoU mechanism is improved, the overall improved algorithm is packaged, and the UA-DERTAC and Visdrone2019 datasets are used for training.

[0141] The improved detection algorithm has higher accuracy in detecting vehicles in traffic scenes, with a 9.8% improvement on the UA-DETRAC dataset and a 1.8% improvement on the more challenging Visdrone2019 dataset, which also proves the strong versatility of the method. In terms of application, the present invention can significantly improve the detection effect of the model for vehicles that are difficult to identify and easily confused for classification, and improve the problem of small targets and vehicle occlusion to a certain extent.

[0142] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A real-time vehicle detection method using feature processing and IoU strategy, characterized in that: include: Acquire an image to be detected including vehicle information; Input the image to be detected into the vehicle detection model generated by feature processing and IoU strategy, and output the vehicle detection result; The method for obtaining a vehicle detection model includes: Set up the feature extraction module to perform multi-scale extraction and fusion of vehicle features based on the Sim_DFC module; Set up a bounding box optimization module to optimize the bounding box regression of the vehicle by combining Shape-IoU and Inner-IoU; Set up a feature fusion module to perform multi-scale fusion and transfer of vehicle features based on the improved BiFPN structure; Determine an initial vehicle detection model based on a feature extraction module, a bounding box optimization module, and a feature fusion module; Obtain a sample data set, and train an initial vehicle detection model based on the sample data set to obtain a vehicle detection model; Set up the feature extraction module, including: The feature extraction module includes three parallel convolutional layers, which are identified as the first convolutional layer, the second convolutional layer and the third convolutional layer from left to right; the third convolutional layer is 3×3 in size, and the first convolutional layer and the second convolutional layer are 1×1 in size; Based on SimAM attention, the features determined by the first convolutional layer are weighted, and the information focus is found from both spatial and channel aspects to highlight important features; The important features are output by the sigmoid function, and the importance of the important features is distributed in the weighted feature map with a weight of 0-1 and passed to the right; In the feature fusion part on the right, the features extracted by Conv, Ghost, and deep separable convolution DSConv are fused with the weighted feature map on the left and then sent to the EMA attention mechanism again to balance the learning of channel and spatial attention and enhance the ability to capture key features; Obtain multiple processed features and concatenate them, combine the multi-dimensionality of the attention mechanism with the multi-scale of the features, and generate the output through the fourth convolutional layer.

2. The real-time vehicle detection method using feature processing and IoU strategy as claimed in claim 1, characterized in that: Methods of acquiring the image to be detected including vehicle information include: direct acquisition method and indirect acquisition method; The direct acquisition method is to obtain images from a preset image database or to shoot images on site; The indirect acquisition method is to obtain a video from a preset video database or shoot a live video, perform image frame processing on the obtained video, and obtain the image to be detected.

3. The real-time vehicle detection method using feature processing and IoU strategy as claimed in claim 1, characterized in that: Set up the bounding box optimization module, including: Introducing Shape-IoU, which calculates the loss by focusing on the shape and scale of the bounding box itself; Inner-IoU is introduced to speed up the regression process by using auxiliary bounding boxes and scale factor ratios; Combining the two, the auxiliary mechanism of Inner-IoU is used to enhance the learning ability of Shape-IoU.

4. The real-time vehicle detection method using feature processing and IoU strategy as claimed in claim 3, characterized in that: The formula for combining Shape-IoU and Inner-IoU includes: Loss Shape-IoU =1-IoU+distance shape +0.5×Ω shape Among them, the GT box and bounding box are represented as b gt and b; IoU represents the intersection-over-union ratio of the two; c represents the length of the minimum hypotenuse of the detection box between the bounding box and the GT box; w gt 、h gt represents the width and height of the GT box; ww and hh are the weight coefficients of the introduced width and height, which are affected by the shape and size of the bounding box; scale is the scale factor, which is set manually; Ω shape Indicates scale penalty; distance shape represents shape loss; and is the center point coordinate of the GT box, and x c and c represents the center coordinates of the anchor box; w and h represent the width and height of the anchor box; substituting w and h into w t t in the equation, we get w w and w h , w w and w h is the ratio of the difference between the width and height weights; Loss Shape-IoU Represents Shape-IoU loss; union=(w gt ×h gt )×(ratio) 2 +(w×h)×(ratio) 2 -inter in, Respectively represent the first calculation parameter, the second calculation parameter, the third calculation parameter and the fourth calculation parameter of the GT frame; b l 、b r 、b t 、b b They represent the first, second, third and fourth calculation parameters of the bounding box respectively; ratio corresponds to the scale transformation factor; inter represents the screening parameters of the GT box and the bounding box; union represents the difference parameters between the GT box and the bounding box; IoU inner Represents Inner-IoU loss; Combine Shape-IoU with Inner-IoU to get the combined loss function Loss Inner-ShapeIoU ; Loss Inner-ShapeIoU =Loss Shape-IoU +IoU-IoU inner Determine the loss function included in the bounding box optimization module.

5. The real-time vehicle detection method using feature processing and IoU strategy as claimed in claim 1, characterized in that: The feature fusion module comprises: The BiFPN module replaces the original ELAN-T module. The BiFPN module removes the module nodes whose fusion contribution is less than the preset threshold on the lateral edge side of the original ELAN-T module. The BiFPN module creates a diagonal path to realize a simplified bidirectional fusion network. For the non-edge intermediate layer, the BiFPN module adds an edge from the original input node to the output node, and simultaneously combines the multi-semantic information of the output node in the transmission.

6. The real-time vehicle detection method using feature processing and IoU strategy as claimed in claim 1, characterized in that: The sample datasets are the UA-DERTAC dataset and the Visdrone2019 dataset.

Images