YOLO-TGB vehicle detection system and method

Abstract

The application discloses a YOLO-TGB vehicle detection system and method, which is based on the combination of a light network GhostNet and an attention mechanism module Transformer, reduces the redundancy of the main network, makes the network more focused on vehicle feature information, and thus maintains good detection accuracy; an improved bidirectional weighted feature pyramid network BiFPN added with a depth separable convolution is used to reduce the parameter size while fusing as many features as possible; a K-Means++ clustering algorithm is used for clustering analysis according to the scale of the vehicle detection target, so that more accurate prior boxes are obtained, and the detection performance is further improved; the effectiveness of the YOLO-TGB network is verified based on the UA-DETRAC vehicle dataset, and experimental results show that, compared with many other target detection networks, the light-weight YOLO-TGB network can maintain relatively good real-time performance while obtaining higher vehicle detection accuracy.

Description

Technical Field

[0001] This invention belongs to the field of intelligent vehicle detection technology, specifically relating to a YOLO-TGB vehicle detection system and method. Background Technology

[0002] Currently, motor vehicles have become an essential means of transportation for people. The rapid increase in the number of vehicles has brought about numerous problems such as traffic congestion, fatigued driving, and traffic accidents, seriously affecting transportation efficiency and the safety of people's lives and property. With the rapid development of the new generation of smart highways, autonomous driving combined with intelligent connected vehicle-road cooperation has become an important way to solve the above problems. Accurate vehicle detection is a prerequisite for realizing autonomous driving and intelligent connected vehicle-road cooperation. From a management perspective, tracking systems based on vehicle detection technology can provide reliable vehicle information, including specific categories, accurate locations, and driving trajectories. This helps in traffic flow analysis, traffic planning optimization, and future vehicle trajectory prediction, improving the utilization rate of vehicle information resources and the work efficiency of management personnel, and ensuring road traffic safety and the safety of people's lives and property. From the driver's perspective, onboard terminals based on vehicle detection technology can monitor the surrounding environment and road traffic conditions in real time through cameras and radar, providing drivers with more reasonable driving suggestions, thereby avoiding traffic violations and reducing the probability of traffic accidents. Since vehicle detection needs to be carried out in different scenarios, such as urban roads, highways, and mountain roads, factors such as lighting, background, weather, and road signs in these scenarios will affect the accuracy of vehicle detection. Meanwhile, in real-world scenarios, multiple overlapping vehicle images often exist in the same detection area, which can easily lead to false detections and missed detections. Achieving high-precision and high-speed vehicle detection in complex traffic scenarios is of great practical significance and value, and it is also a key focus and challenge in current vehicle detection research and exploration.

[0003] In recent years, deep learning-based vehicle detection methods have made significant progress in both detection accuracy and efficiency. In 2015, Joseph et al. proposed YOLO (You Only Look Once), a target detection network with high real-time performance and good detection accuracy. Its basic idea is to eliminate the sliding window, directly divide the original image into a grid, generate feature maps through convolution, and use the corresponding feature elements of the original image to predict targets at the center points of the grid, directly predicting and classifying the object's position. To further improve YOLO's detection performance, researchers have continuously optimized it, successively proposing YOLOv2, YOLOv3, YOLOv4, and YOLOv5. Currently, the YOLO series of networks are widely used in various target detection applications. In 2019, Du et al. proposed a real-time vehicle and traffic light detection method based on YOLOv3. This method can well meet real-time requirements, but its detection accuracy for small traffic targets is not ideal. In 2020, Zhang et al. improved YOLOv3, adding a deeper feature extraction network and four different scale detection layers. This significantly improved recall and accuracy while maintaining detection speed, but the results were susceptible to complex environments. In the same year, Wang et al. improved the feature extraction part of the YOLOv3 network structure to extract deeper features, improved the loss function to enhance small vehicle region detection capabilities, and improved the multi-scale detection method to increase the speed of small vehicle region target detection. This method showed good real-time performance and accuracy, and strong robustness in handling complex environments. However, its feature fusion module used a feature pyramid network, making the detection performance susceptible to the influence of one-sided information flow. Mohamed et al. used ResNet50 and VGG16 as the classification backbone modules of YOLOv4, and introduced PANet to reduce the impact of one-sided information flow on the feature fusion module, achieving better classification accuracy. In 2021, Yao et al. combined YOLOv3 with machine vision, greatly improving the average accuracy of vehicle detection, but also extending the network training cycle. Shi et al. proposed an improved YOLOv3 vehicle detection method that significantly improves detection accuracy in fewer training cycles and exhibits good robustness in complex environments, but its detection speed cannot meet real-time requirements. Yang et al. proposed a real-time YOLOv4 vehicle detection method that can be deployed on embedded platforms, but its vehicle detection efficiency in video streams remains low. Hu et al. proposed an improved YOLOv4-based method that achieves relatively high detection speed without sacrificing detection accuracy, but it still cannot meet the real-time requirements for vehicle target detection in video streams.In 2022, Amrouche et al. further improved the detection speed by using the lightweight YOLOv4-tiny, but sacrificed some accuracy in small target detection. Currently, how to further improve the efficiency and accuracy of vehicle detection in real-world scenarios remains an important direction for future vehicle detection research.

[0004] When detecting small vehicles, their features are difficult to identify due to factors such as complex environment, low resolution, and limited contextual information, which can easily lead to missed or false detections. Summary of the Invention

[0005] The purpose of this invention is to provide a YOLO-TGB vehicle detection system and method to overcome the problem that current vehicle detection networks cannot achieve both good target vehicle detection accuracy and detection efficiency.

[0006] A YOLO-TGB vehicle detection system includes a backbone network, a neck module, and a head module;

[0007] The backbone network consists of GhostNet layers and Transformer layers. The Transformer layers are placed at the end of the GhostNet layers. The GhostNet layers are used to extract feature maps from the input image. The extracted feature maps are then input into the Transformer layers. The Transformer layers unfold the extracted feature map pixels into a one-dimensional information sequence, add position encoding information, and convert it into vector information representation, which is then transmitted to the neck module.

[0008] The neck module includes a spatial pyramid pooling layer and an improved bidirectional weighted feature pyramid network (BiFPN) with added depthwise separable convolution (DSConv). The spatial pyramid pooling layer is used to pool the acquired vector information representation, and then the improved bidirectional weighted feature pyramid network (BiFPN) is used for feature fusion to obtain the fused feature image.

[0009] The head module uses the K-Means++ clustering method to analyze and cluster the fused feature images to obtain vehicle classification results.

[0010] Preferably, the GhostNet layer includes a CBL layer and a G-bneck layer. The CBL layer includes a Conv layer, a BN layer, and a LeakyReLU layer. The Conv layer is a convolutional layer that uses multiple different convolutional kernels to process the input image, resulting in different response feature maps. The BN layer is a batch normalization layer. As a layer of the neural network, the BN layer is placed before the activation function and after the convolutional layers. The BN layer normalizes the different response feature maps obtained. The Leaky ReLU layer uses an activation function to add non-linearity to the convolutional neural network.

[0011] Preferably, the G-bneck layer includes a GhostUnit layer, which contains two Ghost modules. The first Ghost module is used to increase the number of channels, specifying the ratio between the number of output and input channels as the expansion ratio. The second Ghost module reduces the number of channels to match the channels of the shortcut branch.

[0012] Preferably, an input image of size 608×608 is processed through a CBL layer to obtain a feature map of size 304×304. The feature map then enters the first G-bneck1 to expand the number of channels; then it enters the first G-bneck2, where the feature map size changes from 304×304 to 152×152; then it enters the second G-bneck2, where the feature map size changes from 152×152 to 76×76; then it enters G-bneck4 for downsampling, resulting in a feature map of size 38×38; after expanding the number of channels through G-bneck2, it enters G-bneck5, where the feature map output size is 19×19, and then enters the Transformer layer.

[0013] Preferably, the Transformer layer unfolds the image pixels into a one-dimensional shape, adds position encoding information, and then inputs it into the encoder, converting the information in the sequence input to the encoder into a vector representation.

[0014] Preferably, the improved bidirectional weighted feature pyramid network BiFPN upsamples the extracted 19×19 feature map to obtain a 38×38 feature map. This 38×38 feature map is then concat-fused with the corresponding 38×38 feature map from the GhostNet layer to achieve the same scale. The fused feature map is then upsampled to obtain a 76×76 feature map, which is then fused with the 76×76 feature map from the GhostNet layer again to achieve the same scale. Similarly, the fused 76×76 feature map is downsampled twice to obtain feature maps of sizes 38×38 and 19×19, respectively. These 38×38 and 19×19 feature maps are then concat-fused with the corresponding scale features obtained after upsampling and fusion. The 76×76, 38×38, and 19×19 feature maps obtained after multiple feature fusions are input into the corresponding scale head module, and the K-Means++ clustering method is used to analyze and cluster the fused feature images to output the classification results.

[0015] A YOLO-TGB vehicle inspection method includes the following steps;

[0016] S1, extract feature maps from the input image, unfold the extracted feature map pixels into a one-dimensional information sequence, add position encoding information, and then convert it into vector information;

[0017] S2, the obtained vector information representation is pooled, and then the pooled feature image is fused to obtain the fused feature image.

[0018] S3 uses the K-Means++ clustering method to analyze and cluster the fused feature images to obtain vehicle classification results.

[0019] Preferably, the GhostNet layer includes a CBL layer and a G-bneck layer. The CBL layer includes a Conv layer, a BN layer, and a LeakyReLU layer. The Conv layer is a convolutional layer that uses multiple different convolutional kernels to process the input image, resulting in different response feature maps. The BN layer is a batch normalization layer. As a layer of the neural network, the BN layer is placed before the activation function and after the convolutional layers. The BN layer normalizes the different response feature maps obtained. The Leaky ReLU layer uses an activation function to add non-linearity to the convolutional neural network.

[0020] Preferably, the G-bneck layer includes a GhostUnit layer, which contains two Ghost modules. The first Ghost module is used to increase the number of channels, specifying the ratio between the number of output and input channels as the expansion ratio. The second Ghost module reduces the number of channels to match the channels of the shortcut branch.

[0021] Preferably, the Transformer layer unfolds the image pixels into a one-dimensional shape, adds position encoding information, and then inputs it into the encoder, converting the information in the sequence input to the encoder into a vector representation.

[0022] Compared with the prior art, the present invention has the following beneficial technical effects:

[0023] This invention discloses a YOLO-TGB vehicle detection system, based on a lightweight network GhostNet and an attention mechanism module Transformer. This reduces backbone network redundancy while allowing the network to focus more on vehicle feature information, thus maintaining good detection accuracy. An improved bidirectional weighted feature pyramid network (BiFPN) with added depthwise separable convolutions is employed to reduce parameter size while fusing more features. The K-Means++ clustering algorithm is used to perform clustering analysis based on the scale of the vehicle detection target, resulting in more accurate prior boxes and further improving detection performance. The effectiveness of the YOLO-TGB network is verified using the UA-DETRAC vehicle dataset. Experimental results show that, compared with many other object detection networks, the lightweight YOLO-TGB network can achieve higher vehicle detection accuracy while maintaining relatively good real-time performance. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the network structure of a YOLO-TGB vehicle detection system in an embodiment of the present invention.

[0025] Figure 2 This is a schematic diagram of the BiFPN layer structure in an embodiment of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] This invention provides a YOLO-TGB vehicle detection system, including a backbone network, a neck module, and a head module;

[0029] The backbone network consists of GhostNet layers and Transformer layers. The Transformer layers are placed at the end of the GhostNet layers. The GhostNet layers are used to extract feature maps from the input image. The extracted feature maps are then input into the Transformer layers. The Transformer layers unfold the extracted feature map pixels into a one-dimensional information sequence, add position encoding information, and convert it into vector information representation, which is then transmitted to the neck module.

[0030] The neck module includes a Spatial Pyramid Pooling (SPP) layer and an improved BiFPN network with added depthwise separable convolution DSConv. The spatial pyramid pooling layer is used to pool the acquired vector information representation, and then the improved BiFPN network is used for feature fusion to obtain the fused feature image.

[0031] The head module uses the K-Means++ clustering method to analyze and cluster the fused feature images to obtain vehicle classification results, thereby obtaining more accurate prior boxes and further improving detection performance.

[0032] GhostNet layers consist of CBL layers and G-bneck layers. The CBL layers include Conv layers, BN layers, and Leaky ReLU layers. The Conv layers are convolutional layers that process the input image using multiple different convolutional kernels to obtain different response feature maps. The BN layers are batch normalization layers. As a layer in the neural network, the BN layer is placed before the activation function and after the convolutional layers. The BN layer normalizes the different response feature maps obtained. The Leaky ReLU layer uses an activation function to add a non-linear factor to the convolutional neural network.

[0033] A G-bneck layer includes a GhostUnit layer, which contains two Ghost modules. The first Ghost module increases the number of channels, specifying the ratio between the number of output and input channels as the expansion ratio. The second Ghost module decreases the number of channels to match the channels of the shortcut branch. When the stride is 2, a depthwise convolutional layer with a stride of 2 is added between the two Ghost modules.

[0034] Specifically, such as Figure 1 As shown, an input image of size 608×608 is processed through a CBL layer to obtain a feature map of size 304×304. The feature map then enters the first G-bneck1, which expands the number of channels; then it enters the first G-bneck2, where the feature map size changes from 304×304 to 152×152; then it enters the second G-bneck2, where the feature map size changes from 152×152 to 76×76; then it enters G-bneck4 for downsampling, resulting in a feature map of size 38×38; after G-bneck2 expands the number of channels, it enters G-bneck5, where the feature map output size is 19×19, and then enters the Transformer layer.

[0035] The Transformer layer is a model based on the Encoder-Decoder framework; each encoder and decoder module consists of six identical encoders and decoders. The encoder uses a deep neural network. In each layer, the encoder transforms the input sequence into a higher-level feature vector through a series of transformations. The encoder includes Multi-Head Attention, Feed Forward, and Add&Norm. Multi-Head Attention is the core of the encoder, generating a representation vector associated with the input sequence by performing self-attention calculation on each element of the input sequence. Multi-Head Attention consists of multiple Self-Attention modules, each calculating an attention distribution to align different positions in the input sequence. Feed Forward further processes the output of Multi-Head Attention, performing a non-linear transformation on the representation vector to obtain a richer feature representation. Add&Norm is a residual connection and normalization module that adds the outputs of Multi-Head Attention and Feed Forward to obtain a more stable and accurate representation vector. The decoder is a crucial component of the Transformer layer, responsible for converting the high-dimensional representation vector generated by the encoder into the target sequence, realizing sequence generation and prediction. Unlike the encoder, the decoder needs to consider the already generated part of the sequence when generating the target sequence. Therefore, a Masked Multi-HeadAttention structure is needed to mask the position information of the currently generated sequence in order to avoid information leakage and overfitting.

[0036] The Transformer layer unfolds the feature map pixels into a one-dimensional structure, adds positional encoding information, and then inputs it into the encoder. This transforms the information in the encoder sequence into a vector representation. During this process, Multi-Head Attention allows the model to focus on different subspaces simultaneously, better handling the relationships between multiple inputs. The resulting vectors are then summed and normalized before being input into a feedforward neural network. After computation by the feedforward network, the result is fed into the next encoder. The output of the last encoder serves as the input to the decoder. This process involves six decoders, followed by linear and softmax operations to output the final result. The feature map processed by the Transformer layer avoids losing important contextual information during feature extraction, capturing more global information.

[0037] Feature fusion employs an improved bidirectional weighted feature pyramid network, BiFPN, such as... Figure 2As shown, the 19×19 feature map extracted from the backbone network is upsampled to obtain a 38×38 feature map. This 38×38 feature map is then concatted with the corresponding 38×38 feature map in the GhostNet layer to fuse features at the same scale. The fused feature map is then upsampled again to obtain a 76×76 feature map, which is then fused with the 76×76 feature map obtained from the GhostNet layer again at the same scale. Next, the fused 76×76 feature map is downsampled twice to obtain feature maps of sizes 38×38 and 19×19, respectively. These 38×38 and 19×19 feature maps are then concatted with the corresponding scale features obtained after upsampling and fusion. The 76×76, 38×38, and 19×19 feature maps obtained after multiple feature fusions are input into the corresponding scale head module. The K-Means++ clustering method is used to analyze and cluster the fused feature images to output the classification results.

[0038] This application's GhostNet combines linear operations and ordinary convolution. It uses a linear transformation on pre-generated ordinary convolution feature maps to obtain similar feature maps, thus producing a high-dimensional convolution effect while reducing model parameters and computational cost. The module combining convolution and linear operations is called the Ghost module. Theoretically, upgrading ordinary convolution with Ghost modules results in a compression ratio approximately equal to the theoretical speedup ratio, and the computational cost per channel for linear operations is far less than that of ordinary convolution. The Ghost bottleneck layer (G-bnecklayer) is designed to store the Ghost modules.

[0039] By directly modeling variable-length sequences using Transformer layers, the dependencies and positional information within the sequence can be captured directly, enabling parallel computation and accelerating the computational speed. It also offers better scalability, allowing for increased model depth by stacking multiple Transformer layers. Transformers are more robust to object interference and variations in region size in images. Adding Transformer modules to the backbone network allows for the acquisition of more layers of features and contextual information during feature extraction and fusion, reducing false positives and false negatives in vehicle detection caused by complex weather conditions and mutual occlusion.

[0040] The Transformer layer is placed at the end of the GhostNet layers. If a Transformer is added to other stages of the backbone network, the network is still relatively shallow, and adding the Transformer module will trigger boundary regression, thus losing some contextual information that is helpful for judgment. After adding the Transformer, the Transformer unfolds the pixel data of the extracted feature maps into a one-dimensional information sequence, adds positional encoding information, and then inputs it into the encoder. The encoder then converts the input information into a vector representation. During this process, a multi-head attention mechanism is used to allow the model to pay attention to different subspaces simultaneously, better handling the relationships between multiple inputs.

[0041] An improved bidirectional weighted feature pyramid network (BiFPN) is employed, capable of simultaneously transmitting semantic information from high-level features and positional information from low-level features. Additional paths are used when input and output points are located in the same layer to fuse more features. BiFPN utilizes a more efficient feature fusion method and a stackable structural design, effectively addressing issues such as information loss and feature duplication during feature fusion. BiFPN adds extra weights to input features with different resolutions to understand the importance of each feature, thus better collecting features at different scales. Furthermore, BiFPN employs a fast normalization fusion method, improving computational speed.

[0042] The K-Means++ clustering method is used to analyze and cluster based on the fused feature images. Specifically, a specific point is randomly selected from the sample set to become the first cluster center. For each point, the distance between it and existing cluster centers is calculated, using methods such as Euclidean distance to determine the nearest cluster center. This process is repeated until K cluster centers are found. This method of selecting initial cluster centers ensures a more even distribution of initial points, avoiding poor clustering results caused by unfavorable initial point locations, thus improving the clustering performance.

[0043] Experimental verification and comparative analysis of experimental results

[0044] The UA-DETRAC vehicle detection dataset was used for training and validation experiments. UA-DETRAC is a typical real-world dataset suitable for multi-target vehicle detection and tracking. The UA-DETRAC dataset contains real-time vehicle images captured in the dataset. It contains over 140,000 frames of images, including 8,250 manually labeled vehicles. The training set contains approximately 82,085 images, and the test set contains approximately 56,167 images. The dataset covers complex weather conditions, including cloudy, nighttime, sunny, and rainy days, and includes four vehicle types: cars, buses, vans, and other vehicle types. Due to the large size of the UA-DETRAC dataset, 80% of the 26,550 images were randomly selected as the training set, and 20% as the validation set, totaling 8,500 labeled vehicles. Additionally, 12,550 images were selected as the test set.

[0045] The experimental environment setup is shown in Table 1.

[0046] Table 1 Experimental Environment Deployment

[0047]

[0048] Ablation Experiment Results and Analysis

[0049] Attention mechanism selection

[0050] The SE attention mechanism and CBAM attention mechanism were added to the YOLOv4 network, and the backbone network of this application was used. The experimental results are shown in Table 2.

[0051] Table 2 Ablation Experiments Based on Attention Mechanisms

[0052]

[0053] As shown in Table 2, the average detection accuracy of the GhostNet layer and Transformer layer in this application is improved by 1.5% and 0.6% respectively compared with the other two networks.

[0054] Lightweight network options:

[0055] The backbone network CSPDarkNet53 in the YOLOv4 network was replaced with MobileNetv3 and ShuffleNetv2 respectively, and the GhostNet layer of this application was used. The experimental results are shown in Table 3.

[0056] Table 3 Lightweight Network Real-Time Ablation Experiment

[0057]

[0058] Experimental results show that, compared with MobileNetv3 and ShuffleNetv2, using the lightweight network GhostNet reduces more parameters in the network, and improves FPS by 0.6 frames / s and 1.8 frames / s respectively, achieving better performance.

[0059] Detection accuracy verification:

[0060] To verify the effectiveness of the improved YOLO-TGB in terms of detection accuracy, an ablation experiment was conducted using a method of stacking improved modules. The networks used in the experiment are classified as follows:

[0061] Network 1: The original YOLOv4 network.

[0062] Network 2: YOLOv4+Transformer, adding a Transformer attention mechanism module to the YOLOv4 network.

[0063] Network 3: YOLOv4+BiFPN, which adds a BiFPN feature fusion network to the YOLOv4 network to replace the original PANet network.

[0064] Network 4: YOLOv4+Transformer+BiFPN, which adds a Transformer attention mechanism module and a BiFPN feature fusion network to YOLOv4.

[0065] Network 5 (YOLO-TB): YOLOv4+Transformer+BiFPN+K-Means++. It adds a Transformer attention mechanism module to YOLOv4, and adopts BiFPN feature fusion network and K-Means++ clustering method. This network is named YOLO-TB.

[0066] The UA-DETRAC vehicle detection dataset was used, and some key training parameters were set to achieve better results. To better train the model, the batch size was adjusted to 8, the image size to 608×608, the number of training iterations for the entire dataset was adjusted to 300, and the initial learning rate was set to 0.001. A two-stage training strategy was employed throughout the training process. First, the first 50 layers of the original network were frozen and trained to obtain the initial weights of the model. Then, the entire network was unfrozen, and the batch size was adjusted to 4 to complete the second stage of training. By setting these experimental parameters and training strategies, the model can be trained more effectively, improving its accuracy in the vehicle detection task. To ensure the recording of weights without affecting training speed, the weights were saved every 10 epochs.

[0067] The ablation experiment results are shown in Table 4. The performance of networks 1 to 5 is analyzed by comparing them with the performance evaluation indicators of the target detection network.

[0068] Table 4. Network ablation experiments on the UA-DETRAC dataset.

[0069]

[0070] Comparing Network 1 and Network 2, the improved network in Network 2, which incorporates the Transformer attention mechanism, achieves a 1.9% improvement in average detection accuracy, a 13.2MB increase in model size, and a 3% improvement in F1 score compared to the original YOLOv4 network in Network 1. Experiments show that adding the Transformer significantly improves detection accuracy.

[0071] Comparing Network 1 and Network 3, Network 3, which uses BiFPN for feature fusion, achieved an average accuracy improvement of 1.0% compared to the original YOLOv4 network, a model size reduction of 74MB, and a 2% increase in F1 score. This demonstrates that BiFPN optimizes the network model size and can improve detection accuracy.

[0072] Comparing Network 3 and Network 4, Network 4 combines the Transformer attention mechanism module and the BiFPN feature fusion network. Compared with Network 3, the average detection accuracy is improved by 2.1% and the F1 score is improved by 3%, proving that the network combining Transformer and BiFPN has better detection performance.

[0073] Compared with Network 4 and Network 5, Network 5 (YOLO-TB) adds the K-Means++ clustering method to the proposed Transformer attention mechanism module and BiFPN feature fusion network, which improves the average detection accuracy by 0.6% and the F1 score by 3%.

[0074] Compared to Network 1 (original YOLOv4) and the proposed improved Network 5 (YOLO-TB), the average detection accuracy was improved by 3.7%, the network model size was reduced by 64.2 MB, and the F1 score was improved by 8%. Experimental results demonstrate that Network 5, based on the Transformer attention mechanism and BiFPN feature fusion network with added K-Means++ clustering, significantly improves detection performance compared to the unimproved YOLOv4 network.

[0075] The YOLO-TGB vehicle detection system offers a significant performance improvement over the unmodified YOLOv4 network detection system.

[0076] Detection efficiency verification

[0077] To verify the effectiveness of the improved YOLO-TGB in terms of detection efficiency, ablation experiments were conducted using an improved module stacking method. The original YOLOv4, in order to balance accuracy and real-time performance, has a complex network structure, resulting in larger network parameters and a relatively slower detection speed. Experimental results validate the effectiveness of the real-time vehicle detection algorithm YOLO-TGB.

[0078] The networks used in the experiment are classified as follows:

[0079] Network 1: YOLOv4 network.

[0080] Network 2: YOLO-TGB, using the lightweight network GhostNet, combined with the Transformer attention module, using the BiFPN network and adding depthwise separable convolutions, and employing the K-Means++ clustering algorithm.

[0081] The above different algorithms were tested on the same UA-DETRAC vehicle detection dataset, and the experimental results are shown in Table 5.

[0082] Table 5. Real-time Network Ablation Experiment

[0083]

[0084] As shown in Table 5, the original YOLOv4 has a complex network structure to balance accuracy and real-time performance, resulting in a large number of network parameters and a relatively slow detection speed. Network 2 reduces the number of parameters by 53,120,017 and improves the FPS by 19 frames / s, indicating that using the GhostNet network can significantly reduce feature map parameter redundancy. To maintain detection accuracy while keeping the network lightweight, a Transformer attention mechanism module and a BiFPN network were added. This improved detection accuracy but also increased the number of parameters. Therefore, depthwise separable convolutions were used instead, significantly improving the detection speed. Compared to existing networks, Network 2 has a significantly reduced number of parameters and a significantly improved detection speed. Experimental results verify the effectiveness of the real-time vehicle detection algorithm YOLO-TGB.

[0085] Comparative Experimental Results and Analysis

[0086] The YOLOv3, YOLOv4-tiny, YOLOv4, YOLOv5, MobileNet-YOLOv4, and YOLO-TB of this application were compared on the UA-DETRAC vehicle detection dataset, and experimental analysis was conducted using object detection performance evaluation metrics. The experimental results are shown in Table 6.

[0087] Table 6. Experimental Results Comparing Different Target Detection Algorithms

[0088]

[0089] The YOLO-TB network in this application outperforms other algorithms in detection performance. Compared to YOLOv3, YOLOv4 is an improvement on YOLOv3, therefore its overall detection performance is superior. YOLOv4-Tiny is a lightweight version of YOLOv4, offering slightly lower detection accuracy but significantly improved detection speed. MobileNet-YOLOv4 is also a lightweight version of YOLOv4, with slightly lower detection accuracy but a nearly three-fold reduction in model size. It boasts higher detection accuracy than YOLO-Tiny, thus achieving both relatively high detection accuracy and speed. YOLOv5 offers slightly improved accuracy compared to YOLOv4, but its increased model size leaves considerable room for performance improvement. To address this, an improved YOLO-TGB network was proposed, which has a much higher accuracy than other networks. Compared with the YOLOv3 network, the average accuracy is improved by 6.9%, compared with the YOLOv4 network, the average detection accuracy is improved by 3.7%, and the model size is reduced by 22.4%. Compared with the YOLOv5 network, the average detection accuracy is improved by 2.8%, and the model size is reduced by 24.6%.

[0090] The ablation experiment results have verified that the YOLO-TGB lightweight network has good real-time performance, but the simplification of the network structure may lead to a decrease in detection accuracy.

[0091] Ablation experiments have verified that the lightweight YOLO-TGB network has good real-time performance. However, the simplification of the network structure may lead to a decrease in detection accuracy. To verify whether the YOLO-TGB network can significantly improve the real-time performance while maintaining accuracy, comparative experiments were conducted, and the results are shown in Table 7.

[0092] Table 7. Comparison Experiment Results of Lightweight Networks

[0093]

[0094] As shown in the table above, the YOLO-TB network has significantly higher accuracy than the YOLOv4 network. The lightweight network YOLOv4-Tiny, compared to both YOLO-TB and YOLOv4, shows a significant decrease in detection accuracy but an increase in detection speed of 41.9% and 39%, respectively, demonstrating the improvement in real-time performance brought by the lightweight network. However, YOLOv4-Tiny suffers too much accuracy loss, failing to achieve a balance between detection accuracy and real-time performance. Compared to YOLOv4-Tiny, YOLO+ShuffleNetv2 and YOLO+MobileNetv3 show a decrease in detection speed but a significant increase in detection accuracy. YOLO+MobileNetv3 achieves higher detection accuracy while maintaining relatively high detection speed, achieving a balance between high detection accuracy and high detection speed. Compared to the YOLOv4-Tiny and YOLO+ShuffleNetv2 algorithms, YOLO+MobileNetv3 has superior overall detection performance. Compared to YOLOv4, YOLO+MobileNetv3 shows a slight decrease in accuracy but a significant improvement in detection speed, demonstrating the effectiveness of using the lightweight MobileNetv3 network as the backbone. It also proves the significant impact of lightweight networks on the real-time performance of the algorithm. YOLO-TGB, the proposed lightweight network, not only significantly improves detection accuracy compared to the similarly lightweight YOLO+MobileNetv3 but also boasts a substantial advantage in detection speed, proving its ability to better balance accuracy and speed. Comparing YOLOv4 and YOLO-TGB, YOLO-TGB improves detection accuracy by 0.1% and increases FPS from 39.7 frames / s to 58.7 frames / s, indicating that YOLO-TGB's detection speed far exceeds that of YOLOv4, improving real-time performance while maintaining accuracy.

Claims

1. A YOLO-TGB vehicle inspection system, characterized in that, Includes the backbone network, neck module, and head module; The backbone network consists of GhostNet layers and Transformer layers. The Transformer layers are placed at the end of the GhostNet layers. The GhostNet layers are used to extract feature maps from the input image. The extracted feature maps are then input into the Transformer layers. The Transformer layers unfold the extracted feature map pixels into a one-dimensional information sequence, add position encoding information, and convert it into vector information representation, which is then transmitted to the neck module. The neck module includes a spatial pyramid pooling layer and an improved bidirectional weighted feature pyramid network (BiFPN) with added depthwise separable convolution (DSConv). The spatial pyramid pooling layer is used to pool the acquired vector information representation, and then the improved bidirectional weighted feature pyramid network (BiFPN) is used for feature fusion to obtain the fused feature image. The head module uses the K-Means++ clustering method to analyze and cluster the fused feature images to obtain vehicle classification results. The input image of size 608×608 is processed by the CBL layer to obtain a feature map of size 304×304. The feature map then enters the first G-bneck1 to expand the number of channels. After that, it enters the first G-bneck2, and the feature map size is reduced from 304×304 to 152×152. Then it enters the second G-bneck2, and the feature map size is reduced from 152×152 to 76×76. It enters G-bneck4 for downsampling, and the feature map becomes 38×38. After the number of channels is expanded by G-bneck2, it enters G-bneck5 and the output feature map size is 19×19. Then it enters the Transformer layer. The improved bidirectional weighted feature pyramid network BiFPN upsamples the extracted 19×19 feature map to obtain a 38×38 feature map. This 38×38 feature map is then concatted with the corresponding 38×38 feature map from the GhostNet layer to fuse features at the same scale. The fused feature map is then upsampled again to obtain a 76×76 feature map, which is then fused with the 76×76 feature map from the GhostNet layer again at the same scale. Similarly, the fused 76×76 feature map is downsampled twice to obtain feature maps of sizes 38×38 and 19×19, respectively. These 38×38 and 19×19 feature maps are then concatted with the corresponding scale features obtained after upsampling and fusion. The 76×76, 38×38, and 19×19 feature maps obtained after multiple feature fusions are input into the corresponding scale head module. The K-Means++ clustering method is used to analyze and cluster the fused feature images to output the classification results.

2. The YOLO-TGB vehicle inspection system according to claim 1, characterized in that, GhostNet layers consist of CBL layers and G-bneck layers. The CBL layers include Conv layers, BN layers, and Leaky ReLU layers. The Conv layers are convolutional layers that process the input image using multiple different convolutional kernels to obtain different response feature maps. The BN layers are batch normalization layers. As a layer in the neural network, the BN layer is placed before the activation function and after the convolutional layers. The BN layer normalizes the different response feature maps obtained. The Leaky ReLU layer uses an activation function to add a non-linear factor to the convolutional neural network.

3. The YOLO-TGB vehicle inspection system according to claim 1, characterized in that, The G-bneck layer includes the GhostUnit layer, which contains two Ghost modules. The first Ghost module increases the number of channels, specifying the ratio between the number of output and input channels as the expansion ratio. The second Ghost module decreases the number of channels to match the channels of the shortcut branch.

4. The YOLO-TGB vehicle inspection system according to claim 1, characterized in that, The Transformer layer unfolds the image pixels into a one-dimensional shape, adds positional encoding information, and then inputs it into the encoder, transforming the information in the sequence input to the encoder into a vector representation.

5. A YOLO-TGB vehicle inspection method, characterized in that, Includes the following steps; S1, extract feature maps from the input image, unfold the extracted feature map pixels into a one-dimensional information sequence, add position encoding information, and then convert it into vector information; S2, the obtained vector information representation is pooled, and then the pooled feature image is fused to obtain the fused feature image. S3, the K-Means++ clustering method is used to analyze and cluster the fused feature images to obtain vehicle classification results; An input image of size 608×608 is processed through a CBL layer to obtain a feature map of size 304×304. The feature map then enters the first G-bneck1 to expand the number of channels; then it enters the first G-bneck2, where the feature map size changes from 304×304 to 152×152; then it enters the second G-bneck2, where the feature map size changes from 152×152 to 76×76; then it enters G-bneck4 for downsampling, resulting in a feature map of size 38×38; after expanding the number of channels through G-bneck2, it enters G-bneck5, where the feature map output size is 19×19, and then it enters the Transformer layer. The improved bidirectional weighted feature pyramid network BiFPN upsamples the extracted 19×19 feature map to obtain a 38×38 feature map. This 38×38 feature map is then concatted with the corresponding 38×38 feature map from the GhostNet layer to fuse features at the same scale. The fused feature map is then upsampled again to obtain a 76×76 feature map, which is then fused with the 76×76 feature map from the GhostNet layer again at the same scale. Similarly, the fused 76×76 feature map is downsampled twice to obtain feature maps of sizes 38×38 and 19×19, respectively. These 38×38 and 19×19 feature maps are then concatted with the corresponding scale features obtained after upsampling and fusion. The 76×76, 38×38, and 19×19 feature maps obtained after multiple feature fusions are input into the corresponding scale head module. The K-Means++ clustering method is used to analyze and cluster the fused feature images to output the classification results.

6. The YOLO-TGB vehicle inspection method according to claim 5, characterized in that, GhostNet layers consist of CBL layers and G-bneck layers. The CBL layers include Conv layers, BN layers, and Leaky ReLU layers. The Conv layers are convolutional layers that process the input image using multiple different convolutional kernels to obtain different response feature maps. The BN layers are batch normalization layers. As a layer in the neural network, the BN layer is placed before the activation function and after the convolutional layers. The BN layer normalizes the different response feature maps obtained. The Leaky ReLU layer uses an activation function to add a non-linear factor to the convolutional neural network.

7. A YOLO-TGB vehicle inspection method according to claim 5, characterized in that, The G-bneck layer includes the GhostUnit layer, which contains two Ghost modules. The first Ghost module increases the number of channels, specifying the ratio between the number of output and input channels as the expansion ratio. The second Ghost module decreases the number of channels to match the channels of the shortcut branch.

8. The YOLO-TGB vehicle inspection method according to claim 5, characterized in that, The Transformer layer unfolds the image pixels into a one-dimensional shape, adds positional encoding information, and then inputs it into the encoder, transforming the information in the sequence input to the encoder into a vector representation.

Images