Traffic jam intelligent identification method based on residual network
By adopting the residual network structure of the ResNet50 model in traffic congestion recognition, the problem of low recognition accuracy caused by traffic image complexity in the prior art is solved, and higher accuracy and precision are achieved.
Patent Information
- Application Number
- CN202411972959.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-04-25
AI Technical Summary
The road congestion recognition method based on video analysis in the prior art has a high complexity of traffic images, resulting in low recognition accuracy.
The intelligent traffic congestion recognition method based on residual network is adopted, and the traffic images are classified using the ResNet50 model. Low-level features are extracted through convolutional layers. The residual blocks enhance the learning effect through jump connections, the pooling layer reduces the spatial dimension, and the fully connected layer is classified.
It effectively solves the problem of gradient vanishing in deep networks, allowing the network to train deeper levels without losing performance, and improves the accuracy and precision of traffic congestion recognition.
Smart Images
Figure CN119888643A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of machine vision and particularly to an intelligent traffic congestion recognition method based on a residual network. Background Art
[0002] Identifying traffic congestion is an important part of road traffic condition monitoring and helps improve the operating efficiency of the transportation system. Inductive loop detectors were once the main tool for early traffic monitoring, but they provide limited traffic information. Although GPS data and road maps can also be used to detect congestion, they rely on a lot of resources and data collection. Today, many cities have installed low-cost camera networks that can monitor traffic flow more accurately. Hua Cui et al. studied the performance of deep convolutional neural networks (CNNs) in identifying highway traffic congestion states in surveillance camera images, using two classic convolutional networks AlexNet and GoogLeNet to classify congestion states. Ji Yu et al. proposed a method for highway congestion recognition based on the convolutional neural network AlexNet model using images. In view of the problem that the existing road congestion recognition method based on video analysis in the prior art has a low accuracy rate due to the high complexity of traffic images. Summary of the invention
[0003] The purpose of the present invention is to provide a traffic congestion intelligent identification method based on a residual network to address the problem of low accuracy in existing traffic congestion identification.
[0004] The technical solution adopted by the present invention to solve the above technical problems is:
[0005] The traffic congestion intelligent identification method based on residual network includes the following steps:
[0006] Step 1: Obtain the traffic image to be identified;
[0007] Step 2: Input the traffic image to be identified into the trained neural network for traffic congestion classification, wherein the neural network is a ResNet50 model, and the ResNet50 model includes a convolutional layer, a residual block, a pooling layer, and a fully connected layer;
[0008] The convolution layer is used to extract low-level features of the image, the residual block is used to connect the convolution layers through jump connections, the pooling layer is used to reduce the spatial dimension, and the fully connected layer is used to perform classification according to the features of the reduced spatial dimension.
[0009] Furthermore, the specific steps of step one are:
[0010] Obtain surveillance videos of four intersections, and collect images of the surveillance videos at intervals of 100 frames to obtain traffic images.
[0011] Furthermore, the neural network trained in step 2 is obtained by training a training set, and the training set is obtained by the following steps:
[0012] Step 21: Obtain a traffic image and randomly translate the traffic image by 25 pixels in four directions: upward, downward, left, and right;
[0013] Step 22: Enlarge the translated image to twice its original size;
[0014] Step 2 and 3: Crop all the enlarged images through center cropping to obtain square images, and construct the training set.
[0015] Furthermore, the loss function of the neural network is Cross Entropy Loss.
[0016] Furthermore, the trained neural network is trained using an SGD optimizer with a learning rate of 0.0015, a momentum of 0.9, and a batch size of 16.
[0017] The beneficial effects of the present invention are:
[0018] The technical solution of the present application is aimed at the road congestion recognition method based on video analysis in the prior art. For the road congestion recognition task with higher complexity, it needs to be implemented through a deep network. The deep network can achieve higher accuracy. However, the deep network is prone to gradient vanishing and degradation problems during training. In order to ensure that the network can be trained to a deeper level without losing performance and maintaining higher accuracy, a traffic congestion intelligent recognition method based on a residual network is proposed. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 A technical framework for image-based traffic congestion recognition;
[0020] Figure 2 This is an example image of traffic congestion;
[0021] Figure 3 This is an example of an image without traffic congestion;
[0022] Figure 4 An example of the comparison between images before and after enhancement for traffic congestion recognition;
[0023] Figure 5 It is the ResNet network model structure;
[0024] Figure 6 This is the structure diagram of the ResNet50 network model;
[0025] Figure 7 This is the structure diagram of the MobileNetV2 network model;
[0026] Figure 8 This is the structure diagram of the DenseNet121 network model. DETAILED DESCRIPTION
[0027] It should be particularly noted that, in the absence of conflict, the various embodiments disclosed in this application can be combined with each other.
[0028] Specific implementation method 1: The traffic congestion intelligent identification method based on residual network described in this implementation method includes the following steps:
[0029] Step 1: Obtain the traffic image to be identified;
[0030] Step 2: Input the traffic image to be identified into the trained neural network for traffic congestion classification, wherein the neural network is a ResNet50 model, and the ResNet50 model includes a convolutional layer, a residual block, a pooling layer, and a fully connected layer;
[0031] The convolution layer is used to extract low-level features of the image, the residual block is used to connect the convolution layers through jump connections, the pooling layer is used to reduce the spatial dimension, and the fully connected layer is used to perform classification according to the features of the reduced spatial dimension.
[0032] This application can be carried out through three models: ResNet50, MobileNetV2, and DenseNet121. ResNet50 adopts a residual learning structure. By introducing skip connections, it effectively solves the gradient vanishing problem in deep networks, allowing the network to train deeper layers without losing performance, thereby improving the model's expressive power and training efficiency. The model's accuracy, precision and other evaluation indicators are higher than those of MobileNetV2 and DenseNet121, so the residual network is more suitable for identifying traffic congestion.
[0033] In the ResNet50 model, the convolution layer extracts low-level features of the image, the residual block enhances the learning effect through skip connections, the pooling layer reduces the spatial dimension to reduce the computational burden, and finally the fully connected layer performs classification and outputs the probability. In the MobileNetV2 model, the depthwise separable convolution extracts the features of each channel, the reverse residual module increases the number of channels by expanding the 1×1 convolution, and then uses the depthwise separable convolution to extract features, and finally uses the 1×1 convolution to reduce the dimension. Finally, the classification result is output through the fully connected layer. In the DenseNet121 model, the image is passed through the convolution layer to extract low-level features. The dense block consists of multiple convolutional layers, each layer receives the output of the previous layer and enhances the feature representation. The transition layer is used to reduce the amount of computation, usually including convolution and pooling operations. Finally, the feature map is flattened and input into the fully connected layer for classification.
[0034] (1) The traffic congestion intelligent recognition method based on residual network, through the preprocessing of traffic congestion recognition images collected by surveillance video, makes the traffic congestion and non-congestion data sets balanced, avoids the model biased towards the majority class, and improves the performance of the model. By balancing the data set, a lower error rate can be obtained during training, and the model can better learn the characteristics of the minority class, thereby improving the overall prediction accuracy.
[0035] (2) Intelligent identification method of traffic congestion based on residual network. By introducing residual blocks, ResNet50 effectively solves the gradient vanishing problem in deep networks, allowing the network to train deeper layers without losing performance. The depth and residual structure of the model enable it to perform better in the complex task of traffic congestion identification, and it has good generalization ability and low computational complexity. The accuracy of ResNet50 model in traffic congestion identification is higher than that of MobileNetV2 and DenseNet121. MobileNetV2 focuses more on computational efficiency and is suitable for resource-constrained devices, but may not perform as well as ResNet50 when the task complexity is high. DenseNet121 enhances information transmission through dense connections and can provide better performance in some specific tasks, but its computational complexity is high.
[0036] Specific implementation method 2: This implementation method is a further description of the specific implementation method 1. The difference between this implementation method and the specific implementation method 1 is that the specific steps of step 1 are:
[0037] Obtain surveillance videos of four intersections, and collect images of the surveillance videos at intervals of 100 frames to obtain traffic images.
[0038] Specific implementation method three: This implementation method is a further explanation of specific implementation method two. The difference between this implementation method and specific implementation method two is that the trained neural network in step two is obtained by training a training set, and the training set is obtained by the following steps:
[0039] Step 21: Obtain a traffic image and randomly translate the traffic image by 25 pixels in four directions: upward, downward, left, and right;
[0040] Step 22: Enlarge the translated image to twice its original size;
[0041] Step 2 and 3: Crop all the enlarged images through center cropping to obtain square images, and construct the training set.
[0042] Specific implementation method four: This implementation method is a further explanation of specific implementation method three. The difference between this implementation method and specific implementation method three is that the loss function of the neural network is Cross Entropy Loss.
[0043] Specific implementation mode five: This implementation mode is a further explanation of specific implementation mode four. The difference between this implementation mode and specific implementation mode four is that the trained neural network is trained through the SGD optimizer, in which the learning rate is 0.0015, the momentum is 0.9, and the batch size is 16.
[0044] Example:
[0045] The intelligent identification method of traffic congestion based on residual network specifically includes the following steps:
[0046] Step 1: Collect images for traffic congestion recognition through surveillance video;
[0047] The images of traffic congestion recognition in the first embodiment are taken from the surveillance videos of the four intersections of Hongye Road, Hujing Road, Tang'an Road and Tangfu Road on Yingbin Avenue in Dongguan City, Guangdong Province. The video is collected at an interval of every 100 frames, and a total of 23,796 images of traffic congestion recognition are collected, with an image size of 1920×1080. Figure 2 This is an example image of a traffic congestion state in the first embodiment of the present invention. Figure 3 This is an example image of a non-congested traffic state in the first embodiment of the present invention.
[0048] Step 2: preprocessing the collected traffic congestion recognition images;
[0049] The congested dataset was randomly translated by 25 pixels in the four directions of upward, downward, left and right to enhance the dataset, and the congested sample dataset was expanded to twice the original size, totaling 17,490 images. Figure 4 This is an example diagram of the comparison of images before and after enhancement in traffic congestion recognition in an implementation of the present invention.
[0050] Step 3: Create an image dataset for traffic congestion recognition;
[0051] Manually label the collected images and mark them as congested, non-congested, unknown, and damaged. There are 2 damaged images in total, and try to avoid marking images as unknown. Divide the personnel into two groups, each group labels the images, and then compare the results of the two groups. For those with inconsistent labels, the two groups finally reach an agreement after discussion. For the images marked with traffic congestion and non-congestion, the data set is randomly divided into a training set and a test set in a ratio of 8:2. The statistical chart of the number of images of each category in the data set of traffic congestion identification in Example 1 is shown in Table 1.
[0052] Table 1 Image dataset for traffic congestion recognition
[0053]
[0054]
[0055] Step 4: Construct three image classification models for traffic congestion recognition: ResNet50, MobileNetV2, and DenseNet121;
[0056] The image classification model of ResNet traffic congestion recognition constructed according to the data set includes three parts: Input, Backbone, and Output. The ResNet network model structure diagram in Example 1 is shown in FIG. Figure 5 shown.
[0057] The input part uses transforms in PyTorch to preprocess the images in the dataset and then pass them into the model. The preprocessing operation can be divided into normalization transformation and resize transformation. In the normalization transformation operation, the mean [123.675, 116.28, 103.53] and standard deviation [58.395, 57.12, 57.375] of the pre-trained ImageNet dataset are used to pre-train the network, which can make the model training achieve better convergence effect. The resize transformation resizes the input image to 224×224 before inputting the image to the model, which can ensure that the model receives a consistent input size, which is convenient for better training of the model.
[0058] The ResNet residual neural network was proposed to solve the problem of network degradation caused by the increase of network depth. The shortcut connection method introduced in the traffic congestion intelligent identification method based on the ResNet residual network can effectively alleviate the gradient vanishing problem in the network training process and solve the overfitting problem that deep networks are prone to.
[0059] like Figure 6As shown in the figure, the ResNet50 network consists of 49 convolutional layers and 1 fully connected layer, which is divided into 7 parts: the first part is a 7×7 convolutional layer, excluding residual blocks, which mainly performs convolution, regularization, activation function and maximum pooling operations; the second to fifth parts contain Conv2_ to Conv5_ residual layers, which are composed of 3, 4, 6 and 3 residual blocks respectively, and each residual block is composed of 1×1, 3×3 and 1×1 convolutional layers; the sixth part is an average pooling layer, and the seventh part is a fully connected layer. These two parts are used to reduce the number of parameters of the network, prevent overfitting, and output the classification results of the image for traffic congestion recognition. In order to reduce the risk of overfitting and accelerate training, batch normalization layers (BN) are added after all convolutional layers, and rectified linear units (ReLU) are added after the 7×7 convolutional layer and the last layer of each residual block.
[0060] like Figure 7 As shown in the figure, MobileNetV2 is an efficient convolutional neural network. Its core architecture uses an inverted residual block and combines depthwise separable convolution to significantly reduce the amount of computation while maintaining good performance. First, the input image passes through a standard 3×3 convolution layer with a step size of 2, followed by BatchNorm and ReLU6 activation functions for preliminary feature extraction. Then, the core part of the network is the inverted residual block, each of which contains three main steps: first, the dimension is increased by 1×3 convolution to increase the number of channels, thus forming a bottleneck layer; then, the spatial feature extraction is performed using depthwise separable convolution, which greatly reduces the amount of computation; finally, another 1×1 convolution is used to reduce the dimension and restore the lower number of channels. Each inverted residual block also contains a skip connection, that is, the input and output are added through a skip connection, thereby enhancing the flow of information and facilitating training. In addition, linear activation (Linear Bottleneck) is used in the last layer of the inverted residual block to avoid the information loss that may be caused by nonlinear activation functions, thereby retaining more features. Finally, after several inverted residual blocks, the network reduces the feature map to a fixed size through global average pooling, and then classifies and outputs it through a fully connected layer. The entire network architecture significantly reduces the computational complexity through this lightweight design, while still achieving efficient image classification performance.
[0061] like Figure 8As shown in the figure, the structure of the DenseNet121 network input image passes through a 7×7 convolution layer with a step size of 2 for preliminary feature extraction and size reduction, followed by further processing through batch normalization and ReLU activation function. The core part of the network consists of four dense blocks, each of which contains multiple convolution layers (layers 6-12). The input of each convolution layer comes not only from the previous layer, but also includes the output of all previous layers in the block. This dense connection ensures efficient reuse of features and information flow. Each convolution layer usually uses a 3×3 convolution kernel to extract features. Between different dense blocks, transition layers are used to reduce the dimension and size of the feature map. The transition layer usually consists of a 1×1 convolution layer and a 2×2 average pooling layer to reduce the computational burden and avoid overfitting. After the last dense block, the network averages each channel of the feature map through a global average pooling layer to obtain a fixed-length feature vector, and finally the classification output is performed through a fully connected layer. This design enables DenseNet121 to effectively reduce the number of parameters and improve computational efficiency while maintaining high expressive power.
[0062] The Output part predicts the model output results and selects Cross Entropy Loss as the loss function to measure the difference between the two probability distributions. It can well reflect the prediction accuracy of the model and is less sensitive to outliers.
[0063] Step 5: Train the above-mentioned classification model for traffic congestion identification. The SGD optimizer is selected for all models, the learning rate is set to 0.0015, the momentum is set to 0.9, and the batch size is set to 16. Table 2 shows the experimental results of different models, and the models are evaluated through precision, recall, F-score and accuracy statistical indicators.
[0064] Table 2 Comparison of results of different models
[0065]
[0066] Step 6: Classify the images of traffic congestion recognition;
[0067] First, the image is input into the trained model and the local features of the image are extracted through convolution operation. Then, the convolution result is nonlinearly activated through the activation function to enhance the network's expressive power. Next, the pooling layer downsamples the feature map to reduce the spatial dimension and improve computational efficiency. In the deep layers of the network, through multiple layers of convolution and pooling layers, the network is able to extract more and more complex features. The extracted features are then flattened and finally classified through the fully connected layer. Finally, the output layer uses the Softmax function to give the classification result of the image. The training of the entire network adjusts the weights through the loss function and optimization algorithm, so that the classification accuracy is gradually improved until the expected performance is achieved.
[0068] The traffic congestion intelligent recognition method based on residual network can balance the data sets of traffic congestion and non-congestion by preprocessing the images of traffic congestion recognition collected by monitoring video, avoid the model biased towards the majority class, and improve the performance of the model. By balancing the data set, a lower error rate can be obtained during training, and the model can better learn the characteristics of the minority class, thereby improving the overall prediction accuracy.
[0069] Intelligent traffic congestion identification method based on residual network. By introducing residual blocks, ResNet50 effectively solves the gradient vanishing problem in deep networks, allowing the network to train deeper layers without losing performance. The depth and residual structure of the model enable it to perform better in the complex task of traffic congestion identification, and it has good generalization ability and low computational complexity. The accuracy of ResNet50 model in traffic congestion identification is higher than that of MobileNetV2 and DenseNet121. MobileNetV2 pays more attention to computational efficiency and is suitable for resource-constrained devices, but may not perform as well as ResNet50 when the task complexity is high. DenseNet121 enhances information transmission through dense connections and can provide better performance in some specific tasks, but its computational complexity is high.
[0070] It should be noted that the specific implementation is only an explanation and description of the technical solution of the present invention, and cannot be used to limit the scope of protection of the rights. Any partial changes made according to the claims and description of the present invention should still fall within the scope of protection of the present invention.
Claims
1. Intelligent identification method of traffic congestion based on residual network, characterized by The following steps are involved: Step 1: Obtain the traffic image to be identified; Step 2: Input the traffic image to be identified into the trained neural network for traffic congestion classification, wherein the neural network is a ResNet50 model, and the ResNet50 model includes a convolutional layer, a residual block, a pooling layer, and a fully connected layer; The convolution layer is used to extract low-level features of the image, the residual block is used to connect the convolution layers through jump connections, the pooling layer is used to reduce the spatial dimension, and the fully connected layer is used to classify according to the features of the reduced spatial dimension.
2. The traffic congestion intelligent identification method based on residual network according to claim 1 is characterized in that The specific steps of step one are: Obtain surveillance videos of four intersections, and collect images of the surveillance videos at intervals of 100 frames to obtain traffic images.
3. The traffic congestion intelligent identification method based on residual network according to claim 2 is characterized in that The trained neural network in step 2 is obtained by training a training set, and the training set is obtained by the following steps: Step 21: Obtain a traffic image and randomly translate the traffic image by 25 pixels in four directions: upward, downward, left, and right; Step 22: Enlarge the translated image to twice its original size; Step 2 and 3: Crop all the enlarged images through center cropping to obtain square images, and construct the training set.
4. The traffic congestion intelligent identification method based on residual network according to claim 3 is characterized in that The loss function of the neural network is Cross Entropy Loss.
5. The traffic congestion intelligent identification method based on residual network according to claim 4 is characterized in that The trained neural network is trained using the SGD optimizer with a learning rate of 0.0015, a momentum of 0.9, and a batch size of 16.