Expressway real-time congestion identification method based on computer vision technology

Abstract

The invention discloses a real-time highway congestion identification method based on a computer vision technology, relates to the technical field of traffic safety, and aims to solve the problem of low accuracy of real-time highway congestion identification in an existing method. The method comprises the following steps: extracting vehicle track data in a highway monitoring video by using YOLOv5 and Deep SORT algorithms; and the macroscopic and microscopic parameters of the traffic flow are calculated. And through the weight of each parameter during description of the congestion degree, a comprehensive congestion measurement index is constructed. And real-time highway congestion identification is carried out by using the index. According to the technical scheme of the invention, the accuracy of real-time highway congestion identification can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety, and in particular to a real-time highway congestion recognition method based on computer vision technology. Background Art

[0002] Highways are arteries for large-scale, long-distance transportation. Real-time identification of highway traffic congestion levels is a prerequisite for rapidly improving the efficiency of large-scale transportation. Therefore, real-time identification of highway traffic congestion levels has always been a research hotspot in the field of traffic control. At present, traffic congestion identification methods mainly include two categories: based on traditional traffic flow models and based on machine learning. The former mainly uses macroscopic traffic flow models and mathematical equations to describe traffic flow, and uses flow parameters to quantitatively analyze and measure traffic status to determine whether traffic is congested. However, this method cannot accurately characterize the state of traffic congestion, which leads to the problem of low accuracy of real-time congestion identification on highways. Summary of the invention

[0003] The purpose of the present invention is to provide a method for real-time highway congestion recognition based on computer vision technology to address the problem of low accuracy of real-time highway congestion recognition in existing methods.

[0004] The technical solution adopted by the present invention to solve the above technical problems is:

[0005] A method for real-time highway congestion recognition based on computer vision technology comprises the following steps:

[0006] Step 1: Obtain video data of the monitoring area during peak hours and input the video data into YOLOv5 to obtain the vehicle detection frame of each frame;

[0007] Step 2: Based on the vehicle detection frame of each frame, DeepSORT is used to track the target and obtain the location information of the vehicle in each frame image;

[0008] Step 3: Based on the position information of the vehicle in each frame of the image, the traffic volume, traffic flow density, traffic flow speed and average headway time during peak hours are obtained;

[0009] Step 4: Construct a comprehensive congestion measurement index T based on the peak hour traffic volume, traffic flow density, traffic flow speed and average headway. The comprehensive congestion measurement index T is expressed as:

[0010]

[0011] Among them, Q is the traffic volume, K is the traffic flow density, v is the traffic flow speed, h t is the average headway time, w1, w2, w3, w4 are weight coefficients;

[0012] Step 5: Identify congestion based on the comprehensive congestion measurement index T.

[0013] Furthermore, the traffic volume Q is expressed as:

[0014] Q=N / t

[0015] Where N is the total number of vehicles in the observation period, and t is the observation period.

[0016] Furthermore, the traffic flow density K is expressed as:

[0017] K=N / L

[0018] Where L is the length of the observation section.

[0019] Furthermore, the traffic flow speed v is expressed as:

[0020]

[0021] Among them, t i is the time required for the i-th vehicle to pass through section L during the observation period.

[0022] Furthermore, the average headway h t It is expressed as:

[0023]

[0024] Furthermore, the weight coefficient is expressed as:

[0025]

[0026] Where j = 1, 2, 3, 4, g j is the entropy difference coefficient of the jth parameter, H j is the entropy value of the jth parameter, and m is the number of parameters.

[0027] Furthermore, the entropy difference coefficient g of the jth parameter j It is expressed as:

[0028] g j =1-H j .

[0029] Furthermore, the entropy value H of the jth parameter j It is expressed as:

[0030]

[0031] Among them, n is the total number of samples, p ij is the proportion of the jth parameter in the i-th data sample.

[0032] Furthermore, the proportion p of the jth parameter in the i-th data sample ij It is expressed as:

[0033]

[0034] Among them, x ij ' is the value after Min-Max normalization.

[0035] Furthermore, the x ij ' is expressed as:

[0036]

[0037] Among them, x ij is the original data, and are the minimum and maximum values ​​in the original data, respectively.

[0038] The beneficial effects of the present invention are:

[0039] This application uses YOLOv5 and Deep SORT algorithms to extract vehicle trajectory data from highway monitoring videos and calculates macro and micro parameters of traffic flow. A comprehensive congestion measurement index is constructed by weighting each parameter in describing the degree of congestion. The index is used to identify real-time congestion on highways. The technical solution of this application can improve the accuracy of real-time congestion identification on highways. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic diagram of vehicle detection;

[0041] Figure 2 This is a schematic diagram of the surveillance video of the Luzhou section of the Chengdu-Chongqing Ring Expressway;

[0042] Figure 3 This is the congestion comprehensive index diagram for a typical period at 516km+80m;

[0043] Figure 4 This is the congestion comprehensive index diagram for typical time periods at 558km+340m;

[0044] Figure 5 This is the congestion comprehensive index diagram for typical time periods at 585km+170m;

[0045] Figure 6 This is a schematic diagram of the FCM clustering results at 516km+80m;

[0046] Figure 7 This is a schematic diagram of the FCM clustering results at 558km+340m;

[0047] Figure 8 This is a schematic diagram of the FCM clustering results at 585km+170m;

[0048] Fig. 9 This is a schematic diagram of the K-means clustering classification results at 516km+80m;

[0049] Fig.10 This is a schematic diagram of the K-means clustering classification results at 558km+340m;

[0050] Fig.11 This is a schematic diagram of the K-means clustering classification results at 585km+170m. DETAILED DESCRIPTION

[0051] It should be particularly noted that, in the absence of conflict, the various embodiments disclosed in this application can be combined with each other.

[0052] Specific implementation method 1: This implementation method is a real-time highway congestion identification method based on computer vision technology, and the specific steps are as follows:

[0053] Firstly, based on surveillance videos, computer vision technology is used to extract vehicle trajectories and calculate traffic flow macro and micro parameters. Secondly, the entropy method is used to construct a comprehensive measurement index of traffic congestion, and a real-time highway congestion level classification method based on fuzzy C-means is proposed. Finally, empirical analysis is carried out to provide methodological support for the identification and relief of traffic congestion on typical sections of highways.

[0054] The steps of the real-time highway congestion level classification method based on computer vision technology are as follows:

[0055] Step 1: Extracting traffic flow information from surveillance videos

[0056] Computer vision technology (YOLOv5+Deep SORT) and deep convolutional neural networks are used to accurately identify the location and type of vehicles in surveillance videos. Specifically, continuous frame images are extracted from the input video for processing, and YOLOv5 is used to extract features and detect targets on the video images, and the vehicle position is accurately calibrated by the generated bounding box. After the vehicle target is detected, DeepSORT needs to be further applied to track and obtain the vehicle trajectory. Specifically, the deep feature vector of each vehicle is extracted through the convolutional neural network, and the similarity between the vehicle feature vector of the current frame and the feature vector of the previous frame is calculated. The deep learning model and the Hungarian algorithm are used for target matching to ensure that the identity of each vehicle is correctly associated. After matching, DeepSORT performs data association and binds the vehicle in the current frame with the vehicle in the historical frame. If the vehicle can be successfully matched, its tracking status will be updated; otherwise, it may be marked as a new vehicle or a lost target. Finally, using the Kalman filter, DeepSORT updates the vehicle's motion trajectory and predicts its future position, thereby achieving robust and accurate vehicle tracking.

[0057] As attached Figure 1 As shown in the figure, during the vehicle detection process, the vehicle trajectory (including each vehicle's ID, lane direction, location coordinates, etc.) can be accurately extracted to calculate traffic flow parameters including flow, density, speed, number of vehicles, and headway. It should be noted that in order to facilitate real-time traffic analysis, traffic flow information is collected once a minute.

[0058] Step 2: Construction of comprehensive congestion measurement index

[0059] The entropy method is applied to combine the macroscopic parameters of traffic volume Q, traffic flow speed v, traffic flow density K and the microscopic parameters of headway h. t Construct a comprehensive congestion measurement index.

[0060] (1) Traffic volume

[0061] Traffic volume is the total number of vehicles passing through a certain section of the road within a given observation time, and its calculation formula is:

[0062] Q=N / t

[0063] Where: Q is the traffic volume (veh / h), N is the total number of vehicles in the observation period (veh), and t is the observation period (h).

[0064] (2) Traffic flow density

[0065] Traffic flow density is the number of vehicles per unit length of road section at a certain moment, usually expressed as the average value over the total observation time. Its calculation formula is:

[0066] K=N / L

[0067] Where: K is the traffic flow density at a certain moment (veh / km), N is the total number of vehicles in the observation period (veh), and L is the length of the observation section (km).

[0068] (3) Traffic flow speed

[0069] Traffic flow speed refers to the average speed of vehicles passing a certain point. This speed is the average speed of all vehicles passing a certain point. The calculation formula is:

[0070]

[0071] Where: v is the traffic flow speed (km / h), N is the total number of vehicles in the observation period (veh), L is the length of the observation section (km), t i is the time required for the i-th vehicle to pass through section L during the observation period.

[0072] (4) Average headway

[0073] The headway is the time difference between the two vehicles passing a certain point on the road. The average headway of all vehicles on the observed section is taken as the average headway, and its calculation formula is:

[0074]

[0075] Where: h t is the average headway (s), and Q is the traffic volume (veh / h).

[0076] In the field of information theory, entropy is used to measure the uncertainty of a random process, that is, the degree of unpredictability. The amount of information is related to the probability of a random event. The greater the probability of a random event, the greater the amount of information, and vice versa. Information entropy is the expectation of the amount of information generated by all possible random event results. The entropy value is used to determine the discreteness of the four traffic flow parameters. The smaller the entropy value, the greater the entropy weight, the greater the discreteness, and the greater the impact of the parameter on traffic congestion identification. Therefore, the weight of each parameter is determined by the entropy method. The specific steps are as follows:

[0077] Select parameters and data samples to construct the following initial parameter matrix:

[0078]

[0079] Among them, the element x of matrix X ij Represents the value of the jth parameter in the ith data sample, i = 1, 2, ..., n; j = 1, 2, ..., m, n is the total number of samples, and m is the number of parameters.

[0080] In order to ensure that the contribution of each parameter to the comprehensive index is comparable, all parameters need to be standardized so that they have the same dimension and range. This paper adopts Min-Max normalization, that is:

[0081]

[0082] Among them, x ij ' represents the value after Min-Max normalization, x ij is the original data, and are the minimum and maximum values ​​in the original data respectively.

[0083] Calculate the proportion of the jth parameter in the i-th data sample:

[0084]

[0085] Among them, p ij is the proportion of the jth parameter in the i-th data sample, x ij ' is the parameter value after normalization.

[0086] Calculate the entropy value of the jth parameter:

[0087]

[0088] Among them, H j is the entropy value of the jth parameter, and n is the total number of samples.

[0089] Calculate the entropy weight. The entropy weight increases as the entropy value decreases, indicating that the parameter is more important. The specific calculation is as follows:

[0090]

[0091] g j =1-H j

[0092] Among them, g j is the entropy difference coefficient of the jth parameter, w j is the entropy weight of the jth parameter.

[0093] According to the entropy weight, the congestion comprehensive measurement index T can be expressed as:

[0094]

[0095] Among them, w1, w2, w3, and w4 are weight coefficients. The larger the comprehensive metric index value, the more serious the traffic congestion.

[0096] Step 3: Congestion level classification

[0097] Based on the congestion comprehensive measurement index, the traffic congestion level is graded by clustering method. Clustering is one of the common unsupervised machine learning algorithms. It can process large and complex data sets. It can also use the algorithm's association rules, classification and other preprocessing steps to obtain basic data to achieve more accurate and efficient feature extraction or classification. For the classification of the congestion comprehensive measurement index, the data is noisy or has a complex cluster structure. The fuzzy C-means (FCM) clustering algorithm is more suitable because it is a soft clustering algorithm that allows data points to belong to multiple clusters. The relationship between data points and each cluster is represented by the degree of membership. It can handle non-spherical clusters and provide more flexible clustering results. The FCM clustering method is selected to cluster the congestion comprehensive measurement index to achieve the classification of congestion levels.

[0098] The basic steps of the FCM algorithm are as follows:

[0099] ① Initialization: Randomly initialize the membership of each data point to each cluster (membership matrix). Randomly initialize the cluster center.

[0100] ② Calculate cluster center: Calculate each cluster center based on the current membership degree. For one-dimensional data, the cluster center is the weighted average.

[0101] ③ Update membership: Update the membership of each data point according to the current cluster center. Calculate the membership according to the distance from each data point to each cluster center, and weight the distance according to the fuzzy parameter m.

[0102] ④ Repeat steps ② and ③ until the stopping condition is met (such as the membership change is less than a certain threshold or the maximum number of iterations is reached).

[0103] In order to further verify the validity of the FCM clustering results and the practical significance of the constructed comprehensive congestion measurement index, the FCM clustering results were evaluated in two aspects. First, the internal evaluation indicators silhouette coefficient, Davies-Bouldin index, and Calinski-Harabasz index were selected to help judge the stability of the clustering results. Then, K-means was used to cluster and grade the minute-level traffic density and traffic speed, and the overlap coefficient of the FCM clustering data and the K-means clustering data was calculated to verify the practical significance of the comprehensive congestion measurement index and congestion classification.

[0104] (1) Silhouette coefficient

[0105] Silhouette coefficient is a method to evaluate clustering quality, which is used to measure the relationship between each object and its cluster and the nearest neighbor cluster. Its value is between -1 and 1. The closer the value is to 1, the better the clustering effect. Its calculation formula is as follows:

[0106]

[0107] Among them, S(i) is the silhouette coefficient of point i; a(i) is the average distance between i and all points in its cluster; b(i) is the average distance between i and all points in other clusters that are closest to i.

[0108] (2) Davies-Bouldin Index

[0109] Davies-Bouldin Index (DBI) is one of the statistical indicators of the optimal number of clusters. It measures the degree of overlap between clusters. The smaller the value, the better the quality of clustering. It is expressed as:

[0110]

[0111] Where m is the number of clusters; i, j is the number of clusters; d(C i ,C j ) is cluster C i and C j The distance between them; σ(C i ) is the average distance of points within cluster i; σ(C j ) is the average distance of the points within cluster j.

[0112] (3) Calinski-Harabasz index

[0113] The Calinski-Harabasz index is an effective indicator for measuring clustering quality. It measures the ability of clusters to be as close as possible within a cluster and as far apart as possible between clusters. The higher the index value, the better the clustering. Its calculation formula is:

[0114]

[0115] Among them, CHI is the Calinski-Harabasz index; TSS is the square of the total style (tracking sample variance), WSS is the square of the within-group style (within-cluster variance), k is the number of clusters, and p is the number of features.

[0116] The K-means clustering algorithm is simple, intuitive, easy to understand, fast in calculation, and widely applicable. It works better for high-dimensional data and clusters with clear boundaries.

[0117] The basic steps of the K-means algorithm are as follows:

[0118] ① Initialization: Select the number K of clusters to be divided, and randomly initialize K cluster centers (you can also select K points in the sample as the initial cluster centers).

[0119] ② Assign samples: For each sample, calculate the distance between it and the center point of each cluster and assign it to the cluster with the closest distance.

[0120] ③ Update cluster center: For each cluster, calculate the average value of all its samples and use the average value as the new cluster center.

[0121] ④ Repeat steps ② and ③ until the cluster center no longer changes or the maximum number of iterations is reached.

[0122] The overlap coefficient is a measure of how closely two sets overlap, particularly when the data is of different sizes. It is defined as the size of the intersection divided by the size of the smaller set:

[0123]

[0124] Among them, A is set one and B is set two.

[0125] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments.

[0126] Step 1: Extracting traffic flow data

[0127] For the Luzhou section of the Chengdu-Chongqing Ring Expressway, we selected surveillance video data at 516km+80m, 558km+340m, and 585km+170m during the morning, afternoon, and evening peaks during the May Day holiday (see Figure 2 ), and obtain two-way 36h traffic flow data for experimental analysis.

[0128] The selected road section is a basic section of the expressway, where the traffic flow is relatively stable, making it easier to analyze the congestion status.

[0129] The extracted vehicle trajectory information includes each vehicle’s ID, lane direction, time of entering the recognition area, and time of leaving the recognition area. Table 1 shows some of the vehicle trajectory data obtained. Based on the vehicle trajectory information, some traffic flow parameters are extracted, as shown in Table 2.

[0130] Table 1. Snapshot of vehicle trajectory data

[0131]

[0132] Table 2 Traffic flow data example

[0133]

[0134] Calculation of congestion comprehensive index

[0135] Table 3 shows the weight values ​​of various parameters for driving in the two directions of Chengdu and Chongqing during typical time periods in the morning, afternoon and evening.

[0136] Table 3 Weights of four traffic flow parameters

[0137]

[0138]

[0139] Figure 3 , Figure 4 and Figure 5 It is the calculation result of the comprehensive measurement index of two-way congestion in typical time periods at the study location.

[0140] (2) Congestion level classification results

[0141] Figure 6 , Figure 7 Figure 8 The FCM clustering results of the comprehensive measurement index of two-way congestion in typical time periods at the study location. The traffic congestion level in each time period is divided into three levels: level one, level two, and level three. Specifically, the level one congestion level indicates smooth traffic with almost no congestion; the level two congestion level indicates that there is slight traffic congestion, which may have a slight impact on mobility; the level three congestion level indicates that there is mild congestion, and the traffic flow is slightly slowed down but still remains within a relatively smooth range. This classification method provides a detailed description of the congestion status of traffic flow, which helps to more accurately monitor and analyze changes in traffic flow, and provide valuable data support for traffic management departments so that appropriate measures can be taken for optimization and management.

[0142] Table 4 shows the evaluation index of FCM clustering results. When evaluating FCM clustering results, the silhouette coefficient, Davies-Bouldin index and Calinski-Harabasz index provide key quality evaluation indicators. The silhouette coefficient is in the range of 0.5 to 0.58, indicating that the clustering results have a moderate degree of intra-cluster compactness and inter-cluster separation, indicating that the obtained cluster structure is relatively reasonable. The value of Davies-Bouldin index is between 0.49 and 0.6. The lower value reflects the better separation between clusters and the higher compactness within clusters, which further verifies the effectiveness of clustering and the representativeness of clusters. The high value of Calinski-Harabasz index between 189.48 and 300.52 indicates the significant separation and compactness of clusters in the data space in the clustering results. These evaluation indicators show that the clusters generated by the FCM clustering algorithm on the current data set are of high quality and effectiveness.

[0143] Table 4 Evaluation of FCM clustering results

[0144]

[0145] In order to further verify the effectiveness of the FCM clustering results, K-means was used to cluster density-speed. The results are shown in the figure below. Fig. 9 , Fig.10 , Fig.11As shown. According to density and speed, the traffic flow state is divided into three levels: low-density-low-speed cluster, low-density-medium-speed cluster and low-density-high-speed cluster. Among them, the low-density-low-speed cluster represents a state with sparse traffic flow and low vehicle speed, which is usually related to poor traffic fluency and possible mild congestion; the low-density-medium-speed cluster refers to a state with less traffic flow but moderate vehicle speed, which is common in the stage where traffic flows smoothly but has not yet reached high-speed traffic; the low-density-high-speed cluster means that the vehicle speed is high when the traffic flow is small, which usually indicates that the road conditions are good and there is no significant obstruction to traffic flow. This result can be used to verify the practical significance of the FCM clustering results of the proposed congestion comprehensive measurement index.

[0146] In order to further verify the superiority of the constructed congestion comprehensive measurement index, this study also constructed an index T' composed of macro parameters of traffic flow, traffic density and traffic speed, which is expressed as follows:

[0147] T′=w′1K+w′2Q+w′3γ v

[0148] Among them, w'1, w'2 and w'3 are weight coefficients, K represents traffic density, Q represents traffic flow, γ v Stands for speed congestion index.

[0149] In order to compare the comprehensive measurement index T, the macro parameter index T' and the micro parameter index In order to evaluate the effect of partitioning congestion levels, the overlap coefficient between FCM clustering and K-means clustering was calculated, as shown in Table 4.

[0150] Table 4 Overlap coefficient between FCM clustering and K-means clustering under various indicators

[0151]

[0152] As shown in Table 4, the overlap between the two clustering methods is as high as 62.5%, which is higher than the macro parameter T' and micro parameter The results show that the constructed congestion comprehensive measurement index is accurate and reliable. And the hierarchical overlap coefficient of clustering results is as high as 62.5%, indicating that there is a significant consensus on the division of traffic congestion status in FCM clustering and K-means clustering. This further confirms that the congestion comprehensive measurement index can truly reflect the traffic congestion status of highway sections.

[0153] It should be noted that the specific implementation is only an explanation and description of the technical solution of the present invention, and cannot be used to limit the scope of protection of the rights. Any partial changes made according to the claims and description of the present invention should still fall within the scope of protection of the present invention.

Claims

1. A real-time highway congestion identification method based on computer vision technology, characterized in that The following steps are involved: Step 1: Obtain video data of the monitoring area during peak hours and input the video data into YOLOv5 to obtain the vehicle detection frame of each frame; Step 2: Based on the vehicle detection frame of each frame, DeepSORT is used to track the target and obtain the location information of the vehicle in each frame image; Step 3: Based on the position information of the vehicle in each frame of the image, the traffic volume, traffic flow density, traffic flow speed and average headway time during peak hours are obtained; Step 4: Construct a comprehensive congestion measurement index T based on the peak hour traffic volume, traffic flow density, traffic flow speed and average headway. The comprehensive congestion measurement index T is expressed as: Among them, Q is the traffic volume, K is the traffic flow density, v is the traffic flow speed, h t is the average headway time, w1, w2, w3, w4 are weight coefficients; Step 5: Identify congestion based on the comprehensive congestion measurement index T.

2. The method for real-time highway congestion identification based on computer vision technology according to claim 1 is characterized in that The traffic volume Q is expressed as: Q=N / t Where N is the total number of vehicles in the observation period, and t is the observation period.

3. The method for real-time highway congestion identification based on computer vision technology according to claim 2 is characterized in that The traffic flow density K is expressed as: K=N / L Where L is the length of the observation section.

4. The method for real-time highway congestion identification based on computer vision technology according to claim 3 is characterized in that The traffic flow speed v is expressed as: Among them, t i is the time required for the i-th vehicle to pass through section L during the observation period.

5. The method for real-time highway congestion identification based on computer vision technology according to claim 4 is characterized in that The average headway h t It is expressed as:

6. The method for real-time highway congestion identification based on computer vision technology according to claim 5 is characterized in that The weight coefficient is expressed as: Where j = 1, 2, 3, 4, g j is the entropy difference coefficient of the jth parameter, H j is the entropy value of the jth parameter, and m is the number of parameters.

7. The method for real-time highway congestion identification based on computer vision technology according to claim 6 is characterized in that The entropy difference coefficient g of the jth parameter j It is expressed as: g j =1-H j 。 8. The method for real-time highway congestion identification based on computer vision technology according to claim 7 is characterized in that The entropy value H of the jth parameter j It is expressed as: Among them, n is the total number of samples, p ij is the proportion of the jth parameter in the i-th data sample.

9. The method for real-time highway congestion identification based on computer vision technology according to claim 8 is characterized in that The proportion p of the jth parameter in the i-th data sample ij It is expressed as: Among them, x ij ' is the value after Min-Max normalization.

10. The method for real-time highway congestion identification based on computer vision technology according to claim 9, characterized in that The x ij ' is expressed as: Among them, x ij is the original data, and are the minimum and maximum values ​​in the original data, respectively.

Images