A traffic congestion judgment method and system based on a heat map model
By using a heatmap-based traffic congestion assessment method, lane segmentation is performed using grayscale preprocessing, optical flow vector clustering, and Gaussian mixture model to generate heatmaps for multi-dimensional assessment. This method solves the problems of poor foreground extraction performance in complex traffic conditions and high computational cost of deep learning in traditional methods, thus improving both accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- FUZHOU UNIV
- Filing Date
- 2023-03-29
- Publication Date
- 2026-07-31
AI Technical Summary
Traditional traffic congestion assessment methods perform poorly in foreground extraction under complex vehicle conditions, and deep learning-based methods are computationally intensive and have high hardware requirements, resulting in inaccurate assessments and difficulty in applying them to real-world traffic systems.
A heatmap-based approach is adopted, which uses grayscale preprocessing, optical flow vector clustering and Gaussian mixture model to segment lanes, generate a basic template with vehicle location information, and generate a heatmap based on two dimensions: queuing time and queuing length for judgment.
It improves the accuracy and efficiency of traffic congestion assessment, reduces the computational load, and can be applied in real-world traffic systems.
Smart Images
Figure CN116453334B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to methods for judging traffic congestion, and more particularly to a method and system for judging traffic congestion based on a heat map model. Background Technology
[0002] Traditional traffic congestion assessment relies on frame differencing or background subtraction for vehicle flow detection. These methods suffer from poor foreground segmentation and lack adaptability and matching capabilities. In complex traffic conditions or due to camera angle limitations, foreground extraction becomes ineffective, hindering accurate road condition assessment.
[0003] While the more advanced deep learning-based traffic flow detection methods currently available offer high accuracy and robustness, they suffer from high computational demands and complex structures, leading to high hardware performance requirements and making them difficult to apply in real-world traffic systems.
[0004] Furthermore, the two technologies mentioned above only judge traffic congestion from the single dimension of vehicle flow, which makes the judgment of traffic congestion inaccurate. Summary of the Invention
[0005] The purpose of this invention is to provide a traffic congestion judgment method and system based on a heat map model, which further simplifies the traffic congestion judgment process, provides multi-dimensional judgment criteria, and improves the accuracy of judgment while reducing the amount of calculation.
[0006] To achieve the above objectives, the technical solution of this invention is: a traffic congestion judgment method based on a heatmap model. First, the input video stream is preprocessed by grayscale conversion. Then, lane segmentation is performed based on optical flow vector clustering to obtain regions of interest (ROIs) that meet the requirements. Within the ROIs, the foreground is extracted based on a Gaussian mixture model. A basic template with vehicle location information is generated using the obtained foreground image. The basic template is then weighted sequentially based on two dimensions: queuing time and queuing length, before generating a heatmap. Finally, the generated heatmap is analyzed and judged to determine the traffic congestion situation.
[0007] In one embodiment of the present invention, the basic template with vehicle location information is generated by replacing the extracted foreground with a two-dimensional Gaussian kernel function after blob analysis, and the basic template can reduce the impact of camera perspective distortion on the accuracy of vehicle location information.
[0008] In one embodiment of the present invention, the weighting of the queuing time dimension is performed by applying the formula OneNowVal(i,j) = [OneLastVal(i,j) + BaseVal(i,j) / speed] * C to the base template, where OneNowVal(i,j) is the weighted value of row i and column j of the current frame, OneLastVal is the weighted value of row i and column j of the previous frame, BaseVal(i,j) is the base weight of row i and column j in the base template, speed is the weighting speed, and C is a zeroing factor. If the base weight of row i and column j is 0, then C = 0; otherwise, C = 1. After the calculation is completed, the weighted value of the current frame is output.
[0009] In one embodiment of the present invention, the weighting of the queue length dimension is performed by extracting lane width information to perform a secondary weighting on the primary weighting template; the specific formula is: TwoVal(i,j) = OneNowVal(i,j) / W i Where TwoVal(i,j) is the double-weighted value of the current frame at row i and column j, and OneNowVal(i,j) is the single-weighted value of the current frame at row i and column j. i Let be the relative width of the i-th lane in the region of interest (ROI).
[0010] In one embodiment of the present invention, the method of analyzing and judging traffic congestion using the generated heat map is as follows: the lane congestion situation is judged by analyzing the color of each lane in the heat map.
[0011] This invention also provides a traffic congestion judgment system based on a heatmap model, including a foreground extraction module, a heatmap generation module, and a congestion judgment module. The foreground extraction module first performs grayscale preprocessing on the input video stream, then performs lane segmentation based on optical flow vector clustering to obtain regions of interest (ROIs) that meet the requirements, and then extracts the foreground from the ROIs based on a Gaussian mixture model. The heatmap generation module uses the obtained foreground map to generate a basic template with vehicle location information, and then generates a heatmap by weighting the basic template sequentially from two dimensions: queuing time and queuing length. The congestion judgment module uses the generated heatmap to analyze and judge the traffic congestion situation.
[0012] In one embodiment of the present invention, the foreground extraction module is specifically implemented as follows:
[0013] Step A1: Input the overhead surveillance video stream installed at the traffic light intersection, which covers multiple lanes in both directions, and perform RGB to grayscale preprocessing.
[0014] Step A2: Segmenting lanes based on optical flow vector clustering: Calculate the two-dimensional vector field generated by the translation of a certain region between two frames of the preprocessed video using the optical flow method. Take the average vector value of each pixel over a period of time to represent the average motion trend of that pixel. Cluster each pixel into three categories according to the direction of the average vector value: background pixels, outgoing lane pixels, and incoming lane pixels. Segment the outgoing lane region, i.e., the region of interest (ROI).
[0015] Step A3: Use a Gaussian mixture model within the segmented region of interest (ROI) to extract the foreground and separate the background in real time, and output the extracted foreground image.
[0016] In one embodiment of the present invention, step A3 is specifically implemented as follows: First, traverse the pixels frame by frame over a period of time, calculate the mean and variance of different coordinate points, and establish an initial Gaussian model; then, start traversing the preprocessed video stream frame by frame; match each pixel with the corresponding Gaussian model; if no match is found, create a new model, otherwise update the Gaussian model parameters, and then reorder the corresponding Gaussian models according to the size of the weight parameters; if the Gaussian model matched by the current pixel is the model with the largest weight among the Gaussian models corresponding to the pixel position, it is determined to be the background, otherwise it is the foreground.
[0017] In one embodiment of the present invention, the heat map generation module is specifically implemented as follows:
[0018] Step B1: First, use Blob analysis to identify each connected component (i.e., the vehicle) in the foreground image. After obtaining the center point and corresponding border size information of each connected component, replace the connected component with a two-dimensional Gaussian kernel function to obtain a basic template with vehicle position information. The expression of the two-dimensional Gaussian kernel function is:
[0019]
[0020] In the formula, ρ is the correlation coefficient between i and j, μ1 and μ2 are the means of i and j respectively, and σ1 and σ2 are the standard deviations of i and j respectively;
[0021] To reduce the impact of perspective distortion on the vehicle size caused by the camera, the width w of the corresponding frame of the vehicle is used. i and height h i To set the standard deviations σ1 and σ2, the specific formulas are as follows:
[0022]
[0023] Step B2: Weight the basic template based on queuing time, i.e., the longer the queuing time, the higher the corresponding weight; calculate each weight in the basic template using the following formula:
[0024] OneNowVal(i,j)=[OneLastVal(i,j)+BaseVal(i,j) / speed]*C
[0025] Where OneNowVal(i,j) is the weighted value of row i and column j in the current frame, OneLastVal is the weighted value of row i and column j in the previous frame, BaseVal(i,j) is the base weight of row i and column j in the base template, speed is the weighted speed, and C is the zeroing factor. If the base weight of row i and column j is 0, then C = 0, otherwise C = 1. After the calculation is completed, the weighted value of the current frame is output.
[0026] Step B3: Perform secondary weighting on the primary weighted template based on queue length, meaning the longer the queue length, the higher the corresponding weight. The specific weighting steps are as follows:
[0027] Set the pixel values of the oncoming lane region segmented in step A2 to 1, and set the pixel values of other regions in the image frame to 0, thereby obtaining the binarized label map of the region of interest, i.e., the oncoming lane.
[0028] Calculate the relative width W of the i-th lane in the region of interest. i ;
[0029] The reciprocal of the relative lane width is multiplied by each weight value in the corresponding row of the first-order weighted template obtained in step B2, and then summed to obtain the second-order weighted value TwoVal(i,j), thus obtaining the second-order weighted template that reflects the perspective projection characteristics; the specific formula is:
[0030] TwoVal(i,j)=OneNowVal(i,j) / W i
[0031] Step B4: Calculate the average weight of each lane in the secondary weighted template, and then map the average weight of each lane into a pseudo-color image, i.e., a heat map; the smaller the weight, the bluer the mapped heat map, and vice versa.
[0032] In one embodiment of the present invention, the congestion judgment module is specifically implemented by judging congestion by the color of the pseudo-color image generated for each lane; the redder the color, the more congested the road, and the bluer the color, the smoother the road.
[0033] Compared with the prior art, the present invention has the following beneficial effects:
[0034] Existing technologies mostly detect traffic flow across the entire scene in the image, while the lane selection and segmentation technology of this invention can accurately segment the lane area of interest and determine the congestion level of each lane separately, thereby greatly reducing the computational load of subsequent steps and improving the efficiency of subsequent congestion relief.
[0035] Existing technologies cannot simultaneously achieve both high foreground extraction performance and low computational cost. The foreground extraction technology based on the region of interest in this invention is implemented by using a Gaussian mixture model after clustering optical flow vectors to identify lanes of interest. This results in stable foreground extraction performance with low computational cost.
[0036] The invention generates a weighted heatmap to determine the degree of road congestion from two dimensions: queuing time and queuing length. It proposes a first-weighted formula that reflects queuing time by accumulating the frequency of vehicle stops and a second-weighted formula that takes into account perspective distortion. Compared with existing technologies that mostly analyze and judge from the single dimension of traffic flow, the judgment results are more accurate. Attached Figure Description
[0037] Figure 1 This is the overall flowchart of the present invention.
[0038] Figure 2 This is a flowchart of the Gaussian mixture model algorithm in the foreground extraction module of the present invention.
[0039] Figure 3 This is a flowchart illustrating the specific process of the heatmap generation module of the present invention. Detailed Implementation
[0040] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.
[0041] This invention provides a traffic congestion assessment method based on a heatmap model. First, the input video stream is preprocessed by grayscale conversion. Then, lane segmentation is performed based on optical flow vector clustering to obtain regions of interest (ROIs) that meet the requirements. Within the ROIs, a foreground is extracted using a Gaussian mixture model. A base template with vehicle location information is generated using the obtained foreground image. This base template is then weighted sequentially based on two dimensions: queue duration and queue length, before generating a heatmap. Finally, the generated heatmap is analyzed to determine the traffic congestion situation.
[0042] This invention also provides a traffic congestion judgment system based on a heatmap model, including a foreground extraction module, a heatmap generation module, and a congestion judgment module. The foreground extraction module first performs grayscale preprocessing on the input video stream, then performs lane segmentation based on optical flow vector clustering to obtain regions of interest (ROIs) that meet the requirements, and then extracts the foreground from the ROIs based on a Gaussian mixture model. The heatmap generation module uses the obtained foreground map to generate a basic template with vehicle location information, and then generates a heatmap by weighting the basic template sequentially from two dimensions: queuing time and queuing length. The congestion judgment module uses the generated heatmap to analyze and judge the traffic congestion situation.
[0043] The following is a detailed implementation process of the present invention.
[0044] like Figure 1 As shown, this invention discloses a traffic congestion judgment method and system based on a heatmap model. The system includes a foreground extraction module (A), a heatmap generation module (B), and a congestion judgment module (C).
[0045] A foreground extraction module includes the following steps:
[0046] Step A1: Input the overhead surveillance video stream installed at the traffic light intersection, which covers multiple lanes in both directions, and perform RGB to grayscale preprocessing.
[0047] Step A2: Lane segmentation based on optical flow vector clustering. The pre-processed video is used to calculate the two-dimensional vector field generated by the translation of certain regions between two frames using optical flow. The average vector value of each pixel over a period of time is taken to represent the average motion trend of that pixel. According to the direction of the average vector value, the pixels can be clustered into three categories: background pixels, outgoing lane pixels, and incoming lane pixels. The outgoing lane region, i.e., the region of interest (ROI), is then segmented.
[0048] Step A3: Apply a Gaussian mixture model to the segmented ROI region to extract the foreground and separate the background in real time, and output the extracted foreground image. The specific process is as follows: Figure 2 As shown:
[0049] First, traverse the pixels frame by frame over a period of time, calculate the mean and variance of different coordinate points, and establish an initial Gaussian model;
[0050] Then, the preprocessed video stream is traversed frame by frame.
[0051] Each pixel is matched one by one with the Gaussian model at the corresponding pixel position;
[0052] If none of them match, a new model is created; otherwise, the model parameters are updated, and the corresponding models are reordered according to the size of the weight parameters.
[0053] If the model matched to the current pixel is the Gaussian model with the largest weight among the several Gaussian models corresponding to the pixel position, it is judged as the background; otherwise, it is the foreground.
[0054] Output the judgment result;
[0055] The specific implementation process of the heatmap generation module is as follows: Figure 3 As shown, it includes the following steps:
[0056] Step B1: First, use Blob analysis to identify each connected component (i.e., vehicle) in the foreground image, and obtain the center point and corresponding border size information of each connected component. Then, replace the connected component with a two-dimensional Gaussian kernel function to obtain a basic template with vehicle position information. The expression for the two-dimensional Gaussian kernel function in the algorithm is:
[0057]
[0058] In the formula, ρ is the correlation coefficient between i and j, μ1 and μ2 are the means of i and k, respectively, and σ1 and σ2 are the standard deviations of i and j, respectively.
[0059] To reduce the impact of perspective distortion on the vehicle size caused by the camera, it is necessary to consider the width w of the vehicle's corresponding frame. i and height h i To set the standard deviations σ1 and σ2, the specific formulas are as follows:
[0060]
[0061] Step B2: Weight the basic template based on queuing time; the longer the queuing time, the higher the corresponding weight. The calculation process for each weight in the basic template is performed using the following formula:
[0062] OneNowVal(i,j)=[OneLastVal(i,j)+BaseVal(i,j) / speed]*C
[0063] Where OneNowVal(i, j) is the weighted value of row i, column j in the current frame, OneLastVal is the weighted value of row i, column j in the previous frame, BaseVal(i, j) is the base weight of row i, column j in the base template, and speed is the weighted speed. C is a reset factor; if the base weight of row i, column j is 0, then C = 0; otherwise, C = 1. After calculation, the weighted value of the current frame is output.
[0064] Step B3: Perform a second weighting on the first-weighted template based on queue length; that is, the longer the queue, the higher the corresponding weight. Due to the influence of camera perspective distortion, a narrower lane indicates a greater distance from the camera, so the lane width information can be used to indirectly reflect the queue length. The specific weighting steps are as follows:
[0065] The pixel values of the oncoming lane region segmented in step A2 are all set to 1, and the pixel values of other regions in the image frame are all set to 0, thereby obtaining the binarized label map of the region of interest (i.e., the oncoming lane).
[0066] Calculate the relative width W of the i-th lane in the region of interest. i .
[0067] The reciprocal of the relative lane width is multiplied by each weight in the corresponding row of the first-weighted template obtained in step B2, and then summed to obtain the second-weighted value TwoVal(i,j). This yields the second-weighted template that reflects the perspective projection characteristics. The specific formula is: TwoVal(i,j) = OneNowVal(i,j) / W i
[0068] Step B4: Calculate the average weight of each lane in the secondary weighted template, and then map the average weight of each lane into a pseudo-color image, i.e., a heatmap. The smaller the weight, the bluer the mapped heatmap, and vice versa.
[0069] C. Congestion Assessment Module: Congestion is assessed based on the color of the pseudo-color image generated for each lane. The redder the image, the more congested the road; the bluer the image, the smoother the traffic.
[0070] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.
Claims
1. A traffic congestion judgment method based on a heatmap model, characterized in that, First, the input video stream is preprocessed by grayscale conversion. Then, lane segmentation is performed based on optical flow vector clustering to obtain regions of interest (ROIs) that meet the requirements. Foreground is extracted from the ROIs using a Gaussian mixture model. A base template with vehicle location information is generated using the obtained foreground image. The base template is then weighted sequentially based on two dimensions: queue duration and queue length, before generating a heatmap. Finally, the generated heatmap is analyzed to determine the traffic congestion situation. The base template with vehicle location information is generated by replacing the extracted foreground with a two-dimensional Gaussian kernel function after blob analysis. This base template reduces the impact of camera perspective distortion on the accuracy of vehicle location information. The weighting of the queue duration dimension uses a formula... The basic template is weighted once, where For the current frame OK The first weighted value of the column, Previous frame OK The first weighted value of the column, In the basic template OK The basic weight of the column, For weighted velocity, It is a zeroing factor, if the current OK When the basic weight of the column is 0, then =0, otherwise =1, and after calculation, the first weighted value of the current frame is output; the weighting of the queue length dimension is performed by extracting the lane width information to perform a second weighting on the first weighting template; the specific formula is: ,in For the current frame OK The second-order weighted value of the column, For the current frame OK The first weighted value of the column, For the region of interest (ROI) of the th The relative width of the driving lane.
2. The traffic congestion judgment method based on a heatmap model according to claim 1, characterized in that, The method for analyzing and judging traffic congestion using the generated heatmap is as follows: the congestion status of each lane is determined by analyzing the color of each lane in the heatmap.
3. A traffic congestion judgment system based on a heatmap model, characterized in that, The system includes a foreground extraction module, a heatmap generation module, and a congestion assessment module. The foreground extraction module first performs grayscale preprocessing on the input video stream, then performs lane segmentation based on optical flow vector clustering to obtain regions of interest (ROIs) that meet the requirements. The foreground is then extracted from the ROIs using a Gaussian mixture model. The heatmap generation module uses the obtained foreground image to generate a base template with vehicle location information. The base template is then weighted sequentially based on queue duration and queue length before generating a heatmap. The congestion assessment module analyzes and assesses the generated heatmap to determine the traffic congestion situation. The specific implementation of the heatmap generation module is as follows: Step B1: First, use Blob analysis to identify each connected component (i.e., the vehicle) in the foreground image. After obtaining the center point and corresponding border size information of each connected component, replace the connected component with a two-dimensional Gaussian kernel function to obtain a basic template with vehicle position information. The expression of the two-dimensional Gaussian kernel function is: In the formula, 𝜌 is and The correlation coefficients between them, 𝜇1 and 𝜇2 are respectively and The mean of , s1 and s2 are respectively and Standard deviation; To reduce the impact of perspective distortion on the vehicle size caused by the camera, the width of the corresponding frame of the vehicle is used as a reference. and height To set the standard deviations σ1 and σ2, the specific formulas are as follows: Step B2: Weight the basic template based on queuing time, i.e., the longer the queuing time, the higher the corresponding weight; calculate each weight in the basic template using the following formula: in For the current frame OK The first weighted value of the column, Previous frame OK The first weighted value of the column, In the basic template OK The basic weight of the column, For weighted velocity, It is a zeroing factor, if the current OK When the basic weight of the column is 0, then =0, otherwise =1, and after the calculation is completed, the weighted value of the current frame is output; Step B3: Perform secondary weighting on the primary weighted template based on queue length, meaning the longer the queue length, the higher the corresponding weight. The specific weighting steps are as follows: Set the pixel values of the oncoming lane region segmented in step A2 to 1, and set the pixel values of other regions in the image frame to 0, thereby obtaining the binarized label map of the region of interest, i.e., the oncoming lane. Calculate the first in the region of interest relative width of driving lane ; The reciprocal of the relative lane width is multiplied by each weight value in the corresponding row of the first-weighted template obtained in step B2, and then summed to obtain the second-weighted value. This leads to a secondary weighted template that reflects the characteristics of perspective projection; The specific formula is as follows: Step B4: Calculate the average weight of each lane in the secondary weighted template, and then map the average weight of each lane into a pseudo-color image, i.e., a heat map; the smaller the weight, the bluer the mapped heat map, and vice versa.
4. A traffic congestion judgment system based on a heatmap model according to claim 3, characterized in that, The foreground extraction module is implemented as follows: Step A1: Input the overhead surveillance video stream installed at the traffic light intersection, which covers multiple lanes in both directions, and perform RGB to grayscale preprocessing. Step A2: Segmenting lanes based on optical flow vector clustering: Calculate the two-dimensional vector field generated by the translation of a certain region between two frames of the preprocessed video using the optical flow method. Take the average vector value of each pixel over a period of time to represent the average motion trend of that pixel. Cluster each pixel into three categories according to the direction of the average vector value: background pixels, outgoing lane pixels, and incoming lane pixels. Segment the outgoing lane region, i.e., the region of interest (ROI). Step A3: Use a Gaussian mixture model within the segmented region of interest (ROI) to extract the foreground and separate the background in real time, and output the extracted foreground image.
5. A traffic congestion judgment system based on a heatmap model according to claim 4, characterized in that, The specific implementation of step A3 is as follows: First, traverse the pixels frame by frame over a period of time, calculate the mean and variance of different coordinate points, and establish an initial Gaussian model; then, start traversing the preprocessed video stream frame by frame; match each pixel with the corresponding Gaussian model; if no match is found, create a new model, otherwise update the Gaussian model parameters, and then reorder the corresponding Gaussian models according to the size of the weight parameters; if the Gaussian model matched by the current pixel is the model with the largest weight among the Gaussian models corresponding to the pixel position, it is judged as the background, otherwise it is the foreground.
6. A traffic congestion judgment system based on a heatmap model according to claim 3, characterized in that, The congestion judgment module is implemented by judging congestion based on the color of the pseudo-color image generated for each lane. The redder the color, the more congested the road, and the bluer the color, the smoother the road.