Traffic Congestion Assessment Method, Device, Electronic Device and Storage Medium

Abstract

The present application relates to the field of image processing technology, and particularly relates to a traffic congestion evaluation method, device, electronic device and storage medium. Among them, the method includes: obtaining an image frame sequence corresponding to a current road, and determining each lane corresponding to the image frame sequence; starting from the stop line, counting the queuing vehicles in each lane, and calculating the passing time for the trailing vehicle in the queuing vehicles to pass the stop line; calculating a queuing time index corresponding to each lane according to each passing time and the signal light cycle duration of the current road; evaluating the congestion condition of the current road according to each queuing time index. Through the above method, the determination of the traffic congestion condition can be realized without obtaining the speeds of each vehicle and without manual participation, which is convenient and effective, saves time cost and labor cost, and improves the evaluation accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and particularly relates to a traffic congestion evaluation method, device, electronic device and storage medium. Background Art

[0002] At the present stage, with the expansion of urban population and the increase in the number of vehicle ownership, the traffic congestion problem in many big cities is becoming increasingly serious. Many places are actively seeking countermeasures to control congestion, and the real-time traffic congestion evaluation of urban roads is an important prerequisite and component of the urban intelligent transportation system, and is the basis of dynamic navigation and traffic guidance. Traffic managers can judge the congestion condition of urban roads according to the information obtained in real time, conduct real-time traffic guidance, disperse traffic flow, and thus reduce road load and the probability of traffic congestion.

[0003] Existing traffic congestion evaluation methods usually establish a traffic congestion prediction model through speed and use the method of fuzzy inference to obtain the final congestion index to evaluate traffic congestion. Due to this evaluation method, vehicle speed can only be accurately obtained with vehicle-end data, resulting in the difficulty of obtaining vehicle speed, and the fuzzy inference rules are also artificially specified, resulting in inaccurate final results. Summary of the Invention

[0004] Embodiments of the present application provide a traffic congestion evaluation method, device, electronic device and storage medium, which are used to solve the problem that the current traffic congestion evaluation method depends on the speed of each vehicle and requires manual participation, resulting in inaccurate evaluation results.

[0005] In a first aspect, the present application provides a traffic congestion evaluation method, and the method includes:

[0006] Obtain an image frame sequence corresponding to the current road, and determine each lane corresponding to the image frame sequence;

[0007] Start counting the queuing vehicles in each lane from the stop line, and calculate the passing time for the trailing vehicle of the queuing vehicles in each lane to pass the stop line;

[0008] According to each passing time and the signal light cycle time of the corresponding lane, calculate the queuing time index corresponding to each lane;

[0009] Evaluate the congestion condition of the current road according to each queuing time index.

[0010] Through the above method, without obtaining the speed of each vehicle and without manual participation, the determination of traffic congestion condition is realized, which is convenient and effective, saves time cost and labor cost, and improves the evaluation accuracy.

[0011] In a possible embodiment, the determination of each lane corresponding to the image frame sequence includes:

[0012] Segment lane lines in each frame image of the image frame sequence, where the lane lines include lane boundary lines and stop lines;

[0013] Fuse the respective lane lines to obtain the complete lane lines corresponding to the image frame sequence;

[0014] In the perspective of a bird's-eye view, divide each lane through the complete lane lines.

[0015] Through the above method, it is possible to accurately divide each lane in the current road scene based on the image frame sequence collected by the camera.

[0016] In a possible embodiment, the fusing of the respective lane lines to obtain the complete lane lines corresponding to the image frame sequence includes:

[0017] By calculating the intersection over union of the lane lines in the current frame image and the previous frame image in the image frame sequence, determine the same lane line in the previous frame image corresponding to any lane line in the current frame image;

[0018] Fuse the same lane lines in the previous frame image and the current frame image, and cache the fusion result to obtain a cached result;

[0019] Fuse the same lane lines in the next frame image in the image frame sequence and the cached result, and cache the fusion result to obtain an updated cached result until the complete lane lines are obtained.

[0020] Through the above method, a complete lane line and zebra crossing instance are obtained by integrating the segmentation results of multiple frames, avoiding the incomplete situation of lane lines and zebra crossings caused by vehicle occlusion, so as to more accurately divide lanes and determine whether a vehicle passes through the stop line.

[0021] In a possible embodiment, the dividing of each lane through the complete lane lines in the perspective of a bird's-eye view includes:

[0022] Based on the prior information of the zebra crossing in the real world, perform coordinate transformation on the complete lane lines and the image information in the image frame sequence to obtain bird's-eye view information, where the prior information is that the white solid lines forming the zebra crossing are parallel to each other;

[0023] Divide each lane in the bird's-eye view information through the complete lane line information in the bird's-eye view information.

[0024] Through the above method, using the prior information of the size and shape of the zebra crossing in the real world, the entire scene is transformed into a bird's-eye view for calculation, and the lanes can be accurately divided and the time taken for the vehicle to move from a stationary state to pass the stop line can be counted without the need for camera parameters.

[0025] In a possible embodiment, starting from the stop line, the queuing vehicles in each lane are counted, and the passing time for the trailing vehicle in the queuing vehicles to pass the stop line is calculated, including:

[0026] Starting from the stop line, count and determine whether the distance between the front and rear vehicles in each lane is less than a preset threshold;

[0027] If so, determine that the front and rear vehicles belong to the queuing vehicles;

[0028] Among all the queuing vehicles in each lane, determine the trailing vehicle in a stationary state and record the first current timestamp;

[0029] Track each trailing vehicle, and use the time difference between the second current timestamp when each trailing vehicle passes the stop line and the first timestamp as the passing time for each trailing vehicle to pass the stop line.

[0030] Through the above method, without manual participation and without obtaining the vehicle speeds of each vehicle, the time consumed for the trailing vehicle in the queuing vehicles to pass the stop line can be calculated when the road is congested.

[0031] In a possible embodiment, calculating the queuing time index corresponding to each lane according to each passing time and the signal light cycle duration of the current road includes:

[0032] Detect the signal lights in each frame of the image frame sequence and the states of the signal lights, where the states of the signal lights include red lights and green lights;

[0033] When the states of the signal lights in any two adjacent frames in the image frame sequence are different, record the timestamps of the two adjacent frames;

[0034] According to the recorded timestamps and the signal light states corresponding to the timestamps, determine the signal light cycle durations of each lane;

[0035] Obtain the ratios between each passing time and the signal light cycle durations of the corresponding lanes, and use each ratio as the queuing time index corresponding to each lane.

[0036] Through the above method, without manual participation and without obtaining the vehicle speed, by combining the passing time of the trailing vehicles in each lane queuing through the stop line with the signal light cycle duration, the queuing time index of each lane is calculated.

[0037] In a possible embodiment, the evaluating the congestion condition of the current road according to each queuing time index includes:

[0038] Sorting the queuing time indices to obtain the maximum queuing time index;

[0039] Determining the congestion level corresponding to the current road according to the range where the maximum time index is located, where the congestion level includes at least one of unobstructed, slightly congested, moderately congested, and severely congested.

[0040] Through the above method, without manual participation and without obtaining the vehicle speed, the congestion degree of the current road is evaluated according to the queuing time indices of each lane.

[0041] In a second aspect, the present application provides a traffic congestion evaluation device, and the device includes:

[0042] A determination module, configured to obtain an image frame sequence corresponding to a current road and determine each lane corresponding to the image frame sequence;

[0043] A calculation module, configured to count the queuing vehicles in each lane starting from the stop line and calculate the passing time of the trailing vehicles of the queuing vehicles in each lane passing through the stop line; according to each passing time and the signal light cycle duration of the corresponding lane, calculate the queuing time index corresponding to each lane;

[0044] An evaluation module, configured to evaluate the congestion condition of the current road according to each queuing time index.

[0045] In a possible embodiment, the determination module is specifically configured to:

[0046] Segment the lane lines in each frame image of the image frame sequence, where the lane lines include lane side lines and stop lines;

[0047] Fuse each lane line to obtain the complete lane line corresponding to the image frame sequence;

[0048] Divide each lane through the complete lane line from the perspective of a bird's-eye view.

[0049] In a possible embodiment, the determination module is further configured to:

[0050] By calculating the intersection-over-union of the current frame image and the previous frame image in the image frame sequence, the same lane line corresponding to any lane line in the current frame image is determined in the previous frame image;

[0051] Fuse the same lane lines in the previous frame image and the current frame image, and cache the fusion result to obtain a cached result;

[0052] Fuse the next frame image in the image frame sequence with the same lane line in the cached result, and cache the fusion result to obtain an updated cached result until the complete lane line is obtained.

[0053] In a possible embodiment, the determining module is further configured to:

[0054] Based on the prior information of the zebra crossing in the real world, perform coordinate transformation on the complete lane line and the image information in the image frame sequence to obtain bird's-eye view information, where the prior information is that the white solid lines forming the zebra crossing are parallel to each other;

[0055] Divide each lane in the bird's-eye view information through the complete lane line information in the bird's-eye view information.

[0056] In a possible embodiment, the calculating module is specifically configured to:

[0057] Start counting from the stop line, and determine whether the distance between the front and rear vehicles in each lane is less than a preset threshold;

[0058] If so, determine that the front and rear vehicles belong to queuing vehicles;

[0059] Determine the end vehicle in a stationary state among all the queuing vehicles in each lane, and record the first current timestamp;

[0060] Track each end vehicle, and use the time difference between the second current timestamp when each end vehicle passes the stop line and the first timestamp as the passing duration of each end vehicle passing the stop line.

[0061] In a possible embodiment, the calculating module is further configured to:

[0062] Detect the signal lights in each frame image of the image frame sequence and the states of the signal lights, where the states of the signal lights include red lights and green lights;

[0063] When the states of the signal lights in any two adjacent frame images in the image frame sequence are different, record the timestamps of the any two adjacent images;

[0064] Determine the signal light cycle duration of each lane according to each recorded timestamp and the signal light state corresponding to each timestamp;

[0065] Obtain the ratio between each passing duration and the signal light cycle duration of the corresponding lane, and use each ratio as the queuing time index corresponding to each lane.

[0066] In a possible embodiment, the evaluation module is specifically configured to:

[0067] Sort each queuing time index to obtain the maximum queuing time index;

[0068] Determine the congestion level corresponding to the current road according to the range where the maximum time index is located, where the congestion level includes at least one of unobstructed, mildly congested, moderately congested, and severely congested.

[0069] In a third aspect, the present application provides an electronic device, including:

[0070] A memory for storing program instructions;

[0071] A processor for calling the program instructions stored in the memory and executing the steps included in the traffic congestion evaluation method according to any one of the first aspects.

[0072] In a fourth aspect, the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the traffic congestion evaluation method according to any one of the first aspects.

[0073] For the various aspects in the second to fourth aspects above and the possible technical effects that each aspect may achieve, refer to the technical effects that can be achieved according to the first aspect or various possible solutions in the first aspect above, and details will not be repeated here. Description of the Drawings

[0074] Figure 1 It is a schematic diagram of a possible application scenario provided by an embodiment of the present application;

[0075] Figure 2 It is a flowchart of a traffic congestion evaluation method provided by an embodiment of the present application;

[0076] Figure 3 It is an example diagram of queuing vehicles provided by an embodiment of the present application;

[0077] Figure 4 It is a structural diagram of a traffic congestion evaluation device provided by an embodiment of the present application;

[0078] Figure 5 Structural diagram of an electronic device provided by an embodiment of the present application. Specific implementation manner

[0079] To make the purpose, technical solutions and advantages of the present application clearer and more understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application. Without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0080] The terms "first" and "second" in the specification, claims and above-mentioned drawings of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any variations thereof are intended to cover non-exclusive protection. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices. "Multiple" in the present application can represent at least two, for example, it can be two, three or more, and the embodiments of the present application do not make limitations.

[0081] For the convenience of those skilled in the art to understand, some nouns and terms related to the embodiments of the present application are briefly elaborated and explained as follows:

[0082] Instance segmentation: Instance segmentation refers to distinguishing pixels of different classes in an image into different instances, and pixels of the same class have different instance labels.

[0083] Mask: A mask that can distinguish the target from the background, with the target value being 1 and the background value being 0.

[0084] Perspective transformation: Transformation from the 3D world to the 2D image.

[0085] Signal control period: The sum of the red light duration and the green light duration of the traffic signal.

[0086] Maximum queuing time index: The maximum value of the ratio of the duration from when a vehicle first stops to when it passes the stop line in a lane at an intersection to the signal light cycle duration. Refer to the Public Security Industry Standard of the People's Republic of China for the Evaluation Method of Road Traffic Congestion Degree.

[0087] Furthermore, based on the above-mentioned terms and related term explanations, the design concept of this application will be further elaborated and described as follows:

[0088] Existing traffic congestion assessment methods usually establish a traffic congestion prediction model through speed and use fuzzy inference to obtain the final congestion index to achieve the assessment of traffic congestion. Since this assessment method requires vehicle-end data to accurately obtain the vehicle speed, it is usually difficult to obtain the vehicle speed. Moreover, the fuzzy inference rules are also manually specified without a unified standard, resulting in inaccurate final results.

[0089] To solve the above problems, the embodiments of this application provide a traffic congestion assessment method, device, electronic device, and storage medium. By detecting the signal light status, the signal light cycle duration is obtained. Through the scene segmentation method, the vehicle and lane line mask information is obtained, and a lane line fusion mechanism is added. The multi-frame segmentation results are integrated to obtain a complete lane line instance, avoiding the problem that the lane line is incomplete due to vehicle occlusion and unable to accurately divide the lane line. To more accurately determine whether a vehicle passes the stop line, coordinate conversion is performed based on the prior information of the true shape and size of the zebra crossing, and the coordinates of the solid white line within the zebra crossing are set to be parallel to each other, thereby converting the entire scene to a bird's-eye view. Further, in this bird's-eye view, the duration from when the vehicle is stationary to when it passes the stop line is counted, and the queuing time index is calculated in combination with the signal light cycle duration to evaluate the road congestion degree. In this way, without obtaining the speed of each vehicle and without manual participation, the determination of the traffic congestion situation is realized, which is convenient and effective, saves time cost and labor cost, and improves the assessment accuracy.

[0090] Based on the above technical effects, the preferred embodiments of this application will be described below with reference to the accompanying drawings of the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain this application and are not used to limit this application. And without conflict, the embodiments of this application and the features in the embodiments can be combined with each other.

[0091] As Figure 1 shown, it is a schematic diagram of a possible application scenario provided by the embodiments of this application. This application scenario includes: target terminals (101a, 101b) and a server 102. Among them, information interaction can be carried out between the target terminals (101a, 101b) and the server 102 through a communication network. The communication methods adopted by the communication network can include: wireless communication methods and wired communication methods.

[0092] Exemplarily, the target terminals (101a, 101b) can access the network through cellular mobile communication technology and communicate with the server 102. The cellular mobile communication technology includes the 5th Generation Mobile Networks (5G) technology.

[0093] Exemplarily, the target terminals (101a, 101b) can access the network through short-range wireless communication and communicate with the server 102. The short-range wireless communication includes Wireless Fidelity (Wi-Fi) technology.

[0094] The embodiments of the present application do not impose any restrictions on the number of the above devices. For example, Figure 1 as shown, only the target terminals (101a, 101b) and the server 102 are taken as examples for description. Below, a brief introduction to the above devices and their respective functions will be given.

[0095] The target terminals (101a, 101b) are devices that can provide voice and / or data connectivity to users, including: handheld terminal devices with wireless connection functions, in-vehicle terminal devices, etc.

[0096] Exemplarily, the target terminals (101a, 101b) include but are not limited to: mobile phones, tablet computers, laptop computers, palm computers, Mobile Internet Devices (MIDs), wearable devices, Virtual Reality (VR) devices, Augmented Reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in unmanned driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0097] In addition, a client related to traffic congestion assessment can be installed on the target terminals (101a, 101b). The client can be software (such as APPs, browsers, short video software, etc.), or web pages, applets, etc. In the embodiments of the present application, the target terminals (101a, 101b) can use the client related to the above APP compatibility detection and can perform information interaction related to traffic congestion assessment with the server 102.

[0098] Further, the server 102 can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.

[0099] Further, in the embodiment of the present application, the above-mentioned server 102 can be equipped with a traffic congestion assessment service platform corresponding to the above-mentioned client, and this traffic congestion assessment service platform is used to execute traffic congestion assessment services.

[0100] Based on the above application scenarios, the traffic congestion assessment method provided by the embodiments of the present application will be described and explained below with reference to the accompanying drawings. As Figure 2 shown, the embodiments of the present application provide a traffic congestion assessment method, which specifically includes the following steps:

[0101] S201, obtain an image frame sequence corresponding to the current road, and determine each lane corresponding to the image frame sequence;

[0102] In the embodiment of the present application, after obtaining the image frame sequence corresponding to the current road, further, lane lines are segmented in each frame image of the image frame sequence, where the lane lines include lane boundary lines and stop lines. Then, the individual lane lines are fused to obtain the complete lane lines corresponding to the image frame sequence. The specific fusion method is as follows:

[0103] First, by calculating the intersection over union of the lane lines in the current frame image and the previous frame image in the image frame sequence, the same lane line corresponding to any lane line in the current frame image is determined in the previous frame image.

[0104] For example, in the image frame sequence, the current frame image is image A, and the previous frame image of image A is image B. Among them, there are two lane lines a1 and a2 in image A, and two lane lines b1 and b2 in image B. Calculate the intersection over union between the two lane lines a1, a2 and b1, b2 respectively. The lane line pairs for calculating the intersection over union pairwise are (a1, b1), (a1, b2), (a2, b1), (a2, b2), and the four intersection over union results obtained are s1, s2, s3, s4. Among these four results, if s1 > s2, then it can be confirmed that the lane lines a1 and b1 are the same lane line, and further it can be obtained that a2 and b2 are also the same lane line.

[0105] Then, fuse the same lane line in the previous frame image and the current frame image, cache the fusion result to obtain a cached result; and fuse the next frame image in the image frame sequence with the same lane line in the cached result, and cache the fusion result to obtain an updated cached result until a complete lane line is obtained.

[0106] For example, the image frame sequence collected by the camera has 100 frame images. When the second frame image is obtained, fuse the second frame image with the same lane line in the first frame image to obtain a cached result. Then, when the third frame image is obtained, fuse the third frame image with the same lane line in the cached result, and cache the obtained fusion result. Then process the 4th, 5th, ··· frame images until all 100 frame images are fused to obtain a complete lane line.

[0107] Finally, in the bird's-eye view, divide each lane through the above complete lane line.

[0108] Specifically, based on the prior information of the zebra crossing in the real world, perform coordinate transformation on the above complete lane line and the image information in the image frame sequence to obtain bird's-eye view information. The prior information is that the white solid lines forming the zebra crossing are parallel to each other. For example, detect the four corner points of two adjacent white solid lines within the zebra crossing, set the coordinates of these four corner points to form a parallelogram or a rectangle, so as to transform the vehicles and lane lines in the entire image to the bird's-eye view. Although the size is not the same as the real-world size, it does not affect the duration statistics. Further, divide each lane in the bird's-eye view information through the complete lane line information in the bird's-eye view information.

[0109] S202, start counting the queuing vehicles in each lane from the stop line, and calculate the passing duration of the trailing vehicle of the queuing vehicles in each lane passing through the stop line;

[0110] After dividing each lane, start counting the queuing vehicles in each lane from the stop line in the bird's-eye view information. Specifically: start counting from the stop line, and judge whether the distance between the front and rear vehicles in each lane is less than a preset threshold; if it is less than the preset threshold, determine that the front and rear vehicles belong to the queuing vehicles. In this way, the queuing vehicles in each lane can be determined.

[0111] Further, determine the trailing vehicle in a stationary state among all the queuing vehicles in each lane, record the first current timestamp, track each trailing vehicle, and use the time difference between the second current timestamp when each trailing vehicle passes through the stop line and the first timestamp as the passing duration of each trailing vehicle passing through the stop line.

[0112] For example, refer to Figure 3 ,Figure 3 There are four cars, C1, C2, C3, and C4, in the lane. The distance between vehicle C1 and vehicle C2 is d1, the distance between vehicle C2 and vehicle C3 is d2, and the distance between vehicle C3 and vehicle C4 is d3. Among them, d1 and d2 are less than the preset threshold, and d3 is greater than the preset threshold. Therefore, vehicles C1, C2, and C3 are queuing vehicles. When the trailing vehicle C3 is stationary, the current timestamp t1 is recorded, and when the trailing vehicle C3 passes the stop line, the current timestamp t2 is recorded. Then, the passing duration for the trailing vehicle C3 to pass the stop line is t = t2 - t1.

[0113] S203. Calculate the queuing time index for each lane according to each passing duration and the signal light cycle duration of the corresponding lane.

[0114] After calculating the passing duration for the trailing vehicle in the currently queuing vehicles of each lane to pass the stop line, further, detect the signal lights in each frame of the image sequence and the status of the signal lights. The status of the signal lights includes red and green. Then, detect whether the status of the signal lights is the same between any two adjacent frames in the image sequence, and when the status of the signal lights is different between any two adjacent frames in the image sequence, record the timestamps of the two adjacent frames. Then, according to the recorded timestamps and the corresponding signal light status, determine the signal light cycle duration of each lane on the current road.

[0115] For example, the current image sequence includes 50 frames of images. Among them, the signal light of the target lane in the 1st - 15th frames of images is green, the signal light in the 16th - 20th frames of images is yellow, the signal light in the 21st - 35th frames of images is red, and the signal light in the 36th - 50th frames of images is green. Then, the status of the signal light and the corresponding timestamp T1 of the 21st frame of image can be recorded, and the status of the signal light and the corresponding timestamp T2 of the 36th frame of image can be recorded. At this time, according to the status of the signal light and the corresponding timestamp T1 of the 21st frame of image, and the status of the signal light and the corresponding timestamp T2 of the 36th frame of image, the signal light cycle duration of the target lane can be determined as T = T2 - T1.

[0116] After determining the signal light cycle duration of each lane on the current road, further, obtain the ratio between each passing duration and the signal light cycle duration of the corresponding lane, and use each ratio as the queuing time index for each lane. For example, if the passing duration for the trailing vehicle in the target lane of each lane to pass the stop line is t, then the queuing time index for the target lane can be calculated as Q = t / T.

[0117] S204. Evaluate the congestion status of each lane according to each queuing time index.

[0118] After calculating the queuing time index corresponding to each lane, further, the congestion condition of each lane is evaluated according to each queuing index. Specifically:

[0119] First, sort the queuing time indices of each lane to obtain the maximum queuing time index. For example, if the queuing time indices corresponding to the current lanes are Q1, Q2, and Q3 respectively, and Q1 > Q2 > Q3, then the maximum queuing time index can be determined as Q1.

[0120] Then, according to the range where the maximum time index is located, determine the congestion level corresponding to the current road. In the embodiments of the present application, the congestion level is determined according to the road congestion assessment standard, which specifically includes unobstructed, mild congestion, moderate congestion, and severe congestion.

[0121] Specifically, when the maximum queuing time index is between 0 - 55, it is determined that the current road is in an unobstructed state; when the maximum queuing time index is between 55 - 100, it is determined that the current road is in a mild congestion state; when the maximum queuing time index is between 100 - 145, it is determined that the current road is in a moderate congestion state; when the maximum queuing time index is greater than 145, it is determined that the current road is in a severe congestion state.

[0122] Based on the above traffic congestion assessment method, without the need to obtain vehicle - end data and without manual participation, the maximum queuing time index indicator can be calculated intuitively and effectively at a low cost, and then the congestion situation of the current road can be evaluated.

[0123] Moreover, using the prior information of the size and shape of the zebra crossing in the real world, set the corner coordinates of multiple white solid lines within the converted zebra crossing to be parallel to each other, so as to convert the entire scene to a bird's - eye view for calculation. This conversion method can accurately divide lanes and count the time duration from when a vehicle is stationary to when it passes the stop line without the need for camera parameters.

[0124] In addition, when dividing lane lines, the lane line segmentation result of the current frame is fused with the lane line segmentation result of the previous frame. By matching the intersection - over - union ratio of the masks of the two instances to the target, take the union of the two matching lane line masks as the fused result, and then each subsequent frame is merged with this fused result. By synthesizing the segmentation results of multiple frames, a complete lane line and zebra crossing instance are obtained, avoiding the incomplete situation of lane lines and zebra crossings caused by vehicle occlusion, and being able to more accurately divide lanes and determine whether a vehicle has passed the stop line.

[0125] Based on the same inventive concept, an embodiment of the present application provides a traffic congestion assessment device. Please refer to Figure 4 This device includes:

[0126] Determination module 401, configured to obtain an image frame sequence corresponding to the current road and determine each lane corresponding to the image frame sequence;

[0127] Calculation module 402, configured to start counting the queuing vehicles in each lane from the stop line and calculate the passing time for the end vehicle of the queuing vehicles in each lane to pass the stop line; calculate the queuing time index corresponding to each lane according to each passing time and the signal light cycle time of the corresponding lane;

[0128] Evaluation module 403, configured to evaluate the congestion condition of the current road according to each queuing time index.

[0129] In a possible embodiment, the determination module 401 is specifically configured to:

[0130] Segment lane lines in each frame image of the image frame sequence, where the lane lines include lane boundary lines and stop lines;

[0131] Fuse each lane line to obtain a complete lane line corresponding to the image frame sequence;

[0132] Divide each lane through the complete lane line from the perspective of a bird's-eye view.

[0133] In a possible embodiment, the determination module 401 is further configured to:

[0134] Determine the same lane line corresponding to any lane line in the current frame image in the previous frame image by calculating the intersection-over-union ratio of the lane lines in the current frame image and the previous frame image in the image frame sequence;

[0135] Fuse the same lane lines in the previous frame image and the current frame image, and cache the fusion result to obtain a cached result;

[0136] Fuse the same lane lines in the next frame image in the image frame sequence and the cached result, and cache the fusion result to obtain an updated cached result until the complete lane line is obtained.

[0137] In a possible embodiment, the determination module 401 is further configured to:

[0138] Based on the prior information of the zebra crossing in the real world, perform coordinate transformation on the complete lane line and the image information in the image frame sequence to obtain bird's-eye view information, where the prior information is that the white solid lines forming the zebra crossing are parallel to each other;

[0139] Divide each lane in the bird's-eye view information through the complete lane line information in the bird's-eye view information.

[0140] In a possible embodiment, the computing module 402 is specifically configured to:

[0141] Start counting from the stop line, and determine whether the distance between the front and rear vehicles in each lane is less than a preset threshold;

[0142] If so, determine that the front and rear vehicles belong to queuing vehicles;

[0143] Determine the end vehicle in a stationary state among all the queuing vehicles in each lane, and record the first current timestamp;

[0144] Track each end vehicle, and use the time difference between the second current timestamp when each end vehicle passes the stop line and the first timestamp as the passing duration of each end vehicle passing the stop line.

[0145] In a possible embodiment, the computing module 402 is further configured to:

[0146] Detect the signal lights in each frame of the image frame sequence and the states of the signal lights, where the states of the signal lights include red lights and green lights;

[0147] When the states of the signal lights in any two adjacent frames in the image frame sequence are different, record the timestamps of the two adjacent images;

[0148] Determine the signal light cycle duration of each lane according to the recorded timestamps and the signal light states corresponding to the timestamps;

[0149] Obtain the ratio between each passing duration and the signal light cycle duration of the corresponding lane, and use each ratio as the queuing time index corresponding to each lane.

[0150] In a possible embodiment, the evaluation module 403 is specifically configured to:

[0151] Sort the queuing time indices to obtain the maximum queuing time index;

[0152] Determine the congestion level corresponding to the current road according to the range where the maximum time index is located, where the congestion level includes at least one of unobstructed, slightly congested, moderately congested, and severely congested.

[0153] Based on the above traffic congestion evaluation device, without the need to obtain vehicle-end data and without manual participation, it is possible to calculate the maximum queuing time index indicator at a low cost, intuitively and effectively, and then evaluate the congestion situation of the current road.

[0154] Moreover, by using the prior information on the size and shape of the zebra crossing in the real world, the corner coordinates of multiple solid white lines within the converted zebra crossing are set to be parallel to each other, so as to convert the entire scene to a bird's-eye view for calculation. This conversion method can accurately divide lanes and count the time duration of a vehicle from stationary to passing the stop line without camera parameters.

[0155] In addition, when dividing the lane lines, the lane line segmentation result of the current frame is fused with the lane line segmentation result of the previous frame. By matching the intersection over union (IoU) of the masks of two instances as the target, the union of the two matching lane line masks is taken as the fused result, and then each subsequent frame is merged with this fused result. By synthesizing the segmentation results of multiple frames, a complete instance of lane lines and zebra crossings is obtained, avoiding the incomplete situation of lane lines and zebra crossings caused by vehicle occlusion, and enabling more accurate lane division and judgment of whether a vehicle has passed the stop line.

[0156] Based on the same inventive concept, an electronic device is further provided in an embodiment of the present application. The electronic device can implement the functions of the foregoing traffic congestion assessment method device. Refer to Figure 5 , the electronic device includes:

[0157] At least one processor 501, and a memory 502 connected to at least one processor 501. In the embodiment of the present application, the specific connection medium between the processor 501 and the memory 502 is not limited. Figure 5 In Figure 5 , it is taken as an example that the processor 501 and the memory 502 are connected through a bus 500. The bus 500 is represented by a thick line in Figure 5 . The connection manners between other components are only for illustrative purposes and are not to be taken as limiting. The bus 500 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation,

[0158] in Figure 4 it is only represented by a thick line, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 501 can also be called a controller, and there is no limitation on the name.

[0159] Wherein, the processor 501 is the control center of the device, and can connect various parts of the entire control device through various interfaces and lines. By running or executing the instructions stored in the memory 502 and calling the data stored in the memory 502, various functions of the device and process data, so as to monitor the device as a whole.

[0160] In a possible design, the processor 501 may include one or more processing units. The processor 501 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 501 either. In some embodiments, the processor 501 and the memory 502 may be implemented on the same chip, and in some embodiments, they may also be separately implemented on independent chips.

[0161] The processor 501 may be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the traffic congestion assessment method disclosed in combination with the embodiments of the present application may be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0162] As a non-volatile computer-readable storage medium, the memory 502 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 502 may include at least one type of storage medium, for example, it may include flash memory, a hard disk, a multimedia card, a card-type memory, a random access memory (RAM), a static random access memory (SRAM), a programmable read-only memory (PROM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic memory, a magnetic disk, an optical disk, etc. The memory 502 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 502 in the embodiments of the present application may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.

[0163] By programming the design of the processor 501, the code corresponding to the traffic congestion assessment method introduced in the foregoing embodiments can be solidified into the chip, so that the chip can execute when running Figure 2Steps of the traffic congestion assessment method of the illustrated embodiment. How to design and program the processor 501 is a well-known technology to those skilled in the art and will not be elaborated here.

[0164] Based on the same inventive concept, an embodiment of the present application provides a computer-readable storage medium. The computer program product includes: computer program code that, when run on a computer, causes the computer to execute any of the traffic congestion assessment methods described above. Since the principle of solving problems by the above computer-readable storage medium is similar to that of the traffic congestion assessment method, the implementation of the above computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be elaborated.

[0165] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0166] The present application is described with reference to the flowcharts and / or block diagrams of the method, device (system), and computer program product according to the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable traffic congestion assessment devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable traffic congestion assessment devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0167] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable traffic congestion assessment devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0168] These computer program instructions can also be loaded onto a computer or other programmable traffic congestion assessment device, so that a series of user operation steps are executed on the computer or other programmable device to generate a computer-implemented process, thereby providing instructions for implementing the process Figure 1 in one process or multiple processes and / or boxes Figure 1 steps of the functions specified in one box or multiple boxes.

[0169] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these changes and variations.

Claims

1. A traffic congestion assessment method, characterized in that, The method includes: Obtaining an image frame sequence corresponding to the current road and determining each lane corresponding to the image frame sequence; Counting the queuing vehicles in each lane starting from the stop line and calculating the passing duration of the trailing vehicle of the queuing vehicles in each lane passing through the stop line; Calculating the queuing time index corresponding to each lane according to each passing duration and the signal light cycle duration of the corresponding lane; Evaluating the congestion condition of the current road according to each queuing time index; Wherein, the counting the queuing vehicles in each lane starting from the stop line and calculating the passing duration of the trailing vehicle of the queuing vehicles in each lane passing through the stop line includes: Starting to count from the stop line and determining whether the distance between the front and rear vehicles in each lane is less than a preset threshold; If so, determining that the front and rear vehicles belong to the queuing vehicles; Determining the trailing vehicle in a stationary state among all the queuing vehicles in each lane and recording the first current timestamp; Tracking each trailing vehicle and taking the time difference between the second current timestamp when each trailing vehicle passes through the stop line and the first timestamp as the passing duration of each trailing vehicle passing through the stop line.

2. The method according to claim 1, characterized in that, The determining each lane corresponding to the image frame sequence includes: Segmenting lane lines in each frame image of the image frame sequence, wherein the lane lines include lane boundary lines and stop lines; Fusing each lane line to obtain a complete lane line corresponding to the image frame sequence; Dividing each lane in the bird's-eye view perspective through the complete lane line.

3. The method according to claim 2, characterized in that, The fusing each lane line to obtain a complete lane line corresponding to the image frame sequence includes: Determining the same lane line corresponding to any lane line in the current frame image in the previous frame image by calculating the intersection-over-union ratio of the lane lines in the current frame image and the previous frame image in the image frame sequence; Fusing the same lane line in the previous frame image and the current frame image and caching the fusion result to obtain a cached result; Fusing the same lane line in the next frame image in the image frame sequence and the cached result and caching the fusion result to obtain an updated cached result until the complete lane line is obtained.

4. The method according to claim 2, wherein The dividing each lane in the bird's-eye view perspective through the complete lane line includes: Based on the prior information of the zebra crossing in the real world, performing coordinate transformation on the complete lane line and the image information in the image frame sequence to obtain bird's-eye view information, wherein the prior information is that the white solid lines forming the zebra crossing are parallel to each other; Dividing each lane in the bird's-eye view information through the complete lane line information in the bird's-eye view information.

5. The method according to claim 1, wherein The calculating the queuing time index corresponding to each lane according to each passing duration and the signal light cycle duration of the corresponding lane includes: Detecting the signal lights in each frame image of the image frame sequence and the states of the signal lights, wherein the states of the signal lights include red lights and green lights; When the states of the signal lights in any two adjacent frames of the image frame sequence are different, record the timestamps of the two adjacent frames of the image; According to the recorded timestamps and the signal light states corresponding to the timestamps, determine the signal light cycle duration of each lane; Calculate the ratio between each passing duration and the signal light cycle duration of the corresponding lane, and use each ratio as the queuing time index corresponding to each lane.

6. The method according to claim 1, wherein The evaluation of the congestion condition of the current road according to each queuing time index includes: Sort the queuing time indices to obtain the maximum queuing time index; According to the range where the maximum queuing time index is located, determine the congestion level corresponding to the current road, where the congestion level includes at least one of unobstructed, slightly congested, moderately congested, and severely congested.

7. A traffic congestion assessment device, characterized in that, The device includes: A determination module, configured to obtain an image frame sequence corresponding to a current road and determine each lane corresponding to the image frame sequence; A calculation module, configured to count the queuing vehicles in each lane starting from the stop line and calculate the passing duration of the trailing vehicle of the queuing vehicles in each lane passing through the stop line; calculate the queuing time index corresponding to each lane according to each passing duration and the signal light cycle duration of the corresponding lane; An evaluation module, configured to evaluate the congestion condition of the current road according to each queuing time index; Wherein, the calculation module is specifically configured to: Start counting from the stop line and determine whether the distance between the front and rear vehicles in each lane is less than a preset threshold; If so, determine that the front and rear vehicles belong to queuing vehicles; Determine the trailing vehicle in a stationary state among all the queuing vehicles in each lane and record the first current timestamp; Track each trailing vehicle and use the time difference between the second current timestamp when each trailing vehicle passes through the stop line and the first timestamp as the passing duration of each trailing vehicle passing through the stop line.

8. An electronic device, characterized in that, Include: A memory, configured to store program instructions; A processor, configured to call the program instructions stored in the memory and execute the steps included in the method according to any one of claims 1-6 according to the obtained program instructions.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and the computer program includes program instructions, and when the program instructions are executed by a computer, the computer is caused to execute the method according to any one of claims 1-6.

Images