Video-based road regulation and control method and device, computer equipment and storage medium

Abstract

The invention relates to the technical field of traffic control, in particular to a video-based road regulation and control method and device, computer equipment and a storage medium. Vehicle detection and multi-target vehicle tracking are introduced to carry out driving track construction on a road video sequence in a preset time period; a plurality of road sections in a plurality of paths in an area are subjected to congestion judgment according to a constructed driving track, a traffic flow optimization model is applied, traffic data are analyzed in real time, and the crowded road sections are dynamically scheduled, so that the traffic management cost can be effectively reduced, the management efficiency can be improved, and the traffic congestion problem can be relieved; and sustainable development of urban traffic is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic control, and in particular to a video-based road control method, device, computer equipment and storage medium. Background Art

[0002] As the urbanization process accelerates, the living standards of residents have significantly improved. As an important means of transportation in modern life, the number of cars has increased dramatically. This change has greatly enriched people's travel methods and improved the convenience of life, but it has also brought unprecedented pressure and challenges to the urban transportation system. The problem of traffic congestion is becoming increasingly serious, which not only seriously affects the travel efficiency and quality of life of citizens, but also aggravates environmental problems such as air pollution and noise pollution, posing a major threat to the sustainable development of cities.

[0003] Traditionally, traffic management departments rely on manual inspections and surveillance cameras to supervise road vehicles, but this model has significant drawbacks. On the one hand, manual supervision requires a lot of human and material resources, which is costly and inefficient, and it is difficult to cope with increasingly complex traffic conditions; on the other hand, although traditional surveillance cameras can provide a certain degree of visual information, they lack intelligent processing and analysis capabilities, and cannot identify vehicles and track their driving trajectories in real time and accurately, let alone effectively predict and respond to emergencies such as traffic congestion. Therefore, developing an efficient and intelligent traffic management system to achieve real-time monitoring, accurate tracking and dynamic optimization of road vehicles has become the key to solving the current traffic congestion problem. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide a video-based road control method, device, computer equipment and storage medium, which constructs a driving trajectory for a road video sequence within a preset time period by introducing vehicle detection and multi-target vehicle tracking, and makes congestion judgments on several sections of several paths in the area based on the constructed driving trajectory, and applies a traffic flow optimization model to analyze traffic data in real time and dynamically schedule congested sections, which can effectively reduce traffic management costs, improve management efficiency, alleviate traffic congestion problems, and promote the sustainable development of urban transportation.

[0005] In a first aspect, an embodiment of the present application provides a video-based road control method, comprising the following steps:

[0006] Obtaining a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of a plurality of time frames, the road area images include a plurality of paths from a starting point to an end point; the paths correspond to road sections connected by a plurality of road section intersections;

[0007] Inputting the road video sequence into a preset target detection model to perform target detection, and obtaining a vehicle detection result of the road video sequence;

[0008] Inputting the vehicle detection results of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtaining a vehicle trajectory set of the road video sequence, wherein the vehicle trajectory set includes the vehicle trajectories of several vehicles;

[0009] According to the set of driving trajectories in the road video sequence, a congestion judgment is performed on a plurality of road sections in a plurality of paths to obtain a plurality of congested road sections; a plurality of paths associated with the plurality of congested road sections are used as road regulation demand paths to obtain a plurality of road regulation demand paths and monitoring data of the plurality of road regulation demand paths;

[0010] The monitoring data of a plurality of the road control demand paths are input into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and road control is performed according to the traffic flow optimization strategy.

[0011] In a second aspect, an embodiment of the present application provides a video-based road control device, including:

[0012] A road video sequence acquisition module is used to obtain a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from a starting point to an end point; the paths correspond to road sections connected by several road section intersections;

[0013] A vehicle detection module is used to input the road video sequence into a preset target detection model to perform target detection and obtain a vehicle detection result of the road video sequence;

[0014] Driving trajectory construction, for inputting the vehicle detection results of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtaining a driving trajectory set of the road video sequence, wherein the driving trajectory set includes driving trajectories of several vehicles;

[0015] A road control demand judgment module is used to judge the congestion of several sections in several paths according to the driving trajectory set of the road video sequence, and obtain several congested sections; several paths associated with several congested sections are used as road control demand paths, and several road control demand paths and monitoring data of several road control demand paths are obtained;

[0016] The traffic flow optimization module is used to input the monitoring data of several road control demand paths into a preset traffic flow optimization model, obtain a traffic flow optimization strategy, and perform road control according to the traffic flow optimization strategy.

[0017] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the video-based road control method described in the first aspect are implemented.

[0018] In a fourth aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video-based road control method as described in the first aspect are implemented.

[0019] In an embodiment of the present application, a video-based road control method, apparatus, computer equipment and storage medium are provided. By introducing vehicle detection and multi-target vehicle tracking, a driving trajectory is constructed for a road video sequence within a preset time period. According to the constructed driving trajectory, congestion judgment is made on several sections of several paths in the area, and a traffic flow optimization model is applied to analyze traffic data in real time and dynamically schedule congested sections. This can effectively reduce traffic management costs, improve management efficiency, alleviate traffic congestion problems, and promote the sustainable development of urban transportation.

[0020] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A schematic diagram of a flow chart of a video-based road control method provided in one embodiment of the present application;

[0022] Figure 2 A schematic diagram of the process of S3 in the video-based road control method provided in one embodiment of the present application;

[0023] Figure 3 A schematic diagram of the process of S33 in the video-based road control method provided in one embodiment of the present application;

[0024] Figure 4 A schematic diagram of the process of S331 in the video-based road control method provided in one embodiment of the present application;

[0025] Figure 5 A schematic diagram of the process of S3 in the video-based road control method provided in another embodiment of the present application;

[0026] Figure 6A schematic diagram of the process of S4 in the video-based road control method provided in one embodiment of the present application;

[0027] Figure 7 A schematic diagram of the process of S5 in the video-based road control method provided in one embodiment of the present application;

[0028] Figure 8 A schematic diagram of the structure of a video-based road control device provided in one embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0030] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0031] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0032] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0033] See also Figure 1 , Figure 1 A schematic diagram of a flow chart of a video-based road control method provided in one embodiment of the present application, the method comprising the following steps:

[0034] S1: Obtain a road video sequence of a target area within a preset time period.

[0035] The execution subject of the video-based road control method is a road control device of the video-based road control method (hereinafter referred to as the road control device). In an optional embodiment, the road control device can be a computer device, a server, or a server cluster composed of multiple computer devices.

[0036] In this embodiment, the road control device can obtain a road video sequence of the target area input by the user within a preset time period, or obtain a road video sequence of the target area within a preset time period through a preset database, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from a starting point to an end point; the paths correspond to road sections connected by several road section entrances.

[0037] S2: Inputting the road video sequence into a preset target detection model to perform target detection, and obtaining a vehicle detection result of the road video sequence.

[0038] The target detection model adopts the YOLOv5s model. The YOLOv5s model is one of the models in this series. It is the model with the smallest depth and the smallest feature map width, and it also has good results in terms of accuracy. The YOLOv5 model can directly predict the category and position of the target object through a single forward propagation, which greatly improves the detection speed and can still maintain a high accuracy in complex traffic scenes, providing reliable technical support for real-time vehicle detection.

[0039] In this embodiment, the road control device inputs the road video sequence into a preset target detection model, and performs target detection through the target detection model, namely, the input module (Input), backbone network module (Backbone), feature fusion module (Neck) and prediction module (Prediction) of the YOLOv5s model to obtain the vehicle detection result of the road video sequence, wherein the vehicle detection result of the road video sequence includes the vehicle detection results of the road area images of several time frames, and the vehicle detection result includes the detection boxes of several vehicles.

[0040] S3: Inputting the vehicle detection results of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtaining a set of vehicle trajectories of the road video sequence.

[0041] The multi-target tracking model adopts the DeepSORT (Deep Learning for Real-Time Multi-Object Tracking and Segmentation) model. The DeepSORT model combines the advantages of deep learning and traditional tracking algorithms, and can achieve stable tracking effects in multi-target tracking scenarios. By introducing deep learning feature extraction and Kalman filter prediction, it effectively solves problems such as target occlusion and loss, providing a more accurate and robust solution for vehicle tracking.

[0042] In this embodiment, the road control device inputs the vehicle detection results of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtains a driving trajectory set of the road video sequence, wherein the driving trajectory set includes the driving trajectories of several vehicles.

[0043] See also Figure 2 , Figure 2 The flowchart of S3 in the video-based road control method provided in one embodiment of the present application includes steps S31 to S34, which are specifically as follows:

[0044] S31: Taking the road area image of the first time frame as the target image, obtaining vehicle state space information of the target image.

[0045] In this embodiment, the road control device uses the road area image of the first time frame as the target image and obtains the vehicle state space information of the target image, wherein the vehicle state space information includes the center position parameters, aspect ratio and height parameters of the detection frames of several vehicles.

[0046] S32: construct driving trajectories of several vehicles in the target image, predict the trajectory of the target image in the next time frame according to the vehicle state space information of the target image and a preset Kalman filter, and obtain the driving trajectories of several vehicles in the target image in the next time frame.

[0047] In this embodiment, the road control equipment constructs the driving trajectories of several vehicles in the target image, predicts the trajectory of the target image in the next time frame according to the vehicle state space information of the target image and a preset Kalman filter, and obtains the driving trajectories of several vehicles in the target image in the next time frame.

[0048] Specifically, the road control device constructs an eight-dimensional state space (u, v, γ, h, x) of several vehicles in the target image according to the vehicle state space information of the target image. · ,y · ,γ ·,h), where the center position parameter is (u,v), the aspect ratio is γ, the height parameter is h, and (x,y,γ,h) is the change speed of (u,v,γ,h), that is, the first-order derivative of (u,v,γ,h) in the image coordinates. The road control equipment inputs the eight-dimensional state space of several vehicles in the target image into the Kalman filter to predict the trajectory of the target image in the next time frame, and obtains the driving trajectory of several vehicles in the target image in the next time frame.

[0049] S33: According to the detection frames and driving trajectories of several vehicles in the target image in the next time frame, determine whether the detection frames and driving trajectories match, associate the detection frames with the corresponding driving trajectories, and update the driving trajectories of several vehicles in the target image according to the Kalman filter to obtain updated driving trajectories of several vehicles in the target image.

[0050] In this embodiment, the road control device determines whether the detection frame and the driving trajectory match based on the detection frames and driving trajectories of several vehicles in the target image in the next time frame, associates the detection frame to the corresponding driving trajectory, updates the driving trajectories of several vehicles in the target image according to the Kalman filter, and obtains updated driving trajectories of several vehicles in the target image.

[0051] See also Figure 3 , Figure 3 The flowchart of S33 in the video-based road control method provided in one embodiment of the present application includes steps S331 to S332, which are as follows:

[0052] S331: performing feature extraction based on the detection frames and driving trajectories of the plurality of vehicles in the road area image of the next time frame to obtain feature data between the detection frames and driving trajectories of the plurality of vehicles.

[0053] In this embodiment, the road control device performs feature extraction based on the detection frames and driving trajectories of several vehicles in the road area image of the next time frame to obtain feature data between the detection frames and driving trajectories of the several vehicles, wherein the feature data includes motion features and appearance features.

[0054] See also Figure 4 , Figure 4 The flowchart of S331 in the video-based road control method provided in one embodiment of the present application includes steps S3311 to S3312, which are as follows:

[0055] S3311: Based on the detection frames of several vehicles, the driving trajectories and the preset Mahalanobis distance calculation algorithm of the road area image of the next time frame, the Mahalanobis distances between the several driving trajectories and the detection frames of several vehicles are obtained as the motion features.

[0056] In this embodiment, the road control device obtains the Mahalanobis distances between the plurality of vehicle trajectories and the plurality of vehicle detection frames as the motion features according to the plurality of vehicle detection frames, the vehicle trajectories and the preset Mahalanobis distance calculation algorithm of the road area image of the next time frame; wherein the Mahalanobis distance calculation algorithm is:

[0057]

[0058] In the formula, is the Mahalanobis distance between the i-th vehicle trajectory and the j-th vehicle detection box, d j is the position of the detection box of the jth vehicle, y i is the position of the detection box of the jth vehicle in the i-th driving trajectory, which represents the prediction result of the vehicle in the next time frame; is the inverse of the covariance between the target detection position and the tracked position.

[0059] S3312: Based on the Mahalanobis distance and a preset Mahalanobis distance threshold, if the Mahalanobis distance is less than the Mahalanobis distance threshold, it is determined that the vehicle and the driving trajectory are successfully motion-associated. For the driving trajectory and the vehicle that are successfully motion-associated, the cosine distance between the detection frame of the driving trajectory and the vehicle that are successfully motion-associated is obtained according to a preset cosine distance calculation algorithm as the appearance feature.

[0060] In this embodiment, the road control device determines that the vehicle and the driving trajectory are successfully associated in motion according to the Mahalanobis distance and the preset Mahalanobis distance threshold value, if the Mahalanobis distance is less than the Mahalanobis distance threshold value, and for the driving trajectory and the vehicle that are successfully associated in motion, the cosine distance between the driving trajectory and the detection frame of the vehicle that are successfully associated in motion is obtained according to a preset cosine distance calculation algorithm as the appearance feature, wherein the cosine distance calculation algorithm is:

[0061]

[0062] Where, is the cosine distance between the i-th vehicle trajectory with successful appearance association and the detection frame of the j-th vehicle, R i is the ith driving trajectory, is the detection box of the kth vehicle in the ith driving trajectory, r k is the feature vector of the detection box of the kth vehicle, r j is the feature vector of the detection box of the jth vehicle.

[0063] S332: construct a cost matrix based on the feature data between the detection frames and driving trajectories of several vehicles; determine whether the detection frame of the vehicle matches the driving trajectory based on the cost matrix, and if the match is successful, associate the detection frame with the corresponding driving trajectory to obtain the initial driving trajectories of several vehicles in the target image.

[0064] In this embodiment, the road control device constructs a cost matrix based on the feature data between the detection frames and the driving trajectories of the multiple vehicles. Specifically, the road control device linearly weights the appearance features and motion features in the feature data to obtain the cost matrix between the detection frames and the driving trajectories of the multiple vehicles, wherein the cost matrix includes the metric distances between the detection frames of the multiple driving trajectories and the multiple vehicles, and the metric distances are:

[0065]

[0066] In the formula, c (i,j) is the metric distance between the i-th driving trajectory and the detection frame of the j-th vehicle, ω is the weighting coefficient, and when the detection frame of the vehicle is blocked for a long time, ω = 0.

[0067] The road control device determines whether the vehicle detection frame matches the driving trajectory based on the cost matrix. If the match is successful, the detection frame is associated with the corresponding driving trajectory to obtain the initial driving trajectories of several vehicles in the target image.

[0068] Specifically, the road control device determines that the detection frame of the vehicle successfully matches the driving trajectory based on the metric distance between the several driving trajectories and the detection frames of the several vehicles in the cost matrix and a preset metric distance threshold. If the metric distance is less than the metric distance threshold, the road control device determines that the detection frame of the vehicle successfully matches the driving trajectory, otherwise the matching fails.

[0069] S34: taking the road area image of the next time frame as the target image, repeatedly performing trajectory prediction, matching and updating operations until the updated driving trajectories of several vehicles in the road area image of the last time frame are obtained as the driving trajectories, and constructing a driving trajectory set of the road video sequence.

[0070] In this embodiment, the road control device takes the road area image of the next time frame as the target image, and repeatedly performs trajectory prediction, matching and updating operations until the updated driving trajectories of several vehicles in the road area image of the last time frame are obtained as the driving trajectory, and a driving trajectory set of the road video sequence is constructed.

[0071] See also Figure 5 , Figure 5The flowchart of S3 in the video-based road control method provided in another embodiment of the present application includes steps S35 to S37. The steps S35 to S37 are after step S32 and before step S33. Specifically, they are as follows:

[0072] S35: confirming the states of the driving tracks of the plurality of vehicles in the target image to obtain the states of the driving tracks of the plurality of vehicles.

[0073] When the detection target is blocked for a long time, the accuracy of the Kalman filter algorithm is low, and it is easy to have the problem of matching trajectory errors. In this embodiment, the road control equipment uses the Kalman filter to confirm the status of the driving trajectories of several vehicles in the target image, and obtains the status of the driving trajectories of several vehicles, wherein the status includes a confirmed state and a non-confirmed state.

[0074] S36: If the state of the driving trajectory is a confirmation state, a driving trajectory corresponding to the road area image of the previous time frame is cascade matched with a detection frame of the target image of the current time frame to obtain a cascade matching result.

[0075] If the state of the driving trajectory is a confirmation state, in this embodiment, the road control device cascade matches the driving trajectory corresponding to the road area image of the previous time frame with the detection frame of the target image in the current time frame to obtain a cascade matching result, wherein the cascade matching result includes a successfully matched driving trajectory, an unmatched driving trajectory, and an unmatched detection frame.

[0076] S37: If the state of the driving trajectory is unconfirmed, the driving trajectory and the unmatched driving trajectory are respectively matched with the unmatched detection frame to determine whether the unmatched detection frame matches the driving trajectory and the unmatched driving trajectory. If the match is successful, the detection frame is associated with the corresponding driving trajectory.

[0077] If the state of the driving trajectory is an unconfirmed state, in this embodiment, the road control device matches the driving trajectory and the unmatched driving trajectory with the unmatched detection frame respectively, and determines whether the unmatched detection frame matches the driving trajectory and the unmatched driving trajectory. If the match is successful, the detection frame is associated with the corresponding driving trajectory.

[0078] S4: Based on the set of driving trajectories in the road video sequence, congestion judgment is performed on several sections in several paths to obtain several congested sections; several paths associated with the several congested sections are used as road control demand paths to obtain several road control demand paths and monitoring data of several road control demand paths.

[0079] In this embodiment, the road control device performs congestion judgment on several sections in several paths according to the set of driving trajectories in the road video sequence to obtain several congested sections.

[0080] The road control device uses a plurality of paths associated with the plurality of congested road sections as road control demand paths, obtains the plurality of road control demand paths and monitoring data of the plurality of road control demand paths, wherein the monitoring data of the plurality of road control demand paths includes the maximum number of vehicles that can be accommodated on the paths, the maximum delay time, and the general delay time;

[0081] See also Figure 6 , Figure 6 The flowchart of S4 in the video-based road control method provided in one embodiment of the present application includes steps S41 to S44, which are specifically as follows:

[0082] S41: Obtain the cumulative number of vehicles entering and passing through the several road sections, calculate the vehicle inflow rate according to the cumulative number of vehicles entering the road sections, and obtain the vehicle inflow rate of the several road sections.

[0083] In this embodiment, the road control equipment obtains the cumulative number of vehicles entering and passing through the several sections, calculates the vehicle inflow rate according to the cumulative number of vehicles entering each section, and obtains the vehicle inflow rate of the several sections. Specifically, the road control equipment divides the cumulative number of vehicles entering each section within the preset time period by the time corresponding to the time period, and obtains the vehicle inflow rate of the several sections.

[0084] S42: Counting the passing time of several vehicles passing through the several sections, using the upper quartile method, calculating the common time delay parameter according to the passing time of several vehicles passing through the same section, and obtaining the common delay time of the several sections.

[0085] In this embodiment, the road control equipment counts the passing time of several vehicles passing through several of the sections, and uses the upper quartile method to calculate the common time delay parameter based on the passing time of several vehicles passing through the same section to obtain the common delay time of the several sections.

[0086] S43: Obtain the maximum number of vehicles that can be accommodated by the several road sections, and perform a joint solution based on the maximum number of vehicles that can be accommodated by the several road sections, the cumulative number of entering vehicles, the vehicle inflow rate, the general time delay parameter, and the preset arrival curve function and service curve function to obtain the maximum delay time of the several road sections.

[0087] In this embodiment, the road control device obtains the maximum number of vehicles that can be accommodated by the several sections, and performs a joint solution according to the maximum number of vehicles that can be accommodated by the several sections, the cumulative number of vehicles entering, the vehicle inflow rate, the general time delay parameter, and the preset arrival curve function and service curve function to obtain the maximum delay time of the several sections, wherein the arrival curve function is:

[0088] α(t)=σ k +ρ k t

[0089] Where α(t) is the arrival curve value, t is the tth time point, σ k is the maximum number of vehicles that can be accommodated on the kth road section, ρ k is the vehicle inflow rate of the kth road segment;

[0090] The service curve function is:

[0091] β(t)=R k *[(tT k ) + ]

[0092] Where β(t) is the service curve value, R k is the cumulative number of vehicles entering the kth road segment, T k is the general delay time of the kth road segment.

[0093] Specifically, when the road control device obtains the arrival curve value α(t) equal to the service curve value β(t), the time point t on the arrival curve function α and time point t on the service curve function β , solve max(t β -t α ), taking the solution result as the maximum delay time, and obtaining the maximum delay time of several of the road sections.

[0094] S44: Obtain the lengths of the several road sections, obtain the actual travel speeds of the several road sections according to the lengths of the several road sections and the maximum delay time, make congestion judgments on the several road sections according to the actual travel speeds of the several road sections and a preset average travel speed interval, and obtain several congested road sections.

[0095] In this embodiment, the road control device obtains the lengths of the several road sections, and obtains the actual travel speeds of the several road sections according to the lengths of the several road sections and the maximum delay time, wherein the actual travel speed is:

[0096]

[0097] Where V j is the actual travel speed of the jth section, S j is the length of the jth road segment, T △j is the maximum delay time of the jth road segment.

[0098] The road control device makes a congestion judgment on the several road sections according to the actual travel speeds of the several road sections and a preset average travel speed interval. If the actual travel speed is within the travel speed interval, the road section is judged to be a congested section, and several congested road sections are obtained.

[0099] S5: inputting the monitoring data of a plurality of the road control demand paths into a preset traffic flow optimization model, obtaining a traffic flow optimization strategy, and performing road control according to the traffic flow optimization strategy.

[0100] In this embodiment, the road control equipment inputs the monitoring data of several road control demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and performs road control according to the traffic flow optimization strategy.

[0101] See also Figure 7 , Figure 7 The flowchart of S5 in the video-based road control method provided in one embodiment of the present application includes steps S51 to S53, which are specifically as follows:

[0102] S51: Obtain the total vehicle inflow rate of the target area within a preset time period, and obtain the vehicle inflow ratio setting rate of several road control demand paths based on the total vehicle inflow rate, the monitoring data of several road control demand paths, and the vehicle inflow ratio setting rate calculation algorithm in the traffic flow optimization model.

[0103] In this embodiment, the road control device obtains the total vehicle inflow rate of the target area within a preset time period, wherein the total vehicle inflow rate can be obtained by detecting the cumulative number of vehicles entering the target area and dividing the cumulative number of vehicles entering the target area by the time period. The road control device obtains the vehicle inflow ratio setting rate of the plurality of the road control demand paths according to the total vehicle inflow rate, the monitoring data of the plurality of the road control demand paths, and the vehicle inflow ratio setting rate calculation algorithm in the traffic flow optimization model, wherein the vehicle inflow ratio setting rate calculation algorithm is:

[0104]

[0105] In the formula, ρ′ i Set the rate for the vehicle inflow ratio of the i-th road control demand path, Ri is the cumulative number of vehicles entering the i-th road control demand path, σ i is the maximum number of vehicles that can be accommodated on the i-th road control demand path, △ is the maximum delay time of the i-th road control demand path, and the maximum delay time of the road control demand path can be pre-input into the road control device by the user, T i is the general delay time of the ith road regulation demand path, T j is the general delay time of the jth road regulation demand path, ρ all is the total vehicle inflow rate.

[0106] S52: According to the vehicle inflow ratio setting rate of the several road control demand paths and the objective function, queuing delay time constraint condition and vehicle inflow rate constraint condition in the traffic flow optimization model, the queuing delay time is calculated using the single objective programming model method to obtain the queuing delay time of the several road control demand paths.

[0107] The objective function is:

[0108]

[0109] Where M j is the queuing delay time of the j-th road control demand path.

[0110] The queuing delay time constraint is:

[0111]

[0112] In the formula, σ j is the maximum number of vehicles that can be accommodated on the j-th road control demand path, R j is the cumulative number of vehicles entering the jth road control demand path, ρ j Regulate the vehicle inflow rate of the demand path for the jth road.

[0113] The vehicle inflow rate constraint condition is:

[0114]

[0115] In this embodiment, the road control equipment uses a single objective programming model method to calculate the queue delay time based on the vehicle inflow ratio setting rate of several road control demand paths and the objective function, queue delay time constraint, and vehicle inflow rate constraint in the traffic flow optimization model to obtain the queue delay time of several road control demand paths.

[0116] S53: Determine the road control demand path with the maximum queuing delay time and the minimum queuing delay time, adjust the vehicle inflow ratio setting rate of the road control demand path with the maximum queuing delay time and the minimum queuing delay time, calculate the queuing delay time again, and determine the final vehicle inflow ratio setting rate of several road control demand paths according to the queuing delay times of several road control demand paths calculated twice adjacently as the vehicle inflow control rate, and construct the traffic flow optimization strategy.

[0117] In this embodiment, the road control device determines the road control demand path with the maximum queuing delay time and the minimum queuing delay time, adjusts the vehicle inflow ratio setting rate of the road control demand path with the maximum queuing delay time and the minimum queuing delay time, calculates the queuing delay time again, and determines the final vehicle inflow ratio setting rate of several road control demand paths based on the queuing delay times of several road control demand paths calculated twice adjacently, as the vehicle inflow control rate, to construct the traffic flow optimization strategy.

[0118] Specifically, the road control equipment reduces the vehicle inflow ratio setting rate of the road control demand path with the maximum queuing delay time, and increases the vehicle inflow ratio setting rate of the road control demand path with the minimum queuing delay time, so as to adjust the vehicle inflow ratio setting rate of the road control demand path with the maximum queuing delay time and the minimum queuing delay time.

[0119] The road control device calculates the queuing delay time according to the vehicle inflow ratio setting rate of the adjusted road control demand path, the vehicle inflow ratio setting rates of other road control demand paths, and the objective function, queuing delay time constraint condition, and vehicle inflow rate constraint condition in the traffic flow optimization model, and obtains the queuing delay time of several road control demand paths calculated next time; if the maximum queuing delay time of the road control demand path calculated next time is consistent with the maximum queuing delay time calculated last time, the vehicle inflow ratio setting rate of several road control demand paths calculated next time is used as the final vehicle inflow ratio setting rate; if the maximum queuing delay time of the road control demand path calculated next time is consistent with the maximum queuing delay time calculated last time, The maximum queuing delay time calculated last time is inconsistent. The road control equipment repeatedly determines the road control demand path with the maximum queuing delay time and the minimum queuing delay time, performs the vehicle inflow ratio setting rate adjustment and queuing delay time calculation operations, until the maximum queuing delay time of the road control demand path calculated next time is consistent with the maximum queuing delay time calculated last time, obtains the vehicle inflow control rate of several road control demand paths, constructs the traffic flow optimization strategy, analyzes traffic data in real time, and performs dynamic scheduling in a way that reduces the average delay and maximum delay difference of each road section, thereby solving the problem of alleviating traffic congestion and achieving the effect of effectively reducing traffic management costs, improving management efficiency, and promoting the sustainable development of urban transportation.

[0120] Please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of a video-based road control device provided by an embodiment of the present application. The device can implement all or part of the video-based road control device through software, hardware, or a combination of both. The device 8 includes:

[0121] The road video sequence acquisition module 81 is used to obtain a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from a starting point to an end point; the paths correspond to road sections connected by several road section intersections;

[0122] A vehicle detection module 82, used for inputting the road video sequence into a preset target detection model to perform target detection and obtain a vehicle detection result of the road video sequence;

[0123] Driving trajectory construction 83, for inputting the vehicle detection result of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtaining a driving trajectory set of the road video sequence, wherein the driving trajectory set includes driving trajectories of several vehicles;

[0124] The road control demand judgment module 84 is used to judge the congestion of several sections in several paths according to the driving trajectory set of the road video sequence, and obtain several congested sections; use several paths associated with several congested sections as road control demand paths, and obtain several road control demand paths and monitoring data of several road control demand paths;

[0125] The traffic flow optimization module 85 is used to input the monitoring data of a plurality of the road control demand paths into a preset traffic flow optimization model, obtain a traffic flow optimization strategy, and perform road control according to the traffic flow optimization strategy.

[0126] In an embodiment of the present application, a road video sequence of a target area within a preset time period is obtained through a road video sequence acquisition module, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from a starting point to an end point; the paths correspond to road sections connected by several road section intersections; through a vehicle detection module, the road video sequence is input into a preset target detection model for target detection to obtain a vehicle detection result of the road video sequence; through driving trajectory construction, the vehicle detection result of the road video sequence is input into a preset multi-target tracking model for target tracking and trajectory construction to obtain a driving trajectory of the road video sequence. Trajectory set, wherein the driving trajectory set includes the driving trajectories of several vehicles; through the road control demand judgment module, according to the driving trajectory set of the road video sequence, several sections in several paths are judged to be congested, and several congested sections are obtained; several paths associated with several congested sections are used as road control demand paths, and several road control demand paths and monitoring data of several road control demand paths are obtained; through the traffic flow optimization module, the monitoring data of several road control demand paths are input into the preset traffic flow optimization model, and the traffic flow optimization strategy is obtained, and the road control is performed according to the traffic flow optimization strategy. By introducing vehicle detection and multi-target vehicle tracking to construct driving trajectories for road video sequences within a preset time period, congestion judgment is performed on several sections in several paths in the area according to the constructed driving trajectories, and the traffic flow optimization model is applied to analyze traffic data in real time, and the congested sections are dynamically scheduled, which can effectively reduce traffic management costs, improve management efficiency, alleviate traffic congestion problems, and promote the sustainable development of urban transportation.

[0127] Please refer to Fig. 9 , Fig. 9The computer device 9 is a schematic diagram of a structure of a computer device provided in an embodiment of the present application. The computer device 9 includes: a processor 91, a memory 92, and a computer program 83 stored in the memory 92 and executable on the processor 91; the computer device may store multiple instructions, which are suitable for being loaded and executed by the processor 91 Figures 1 to 6 The method steps shown in the figure can be found in the specific execution process. Figures 1 to 6 The specific description shown will not be repeated here.

[0128] Among them, the processor 91 may include one or more processing cores. The processor 91 uses various interfaces and lines to connect various parts in the server, and executes various functions and processes data of the video-based road control device 8 by running or executing instructions, programs, code sets or instruction sets stored in the memory 92, and calling the data in the memory 92. Optionally, the processor 91 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programble Logic Array, PLA). The processor 91 can integrate one or more combinations of a central processing unit 91 (Central Processing Unit, CPU), a graphics processor 91 (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 91, and it can be implemented by a single chip.

[0129] Among them, the memory 92 may include a random access memory 92 (Random Access Memory, RAM), and may also include a read-only memory 92 (Read-Only Memory). Optionally, the memory 92 includes a non-transitory computer-readable storage medium. The memory 92 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 92 may also be optionally at least one storage device located away from the aforementioned processor 91.

[0130] The present application also provides a storage medium that can store multiple instructions, which are suitable for the processor to load and execute the above-mentioned Figures 1 to 6 The method steps shown in the figure can be found in the specific execution process. Figures 1 to 6 The specific description shown will not be repeated here.

[0131] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0132] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0133] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraint algorithm of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0134] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0138] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications to the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims and equivalent technologies of the present invention, the present invention is also intended to include these changes and modifications.

Claims

1. A video-based road control method, characterized in that: The following steps are involved: Obtaining a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of a plurality of time frames, the road area images include a plurality of paths from a starting point to an end point; the paths correspond to road sections connected by a plurality of road section intersections; Inputting the road video sequence into a preset target detection model to perform target detection, and obtaining a vehicle detection result of the road video sequence; Inputting the vehicle detection results of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtaining a vehicle trajectory set of the road video sequence, wherein the vehicle trajectory set includes the vehicle trajectories of several vehicles; According to the set of driving trajectories in the road video sequence, a congestion judgment is performed on a plurality of road sections in a plurality of paths to obtain a plurality of congested road sections; a plurality of paths associated with the plurality of congested road sections are used as road regulation demand paths to obtain a plurality of road regulation demand paths and monitoring data of the plurality of road regulation demand paths; The monitoring data of a plurality of the road control demand paths are input into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and road control is performed according to the traffic flow optimization strategy.

2. The video-based road control method according to claim 1, characterized in that: The vehicle detection result of the road video sequence includes vehicle detection results of road area images of several time frames, and the vehicle detection result includes detection frames of several vehicles; The vehicle detection results of the road video sequence are input into a preset multi-target tracking model to perform target tracking and trajectory construction, and a set of vehicle trajectories of the road video sequence is obtained, including the steps of: Taking the road area image of the first time frame as the target image, obtaining the vehicle state space information of the target image, wherein the vehicle state space information includes the center position parameters, aspect ratio and height parameters of the detection frames of several vehicles; Constructing driving trajectories of several vehicles in the target image, predicting the trajectory of the target image in the next time frame according to the vehicle state space information of the target image and a preset Kalman filter, and obtaining the driving trajectories of several vehicles in the target image in the next time frame; According to the detection frames and driving trajectories of the several vehicles in the target image in the next time frame, determine whether the detection frames and driving trajectories match, associate the detection frames with the corresponding driving trajectories, and update the driving trajectories of the several vehicles in the target image according to the Kalman filter to obtain updated driving trajectories of the several vehicles in the target image; The road area image of the next time frame is used as the target image, and the trajectory prediction, matching and updating operations are repeatedly performed until the updated driving trajectories of several vehicles in the road area image of the last time frame are obtained as the driving trajectories, and the driving trajectory set of the road video sequence is constructed.

3. The video-based road control method according to claim 2, characterized in that: The method of judging whether the detection frames and the driving trajectories of the plurality of vehicles in the target image of the next time frame match each other comprises the steps of: Extracting features based on the detection frames and driving trajectories of the multiple vehicles in the road area image of the next time frame to obtain feature data between the detection frames and driving trajectories of the multiple vehicles; A cost matrix is ​​constructed based on feature data between detection frames and driving trajectories of several vehicles. Whether the detection frame of the vehicle matches the driving trajectory is determined based on the cost matrix. If the match is successful, the detection frame is associated with the corresponding driving trajectory to obtain the initial driving trajectories of several vehicles in the target image.

4. The video-based road control method according to claim 3, characterized in that: The feature data includes motion features and appearance features; The method of extracting features from the detection frames and driving trajectories of several vehicles in the road area image of the next time frame to obtain feature data between the detection frames and driving trajectories of several vehicles comprises the following steps: According to the detection frames of several vehicles, the driving tracks and the preset Mahalanobis distance calculation algorithm of the road area image of the next time frame, the Mahalanobis distances between the several driving tracks and the detection frames of several vehicles are obtained as the motion features; wherein the Mahalanobis distance calculation algorithm is: In the formula, is the Mahalanobis distance between the i-th vehicle trajectory and the j-th vehicle detection box, d j is the position of the detection box of the jth vehicle, y i is the position of the detection box of the jth vehicle in the i-th driving trajectory, which represents the prediction result of the vehicle in the next time frame; is the inverse of the covariance between the target detection position and the tracking position; According to the Mahalanobis distance and the preset Mahalanobis distance threshold, if the Mahalanobis distance is less than the Mahalanobis distance threshold, it is determined that the vehicle is successfully associated with the driving trajectory. For the driving trajectory and the vehicle that are successfully associated with the motion, the cosine distance between the driving trajectory and the detection frame of the vehicle that are successfully associated with the motion is obtained according to a preset cosine distance calculation algorithm as the appearance feature, wherein the cosine distance calculation algorithm is: Where, is the cosine distance between the i-th vehicle trajectory with successful appearance association and the detection frame of the j-th vehicle, r i is the ith driving trajectory, is the detection box of the kth vehicle in the ith driving trajectory, r k is the feature vector of the detection box of the kth vehicle, r j is the feature vector of the detection box of the jth vehicle.

5. The video-based road control method according to claim 4, characterized in that: The method of predicting the trajectory of the target image of the next time frame according to the vehicle state space information of the target image and a preset Kalman filter to obtain the driving trajectories of several vehicles of the target image of the next time frame also includes the steps of: Confirming the states of the driving trajectories of the plurality of vehicles in the target image to obtain the states of the driving trajectories of the plurality of vehicles, wherein the states include a confirmed state and a non-confirmed state; If the state of the driving trajectory is a confirmation state, a driving trajectory corresponding to the driving trajectory in the road area image of the previous time frame is cascade matched with a detection frame of the target image of the current time frame to obtain a cascade matching result, wherein the cascade matching result includes a successfully matched driving trajectory, an unmatched driving trajectory, and an unmatched detection frame; If the state of the driving trajectory is unconfirmed, the driving trajectory and the unmatched driving trajectory are respectively matched with the unmatched detection frame to determine whether the unmatched detection frame matches the driving trajectory and the unmatched driving trajectory. If the match is successful, the detection frame is associated with the corresponding driving trajectory.

6. The video-based road control method according to claim 2, characterized in that: The method of performing congestion judgment on a plurality of road sections in a plurality of paths according to the set of vehicle trajectories in the road video sequence to obtain a plurality of congested road sections comprises the steps of: Obtaining the cumulative number of vehicles entering and passing through the plurality of road sections, calculating the vehicle inflow rate according to the cumulative number of vehicles entering the road section, and obtaining the vehicle inflow rate of the plurality of road sections; Counting the passing time of several vehicles passing through the several sections, using the upper quartile method, calculating the general time delay parameter according to the passing time of several vehicles passing through the same section, and obtaining the general delay time of the several sections; The maximum number of vehicles that can be accommodated by the plurality of road sections is obtained, and a joint solution is performed according to the maximum number of vehicles that can be accommodated by the plurality of road sections, the cumulative number of vehicles entering, the vehicle inflow rate, the general time delay parameter, and the preset arrival curve function and service curve function to obtain the maximum delay time of the plurality of road sections, wherein the arrival curve function is: α(t)=σ k +r k t Where α(t) is the arrival curve value, t is the tth time point, σ k is the maximum number of vehicles that the k-th road section can accommodate, ρ k is the vehicle inflow rate of the kth road segment; The service curve function is: β(t)=R k *[(t-T k ) + ] Where β(t) is the service curve value, R k is the cumulative number of vehicles entering the kth road segment, T k is the general delay time of the kth road segment; The lengths of the several road sections are obtained, and the actual travel speeds of the several road sections are obtained according to the lengths of the several road sections and the maximum delay time. The congestion of the several road sections is determined according to the actual travel speeds of the several road sections and a preset average travel speed interval, so as to obtain several congested road sections.

7. The video-based road control method according to claim 3, characterized in that: The monitoring data of the plurality of road regulation demand paths include the maximum number of vehicles that can be accommodated on the paths, the maximum delay time, and the general delay time; The monitoring data of the plurality of road regulation demand paths are input into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, including the steps of: The total vehicle inflow rate of the target area within a preset time period is obtained, and the vehicle inflow ratio setting rates of the plurality of the road control demand paths are obtained according to the total vehicle inflow rate, the monitoring data of the plurality of the road control demand paths, and the vehicle inflow ratio setting rate calculation algorithm in the traffic flow optimization model, wherein the vehicle inflow ratio setting rate calculation algorithm is: In the formula, ρ′ i Set the rate for the vehicle inflow ratio of the i-th road control demand path, R i is the cumulative number of vehicles entering the i-th road control demand path, σ i is the maximum number of vehicles that can be accommodated on the i-th road regulation demand path, △ is the maximum delay time of the i-th road regulation demand path, T i is the general delay time of the ith road regulation demand path, T j is the general delay time of the jth road regulation demand path, ρ all is the total vehicle inflow rate; According to the vehicle inflow ratio setting rate of the several road regulation demand paths and the objective function, queuing delay time constraint condition and vehicle inflow rate constraint condition in the traffic flow optimization model, the queuing delay time is calculated by using the single objective programming model method to obtain the queuing delay time of the several road regulation demand paths, wherein the objective function is: Where M j is the queuing delay time of the jth road control demand path; The queuing delay time constraint is: In the formula, σ j is the maximum number of vehicles that can be accommodated on the j-th road control demand path, R j is the cumulative number of vehicles entering the jth road control demand path, ρ j The vehicle inflow rate of the j-th road demand path; The vehicle inflow rate constraint condition is: Determine the road control demand path with the maximum queuing delay time and the minimum queuing delay time, adjust the vehicle inflow ratio setting rate of the road control demand path with the maximum queuing delay time and the minimum queuing delay time, calculate the queuing delay time again, and determine the final vehicle inflow ratio setting rate of several road control demand paths according to the queuing delay times of several road control demand paths calculated twice adjacently as the vehicle inflow control rate, and construct the traffic flow optimization strategy.

8. A video-based road control device, characterized in that: include: A road video sequence acquisition module is used to obtain a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from a starting point to an end point; the paths correspond to road sections connected by several road section intersections; A vehicle detection module is used to input the road video sequence into a preset target detection model to perform target detection and obtain a vehicle detection result of the road video sequence; Driving trajectory construction, for inputting the vehicle detection results of the road video sequence into a preset multi-target tracking model to perform target tracking and trajectory construction, and obtaining a driving trajectory set of the road video sequence, wherein the driving trajectory set includes driving trajectories of several vehicles; A road control demand judgment module is used to judge the congestion of several sections in several paths according to the driving trajectory set of the road video sequence, and obtain several congested sections; several paths associated with several congested sections are used as road control demand paths, and several road control demand paths and monitoring data of several road control demand paths are obtained; The traffic flow optimization module is used to input the monitoring data of several road control demand paths into a preset traffic flow optimization model, obtain a traffic flow optimization strategy, and perform road control according to the traffic flow optimization strategy.

9. A computer device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the video-based road control method as described in any one of claims 1 to 7 are implemented.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the video-based road control method according to any one of claims 1 to 7 are implemented.

Images