Video-based road regulation method and device, computer device and storage medium

Abstract

The present application relates to the technical field of traffic control, in particular to a road regulation method and device based on video, computer equipment and storage medium, by introducing vehicle detection and multi-target vehicle tracking, the driving trajectory of the road video sequence in the preset time period is constructed, according to the driving trajectory, the congestion of several road sections in several paths in the region is judged, and the traffic flow optimization model is applied, the traffic data is analyzed in real time, the dynamic scheduling of the congested road section can effectively reduce the traffic management cost, improve the management efficiency, alleviate the traffic congestion problem, and promote the sustainable development of urban traffic.

Description

Technical Field

[0001] This invention relates to the field of traffic control technology, and in particular to a video-based road control method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the accelerated pace of urbanization and the significant improvement in residents' living standards, the number of cars, as an important means of transportation in modern life, has increased dramatically. This change has greatly enriched people's travel options and improved the convenience of life, but it has also brought unprecedented pressure and challenges to urban transportation systems. Traffic congestion is becoming increasingly serious, not only severely affecting citizens' travel efficiency and quality of life, but also exacerbating environmental problems such as air and noise pollution, posing a significant threat to the sustainable development of cities.

[0003] Traditionally, traffic management departments rely on manual patrols and surveillance cameras to monitor road vehicles, but this model has significant drawbacks. On the one hand, manual monitoring requires substantial human and material resources, resulting in high costs and low efficiency, making it difficult to cope with increasingly complex traffic conditions. On the other hand, while traditional surveillance cameras provide some visual information, they lack intelligent processing and analysis capabilities, failing to identify vehicles accurately and in real-time, track their trajectories, or effectively predict and respond to emergencies such as traffic congestion. Therefore, developing an efficient and intelligent traffic management system to achieve real-time monitoring, precise tracking, and dynamic optimization of road vehicles is key to solving the current traffic congestion problem. Summary of the Invention

[0004] Based on this, the purpose of this invention is to provide a video-based road control method, device, computer equipment, and storage medium. By introducing vehicle detection and multi-target vehicle tracking, driving trajectories are constructed from road video sequences within a preset time period. Based on the constructed driving trajectories, congestion is determined for several road segments on several paths in the area. A traffic flow optimization model is applied to analyze traffic data in real time and dynamically schedule congested road segments. This can effectively reduce traffic management costs, improve management efficiency, alleviate traffic congestion, and promote the sustainable development of urban transportation.

[0005] In a first aspect, embodiments of this application provide a video-based road control method, comprising the following steps:

[0006] Obtain a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from the start point to the end point; each path corresponds to several road segments connected by intersections.

[0007] The road video sequence is input into a preset target detection model for target detection to obtain the vehicle detection results of the road video sequence;

[0008] The vehicle detection results of the road video sequence are input into a preset multi-target tracking model for target tracking and trajectory construction to obtain a set of driving trajectories of the road video sequence, wherein the set of driving trajectories includes the driving trajectories of several vehicles.

[0009] Based on the set of driving trajectories in the road video sequence, congestion is determined for several road segments in several paths to obtain several congested road segments; several paths associated with several congested road segments are taken as road control demand paths to obtain several road control demand paths and monitoring data of several road control demand paths.

[0010] The monitoring data of several road regulation demand paths are input into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and road regulation is carried out according to the traffic flow optimization strategy.

[0011] Secondly, embodiments of this application provide a video-based road control device, comprising:

[0012] The road video sequence acquisition module is used to obtain a road video sequence of a target area within a preset time period. The road video sequence includes road area images of several time frames, and the road area images include several paths from the start point to the end point. Each path corresponds to several road segments connected by intersections.

[0013] The vehicle detection module is used to input the road video sequence into a preset target detection model for target detection and obtain the vehicle detection results of the road video sequence.

[0014] Driving trajectory construction is used to input the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction, to obtain a set of driving trajectories of the road video sequence, wherein the set of driving trajectories includes the driving trajectories of several vehicles;

[0015] The road regulation demand judgment module is used to determine the congestion of several road segments in several paths based on the driving trajectory set of the road video sequence, and obtain several congested road segments; and to take several paths associated with several congested road segments as road regulation demand paths, and obtain several road regulation demand paths and monitoring data of several road regulation demand paths.

[0016] The traffic flow optimization module is used to input monitoring data of several road regulation demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and to perform road regulation according to the traffic flow optimization strategy.

[0017] Thirdly, embodiments of this application provide a computer device, including: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, it implements the steps of the video-based road control method as described in the first aspect.

[0018] Fourthly, embodiments of this application provide a storage medium storing a computer program that, when executed by a processor, implements the steps of the video-based road control method described in the first aspect.

[0019] This application provides a video-based road control method, apparatus, computer device, and storage medium. By introducing vehicle detection and multi-target vehicle tracking, driving trajectories are constructed from road video sequences within a preset time period. Based on the constructed driving trajectories, congestion is assessed for several road segments along several paths in the area. A traffic flow optimization model is applied to analyze traffic data in real time and dynamically schedule congested road segments. This can effectively reduce traffic management costs, improve management efficiency, alleviate traffic congestion, and promote the sustainable development of urban transportation.

[0020] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0021] Figure 1 A schematic flowchart of a video-based road control method provided in one embodiment of this application;

[0022] Figure 2 A schematic flowchart of S3 in a video-based road control method provided in one embodiment of this application;

[0023] Figure 3 A schematic flowchart of step S33 in a video-based road control method provided in one embodiment of this application;

[0024] Figure 4 A schematic flowchart of step S331 in a video-based road control method provided in one embodiment of this application;

[0025] Figure 5 A flowchart illustrating step S3 in a video-based road control method provided in another embodiment of this application;

[0026] Figure 6A schematic flowchart of step S4 in a video-based road control method provided in one embodiment of this application;

[0027] Figure 7 A schematic flowchart of step S5 in a video-based road control method provided in one embodiment of this application;

[0028] Figure 8 A schematic diagram of the structure of a video-based road control device provided in one embodiment of this application;

[0029] Figure 9 This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0032] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0033] Please see Figure 1 , Figure 1 The following is a flowchart illustrating a video-based road control method according to an embodiment of this application. The method includes the following steps:

[0034] S1: Obtain the road video sequence of the target area within a preset time period.

[0035] The execution entity of the video-based road control method is a video-based road control device (hereinafter referred to as the road control device). In an optional embodiment, the road control device may be a computer device, a server, or a server cluster composed of multiple computer devices.

[0036] In this embodiment, the road control device can obtain a road video sequence of a target area within a preset time period input by the user, or it can obtain a road video sequence of a target area within a preset time period through a preset database. The road video sequence includes road area images of several time frames, and the road area images include several paths from the starting point to the ending point; each path corresponds to several road segments connected by intersections.

[0037] S2: Input the road video sequence into a preset target detection model to perform target detection and obtain the vehicle detection results of the road video sequence.

[0038] The target detection model adopts the YOLOv5s model, which is one of the models in this series. It has the smallest depth and the smallest feature map width, and it also has good accuracy. The YOLOv5 model can directly predict the category and location of the target object through a single forward propagation, which greatly improves the detection speed and can still maintain a high accuracy in complex traffic scenarios, providing a reliable technical guarantee for real-time vehicle detection.

[0039] In this embodiment, the road control device inputs the road video sequence into a preset target detection model. The target detection model, namely the YOLOv5s model, consists of an input module, a backbone module, a feature fusion module, and a prediction module, to perform target detection and obtain the vehicle detection results of the road video sequence. The vehicle detection results of the road video sequence include vehicle detection results of road area images from several time frames, and the vehicle detection results include detection boxes of several vehicles.

[0040] S3: Input the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction to obtain a set of driving trajectories of the road video sequence.

[0041] The multi-object tracking model adopts the DeepSORT (Deep Learning for Real-Time Multi-Object Tracking and Segmentation) model. The DeepSORT model integrates the advantages of deep learning and traditional tracking algorithms, and can achieve stable tracking results in multi-object tracking scenarios. By introducing deep learning feature extraction and Kalman filter prediction, it effectively solves problems such as target occlusion and loss, and provides a more accurate and robust solution for vehicle tracking.

[0042] In this embodiment, the road control device inputs the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction to obtain a set of driving trajectories of the road video sequence, wherein the set of driving trajectories includes the driving trajectories of several vehicles.

[0043] Please see Figure 2 , Figure 2 The flowchart of S3 in the video-based road control method provided in one embodiment of this application includes steps S31 to S34, as follows:

[0044] S31: Take the road area image of the first time frame as the target image and obtain the vehicle state space information of the target image.

[0045] In this embodiment, the road control device uses the road area image of the first time frame as the target image to obtain the vehicle state space information of the target image. The vehicle state space information includes the center position parameters, aspect ratio, and height parameters of the detection boxes of several vehicles.

[0046] S32: Construct the driving trajectories of several vehicles in the target image, and predict the trajectory of the target image in the next time frame based on the vehicle state space information of the target image and a preset Kalman filter to obtain the driving trajectories of several vehicles in the target image in the next time frame.

[0047] In this embodiment, the road control device constructs the driving trajectories of several vehicles in the target image, and predicts the trajectory of the target image in the next time frame based on the vehicle state space information of the target image and a preset Kalman filter, thereby obtaining the driving trajectories of several vehicles in the target image in the next time frame.

[0048] Specifically, the road control equipment constructs an eight-dimensional state space (u, v, γ, h, x) for several vehicles in the target image based on the vehicle state space information of the target image. · ,y · ,γ ·The system defines a coordinate system (x, y, γ, h), where the center position parameter is (u, v), the aspect ratio is γ, the height parameter is h, and (x, y, γ, h) is the rate of change of (u, v, γ, h), i.e., the first derivative of (u, v, γ, h) in the image coordinates. The road control device inputs the eight-dimensional state space of several vehicles in the target image into a Kalman filter to predict the trajectory of the target image in the next time frame, thereby obtaining the driving trajectories of several vehicles in the target image in the next time frame.

[0049] S33: Based on the detection boxes and driving trajectories of several vehicles in the target image of the next time frame, determine whether the detection boxes and driving trajectories match, associate the detection boxes with the corresponding driving trajectories, update the driving trajectories of several vehicles in the target image according to the Kalman filter, and obtain the updated driving trajectories of several vehicles in the target image.

[0050] In this embodiment, the road control device determines whether the detection boxes and driving trajectories of several vehicles in the target image of the next time frame match, associates the detection boxes with the corresponding driving trajectories, and updates the driving trajectories of several vehicles in the target image according to the Kalman filter to obtain the updated driving trajectories of several vehicles in the target image.

[0051] Please see Figure 3 , Figure 3 The flowchart of S33 in the video-based road control method provided in one embodiment of this application includes steps S331 to S332, as follows:

[0052] S331: Based on the detection boxes and driving trajectories of several vehicles in the road area image of the next time frame, feature extraction is performed to obtain feature data between the detection boxes and driving trajectories of several vehicles.

[0053] In this embodiment, the road control device performs feature extraction based on the detection boxes and driving trajectories of several vehicles in the road area image of the next time frame, and obtains feature data between the detection boxes and driving trajectories of several vehicles, wherein the feature data includes motion features and appearance features.

[0054] Please see Figure 4 , Figure 4 The flowchart of S331 in the video-based road control method provided in one embodiment of this application includes steps S3311 to S3312, as follows:

[0055] S3311: Based on the detection boxes of several vehicles, driving trajectories, and a preset Mahalanobis distance calculation algorithm of the road area image of the next time frame, obtain the Mahalanobis distance between several driving trajectories and several vehicle detection boxes, and use it as the motion feature.

[0056] In this embodiment, the road control device obtains the Mahalanobis distance between several vehicle detection frames and several vehicle detection frames based on the detection frames of several vehicles in the road area image of the next time frame, the vehicle trajectories, and a preset Mahalanobis distance calculation algorithm, as the motion feature; wherein, the Mahalanobis distance calculation algorithm is:

[0057]

[0058] In the formula, Let d be the Mahalanobis distance between the detection boxes of the i-th driving trajectory and the j-th vehicle. j Let y be the position of the detection frame for the j-th vehicle. i Let be the position of the detection box of the j-th vehicle in the i-th driving trajectory, so as to represent the prediction result of the vehicle in the next time frame; It is the reciprocal of the covariance between the target detection location and the tracking location.

[0059] S3312: Based on the Mahalanobis distance and a preset Mahalanobis distance threshold, if the Mahalanobis distance is less than the Mahalanobis distance threshold, it is determined that the vehicle and the driving trajectory are successfully associated. For the driving trajectory and the vehicle that are successfully associated, the cosine distance between the detection boxes of the driving trajectory and the vehicle that are successfully associated is obtained according to a preset cosine distance calculation algorithm, and is used as the appearance feature.

[0060] In this embodiment, the road control device determines that the vehicle and the driving trajectory are successfully associated based on the Mahalanobis distance and a preset Mahalanobis distance threshold. If the Mahalanobis distance is less than the Mahalanobis distance threshold, the device calculates the cosine distance between the detection frames of the successfully associated driving trajectory and the vehicle using a preset cosine distance calculation algorithm. This cosine distance is then used as the appearance feature. The cosine distance calculation algorithm is as follows:

[0061]

[0062] In the formula, is the cosine distance between the detection box of the i-th driving trajectory and the j-th vehicle after successful appearance association, and R is... i For the i-th driving trajectory, For the detection box of the k-th vehicle in the i-th driving trajectory, r k Let r be the feature vector of the detection box for the k-th vehicle. j Let be the feature vector of the detection box for the j-th vehicle.

[0063] S332: Construct a cost matrix based on the feature data between the detection boxes of several vehicles and their driving trajectories; determine whether the detection boxes of the vehicles match the driving trajectories based on the cost matrix; if a match is successful, associate the detection boxes with the corresponding driving trajectories to obtain the initial driving trajectories of several vehicles in the target image.

[0064] In this embodiment, the road control device constructs a cost matrix based on feature data between the detection frames and driving trajectories of several vehicles. Specifically, the road control device linearly weights the appearance features and motion features in the feature data to obtain the cost matrix between the detection frames and driving trajectories of several vehicles. The cost matrix includes the metric distance between several driving trajectories and the detection frames of several vehicles, and the metric distance is:

[0065]

[0066] In the formula, c (i,j) Let ω be the metric distance between the detection box of the i-th driving trajectory and the j-th vehicle, and let ω be the weighting coefficient. When the detection box of a vehicle is occluded for a long time, ω = 0.

[0067] The road control equipment determines whether the vehicle's detection box matches the driving trajectory based on the cost matrix. If a match is successful, the detection box is associated with the corresponding driving trajectory to obtain the initial driving trajectories of several vehicles in the target image.

[0068] Specifically, the road control equipment determines that the vehicle's detection frame and the driving trajectory are successfully matched if the measured distance between several driving trajectories and several vehicle detection frames in the cost matrix and a preset measured distance threshold is less than the measured distance threshold; otherwise, the matching fails.

[0069] S34: Using the road area image of the next time frame as the target image, repeatedly perform trajectory prediction, matching, and update operations until the updated driving trajectories of several vehicles in the road area image of the last time frame are obtained, and use them as the driving trajectories to construct the driving trajectory set of the road video sequence.

[0070] In this embodiment, the road control device uses the road area image of the next time frame as the target image and repeatedly performs trajectory prediction, matching and update operations until it obtains the updated driving trajectories of several vehicles in the road area image of the last time frame, which are then used as the driving trajectories to construct the driving trajectory set of the road video sequence.

[0071] Please see Figure 5 , Figure 5A flowchart illustrating step S3 of a video-based road control method provided in another embodiment of this application includes steps S35 to S37, wherein steps S35 to S37 are after step S32 and before S33, as follows:

[0072] S35: Confirm the status of the driving trajectories of several vehicles in the target image to obtain the status of the driving trajectories of several vehicles.

[0073] When the target to be detected is occluded for a long time, the accuracy of the Kalman filter algorithm is low and it is easy to encounter the problem of matching trajectory errors. In this embodiment, the road control device uses the Kalman filter to confirm the status of the driving trajectory of several vehicles in the target image and obtain the status of the driving trajectory of several vehicles. The status includes confirmed status and unconfirmed status.

[0074] S36: If the state of the driving trajectory is confirmed, the driving trajectory in the road area image of the previous time frame is cascaded and matched with the detection box of the target image in the current time frame to obtain the cascaded matching result.

[0075] If the driving trajectory is in a confirmed state, in this embodiment, the road control device performs a cascaded matching of the driving trajectory corresponding to the road area image in the previous time frame with the detection box of the target image in the current time frame to obtain a cascaded matching result. The cascaded matching result includes successfully matched driving trajectories, unmatched driving trajectories, and unmatched detection boxes.

[0076] S37: If the driving trajectory is in an unconfirmed state, the driving trajectory and the unmatched driving trajectory are matched with the unmatched detection box respectively. It is determined whether the unmatched detection box matches the driving trajectory and the unmatched driving trajectory. If the match is successful, the detection box is associated with the corresponding driving trajectory.

[0077] If the driving trajectory is in an unconfirmed state, in this embodiment, the road control device will match the driving trajectory and the unmatched driving trajectory with the unmatched detection box respectively, and determine whether the unmatched detection box matches the driving trajectory and the unmatched driving trajectory. If the match is successful, the detection box will be associated with the corresponding driving trajectory.

[0078] S4: Based on the set of driving trajectories in the road video sequence, determine the congestion of several road segments in several paths to obtain several congested road segments; take several paths associated with several congested road segments as road control demand paths to obtain several road control demand paths and monitoring data of several road control demand paths.

[0079] In this embodiment, the road control device determines the congestion of several road segments in several paths based on the set of driving trajectories in the road video sequence, and obtains several congested road segments.

[0080] The road control equipment uses several paths associated with several congested road sections as road control demand paths, and obtains several road control demand paths and several monitoring data of the road control demand paths. The monitoring data of the road control demand paths includes the maximum number of vehicles that the path can accommodate, the maximum delay time, and the general delay time.

[0081] Please see Figure 6 , Figure 6 The flowchart of S4 in the video-based road control method provided in one embodiment of this application includes steps S41 to S44, as follows:

[0082] S41: Obtain the cumulative number of vehicles entering and passing through several road segments, calculate the vehicle inflow rate based on the cumulative number of vehicles entering the road segments, and obtain the vehicle inflow rate of several road segments.

[0083] In this embodiment, the road control device obtains the cumulative number of vehicles entering and passing through several road segments, and calculates the vehicle inflow rate based on the cumulative number of vehicles entering each road segment to obtain the vehicle inflow rate of several road segments. Specifically, the road control device divides the cumulative number of vehicles entering each road segment within the preset time period by the time corresponding to the time period to obtain the vehicle inflow rate of several road segments.

[0084] S42: Statistically calculate the passage time of several vehicles passing through several road segments, and use the upper quarter point method to calculate the general time delay parameter based on the passage time of several vehicles passing through the same road segment to obtain the general delay time of several road segments.

[0085] In this embodiment, the road control equipment counts the passage time of several vehicles passing through several road segments, and uses the upper quarter point method to calculate the general time delay parameter based on the passage time of several vehicles passing through the same road segment to obtain the general delay time of several road segments.

[0086] S43: Obtain the maximum number of vehicles that can be accommodated for several road segments, and perform a joint solution based on the maximum number of vehicles that can be accommodated for several road segments, the cumulative number of vehicles entering, the vehicle inflow rate, the general time delay parameter, and the preset arrival curve function and service curve function to obtain the maximum delay time of several road segments.

[0087] In this embodiment, the road control equipment obtains the maximum number of vehicles that several road segments can accommodate. Based on the maximum number of vehicles that several road segments can accommodate, the cumulative number of entering vehicles, the vehicle inflow rate, the general time delay parameter, and a preset arrival curve function and service curve function, it performs a joint solution to obtain the maximum delay time of several road segments. The arrival curve function is:

[0088] α(t)=σ k +ρ k t

[0089] In the formula, α(t) is the arrival curve value, t is the t-th time point, and σ k Let ρ be the maximum number of vehicles that the k-th road segment can accommodate. k Let be the vehicle inflow rate of the k-th road segment;

[0090] The service curve function is:

[0091] β(t)=R k *[(tT k ) + ]

[0092] In the formula, β(t) is the service curve value, and R k T represents the cumulative number of vehicles entering the k-th road segment. k Let be the general delay time for the k-th road segment.

[0093] Specifically, when the road control equipment obtains the arrival curve value α(t) equal to the service curve value β(t), the time point t on the arrival curve function is... α and the time point t on the service curve function β Solve for max(t) β -t α The solution result is used as the maximum delay time to obtain the maximum delay time of several road segments.

[0094] S44: Obtain the length of several road segments, obtain the actual travel speed of several road segments based on the length of several road segments and the maximum delay time, and determine the congestion of several road segments based on the actual travel speed of several road segments and the preset average travel speed range to obtain several congested road segments.

[0095] In this embodiment, the road control device obtains the lengths of several road segments, and based on the lengths of the several road segments and the maximum delay time, obtains the actual travel speeds of the several road segments, wherein the actual travel speeds are:

[0096]

[0097] In the formula, V j S is the actual travel speed of the j-th road segment. j Let T be the length of the j-th road segment. △j Let be the maximum delay time for the j-th road segment.

[0098] The road control equipment determines the congestion of several road segments based on the actual travel speed of the road segments and the preset average travel speed range. If the actual travel speed is within the travel speed range, the road segment is determined to be a congested road segment, and several congested road segments are obtained.

[0099] S5: Input the monitoring data of several road regulation demand paths into the preset traffic flow optimization model to obtain the traffic flow optimization strategy, and perform road regulation according to the traffic flow optimization strategy.

[0100] In this embodiment, the road control device inputs monitoring data of several road control demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and performs road control according to the traffic flow optimization strategy.

[0101] Please see Figure 7 , Figure 7 The flowchart of S5 in the video-based road control method provided in one embodiment of this application includes steps S51 to S53, as follows:

[0102] S51: Obtain the total vehicle inflow rate of the target area within a preset time period, and obtain the vehicle inflow ratio setting rate of the several road control demand paths based on the total vehicle inflow rate, the monitoring data of several road control demand paths, and the vehicle inflow ratio setting rate calculation algorithm in the traffic flow optimization model.

[0103] In this embodiment, the road control equipment obtains the total vehicle inflow rate of the target area within a preset time period. The total vehicle inflow rate can be obtained by detecting the cumulative number of vehicles entering the target area and dividing the cumulative number of vehicles by the time period. Based on the total vehicle inflow rate, monitoring data of several road control demand paths, and the vehicle inflow ratio setting rate calculation algorithm in the traffic flow optimization model, the road control equipment obtains the vehicle inflow ratio setting rate for several road control demand paths. The vehicle inflow ratio setting rate calculation algorithm is as follows:

[0104]

[0105] In the formula, ρ′ i A rate, R, is set for the proportion of vehicles flowing into the i-th road's demand path.i Let σ be the cumulative number of vehicles entering the i-th road regulation demand path. i Let T be the maximum number of vehicles that can be accommodated for the i-th road control demand path, and Δ be the maximum delay time for the i-th road control demand path. The maximum delay time for the road control demand path can be pre-input by the user into the road control equipment. i Let T be the general delay time of the i-th road regulation demand path. j Let ρ be the general delay time of the j-th road regulation demand path. all This represents the total vehicle inflow rate.

[0106] S52: Based on the vehicle inflow ratio setting rate of several road control demand paths and the objective function, queuing delay time constraint, and vehicle inflow rate constraint in the traffic flow optimization model, the queuing delay time is calculated using a single-objective programming model method to obtain the queuing delay time of several road control demand paths.

[0107] The objective function is:

[0108]

[0109] In the formula, M j Let be the queuing delay time for the j-th road regulation demand path.

[0110] The queuing delay time constraint is as follows:

[0111]

[0112] In the formula, σ j R represents the maximum number of vehicles that can be accommodated for the j-th road traffic control path. j Let ρ be the cumulative number of vehicles entering the j-th road regulation demand path. j Let be the vehicle inflow rate of the j-th road regulation demand path.

[0113] The vehicle inflow rate constraint is as follows:

[0114]

[0115] In this embodiment, the road control device sets the rate based on the vehicle inflow ratio of several road control demand paths and the objective function, queuing delay time constraint, and vehicle inflow rate constraint in the traffic flow optimization model, and uses a single-objective programming model method to calculate the queuing delay time to obtain the queuing delay time of several road control demand paths.

[0116] S53: Determine the road control demand paths with the maximum and minimum queuing delay times, adjust the vehicle inflow ratio setting rate of the road control demand paths with the maximum and minimum queuing delay times, recalculate the queuing delay time, and determine the final vehicle inflow ratio setting rate of the road control demand paths based on the queuing delay times of several road control demand paths calculated in two adjacent calculations, as the vehicle inflow control rate, and construct the traffic flow optimization strategy.

[0117] In this embodiment, the road control equipment determines the road control demand paths with the maximum and minimum queuing delay times, adjusts the vehicle inflow ratio setting rate for the road control demand paths with the maximum and minimum queuing delay times, recalculates the queuing delay time, and determines the final vehicle inflow ratio setting rate for several road control demand paths based on the queuing delay times of several adjacent road control demand paths, which is used as the vehicle inflow control rate to construct the traffic flow optimization strategy.

[0118] Specifically, the road control equipment reduces the vehicle inflow ratio setting rate of the road control demand path with the maximum queuing delay time and increases the vehicle inflow ratio setting rate of the road control demand path with the minimum queuing delay time, so as to adjust the vehicle inflow ratio setting rate of the road control demand paths with the maximum queuing delay time and the minimum queuing delay time.

[0119] The road control equipment calculates the queuing delay time based on the adjusted vehicle inflow ratio setting rate of the road control demand path, the vehicle inflow ratio setting rate of other road control demand paths, and the objective function, queuing delay time constraint, and vehicle inflow rate constraint in the traffic flow optimization model. This yields the queuing delay time for several road control demand paths to be calculated in the next iteration. If the maximum queuing delay time of the road control demand path in the next calculation is consistent with the maximum queuing delay time calculated in the previous calculation, the vehicle inflow ratio setting rate of the several road control demand paths in the next calculation is taken as the final vehicle inflow ratio setting rate. If the maximum queuing delay time of the road control demand path in the next calculation is consistent with... If the maximum queuing delay time calculated in the previous calculation is inconsistent, the road control equipment repeatedly determines the road control demand path with the maximum and minimum queuing delay times, performs vehicle inflow ratio setting rate adjustment and queuing delay time calculation operations, until the maximum queuing delay time of the road control demand path calculated in the next calculation is consistent with the maximum queuing delay time calculated in the previous calculation. This obtains the vehicle inflow control rate for several road control demand paths, constructs the traffic flow optimization strategy, and analyzes traffic data in real time. Dynamic scheduling is performed to reduce the difference between the average delay and the maximum delay of each road segment, thus solving the problem of alleviating traffic congestion and effectively reducing traffic management costs, improving management efficiency, and promoting the sustainable development of urban transportation.

[0120] Please refer to Figure 8 , Figure 8 This is a schematic diagram of a video-based road control device according to an embodiment of this application. The device can be implemented entirely or partially through software, hardware, or a combination of both. The device 8 includes:

[0121] The road video sequence acquisition module 81 is used to acquire a road video sequence of a target area within a preset time period. The road video sequence includes road area images of several time frames, and the road area images include several paths from the starting point to the ending point. Each path corresponds to several road segments connected by intersections.

[0122] The vehicle detection module 82 is used to input the road video sequence into a preset target detection model for target detection and obtain the vehicle detection result of the road video sequence;

[0123] The vehicle trajectory construction 83 is used to input the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction, so as to obtain a set of vehicle trajectories of the road video sequence, wherein the set of vehicle trajectories includes the vehicle trajectories of several vehicles.

[0124] The road regulation demand judgment module 84 is used to judge the congestion of several road segments in several paths based on the driving trajectory set of the road video sequence, and obtain several congested road segments; and to take several paths associated with several congested road segments as road regulation demand paths, and obtain several road regulation demand paths and monitoring data of several road regulation demand paths.

[0125] The traffic flow optimization module 85 is used to input the monitoring data of several road regulation demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and to perform road regulation according to the traffic flow optimization strategy.

[0126] In this embodiment, a road video sequence acquisition module obtains a road video sequence of a target area within a preset time period. The road video sequence includes road area images of several time frames, and each road area image includes several paths from a starting point to an end point. Each path corresponds to several road segments connected by intersections. A vehicle detection module inputs the road video sequence into a preset target detection model for target detection, obtaining vehicle detection results for the road video sequence. A driving trajectory construction module inputs the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction, obtaining the driving trajectory of the road video sequence. The system employs a trajectory set, comprising the trajectories of several vehicles. A road control demand judgment module, based on the trajectories of the road video sequence, determines congestion on several road segments within several paths, identifying several congested road segments. Several paths associated with these congested road segments are then used as road control demand paths, yielding several road control demand paths and their monitoring data. A traffic flow optimization module inputs the monitoring data of these road control demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy. Road control is then implemented based on this strategy. By introducing vehicle detection and multi-target vehicle tracking to construct trajectories from road video sequences within a preset time period, and by determining congestion on several road segments within several paths in the region based on the constructed trajectories, and by applying a traffic flow optimization model to analyze traffic data in real time and dynamically schedule congested road segments, the system can effectively reduce traffic management costs, improve management efficiency, alleviate traffic congestion, and promote the sustainable development of urban transportation.

[0127] Please refer to Figure 9 , Figure 9This is a schematic diagram of the structure of a computer device provided in one embodiment of this application. The computer device 9 includes: a processor 91, a memory 92, and a computer program 83 stored in the memory 92 and executable on the processor 91; the computer device can store multiple instructions, which are adapted to be loaded and executed by the processor 91. Figures 1 to 6 The method steps shown can be found in the following document for detailed execution process. Figures 1 to 6 The specific details shown will not be repeated here.

[0128] The processor 91 may include one or more processing cores. The processor 91 connects to various parts of the server using various interfaces and lines, and executes various functions and processes data of the video-based road control device 8 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 92, and by calling data stored in the memory 92. Optionally, the processor 91 may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor 91 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required to be displayed on the touch screen; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor 91 and may be implemented as a separate chip.

[0129] The memory 92 may include random access memory (RAM) or read-only memory. Optionally, the memory 92 may include a non-transitory computer-readable storage medium. The memory 92 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 92 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions), instructions for implementing the various method embodiments described above, etc.; the data storage area may store data involved in the various method embodiments described above, etc. Optionally, the memory 92 may also be at least one storage device located remotely from the aforementioned processor 91.

[0130] This application embodiment also provides a storage medium that can store multiple instructions, which are adapted to be loaded and executed by a processor as described above. Figures 1 to 6 The method steps shown can be found in the following document for detailed execution process. Figures 1 to 6 The specific details shown will not be repeated here.

[0131] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0132] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0133] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the algorithm. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0134] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / terminal devices and methods can be implemented in other ways. For example, the apparatus / terminal device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0135] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0136] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0137] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms.

[0138] This invention is not limited to the above-described embodiments. If any modifications or variations to this invention do not depart from the spirit and scope of this invention, and if such modifications and variations fall within the scope of the claims and equivalent technologies of this invention, then this invention also intends to include such modifications and variations.

Claims

1. A video-based road control method, characterized in that, Includes the following steps: Obtain a road video sequence of a target area within a preset time period, wherein the road video sequence includes road area images of several time frames, and the road area images include several paths from the start point to the end point; each path corresponds to several road segments connected by intersections. The road video sequence is input into a preset target detection model for target detection to obtain the vehicle detection results of the road video sequence; The vehicle detection results of the road video sequence are input into a preset multi-target tracking model for target tracking and trajectory construction to obtain a set of driving trajectories of the road video sequence, wherein the set of driving trajectories includes the driving trajectories of several vehicles. Based on the set of driving trajectories of the road video sequence, the cumulative number of vehicles entering and passing through several road segments is obtained. Based on the cumulative number of vehicles entering the road segments, the vehicle inflow rate is calculated to obtain the vehicle inflow rate of several road segments. The transit times of several vehicles on several road segments are statistically analyzed. Using the upper quartile point method, a general time delay parameter is calculated based on the transit times of several vehicles on the same road segment to obtain the general delay time of several road segments. The maximum number of vehicles that can be accommodated for several road segments is obtained. Based on the maximum number of vehicles that can be accommodated for several road segments, the cumulative number of entering vehicles, the vehicle inflow rate, the general time delay parameter, and a preset arrival curve function and service curve function, a joint solution is performed to obtain the maximum delay time for several road segments. The arrival curve function is: In the formula, To reach the curve value, For the first t At a certain point in time, For the first k The maximum number of vehicles that each road section can accommodate. For the first k The vehicle inflow rate of each road segment; The service curve function is: In the formula, For service curve values, For the first k The cumulative number of vehicles entering each road segment. For the first k The average delay time for each road segment; The lengths of several road segments are obtained. Based on the lengths of several road segments and the maximum delay time, the actual travel speeds of several road segments are obtained. Based on the actual travel speeds of several road segments and a preset average travel speed range, congestion is determined for several road segments to obtain several congested road segments. Several paths associated with several congested road sections are taken as road regulation demand paths, and several road regulation demand paths and several monitoring data of the road regulation demand paths are obtained. The monitoring data of several road regulation demand paths are input into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and road regulation is carried out according to the traffic flow optimization strategy.

2. The video-based road control method according to claim 1, characterized in that: The vehicle detection results of the road video sequence include vehicle detection results of road area images of several time frames, and the vehicle detection results include detection boxes of several vehicles. The step of inputting the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction to obtain a set of driving trajectories for the road video sequence includes the following steps: The road area image of the first time frame is used as the target image to obtain the vehicle state space information of the target image. The vehicle state space information includes the center position parameters, aspect ratio and height parameters of the detection boxes of several vehicles. Construct the driving trajectories of several vehicles in the target image, and predict the trajectory of the target image in the next time frame based on the vehicle state space information of the target image and a preset Kalman filter to obtain the driving trajectories of several vehicles in the target image in the next time frame. Based on the detection boxes and driving trajectories of several vehicles in the target image of the next time frame, it is determined whether the detection boxes and driving trajectories match, the detection boxes are associated with the corresponding driving trajectories, and the driving trajectories of several vehicles in the target image are updated according to the Kalman filter to obtain the updated driving trajectories of several vehicles in the target image. Using the road area image of the next time frame as the target image, the trajectory prediction, matching, and update operations are repeatedly performed until the updated driving trajectories of several vehicles in the road area image of the last time frame are obtained, which are then used as the driving trajectories to construct the driving trajectory set of the road video sequence.

3. The video-based road control method according to claim 2, characterized in that, The step of determining whether the detection boxes and driving trajectories of several vehicles in the target image of the next time frame match includes the following steps: Feature extraction is performed based on the detection boxes and driving trajectories of several vehicles in the road area image of the next time frame to obtain feature data between the detection boxes and driving trajectories of several vehicles. A cost matrix is ​​constructed based on the feature data between the detection boxes of several vehicles and their driving trajectories. The cost matrix is ​​used to determine whether the detection boxes of the vehicles match the driving trajectories. If a match is successful, the detection boxes are associated with the corresponding driving trajectories to obtain the initial driving trajectories of several vehicles in the target image.

4. The video-based road control method according to claim 3, characterized in that: The feature data includes motion features and appearance features; The step of extracting features from the detection boxes and trajectories of several vehicles in the road area image of the next time frame to obtain feature data between the detection boxes and trajectories of several vehicles includes the following steps: Based on the detection boxes and driving trajectories of several vehicles in the road area image of the next time frame, and a preset Mahalanobis distance calculation algorithm, the Mahalanobis distance between several driving trajectories and several vehicle detection boxes is obtained as the motion feature; wherein, the Mahalanobis distance calculation algorithm is: In the formula, For the first i driving trajectory and the first j The Mahalanobis distance between the detection frames of each vehicle For the first j The location of the detection box for each vehicle. For the first i The first driving trajectory j The position of the detection box for each vehicle is used to indicate the prediction result for the vehicle in the next time frame; It is the reciprocal of the covariance between the target detection location and the tracking location; Based on the Mahalanobis distance and a preset Mahalanobis distance threshold, if the Mahalanobis distance is less than the Mahalanobis distance threshold, it is determined that the vehicle and the driving trajectory are successfully associated. For the driving trajectory and vehicle with successfully associated motion, the cosine distance between the detection boxes of the successfully associated driving trajectory and the vehicle is obtained according to a preset cosine distance calculation algorithm, and is used as the appearance feature. The cosine distance calculation algorithm is as follows: In the formula, The first one that successfully associated with appearance i The driving trajectory and the first j The cosine distance between the detection frames of each vehicle. For the first i Driving trajectory, For the first i The first driving trajectory k The detection frame for each vehicle. For the first k The feature vector of the detection box for each vehicle. For the first j The feature vector of the detection box for each vehicle.

5. The video-based road control method according to claim 4, characterized in that: The step of predicting the trajectory of the target image in the next time frame based on the vehicle state space information of the target image and a preset Kalman filter to obtain the driving trajectories of several vehicles in the target image in the next time frame further includes the following steps: The status of the driving trajectories of several vehicles in the target image is confirmed to obtain the status of the driving trajectories of several vehicles, wherein the status includes a confirmed state and an unconfirmed state. If the driving trajectory is in a confirmed state, the driving trajectory corresponding to the road area image in the previous time frame is cascaded and matched with the detection box of the target image in the current time frame to obtain a cascaded matching result. The cascaded matching result includes successfully matched driving trajectories, unmatched driving trajectories, and unmatched detection boxes. If the driving trajectory is in an unconfirmed state, the driving trajectory and the unmatched driving trajectory are matched with the unmatched detection boxes respectively. It is determined whether the unmatched detection box matches the driving trajectory and the unmatched driving trajectory. If the match is successful, the detection box is associated with the corresponding driving trajectory.

6. The video-based road control method according to claim 3, characterized in that: The monitoring data for the road regulation demand paths mentioned in the several clauses include the maximum number of vehicles that the path can accommodate, the maximum delay time, and the general delay time. The step of inputting monitoring data from several road regulation demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy includes the following steps: The total vehicle inflow rate of the target area within a preset time period is obtained. Based on the total vehicle inflow rate, monitoring data of several road control demand paths, and the vehicle inflow ratio setting rate calculation algorithm in the traffic flow optimization model, the vehicle inflow ratio setting rate of several road control demand paths is obtained. The vehicle inflow ratio setting rate calculation algorithm is as follows: In the formula, For the first i The rate is set based on the proportion of vehicles flowing into each road's demand path. For the first i The cumulative number of vehicles entering each road traffic control demand path. For the first i The maximum number of vehicles that can be accommodated for each road traffic control demand path. For the first i The maximum delay time of each road regulation demand path For the first i The general delay time of demand path for road regulation. For the first j The general delay time of demand path for road regulation. This represents the total vehicle inflow rate. Based on the vehicle inflow ratio setting rate for several road control demand paths and the objective function, queuing delay time constraint, and vehicle inflow rate constraint in the traffic flow optimization model, a single-objective programming model method is used to calculate the queuing delay time for several road control demand paths. The objective function is: In the formula, For the first j Queuing delay time for each road traffic control demand path; The queuing delay time constraint is as follows: In the formula, For the first j The maximum number of vehicles that can be accommodated for each road traffic control demand path. For the first j The cumulative number of vehicles entering each road traffic control demand path. For the first j Vehicle inflow rate of each road control demand path; The vehicle inflow rate constraint is as follows: The maximum and minimum queuing delay times of road control demand paths are determined. The vehicle inflow ratio setting rate of the road control demand paths with the maximum and minimum queuing delay times is adjusted. The queuing delay time is recalculated. Based on the queuing delay times of several road control demand paths calculated in two consecutive calculations, the final vehicle inflow ratio setting rate of several road control demand paths is determined as the vehicle inflow control rate, and the traffic flow optimization strategy is constructed.

7. A video-based road control device, characterized in that, include: The road video sequence acquisition module is used to obtain a road video sequence of a target area within a preset time period. The road video sequence includes road area images of several time frames, and the road area images include several paths from the start point to the end point. Each path corresponds to several road segments connected by intersections. The vehicle detection module is used to input the road video sequence into a preset target detection model for target detection and obtain the vehicle detection results of the road video sequence. Driving trajectory construction is used to input the vehicle detection results of the road video sequence into a preset multi-target tracking model for target tracking and trajectory construction, to obtain a set of driving trajectories of the road video sequence, wherein the set of driving trajectories includes the driving trajectories of several vehicles; The road regulation demand judgment module is used to obtain the cumulative number of vehicles entering and passing through several road segments based on the driving trajectory set of the road video sequence, and to calculate the vehicle inflow rate based on the cumulative number of vehicles entering the road segments to obtain the vehicle inflow rate of several road segments. The transit times of several vehicles on several road segments are statistically analyzed. Using the upper quartile point method, a general time delay parameter is calculated based on the transit times of several vehicles on the same road segment to obtain the general delay time of several road segments. The maximum number of vehicles that can be accommodated for several road segments is obtained. Based on the maximum number of vehicles that can be accommodated for several road segments, the cumulative number of entering vehicles, the vehicle inflow rate, the general time delay parameter, and a preset arrival curve function and service curve function, a joint solution is performed to obtain the maximum delay time for several road segments. The arrival curve function is: In the formula, To reach the curve value, For the first t At a certain point in time, For the first k The maximum number of vehicles that each road section can accommodate. For the first k The vehicle inflow rate of each road segment; The service curve function is: In the formula, For service curve values, For the first k The cumulative number of vehicles entering each road segment. For the first k The average delay time for each road segment; The lengths of several road segments are obtained. Based on the lengths of several road segments and the maximum delay time, the actual travel speeds of several road segments are obtained. Based on the actual travel speeds of several road segments and a preset average travel speed range, congestion is determined for several road segments to obtain several congested road segments. Several paths associated with several congested road sections are taken as road regulation demand paths, and several road regulation demand paths and several monitoring data of the road regulation demand paths are obtained. The traffic flow optimization module is used to input monitoring data of several road regulation demand paths into a preset traffic flow optimization model to obtain a traffic flow optimization strategy, and to perform road regulation according to the traffic flow optimization strategy.

8. A computer device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor; the computer program, when executed by the processor, implements the steps of the video-based road control method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a computer program that, when executed by a processor, implements the steps of the video-based road control method as described in any one of claims 1 to 6.

Images