Traffic jam detection and early warning method based on unmanned aerial vehicle video stream

Abstract

The invention discloses a traffic jam detection and early warning method based on an unmanned aerial vehicle video stream, and belongs to the technical field of intelligent traffic. The system is combined with a floating vehicle triggering model, collects road videos in real time through an unmanned aerial vehicle, and realizes high-precision vehicle detection and tracking in combination with improved YOLOv8 and DeepSort algorithms; unmanned aerial vehicle data and floating vehicle GPS data are fused, three elements of traffic flow are quantified, and a dual-mechanism congestion detection model is constructed; respectively utilizing a GRU cooperative traffic wave theory to predict the congestion dissipation duration and utilizing an improved GRU + GCN model to predict the traffic situation; the signal lamps are dynamically regulated and controlled through multi-agent reinforcement learning, and active congestion relieving is achieved. The method has the advantages of wide-area coverage, high-precision perception and intelligent response, and the urban traffic control efficiency is remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of intelligent transportation and relates to a traffic congestion detection and early warning method based on UAV video stream, which integrates target detection, localization and tracking in UAV aerial video stream, traffic congestion event detection, congestion dissipation duration prediction, traffic situation prediction and multi-signal light control. Background Technology

[0002] With the acceleration of urbanization and the continuous increase in the number of motor vehicles in my country, urban traffic congestion has evolved from a localized problem to a systemic "urban disease." Currently, mainstream static sensor or floating car GPS data for congestion detection suffers from the following problems: it can only provide macroscopic road segment-level congestion location, failing to acquire more evolutionary data; in low-traffic areas, the detection effect is poor due to sparse samples. Limited by the insufficient precision of monitoring methods, existing approaches struggle to comprehensively and accurately grasp the congestion evolution of the entire road network. Furthermore, due to the single data source and low update frequency, traditional technologies often fail to promptly detect early signs of congestion and effectively predict and intervene, thus hindering the efficiency of traffic management. Summary of the Invention

[0003] This invention proposes a traffic congestion detection and early warning method based on UAV video streams, aiming to address the core problems of existing traffic perception technologies in complex traffic scenarios, such as numerous coverage blind spots, delayed response, unclear causes, inaccurate predictions, and lack of service loops. By constructing an integrated architecture of "air-ground dynamic perception - multimodal fusion analysis - collaborative guidance," this invention innovatively integrates a lightweight visual model, dual-mechanism congestion detection, and a physical-data hybrid prediction model that fuses air and ground data. This enables accurate detection, cause identification, dissipation time prediction, and proactive traffic management of traffic congestion events, providing a high-precision, high-timeliness, and highly interpretable intelligent solution for urban traffic governance.

[0004] In response to traffic congestion events, drones, with their dynamic, maneuverable, and wide-coverage characteristics, can provide real-time, dynamic aerial imagery supplemented by floating car data. This overcomes the limitations of macroscopic positioning, enabling a detailed depiction of the causes and evolution of congestion, improving overall coverage, enhancing monitoring accuracy and timeliness, and supporting proactive prediction and intervention. In conclusion, the application of drones in traffic congestion event detection, prediction, and optimization not only improves the comprehensiveness and real-time nature of data collection but also provides a more intelligent, dynamic, and proactive solution for traffic management.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a traffic congestion detection and early warning method based on UAV video stream, comprising the following steps:

[0006] Step 1: Determine whether to deploy drones based on the floating car-triggered drone detection algorithm model;

[0007] Step 2: Control the drone or use waypoint flight to make the drone hover or fly at a constant speed above the road, and obtain parameters such as altitude, gimbal tilt angle, cruise speed, and position information through the ground station, and send these parameters back to the server;

[0008] Step 3: Push the video stream from the ground station to the server using the RTMP protocol. The server reads the RTMP address to obtain the video stream.

[0009] Step 4: Detect vehicles using the improved YOLOv8 algorithm on the video stream;

[0010] Step 5: Use the DeepSort algorithm to perform vehicle tracking on the video stream based on YOLOv8 algorithm detection;

[0011] Step 6: Calculate the road length based on the drone's basic parameters and quantify the three elements of traffic volume;

[0012] Step 7: Use a database to record the data corresponding to each time point and spatial location;

[0013] 7.1: Record the current drone parameters, road network ID, and traffic flow data in database 1;

[0014] 7.2: Receive de-identified floating car GPS trajectory data from the cooperating platform, including timestamps, latitude and longitude, and instantaneous speed; associate trajectory points with specific road segments in the road network topology through map matching, and aggregate to generate road segment-level macro indicators, which are then stored in database 2;

[0015] Step 8: Perform dual-mechanism traffic congestion detection on the obtained data: the road segment accumulation change congestion detection algorithm determines activation, and the fuzzy logic congestion detection algorithm determines the level of congestion.

[0016] 8.1: Delineate upstream and downstream virtual detection lines in the image coordinate system; use DeepSort data stored in the database to count the number of vehicles crossing the upstream section per unit time, and make judgments based on changes in road segment accumulation;

[0017] 8.2: Using fuzzy logic, a multi-indicator fusion congestion level determination model is constructed. The three traffic flow parameters are selected as input indicators. For each indicator, a trapezoidal membership function is designed to describe its membership degree to each congestion level.

[0018] Step 9: Using database data as input, the algorithm GRU neural network is used as the core to predict the three elements of traffic flow, and traffic congestion dissipation time is predicted by combining traffic wave theory.

[0019] Step 10: Using database data as input, predict traffic conditions based on the improved GRU+GCN. GCN is responsible for processing road topology information, and GRU is responsible for obtaining the time dependency of traffic data.

[0020] Step 11: Based on the congestion detection and traffic situation prediction in the previous steps, use multi-agent reinforcement learning to control multiple traffic lights and coordinate to alleviate traffic congestion events.

[0021] Preferably, the floating car-triggered drone detection algorithm model in step 1 is defined as follows: the congestion pheromone concentration of road segment i at time t is... ,

[0022]

[0023] in, This represents the residual pheromone content from the previous moment; The natural decay coefficient is in the range of [0,1], simulating the natural dissipation of congestion; Normalized speed ratio , Let k be the real-time speed of the kth floating car on this road segment; This represents the free-flow velocity of that road section; For the excitation function, the bipolar hyperbolic tangent function

[0024]

[0025] in, The optimal velocity threshold, i.e., the phase transition point; This is the scaling factor;

[0026] Simultaneously construct a utility function To measure the current concentration of congestion pheromones Do we need to take off?

[0027]

[0028] in The Sigmoid saturation function normalizes the pheromone concentration to the (0,1) interval; This represents the Euclidean distance from the drone's current location to the target location; This represents the current percentage of the drone's battery power. Value reward coefficient; This represents the cost penalty coefficient.

[0029] Preferably, the algorithm improvements for UAV video stream detection in step 4, specifically for YOLOv8, include: reducing the depth multiplication factor from 1.0 to 0.75 to reduce the number of repeated stackings of C2f modules in each stage; reducing the width multiplication factor from 1.25 to 0.75 to reduce the number of channels in each convolutional layer; embedding CoordAtt modules after the P2, P3, and P4 feature layers in the backbone; and adopting a lightweight BiFPN approach in the head section: adding a bottom-up GhostConv + BiFPN_Add2 path after the traditional FPN upsampling path to achieve bidirectional cross-scale information interaction; replacing some standard convolutions with GhostConv to further reduce computational load with almost no loss of accuracy; retaining SPPF to expand the receptive field while avoiding heavy modules such as SPPC and SPC, thus balancing context modeling and inference speed.

[0030] Preferably, in step 8, the dual-mechanism traffic congestion detection algorithm, specifically the congestion detection algorithm based on changes in road segment accumulation, involves defining upstream and downstream virtual detection lines in a coordinate system; and using data stored in the database by DeepSort to count the number of vehicles crossing the upstream section per unit time. , , as well as If the current floating car sample density of the road segment If the data exceeds a preset threshold, a weighted fusion strategy is used to correct the traffic; otherwise, pure drone data is used.

[0031]

[0032]

[0033] Among them, weight Adjustments are made based on the proportion of floating car sample density exceeding a threshold, for pure drone data. =1,

[0034] The specific judgment conditions are as follows:

[0035]

[0036] Where Croad is the maximum number of vehicles that the road segment can accommodate, and A(t) is the cumulative number of vehicles accumulated up to time t; To ensure effective continuous observation duration; when all these conditions are met, it is judged as congestion; combining the ratio of cumulative accumulation to road capacity and average vehicle speed, the congestion level is further classified;

[0037] In step 8.2, the specific algorithm process for traffic congestion judgment based on fuzzy logic is as follows: A multi-indicator fusion congestion level determination model is constructed; three traffic flow parameters are selected as input indicators; for each indicator, a trapezoidal membership function is designed to describe its membership degree to each congestion level; gradient membership model:

[0038]

[0039] Multi-indicator weighted fuzzy synthesis:

[0040] The final congestion level is selected based on the highest overall membership level.

[0041]

[0042] in, Let x be the membership degree of the observed value x to the k-th congestion level, with a value range of [0,1]. , , , represents the boundary parameters of the k-th level membership function; This represents the membership vector of the density index across five levels. This represents the membership vector of the speed index across five levels. This represents the membership vector of the traffic indicator across five levels. , , These are the weighting coefficients for the three indicators: density, velocity, and flow rate. The final congestion level is determined at time t.

[0043] Preferably, in step 9, the GRU neural network algorithm is used as the core to predict the three elements of traffic flow, and traffic congestion dissipation time is predicted by combining traffic wave theory: using the traffic density at time t Average speed Traffic flow The input is fed into a gated recurrent unit model, and the output is a predicted traffic state for future time periods:

[0044]

[0045] in, To output the weight matrix; This is the output bias vector; Let's define the hidden state of the GRU at time t. Then, we use the congestion dissipation model from classical traffic wave theory for further calculations.

[0046]

[0047]

[0048]

[0049]

[0050] The congestion event's resolution time is approximately T0. The wave velocity of the surging wave is W1, and the wave velocity of the dissipating wave is W2. The time between the two waves' meeting is T. The traffic density before the event resolution is K1, and the density corresponding to the road's capacity at this time is KS1. The predicted traffic density after the event resolution is K2. At this point, the density corresponding to the road capacity is KS2, and the vehicle speed after the event is resolved returns to the same speed as before the event, v0. The queue length is x, and the dissipation time is T*.

[0051] The Gaussian priors are established based on the event type e identified by the improved YOLOv8 model in the drone video stream and the lane number L, and then updated using Bayesian methods based on the observations:

[0052]

[0053] in, After the nth observation, for a traffic event of type e and number of lanes L, the estimated road critical density is... The posterior mean; The posterior variance of the critical density estimate after the (n-1)th update; Noise variance of critical density observations; The actual critical density observation value corresponding to event type e and lane number L observed in the nth observation.

[0054] Preferably, step 10 is based on improved GRU+GCN traffic situation prediction, where the first step is spatial dependency modeling, which extracts the spatial correlation between adjacent road segments in the road network through graph convolution:

[0055]

[0056] in =A+I is an adjacency matrix with self-loops. Its degree matrix, For learnable space weights, (x) is the Sigmoid activation function. For spatial embedding;

[0057]

[0058] Insert a spatiotemporal attention block here, and after residual connection and layer normalization, reshape it back to the spatiotemporal dimension. At the end of time step t−1, the model outputs the predicted value for the next time step t:

[0059]

[0060] It contains future density, speed, and traffic flow forecasts for each road segment, which can be used to calculate the probability of congestion or as input for traffic wave models, and its effects can propagate along the road network.

[0061] Preferably, in step 11, based on the congestion detection and traffic situation prediction in the preceding steps, the reward and penalty function of the multi-agent PPO is:

[0062]

[0063] Where t represents time and i represents the traffic light agent. The average speed at the intersection. The amount of speed decrease, The proportion of low-speed vehicles. To obscure the congestion index, The magnitude of the sudden change in vehicle acceleration. The improved GRU+GCN traffic situation prediction model outputs the probability of future short-term congestion. , , , as well as As weight.

[0064] Local observation vector of each agent i It is composed of the following nine types of features: lane congestion vector, average speed, calculated based on floating car data sampling, the proportion of sampled vehicles with instantaneous speeds greater than the threshold, fuzzy congestion level, acceleration change amplitude, start-stop ratio, average direction angle, and current phase encoding. In addition, the system introduces a neighbor feature aggregation mechanism: for each agent, the average queue length of its neighboring intersections and the mean of the phase one-hot vector are calculated and appended to the end of the observation. The action space of each agent is defined as a discrete set, corresponding to the selectable signal phase index.

[0065] Compared with the prior art, the present invention has the following advantages:

[0066] 1. This invention utilizes the flexible deployment and wide-area overhead view of UAVs to collect real-time video of roads of any level. Combined with video detection and tracking algorithms, it obtains the microscopic state of the road, compensating for the coverage blind spots and sampling biases of ground floating car data.

[0067] 2. At the same time, the changes in road segment accumulation are combined with a fuzzy logic model to integrate multiple indicators of density, speed, and flow, outputting congestion levels, which improves the accuracy of congestion detection and provides a basis for refined management.

[0068] 3. Data fusion of drone and floating car data, with drone data as the primary source and floating car data as a supplement, expands data source channels, reduces costs, and improves the credibility of data analysis;

[0069] 4. A series of response chains were constructed after traffic congestion events were detected. If congestion occurs, the time for traffic congestion to dissipate is predicted and multi-signal light control is used to alleviate it. If congestion does not occur, traffic situation is predicted and multi-signal light control is used preventively to alleviate traffic flow accumulation. Attached Figure Description

[0070] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0071] Figure 1 is a flowchart of a traffic congestion detection and early warning method based on drone video streams. Detailed Implementation

[0072] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structure, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0073] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0074] Compared to existing methods, the traffic congestion event detection, prediction, and optimization method based on UAV video streams offers cost savings, reduced manpower and material resources, and dynamic flexibility. Figure 1 The specific implementation method of the present invention is as follows:

[0075] Step 1: Determine whether to deploy drones based on the floating car-triggered drone detection algorithm model;

[0076] Step 2: Control the drone or use waypoint flight to make the drone hover or fly at a constant speed above the road, and obtain parameters such as altitude, gimbal tilt angle, cruise speed, and position information through the ground station, and send these parameters back to the server;

[0077] Step 3: Push the video stream from the ground station to the server using the RTMP protocol. The server reads the RTMP address to obtain the video stream.

[0078] Step 4: Detect vehicles using the improved YOLOv8 algorithm on the video stream;

[0079] Step 5: Use the DeepSort algorithm to perform vehicle tracking on the video stream based on YOLOv8 algorithm detection;

[0080] Step 6: Calculate the road length based on the drone's basic parameters and quantify the three elements of traffic volume;

[0081] Step 7: Use a database to record the data corresponding to each time point and spatial location;

[0082] 7.1: Record the current drone parameters, road network ID, and traffic flow data in database 1;

[0083] 7.2: Receive de-identified floating car GPS trajectory data from the cooperating platform, including timestamps, latitude and longitude, and instantaneous speed; associate trajectory points with specific road segments in the road network topology through map matching, and aggregate to generate road segment-level macro indicators, which are then stored in database 2;

[0084] Step 8: Perform dual-mechanism traffic congestion detection on the obtained data: the road segment accumulation change congestion detection algorithm determines activation, and the fuzzy logic congestion detection algorithm determines the level of congestion.

[0085] 8.1: Define upstream and downstream virtual detection lines in the image coordinate system; use DeepSort data stored in the database to count the number of vehicles crossing the upstream section per unit time. , , as well as The judgment is based on changes in the road section's accumulation volume;

[0086] 8.2: A multi-indicator fusion congestion level determination model is constructed using fuzzy logic. Three traffic flow parameters are selected as input indicators. For each indicator, a trapezoidal membership function is designed to describe its membership degree to each congestion level.

[0087] Step 9: Using database data as input, the algorithm GRU neural network is used as the core to predict the three elements of traffic flow, and traffic congestion dissipation time is predicted by combining traffic wave theory.

[0088] Step 10: Using database data as input, predict traffic conditions based on the improved GRU+GCN. GCN is responsible for processing road topology information, and GRU is responsible for obtaining the time dependency of traffic data.

[0089] Step 11: Based on the congestion detection and traffic situation prediction in the previous steps, use multi-agent reinforcement learning to control multiple traffic lights and coordinate to alleviate traffic congestion events.

[0090] In step 1, the floating car-triggered drone detection algorithm model is defined as the congestion pheromone concentration of road segment i at time t as... ,

[0091]

[0092] in, This represents the residual pheromone content from the previous moment; The natural decay coefficient is in the range of [0,1], simulating the natural dissipation of congestion; Normalized speed ratio , Let k be the real-time speed of the kth floating car on this road segment; This represents the free-flow velocity of that road section; For the excitation function, the bipolar hyperbolic tangent function

[0093]

[0094] in, The optimal velocity threshold, i.e., the phase transition point; This is the scaling factor.

[0095] Simultaneously construct a utility function To measure the current concentration of congestion pheromones Do we need to take off?

[0096]

[0097] in The Sigmoid saturation function normalizes the pheromone concentration to the (0,1) interval; This represents the Euclidean distance from the drone's current location to the target location; This represents the current percentage of the drone's battery power. Value reward coefficient; This represents the cost penalty coefficient.

[0098] In step 4, the improvements to YOLOv8 include: reducing the depth multiplication factor from 1.0 to 0.75 to reduce the number of repeated stackings of C2f modules in each stage; reducing the width multiplication factor from 1.25 to 0.75 to reduce the number of channels in each convolutional layer; embedding CoordAtt modules after the P2, P3, and P4 feature layers in the backbone; and adopting a lightweight BiFPN approach in the head part: adding a bottom-up GhostConv + BiFPN_Add2 path after the traditional FPN upsampling path to achieve bidirectional cross-scale information interaction; replacing some standard convolutions with GhostConv to further reduce computational cost with almost no loss of accuracy; and retaining SPPF to expand the receptive field while avoiding heavy modules such as SPPC and SPC, thus balancing context modeling and inference speed.

[0099] In step 6, the specific steps for quantifying the three elements of traffic volume are as follows, with vehicle density as follows:

[0100]

[0101] Where k(t) represents the traffic density at time t; NROI(t) represents the number of vehicles in the detection area; and L represents the length of the detection area.

[0102] Velocity calculation based on world coordinate system:

[0103]

[0104] The average speed is calculated using the instantaneous speeds of each vehicle.

[0105]

[0106] Where Pi represents the transformed world coordinates of the vehicle's bottom center point, and ∆t represents the inter-frame time interval. The instantaneous speed of the i-th vehicle at time t Let t be the average speed of all vehicles within the detection area.

[0107] Traffic flow:

[0108]

[0109] in, Let be the traffic flow at time t.

[0110] In step 8.1, the congestion detection algorithm based on changes in road segment accumulation involves defining upstream and downstream virtual detection lines in a coordinate system; and using data stored in the database by DeepSort to count the number of vehicles crossing the upstream section per unit time. , , as well as If the current floating car sample density of the road segment If the data exceeds a preset threshold, a weighted fusion strategy is used to correct the traffic; otherwise, pure drone data is used.

[0111]

[0112]

[0113] Among them, weight Adjustments are made based on the proportion of floating car sample density exceeding a threshold, for pure drone data. =1,

[0114] The specific judgment conditions are as follows:

[0115]

[0116] Where Croad is the maximum number of vehicles that the road segment can accommodate, and A(t) is the cumulative number of vehicles accumulated up to time t; The effective continuous observation period is determined. When all these conditions are met, congestion is identified. Congestion levels are further classified by combining the ratio of accumulated traffic volume to road capacity and average vehicle speed, among other indicators.

[0117] In step 8.2, the specific algorithm for traffic congestion judgment based on fuzzy logic involves constructing a multi-indicator fusion congestion level determination model. Three traffic flow parameters are selected as input indicators. For each indicator, a trapezoidal membership function is designed to describe its membership degree to each congestion level. Gradient membership model:

[0118]

[0119] Multi-indicator weighted fuzzy synthesis:

[0120] The final congestion level is selected based on the highest overall membership level.

[0121]

[0122] in, Let x be the membership degree of the observed value x to the k-th congestion level, with a value range of [0,1]. , , , represents the boundary parameters of the k-th level membership function; This represents the membership vector of the density index across five levels. This represents the membership vector of the speed index across five levels. This represents the membership vector of the traffic indicator across five levels. , , These are the weighting coefficients for the three indicators: density, velocity, and flow rate. The final congestion level is determined at time t.

[0123] In step 9, the improved GRU algorithm combined with traffic wave theory is to use the GRU model to predict the changes in the three elements of traffic flow after congestion occurs.

[0124] When there is insufficient data for GRU computation, data from free-flow moments when no congestion occurred is used as the input for the three parameters of the dissipated traffic flow. This prediction and backoff system ensures the accuracy, scalability, and robustness of the prediction.

[0125] The hidden state update of the GRU cell at time t is as follows:

[0126]

[0127] The input vector is defined as:

[0128]

[0129] Let be the hidden state vector at time t−1; To update the door; To reset the door; Candidate hidden state; , , The input weight matrix is ​​used for the linear transformation of the update gate, reset gate, and candidate state, respectively, where d is the hidden layer dimension and n is the input feature dimension; , , The cyclic weight matrix is ​​used to connect the hidden states from the previous time step. ; , , These are bias vectors, corresponding to the offsets of the update gate, reset gate, and candidate state, respectively. (x) is the Sigmoid activation function, defined as having an output range of (0,1); (x) is the hyperbolic tangent function, with an output range of (−1, 1); For Hadamard product,

[0130] The output layer of GRU is:

[0131]

[0132] in, To output the weight matrix; This is the output bias vector; The predicted traffic density at time t. The predicted average velocity at time t Let be the predicted traffic flow at time t.

[0133] Predict ), , As road traffic parameters that dissipate after the causal events of congestion are resolved, the congestion dissipation model of classical traffic wave theory is then used for further calculation.

[0134]

[0135]

[0136]

[0137]

[0138] The congestion event's resolution time is approximately T0. The wave velocity of the surging wave is W1, and the wave velocity of the dissipating wave is W2. The time between the two waves' meeting is T. The traffic density before the event resolution is K1, and the density corresponding to the road's capacity at this time is KS1. The predicted traffic density after the event resolution is K2. At this point, the density corresponding to the road capacity is KS2, and the vehicle speed after the event is resolved returns to the same speed as before the event, v0. The queue length is x, and the dissipation time is T*.

[0139] The road capacity at the time of the incident was determined by training a vehicle accident dataset using an improved YOLOv8 model and detecting drone video streams to classify congestion events as non-accident and accident-related, further categorizing accidents into single-lane and multi-lane types. Based on existing road network data, an initial...

[0140] This distribution is updated online with the observed data. Let's assume that after the nth observation of an event of type e and number of lanes L, we obtain the observed value of its critical density. And assume that the observed noise follows a variance of Gaussian distribution:

[0141]

[0142] Utilizing the conjugate prior property, the posterior distribution remains a Gaussian distribution. Let the parameter after the (n-1)th update be denoted as ^2. Then the Bayesian update formula for the nth time is:

[0143]

[0144] That is, the critical density for the current event (e,L). The optimal estimate is used for calculating the rush wave velocity in subsequent traffic wave models:

[0145]

[0146] This allows us to obtain the road traffic volume at the time of an event, which is updated and changes as data accumulates.

[0147] In step 10, as the drone continues to be stationed or conducts multiple patrols, the system gradually accumulates historical data for target road segment i. The system automatically switches to GRU prediction mode, inputs its own and upstream road segment's historical traffic sequences, and performs end-to-end traffic situation prediction:

[0148] Spatial dependency modeling extracts the spatial correlation between adjacent road segments in the road network through graph convolution:

[0149]

[0150] in =A+I is an adjacency matrix with self-loops. Its degree matrix, For learnable space weights, (x) is the Sigmoid activation function. For spatial embedding.

[0151]

[0152] Insert a spatiotemporal attention block here, and after residual connection and layer normalization, reshape it back to the spatiotemporal dimension. At the end of time step t−1, the model outputs the predicted value for the next time step t:

[0153]

[0154] It contains future density, speed, and traffic flow forecasts for each road segment, which can be used to calculate the probability of congestion or as input for traffic wave models, and its effects can propagate along the road network.

[0155] In step 11, based on the congestion detection and traffic situation prediction in the previous steps, multi-agent reinforcement learning is used to control multiple traffic lights to coordinate and alleviate traffic congestion events. The reward and penalty function is as follows:

[0156]

[0157] Where t represents time and i represents the traffic light agent. The average speed at the intersection. The amount of speed decrease, The proportion of low-speed vehicles. To obscure the congestion index, The magnitude of the sudden change in vehicle acceleration. The improved GRU+GCN traffic situation prediction model outputs the probability of future short-term congestion. , , , as well as As weight.

[0158] Local observation vector of each agent i The system is composed of nine features: lane congestion vector, average speed, calculated based on floating car data sampling, the proportion of sampled vehicles with instantaneous speeds exceeding a threshold, fuzzy congestion level, acceleration change magnitude, start-stop ratio, average direction angle, and current phase encoding. Furthermore, the system introduces a neighbor feature aggregation mechanism: for each agent, it calculates the average queue length of its neighboring intersections and the mean of the one-hot phase vectors, appending this to the end of the observation. The action space of each agent is defined as a discrete set, corresponding to selectable signal phase indices.

[0159] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for traffic congestion detection and early warning based on UAV video stream, characterized in that, Includes the following steps: Step 1: Determine whether to deploy drones based on the floating car-triggered drone detection algorithm model; Step 2: Control the drone or use waypoint flight to make the drone hover or fly at a constant speed above the road, and obtain parameters such as altitude, gimbal tilt angle, cruise speed, and position information through the ground station, and send these parameters back to the server; Step 3: Push the video stream from the ground station to the server using the RTMP protocol. The server reads the RTMP address to obtain the video stream. Step 4: Detect vehicles using the improved YOLOv8 algorithm on the video stream; Step 5: Use the DeepSort algorithm to perform vehicle tracking on the video stream based on YOLOv8 algorithm detection; Step 6: Calculate the road length based on the drone's basic parameters and quantify the three elements of traffic volume; Step 7: Use a database to record the data corresponding to each time point and spatial location; 7.1: Record the current drone parameters, road network ID, and traffic flow data in database 1; 7.2: Receive de-identified floating car GPS trajectory data from the cooperating platform, including timestamps, latitude and longitude, and instantaneous speed; associate trajectory points with specific road segments in the road network topology through map matching, and aggregate to generate road segment-level macro indicators, which are then stored in database 2; Step 8: Perform dual-mechanism traffic congestion detection on the obtained data: the road segment accumulation change congestion detection algorithm determines activation, and the fuzzy logic congestion detection algorithm determines the level of congestion. 8.1: Delineate upstream and downstream virtual detection lines in the image coordinate system; use DeepSort data stored in the database to count the number of vehicles crossing the upstream section per unit time, and make judgments based on changes in road segment accumulation; 8.2: Using fuzzy logic, a multi-indicator fusion congestion level determination model is constructed. The three traffic flow parameters are selected as input indicators. For each indicator, a trapezoidal membership function is designed to describe its membership degree to each congestion level. Step 9: Using database data as input, the algorithm GRU neural network is used as the core to predict the three elements of traffic flow, and traffic congestion dissipation time is predicted by combining traffic wave theory. Step 10: Using database data as input, predict traffic conditions based on the improved GRU+GCN. GCN is responsible for processing road topology information, and GRU is responsible for obtaining the time dependency of traffic data. Step 11: Based on the congestion detection and traffic situation prediction in the previous steps, use multi-agent reinforcement learning to control multiple traffic lights and coordinate to alleviate traffic congestion events.

2. The traffic congestion detection and early warning method based on UAV video stream according to claim 1, characterized in that, The floating car-triggered drone detection algorithm model in step 1 is defined as follows: the congestion pheromone concentration of road segment i at time t is... , in, This represents the residual pheromone content from the previous moment; The natural decay coefficient is in the range of [0,1], simulating the natural dissipation of congestion; Normalized rate ratio , Let k be the real-time speed of the kth floating car on this road segment; This represents the free-flow velocity of that road section; For the excitation function, the bipolar hyperbolic tangent function in, The optimal velocity threshold, i.e., the phase transition point; This is the scaling factor; Simultaneously construct a utility function To measure the current concentration of congestion pheromones Do we need to take off? in The Sigmoid saturation function normalizes the pheromone concentration to the (0,1) interval; This represents the Euclidean distance from the drone's current location to the target location; This represents the current percentage of the drone's battery power. Value reward coefficient; This represents the cost penalty coefficient.

3. The traffic congestion detection and early warning method based on UAV video stream according to claim 1, characterized in that, In step 4, the algorithm improvements for UAV video stream detection, specifically for YOLOv8, include: reducing the depth multiplication factor from 1.0 to 0.75 to reduce the number of repeated stackings of C2f modules in each stage; reducing the width multiplication factor from 1.25 to 0.75 to reduce the number of channels in each convolutional layer; embedding CoordAtt modules after the P2, P3, and P4 feature layers in the backbone; and adopting a lightweight BiFPN approach in the head section: adding a bottom-up GhostConv + BiFPN_Add2 path after the traditional FPN upsampling path to achieve bidirectional cross-scale information interaction; replacing some standard convolutions with GhostConv to further reduce computational cost with almost no loss of accuracy; and retaining SPPF to expand the receptive field while avoiding heavy modules such as SPPC and SPC, thus balancing context modeling and inference speed.

4. The traffic congestion detection and early warning method based on UAV video stream according to claim 1, characterized in that, In step 8, the dual-mechanism traffic congestion detection algorithm, specifically the congestion detection algorithm based on changes in road segment accumulation, involves defining upstream and downstream virtual detection lines in a coordinate system; and using data stored in the database by DeepSort to count the number of vehicles crossing the upstream section per unit time. , as well as If the current floating car sample density of the road segment If the data exceeds a preset threshold, a weighted fusion strategy is used to correct the traffic; otherwise, pure drone data is used. Among them, weight Adjustments are made based on the proportion of floating car sample density exceeding a threshold, for pure drone data. = 1, The specific judgment conditions are as follows: Where Croad is the maximum number of vehicles that the road segment can accommodate, and A(t) is the cumulative number of vehicles accumulated up to time t; To ensure effective continuous observation duration; when all these conditions are met, it is judged as congestion; combining the ratio of cumulative accumulation to road capacity and average vehicle speed, the congestion level is further classified; In step 8.2, the specific algorithm process for traffic congestion judgment based on fuzzy logic is as follows: A multi-indicator fusion congestion level determination model is constructed; three traffic flow parameters are selected as input indicators; for each indicator, a trapezoidal membership function is designed to describe its membership degree to each congestion level; gradient membership model: Multi-indicator weighted fuzzy synthesis: The final congestion level is selected based on the highest overall membership level. in, Let x be the membership degree of the observed value x to the k-th congestion level, with a value range of [0,1]. , , , represents the boundary parameters of the k-th level membership function; This represents the membership vector of the density index across five levels. This represents the membership vector of the speed index across five levels. This represents the membership vector of the traffic indicator across five levels. , , These are the weighting coefficients for the three indicators: density, velocity, and flow rate. The final congestion level is determined at time t.

5. The traffic congestion detection and early warning method based on UAV video stream according to claim 1, characterized in that, In step 9, the GRU neural network algorithm is used as the core to predict the three elements of traffic flow, and traffic congestion dissipation time is predicted by combining traffic wave theory: based on the traffic density at time t. Average speed Traffic flow The input is fed into a gated recurrent unit model, and the output is a predicted traffic state for future time periods: in, To output the weight matrix; This is the output bias vector; Let's define the hidden state of the GRU at time t. Then, we use the congestion dissipation model from classical traffic wave theory for further calculations. The congestion event's approximate resolution time is T0. The wave velocity of the surging wave is W1, and the wave velocity of the dissipating wave is W2. The time between the two waves' meeting is T. The traffic density before the event resolution is K1, and the density corresponding to the road's capacity at this time is KS1. The predicted traffic density after the event resolution is K2. At this point, the density corresponding to the road capacity is KS2, and the vehicle speed after the event is resolved returns to the same speed as before the event, v0. The queue length is x, and the dissipation time is T*. The Gaussian priors are established based on the event type e identified by the improved YOLOv8 model in the drone video stream and the lane number L, and then updated using Bayesian methods based on the observations: in, After the nth observation, for a traffic event of type e and number of lanes L, the estimated road critical density is... The posterior mean; The posterior variance of the critical density estimate after the (n-1)th update; Noise variance of critical density observations; The actual critical density observation value corresponding to event type e and lane number L observed in the nth observation.

6. The traffic congestion detection and early warning method based on UAV video stream according to claim 1, characterized in that... Step 10 is traffic situation prediction based on the improved GRU+GCN, which first involves spatial dependency modeling, extracting the spatial correlation between adjacent road segments in the road network through graph convolution: ,in =A+I is an adjacency matrix with self-loops. Its degree matrix, For learnable space weights, (x) is the Sigmoid activation function. For spatial embedding; A spatiotemporal attention block is inserted here. After residual connection and layer normalization, it is reshaped back to the spatiotemporal dimension. At the end of time step t−1, the model outputs the predicted value for the next time step t: It contains future density, speed, and traffic flow forecasts for each road segment, which can be used to calculate the probability of congestion or as input for traffic wave models, and its effects can propagate along the road network.

7. The traffic congestion detection and early warning method based on UAV video stream according to claim 1, characterized in that... In step 11, based on the congestion detection and traffic situation prediction in the previous steps, the reward and penalty function for the multi-agent PPO is: Where t represents time and i represents the traffic light agent. The average speed at the intersection. The amount of speed decrease, The proportion of low-speed vehicles. To obscure the congestion index, The magnitude of the sudden change in vehicle acceleration. The improved GRU+GCN traffic situation prediction model outputs the probability of future short-term congestion. as well as The weights are the local observation vectors of each agent i. It is composed of the following nine types of features: lane congestion vector, average speed, calculated based on floating car data sampling, the proportion of sampled vehicles with instantaneous speeds greater than the threshold, fuzzy congestion level, acceleration change amplitude, start-stop ratio, average direction angle, and current phase encoding. In addition, the system introduces a neighbor feature aggregation mechanism: for each agent, the average queue length of its neighboring intersections and the mean of the phase one-hot vector are calculated and appended to the end of the observation. The action space of each agent is defined as a discrete set, corresponding to the selectable signal phase index.

Images