Intelligent networked traffic signal and vehicle cloud edge collaborative control method

Abstract

The application discloses a kind of intelligent network connection traffic signal lamp and vehicle cloud edge collaborative control method, utilize the graph structure traffic flow information formed by the topological relationship between multiple traffic intersections to predict overall traffic flow direction, and plan the phase length of each traffic signal lamp according to this, improve the coupling between multiple intelligent network connection traffic intersections, simultaneously, using the method of federated learning, build the cloud edge collaborative training mode of traffic signal lamp length planning network, so that each edge device is guided by cloud device, form the overall intelligent traffic of cloud edge collaboration, effectively reduce the training round of edge device, improve training efficiency, reduce edge load.

Description

Technical Field

[0001] This invention belongs to the field of automatic control technology, specifically relating to a method for intelligent connected traffic lights and vehicle cloud-edge collaborative control. Background Technology

[0002] Intelligent connected traffic intersections mainly consist of intelligent connected vehicles and intelligent roadside equipment. They rely on vehicle-to-vehicle communication and vehicle-to-roadside equipment communication technologies to plan the trajectories of intelligent connected vehicles within the intersection and the duration of traffic lights, in order to alleviate traffic congestion and improve traffic efficiency.

[0003] Currently, intelligent connected vehicle trajectory and traffic light planning methods at intelligent connected traffic intersections primarily focus on the case of a single intersection. These methods solve optimization problems to plan the future trajectories of intelligent connected vehicles within a certain time domain at the intersection, and use reinforcement learning to determine the duration of the next phase of the traffic lights based on the current traffic conditions at the individual intersection. However, this approach is limited to the case of a single intersection and ignores the interrelationships and mutual influences between multiple intersections. Therefore, existing methods cannot predict future traffic flow information, limiting further improvements in traffic efficiency. Furthermore, current intelligent connected traffic intersection planners are mainly deployed on roadside equipment, meaning they perform information collection, planning problem solving, and control signal distribution locally at the intersection. This places high demands on the computing power of the computing units on the roadside equipment and increases deployment costs. Summary of the Invention

[0004] In view of this, the present invention provides a cloud-edge collaborative control method for intelligent connected traffic lights and vehicles, which realizes overall collaborative control of traffic lights and vehicles at multiple traffic intersections when traffic flows at multiple intersections affect each other.

[0005] This invention provides a method for intelligent connected traffic lights and vehicle cloud-edge collaborative control, comprising the following steps:

[0006] Step 1: Construct a multi-intersection traffic map flow prediction model based on a spatiotemporal graph convolutional neural network to predict traffic flow for traffic maps composed of multiple intelligent connected traffic intersections. The multi-intersection traffic map flow prediction model is as follows: in, H represents the predicted inbound and outbound traffic flow information for intersection n at time t+1, where H is the embedding matrix; the loss function required for training the multi-intersection traffic map flow prediction model. The backpropagation algorithm is used to train a multi-intersection traffic map flow prediction model until the loss function is reached. convergence;

[0007] Step 2: Construct a traffic light duration planning network based on reinforcement learning, with the following state: N is the number of vehicles in each lane in the direction of entry, Q f The Period represents the future traffic flow of each lane predicted by the multi-intersection traffic map flow prediction model. now The duration of the current signal phase; the possible set of actions, A = {Duration}, represents the duration of the next phase. lower_bound Duration lower_bound +Δt,...,Duration upper_bound Duration lower_bound Duration is the shortest duration of a traffic light phase. upper_bound The maximum duration of the traffic light phase is given by Δt, where Δt represents the minimum change. The reward is... Duration is the duration of the signal phase, N out N represents the number of vehicles exiting the intersection during the duration of this phase. in Q represents the number of vehicles entering the intersection during the duration of this phase. out Q is the outbound flow rate calculated during the duration of this phase. in The inbound flow rate is calculated within the duration of this phase, where ω1 is the net flow rate weight coefficient and ω2 is the penalty time coefficient; the traffic light duration planning network is trained based on the loss function.

[0008] Step 3: Based on the traffic light duration constraint, a hybrid integer programming method is used to establish an intelligent connected vehicle speed planner; horizontal federated learning is used to complete the cloud-edge collaborative training of multiple traffic light duration planning networks;

[0009] Step 4: In practical use, the edge devices at each intelligent connected traffic intersection obtain traffic status information and upload it to the cloud server. Based on the received traffic status information, the cloud server uses a multi-intersection traffic map flow prediction model to predict traffic flow and obtain traffic flow prediction information for a certain time domain in the future. Based on the traffic status information and traffic flow prediction information, the cloud server completes the collaborative model training of the traffic signal duration planning network deployed in the client through federated learning. Each client uses the trained traffic signal duration planning network to plan the duration of the next signal phase, and then uses the duration planning of the next signal phase as a signal constraint to complete the speed planning of intelligent connected vehicles through the deployed intelligent connected vehicle speed planner, thereby realizing the collaborative control of traffic lights and vehicles.

[0010] Furthermore, the loss function of the traffic light duration planning network is:

[0011] Where r is the reward, s is the current state, a is the current action, and s′ is the state of the next round; γ is the discount factor used to measure the importance of future rewards in the current decision; θ is the evaluation network parameter, and θ′ is the target network parameter.

[0012] Furthermore, the cloud-edge collaborative training of multiple traffic light duration planning networks using lateral federated learning includes: deploying the traffic light duration planning network for each intersection on an independent client as a client model; deploying the traffic light duration planning network on a cloud server as a cloud model, with its parameters updated by aggregating the parameters of each client model using federated learning; during training, the client first downloads the parameters of the cloud model as the parameters of the client model, then uses the loss function calculated from the current data to obtain the posterior gradient of the client network parameters, and then calculates the prior gradient without using the current data, sets weights to calculate the mean of the prior gradient and the posterior gradient, thereby updating the network parameters, and sending the updated client model parameters to the cloud server; after receiving the client model parameters returned by all clients in this round, the cloud server aggregates and updates the cloud model parameters, and then enters the next round of training; until the training of all traffic light duration planning networks is completed.

[0013] Beneficial effects:

[0014] This invention utilizes the graph-structured traffic flow information formed by the topological relationships between multiple traffic intersections to predict the overall traffic flow direction and plan the phase duration of each traffic light accordingly. This improves the coupling between multiple intelligent connected traffic intersections. At the same time, it adopts a federated learning method to build a cloud-edge collaborative training mode for the traffic light duration planning network. This allows each edge device to form a cloud-edge collaborative intelligent transportation system guided by the cloud device, effectively reducing the number of training rounds for edge devices, improving training efficiency, and reducing edge load. Attached Figure Description

[0015] Figure 1 This invention provides a flowchart illustrating a cloud-edge collaborative control method for intelligent connected traffic lights and vehicles. Detailed Implementation

[0016] The following examples illustrate the invention in detail.

[0017] This invention provides a method for intelligent connected traffic lights and vehicle cloud-edge collaborative control, the flowchart of which is shown below. Figure 1 As shown, the main steps include:

[0018] Step 1: Construct a multi-intersection traffic map flow prediction model based on a spatiotemporal graph convolutional neural network to predict traffic flow for traffic maps composed of multiple intelligent connected traffic intersections. The multi-intersection traffic map flow prediction model is as follows: in, H represents the predicted inbound and outbound traffic flow information for intersection n at time t+1, where H is the embedding matrix; the loss function required for training the multi-intersection traffic map flow prediction model. The backpropagation algorithm is used to train a multi-intersection traffic map flow prediction model until the loss function is reached. convergence.

[0019] This invention uses x t,n This represents the traffic flow information at intersection n at time t, x t,n =[Q t,n,N Q t,n,E Q t,n,S Q t,n,W ], Q t,n,N =[Q t,n,N,in Q t,n,N,out [Q] represents the inbound and outbound traffic flow of the intersection numbered n at time t on its northbound road. t,n,E =[Q t,n,E,in Q t,n,E,out [Q] represents the inbound and outbound traffic flow of the intersection numbered n at time t, on its eastbound road. t,n,S =[Q t,n,S,in Q t,n,S,out [Q] represents the inbound and outbound traffic flow of the intersection numbered n at time t, on its southbound road. t,n,W =[Q t,n,W,in Q t,n,W,out [] represents the inbound and outbound traffic flow of the traffic intersection numbered n at time t.

[0020] Among them, the loss function for:

[0021]

[0022] Where i∈[1,...,8] takes values ​​corresponding to northbound entry, northbound exit, eastbound entry, eastbound exit, southbound entry, southbound exit, westbound entry, and westbound exit, respectively. This represents the true value of the corresponding flow direction in the dataset. ω represents the predicted value for the corresponding flow direction. i This represents the weight parameter, which reflects the degree of influence of each direction on the overall traffic flow efficiency and is continuously updated during network training.

[0023] Specifically, the process of constructing a multi-intersection traffic map flow prediction model includes:

[0024] Define the traffic map: in, For the image The set of vertices, that is, the set of all traffic intersection numbers, has a size of That is, the number of traffic intersections N that make up the traffic map; ε is the set of edges, representing the connection relationship between any two traffic intersections; Let be the adjacency matrix of the graph. This is a set of traffic flow information for each intersection from time tT to time t.

[0025] For a traffic intersection n in the traffic map, at a certain time t, its traffic flow information x t,n As shown in the following formula: x t,n =[Q t,n,N Q t,n,E Q t,n,S Q t,n,W ], where Q t,n,N =[Q t,n,N,in Q t,n,N,out Let ] represent the inbound and outbound traffic flow of the northbound road at time t for intersection n. This can be used to describe the directional traffic flow information in the traffic map.

[0026] Using a spatio-temporal (ST) convolutional network to process traffic maps The process is performed to obtain the embedding vector. Specifically as follows:

[0027]

[0028] E t =SC(B t A)

[0029] Where TC(*) represents temporal convolution, and SC(*) represents spatial convolution. Let be the region embedding vector at time t, and its nth row. Let D represent the region embedding vector with the traffic intersection number n, and let T represent the dimension of the embedding vector. out E represents the length of the embedded sequence output by the temporal convolution encoder after convolution operations. t This represents the result of processing spatial information at time t after spatial convolution.

[0030] After processing through multiple TC→SC→TC structures, the final embedding matrix form is obtained: Traffic flow prediction for the next time step can be obtained from the embedding matrix H through a multilayer perceptron (MLP): in, This represents the predicted inbound and outbound traffic flow information for the traffic intersection with the number n at time t+1.

[0031] Step 2: Construct a traffic light duration planning network based on reinforcement learning, including state (Situation, S), action (A), and reward (Reward, R), as follows:

[0032] State S is Where N represents the number of vehicles in each lane within the sensing range of the intelligent traffic intersection in the direction of entry, N = [N N N E N S N W ], N N N represents the number of vehicles entering the northbound lane. E N represents the number of vehicles entering the lane from the east. S N represents the number of vehicles entering the southbound lane. W Indicates the number of vehicles entering the westbound lane; Q f Q represents the future traffic flow in each direction predicted by the multi-intersection traffic map flow prediction model. f =[Q f,N Q f,E Q f,S Q f,W ], Q f,N Q represents future northbound traffic flow. f,E Q represents future eastbound traffic flow. f,S Q represents future southbound traffic flow. f,W Indicates future westbound traffic flow; Period now It indicates the duration of the current signal phase.

[0033] Action A represents the possible set of durations of the next phase, specifically: A = {Duration} lower_bound Duration lower_bound +Δt,...,Duration upper_bound}, where Duration lower_bound The duration indicates the shortest duration of a traffic light phase. upper_bound Δt represents the maximum duration of the traffic light phase, and Δt represents the minimum change, usually taken as Δt = 1s.

[0034] Reward R is in, Duration represents the duration of the signal phase, N out N represents the number of vehicles exiting the intersection during the duration of that phase. in Q represents the number of vehicles entering the intersection during the duration of that phase. out Q represents the outbound flow rate calculated during the duration of this phase. in This represents the inbound flow calculated during the duration of this phase, where ω1 is the net flow weighting coefficient and ω2 is the penalty time coefficient.

[0035] The loss function for training this reinforcement learning-based traffic light duration planning network is constructed as follows: Where r is the reward, s is the current state, a is the current action, s′ is the state of the next round, γ is the discount factor used to measure the importance of future rewards in the current decision, θ is the parameters of the evaluation network, and θ′ is the parameters of the target network. After a certain number of iterations, the parameters of the evaluation network will be copied to the target network.

[0036] The traffic light duration planning network was trained using the backpropagation method based on the constructed loss function.

[0037] Since traffic flow is constantly changing, intelligent connected traffic intersection scenarios should be able to adaptively plan the duration of the next signal phase based on current and future traffic flow information within a certain time domain, in order to maximize traffic flow. Therefore, this invention constructs a traffic signal duration planning network based on reinforcement learning.

[0038] Step 3: Considering the traffic light duration constraint, a speed planner for intelligent connected vehicles is established using a mixed integer programming method.

[0039] For a specific lane in a four-way intersection of an intelligent connected traffic system, the speed planning problem for intelligent connected vehicles is established using model predictive control methods, as shown in the following formula:

[0040]

[0041] d sig -d i (k)≤M·(1-p i ),d i (k)-d sig ≤M·p i ,

[0042]

[0043] Where, N OHdt represents the final time in the prediction time domain, ω1, ω2, ω3, and ω4 are all weighting parameters, and s represents the final time in the prediction time domain. i l i are the relaxation factors in the following model and the safety distance constraint, respectively; d(k), v(k), and a(k) are the distance traveled, speed, and acceleration at time k, respectively; and M is a very large value. i ∈{0,1} is an indicator variable; when vehicle i can pass through the intersection in the current green phase, p i =1, otherwise p i =0. The function sig(k) represents the traffic light phase at time k. τ is a parameter in the GHR (Gas-Road Harness) model simulating human drivers, indicating that the following vehicle tends to reach the same speed as the preceding vehicle within τ seconds. N(t0) represents the set of all vehicles in the lane at time t0, including connected vehicles and human-driven vehicles. h (t0) represents the set of all human-driven vehicles in the lane at time t0. h is the headway. min This is the minimum following distance when the vehicle is stationary. min v max a min a max These are the upper and lower limits for velocity and acceleration, respectively.

[0044] Step 4: Employ horizontal federated learning to perform cloud-edge collaborative training on multiple traffic light duration planning networks. Each traffic intersection's traffic light duration planning network is deployed on an independent client as a client model, using C... n (n=1,...,N) represents a traffic light duration planning network deployed on a cloud server as a cloud model. Its parameters are updated by aggregating the parameters of each client model using a federated learning method, denoted as S. During training, the client first downloads the parameters of the cloud model as its own parameters. Then, it uses a loss function calculated from the current data to obtain the posterior gradient of the client network parameters. Next, it calculates the prior gradient without using the current data, and calculates the mean of the prior and posterior gradients with certain weights. This is used to update the network parameters, and the updated client model parameters are sent to the cloud server. After receiving all the client model parameters returned by the clients in this round, the cloud server aggregates and updates the cloud model parameters before entering the next round of training. This process continues until all traffic light duration planning networks have been trained. Specifically, the following steps are included:

[0045] Step 4.1: Design the ClientUpdate function.

[0046] The input is: the prior distribution mean μ of the cloud model for parameter θ. SThe prior distribution covariance Σ of the cloud model for parameter θ S Number of cloud training rounds r N Output: The mean μ of the updated prior distribution of the local client model with ID n. n Prior distribution covariance Σ n .

[0047] Step 4.1.1, Initialization: Client model parameters θ = μ S Clear μ n and Σ n .

[0048] Step 4.1.2, Loop: The local training rounds range from 1 to E, where the i-th training round includes:

[0049] From the total dataset D n We take B data points for this training. For the b-th data point, we calculate the gradient g of the parameter θ: Among them, Loss n (θ) represents the network loss function for client n;

[0050] Then perform the following calculations:

[0051] diag(∑ n ) = diag(∑ n )+diag(gg)

[0052]

[0053] Here, diag() is a function that forms a diagonal matrix. Let λ be the prior loss function, λ be the balancing factor between the posterior and prior loss gradients, and lr be the learning rate.

[0054] Step 4.1.3: Calculate the mean μ of the updated prior distribution of the local client model with ID n according to the following formula. n Prior distribution covariance Σ n μ n =θ, Return μ n and Σ n .

[0055] Step 4.2: Design the cloud model parameter aggregation function.

[0056] Step 4.2.1 Initialization: Initialize the cloud model parameters θ, and the prior mean μ of parameter θ. S The prior covariance of the parameters ∑ S .

[0057] Step 4.2.2, Loop: For the r-th round of training (r=1, 2...), for the n-th client (n=1,..., N), execute the following function and receive the return value: μ n ,Σ n =ClientUpdate(μ s ,Σ S ,r N );

[0058] The mean μ of the model parameters for each client is aggregated using the following formula. n ∑ covariance n To obtain the prior mean μ of the cloud model parameters after this round of training. S and parameter prior covariance ∑ S :

[0059]

[0060]

[0061] Where the superscript -1 represents the reciprocal, and the parameter π n This represents the preset contribution weight of each client to the cloud, satisfying...

[0062] Step 4.3: The method of combining federated learning with traffic light duration planning networks at multi-intelligent connected intersections:

[0063] Step 4.3.1: In the case of multiple intelligent connected traffic intersections, the traffic light duration planning network for each intersection is an independent client, denoted by C. n (n=1,...,N) represents the expression.

[0064] Step 4.3.2: On the cloud server, deploy a traffic light duration planning network as the main network model. Its parameters are updated by aggregating the parameters of each client model through federated learning, denoted by S.

[0065] Step 4.3.3: As mentioned in Step 2, construct the traffic light duration planning network based on reinforcement learning and the loss function Loss. n The defined method involves the client first downloading the parameters of the cloud model as the parameters of its own model. Then, it uses a loss function calculated from the current data to obtain the posterior gradient of the client's network parameters. Next, without using the current data, the prior gradient is calculated. The mean of the prior and posterior gradients is calculated with certain weights, and the network parameters are updated accordingly before being returned to the cloud.

[0066] Step 4.3.4: After receiving the network parameters returned by all clients in this round, the cloud server aggregates and updates the cloud model parameters and proceeds to the next round.

[0067] Step 5: In practical use, the edge devices at each intelligent connected traffic intersection obtain traffic status information within the scene through traffic perception. The traffic status information includes vehicle position, speed, and direction of travel, the current traffic light phase, and the position and speed of pedestrians and other traffic participants. This traffic status information is then uploaded to the cloud server. Based on the received traffic status information, the cloud server uses a multi-intersection traffic map flow prediction model to predict traffic flow, obtaining traffic flow prediction information for a certain time domain in the future. Based on the traffic status information and traffic flow prediction information, the cloud server completes the collaborative model training of the traffic light duration planning network deployed within the client through federated learning. Each client uses the trained traffic light duration planning network to plan the duration of the next signal phase, and then uses the duration planning of the next signal phase as a signal constraint to complete the speed planning of intelligent connected vehicles through the deployed intelligent connected vehicle speed planner.

[0068] The cloud server and client (edge) respectively deploy a multi-intersection traffic map flow prediction model and a traffic light duration planning network. In addition, the client also needs to perform traffic perception tasks. The specific workflow of cloud-edge collaboration is as follows:

[0069] Step 5.1: The edge devices at each intelligent connected traffic intersection perform traffic perception, obtain traffic status information within the scene, and upload it to the cloud server.

[0070] Step 5.2: Based on the traffic information uploaded by each intelligent connected intersection, the cloud server executes the task of the multi-intersection traffic map flow prediction model to obtain traffic flow prediction information for a certain time domain in the future.

[0071] Step 5.3: Based on the traffic status information and traffic flow prediction information obtained from traffic perception, the cloud executes the task of the traffic signal duration planning network. Through federated learning, it conducts collaborative model training with the edge traffic signal duration planning network and performs duration planning for the next signal phase at each traffic intersection.

[0072] Step 5.4: The edge devices at each intelligent connected traffic intersection use the duration of the next phase signal planned by the traffic signal duration planning network as a signal constraint to perform speed planning tasks for intelligent connected vehicles in the scenario.

[0073] Step 5.5: Repeat the above steps at certain sampling to planning intervals.

[0074] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for intelligent connected traffic lights and vehicle cloud-edge collaborative control, characterized in that, Includes the following steps: Step 1: Construct a multi-intersection traffic map flow prediction model based on a spatiotemporal graph convolutional neural network to predict traffic flow for traffic maps composed of multiple intelligent connected traffic intersections. The multi-intersection traffic map flow prediction model is as follows: in, H represents the predicted inbound and outbound traffic flow information for intersection n at time t+1, where H is the embedding matrix; the loss function required for training the multi-intersection traffic map flow prediction model. The backpropagation algorithm is used to train a multi-intersection traffic map flow prediction model until the loss function is reached. convergence; Step 2: Construct a traffic light duration planning network based on reinforcement learning, with the following state: N is the number of vehicles in each lane in the direction of entry, Q f The Period represents the future traffic flow of each lane predicted by the multi-intersection traffic map flow prediction model. now The duration of the current signal phase; the possible set of actions, A = {Duration}, represents the duration of the next phase. lower_bound Duration lower_bound +Δt,...,Duration upper_bound Duration lower_bound Duration is the shortest duration of a traffic light phase. upper_bound The maximum duration of the traffic light phase is given by Δt, where Δt represents the minimum change. The reward is... Duration is the duration of the signal phase, N out N represents the number of vehicles that exit the intersection during the duration of this phase. in Q represents the number of vehicles that enter the intersection during the duration of this phase. out Q is the outbound flow rate calculated during the duration of this phase. in The inbound flow rate is calculated within the duration of this phase, where ω1 is the net flow rate weight coefficient and ω2 is the penalty time coefficient; the traffic light duration planning network is trained based on the loss function. Step 3: Based on the traffic light duration constraint, a hybrid integer programming method is used to establish an intelligent connected vehicle speed planner; horizontal federated learning is used to complete the cloud-edge collaborative training of multiple traffic light duration planning networks; Step 4: In practical use, the edge devices at each intelligent connected traffic intersection obtain traffic status information and upload it to the cloud server. Based on the received traffic status information, the cloud server uses a multi-intersection traffic map flow prediction model to predict traffic flow and obtain traffic flow prediction information for a certain time domain in the future. Based on the traffic status information and traffic flow prediction information, the cloud server completes the collaborative model training of the traffic signal duration planning network deployed in the client through federated learning. Each client uses the trained traffic signal duration planning network to plan the duration of the next signal phase, and then uses the duration planning of the next signal phase as a signal constraint to complete the speed planning of intelligent connected vehicles through the deployed intelligent connected vehicle speed planner, thereby realizing the collaborative control of traffic lights and vehicles.

2. The intelligent connected traffic signal light and vehicle cloud-edge collaborative control method according to claim 1, characterized in that, The loss function of the traffic light duration planning network is: Where r is the reward, s is the current state, a is the current action, and s′ is the state of the next round; γ is the discount factor used to measure the importance of future rewards in the current decision; θ is the evaluation network parameter, and θ′ is the target network parameter.

3. The intelligent connected traffic signal light and vehicle cloud-edge collaborative control method according to claim 1, characterized in that, The method employs horizontal federated learning to perform cloud-edge collaborative training of multiple traffic light duration planning networks. This includes: deploying the traffic light duration planning network for each intersection on independent clients as client models; deploying the traffic light duration planning network on a cloud server as a cloud model, with its parameters updated by aggregating the parameters of each client model using federated learning; during training, the client first downloads the parameters of the cloud model as its own parameters, then uses a loss function calculated from the current data to obtain the posterior gradient of the client network parameters, and then calculates the prior gradient without using the current data. Weights are then set to calculate the mean of the prior and posterior gradients, which is used to update the network parameters. The updated client model parameters are then sent to the cloud server; after receiving the client model parameters returned by all clients in the current round, the cloud server aggregates and updates the cloud model parameters before proceeding to the next round of training; this process continues until all traffic light duration planning networks are trained.

Images