Intelligent network connection traffic signal lamp and vehicle cloud side cooperative control method

Abstract

The invention discloses an intelligent network connection traffic signal lamp and vehicle cloud side cooperative control method, and the method comprises the steps: predicting the overall traffic flow direction through the traffic flow information of a graph structure formed by the topological relation between a plurality of traffic intersections, and planning the phase duration of each traffic signal lamp according to the predicted traffic flow direction; the method improves the coupling among a plurality of intelligent network connection traffic intersections, employs a federal learning method, builds a cloud-side cooperative training mode of a traffic signal lamp time length planning network, enables each side end device to form a cloud-side cooperative intelligent traffic whole under the guidance of a cloud end device, effectively reduces the training turns of the side end devices, and improves the training efficiency. The training efficiency is improved, and the side end load is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of automatic control technology, and specifically relates to a method for collaborative control of intelligent networked traffic lights and vehicle cloud-edge. Background Art

[0002] Intelligent connected traffic intersections are mainly composed of intelligent connected vehicles and intelligent roadside equipment, and rely on vehicle-to-vehicle communication and vehicle-to-roadside equipment communication technology to realize the trajectory planning of intelligent connected vehicles in traffic intersections and the duration planning of traffic lights, so as to alleviate traffic congestion and improve traffic efficiency.

[0003] Currently, methods for planning trajectories and traffic lights for connected vehicles at intelligently connected traffic intersections primarily focus on the situation at a single intersection. These methods solve optimization problems to plan trajectories for connected vehicles within the intersection for a specific time period in the future. Furthermore, reinforcement learning methods are used to determine the duration of the next traffic light phase based on the current traffic conditions at the single intersection. This approach is limited to the situation at a single intersection and ignores the interplay between traffic flows at multiple intersections. Consequently, existing methods are unable to predict future traffic flow information, limiting further improvements in traffic efficiency. Furthermore, current intelligently connected traffic intersection planners are primarily deployed on roadside equipment, performing information collection, planning problem solving, and control signal issuance tasks locally at the intersection. This places high demands on the computing power of the roadside equipment and increases the cost of technology deployment. Summary of the Invention

[0004] In view of this, the present invention provides a method for cloud-edge collaborative control of intelligent connected traffic lights and vehicles, which realizes overall collaborative control of traffic lights and vehicles at multiple traffic intersections in response to the situation where traffic flows between multiple traffic intersections influence each other.

[0005] The present invention provides a method for collaboratively controlling intelligent networked traffic lights and vehicle cloud-edge systems, comprising the following steps:

[0006] Step 1: Build a multi-intersection traffic map flow prediction model based on the spatiotemporal graph convolutional neural network to realize flow prediction for the traffic map composed of multiple intelligent connected traffic intersections. The multi-intersection traffic map flow prediction model is: in, represents the predicted four-way incoming and outgoing traffic flow information of the traffic intersection numbered n at time t+1, H is the embedding matrix; the loss function required for the multi-intersection traffic map flow prediction model training The back propagation algorithm is used to train the multi-intersection traffic map flow prediction model until the loss function convergence;

[0007] Step 2: Build a traffic light duration planning network based on reinforcement learning. The state is N is the number of vehicles in each lane in the direction of entry, Q f is the future traffic flow of each lane predicted by the multi-intersection traffic graph flow prediction model, Period now is the duration of the current signal phase; the action is the possible set of the duration of the next phase A = {Duration lower_bound ,Duration lower_bound +Δt,...,Duration upper_bound Duration lower_bound Duration is the shortest duration of the signal light phase. upper_bound is the maximum duration of the signal light phase, Δt represents the minimum change; the reward is Duration is the duration of the signal phase, N out is the number of vehicles exiting the intersection during the duration of the phase, N in is the number of vehicles entering the intersection during the duration of the phase, Q out is the outgoing flow calculated during the duration of the phase, Q in is the incoming flow calculated during the phase duration, ω1 is the net flow weight coefficient, and ω2 is the penalty time coefficient; the training of the traffic light duration planning network is completed based on the loss function;

[0008] Step 3: Based on the traffic light duration constraints, a mixed integer programming method is used to establish an intelligent connected vehicle speed planner. Horizontal federated learning is used to complete cloud-edge collaborative training of multiple traffic light duration planning networks.

[0009] Step 4. In actual use, the edge device at each intelligent connected traffic intersection obtains traffic status information and uploads it to the cloud server; the cloud server uses a multi-intersection traffic map flow prediction model to predict traffic flow based on the received traffic status information, and obtains traffic flow prediction information within a certain time domain in the future; the cloud server completes the collaborative model training of the traffic light duration planning network deployed in the client through federated learning based on the traffic status information and traffic flow prediction information; each client uses the trained traffic light duration planning network to plan the duration of the next signal phase, and then uses the duration plan of the next signal phase as a signal constraint to complete the speed planning of the intelligent connected vehicle through the deployed intelligent connected vehicle speed planner, thereby realizing collaborative control of traffic lights and vehicles.

[0010] Furthermore, the loss function of the traffic light duration planning network is:

[0011] Where r is the reward, s is the current state, a is the current action, and s′ is the state of the next round; γ is the discount factor, which is used to measure the importance of future rewards in the current decision; θ is the evaluation network parameter, and θ′ is the target network parameter.

[0012] Furthermore, the use of horizontal federated learning to complete cloud-edge collaborative training of multiple traffic light timing planning networks includes: the traffic light timing planning network of each traffic intersection is deployed on an independent client as a client model; the traffic light timing planning network is deployed on the cloud server as a cloud model, and its parameter update is obtained by aggregating the parameters of each client model through the federated learning method; during the training process, the client first downloads the parameters of the cloud model as the parameters of the client model, and then uses the loss function calculated by the current data to obtain the posterior gradient of the client network parameters, and then calculates the prior gradient without using the current data, and calculates the mean of the prior gradient and the posterior gradient by setting the weight to update the network parameters, and sends the updated client model parameters to the cloud server; after the cloud server receives the client model parameters returned by all clients in this round, it aggregates and updates the cloud model parameters, and then enters the next round of training; until the training of all traffic light timing planning networks is completed.

[0013] Beneficial effects:

[0014] The present invention uses the graph-structured traffic flow information formed by the topological relationship between multiple traffic intersections to predict the overall traffic flow direction, and uses this to plan the phase duration of each traffic light, thereby improving the coupling between multiple intelligent connected traffic intersections. At the same time, the federated learning method is adopted to build a cloud-edge collaborative training mode for the traffic light duration planning network, so that each edge device is guided by the cloud device to form a cloud-edge collaborative intelligent traffic whole, effectively reducing the training rounds of edge devices, improving training efficiency, and reducing edge load. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 A schematic flow chart of a method for collaborative control of intelligent connected traffic lights and vehicle cloud-edge provided by the present invention. DETAILED DESCRIPTION

[0016] The present invention is described in detail with reference to the following examples.

[0017] The present invention provides a method for collaborative control of intelligent networked traffic lights and vehicle cloud-edge, the flow chart of which is as follows: Figure 1 As shown, it mainly includes the following steps:

[0018] Step 1: Build a multi-intersection traffic map flow prediction model based on the spatiotemporal graph convolutional neural network to realize flow prediction for the traffic map composed of multiple intelligent connected traffic intersections. The multi-intersection traffic map flow prediction model is: in, represents the predicted four-way incoming and outgoing traffic flow information of the traffic intersection numbered n at time t+1, H is the embedding matrix; the loss function required for the multi-intersection traffic map flow prediction model training The back propagation algorithm is used to train the multi-intersection traffic map flow prediction model until the loss function convergence.

[0019] The present invention adopts x t,n represents the traffic flow information of the traffic intersection numbered n at time t, x t,n =[Q t,n,N ,Q t,n,E ,Q t,n,S ,Q t,n,W ], Q t,n,N =[Q t,n,N,in ,Q t,n,N,out ] represents the inbound and outbound traffic of the northbound road at the traffic intersection numbered n at time t, Q t,n,E =[Q t,n,E,in ,Q t,n,E,out ] represents the incoming and outgoing traffic of the eastbound road at the traffic intersection numbered n at time t, Q t,n,S =[Q t,n,S,in ,Q t,n,S,out ] represents the incoming and outgoing traffic volume of the southbound road at the traffic intersection numbered n at time t, Q t,n,W =[Q t,n,W,in ,Q t,n,W,out ] represents the incoming and outgoing traffic flow of the westbound road at the traffic intersection numbered n at time t.

[0020] Among them, the loss function for:

[0021]

[0022] Among them, i∈[1,...,8] values ​​correspond to northbound entry, northbound exit, eastbound entry, eastbound exit, southbound entry, southbound exit, westbound entry, and westbound exit; Indicates the true value of the flow direction of the dataset. Represents the predicted value of the corresponding flow direction, ω i Represents the weight parameter, which reflects the influence of each direction on the overall traffic flow efficiency and is continuously updated during the network training process.

[0023] Specifically, the construction process of the multi-intersection traffic graph flow prediction model includes:

[0024] Define the traffic map: in, For the picture The vertex set, that is, the set of all traffic intersection numbers, its size That is, the number of traffic intersections N that constitute the traffic graph; ε is the set of edges, which represents the connection relationship between any two traffic intersections; is the adjacency matrix of the graph, It is the flow information set of each traffic intersection from time tT to time t.

[0025] For a certain traffic intersection n in the traffic map, at a certain time t, its flow information x t,n As shown in the following formula: t,n =[Q t,n,N ,Q t,n,E ,Q t,n,S ,Q t,n,W ], where Q t,n,N =[Q t,n,N,in ,Q t,n,N,out ] represents the inbound and outbound traffic flow of the northbound road at intersection n at time t. This can describe the directional traffic information in the traffic graph.

[0026] Using spatio-temporal (ST) convolutional network to train traffic maps Processing to obtain the embedding vector The details are as follows:

[0027]

[0028] E t =SC(B t ,A)

[0029] Among them, TC(*) is temporal convolution, SC(*) is spatial convolution, is the region embedding vector at time t, its nth row represents the embedding vector of the region numbered as traffic intersection n, D represents the dimension of the embedding vector, T out is the length of the embedded sequence output by the temporal convolutional encoder after convolution operation, E t is the processing result of spatial information at time t after spatial convolution.

[0030] After multiple TC→SC→TC structure processing, the final embedding matrix form is obtained: The traffic flow prediction for the next moment can be obtained by using the embedding matrix H through a multilayer perceptron (MLP): in, Represents the predicted four-way incoming and outgoing traffic flow information for the traffic intersection numbered n at time t+1.

[0031] Step 2: Build a traffic light duration planning network based on reinforcement learning, which includes state (Situation, S), action (Action, A), and reward (Reward, R), as follows:

[0032] State S is Where N represents the number of vehicles in the direction of entry in each lane within the sensing range of the intelligent traffic intersection, N = [N N ,N E ,N S ,N W ],N N Indicates the number of vehicles entering the northbound lane, N E Indicates the number of vehicles entering the eastbound lane, N S Indicates the number of vehicles entering the southbound lane, N W Indicates the number of vehicles entering the westbound lane; Q f represents the future traffic flow of each lane predicted by the multi-intersection traffic graph flow prediction model, Q f =[Q f,N ,Q f,E ,Q f,S ,Q f,W ], Q f,N represents the future northbound traffic flow, Q f,E represents the future eastbound traffic flow, Q f,S represents the future southbound traffic flow, Q f,W Indicates the future westbound traffic flow; Period now Indicates the duration of the current signal phase.

[0033] Action A represents the possible set of durations of the next phase, specifically: A = {Duration lower_bound ,Duration lower_bound +Δt,...,Duration upper_bound}, where Duration lower_bound Indicates the shortest duration of the signal light phase, Duration upper_bound It indicates the maximum duration of the signal light phase, and Δt indicates the minimum change, which is usually set to Δt = 1s.

[0034] The reward R is in, Duration indicates the duration of the signal phase, N out Indicates the number of vehicles exiting the intersection during the duration of the phase, N in represents the number of vehicles entering the intersection during the duration of the phase, Q out represents the outgoing flow calculated during the duration of the phase, Q in represents the incoming flow calculated during the duration of the phase, ω1 is the net flow weight coefficient, and ω2 is the penalty time coefficient.

[0035] The loss function constructed for training the reinforcement learning-based traffic light duration planning network is as follows: Where r is the reward, s is the current state, a is the current action, s′ is the state of the next round, γ is the discount factor, which is used to measure the importance of future rewards in the current decision, θ is the evaluation network parameter, and θ′ is the target network parameter. After a certain number of iterations, the parameters of the evaluation network will be copied to the target network.

[0036] Through the back-propagation method, the training of the traffic light duration planning network is completed based on the constructed loss function.

[0037] Because traffic flow is constantly changing, intelligent connected traffic intersections should be able to adaptively plan the duration of the next signal phase based on current and future traffic flow information over a specific time domain to maximize traffic flow. Therefore, this paper constructs a traffic light duration planning network based on reinforcement learning.

[0038] Step 3: Considering the traffic light duration constraint, a mixed integer programming method is used to establish the speed planner for intelligent connected vehicles.

[0039] For one of the four lanes in an intelligent connected traffic intersection, the model predictive control method is used to establish the speed planning problem of the intelligent connected vehicle, as shown in the following formula:

[0040]

[0041] d sig -d i (k)≤M·(1-p i ),d i (k)-d sig ≤M·p i ,

[0042]

[0043] Among them, N OHrepresents the final moment of the prediction time domain, dt is the discrete time step, ω1, ω2, ω3, ω4 are all weight parameters, s i 、l i are the relaxation factors in the following model and safety distance constraint, d(k), v(k), a(k) are the distance traveled, speed, and acceleration at time k, respectively, and M is a relatively large value. i ∈{0,1} is an indicator variable. When vehicle i can pass through the intersection in the current green phase, p i =1, otherwise p i = 0. The function sig(k) represents the phase of the traffic light at time k. τ is a parameter in the GHR model that simulates a human driver, indicating that the following vehicle tends to reach the same speed as the preceding vehicle within τ seconds. N(t0) represents the set of all vehicles in the lane at time t0, including intelligent connected vehicles and human-driven vehicles. N h (t0) represents the set of all human-driven vehicles in the lane at time t0. h is the headway, h min is the minimum following distance when the vehicle is stationary. min 、v max 、a min 、a max They are the upper and lower limits of speed and acceleration respectively.

[0044] Step 4: Use horizontal federated learning to perform cloud-edge collaborative training on multiple traffic light duration planning networks. The traffic light duration planning network at each traffic intersection is deployed on an independent client as a client model. n (n=1,...,N) represents; a traffic light duration planning network is deployed on the cloud server as a cloud model, and its parameter updates are obtained by aggregating the model parameters of each client through the federated learning method, which is represented by S; during the training process, the client first downloads the parameters of the cloud model as the parameters of the client model, and then uses the loss function calculated by the current data to obtain the posterior gradient of the client network parameters, and then calculates the prior gradient without using the current data, calculates the mean of the prior gradient and the posterior gradient with a certain weight, and updates the network parameters, and sends the updated client model parameters to the cloud server; after the cloud server receives the client model parameters returned by all clients in this round, it aggregates and updates the cloud model parameters and enters the next round of training; until the training of all traffic light duration planning networks is completed. Specifically, it includes the following steps:

[0045] Step 4.1. Design the ClientUpdate function.

[0046] Input: The cloud model's prior distribution mean μ for parameter θ S, the prior distribution covariance Σ of the cloud model for parameter θ S , number of cloud training rounds r N ; Output: the updated prior distribution mean μ of the local client model numbered n n , prior distribution covariance Σ n .

[0047] Step 4.1.1. Initialization: Client model parameters θ = μ S , clear μ n and Σ n .

[0048] Step 4.1.2, loop: Local training rounds from 1 to E, where the i-th training includes:

[0049] From the total data set D n Take B data for this training, and for the bth data, calculate the gradient g of the parameter θ: Among them, Loss n (θ) represents the network loss function of client n;

[0050] Then complete the following calculation:

[0051] diag(∑ n )=diag(∑ n )+diag(gg)

[0052]

[0053] Among them, diag() is a function that forms a diagonal matrix, is the prior loss function, λ is the balance factor that balances the posterior loss gradient and the prior loss gradient, and lr is the learning rate.

[0054] Step 4.1.3. Calculate the updated prior distribution mean μ of the local client model numbered n according to the following formula: n , prior distribution covariance Σ n :μ n =θ, Return μ n and Σ n .

[0055] Step 4.2: Design the cloud-based model parameter aggregation function.

[0056] Step 4.2.1. Initialization: Initialize the cloud model parameters θ and the prior mean μ of the parameters θ S , the prior covariance of the parameters ∑ S .

[0057] Step 4.2.2, loop: For the r=1, 2... rounds of training: For the n=1,..., N clients, execute the following function and receive the return value: μ n ,Σ n =ClientUpdate(μ s ,Σ S ,r N );

[0058] The following formula is used to aggregate the mean model parameter μ of each client: n and covariance∑ n , to obtain the prior mean μ of the cloud model parameters after this round of training S and parameter prior covariance ∑ S :

[0059]

[0060]

[0061] Among them, the superscript -1 represents the reciprocal, and the parameter π n Indicates the preset contribution weight of each client to the cloud, satisfying

[0062] Step 4.3: Combining Federated Learning with the Traffic Light Duration Planning Network at Multiple Intelligent Connected Traffic Intersections:

[0063] Step 4.3.1: In the case of multiple intelligent networked traffic intersections, the signal light duration planning network at each traffic intersection is an independent client, with C n (n=1,...,N) represents.

[0064] Step 4.3.2: On the cloud server, a traffic light duration planning network is also deployed as the main network model. Its parameters are updated by aggregating the parameters of each client model using the federated learning method, represented by S.

[0065] Step 4.3.3: As mentioned in step 2, the construction method of the traffic light duration planning network based on reinforcement learning and the loss function Loss n The client first downloads the parameters of the cloud model as the client model parameters, then uses the loss function calculated based on the current data to obtain the posterior gradient of the client network parameters. Next, the prior gradient is calculated without using the current data. The mean of the prior and posterior gradients is calculated with a certain weight, and this is used to update the network parameters and return to the cloud.

[0066] Step 4.3.4: After receiving the network parameters returned by all clients in this round, the cloud server aggregates and updates the cloud model parameters and enters the next round.

[0067] Step 5. In actual use, the edge device of each intelligent connected traffic intersection obtains traffic status information in the scene through traffic perception. The traffic status information includes vehicle position, speed, driving direction, current signal light phase, and the position and speed of other traffic participants such as pedestrians, and uploads the traffic status information to the cloud server; the cloud server uses the multi-intersection traffic map flow prediction model to predict traffic flow based on the received traffic status information, and obtains traffic flow prediction information within a certain time domain in the future; the cloud server completes the collaborative model training of the traffic light duration planning network deployed in the client through federated learning based on the traffic status information and traffic flow prediction information; each client uses the trained traffic light duration planning network to plan the duration of the next signal phase, and then uses the duration plan of the next signal phase as the signal constraint to complete the speed planning of the intelligent connected vehicle through the deployed intelligent connected vehicle speed planner.

[0068] The cloud server and the client (edge) deploy a multi-intersection traffic flow prediction model and a traffic light duration planning network respectively. In addition, the client also needs to perform traffic perception tasks. The specific workflow of cloud-edge collaboration is as follows:

[0069] Step 5.1: The edge devices at each intelligent connected traffic intersection perform traffic perception, obtain traffic status information within the scene, and upload it to the cloud server.

[0070] Step 5.2: The cloud server executes the task of the multi-intersection traffic map flow prediction model based on the traffic information uploaded by each intelligent connected intersection, and obtains the traffic flow prediction information within a certain time domain in the future.

[0071] In step 5.3, the cloud executes the task of the traffic light duration planning network based on the traffic status information and traffic flow prediction information obtained by traffic perception. Through federated learning, it conducts collaborative model training with the edge traffic light duration planning network and plans the duration of the next signal phase at each traffic intersection.

[0072] Step 5.4: The edge devices at each intelligent connected traffic intersection use the next phase signal duration planned by the traffic light duration planning network as a signal constraint to perform speed planning tasks for the intelligent connected vehicles in the scene.

[0073] Step 5.5: Repeat the above steps at a certain sampling to planning interval.

[0074] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for collaborative control of intelligent networked traffic lights and vehicle cloud-edge, characterized in that: The following steps are involved: Step 1: Build a multi-intersection traffic map flow prediction model based on the spatiotemporal graph convolutional neural network to realize flow prediction for the traffic map composed of multiple intelligent connected traffic intersections. The multi-intersection traffic map flow prediction model is: in, represents the predicted four-way incoming and outgoing traffic flow information of the traffic intersection numbered n at time t+1, H is the embedding matrix; the loss function required for the multi-intersection traffic map flow prediction model training The back propagation algorithm is used to train the multi-intersection traffic map flow prediction model until the loss function convergence; Step 2: Build a traffic light duration planning network based on reinforcement learning. The state is N is the number of vehicles in each lane in the direction of entry, Q f is the future traffic flow of each lane predicted by the multi-intersection traffic graph flow prediction model, Period now is the duration of the current signal phase; the action is the possible set of the duration of the next phase A = {Duration lower_bound ,Duration lower_bound +Δt,...,Duration upper_bound Duration lower_bound Duration is the shortest duration of the signal light phase. upper_bound is the maximum duration of the signal light phase, Δt represents the minimum change; the reward is Duration is the duration of the signal phase, N out is the number of vehicles exiting the intersection during the duration of the phase, N in is the number of vehicles entering the intersection during the duration of the phase, Q out is the outgoing flow calculated during the duration of the phase, Q in is the incoming flow calculated during the phase duration, ω1 is the net flow weight coefficient, and ω2 is the penalty time coefficient; the training of the traffic light duration planning network is completed based on the loss function; Step 3: Based on the traffic light duration constraints, a mixed integer programming method is used to establish an intelligent connected vehicle speed planner. Horizontal federated learning is used to complete cloud-edge collaborative training of multiple traffic light duration planning networks. Step 4. In actual use, the edge device at each intelligent connected traffic intersection obtains traffic status information and uploads it to the cloud server; the cloud server uses a multi-intersection traffic map flow prediction model to predict traffic flow based on the received traffic status information, and obtains traffic flow prediction information within a certain time domain in the future; the cloud server completes the collaborative model training of the traffic light duration planning network deployed in the client through federated learning based on the traffic status information and traffic flow prediction information; each client uses the trained traffic light duration planning network to plan the duration of the next signal phase, and then uses the duration plan of the next signal phase as a signal constraint to complete the speed planning of the intelligent connected vehicle through the deployed intelligent connected vehicle speed planner, thereby realizing collaborative control of traffic lights and vehicles.

2. The method for collaborative control of intelligent networked traffic lights and vehicle cloud-edge according to claim 1 is characterized in that: The loss function of the traffic light duration planning network is: Where r is the reward, s is the current state, a is the current action, and s′ is the state of the next round; γ is the discount factor, which is used to measure the importance of future rewards in the current decision; θ is the evaluation network parameter, and θ′ is the target network parameter.

3. The method for collaborative control of intelligent networked traffic lights and vehicle cloud-edge according to claim 1 is characterized in that: The method uses horizontal federated learning to complete cloud-edge collaborative training of multiple traffic light timing planning networks, including: the traffic light timing planning network of each traffic intersection is deployed on an independent client as a client model; the traffic light timing planning network is deployed on a cloud server as a cloud model, and its parameter updates are obtained by aggregating the parameters of each client model through a federated learning method; during the training process, the client first downloads the parameters of the cloud model as the parameters of the client model, and then uses the loss function calculated by the current data to obtain the posterior gradient of the client network parameters, and then calculates the prior gradient without using the current data, and calculates the mean of the prior gradient and the posterior gradient with the set weights to update the network parameters, and sends the updated client model parameters to the cloud server; after the cloud server receives the client model parameters returned by all clients in this round, it aggregates and updates the cloud model parameters and then enters the next round of training; until the training of all traffic light timing planning networks is completed.

Images