A traffic signal control method for double intersections based on Double DQN and convolutional neural network

Abstract

A double intersection traffic signal control method based on Double DQN and convolutional neural network, the specific steps are as follows: S1, the lanes of two adjacent intersections are segmented, and the road network vehicle traffic information data between the two intersections is collected in units of road sections, and the road network image information data M is collected at the same time. k and road network signal light phase information data P k ; S2, pre-process the data in step S1, and input the data into the convolutional neural network to obtain the road network traffic state-action set {S k ,P k}, where S k The road network traffic state vector is formed by connecting the road network image information data and the vehicle traffic information data, and a simulation model of the traffic simulation platform is built; S3, using the road network traffic state-action set {S k ,P k} and the constructed simulation model to train the Double DQN network, and update the parameters of the target Q network in the Double DQN network; S4, according to the trained deep neural network, the road network signal light phase P required at the next moment at the intersection k+1 .

Description

Technical Field

[0001] The invention belongs to the field of traffic control and relates to a double intersection traffic signal control method based on Double DQN and convolutional neural network. Background Art

[0002] With the rapid popularization of new energy vehicles, the total number of cars in my country has increased rapidly. The relatively backward road network infrastructure and outdated traffic timing schemes have led to increasingly severe traffic congestion. Compared with improving road network traffic facilities, optimizing the traffic timing scheme at intersections is more cost-effective. Traditional signal light control methods almost all use timing schemes, which cannot cope with the strong real-time characteristics of urban road networks. Moreover, the traffic data of urban road networks is now large in volume and redundant in data, so the existing traffic signal control methods cannot process data in a timely manner and cannot control traffic congestion well. Summary of the invention

[0003] In view of the problems existing in the above background technology introduction, the purpose of the present invention is to provide a dual-intersection traffic signal control method based on DoubleDQN and convolutional neural network, which integrates the vehicle traffic segmentation information obtained in real time into a data set for training Double DQN and convolutional neural network, and then uses the trained neural network to control the traffic lights at the intersection, thereby alleviating traffic congestion.

[0004] The technical solution adopted by the present invention is:

[0005] A double intersection traffic signal control method based on Double DQN and convolutional neural network, the specific steps are as follows:

[0006] S1. Divide the lanes of two adjacent intersections into sections, collect the road network vehicle traffic information data between the two intersections in sections, and collect the road network image information data M at the same time. k and road network signal light phase information data P k ;

[0007] S2, preprocess the data in step S1, and input the data into the convolutional neural network to obtain the road network traffic state-action set {S k ,P k}, where S k The road network traffic state vector is formed by connecting the road network image information data and the road network vehicle traffic information data, and a simulation model of the traffic simulation platform is built;

[0008] S3, using the road network traffic status-action set {S k ,P k} and the constructed simulation model to train the Double DQN network and update the parameters of the target Q network in the Double DQN network;

[0009] S4: Based on the trained deep neural network, the road network signal light phase P required at the intersection at the next moment k+1 for:

[0010] P k+1 =argmax(Q'(S k ,P k ;ω')),

[0011] Where Q'(S k ,P k ; ω') is the Q' value of the target Q network, and the arg max function is Q'(S k ,P k ; ω') is the value of the variable when it reaches its maximum value, and ω' is the parameter of the target Q network.

[0012] Furthermore, the steps of acquiring the road network vehicle traffic information data in step S1 are as follows:

[0013] S11, according to the characteristics of traffic flow data, the data sequence is processed into discrete time data, the discrete time step is unified, and k = 1, 2, ..., K is used to represent it, where k is the kth moment;

[0014] S12, for the road network of two adjacent intersections, firstly divide each lane entering the intersection into several small segments of equal length, and use n = 1, 2, ..., N to number the number of segments, where n represents the nth segment. For the nth segment at time k, the road network vehicle traffic information data is expressed as {l n,k ,v n,k}, where l n,k Indicates the number of vehicles on the road section at the current moment, v n,k Indicates the average speed of vehicles on the road section at the current moment;

[0015] S13, after collecting the data of all road sections, for the data of the two intersections, two vectors are used to describe the road network vehicle traffic information data {L k ,V k},in

[0016] L k = {l n,k :n=1,2,...,N},V k = {v n,k :n=1,2,...,N}.

[0017] Further, the road network image information data is k Indicates that Mk ={m n,k :n=1,2,...,N C}, for two adjacent intersections, each camera in the road network uses n = 1, 2, ..., N C Number, n represents the nth camera, for the nth camera at time k, the image information data of a single camera is represented as {m n,k}.

[0018] Further, the road network signal light phase information data is P k Indicates that P k ={p ,n k :=n 1,2, D .N, for two adjacent intersections, each signal light in the network uses n=1,2,...,N D Number, n represents the nth signal light, for the nth signal light at time k, its signal light phase information data is expressed as {p n,k}.

[0019] Further, the data processing steps of step S2 are as follows:

[0020] S21, delete abnormal data;

[0021] S22, obtain the intersection vehicle information in sections, and then use the obtained intersection road network vehicle traffic information data as L k -V k The structure is combined into the final road network vehicle traffic information data vector;

[0022] S23, collect the road network image information data M k Input into the convolutional neural network to output image features It is combined with the road network vehicle traffic information data vector {L k ,V k}Merge into state set Then, according to the road network signal light phase information data P k , forming the final road network traffic state-action set {S k ,P k}.

[0023] Furthermore, the abnormal data in step S21 include that the number of vehicles exceeds the maximum total number of vehicles that the lane can accommodate, and the average speed of vehicles exceeds the speed limit requirement of the urban road.

[0024] Further, the image feature M of the road network image information data extracted by the convolutional neural network in step S23 k CNNIt contains the pixel position of each vehicle's center point in the graph and the number of vehicles in the road section, indicating the relative position of each vehicle in the road network and the current congestion level of the road section.

[0025] Furthermore, in step S2, a simulation model of the traffic simulation platform is built. First, the road network is accurately modeled by importing the target vector map, and then the traffic flow is simulated. The lane traffic flow weight change information of each entrance of the double intersection every hour is obtained by the OD estimation method based on the license plate recognition data. The specific steps are as follows:

[0026] The first step is to obtain the average total traffic flow W within the required simulation time based on the vehicle information of the license plate recognition data. sum ,The total traffic volume is averaged by calculating multiple time periods;

[0027] The second step is to calculate the traffic flow weight q for each time period according to the time information of the license plate recognition data. i ,i=1,2,...,I, where i represents the i-th hour;

[0028] The third step is to calculate the number of vehicles passing each hour in the simulation time. i =q i W sum , and assign the number of vehicles passing each hour to the simulation model.

[0029] Further, the specific steps of updating the parameters of the target Q network in the Double DQN network in step S3 are as follows:

[0030] S31, initialize the parameters ω of the current Q network and the parameters ω' of the target Q network Q', and clear the experience feedback

[0031] Put pool D;

[0032] S32, for any time {S k ,P k}, define the reward value rk in Double DQN neural network learning:

[0033]

[0034] Among them, w1, w2, w3 are the reward conversion coefficients corresponding to different traffic information, C is the number of signal light switching times, 0 means that the phase of the signal light has not changed compared with the previous moment, and 1 means that it has changed;

[0035] S33, the current moment {S k ,P k} and reward r k Put it into experience pool D;

[0036] S34, take out sample information from the experience replay pool, and combine it with the information at the current moment to calculate the ω value of the current Q network Q, and the discount factor γ is set to 0.99:

[0037]

[0038] S35, use the mean square error loss function L and update the Q' parameter ω' of the target Q network through back propagation of the neural network:

[0039]

[0040] Among them, m is the number of sample information taken from the experience pool in one training, a′ is the optimal action at the next moment, and j is the number of iterations;

[0041] S36, continuously updating the neural network parameters until the maximum number of iterations is reached.

[0042] Compared with the prior art, the present invention has the following significant advantages: the vehicle traffic segmentation information obtained in real time is integrated into a data set for training Double DQN and convolutional neural networks, and then the trained neural network is used to control the traffic lights at the intersection, thereby alleviating traffic congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of the Double DQN and convolutional neural network training of the present invention.

[0044] Figure 2 It is a road network schematic diagram of a double intersection of the present invention. DETAILED DESCRIPTION

[0045] The present invention is further described below in conjunction with specific embodiments, but the present invention is not limited to these specific embodiments. Those skilled in the art should recognize that the present invention covers all possible alternatives, improvements and equivalents within the scope of the claims.

[0046] See also Figure 1 This embodiment provides a double intersection traffic signal control method based on Double DQN and convolutional neural network, and the specific steps are as follows:

[0047] S1. Divide the lanes of two adjacent intersections into sections, collect the road network vehicle traffic information data between the two intersections in sections, and collect the road network image information data M at the same time. k and road network signal light phase information data P k ;

[0048] The steps for obtaining vehicle traffic information data are as follows:

[0049] S11, according to the characteristics of traffic flow data, the data sequence is processed into discrete time data, the discrete time step is unified, and k = 1, 2, ..., K is used to represent it, where k is the kth moment;

[0050] S12, for the road network of two adjacent intersections, firstly divide each lane entering the intersection into several small segments of equal length, and use n = 1, 2, ..., N to number the number of segments, where n represents the nth segment. For the nth segment at time k, the road network vehicle traffic information data is expressed as {l n,k ,v n,k}, where l n,k Indicates the number of vehicles on the road section at the current moment, v n,k Indicates the average speed of vehicles on the road section at the current moment;

[0051] S13, after collecting the data of all road sections, for the data of the two intersections, two vectors are used to describe the road network vehicle traffic information data {L k ,V k}, where L k = {l n,k :n=1,2,...,N},V k = {v n,k :n=1,2,...,N}.

[0052] According to traffic flow theory, the three basic quantities of traffic flow are flow rate, density and speed. If any two of them are known, the third term can be obtained. Therefore, after obtaining the vehicle density vector and the vehicle average speed vector, the road network vehicle information is considered complete. The present invention facilitates the difference processing of data of different sections by segmenting the road network, thereby facilitating the neural network to screen more critical section data and assign higher decision weights.

[0053] For two adjacent intersections, each camera in the road network uses n=1,2,...,N C Number, n represents the nth camera, for the nth camera at time k, the image information data of a single camera is represented as {m n,k}, road network image information data is used by M k Indicates that M k ={m n,k :n=1,2,...,N C}.

[0054] For two adjacent intersections, each signal light in the network uses n=1,2,...,N D Number, n represents the nth signal light, for the nth signal light at time k, its signal light phase information data is expressed as {pn,k}, the road network signal light phase information data is P k Indicates that P k ={p n,k :n=1,2,...,N D}.

[0055] S2, preprocess the data in step S1, and input the data into the convolutional neural network to obtain the road network traffic state-action set {S k ,P k}, where S k The road network traffic state vector is formed by connecting the road network image information data and the road network vehicle traffic information data, and a simulation model of the traffic simulation platform is built;

[0056] The data processing steps of step S2 are as follows:

[0057] S21, deleting abnormal data; abnormal data includes the number of vehicles exceeding the maximum total number of vehicles that the lane can accommodate, and the average speed of vehicles exceeding the speed limit requirement of urban roads.

[0058] S22, obtain the intersection vehicle information in sections, and then use the obtained intersection road network vehicle traffic information data as L k -V k The structure is combined into the final road network vehicle traffic information data vector;

[0059] S23, collect the road network image information data M k Input into the convolutional neural network to output image features It is combined with the road network vehicle traffic information data vector {L k ,V k}Merge into state set Then according to the road network signal light phase information data P k , forming the final road network traffic state-action set {S k ,P k}.

[0060] The image features M of the road network image information data extracted by the convolutional neural network k CNN It contains the pixel position of each vehicle's center point in the graph and the number of vehicles in the road section, indicating the relative position of each vehicle in the road network and the current congestion level of the road section.

[0061] The simulation model of the traffic simulation platform is built by first accurately modeling the road network by importing the target vector map, and then simulating the traffic flow. The lane traffic flow weight change information of each entrance of the double intersection every hour is obtained by the OD estimation method based on license plate recognition data. The specific steps are as follows:

[0062] The first step is to obtain the average total traffic flow W within the required simulation time based on the vehicle information of the license plate recognition data. sum ,The total traffic volume is averaged by calculating multiple time periods;

[0063] The second step is to calculate the traffic flow weight q for each time period according to the time information of the license plate recognition data. i ,i=1,2,...,I, where i represents the i-th hour;

[0064] The third step is to calculate the number of vehicles passing each hour in the simulation time. i =q i W sum , and assign the number of vehicles passing each hour to the simulation model.

[0065] S3, using the road network traffic status-action set {S k ,P k} and the constructed simulation model to train the Double DQN network and update the parameters of the target Q network in the Double DQN network;

[0066] Among them, the specific steps for updating the parameters of the target Q network in the Double DQN network are as follows:

[0067] S31, initialize the parameters ω of the current Q network and the parameters ω' of the target Q network Q', and clear the experience feedback

[0068] Put pool D;

[0069] S32, for any time {S k ,P k}, define the reward value r in Double DQN neural network learning k :

[0070]

[0071] Wherein w1, w2, w3 are reward conversion coefficients corresponding to different traffic information, which can be set according to specific traffic needs. In this embodiment, w1=2, w2=1, w3=0.5 are set. C is the number of signal light conversions. 0 means that the phase of the signal light has not changed compared with the previous moment, and 1 means that it has changed.

[0072] S33, the current moment {S k ,P k} and reward r k Put it into experience pool D;

[0073] S34, taking sample information from the experience replay pool, and combining it with the information at the current moment, calculating the ω value of the current Q network Q, which is the γ discount factor, which is set to 0.99 in this embodiment:

[0074]

[0075] S35, use the mean square error loss function L and update the Q' parameter ω' of the target Q network through back propagation of the neural network:

[0076]

[0077] Among them, m is the number of sample information taken from the experience pool in one training, a′ is the optimal action at the next moment, and j is the number of iterations;

[0078] S36, continuously updating the neural network parameters until the maximum number of iterations is reached.

[0079] S4: Based on the trained deep neural network, the road network signal light phase P required at the intersection at the next moment k+1 for:

[0080] P k+1 =argmax(Q'(S k ,P k ;ω')),

[0081] Where Q'(S k ,P k ; ω') is the Q' value of the target Q network, and the arg max function is Q'(S k ,P k ; ω') is the value of the variable when it reaches its maximum value, and ω' is the parameter of the target Q network.

[0082] The present invention first divides the lanes of the double intersection into sections, collects the vehicle traffic information and phase information in the double intersection in units of sections, and obtains the training data set of Double DQN and convolutional neural network after certain data preprocessing. Based on the data set, a simulation environment is built to train the neural network. The optimal timing scheme of the traffic lights of the road network can be obtained through the real-time vehicle information and the trained neural network. The traffic light controller finally obtained by the invention can well handle the strong dynamic characteristics of the urban road network.

[0083] See also Figure 2 This embodiment takes a two-way four-lane double-intersection road network built on the micro-simulation software SUMO as an example, and uses high-quality checkpoint passing data to fit the vehicle following model parameters and lane changing model parameters.

[0084] Taking the traffic micro-simulation software SUMO as an example, the above method was used to obtain a double-intersection traffic signal control strategy based on Double DQN and convolutional neural network. As a result, the average waiting time of vehicles was reduced by 20%.

[0085] The present invention obtains vehicle traffic segmentation information in real time and integrates it into a data set for training Double DQN and convolutional neural networks. The trained neural networks are then used to control the traffic lights at intersections, thereby alleviating traffic congestion.

[0086] The above description is about the excellent effect shown by an embodiment of the present invention. Obviously, the present invention is not only suitable for the above embodiment, but also can be implemented with various changes without departing from the basic spirit of the present invention and without exceeding the content involved in the essential content of the present invention.

Claims

1. A double intersection traffic signal control method based on Double DQN and convolutional neural network, the specific steps are as follows: S1. Divide the lanes of two adjacent intersections into sections, collect the road network vehicle traffic information data between the two intersections in sections, and collect the road network image information data M at the same time. k and road network signal light phase information data P k ; S2, preprocess the data in step S1, and input the data into the convolutional neural network to obtain the road network traffic state-action set {S k ,P k }, where S k The road network traffic state vector is formed by connecting the road network image information data and the road network vehicle traffic information data, and a simulation model of the traffic simulation platform is built; S3, using the road network traffic status-action set {S k ,P k } and the constructed simulation model to train the Double DQN network and update the parameters of the target Q network in the Double DQN network; The specific steps for updating the parameters of the target Q network in the Double DQN network are as follows: S31, initialize the parameters ω of the current Q network and the parameters ω' of the target Q network Q', and clear the experience replay pool D; S32, for any time {S k ,P k }, define the reward value r in Double DQN neural network learning k : Among them, w1, w2, w3 are the reward conversion coefficients corresponding to different traffic information, C is the number of signal light conversions, 0 means that the phase of the signal light has not changed compared with the previous moment, and 1 means that it has changed, L k represents the sequence set of the number of vehicles on the road section; V k Represents the average speed sequence set of vehicles on the road section; S33, the current moment {S k ,P k } and reward r k Put it into experience pool D; S34, take out sample information from the experience replay pool, and combine it with the information at the current moment to calculate the ω value of the current Q network Q, where γ is the discount factor: S35, use the mean square error loss function L and update the Q' parameter ω' of the target Q network through back propagation of the neural network: in, m is the number of sample information taken from the experience pool in one training, a′ is the optimal action at the next moment, and j is the number of iterations; S36, continuously updating the neural network parameters until the maximum number of iterations is reached; S4: Based on the trained deep neural network, the road network signal light phase P required at the intersection at the next moment k+1 for: P k+1 =argmax(Q'(S k ,P k ;ω')), Where Q'(S k ,P k ; ω') is the Q' value of the target Q network, and the arg max function is Q'(S k ,P k ; ω') is the value of the variable when it reaches its maximum value, and ω' is the parameter of the target Q network.

2. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 1, characterized in that: The steps for acquiring road network vehicle traffic information data in step S1 are as follows: S11, according to the characteristics of traffic flow data, the data sequence is processed into discrete time data, the discrete time step is unified, and k = 1, 2, ..., K is used to represent it, where k is the kth moment; S12, for the road network of two adjacent intersections, firstly divide each lane entering the intersection into several small segments of equal length, and use n = 1, 2, ..., N to number the number of segments, where n represents the nth segment. For the nth segment at time k, the road network vehicle traffic information data is expressed as {l n,k ,v n,k }, where l n,k Indicates the number of vehicles on the road section at the current moment, v n,k Indicates the average speed of vehicles on the road section at the current moment; S13, after collecting the data of all road sections, for the data of the two intersections, two vectors are used to describe the road network vehicle traffic information data {L k ,V k }, where L k = {l n,k :n=1,2,...,N},V k = {v n,k :n=1,2,...,N}.

3. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 1, characterized in that: The road network image information data is represented by M k Indicates that M k ={m n,k :n=1,2,...,N C }, for two adjacent intersections, each camera in the road network uses n = 1, 2, ..., N C Number, n represents the nth camera, for the nth camera at time k, the image information data of a single camera is represented as {m n,k }.

4. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 1, characterized in that: The road network signal light phase information data is P k Indicates that P k ={p n,k :n=1,2,...,N D }, for two adjacent intersections, each signal light in the network is n=1,2,...,N D Number, n represents the nth signal light, for the nth signal light at time k, its signal light phase information data is expressed as {p n,k }.

5. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 2, characterized in that: The data processing steps of step S2 are as follows: S21, delete abnormal data; S22, obtain the intersection vehicle information in sections, and then use the obtained intersection road network vehicle traffic information data as L k -V k The structure is combined into the final road network vehicle traffic information data vector; S23, collect the road network image information data M k Input into the convolutional neural network to output image features It is combined with the road network vehicle traffic information data vector {L k ,V k }Merge into state set Then according to the road network signal light phase information data P k , forming the final road network traffic state-action set {S k ,P k }.

6. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 5, characterized in that: The abnormal data in step S21 include that the number of vehicles exceeds the maximum total number of vehicles that the lane can accommodate and the average speed of vehicles exceeds the speed limit requirement of the urban road.

7. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 5, characterized in that: Image features of the road network image information data extracted by the convolutional neural network in step S23 It contains the pixel position of each vehicle's center point in the graph and the number of vehicles in the road section, indicating the relative position of each vehicle in the road network and the current congestion level of the road section.

8. The double intersection traffic signal control method based on Double DQN and convolutional neural network according to claim 2, characterized in that: In step S2, a simulation model of the traffic simulation platform is built. First, the road network is accurately modeled by importing the target vector map, and then the traffic flow is simulated. The lane traffic flow weight change information of each entrance of the double intersection every hour is obtained by the OD estimation method based on the license plate recognition data. The specific steps are as follows: The first step is to obtain the average total traffic flow W within the required simulation time based on the vehicle information of the license plate recognition data. sum ,The total traffic volume is averaged by calculating multiple time periods; The second step is to calculate the traffic flow weight q for each time period according to the time information of the license plate recognition data. i ,i=1,2,...,I, where i represents the i-th hour; The third step is to calculate the number of vehicles passing each hour in the simulation time. i =q i W sum , and assign the number of vehicles passing each hour to the simulation model.

Images