Multi-traffic-signal-lamp intelligent control method based on two-stage circulation communication

Through the dual-stage cyclic communication framework and attention mechanism recurrent neural network, information sharing and intelligent control between traffic lights at multiple intersections is realized, and the bottleneck problem of coordinated control of traffic lights at multiple intersections is solved, and the traffic efficiency and signal lights are improved. control accuracy.

CN119091649APending Publication Date: 2024-12-06NORTHWESTERN POLYTECHNICAL UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411269915.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-11
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The prior art ignores the coordinated interaction between adjacent intersections in the control of traffic lights from multiple intersections, making it difficult to effectively solve the traffic congestion problem, and the high sensitivity of real-time data affects the timeliness and accuracy of control.

Method used

Using an intelligent control method based on dual-stage cyclic communication, a dual-stage cyclic communication framework is built through attention mechanism, cyclic gate unit and reinforcement learning, information transmission between traffic lights and neural network parameters are realized, and an intelligent control model is generated.

Benefits of technology

It effectively reduces the average waiting time of vehicles in the road network, improves the traffic capacity of intersections, alleviates traffic congestion, and realizes intelligent and efficient traffic light control.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091649A_ABST
    Figure CN119091649A_ABST
Patent Text Reader

Abstract

The invention provides a multi-traffic-signal-lamp intelligent control method based on two-stage circulation communication, and the method comprises the steps: S1, constructing a state coding module, and converting original traffic signal lamp data into state codes through the state coding module; s2, constructing a circulation gating module based on an attention mechanism, and converting the state code into state representation through the circulation gating module based on the attention mechanism; s3, constructing a two-stage circulation communication framework, realizing information transmission among the traffic signal lamps through the two-stage circulation communication framework, and transmitting state representation generated after communication is completed to the reinforcement learning network; and S4, training the reinforcement learning network, updating the neural network parameters of each traffic signal lamp, and obtaining a traffic signal lamp intelligent control model. The method can achieve the intelligent control of a plurality of traffic lights between the intersections, effectively reduces the average waiting time of vehicles in a traffic network, improves the traffic capacity of the intersections, and alleviates the traffic congestion.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of reinforcement learning intelligent collaborative control, and specifically relates to a multi-traffic signal light intelligent control method based on two-stage cyclic communication. Background Art

[0002] Transportation systems play an important role in society, providing mobility and accessibility for people and goods. The growing population and urbanization have placed an increasing burden on the existing transportation network, exacerbating traffic congestion. In order to ensure the safety and stable traffic flow of pedestrians and vehicles, traffic signal control and management at road intersections is a challenging problem in the transportation system.

[0003] Traffic signal control is an important aspect of urban traffic management, which can significantly affect traffic flow and improve road safety. Traditional methods control traffic signals through fixed time or manually set heuristics, but these methods are predefined and cannot adapt to the high dynamics and high uncertainty in traffic operation. In the field of intelligent control, reinforcement learning has shown strong control capabilities. However, most existing reinforcement learning methods consider one intersection alone, ignoring the interaction between traffic lights at adjacent intersections, and can only achieve suboptimal performance; secondly, in algorithms that consider multiple intersections, the lack of coordination between multiple traffic lights at each intersection due to the inability to achieve efficient communication.

[0004] CN114495506A discloses a multi-intersection signal light control system and method based on traffic flow prediction and reinforcement learning, which reduces the scale of data communication and deployment costs through joint control. CN116758767A discloses a traffic light control method based on multi-strategy reinforcement learning, which uses a classification width learning system and an evaluation width learning system to determine the complexity of the traffic system and calculate the optimal control strategy. However, these two methods are highly sensitive to real-time data, which may affect the timeliness and accuracy of traffic light control. CN116863729A discloses a multi-agent road traffic signal control method based on a deep reinforcement learning algorithm, which takes into account multiple action spaces and new state spaces as well as new reward functions. However, the coordination between agents is very complex and requires a high degree of communication and synchronization. CN117409593A discloses a multi-intersection traffic signal control method based on deep reinforcement learning, which involves deep learning target detection technology, requires a large amount of labeled data, and the training process is time-consuming. CN117809469A discloses a traffic light timing control method and system based on deep reinforcement learning, which improves learning stability and accuracy of strategy evaluation by introducing experience replay and target Q value network, but is sensitive to network structure and parameter selection, resulting in unstable learning process.

[0005] With the continuous development of machine learning, deep learning and reinforcement learning, advanced technologies such as attention mechanism, recurrent neural network, deep deterministic strategic gradient have emerged, providing new ideas for breaking the bottleneck of intelligent control of traffic lights. In view of the above background, the present invention is committed to using communication-based reinforcement learning to realize intelligent control of multiple traffic lights to solve the congestion problem between multiple intersections. Summary of the invention

[0006] In order to overcome the above technical problems, the purpose of the present invention is to provide a multi-traffic signal light intelligent control method based on two-stage cyclic communication, by using attention mechanism, cyclic gating unit and reinforcement learning and other technologies, a two-stage cyclic communication framework is constructed to intelligently control traffic lights. The intelligent control method proposed in the present invention can be used in scenarios where multiple intersections in an urban road network affect each other and traffic flow is complex and changeable, and can exchange information with other intersections in real time, thereby learning global traffic status information and alleviating traffic congestion.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is: a multi-traffic signal light intelligent control method based on two-stage cyclic communication, comprising the following steps:

[0008] Step S1: construct a state encoding module, and convert the original traffic light data into state encoding through the state encoding module;

[0009] Step S2: construct a recurrent gating module based on the attention mechanism, and convert the state encoding into state representation through the recurrent gating module based on the attention mechanism;

[0010] Step S3: construct a two-stage loop communication architecture, use the two-stage loop communication architecture to transmit information between traffic lights, and transmit the state representation generated after the communication is completed to the reinforcement learning network;

[0011] Step S4: Train the reinforcement learning network, update the neural network parameters of each traffic light, and obtain the traffic light intelligent control model.

[0012] Preferably, step S1 constructs a state coding module, and converts the original traffic light data into a state code through the state coding module, specifically:

[0013] Step S1.1: Define the observation space of traffic light data in represents the observation of the i-th traffic light at time t;

[0014] Step S1.2: For each traffic light, construct a feature encoding module F consisting of a linear layer and a leaky linear rectifier activation function. enc ;

[0015] Step S1.3: Through the feature encoding module of the i-th traffic light The observation data O of the i-th traffic light t Convert to the state code of the i-th traffic light

[0016] Preferably, step S2 constructs a recurrent gating module based on an attention mechanism, and converts the state encoding into a state representation through the recurrent gating module based on the attention mechanism, specifically:

[0017] Step S2.1: Build an attention-based recurrent gating module F for each traffic light att , including the reset gate f r , update gate f u , cell state gate f c , attention gate f a and output gate f o ,Each gate consists of a fully connected layer;

[0018] Step S2.2: For the i-th traffic light, encode the state s t Considered as the current attention recurrent gating module F att The input x t , by resetting gate f r The cell state c at time t-1 t-1 and the current input x t Convert to reset information t , the calculation process is as follows:

[0019] r t =f r (c t-1 ,x t )

[0020] =sigmoid(W r ·[c t-1 ,x t ])

[0021] Among them, W r Represents f r The conversion matrix, sigmod(·) represents the sigmoid activation function;

[0022] Step S2.2: By updating the gate f u c t-1 and x t Convert to update information t , the calculation process is as follows:

[0023] u t =f u (ct-1 ,x t )

[0024] =sigmoid(W z ·[c t-1 ,x t ])

[0025] Among them, W z Represents f u The transformation matrix of

[0026] Step S2.3: Gate the cell state f c c t-1 、x t 、r t and u t Transition to cell state c t , the calculation process is as follows:

[0027] c t =f c (c t-1 ,x t ,r t ,u t )

[0028] =(1-u t )·c t-1 +tanh(W c ·[r t ·c t-1 ,x t ])

[0029] Where W c Represents f c The transformation matrix of , tanh(·) represents the hyperbolic tangent activation function;

[0030] Step S2.4: Pass the attention gate f a The state vector h at time t-1 t-1 and x t Converted to attention value a t , the calculation process is as follows:

[0031] a t =f a (x t ,h t-1 )

[0032] =softmax(W a ·[x t ,h t-1 ])

[0033] Among them, W a Represents f aThe transformation matrix of , softmax(·) represents the normalized exponential activation function;

[0034] Step S2.5: Through the output gate f o A t 、c t and h t-1 Convert to F att The output h at time t t , also called the state vector at time t, the calculation process is as follows:

[0035] h t =f o (a t ,c t ,h t-1 )

[0036] =tanh(W o ·[a t ,c t ,h t-1 ])

[0037] Among them, W o Represents f o The transformation matrix.

[0038] Preferably, step S3 constructs a two-stage loop communication architecture, through which information is transmitted between traffic lights, and the state representation generated after the communication is completed is transmitted to the reinforcement learning network, specifically:

[0039] Step S3.1: The state vector generated by the i-th traffic light The state code of the i+1th traffic light at time t and the cell state of the i+1th traffic light at time t-1 Attention-based recurrent gating module passed to the i+1th traffic light Get the state vector generated by the i+1th traffic light based on i and the cell state of the i+1th traffic light at time t based on i. It is expressed as:

[0040]

[0041] Step S3.2: transmitting information in the order of traffic light numbers (i+1, ..., n, 1, ..., i-1);

[0042] The way of information transmission is: the state vector generated by the k-1th traffic light based on i is The state code of the kth traffic light at time t and the cell state of the kth traffic light at time t-1 Attention-based recurrent gating module passed to the kth traffic light Get the state vector generated by k traffic lights based on i and the detailed state of the kth traffic light at time t based on i It is expressed as:

[0043]

[0044] Where k represents the number of the current traffic light and is not i, and n represents the total number of traffic lights. In this way, the state vector generated by the i-1th traffic light based on i is obtained.

[0045] Preferably, step S4 trains a reinforcement learning network, updates the neural network parameters of each traffic light, and obtains a traffic light intelligent control model, specifically:

[0046] Step S4.1: For each traffic light i, initialize the policy network and the target network, where the policy network and the target network are composed of three fully connected layers respectively;

[0047] Step S4.2: Set the state vector of the i-th traffic light As the input of the policy network, we get the output action a of the traffic light t ;

[0048] Step S4.3: After all lights perform actions, we get the reward r and the observation o of all lights at the next moment. t ',o t 'After the two-stage loop communication architecture, the state vector h is obtained t ';

[0049] Step S4.4: Set the value network objective function L c (·), which means as follows:

[0050]

[0051] Where γ is the discount factor;

[0052] Step S4.5: Setting the policy network objective function It is expressed as follows:

[0053]

[0054] Step S4.6: Update the strategy network and the value network, repeat the training process until the training is completed, and generate a traffic light intelligent control model.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] (1) The present invention uses a recursive network based on a gated attention mechanism to communicate traffic lights at various intersections. In the first level of recursion, the hidden states of all traffic lights are regarded as a sequence. In the second level of recursion, the hidden state sequence of each agent is integrated into the cell state. The communication message and memory are separated in the recursive model, so that the proposed communication flow can adapt to the changing communication objects under the condition of limited communication, and the information contained in the communication result is fair and equal to each traffic light.

[0057] (2) The present invention effectively reduces the average waiting time of vehicles in the road network, improves the traffic capacity of intersections, alleviates traffic congestion, and realizes intelligent and efficient traffic signal light control. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 is a flow chart of a method for intelligently controlling multiple traffic lights based on two-stage cyclic communication of the present invention;

[0059] Figure 2 Schematic diagram of a recurrent gating module based on an attention mechanism in the method of the present invention;

[0060] Figure 3 It is a schematic diagram of a two-stage loop communication architecture in the method of the present invention;

[0061] Figure 4 It is a schematic diagram of the change of average travel time of the method of the present invention and other two control methods during the training process. DETAILED DESCRIPTION

[0062] Unless otherwise specified, any feature disclosed throughout this specification may be replaced by an alternative feature that is equivalent or has a similar purpose. That is, each feature is only an example in a series of equivalent or similar features.

[0063] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention will be described in more detail below in conjunction with the drawings of the specification. It should be noted that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0064] Example

[0065] Based on previous work, this paper innovatively proposes a multi-traffic signal light intelligent control method based on two-stage cyclic communication, aiming to solve the problem of coordinated control of traffic lights at multiple intersections and improve road traffic efficiency. The method proposed in this paper is based on a recurrent neural network with an attention mechanism. Through a two-stage cyclic communication architecture, the traffic lights at each intersection are allowed to receive and integrate traffic status information from other intersections, thereby better perceiving the global traffic conditions.

[0066] Reference Figure 1 According to the present invention, a multi-traffic signal light intelligent control method based on two-stage cyclic communication includes:

[0067] Step S1 constructs a state coding module, and converts the original traffic light data into state coding through the state coding module, specifically:

[0068] Step S1.1: Define the observation space of traffic light data in represents the observation of the i-th traffic light at time t.

[0069] Step S1.2: For each traffic light, construct a feature encoding module F consisting of a linear layer and a leaky linear rectifier activation function. enc .

[0070] Step S1.3: Through the feature encoding module of the i-th traffic light The observation data O of the i-th traffic light t Convert to the state code of the i-th traffic light

[0071] Reference Figure 2 , step S2 constructs a recurrent gating module based on the attention mechanism, and converts the state encoding into a state representation through the recurrent gating module based on the attention mechanism, specifically:

[0072] Step S2.1: Build an attention-based recurrent gating module F for each traffic light att , including the reset gate f r , update gate f u , cell state gate f c , attention gate f a and output gate f o , each gate consists of a fully connected layer.

[0073] Step S2.2: For the i-th traffic light, encode the state s t Considered as the current attention recurrent gating module F att The input x t , by resetting gate f rThe cell state c at time t-1 t-1 and the current input x t Convert to reset information t , the calculation process is as follows:

[0074] r t =f r (c t-1 ,x t )

[0075] =sigmoid(W r ·[c t-1 ,x t ])

[0076] Among them, W r Represents f r The conversion matrix of , sigmod(·) represents the sigmoid activation function.

[0077] Step S2.2: By updating the gate f u c t-1 and x t Convert to update information t , the calculation process is as follows:

[0078] u t =f u (c t-1 ,x t )

[0079] =sigmoid(W z ·[c t-1 ,x t ])

[0080] Among them, W z Represents f u The transformation matrix.

[0081] Step S2.3: Gate the cell state f c c t-1 、x t 、r t and u t Transition to cell state c t , the calculation process is as follows:

[0082] c t =f c (c t-1 ,x t ,r t ,u t )

[0083] =(1-u t )·c t-1 +tanh(Wc ·[r t ·c t-1 ,x t ])

[0084] Where W c Represents f c , tanh(·) represents the hyperbolic tangent activation function.

[0085] Step S2.4: Pass the attention gate f a The state vector h at time t-1 t-1 and x t Converted to attention value a t , the calculation process is as follows:

[0086] a t =f a (x t ,h t-1 )

[0087] =softmax(W a ·[x t ,h t-1 ])

[0088] Among them, W a Represents f a , and softmax(·) represents the normalized exponential activation function.

[0089] Step S2.5: Through the output gate f o A t 、c t and h t-1 Convert to F att The output h at time t t , also called the state vector at time t, the calculation process is as follows:

[0090] h t =f o (a t ,c t ,h t-1 )

[0091] =tanh(W o ·[a t ,c t ,h t-1 ])

[0092] Among them, W o Represents f o The transformation matrix.

[0093] Reference Figure 3, step S3 constructs a two-stage loop communication architecture, through which information is transmitted between traffic lights, and the state representation generated after the communication is completed is transmitted to the reinforcement learning network, specifically:

[0094] Step S3.1: The state vector generated by the i-th traffic light The state code of the i+1th traffic light at time t and the cell state of the i+1th traffic light at time t-1 Attention-based recurrent gating module passed to the i+1th traffic light Get the state vector generated by the i+1th traffic light based on i and the cell state of the i+1th traffic light at time t based on i. It is expressed as:

[0095]

[0096] Step S3.2: Information is transmitted in the order of traffic light numbers (i+1, ..., n, 1, ..., i-1). The information transmission method is: the state vector generated by the k-1th traffic light based on i is The state code of the kth traffic light at time t and the cell state of the kth traffic light at time t-1 Attention-based recurrent gating module passed to the kth traffic light Get the state vector generated by k traffic lights based on i and the detailed state of the kth traffic light at time t based on i It is expressed as:

[0097]

[0098] Where k represents the number of the current traffic light and is not i, and n represents the total number of traffic lights. Then, the state vector generated by the i-1th traffic light based on i is obtained in this way:

[0099] Step S4 trains the reinforcement learning network, updates the neural network parameters of each traffic light, and obtains the intelligent control model of the traffic light, which is specifically:

[0100] Step S4.1: For each traffic light i, initialize the policy network and the target network, where the policy network and the target network are composed of three fully connected layers respectively.

[0101] Step S4.2: Set the state vector of the i-th traffic light As the input of the policy network, we get the output action a of the traffic light t .

[0102] Step S4.3: After all lights perform actions, we get the reward r and the observation o of all lights at the next moment. t ',o t 'After the two-stage loop communication architecture, the state vector h is obtained t '.

[0103] Step S4.4: Set the value network objective function L c (·), which means as follows:

[0104]

[0105] Here, γ is the discount factor.

[0106] Step S4.5: Setting the policy network objective function It is expressed as follows:

[0107]

[0108] Step S4.6: Update the strategy network and the value network, repeat the training process until the training is completed, and generate a traffic light intelligent control model.

[0109] After executing steps 1 to 4, the experimental verification is carried out through the traffic simulator platform CityFlow. In a certain city traffic data, the road network contains 34 intersections. The traffic flow is generated by the traffic trajectories collected by 80 red light cameras and 16 surveillance cameras within one hour. Within 3600 seconds after data cleaning, there are 1775 valid driving records. The method proposed in the present invention is compared with the traditional method, wherein the comparison method adopted is the CoLight method that uses the graph attention network (GAT) to enhance the spatial communication between adjacent intersections, and the PressLight method that optimizes each intersection by defining the pressure value. The vehicle traffic flow dataset is described by three dimensions: the starting position o, the departure time t, and the end position d. The starting position and the end position represent the position of the vehicle on the road network, and the vehicle's turning choice is based on a 10% probability of turning left, a 60% probability of going straight, and a 30% probability of turning right.

[0110] The experimental parameters of the method proposed in the present invention are as follows: learning rate is 0.0001, discount factor γ is 0.95, sample batch is 512, and the number of training rounds is 100. The average travel time of the three methods during the training process changes as follows Figure 4The final experimental results are as follows: In the three methods proposed in the present invention, CoLight and PressLight, the average driving time of each cycle is 572 seconds, 1574 seconds and 2220 seconds, and the average number of vehicles passing through each cycle is 2377, 1257 and 841 respectively.

[0111] The above embodiments are only preferred embodiments of the present invention, and are not intended to limit the scope of the present invention. For those skilled in the art, without departing from the spirit of the present invention, the present invention may be subjected to several modifications, improvements and equivalent substitutions, which shall all fall within the scope of protection defined in the claims of the present invention.

Claims

1. A multi-traffic signal light intelligent control method based on two-stage cyclic communication, characterized in that: include: Step S1: construct a state encoding module, and convert the original traffic light data into state encoding through the state encoding module; Step S2: construct a recurrent gating module based on the attention mechanism, and convert the state encoding into state representation through the recurrent gating module based on the attention mechanism; Step S3: construct a two-stage loop communication architecture, use the two-stage loop communication architecture to transmit information between traffic lights, and transmit the state representation generated after the communication is completed to the reinforcement learning network; Step S4: Train the reinforcement learning network, update the neural network parameters of each traffic light, and obtain the traffic light intelligent control model.

2. The multi-traffic signal light intelligent control method based on two-stage cyclic communication according to claim 1 is characterized in that: The step S1 comprises: Step S1.1: Define the observation space of traffic light data in represents the observation of the i-th traffic light at time t; Step S1.2: For each traffic light, construct a feature encoding module F consisting of a linear layer and a leaky linear rectifier activation function. enc ; Step S1.3: Through the feature encoding module of the i-th traffic light The observation data O of the i-th traffic light t Convert to the state code of the i-th traffic light 3. The multi-traffic signal light intelligent control method based on two-stage cyclic communication according to claim 1 is characterized in that: The step S2 comprises: Step S2.1: Build an attention-based recurrent gating module F for each traffic light att , including the reset gate f r , update gate f u , cell state gate f c , attention gate f a and output gate f o ,Each gate consists of a fully connected layer; Step S2.2: For the i-th traffic light, encode the state s t As the current attention loop gating module F att The input x t , by resetting gate f r The cell state c at time t-1 t-1 and the current input x t Convert to reset information t , the calculation process is as follows: r t =f r (c t-1 ,x t ) =sigmoid(W r ·[c t-1 ,x t ]) Among them, W r Represents f r The conversion matrix, sigmod(·) represents the sigmoid activation function; Step S2.2: By updating the gate f u c t-1 and x t Convert to update information t , the calculation process is as follows: u t =f u (c t-1 ,x t ) =sigmoid(W z ·[c t-1 ,x t ]) Among them, W z Represents f u The transformation matrix of Step S2.3: Gate the cell state f c c t-1 、x t 、r t and u t Transition to cell state c t , the calculation process is as follows: c t =f c (c t-1 ,x t ,r t ,u t ) =(1-u t )·c t-1 +tanh(W c ·[r t ·c t-1 ,x t ]) Where W c Represents f c The transformation matrix of , tanh(·) represents the hyperbolic tangent activation function; Step S2.4: Pass the attention gate f a The state vector h at time t-1 t-1 and x t Converted to attention value a t , the calculation process is as follows: a t =f a (x t ,h t-1 ) =softmax(W a ·[x t ,h t-1 ]) Among them, W a Represents f a The transformation matrix of , softmax(·) represents the normalized exponential activation function; Step S2.5: Through the output gate f o A t 、c t and h t-1 Convert to F att The output h at time t t , also called the state vector at time t, the calculation process is as follows: h t =f o (a t ,c t ,h t-1 ) =tanh(W o ·[a t ,c t ,h t-1 ]) Among them, W o Represents f o The transformation matrix.

4. The multi-traffic signal light intelligent control method based on two-stage cyclic communication according to claim 1 is characterized in that: The step S3 comprises: Step S3.1: The state vector generated by the i-th traffic light The state code of the i+1th traffic light at time t and the cell state of the i+1th traffic light at time t-1 Attention-based recurrent gating module passed to the i+1th traffic light Get the state vector generated by the i+1th traffic light based on i and the cell state of the i+1th traffic light at time t based on i It is expressed as: Step S3.2: transmitting information in the order of traffic light numbers (i+1, ..., n, 1, ..., i-1); The way of information transmission is: the state vector generated by the k-1th traffic light based on i is The state code of the kth traffic light at time t and the cell state of the kth traffic light at time t-1 Attention-based recurrent gating module passed to the kth traffic light Get the state vector generated by k traffic lights based on i and the detailed state of the kth traffic light at time t based on i It is expressed as: Wherein, k represents the number of the current traffic light and is not i, and n represents the total number of traffic lights; In this way, the state vector generated by the i-1th traffic light based on i is obtained 5. The multi-traffic signal light intelligent control method based on two-stage cyclic communication according to claim 1 is characterized in that: The step S4 comprises: Step S4.1: For each traffic light i, initialize the policy network and the target network, where the policy network and the target network are composed of three fully connected layers respectively; Step S4.2: Set the state vector of the i-th traffic light As the input of the policy network, we get the output action a of the traffic light t ; Step S4.3: After all lights perform actions, we get the reward r and the observation o′ of all lights at the next moment t , o′ t The state vector h′ is obtained through a two-stage loop communication architecture. t ; Step S4.4: Set the value network objective function L c (·), which means as follows: Where γ is the discount factor; Step S4.5: Setting the policy network objective function It is expressed as follows: Step S4.6: Update the strategy network and the value network, repeat the training process until the training is completed, and generate a traffic light intelligent control model.

Citation Information

Patent Citations

  • Traffic signal lamp control method based on multi-strategy reinforcement learning

    CN116758767A

  • Multi-agent road traffic signal control method based on deep reinforcement learning algorithm

    CN116863729A

  • Multi-intersection traffic signal control method based on deep reinforcement learning

    CN117409593A

  • Traffic signal lamp timing regulation and control method and system based on deep reinforcement learning

    CN117809469A