A traffic light control method and terminal based on neighborhood information fusion

Abstract

The application discloses a traffic light control method and a terminal based on neighborhood information fusion, and the method comprises the following steps: decomposing a target area traffic road network into a plurality of basic subnets; for each traffic light in the traffic road network, acquiring traffic flow information and traffic light state information of four sides of the traffic light, and carrying out digital processing on the traffic flow information and the traffic light information to obtain digital traffic flow state information; constructing a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion according to the subnets in the traffic road network and the obtained digital traffic flow state information; and based on the constructed multi-agent reinforcement learning traffic light control model, carrying out neighborhood model fusion, and controlling the traffic lights in the traffic road network according to the fused model. The application decomposes the traffic network into more basic subnets and proposes a neighborhood information fusion method, so that each traffic light only needs to consider the directly adjacent intersections, and the traffic light optimization efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of terminal applications, and in particular to a traffic light control method and terminal based on neighborhood information fusion. Background Technology

[0002] Intelligent traffic light control is a promising method for alleviating road congestion and improving road efficiency without requiring modifications or upgrades to existing urban road network infrastructure. However, how to dynamically optimize traffic light control strategies in real time to improve the efficiency of the road network remains a key research issue in the field of intelligent transportation.

[0003] In recent years, with the rapid development of artificial intelligence technology, traffic light control strategies based on reinforcement learning (especially multi-agent reinforcement learning) have received widespread attention and research. However, existing technologies and methods still have some problems: (1) Most existing reinforcement learning models for optimizing traffic network efficiency rely on the global information of the traffic network, resulting in the problem of state and action space explosion, especially when the traffic network is very large. (2) Existing multi-agent reinforcement learning traffic light control models suffer from low training efficiency.

[0004] Therefore, existing technologies still need improvement. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a traffic light control method and terminal based on neighborhood information fusion, in order to solve the problem of low efficiency of existing traffic light optimization based on reinforcement learning, which is in response to the shortcomings of the prior art.

[0006] The technical solution adopted by this invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides a traffic light control method based on neighborhood information fusion, the traffic light control method based on neighborhood information fusion comprising the following steps:

[0008] The traffic network of the target area is decomposed into multiple basic subnetworks; each subnetwork includes: the target intersection and adjacent intersections;

[0009] For each traffic light in the traffic network, traffic flow information and traffic light status information from all four sides of the traffic light are acquired, and the traffic flow information and traffic light information are digitized to obtain digitized traffic flow status information; wherein, the traffic flow information includes: vehicle position information and vehicle speed information;

[0010] Based on the subnets in the traffic network and the obtained digital traffic flow state information, a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion is constructed.

[0011] Based on the constructed multi-agent reinforcement learning traffic light control model, neighborhood model fusion is performed, and the traffic lights in the traffic network are controlled according to the fused model.

[0012] In one implementation, the multi-agent reinforcement learning traffic light control model includes: a convolutional neural network, a recurrent neural network, and a graph neural network;

[0013] The construction of a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion includes:

[0014] For each traffic light at the target intersection in the subnet, the digitized traffic flow state information is converted into a position matrix and a velocity matrix, and then processed using a convolutional neural network to obtain the physical characteristics of the traffic flow.

[0015] For each traffic light at the target intersection in the subnet, its historical state is collected and processed using a recurrent neural network to obtain traffic flow dynamics characteristics;

[0016] For each target intersection traffic light in the subnet, collect the current traffic light and the historical state information of adjacent intersections, and process them using a graph neural network to obtain traffic flow interaction features;

[0017] The traffic flow physical features, traffic flow interaction features, and traffic light status are concatenated in vector form and combined with traffic light action information to obtain the value of the current action in the current traffic flow status and traffic light status.

[0018] Determine the optimal action for the traffic light under the current conditions.

[0019] In one implementation, the physical characteristics of the traffic flow are:

[0020]

[0021] in, and Let be the position matrix and velocity matrix at time t, respectively;

[0022] φ i and These are the model parameters and output of the convolutional neural network CNNi, respectively.

[0023] In one implementation, the traffic flow dynamics characteristics are:

[0024]

[0025] in, This is the length of the coupling waiting queue;

[0026] c represents the historical time span;

[0027] and These are the recurrent neural networks (RNNs). i The model parameters and output.

[0028] In one implementation, the traffic flow interaction feature is:

[0029]

[0030] in, ψ i and These are the graph neural networks (GNNs). i The model parameters and output.

[0031] In one implementation, the traffic light control model based on the constructed multi-agent reinforcement learning model is used to fuse neighborhood models, and the traffic lights in the traffic network are controlled according to the fused model, including:

[0032] Initialize the reinforcement learning model parameters and model training hyperparameters for each traffic light;

[0033] Choose the traffic light action based on a greedy strategy;

[0034] The reward is designed based on the length of the vehicle waiting sequence after the traffic light action, and an experience sample is obtained and stored in the local dataset.

[0035] Based on the obtained empirical samples, several samples are randomly selected from the local dataset, and the parameters of the deep Q network are optimized using the gradient descent method to minimize the loss function;

[0036] For each traffic light, the current traffic light is controlled based on the local deep Q-network model, and the model parameters of adjacent traffic lights in the subnet are collected. The model parameters of the current traffic light are updated through the neighborhood model aggregation method.

[0037] In one implementation, the model parameters of the current traffic light are:

[0038]

[0039] Among them, K i For subnet Number of traffic lights in China;

[0040] θ′ j These are the parameters for the neighborhood model;

[0041] The model aggregates weight vectors, the dimension of which depends on the subnet. Size.

[0042] In one implementation, updating the model parameters of the current traffic light then includes:

[0043] Determine if the termination condition is met:

[0044] If the current iteration count is greater than the total number of iterations, then stop updating;

[0045] If the current iteration number is less than or equal to the total iteration number, then continue to select traffic light actions according to the greedy strategy until the termination condition is met.

[0046] In a second aspect, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a traffic light control program based on neighborhood information fusion, and the traffic light control program based on neighborhood information fusion, when executed by the processor, is used to implement the traffic light control method based on neighborhood information fusion as described in the first aspect.

[0047] Thirdly, the present invention provides a storage medium storing a traffic light control program based on neighborhood information fusion, wherein the traffic light control program based on neighborhood information fusion, when executed by a processor, is used to implement the traffic light control method based on neighborhood information fusion as described in the first aspect.

[0048] The present invention, by employing the above technical solution, has the following effects:

[0049] This invention decomposes the traffic network into more basic subnets and proposes a neighborhood information fusion method. Each traffic light only needs to consider the intersections directly adjacent to it, which greatly reduces the dimensionality of the state and action space and avoids the "curse of dimensionality" problem. Furthermore, a neighborhood state fusion method is proposed. When collecting neighborhood information, only the lane waiting queue lengths of adjacent intersections coupled with the target traffic intersection are collected, which greatly reduces the computational complexity and improves the model performance. At the same time, each traffic light only needs to communicate with the traffic lights connected to it and share model parameters, without the need for coordination by a central server, which effectively avoids communication congestion and single point of failure problems. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0051] Figure 1 This is a flowchart of a traffic light control method based on neighborhood information fusion in one implementation of the present invention.

[0052] Figure 2 This is a schematic diagram of the traffic network and subnet decomposition in one implementation of the present invention.

[0053] Figure 3 This is a flowchart of the neighborhood state fusion process in one implementation of the present invention.

[0054] Figure 4 This is a schematic diagram of a coupled waiting queue in one implementation of the present invention.

[0055] Figure 5 This is a functional schematic diagram of the terminal in one implementation of the present invention.

[0056] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0058] Exemplary methods

[0059] Traffic light control strategies based on reinforcement learning (especially multi-agent reinforcement learning) still have some problems: (1) Most existing reinforcement learning models for optimizing traffic network efficiency rely on global information of the traffic network, resulting in the problem of state and action space explosion, especially when the traffic network is very large. (2) Existing multi-agent reinforcement learning traffic light control models have the problem of low training efficiency.

[0060] To address the aforementioned technical problems, this invention provides a multi-agent reinforcement learning-based traffic light control method based on neighborhood information fusion to improve traffic network efficiency. This method includes neighborhood state fusion and neighborhood model fusion. Its main feature is that it decomposes the traffic network into smaller, more fundamental subnetworks. Each traffic light only needs to communicate with its directly adjacent traffic lights, sharing traffic flow and model information. Through neighborhood information fusion, a multi-agent reinforcement learning-based traffic light control model is trained, thereby improving traffic network efficiency.

[0061] like Figure 1 As shown, this embodiment of the invention provides a traffic light control method based on neighborhood information fusion. The traffic light control method based on neighborhood information fusion includes the following steps:

[0062] Step S100: Decompose the traffic network of the target area into multiple basic sub-networks.

[0063] In this embodiment, the traffic light control method based on neighborhood information fusion is applied to a terminal, which includes, but is not limited to, devices such as computers and mobile terminals; specifically, the terminal is a terminal device used to control the operation of each traffic light (i.e., traffic light signals) in the traffic network.

[0064] When implementing the traffic light control method based on neighborhood information fusion, it is necessary to obtain urban traffic network information. The urban traffic network information can be road network information composed of certain main roads and other roads connected to them, or it can be road network information of a certain part of the urban administrative area. Based on the urban traffic network information, traffic light information in the urban traffic network information can be obtained. The obtained traffic light information is traffic light information within a preset area. The preset area can be the jurisdiction area of ​​multiple traffic lights adjacent to a certain traffic light, or it can be a designated range area centered on a certain traffic light.

[0065] like Figure 2 As shown in this embodiment, after obtaining urban traffic network information, the target area traffic network can be decomposed into many basic subnetworks, each of which consists of a target intersection and an intersection directly adjacent to it.

[0066] Unlike existing multi-agent reinforcement learning traffic light control strategies based on a centralized training and decentralized execution architecture, this invention proposes a neighborhood information fusion method by decomposing the traffic network into more basic subnets. Each traffic light only needs to consider the intersections directly adjacent to it, which greatly reduces the dimensionality of the state and action space and avoids the "curse of dimensionality" problem.

[0067] like Figure 1 As shown, in one implementation of the present invention, the traffic light control method based on neighborhood information fusion further includes the following steps:

[0068] Step S200: For each traffic light in the traffic network, acquire the traffic flow information and traffic light status information on all four sides of the traffic light, and digitize the traffic flow information and traffic light information to obtain digitized traffic flow status information.

[0069] In this embodiment, for each traffic light in the traffic network, a corresponding camera, radar, and inductive loop can be identified. The traffic flow status of the area governed by the traffic light is obtained based on the camera, radar, and inductive loop. The traffic flow status includes traffic light status information (e.g., red light, green light, and yellow light) and traffic flow information around the traffic light. Furthermore, the traffic flow information includes vehicle location information and vehicle speed information. After obtaining the traffic status, the traffic light status information and the traffic flow status information around the traffic light are digitized to obtain digitized traffic flow status information.

[0070] like Figure 1 As shown, in one implementation of the present invention, the traffic light control method based on neighborhood information fusion further includes the following steps:

[0071] Step S300: Based on the subnets in the traffic network and the obtained digital traffic flow state information, construct a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion.

[0072] In this embodiment, based on the subnets in the traffic network in step S100 and the digital traffic flow state information obtained in step S200, a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion can be constructed, which can effectively extract the spatiotemporal coupling information of traffic flow and improve model performance. Furthermore, based on this model, through neighborhood model fusion, each traffic light can achieve cooperative training without relying on server coordination, thereby improving model training efficiency.

[0073] In one implementation of this embodiment, step S300 specifically includes the following steps:

[0074] Step S310: For each subnet target intersection traffic light, the digitized traffic flow state information is converted into a position matrix and a velocity matrix, and processed using a convolutional neural network to obtain the traffic flow physical characteristics.

[0075] like Figure 3 As shown, in this embodiment, the multi-agent reinforcement learning traffic light control model includes: convolutional neural network, recurrent neural network, and graph neural network.

[0076] In constructing this model, for each target intersection traffic light i in the subnet, the traffic flow information is digitized into a position matrix and a velocity matrix, and then processed using a convolutional neural network to obtain the physical characteristics of the traffic flow, specifically:

[0077]

[0078] in, and Let be the position matrix and velocity matrix at time t, respectively;

[0079] φ i and The convolutional neural network CNN is respectively i The model parameters and output.

[0080] In one implementation of this embodiment, step S300 further includes the following steps:

[0081] Step S320: For each target intersection traffic light in the subnet, collect its historical state and process it using a recurrent neural network to obtain traffic flow dynamics characteristics.

[0082] In this embodiment, for each target intersection traffic light i in the subnet, its historical state is collected. The traffic flow dynamics characteristics were then obtained using a recurrent neural network, specifically:

[0083]

[0084] in, This is the length of the coupling waiting queue;

[0085] c represents the historical time span;

[0086] and These are the model parameters and output of the recurrent neural network RNNi, respectively.

[0087] Specifically, the coupled waiting queue refers to a queue that affects the traffic flow at the target intersection. Taking the traffic flow on the left side of target intersection i as an example (e.g., Figure 4 As shown, the solid arrows represent the coupled waiting queues. The traffic flow that affects the target intersection comes from the left side of intersection j, specifically including: traffic flow from the left side of intersection j that goes straight through intersection j, traffic flow from above intersection j that turns left through intersection j, and traffic flow from below intersection j that turns right through intersection j. Similarly, the coupled waiting queues in other directions of the target intersection can be obtained.

[0088] In one implementation of this embodiment, step S300 further includes the following steps:

[0089] Step S330: For each target intersection traffic light in the subnet, collect the current traffic light and the historical state information of adjacent intersections, and process it using a graph neural network to obtain traffic flow interaction features;

[0090] In this embodiment, for each traffic light i at the target intersection of the subnet, historical state information of itself and its adjacent intersections is collected. Then, a graph neural network is used to process the traffic flow interaction features, specifically:

[0091]

[0092] in, ψ i and These are the graph neural networks (GNNs). i The model parameters and output.

[0093] Specifically, in subnets In the process, the historical state of the target intersection is calculated. Historical status information of its adjacent intersections The correlation degree was determined, and α was obtained by normalization using the softmax function. ij And thus obtain Where σ is the activation function and w is the linear transformation matrix.

[0094] In one implementation of this embodiment, step S300 further includes the following steps:

[0095] Step S340: The traffic flow physical features, the traffic flow interaction features, and the traffic light status are concatenated in vector form and combined with the traffic light action information to obtain the value of the current action in the current traffic flow status and traffic light status.

[0096] Step S350: Determine the optimal action of the traffic light under the current state.

[0097] In this embodiment, the above-mentioned traffic flow physical characteristics are... Traffic flow interaction characteristics Traffic light status By concatenating the vectors and combining them with traffic light action information, the value Q(s) of the action under the current traffic flow state and traffic light state is obtained. i a i θ i ), where θ i ={φ i , φ i , ψ i} represents the parameters of the reinforcement learning model, s i and a i The analysis includes the current traffic state and the traffic light's action; finally, the optimal action of the traffic light under the current state is obtained through analysis.

[0098] Unlike existing traffic flow state information extraction methods, this invention proposes a neighborhood state fusion method that combines convolutional neural networks, recurrent neural networks, and graph neural networks to extract the spatiotemporal coupling information of traffic flow. Specifically, when collecting neighborhood information, only the waiting queue lengths of lanes at adjacent intersections coupled with the target traffic intersection are collected, significantly reducing computational complexity and improving model performance.

[0099] like Figure 1 As shown, in one implementation of the present invention, the traffic light control method based on neighborhood information fusion further includes the following steps:

[0100] Step S400: Based on the constructed multi-agent reinforcement learning traffic light control model, perform neighborhood model fusion, and control the traffic lights in the traffic network according to the fused model.

[0101] In this embodiment, based on the multi-agent reinforcement learning model established in step S300, the traffic lights are controlled by combining the neighborhood model fusion. The control process is as follows: first, the parameters of the traffic light model are initialized; then, information fusion is performed by combining the neighborhood model; the parameters of the traffic lights in the subnet are updated according to the fused information; and the traffic lights are controlled according to the updated parameters.

[0102] In one implementation of this embodiment, step S400 specifically includes the following steps:

[0103] Step S410: Initialize the reinforcement learning model parameters and model training hyperparameters for each traffic light;

[0104] Step S420: Select the traffic light action according to the greedy strategy;

[0105] Step S430: Design a reward based on the length of the vehicle waiting sequence after the traffic light action, obtain an experience sample, and save the experience sample in the local dataset;

[0106] Step S440: Based on the obtained empirical samples, randomly select several samples from the local dataset, and use the gradient descent method to optimize the parameters of the deep Q network and minimize the loss function;

[0107] Step S450: For each traffic light, control the current traffic light based on the local deep Q network model, collect the model parameters of adjacent traffic lights in the subnet, and update the model parameters of the current traffic light through the neighborhood model aggregation method.

[0108] In this embodiment, the reinforcement learning model parameters θ and model training hyperparameters for each traffic light are initialized. The model parameters include: learning rate η, discount factor γ, ∈ in the greedy policy, total number of iterations T, current iteration number t, etc.

[0109] Next, a traffic light action is selected based on a greedy strategy, and a reward is designed based on the length of the vehicle waiting sequence after the traffic light action. Here, I represents the total number of lanes at the traffic light intersection, and li is the length of the waiting sequence of vehicles in the i-th lane after the traffic light activates, thus providing an empirical sample. s′ represents the traffic state of the subnet after executing traffic light action a, and empirical samples are stored in the local dataset D = {d1, d2, ... d}. n In the context of}, n represents the total number of samples.

[0110] Based on the obtained empirical samples, some samples are randomly selected from the local dataset D, and the gradient descent method is used to optimize the parameters θ of the deep Q-network, thereby minimizing the loss function. here For the target value, θ - Let θ be the target network parameters, a′ be the action selected in the next state s′, and the model parameter update rule be θ. i =θ i -ηe(θ i ), here θ i and θ i Let e(θ) be the model parameters before and after the local update of the i-th traffic light. i ) is its corresponding gradient.

[0111] For each traffic light i, based on the traffic subnet in step S100 Traffic light i collects parameters from its neighbor model. Then, the model parameters θ′ of traffic light i are updated using the neighborhood model aggregation method. i Specifically: Here K i For subnet The number of traffic lights in China Aggregate weight vectors for the model; their dimensions depend on the subnet. Size.

[0112] In one implementation of this embodiment, the following steps are included after step S450:

[0113] Step S460: Determine whether the termination condition is met.

[0114] Step S470: If the current iteration count is greater than the total iteration count, then stop updating;

[0115] Step S480: If the current iteration number is less than or equal to the total iteration number, continue to select traffic light actions according to the greedy strategy until the termination condition is met.

[0116] In this embodiment, the final termination condition is determined. If the current iteration number t is greater than the total iteration number T, the model parameters of the current traffic light are stopped from being updated. Otherwise, the process proceeds to step S420, where other traffic light actions are selected according to a greedy strategy.

[0117] Unlike existing distributed training strategies based on federated learning, this invention proposes a neighborhood model fusion method. Each traffic light only needs to communicate with the traffic lights connected to it to share model parameters, without the need for coordination from a central server, effectively avoiding communication blockage and single point of failure problems.

[0118] The above-described technical solution in this embodiment has the following effects:

[0119] This embodiment decomposes the traffic network into more basic subnets and proposes a neighborhood information fusion method. Each traffic light only needs to consider the intersections directly adjacent to it, which greatly reduces the dimensionality of the state and action space and avoids the "curse of dimensionality" problem. Furthermore, a neighborhood state fusion method is proposed. When collecting neighborhood information, only the lane waiting queue lengths of adjacent intersections coupled with the target traffic intersection are collected, which greatly reduces the computational complexity and improves the model performance. At the same time, each traffic light only needs to communicate with the traffic lights connected to it and share model parameters, without the need for coordination by a central server, which effectively avoids communication blockage and single point of failure problems.

[0120] Exemplary device

[0121] Based on the above embodiments, the present invention also provides a terminal, the principle block diagram of which can be as follows: Figure 5 As shown.

[0122] The terminal includes: a processor, a memory, an interface, a display screen, and a communication module connected via a system bus; wherein, the processor of the terminal provides computing and control capabilities; the memory of the terminal includes a storage medium and internal memory; the storage medium stores the operating system and computer programs; the internal memory provides an environment for the operation of the operating system and computer programs in the storage medium; the interface is used to connect to external terminal devices, such as mobile terminals and computers; the display screen is used to display corresponding traffic light control information based on neighborhood information fusion; and the communication module is used to communicate with a cloud server or mobile terminal.

[0123] When executed by a processor, this computer program is used to implement a traffic light control method based on neighborhood information fusion.

[0124] It will be understood by those skilled in the art that Figure 5The schematic diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal to which the present invention is applied. A specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0125] In one embodiment, a terminal is provided, comprising: a processor and a memory, the memory storing a traffic light control program based on neighborhood information fusion, the traffic light control program based on neighborhood information fusion being executed by the processor to implement the traffic light control method based on neighborhood information fusion as described above.

[0126] In one embodiment, a storage medium is provided, wherein the storage medium stores a traffic light control program based on neighborhood information fusion, which, when executed by a processor, is used to implement the traffic light control method based on neighborhood information fusion as described above.

[0127] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory.

[0128] In summary, this invention provides a traffic light control method and terminal based on neighborhood information fusion. The method includes: decomposing the traffic network of a target area into multiple basic subnets; for each traffic light in the traffic network, acquiring traffic flow information and traffic light status information from all four sides of the traffic light, and digitizing the traffic flow information and traffic light information to obtain digitized traffic flow status information; constructing a multi-agent reinforcement learning traffic light control model based on neighborhood status fusion according to the subnets in the traffic network and the obtained digitized traffic flow status information; performing neighborhood model fusion based on the constructed multi-agent reinforcement learning traffic light control model, and controlling the traffic lights in the traffic network according to the fused model. This invention improves traffic light optimization efficiency by decomposing the traffic network into more basic subnets and proposing a neighborhood information fusion method, allowing each traffic light to consider only its directly adjacent intersections.

[0129] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A traffic light control method based on neighborhood information fusion, characterized in that, The traffic light control method based on neighborhood information fusion includes the following steps: The traffic network of the target area is decomposed into multiple basic subnetworks; each subnetwork includes: the target intersection and adjacent intersections; For each traffic light in the traffic network, traffic flow information and traffic light status information from all four sides of the traffic light are acquired, and the traffic flow information and traffic light information are digitized to obtain digitized traffic flow status information; wherein, the traffic flow information includes: vehicle position information and vehicle speed information; Based on the subnets in the traffic network and the obtained digital traffic flow state information, a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion is constructed. Based on the constructed multi-agent reinforcement learning traffic light control model, neighborhood model fusion is performed, and the traffic lights in the traffic network are controlled according to the fused model, including: Initialize the reinforcement learning model parameters and model training hyperparameters for each traffic light; Choose the traffic light action based on a greedy strategy; The reward is designed based on the length of the vehicle waiting sequence after the traffic light action, and an experience sample is obtained and stored in the local dataset. Based on the obtained empirical samples, several samples are randomly selected from the local dataset, and the parameters of the deep Q network are optimized using the gradient descent method to minimize the loss function; For each traffic light, the current traffic light is controlled based on the local deep Q-network model, and the model parameters of adjacent traffic lights in the subnet are collected. The model parameters of the current traffic light are updated through the neighborhood model fusion method.

2. The traffic light control method based on neighborhood information fusion according to claim 1, characterized in that, The multi-agent reinforcement learning traffic light control model includes: convolutional neural network, recurrent neural network, and graph neural network; The construction of a multi-agent reinforcement learning traffic light control model based on neighborhood state fusion includes: For each traffic light at the target intersection in the subnet, the digitized traffic flow state information is converted into a position matrix and a velocity matrix, and then processed using a convolutional neural network to obtain the traffic flow physical features; wherein, the traffic flow physical features are: in, and They are respectively Position and velocity matrices at each moment; and The convolutional neural network is respectively Model parameters and output; For each traffic light at the target intersection in the subnet, its historical state is collected and processed using a recurrent neural network to obtain traffic flow dynamics features; wherein, the traffic flow dynamics features are: in, This is the length of the coupling waiting queue; and The recurrent neural network is respectively Model parameters and output; For each target intersection traffic light in the subnet, the historical state information of the current traffic light and adjacent intersections is collected and processed using a graph neural network to obtain traffic flow interaction features; wherein, the traffic flow interaction features are: in, , and The graph neural network is respectively Model parameters and output; In subnet In the process, the historical state of the target intersection is calculated. Historical status information of its adjacent intersections The correlation degree was obtained, and then normalized using the softmax function. And thus obtain ;in, For activation function, It is a linear transformation matrix; The traffic flow physical features, traffic flow interaction features, and traffic light status are concatenated in vector form and combined with traffic light action information to obtain the value of the current action in the current traffic flow status and traffic light status. Determine the optimal action for the traffic light under the current conditions.

3. The traffic light control method based on neighborhood information fusion according to claim 1, characterized in that, The model parameters of the current traffic light are: in, For subnet Number of traffic lights in China; These are the parameters for the neighborhood model; The model fusion weight vector has a dimension that depends on the subnet. Size.

4. The traffic light control method based on neighborhood information fusion according to claim 1, characterized in that, The process of updating the model parameters of the current traffic light includes: Determine if the termination condition is met: If the current iteration count is greater than the total number of iterations, then stop updating; If the current iteration number is less than or equal to the total iteration number, then continue to select traffic light actions according to the greedy strategy until the termination condition is met.

5. A terminal, characterized in that, include: The processor and memory, wherein the memory stores a traffic light control program based on neighborhood information fusion, and the traffic light control program based on neighborhood information fusion, when executed by the processor, is used to implement the traffic light control method based on neighborhood information fusion as described in any one of claims 1-4.

6. A storage medium, characterized in that, The storage medium stores a traffic light control program based on neighborhood information fusion, which, when executed by a processor, is used to implement the traffic light control method based on neighborhood information fusion as described in any one of claims 1-4.

Images