Traffic signal control optimization method and device and electronic equipment

Abstract

The invention provides a traffic signal control optimization method and device and electronic equipment, and the method comprises the steps: obtaining multi-source traffic data, an exploration rate and a current parameter vector corresponding to a current time step of an optimization hierarchy, determining a large language model or a corresponding reinforcement learning model as an action source according to the exploration rate, determining a current action according to a current state and an action source, determining an adjusted parameter vector based on the current action and the current parameter vector, determining green light time of a target phase based on the adjusted parameter vector so as to control traffic signals of an optimization level to obtain new multi-source traffic data, and determining the adjusted parameter vector as a new current parameter vector, continuing to control the traffic signal of the optimization level, storing control experience data of the optimization level in a preset first time period, and optimizing the reinforcement learning model corresponding to the optimization level; according to the method, the black box decision problem is solved by adjusting parameters with physical meanings, the action source is controlled through the exploration rate, the advantages of the two models can be combined, and the disadvantages are reduced.

Description

Technical Field

[0001] This invention relates to the field of traffic control technology, and in particular to a traffic signal control optimization method, apparatus, and electronic equipment. Background Technology

[0002] With the acceleration of urbanization and the continuous growth of motor vehicle ownership, urban transportation systems face severe congestion challenges. Traffic signal control, as a key means to improve road network efficiency, has become a core area of ​​research in Intelligent Transportation Systems (ITS) due to its adaptive optimization capabilities. Traditional control methods, such as timed control and inductive control, while having the advantage of simple structure, often exhibit limitations in dynamic response lag and insufficient cross-regional coordination when dealing with complex time-varying traffic flow and sudden events.

[0003] Among related technologies, Deep Reinforcement Learning (DRL) demonstrates powerful perception and decision-making potential through trial and error between agents and the environment. However, the end-to-end data-driven approach faces significant challenges in practical engineering applications, particularly in the transition from simulation to reality (Sim2Real). These challenges include the lack of interpretability of black-box models, the tendency for extreme actions to violate traffic common sense during the exploration phase, and the low training efficiency due to the difficulty in integrating the experience of mature traffic engineering experts. Furthermore, Large Language Models (LLMs), with their superior semantic understanding and common-sense reasoning capabilities, can handle unstructured expert rules, but suffer from high inference latency and potential "illusion" risks in real-time closed-loop control. Summary of the Invention

[0004] This invention provides a traffic signal control optimization method, apparatus, and electronic device to solve the technical problems of low training efficiency in deep reinforcement learning and inference delay and potential illusion in large language models during traffic signal control.

[0005] This invention provides a traffic signal control optimization method, comprising: acquiring multi-source traffic data, exploration rate, and current parameter vector corresponding to an optimization level at the current time step, wherein the optimization level includes single-intersection optimization or regional optimization composed of multiple single intersections, and the parameter vector includes multiple parameters for indirectly controlling traffic signals and having physical meaning; determining a large language model or a corresponding reinforcement learning model as an action source based on the exploration rate, and determining a current action based on the current state and the action source, wherein the current state is constructed based on the multi-source traffic data, and the current action is used to characterize the adjustment amount of the current parameter vector; determining an adjusted parameter vector based on the current action and the current parameter vector, and determining the green light time of the target phase based on the adjusted parameter vector to control the traffic signal of the optimization level, obtaining new multi-source traffic data, and determining the adjusted parameter vector as a new current parameter vector to continue controlling the traffic signal of the optimization level; storing control experience data of the optimization level within a preset first time period, and optimizing the reinforcement learning model corresponding to the optimization level, wherein the control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

[0006] In one embodiment of the present invention, the determination of the optimization level includes: if the parameter performance index of a single intersection improves within a preset second time period, then the single intersection is determined as an active intersection, and the optimization level is determined as single intersection optimization, wherein the parameter performance index is obtained based on the intersection real-time reward of the single intersection; if the parameter performance index of an active intersection does not improve within the preset second time period, then the active intersection is determined as a stable intersection; if there are adjacent stable intersections, and the number of stable intersections is greater than a preset number threshold, then the optimization level is determined as regional optimization, and the adjacent stable intersections are determined as optimization regions.

[0007] In one embodiment of the present invention, before determining the green light time of the target phase based on the adjusted parameter vector, the method further includes: constructing a regional joint state based on multi-source traffic data of each single intersection and inputting it into an upper-layer strategy to obtain a coordination mode for each single intersection. The upper-layer strategy aims to maximize the regional joint reward of the optimized region. If the value type of the coordination mode of a single intersection is a continuous value, the current action is scaled. If the value type of the coordination mode of a single intersection is a discrete value, the strategy is frozen or allowed to be adjusted. The adjustment strategy includes optimizing the underlying strategy of the corresponding single intersection and / or adjusting the parameter vector of the corresponding single intersection to obtain the optimization range of the single intersection. The upper-layer strategy and the underlying strategy of the single intersection are obtained based on the corresponding reinforcement learning model.

[0008] In one embodiment of the present invention, the parameter vector includes a historical gain coefficient for characterizing the weight of controlling historical data fusion, a basic bias time for characterizing the vehicle start-up loss time, a basic passage time for characterizing the time required to release a unit vehicle, and a neighbor gain coefficient for characterizing the weight of controlling neighbor prediction information fusion.

[0009] In one embodiment of the present invention, the large language model associates problem patterns with corresponding adjustment logic to obtain the current action in the early stage of exploration; if the problem pattern is that there is a green light that is not cleared, the value of the first type of parameter is reduced and / or the quality of multi-source traffic data is detected, the first type of parameter including at least one of the basic passage time, the basic bias time and the historical gain coefficient; if the problem pattern is that there are vehicles that have not been released, the value of the second type of parameter is increased and / or the sudden change in traffic demand is detected, the second type of parameter including at least one of the basic passage time, the basic bias time and the historical gain coefficient; if the problem pattern is that there is data imbalance in different directions, the fault status of the corresponding data acquisition equipment is detected, and / or the correctness of the road configuration is detected; if the problem pattern is that there is control lag, the value of the third type of parameter is reduced, and / or the time prediction parameter corresponding to the model is adjusted, the third type of parameter including the historical gain coefficient and / or the queue length of historical data, the time prediction parameter including the decision time step corresponding to the current action and / or the model update frequency corresponding to the reinforcement learning model.

[0010] In one embodiment of the present invention, determining the green light time of a target phase based on the adjusted parameter vector includes: determining the phase with the highest vehicle queuing pressure as the target phase, or determining each phase of the optimization level as the target phase, wherein the vehicle queuing pressure is used to characterize the total number of target queue lengths of the lanes managed by the phase, and the target queue length is determined based on the real-time queue length of vehicles in the lane, the adjusted neighbor gain coefficient, and the adjusted historical gain coefficient; determining the green light time of the target phase based on the adjusted base offset time, the adjusted base passage time, and the maximum value of the target queue length of the lanes managed by the target phase; wherein the adjusted parameter vector includes the adjusted historical gain coefficient, the adjusted base offset time, the adjusted base passage time, and the adjusted neighbor gain coefficient.

[0011] In one embodiment of the present invention, the target queue length is determined based on the real-time queue length of vehicles in the lane, the adjusted neighbor gain coefficient, and the adjusted historical gain coefficient, including: determining the effective queue length based on the real-time queue length, the historical queue length, and the adjusted historical gain coefficient of vehicles in the lane; correcting the effective queue length according to the predicted queue length and the adjusted neighbor gain coefficient to obtain the target queue length; wherein, the predicted queue length is obtained based on a pre-trained multi-intersection queue length prediction model and the historical queue length corresponding to each of the single intersections.

[0012] In one embodiment of the present invention, the regional joint reward for regional optimization includes: determining the regional joint reward based on the mean and variance of the instantaneous rewards of all single intersections within the optimization region, and the penalty coefficient corresponding to the variance.

[0013] This invention provides a traffic signal control optimization device, comprising: an acquisition module, configured to acquire multi-source traffic data, an exploration rate, and a current parameter vector corresponding to an optimization level at the current time step, wherein the optimization level includes single-intersection optimization or regional optimization composed of multiple single intersections, and the parameter vector includes multiple parameters for indirectly controlling traffic signals and having physical meaning; and an action generation module, configured to determine a large language model or a corresponding reinforcement learning model as an action source based on the exploration rate, and determine a current action based on the current state and the action source, wherein the current state is constructed based on the multi-source traffic data, and the current action is used to characterize the adjustment amount of the current parameter vector. The traffic signal control module is used to determine an adjusted parameter vector based on the current action and the current parameter vector, and to determine the green light time of the target phase based on the adjusted parameter vector, so as to control the traffic signal of the optimization level, obtain new multi-source traffic data, and determine the adjusted parameter vector as the new current parameter vector to continue controlling the traffic signal of the optimization level; the model optimization module is used to store the control experience data of the optimization level within a preset first time period, and optimize the reinforcement learning model corresponding to the optimization level. The control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

[0014] The present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device enables the traffic signal control optimization method as described in any of the above embodiments.

[0015] The beneficial effects of this invention are as follows: The traffic signal control optimization method, device, and electronic equipment proposed in this invention reduce the risk of overfitting the reinforcement learning model to the absolute state of the environment by adjusting parameters with clear physical meaning, and solve the black-box decision-making problem. Furthermore, by controlling the source of the action through the exploration rate, the advantages of large language models and reinforcement learning models can be combined. For example, large language models can combine unstructured expert experience, and reinforcement learning models can make efficient decisions while reducing the disadvantages of both. For example, if the large language model is misled, the reinforcement learning model can output actions that violate traffic common sense, thus balancing exploration efficiency and control stability, thereby continuously, safely, and efficiently adjusting the parameter vector. Attached Figure Description

[0016] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0017] In the attached diagram:

[0018] Figure 1 A schematic diagram of an exemplary system architecture provided in an embodiment of the present invention; Figure 2 This is a flowchart illustrating a traffic signal control optimization method provided in one embodiment of the present invention; Figure 3 This is a schematic diagram of the logic for progressive region optimization provided in one embodiment of the present invention; Figure 4 This is a logical diagram illustrating reinforcement learning training and deployment in one embodiment of the present invention; Figure 5 This is a block diagram of a traffic signal control optimization device provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a computer system for an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0019] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0020] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0021] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0022] In related technologies, end-to-end data-driven traffic signal control methods based on Deep Reinforcement Learning (DRL) are currently the mainstream research direction in the field of intelligent transportation. Representative algorithms include value-based Deep Q-Network (DQN), and asynchronous advantage actor-critic (A3C) and proximal policy optimization (PPO) based on policy gradients. The core of this type of scheme lies in constructing a closed-loop "perception-decision" framework. High-dimensional real-time traffic flow data such as lane-level queue length, cumulative waiting time, and signal phase are acquired through detection devices deployed at intersections, and these are mapped into state vector inputs. The agent uses deep neural networks as nonlinear function approximators to directly output the control action for the next moment, such as discrete phase selection or continuous green light duration adjustment. During training, the agent interacts with the simulation environment through trial and error, aiming to maximize long-term cumulative rewards (such as minimizing average delay or maximizing traffic volume). It updates network parameters via backpropagation, thereby achieving a direct end-to-end mapping from the original traffic state to control commands. This method aims to eliminate reliance on manually preset rules and achieve automated evolution of control strategies.

[0023] However, significant shortcomings remain in practical engineering deployments. First, the lack of safety constraints in stochastic exploration means these methods rely on random strategies for optimization when facing unknown states. Due to the lack of explicit constraints based on traffic engineering common sense, they are prone to producing commands that violate safety regulations, such as extremely short green light intervals or phase deadlock. Second, the entirely data-driven "whiteboard learning" model results in a lack of domain knowledge, failing to leverage the experience of mature traffic engineering experts, leading to slow convergence and low sample efficiency during the "cold start" phase. Furthermore, the domain transfer gap from simulation to reality (Sim2Real) is a key bottleneck. The idealized simulation environment differs significantly from the real environment, which contains noise and latency, and the lack of online adaptive fine-tuning mechanisms for key control hyperparameters causes a significant drop in strategy performance at real intersections. Finally, the black-box nature of deep neural network decision-making processes makes them lack interpretability, failing to meet the stringent requirements of traffic management departments for system transparency and robustness.

[0024] In addition, model-based and heuristic rule-based adaptive traffic signal control is the most widely used technical solution, with the Sydney Coordinated Adaptive Traffic System (SCATS) and the Split, Cycle and Offset Optimization Technique (SCOOT) system as typical examples. The core working principle of this type of solution is to adjust parameters based on traffic flow theory models (such as Webster's formula and saturation balance theory) and preset heuristic rules. The system typically adopts a hierarchical control architecture. At the intersection level, sensors collect periodic flow and occupancy data, explicitly maintaining a set of key control parameters including cycle duration, green ratio, and phase difference. Based on the real-time calculated saturation, a matching timing scheme is selected from a preset scheme library, or incremental fine-tuning of the current parameters is performed based on a fixed step size. At the regional level, simplified linear models or "green wave" theory are typically used to adjust the phase difference between intersections to achieve coordinated control of arterial roads. This method has high engineering reliability and clear traffic engineering theoretical support, but its limitations are becoming increasingly apparent when dealing with the complex traffic environments of modern cities. First, it heavily relies on human experience and manual calibration. Its performance is extremely dependent on the initial settings of basic parameters such as saturation flow and startup loss time. Once the traffic environment changes, the fixed rule base is difficult to adapt adaptively, resulting in high maintenance costs. Second, the control logic based on heuristic rules is inherently linear and relatively static, making it difficult to handle nonlinear and complex operating conditions beyond common sense, such as intersection deadlocks and sudden accident overflows. In addition, the system response has a significant lag and low data utilization. It usually makes decisions based on aggregated statistical data over a long time window, failing to extract high-dimensional data features such as vehicle trajectories, thus limiting control accuracy. Finally, its regional coordination mechanism is mostly based on simplified models, making it difficult to handle dynamic collaborative optimization under complex road network structures, and it is prone to getting trapped in local optima.

[0025] Please see Figure 1 , Figure 1 This is a schematic diagram of an exemplary system architecture provided in an embodiment of the present invention. Figure 1 As shown, the system architecture may include a data acquisition device 110 and a computer device 120. The computer device may be at least one of an embedded computer, a neural network computer, or a general-purpose computer. The data acquisition device 110 includes detection equipment to obtain multi-source traffic data, such as lane-level queue lengths, and transmits it to the computer device for traffic signal control optimization.

[0026] For example, computer device 120 acquires multi-source traffic data, exploration rate, and current parameter vector corresponding to the optimization level at the current time step. The optimization level includes single-intersection optimization or regional optimization composed of multiple single intersections. The parameter vector includes multiple parameters that are used to indirectly control traffic signals and have physical meaning. Based on the exploration rate, a large language model or the corresponding reinforcement learning model is determined as the action source. The current action is determined based on the current state and the action source. The current state is constructed based on multi-source traffic data. The current action is used to characterize the adjustment amount of the current parameter vector. Based on the current action and the current parameter vector, the adjusted parameter vector is determined. Based on the adjusted parameter vector, the green light time of the target phase is determined to control the traffic signal of the optimization level. New multi-source traffic data is obtained, and the adjusted parameter vector is determined as the new current parameter vector to continue controlling the traffic signal of the optimization level. Control experience data of the optimization level within a preset first time period is stored, and the reinforcement learning model corresponding to the optimization level is optimized. The control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

[0027] In related technologies, deep reinforcement learning suffers from low training efficiency in traffic signal control, while large language models suffer from inference delays and potential illusions.

[0028] To address the aforementioned technical problems, this invention provides a traffic signal control optimization method, apparatus, and electronic device. The implementation details of the technical solutions in the embodiments of this invention are described in detail below.

[0029] Please see Figure 2 , Figure 2 This is a flowchart illustrating a traffic signal control optimization method provided in one embodiment of the present invention. Figure 2 As shown, in an exemplary embodiment, the traffic signal control optimization method includes at least steps S210 to S240, which are described in detail below: Step S210: Obtain the multi-source traffic data, exploration rate, and current parameter vector corresponding to the optimization level at the current time step.

[0030] The optimization levels include single-intersection optimization or area optimization consisting of multiple single intersections, and the parameter vector includes multiple parameters that are used to indirectly control traffic signals and have physical meaning.

[0031] In one embodiment of the present invention, the multi-source traffic data includes real-time data, historical data, topology data, and neighbor prediction data.

[0032] In one embodiment of the present invention, the original data set composed of multi-source traffic data is as follows:

[0033] in, For the original data set, For real-time data vectors, For historical data queue, For topology data, Predict data for neighbors.

[0034] In one embodiment of the present invention, at time This identifies the discrete time step corresponding to the data collection. (Passing through the intersection) Identify the intersection index of a single intersection.

[0035] In one embodiment of the present invention, the current time step is used to characterize the current discrete time step.

[0036] In one embodiment of the present invention, the discrete time step and the decision time step of the action source are... There is a corresponding relationship.

[0037] In one embodiment of the present invention, the real-time data vector is as follows:

[0038] in, For real-time data vectors, For a moment The lane queue length vector, For a moment Lane waiting time vector, For a moment The lane density vector, For a moment The headway statistics vector.

[0039] In one embodiment of the present invention, , The dimension of a vector is , This refers to the number of lanes at the intersection, or the lane dimension used for state construction. For example, =[8,3,0,...]; =[35,12,0,...].

[0040] In one embodiment of the present invention, the historical data queue is as follows:

[0041] in, For historical data queue, The smoothed historical lane queue length, For the smoothed historical lane waiting time, This represents the smoothed historical lane density.

[0042] In one embodiment of the present invention, the sliding window smoothing of historical data corresponding to historical features is as follows:

[0043] in, For at any time Smoothed historical characteristics The set of time steps within the sliding window. For time Historical characteristics of change The base of the time steps within the sliding window. This is the span parameter of the sliding window.

[0044] In one embodiment of the present invention, the topology data is as follows:

[0045] in, For topology data, The feature matrix representing the lane correspondence managed by the phase. It is an adjacency matrix representing the neighbor relationships between multiple intersections.

[0046] In one embodiment of the present invention, the neighbor prediction data is as follows:

[0047] in, Predict data for neighbors, To predict queue length based on neighboring intersections, This is a prediction of lane waiting time based on neighboring intersections.

[0048] In one embodiment of the present invention, the determination of the optimization level includes: if the parameter performance index of a single intersection improves within a preset second time period, the single intersection is determined as an active intersection, and the optimization level is determined as single intersection optimization, with the parameter performance index obtained based on the intersection's real-time reward; if the parameter performance index of an active intersection does not improve within the preset second time period, the active intersection is determined as a stable intersection; if there are adjacent stable intersections, and the number of stable intersections is greater than a preset number threshold, the optimization level is determined as regional optimization, and the adjacent stable intersections are determined as optimization regions.

[0049] In one embodiment of the present invention, the mathematical framework of a Markov Decision Process (MDP) is used to describe the sequential decision problem, thereby realizing the modeling of the parameter adjustment task of the present invention, including the state space, action space, transition probability, reward function, and discount factor.

[0050] In one embodiment of the present invention, the state during single-intersection optimization is represented by a state vector, and the state during region optimization is represented by a region joint state.

[0051] In one embodiment of the present invention, the state vector corresponding to a single intersection is as follows:

[0052] in, For the intersection The state vector, For lane feature compression function, For the intersection Real-time queue length, For the intersection Average waiting time characteristics, For the intersection The parameter vector, For the neighbor parameter aggregation function, For the neighbor's intersection The parameter vector, intersection Meet at the neighbor's intersection.

[0053] In one embodiment of the present invention, the vehicle start-up loss time includes driver reaction time, vehicle acceleration time, and safety interval, thereby obtaining the complete time from the green light turning on to the first vehicle passing the stop line.

[0054] In one embodiment of the present invention, the region joint state corresponding to region optimization is as follows:

[0055] in, In a state of regional unity, For the intersection The state vector, To optimize the region Topological graph features, This is a vector concatenation operation.

[0056] In one embodiment of the present invention, the parameter vector includes a historical gain coefficient for characterizing the weight of controlling historical data fusion, a basic bias time for characterizing the vehicle start-up loss time, a basic passage time for characterizing the time required to release a unit vehicle, and a neighbor gain coefficient for characterizing the weight of controlling neighbor prediction information fusion.

[0057] In one embodiment of the present invention, the standardized parameter vector is defined as follows:

[0058] in, For the intersection The parameter vector, This is the historical gain coefficient. Based on the base bias time, Based on the basic passage time, This is the neighbor gain coefficient.

[0059] In one embodiment of the present invention, parameters with physical meaning can provide constraints and adjustment basis that can be directly used for engineering verification, thereby solving the problems of insufficient safety constraints and insufficient interpretable information for decision-making in the exploration phase.

[0060] In one embodiment of the present invention, the determination of the intersection instant reward includes: determining the intersection instant reward based on the average waiting time of a single intersection at the current time step, the lane queue length vector, the preset constraint violation indication function, and the corresponding reward weight.

[0061] In one embodiment of the present invention, the instant intersection reward for a single intersection is as follows:

[0062] in, For at any time Instant rewards at intersections As the first reward weight, This represents the average waiting time. As the second reward weight, For a moment The lane queue length vector, This is the third reward weight. To constrain the violation indicator function.

[0063] In one embodiment of the present invention, the three reward weights can be set to 0.6, 0.3, and 0.1 respectively, in order to prioritize ensuring passage timeliness.

[0064] In one embodiment of the present invention, parameter performance evaluation is represented by a parameter vector. The expected cumulative discount rewards are as follows:

[0065] in, For parameter vectors Performance evaluation values, As expected, For a moment Instant rewards at intersections As a discount factor, For time range, Given a parameter vector .

[0066] In one embodiment of the present invention, the discount factor I recommend 0.95. In one embodiment of the present invention, the regional joint reward for regional optimization includes: determining the regional joint reward based on the mean and variance of the instantaneous rewards of all single intersections within the optimization region, and the penalty coefficient corresponding to the variance.

[0067] In one embodiment of the present invention, the regional joint reward is as follows:

[0068] in, For regional joint awards, To optimize the intersection set within the area cardinality of the set For the intersection Instant rewards at intersections This is the penalty coefficient for variance. Let Variance be the variance.

[0069] In one embodiment of the present invention, the second time period is preset to be 14 days.

[0070] In one embodiment of the present invention, the preset quantity threshold is set to 2. The area triggering condition is that there are adjacent stable intersections, and the number of stable intersections is greater than or equal to 2.

[0071] In one embodiment of the present invention, the decision time steps, i.e., the decision cycles, corresponding to different optimization levels and different optimization scales within a region are different. This hierarchical setting determines the time scale between the Markov decision process chains, aiming to balance micro-level precise tuning with macro-level flow stability. For example:

[0072] in, The decision time step is set as follows: For an independent single intersection, a high-frequency fine-tuning of 5 minutes is used to quickly respond to local fluctuations; when 2 to 10 single intersections form an associated area, the decision time step is extended to 15 minutes to smooth the transmission of traffic waves within the area; for large-scale areas with more than 10 intersections, a decision cycle of 30 minutes is used to ensure the global balance of macro traffic flow and reduce control oscillations.

[0073] In one embodiment of the present invention, in the bottom-up optimization evolution strategy, regional dynamic aggregation is triggered by monitoring the convergence state of a single intersection. A reward evaluation mechanism with a variance penalty term is introduced during the regional optimization stage, aiming to maximize regional traffic efficiency while constraining performance differences between sub-intersections. This solves the coordination conflicts introduced by the expansion of joint state dimensions and uneven distribution of multi-intersection benefits, the potential inconsistency between local and overall objectives in the regional optimization process, and the problems of dimensionality curse and coordination mismatch in large-scale road network control.

[0074] In one embodiment of the present invention, time action Defined as the amount of adjustment to the parameter vector, such as the fine-tuning increment, i.e. The action space constraints are as follows:

[0075] in, For the action space of a single intersection, For the intersection The parameter adjustment action of the parameter vector. The dimension of the parameter vector is 4-dimensional. This is the lower bound of the parameter vector. For the intersection The parameter vector, This is the upper bound of the parameter vector.

[0076] In one embodiment of the present invention, the upper limit and the lower limit of the parameter vector are preset.

[0077] In one embodiment of the present invention, the single adjustment range constraint is as follows:

[0078] in, For the first The parameters at time... The adjustment amount, For the intersection The The values ​​of the parameters before adjustment.

[0079] Step S220: Based on the exploration rate, determine the large language model or the corresponding reinforcement learning model as the action source, and determine the current action based on the current state and the action source.

[0080] The current state is constructed based on multi-source traffic data, and the current action is used to characterize the adjustment amount of the current parameter vector.

[0081] This invention can achieve optimal performance at each intersection through the synergy of a Large Language Model (LLM) and a Reinforcement Learning (RL) model. Standardized parameter vector Continuous, safe, and efficient adaptive fine-tuning is performed, and the process is gradually extended to regional optimization of intersection joints.

[0082] In one embodiment of the present invention, the action source is selected as follows:

[0083] in, For a moment The action, For large language models, For the intersection The state vector, To adjust historical records, The principle function for parameter adjustment. For prompt word template function, For exploration rate; To optimize the policy network in the hierarchical correspondence reinforcement learning model, including the single-way policy network. or regional strategy network ; For a moment The state, including or regional joint state .

[0084] This invention combines knowledge-driven (LLM heuristic exploration) and data-driven (RL) decision-making mechanisms, employing a greedy strategy to balance exploration efficiency and control stability. During the exploration phase, it uses probability... LLM is used to generate parameter adjustment suggestions, and RL strategies are used for precise fine-tuning during the utilization phase, thereby achieving hybrid intelligent decision-making.

[0085] In one embodiment of the present invention, the prompt word template function is as follows:

[0086] in, For prompt word template function, For the intersection The state vector, To adjust historical records, The principle function for parameter adjustment; For connection, The current intersection status and parameters, including lane queue length. Lane waiting time and parameter vector ; Topological information, including feature matrices Dimensions and neighbor intersection set The dimension; Principle function for parameter adjustment , Adjusting historical records LLM outputs action vectors .

[0087] In one embodiment of the present invention, the present invention drives LLM to perform traffic logic reasoning and decision-making by aggregating textual instructions of real-time traffic status, road network topology and expert experience rules.

[0088] In one embodiment of the present invention, the prompt word template function includes information such as role settings, input information templates, thinking process requirements, and output format specifications. The role settings include core rules and the activation of a deep focus state.

[0089] In one embodiment of the present invention, an example of a core rule is as follows: "1. You can only adjust four parameters: HISTORY_GAIN, BASEBIAS, BASEPASSCAR, and NB_GAIN; 2. Each adjustment must be minor, with the change in each parameter not exceeding +10% of the current value; 3. Your decisions must be based on traffic engineering principles and actual traffic data; 4. Output must strictly adhere to the JSON format and include the fields: reasoning, adjustment, and new_parameters (adjusted parameters). 5. You possess the ability to deeply analyze traffic patterns and identify the coupling relationships between parameters. In one embodiment of the present invention, the input information template includes the current intersection status, historical adjustment records, and manual rectification logic reference.

[0090] In one embodiment of the present invention, the manual rectification logic includes at least one of the following: green light idling processing logic, vehicle incomplete idling processing logic, data imbalance processing logic, and adaptive lag processing logic, thereby solving the problem that expert experience and parameter adjustment rules lack computable representation.

[0091] In one embodiment of the present invention, the large language model associates problem patterns with corresponding adjustment logic to obtain the current action in the early stage of exploration. If the problem pattern is that there is a green light that is not cleared, the value of the first type of parameter is reduced and / or the quality of multi-source traffic data is detected. The first type of parameter includes at least one of the basic passage time, basic bias time, and historical gain coefficient. If the problem pattern is that there are vehicles that have not been released, the value of the second type of parameter is increased and / or the sudden change in traffic demand is detected. The second type of parameter includes at least one of the basic passage time, basic bias time, and historical gain coefficient. If the problem pattern is that there is an imbalance of data in different directions, the fault status of the corresponding data acquisition equipment is detected, and / or the correctness of the road configuration is detected. If the problem pattern is that there is control lag, the value of the third type of parameter is reduced and / or the time prediction parameter corresponding to the model is adjusted. The third type of parameter includes the historical gain coefficient and / or the queue length of historical data. The time prediction parameter includes the decision time step corresponding to the current action and / or the model update frequency corresponding to the reinforcement learning model.

[0092] In one embodiment of the present invention, the meaning of the problem mode is defined in the prompt words. For example, "vehicles not fully released" indicates at least one of the following: the actual green light time is less than the maximum green light time, there are still queuing vehicles when the green light ends, and the number of vehicles not released is greater than a preset vehicle number threshold (e.g., 3 vehicles). "Green light idling" indicates at least one of the following: the actual green light time is greater than the minimum green light time, no vehicles pass through during the green light period, and the idling time is greater than a preset idling time (e.g., 3 seconds). "Data imbalance in different directions" indicates at least one of the following: significant differences in multi-source traffic data collected from different directions, simultaneous occurrence of idling and not fully released traffic, and data fluctuation amplitude greater than a preset amplitude threshold (e.g., 30%). "Control lag" indicates that the control scheme is significantly lagging behind the actual traffic flow, there is maladaptation when switching decision cycles, and the influence of historical patterns is too great.

[0093] In one embodiment of the present invention, the quality of multi-source traffic data includes data that is too large and historical data that contains noise.

[0094] In one embodiment of the present invention, sudden changes in traffic demand include sudden surges in traffic volume and failure of predictive models to respond in a timely manner.

[0095] In one embodiment of the present invention, the fault states of the data acquisition device include improper installation location, data being too large or too small.

[0096] In one embodiment of the present invention, the correctness of road configuration includes: incorrect configuration of lane number changes and abnormal distribution of turning traffic flow.

[0097] In one embodiment of the present invention, the thought process requires an example: Please follow the steps below for in-depth analysis. 1. Pattern Recognition: Analyze current traffic status data to identify whether typical question-answering patterns exist; 2. Parameter impact analysis: Evaluate the weight of each parameter's impact on the current problem; 3. Historical learning: Learn from the experiences of success and failure by studying historical records; 4. Consideration of coupling relationships: Analyze the mutual influence relationships between parameters; 5. Minor Adjustment Decision: Based on the above analysis, determine the optimal minor adjustment plan. In one embodiment of the present invention, the output format specification may further include prediction information after parameter adjustment, such as expected changes in queue length and expected changes in waiting time.

[0098] In one embodiment of the present invention, the limitations of the decision-making include the engineering significance of adjusting each parameter and the adjustment principle of the value range. For example, the meaning and calculation method of the basic bias time: the sum of driver reaction time, vehicle acceleration time, and safety interval, with a typical value range of 3.0 to 8.0 seconds. Another example is the usage of the neighbor gain coefficient and the historical gain coefficient, referring to equations (25) and (26), thereby realizing traffic flow wave propagation prediction, upstream and downstream coordination, and regional optimization.

[0099] In one embodiment of the invention, expert experience is represented by threshold conditions and parameter increments. For example, the parameter adjustment increment is ≤ 10% in a single step.

[0100] For example, regarding a green light being left vacant, the action is: BASEPASSCAR <- BASEPASSCAR (1 - 0.05) or BASEBIAS<- BASEBIAS (1 - 0.05) to reduce the base travel time by 5% or the base offset time by 5%; For vehicles that have not been released, the action is: BASEPASSCAR <- BASEPASSCAR (1 + 0.05) or BASEBIAS<- BASEBIAS (1 + 0.05) to increase the base passage time by 5% or increase the base offset time by 5%; For control lag, the action is: HISTORY_GAIN <- min(HISTORY_GAIN, 1.0), and NB_GAIN is checked to ensure that if the historical gain coefficient is less than or equal to 1.0, the original value is kept unchanged, and if the historical gain coefficient is greater than 1.0, it is clipped to 1.0.

[0101] In one embodiment of the present invention, the present invention provides a complete method for constructing prompt words for a large language model, as follows: #Character Setting and Hypnotic Commands You are a professional traffic signal control parameter optimization expert with in-depth knowledge of traffic engineering theory and reinforcement learning. Please strictly follow the following rules for thinking and output: Core Rules : 1. You can only adjust four parameters: HISTORY_GAIN, BASEBIAS, BASEPASSCAR, and NB_GAIN. 2. Each adjustment must be minute, with the change in each parameter not exceeding ±10% of its current value. 3. Your decisions must be based on traffic engineering principles and actual traffic data. 4. The output must strictly adhere to the JSON format and include the fields for reasoning, adjustment, and new_parameters. 5. You possess the ability to deeply analyze traffic patterns and identify the coupling relationships between parameters. Deep focus activation From this moment on, you enter a deep thinking state as a traffic parameter optimization expert. All irrelevant information will be filtered out, and you will focus solely on providing scientific and reasonable parameter adjustment suggestions.

[0102] #Input Information Template ## Current intersection status {"intersection_id": "Unique identifier for the intersection", "current_parameters": { "HISTORY_GAIN": Current value, "BASEBIAS": Current value, "BASEPASSCAR": Current value, "NB_GAIN": Current value}, "traffic_state": { "queue_lengths": [array of queue lengths for each lane], "waiting_times": [waiting time in each direction], "throughput": Current throughput "headway_stats": Headway statistics }, "topology": { "phase_matrix": Phase-lane matrix, "neighbor_intersections": Information on adjacent intersections. "road_geometry": Road channelization information } } ## Historical Adjustment Record [ { "timestamp": "timestamp", "old_parameters": Parameters before adjustment "new_parameters": Adjusted parameters, "performance_change": Performance change, "adjustment_reason": Reason for adjustment } / / ... Recent 5 adjustments ] ## Reference for Manual Rectification Logic { "problem_patterns": { "green_light_waste": "Logic for handling green light idling", "vehicle_not_cleared": "Logic for handling vehicles not yet cleared", "data_imbalance": "Logic for handling data imbalance", "adaptive_lag": "Adaptive lag processing logic" } } # Requirements for the thinking process Please follow these steps for in-depth analysis: 1. Pattern recognition Analyze current traffic data to identify typical problem patterns. 2. Parameter Influence Analysis : Evaluate the weight of each parameter's impact on the current problem 3. History Learning Learn from the successes and failures of historical adjustment records. 4. Consideration of coupling relationship Analyze the interaction relationships between parameters. 5. Minor adjustment decisions Based on the above analysis, the optimal minor adjustment scheme is determined. # Output format specifications json { "reasoning": { "problem_identification": "Description of the main traffic problem identified", "parameter_analysis": "Analysis of the impact of each parameter", "historical_reference": "Historical adjustment reference", "adjustment_strategy": Explanation of the adjustment strategy principle }, "adjustment": { "HISTORY_GAIN": Adjustment amount, "BASEBIAS": Adjustment amount, "BASEPASSCAR": Adjustment amount "NB_GAIN": Adjustment amount }, "new_parameters": { "HISTORY_GAIN": Adjusted value, "BASEBIAS": Adjusted value, "BASEPASSCAR": Adjusted value "NB_GAIN": Adjusted value }, "expected_impact": { "queue_length_change": "Expected change in queue length", "waiting_time_change": "Expected change in waiting time", "throughput_change": "Expected change in throughput", "risk_assessment": "Risk assessment" } } ``` Significance of traffic engineering : \[ Q_{\text{effective}} = (1-\alpha) \times Q_{\text{real}} + \alpha\times Q_{\text{hist}} \] Where $\alpha$ is related to HISTORY_GAIN BASEBIAS (Basic Bias Time) Mathematical expression : \[ T_{\text{phase}} = \beta \times \text{max}(M_{\text{stages},i}) +\gamma \] Where $\gamma = \text{BASEBIAS}$ Principle Explanation : - effect The fixed bias term in the phase-time calculation represents the vehicle start-up loss time. - Physical meaning The complete time from when the green light turns on to when the first car crosses the stop line. - Calculation benchmark : \[ \text{BASEBIAS} = \frac{1}{N} \sum_{i=1}^N (T_{\text{start}}^i - \text{ BASEPASSCAR}) \] Where $T_{\text{start}}^i$ represents the start-up and passage time of the vehicle in the i-th direction. Significance of traffic engineering : - Includes: driver reaction time + vehicle acceleration time + safety interval - Typical range: 3.0-8.0 seconds BASEPASSCAR (Basic Passage Time) Mathematical expression : \[ T_{\text{phase}} = \beta \times \text{max}(M_{\text{stages},i}) +\gamma \] Where $\beta = \text{BASEPASSCAR}$ Principle Explanation : - effect : Passage time coefficient required for vehicles queuing at the unit - Physical meaning Average minimum headway - Calculation benchmark : \[ \text{BASEPASSCAR} = \frac{1}{4} \sum_{d=1}^4 \min(\{h_{t,d} | t \in [1,T ]\}) \] Where $h_{t,d}$ is the headway of the vehicle in direction d at time t. Significance of traffic engineering : - Reflects: Lane capacity, driver behavior characteristics, and vehicle performance - Typical range: 2.5-5.0 seconds NB_GAIN (Neighbor Gain Coefficient) Mathematical expression : [Q_{\text{correction}} = w_2 \times Q_{\text{prediction}} + (1-w_2) \times Q_{\text{current}} \] Where $w_2 = \text{NBGAIN} \times w$, and $w$ is the accuracy of the prediction model. Principle Explanation : - effect : Control the weight of neighboring intersection prediction data in local decision-making - Physical meaning The extent of the impact of regional coordination - Calculation benchmark : [ w = \frac{\text{prediction accuracy}}{100\%}, \quad \text{NBGAIN} \in [0,1]\] Significance of traffic engineering : - Achieve: Traffic flow wave propagation prediction, upstream and downstream coordination, and regional optimization Problem Pattern Recognition and Rectification Logic Green light emptying handling logic Problem characteristics : - Green light duration > minimum green time No vehicles pass through during the green light period. - Idle time $\geq 3$ seconds Root cause analysis : 1. Parameter settings are too high : - BASEBIAS or BASEPASSCAR settings are too high - Overweighting of historical data leads to delayed response. 2. Data quality issues : - Detector data is too large - Historical data contains noise. Logic for handling vehicles not yet fully released Problem characteristics : - Green light duration < maximum green light duration - There are still vehicles queuing when the green light ends - Number of vehicles not released: $3 Root cause analysis : 1. Parameter settings are too low : - BASEBIAS or BASEPASSCAR settings are too low - Overweighting of real-time data leads to underestimation 2. Sudden Changes in Transportation Demand : - Sudden surge in traffic - The predictive model failed to respond in a timely manner Data imbalance handling logic Problem characteristics : - Significant differences in detector data from different orientations - Both empty and incomplete play occur simultaneously - Data fluctuation range >30% Root cause analysis : 1. Detector malfunction : - Some detector data are too high / too low - Improper installation location 2. Road channelization problem : - Lane number change not configured correctly - Abnormal flow distribution during turns Adaptive hysteresis processing logic Problem characteristics : - The control scheme is significantly outdated compared to actual traffic flow. - Incompatibility during cycle switching - Historical patterns have too much influence Root cause analysis : 1. Historical weighting is too high : - HISTORYGAIN setting is too high - Historical data queue is too long 2. Predictive model lag : - The predicted time step is unreasonable. - Insufficient model update frequency.

[0103] In one embodiment of the present invention, the policy network in the reinforcement learning model corresponding to the optimization level is as follows:

[0104] in, To optimize the policy network in a hierarchical reinforcement learning model, it means that the current policy parameters... Below, in state Select action The probability distribution; The distribution is Gaussian, meaning the actions follow a mean of 1 / 2. Standard deviation is It follows a normal distribution.

[0105] In one embodiment of the present invention, at each decision time step Build status And select the appropriate action source from the two types of action sources. fine-tuning Large language models and reinforcement learning models. Action source selection adopts... Greedy strategy: Among them, For exploration rate (e.g.) = 0.1). During initial deployment or performance plateauing, increase... The initial approach leverages LLM (Low-Level Learning) for exploratory parameter tuning based on common sense and expert rules; once sufficient experience is accumulated, it primarily relies on a well-trained reinforcement learning model for stable utilization. This hybrid decision-making mechanism can be applied to scenarios where hyperparameters are fine-tuned online using domain knowledge.

[0106] Step S230: Determine the adjusted parameter vector based on the current action and the current parameter vector, and determine the green light time of the target phase based on the adjusted parameter vector to control the traffic signal at the optimization level, obtain new multi-source traffic data, and determine the adjusted parameter vector as the new current parameter vector to continue controlling the traffic signal at the optimization level.

[0107] In one embodiment of the present invention, the adjusted parameter vector is controlled between the lower limit and the upper limit of the parameter vector by a pruning operator. The determination of the adjusted parameter vector, i.e., the parameter update method, is as follows:

[0108] in, For the adjusted parameter vector, For the trimming operator, For the intersection The parameter vector, For a moment The action, This is the lower bound of the parameter vector. This is the upper bound of the parameter vector.

[0109] In one embodiment of the present invention, if ,but This means adding the corresponding increment to each of the four parameters.

[0110] In one embodiment of the present invention, before determining the green light time of the target phase based on the adjusted parameter vector, the method further includes: constructing a regional joint state based on multi-source traffic data of each intersection and inputting it into the upper-level strategy to obtain the coordination mode of each intersection. The upper-level strategy aims to maximize the regional joint reward of regional optimization. If the value type of the coordination mode of an intersection is a continuous value, the current action is scaled. If the value type of the coordination mode of an intersection is a discrete value, the strategy is frozen or allowed to be adjusted. The adjustment strategy includes optimizing the underlying strategy of the corresponding intersection and / or adjusting the parameter vector of the corresponding intersection to obtain the optimization range of the intersection. The upper-level strategy and the underlying strategy of the intersection are obtained based on the corresponding reinforcement learning model.

[0111] In one embodiment of the present invention, for each intersection Maintain an independent underlying strategy Its input is a single-intersection state. The output is the action corresponding to the parameter adjustment at this intersection: The training of the underlying strategy still employs a hybrid decision-making approach: based on probability. Generated by LLM With probability Depend on The optimization objective is to maximize the parameter performance evaluation of a single intersection. .

[0112] In one embodiment of the present invention, an upper-level strategy is introduced. Its input is the joint state of the region determined by the state vectors of multiple single intersections, and its output is an M-dimensional coordination vector. This is used to control the optimization process of each underlying strategy. Based on the coordination vector... The value type can be defined in two coordination modes: continuous value mode and discrete value mode.

[0113] In one embodiment of the invention, actions are discounted based on continuous values; for example, in a continuous value mode, the coordination vector... Coordination mode of each single intersection .when When =1, the intersection Adjust at normal speed; when When =0, the intersection The parameters remain unchanged; when When ∈(0,1), the adjustment range is scaled proportionally. Practical application actions. For action The result after discount:

[0114] in, For the intersection Actions after discount, For the intersection Coordination mode, For the intersection The action.

[0115] In one embodiment of the present invention, the parameter update method is modified as follows:

[0116] in, For the intersection The adjusted parameter vector, For the trimming operator, For the intersection The parameter vector, For the intersection Coordination mode, For a moment The action, This is the lower bound of the parameter vector. This is the upper bound of the parameter vector.

[0117] In one embodiment of the present invention, freezing or allowing adjustment of the strategy includes: if the value of the coordination mode of a single intersection is a first type of discrete value, then freezing the underlying strategy of the single intersection, and / or freezing the parameter vector of the single intersection and executing only the current parameter vector; if the value of the coordination mode of a single intersection is a second type of discrete value, then allowing optimization of the underlying strategy of the single intersection, and / or allowing adjustment of the parameter vector of the single intersection to execute the adjusted parameter vector.

[0118] In one embodiment of the present invention, the first type of discrete value can be set to 0, and the second type of discrete value can be set to 1.

[0119] In one embodiment of the present invention, in a discrete-valued pattern, the coordination vector Coordination mode of each single intersection This indicates whether or not access to the intersection is permitted. Adjust the parameters. When When =1, it is permissible to cross the intersection. Training and adjustment; when When =0, the parameters of intersection i are frozen, and the underlying strategy is not trained or updated; only the current parameters are executed.

[0120] In one embodiment of the present invention, the actual action applied is as follows:

[0121] in, For the intersection Practical application of actions, For the intersection The coordination model.

[0122] In one embodiment of the present invention, the parameter update method is as follows:

[0123] in, For the intersection The adjusted parameter vector, For the trimming operator, For the intersection The parameter vector, For the intersection of , This is the lower bound of the parameter vector. The upper bound of the parameter vector. For the intersection The coordination model.

[0124] In one embodiment of the present invention, the upper-layer strategy With underlying strategy Collaborative training is possible. In each decision cycle... t Execute sequentially: Collect region joint status and the status of each intersection The upper-level strategy is based on... Generate coordination vector For each intersection If parameter adjustment is allowed ( >0 or =1), then the underlying strategy or LLM is based on Generate Actions And calculate the actual application actions according to the above rules. ,otherwise =0. Application At each intersection, after executing a decision-making cycle, collect the single-intersection reward. Joint rewards with regions .use Update all underlying strategies (Only for) >0 intersection). Use Update upper-level strategy .

[0125] In one embodiment of the present invention, compared with incremental regional optimization, optimization based on upper-level strategies no longer requires waiting for a single intersection to stabilize before triggering regional optimization. Instead, it coordinates the optimization process of each intersection in real time and dynamically through upper-level strategies. The coordination vector output by the upper-level strategies can be regarded as an attention mechanism, enabling regional optimization to flexibly allocate optimization resources: in scenarios requiring global coordination, the adjustment speed of some intersections can be reduced or their parameters can be temporarily frozen to avoid local optimization interfering with the overall equilibrium. This is particularly suitable for scenarios with complex road network topologies and significant mutual influence of traffic flows.

[0126] In one embodiment of the invention, the invention indirectly achieves flexible and diverse adaptive control of traffic signals by fine-tuning four standardized parameters with clear physical meanings, such as variable cycle, maximum green light time adjustment, and maximum pressure method. Its core principle lies in influencing the calculation of intersection phase time and the estimation of traffic conditions through these parameters.

[0127] In one embodiment of the present invention, determining the green light time of a target phase based on an adjusted parameter vector includes: determining the phase with the highest vehicle queuing pressure as the target phase, or determining each phase of the optimization level as the target phase, where vehicle queuing pressure is used to characterize the total number of target queue lengths of the lanes managed by the phase, and the target queue length is determined based on the real-time queue length of vehicles in the lane, the adjusted neighbor gain coefficient, and the adjusted historical gain coefficient; determining the green light time of the target phase based on the adjusted base offset time, the adjusted base passage time, and the maximum value of the target queue length of the lanes managed by the target phase; wherein the adjusted parameter vector includes the adjusted historical gain coefficient, the adjusted base offset time, the adjusted base passage time, and the adjusted neighbor gain coefficient.

[0128] In one embodiment of the present invention, the target queue length of the lane managed by the phase is determined by the phase queuing matrix. storage, It is a matrix of (number of phases × number of lanes), representing the number of vehicles queuing in each lane managed by that phase, with unmanaged lanes having a position of 0.

[0129]

[0130] in, For phase queuing matrix, For the managed lanes The target queue length.

[0131] In one embodiment of the present invention, the target queue length is determined based on the real-time queue length of vehicles in the lane, the adjusted neighbor gain coefficient, and the adjusted historical gain coefficient, including: determining the effective queue length based on the real-time queue length of vehicles in the lane, the historical queue length, and the adjusted historical gain coefficient; correcting the effective queue length according to the predicted queue length and the adjusted neighbor gain coefficient to obtain the target queue length; wherein, the predicted queue length is obtained based on a pre-trained multi-intersection queue length prediction model and the historical queue length corresponding to each single intersection.

[0132] In one embodiment of the present invention, the effective queue length is as follows:

[0133] in, For effective queue length, The value is equal to the adjusted historical gain coefficient. For real-time queue length, This represents the historical queue length.

[0134] In one embodiment of the present invention, the target queue length is as follows: Equation (26) in, For the target queue length, , This is the adjusted neighbor gain coefficient. To improve the accuracy of the prediction model, To predict queue length, The effective queue length.

[0135] In one embodiment of the present invention, the multi-intersection queue length prediction model is as follows:

[0136] in, To predict queue length, It is a SpatioTemporalGraph Neural Network (ST-GNN). For the region All intersections Historical queue length sequence This is a topological diagram of the intersections. To predict the time step.

[0137] In one embodiment of the present invention, the spatiotemporal graph neural network can effectively capture the spatiotemporal dependence of traffic flow, and is suitable for multi-intersection collaborative prediction scenarios.

[0138] In one embodiment of the present invention, the prediction time step With decision-making time step There is a corresponding relationship.

[0139] In one embodiment of the present invention, the vehicle queuing pressure is as follows:

[0140] in, For phase All lanes under management The total number of vehicles queuing.

[0141] In one embodiment of the present invention, the green light duration for one phase is as follows:

[0142] in, For green light time, This is the adjusted basic travel time. For phase queuing matrix, This represents the maximum number of vehicles queuing in the lane managed under this phase. This is the adjusted base offset time.

[0143] In one embodiment of the present invention, the green light time for each phase is calculated by directly using formula (29), and the period length is the sum of the green light times for each phase, thereby realizing variable period adaptive control.

[0144] In one embodiment of the present invention, the phase with the greatest vehicle queuing pressure is selected as the next release phase, and the green light time is calculated using equation (29).

[0145] In one embodiment of the present invention, the maximum number of vehicles queuing in the lane managed under this phase is as follows:

[0146] in, For phase Managed lanes The maximum number of vehicles queuing on the platform.

[0147] In one embodiment of the present invention, typical rectification logic in traffic engineering is encoded into LLM reasoning rules and used as the action space of RL. By performing minor iterative corrections on these parameters with clear physical meaning, refined and interpretable interventions are achieved for adaptive control logic at intersections, such as cycle time, green light duration, and phase selection.

[0148] In one embodiment of the present invention, the current strategy parameters can be evaluated using performance evaluation values ​​and preset performance thresholds to determine whether convergence has occurred, which is also a condition for stopping model training. The corresponding optimal parameter set is as follows:

[0149] in, For the optimal set of parameters, For parameters Performance evaluation values, This is a preset performance threshold.

[0150] The progressive region optimization framework proposed in this invention decouples the large language model from the reinforcement learning model into independent action generation modules, which are then embedded into a multi-scale optimization loop. Please refer to [link / reference]. Figure 3 , Figure 3 This is a logical schematic diagram of progressive region optimization provided in one embodiment of the present invention. Figure 3 As shown, the process begins by collecting multi-source traffic data (real-time / historical / topology / predictive): inputting the original dataset; selecting the optimization level (single intersection / region) based on the current optimization maturity; flexibly switching action sources (probe vs. execute) based on a greedy strategy: in the probing phase, LLM decision-making is used to generate heuristic parameter adjustment suggestions that conform to traffic engineering principles; in the execution phase, RL decision-making is used to achieve precise parameter differentiation; in LLM decision-making: parameter adjustment suggestions, i.e., actions, are constructed, inferred, and output through prompts; in RL decision-making, states are constructed based on the original dataset, thereby inferring and outputting actions; all generated actions are executed and their performance is evaluated in an environment verification module including the actual road network; incremental evolution (stability check / region expansion): incremental evolution from isolated single intersection nodes to collaborative regions is carried out through the stability drive of single intersections, gradually incorporating stable intersections into the regional optimization; regional joint optimization (Joint-R): based on regional joint rewards... Achieve regional optimization.

[0151] Step S240: Store the control experience data of the optimization level within the preset first time period, and optimize the reinforcement learning model corresponding to the optimization level. The control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

[0152] In one embodiment of the present invention, the control experience data includes a single-intersection experience buffer and a regional experience buffer, thereby updating the single-intersection policy and the regional policy. Regional optimization uses a stable set of intersections to trigger dynamic aggregation, and the regional joint reward adopts... .

[0153] In one embodiment of the present invention, the preset first time period corresponding to different optimization levels may be the same or different.

[0154] In one embodiment of the present invention, the preset first time period can be set to 7 days. The buffer time step corresponding to different optimization levels can be different. For example, the first buffer time step corresponding to single intersection optimization is 5 minutes, and the buffer time step corresponding to regional optimization is 60 minutes.

[0155] In one embodiment of the present invention, the present invention can construct buffering mechanisms for long-period data of a single intersection and short-period data of a region respectively, supporting model training at different spatial scales and temporal resolutions.

[0156] In one embodiment of the present invention, deep convergence of the policy network is achieved through a single-intersection empirical buffer, as follows:

[0157] in, As a buffer for single-intersection experience, For a moment The state vector, For a moment The action, For a moment Instant rewards at intersections The moment after the action is performed The state vector, This is a buffer time step for single-intersection experience.

[0158] In one embodiment of the present invention, adaptive updating of the regional policy is achieved through a regional experience buffer, as follows:

[0159] in, As a buffer for regional experience, To optimize the intersection aggregation within the area, To optimize the region Inner Moments Regional unity To optimize the region All intersections within the area are at the designated time. The action, To optimize the region At any moment Regional joint rewards, Optimize the region after the action is performed. All intersections within the area are at the designated time. Regional unity This serves as a buffer time step for regional experience.

[0160] In one embodiment of the present invention, in a reinforcement learning model, actions in the non-probe phase are generated through Proximal Policy Optimization (PPO) or Deep Deterministic Policy Gradient (DDPG).

[0161] In one embodiment of the present invention, in PPO, the policy network is optimized through value network, time difference error, generalized advantage estimation, and policy pruning.

[0162] In one embodiment of the present invention, the value network is as follows:

[0163] in, For state The value estimate, representing the state Initially, the total discount reward expected to be obtained, For the expectation, For a moment Instant rewards As a discount factor, This is the initial state.

[0164] In one embodiment of the present invention, the time difference error is as follows:

[0165] in, For a moment Time difference error, For a moment Instant rewards As a discount factor, For a moment state, For a moment state, For state The value estimate, For state Value estimate.

[0166] In one embodiment of the present invention, Generalized Advantage Estimation (GAE) is:

[0167] in, For a moment The advantage estimate, As a discount factor, For the generalized dominance estimation coefficients, a trade-off between controllable bias and variance is considered. For a moment Time difference error, For time steps, This represents the total number of steps.

[0168] In one embodiment of the present invention, the strategy pruning objective is as follows:

[0169] in, For the current strategy parameters The cutting target, For the expectation, For the current strategy parameters The probability ratio of actions, For a moment The advantage estimate, For the clipping function, This is the cutting factor.

[0170] In one embodiment of the present invention, the probability of the action is as follows:

[0171] in, The probability ratio of actions. To the current strategy parameters Below, at the moment state vector Choose action The probability, For the strategy parameters in the previous round Below, at the moment state vector Choose action The probability of.

[0172] In one embodiment of the present invention, please refer to Figure 4 , Figure 4 This is a logical diagram illustrating reinforcement learning training and deployment according to one embodiment of the present invention. Figure 4As shown, the entire process is decoupled into an online deployment loop and an offline training loop, thus solving the problems of data distribution differences and insufficient online adaptation mechanisms in the migration from simulation to reality. In the online deployment loop: the state S_t is extracted from the traffic environment of the actual road network, the action a_t is extracted by the PPO policy network, and deployed to the traffic environment of the actual road network. The traffic environment provides immediate reward r_t to evaluate the quality of the action. In the training loop: through a many-to-one aggregation method, the state S_t, action a_t, immediate reward r_t corresponding to multiple buffer time steps, and the state S_t+1 after the action are executed are stored in a multi-scale experience buffer; the PPO optimizer periodically samples batch data, calculates the policy loss and value loss after pruning, and thus synchronously updates the weight parameters of the policy network and value network, such as through the Adam optimization algorithm.

[0173] In one embodiment of the present invention, the policy network of the reinforcement learning model is replaced from proximal policy optimization to deep deterministic policy gradient (DDPG).

[0174] In one embodiment of the present invention, the deterministic action output by the Actor network is as follows:

[0175] in, For a moment The action, For Actor networks, For a moment The state.

[0176] In one embodiment of the invention, the Critic network outputs action value:

[0177] in, Let the action value function be... For a moment state, For a moment The action.

[0178] In one embodiment of the present invention, the Critic network employs a target network with soft updates. as follows:

[0179] in, The target value for the Critic network, For a moment Instant rewards As a discount factor, For the target Critic network, To perform the action at time state, For the target Actor network.

[0180] In one embodiment of the present invention, the parameters of the Critic network are updated by minimizing the squared error, as follows:

[0181] in, The time difference error of the Critic network. For a moment state, For a moment The action, The target value for the Critic network, This represents the mean square error.

[0182] In one embodiment of the present invention, the parameters of the Actor network are updated through a deterministic policy gradient, as follows:

[0183] in, Let the policy performance objective function be... For policy gradient, Mean square error, For the Critic network in state Output value related to action The gradient in the action Calculation at point, For the Actor network in state The output value is related to its parameters. The gradient.

[0184] This invention, based on common-sense reasoning using a large language model and progressive adaptive fine-tuning of traffic signal parameters using a reinforcement learning model, offers significant technical advantages in improving control safety, expert knowledge fusion efficiency, and cross-domain transfer capabilities through the organic integration of hybrid intelligent driving and a progressive evolutionary architecture. First, an LLM (Local Language Model) is used as the meta-controller in the exploration phase, employing structured cue words to explicitly constrain parameter adjustment logic. Compared to the stochastic exploration of reinforcement learning in related technologies, this ensures that the fine-tuning process conforms to common-sense traffic engineering principles, such as allowing green lights to remain idle and preventing deadlocks, effectively eliminating the safety risks of black-box decision-making. Furthermore, each parameter adjustment has a clear explanation based on logical reasoning, significantly enhancing the transparency and credibility of the control law.

[0185] This invention significantly enhances the transferability of strategies from simulation to reality through an indirect control method using a standardized parameter system. By fine-tuning parameters with clear physical meaning, such as historical gain coefficients and basic bias time, the risk of overfitting the reinforcement learning model to the absolute state of the environment is reduced, enabling it to adapt to the differences in data distribution at real intersections. This improves robustness under complex time-varying traffic flows without requiring large-scale retraining. Furthermore, the bottom-up progressive evolution strategy, combined with regional joint rewards incorporating variance penalties, enforces constraints on performance differences between sub-nodes during the expansion from single intersections to regional optimization. This effectively resolves local interest conflicts in multi-agent cooperation, achieving a smooth transition and global equilibrium of road network traffic flow.

[0186] This invention significantly reduces the ineffective exploration space by automating the manual rectification logic into the LLM inference process. Combined with a multi-timescale experience replay mechanism, it significantly improves data utilization and model convergence stability. Actual deployment data shows that this method can reduce average waiting time by 15%–25%, vehicle queue length by 20%–30%, and improve parameter convergence stability by 40% in test road networks. Therefore, this invention not only avoids the safety risks and high manual maintenance costs of related technologies but also provides a systematic path for the adaptive optimization of large-scale urban traffic signals that combines engineering feasibility and algorithmic advancement.

[0187] Please see Figure 5 , Figure 5 This is a block diagram of a traffic signal control optimization device provided in one embodiment of the present invention. This device can be applied to... Figure 1 The implementation environment shown is specifically configured in computer device 120. This device can also be applied to other exemplary implementation environments and specifically configured in other devices; this embodiment does not limit the implementation environment to which the device is applicable.

[0188] like Figure 5 As shown, a traffic signal control optimization device 500 according to an embodiment of the present invention includes: an acquisition module 510, an action generation module 520, a traffic signal control module 530, and a model optimization module 540.

[0189] The acquisition module 510 is used to acquire the multi-source traffic data, exploration rate and current parameter vector corresponding to the optimization level at the current time step. The optimization level includes single-intersection optimization or regional optimization composed of multiple single intersections. The parameter vector includes multiple parameters that are used to indirectly control traffic signals and have physical meaning. The action generation module 520 is used to determine the large language model or the corresponding reinforcement learning model as the action source based on the exploration rate, and to determine the current action based on the current state and the action source. The current state is constructed based on multi-source traffic data, and the current action is used to represent the adjustment amount of the current parameter vector. Traffic signal control module 530 is used to determine the adjusted parameter vector based on the current action and the current parameter vector, and to determine the green light time of the target phase based on the adjusted parameter vector, so as to control the traffic signal at the optimization level, obtain new multi-source traffic data, and determine the adjusted parameter vector as the new current parameter vector to continue to control the traffic signal at the optimization level. The model optimization module 540 is used to store the control experience data of the preset first time period optimization level and optimize the reinforcement learning model corresponding to the optimization level. The control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

[0190] It should be noted that the traffic signal control optimization device and the traffic signal control optimization method provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the traffic signal control optimization device provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation here.

[0191] Embodiments of the present invention also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the traffic signal control optimization method provided in the above embodiments.

[0192] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a computer system for an electronic device provided in one embodiment of the present invention. Figure 6 The computer system 600 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0193] like Figure 6As shown, the computer system 600 includes a central processing unit 601, which can perform various appropriate actions and processes based on a program stored in a read-only memory 602 or a program loaded from a storage section 608 into a random access memory 603, such as performing the methods described in the above embodiments. The random access memory 603 also stores various programs and data required for system operation. The central processing unit 601, the read-only memory 602, and the random access memory 603 are interconnected via a bus 604. An input / output interface 605 is also connected to the bus 604.

[0194] The following components are connected to the input / output interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0195] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit 601, it performs various functions defined in the system of the present invention.

[0196] The computer-readable medium shown in the embodiments of the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory (EPROM), flash memory, optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. Computer programs contained on computer-readable media can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0197] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0198] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0199] In the above embodiments, unless otherwise specified, the use of ordinal numbers such as "first" and "second" to describe common objects only indicates that they refer to different instances of the same object, rather than indicating that the objects being described must be in a given order, whether temporally, spatially, sequentially, or in any other way.

[0200] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A traffic signal control optimization method, characterized in that, The method includes: Obtain the multi-source traffic data, exploration rate, and current parameter vector corresponding to the optimization level at the current time step. The optimization level includes single-intersection optimization or regional optimization composed of multiple single intersections. The parameter vector includes multiple parameters that are used to indirectly control traffic signals and have physical meaning. The large language model or the corresponding reinforcement learning model is identified as the action source based on the exploration rate, and the current action is determined based on the current state and the action source. The current state is constructed based on the multi-source traffic data, and the current action is used to characterize the adjustment amount of the current parameter vector. Based on the current action and the current parameter vector, an adjusted parameter vector is determined, and based on the adjusted parameter vector, the green light time of the target phase is determined to control the traffic signal of the optimization level, thereby obtaining new multi-source traffic data. The adjusted parameter vector is then determined as the new current parameter vector, and the traffic signal of the optimization level continues to be controlled. The control experience data of the optimization level within a preset first time period is stored, and the reinforcement learning model corresponding to the optimization level is optimized. The control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

2. The traffic signal control optimization method according to claim 1, characterized in that, The determination of the optimization level includes: If the performance indicators of a single intersection improve within a preset second time period, the single intersection is identified as an active intersection, and the optimization level is determined to be single intersection optimization. The performance indicators are obtained based on the real-time rewards of the single intersection. If the performance parameters of an active intersection do not improve within a preset second time period, the active intersection will be identified as a stable intersection. If there are adjacent stable intersections, and the number of stable intersections is greater than a preset threshold, then the optimization level is determined to be regional optimization, and the adjacent stable intersections are determined to be the optimization region.

3. The traffic signal control optimization method according to claim 1, characterized in that, Before determining the green light time for the target phase based on the adjusted parameter vector, the process also includes: The regional joint state is constructed based on the multi-source traffic data of each single intersection and input into the upper-level strategy to obtain the coordination mode of each single intersection. The upper-level strategy aims to maximize the regional joint reward of the optimized region. If the value type of the coordination mode of a single intersection is continuous, then the current action is scaled. If the value type of the coordination mode of a single intersection is discrete, then the adjustment strategy is frozen or allowed. The adjustment strategy includes optimizing the underlying strategy corresponding to the single intersection and / or adjusting the parameter vector of the corresponding single intersection to obtain the optimization range of the single intersection. The upper-layer strategy and the lower-layer strategy of the single-intersection are obtained based on the corresponding reinforcement learning model.

4. The traffic signal control optimization method according to claim 1, characterized in that, The parameter vector includes a historical gain coefficient for characterizing the weights of the historical data fusion, a basic bias time for characterizing the vehicle start-up loss time, a basic passage time for characterizing the time required to release a unit vehicle, and a neighbor gain coefficient for characterizing the weights of the neighbor prediction information fusion.

5. The traffic signal control optimization method according to claim 4, characterized in that, The large language model associates problem patterns with corresponding adjustment logic to obtain the current action in the early stages of exploration; If the problem pattern is that there is a green light that is not used, then reduce the value of the first type of parameter and / or detect the quality of multi-source traffic data. The first type of parameter includes at least one of the basic travel time, the basic offset time, and the historical gain coefficient. If the problem pattern is that there are vehicles that have not been released, then the value of the second type of parameter is increased and / or a sudden change in traffic demand is detected. The second type of parameter includes at least one of the basic passage time, the basic bias time, and the historical gain coefficient. If the problem pattern is data imbalance in different directions, then the fault status of the corresponding data acquisition device is detected, and / or the correctness of the road configuration is detected; If the problem pattern is characterized by control lag, then the value of the third type of parameter is reduced, and / or the time prediction parameter corresponding to the model is adjusted. The third type of parameter includes the historical gain coefficient and / or the queue length of historical data, and the time prediction parameter includes the decision time step corresponding to the current action and / or the model update frequency corresponding to the reinforcement learning model.

6. The traffic signal control optimization method according to claim 4, characterized in that, Determining the green light time for the target phase based on the adjusted parameter vector includes: The phase with the highest vehicle queuing pressure is determined as the target phase, or each phase of the optimization level is determined as the target phase. The vehicle queuing pressure is used to characterize the total number of target queue lengths of the lanes managed by the phase. The target queue length is determined based on the real-time queue length of vehicles in the lane, the adjusted neighbor gain coefficient, and the adjusted historical gain coefficient. The green light time of the target phase is determined based on the adjusted base offset time, the adjusted base passage time, and the maximum value of the target queue length of the lane managed by the target phase. The adjusted parameter vector includes the adjusted historical gain coefficient, the adjusted base bias time, the adjusted base travel time, and the adjusted neighbor gain coefficient.

7. The traffic signal control optimization method according to claim 6, wherein the target queue length is determined based on the real-time queue length of vehicles in the lane, the adjusted neighbor gain coefficient, and the adjusted historical gain coefficient, comprising: The effective queue length is determined based on the real-time queue length, historical queue length, and adjusted historical gain coefficient of vehicles in the lane. The effective queue length is corrected based on the predicted queue length and the adjusted neighbor gain coefficient to obtain the target queue length; The predicted queue length is obtained based on a pre-trained multi-intersection queue length prediction model and the historical queue length corresponding to each single intersection.

8. The traffic signal control optimization method according to any one of claims 1-7, characterized in that, The regional joint reward for regional optimization includes: The regional joint reward is determined based on the mean and variance of the instantaneous rewards of all single intersections within the optimized region, and the penalty coefficient corresponding to the variance.

9. A traffic signal control optimization device, characterized in that, The device includes: The acquisition module is used to acquire multi-source traffic data, exploration rate and current parameter vector corresponding to the optimization level at the current time step. The optimization level includes single-intersection optimization or regional optimization composed of multiple single intersections. The parameter vector includes multiple parameters that are used to indirectly control traffic signals and have physical meaning. An action generation module is used to determine a large language model or a corresponding reinforcement learning model as an action source based on the exploration rate, and to determine the current action based on the current state and the action source. The current state is constructed based on the multi-source traffic data, and the current action is used to characterize the adjustment amount of the current parameter vector. The traffic signal control module is used to determine the adjusted parameter vector based on the current action and the current parameter vector, and to determine the green light time of the target phase based on the adjusted parameter vector, so as to control the traffic signal of the optimization level, obtain new multi-source traffic data, and determine the adjusted parameter vector as the new current parameter vector to continue to control the traffic signal of the optimization level. The model optimization module is used to store the control experience data of the optimization level within a preset first time period, and optimize the reinforcement learning model corresponding to the optimization level. The control experience data includes the current state, the current action, and the next state vector after executing the adjusted parameter vector.

10. An electronic device, characterized in that, The electronic device includes: One or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the traffic signal control optimization method as described in any one of claims 1 to 8.

Images