Phase sequence-duration coordinated fine-grained artificial intelligence traffic signal control method and device

Abstract

The invention belongs to the technical field of intelligent traffic control, and discloses a phase sequence-duration coordinated fine-grained artificial intelligence traffic signal control method and device. The method comprises the following steps: firstly, obtaining vehicle position and speed data of each entrance lane of a signalized intersection, dividing cells with gradual change lengths, collecting vehicle count and average speed in the cells, constructing a two-channel 3D traffic state tensor, and realizing fine-grained sensing; secondly, a Transform encoder is adopted to process a state tensor output global representation, a state value and a dominant function are subjected to parallel calculation based on a D3QN architecture, Q value overestimation is avoided through a double-Q network, and artificial intelligence decision optimization is completed; thirdly, introducing a priority experience playback mechanism, determining a sample priority and sampling according to a TD error, adjusting loss and optimizing training efficiency in combination with an importance weight, designing a phase sequence-duration coordination action pair, constraining the maximum green light duration and continuous expansion times of a straight-going / left-turning phase sequence, avoiding phase sequence hunger, and realizing fine-grained action control.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of intelligent traffic control, and more particularly relates to a phase-time coordination fine-grained artificial intelligence traffic signal control method and device. BACKGROUND

[0002] The signal intersection is the core node and typical bottleneck area of the urban traffic network, and the signal control efficiency directly determines the road network capacity. The traditional control method (fixed timing, induction control) has significant limitations: the phase sequence and time length of the fixed timing are both preset fixed values, which cannot adapt to dynamic traffic flow and easily cause long queues or green light waste; the induction control only adjusts the time length according to a single flow threshold, without establishing a "phase sequence optimization and time length adaptation" coordination logic, which lacks flexibility and precision and is difficult to cope with complex scenarios.

[0003] The application of artificial intelligence technology (especially deep reinforcement learning) in traffic signal control is gradually increasing, but there are still key defects, and the core problem focuses on "lack of phase-time coordination and insufficient fine-grained support": first, the fine-grained perception is insufficient, existing methods mostly use single indicators such as queue length and vehicle count, which cannot capture the fine-grained differences in space coupling and vehicle speed between lanes, resulting in one-sided perception and difficulty in providing accurate support for phase-time coordination decision-making; second, the Q value estimation bias is large, the "Q value overestimation" problem of traditional DQN easily causes unstable control strategies (such as frequent phase switching and improper time length allocation), which destroys the coordinated matching of phase and time length; third, the sample utilization is low, and the uniform experience replay is insufficient for "phase-time coordination key scene (sudden queue, phase switching boundary)" sample sampling, which slows down the model convergence and affects the iterative optimization of the coordination strategy; fourth, the phase-time coordination is lacking, the action space is multiple, and only the phase sequence or the time length is adjusted, without realizing the fine-grained coordination of the two, and without a dynamic coordination feedback mechanism based on the perception results, which easily causes phase starvation or time length redundancy, and cannot form a "fine-grained perception - phase - time coordination - feedback optimization" closed-loop control.

[0004] Therefore, there is an urgent need for an artificial intelligence technology solution that can realize fine-grained perception, build phase-time coordination logic, stably optimize the control strategy, and efficiently utilize training samples to solve the deficiencies of existing technologies. SUMMARY

[0005] In view of the above-mentioned defects of the prior art, such as "lack of fine-grained perception, lack of phase-time coordination, and lack of artificial intelligence decision-making closed loop", the present application provides a phase-time coordination fine-grained artificial intelligence traffic signal control method, characterized by the following steps: Step 1: Obtain vehicle position and speed data for each entrance lane of the signalized intersection, divide each lane into cells of varying lengths, collect vehicle counts and average speeds within each cell, construct a 2-channel 3D traffic state tensor, and achieve fine-grained traffic state perception. Step 2: The traffic state tensor is flattened, linearly embedded, positionally encoded, and multi-attention layer calculated using a Transformer encoder to output a global traffic state representation, providing perceptual support for phase sequence-duration collaborative decision-making. Step 3: Based on the D3QN architecture, the state value function and the advantage function are computed in parallel. The action Q value is obtained through the dual Q network mechanism of the main network selecting actions and the target network evaluating the target Q value to avoid Q value overestimation, thus building an artificial intelligence decision bridge that perceives phase sequence-duration actions. Step 4: Introduce a priority experience replay mechanism, calculate sample priority based on TD error and sample, and adjust the loss in combination with importance weight to optimize model training efficiency and strengthen the closed-loop learning of phase sequence-duration collaboration. Step 5: Design a "phase sequence-duration" coordinated action pair, constraining the maximum green light duration of the straight-going phase sequence to 60s and the left-turn phase sequence to 30s, and the same phase sequence can be extended a maximum of 3 times consecutively to avoid phase sequence starvation and achieve fine-grained coordinated control of phase sequence and duration. Step 6: Construct a multi-objective reward function based on the number of vehicles in the queue and the average delay per vehicle to provide feedback signals for the phase sequence-duration collaborative closed loop and guide the model to reduce queuing and delay at intersections; Step 7: Train the model using the ε-greedy exploration strategy. After training, input real-time traffic status data and output the optimal signal phase sequence and green light duration adjustment instructions to form a fine-grained artificial intelligence traffic signal closed-loop control with phase sequence-duration coordination.

[0006] In one implementation, a fine-grained sensing core carrier for constructing a two-channel 3D traffic state tensor with phase-time coordination is used. Among them, the channel These are the vehicle counting matrix and the average speed matrix, respectively, with a number of rows. Corresponding to the 120 lanes at the four entrances (east, west, south, and north) of the intersection, the number of rows... Each lane has 10 cells; the lengths of the cells from upstream to downstream are 60m, 40m, 30m, 20m, 10m, 8m, 8m, 8m, 8m, 8m, and 8m respectively. Non-uniform division enables high-granularity perception near the intersection, providing accurate input for phase sequence-duration collaborative decision-making.

[0007] In an embodiment, the Transformer encoder is a phase-duration perception representation key module, which includes an L-layer stacked structure, each layer including a multi-head self-attention module, a dropout module, a residual normalization module, and a feedforward network module; the position encoding is a learnable position vector, which is added to the linearly embedded tensor to capture the relative position relationship of the cells and improve the fine-grained perception spatial correlation of the phase-duration coordination.

[0008] In an embodiment, the priority experience replay mechanism is a phase-duration coordination training optimization component, and the sample priority , is a small constant of 10 -6 , and is a priority coefficient, and the sampling probability ; the importance weight , N is the capacity of the replay buffer; β is a correction coefficient and is linearly increasing; represents the normalized importance weight; it is the value after normalization processing of the original importance weight , and the purpose is to avoid excessive influence of the weight on the loss function, so that the training is more stable; represents the maximum value of all sample importance weights; by dividing the original importance weight by this maximum value, the normalization of the importance weight is realized, which ensures that the weight is within a reasonable range, improves the training efficiency of the PER mechanism, and strengthens the stability of the phase-duration coordination closed-loop learning.

[0009] In an embodiment, the multi-objective reward function is a phase-duration coordination feedback key, and its expression is , wherein is a weight coefficient and takes a value of 0.5, is the number of queued vehicles in the cell area, is the average delay of a single vehicle in the area, is the maximum queuing capacity of the area, is the maximum single vehicle delay, represents the number of delayed vehicles in the area, i.e., the number of vehicles participating in the calculation of the average delay of a single vehicle, which is used to normalize the average delay of a single vehicle, so that the calculation dimensions of the two objectives of queuing and delay in the reward function remain consistent, thereby realizing the balanced optimization of multiple objectives, converting the fine-grained perception result into a feedback signal of phase-duration coordination, and guiding the model to reduce the queuing and delay at the intersection.

[0010] In an embodiment, the epsilon-greedy exploration strategy is a phase-time collaborative exploration guarantee, wherein epsilon is exponentially decayed from 0.1 to 0.01 by a decay factor of 0.995; the target network is updated every 20 steps by a soft update mechanism, balancing the exploration and utilization of the phase-time collaborative closed loop.

[0011] The application also relates to a phase-time collaborative fine-grained artificial intelligence traffic signal control device, comprising: A state acquisition and tensor construction module is configured to acquire signal intersection vehicle position and speed data, divide cells, and construct a 2-channel 3D traffic state tensor, thereby providing fine-grained perception input for phase-time collaboration. A Transformer encoding module is configured to perform flattening, embedding, position encoding, and multi-attention layer calculation on the traffic state tensor, thereby outputting global state representation and linking fine-grained perception and phase-time collaborative decision-making. A D3QN value calculation module is configured to calculate state value functions and advantage functions in parallel based on a duel double Q network architecture, thereby outputting action Q values and providing a basis for phase-time collaborative decision-making. A PER optimization training module is configured to implement priority experience replay, sample based on TD error, and adjust loss weight, thereby optimizing the training efficiency of the phase-time collaborative model. A phase-time collaborative decision-making execution module is configured to generate a phase-time collaborative action pair, control the signal lamp in combination with the time constraint, and execute fine-grained collaborative actions. A reward calculation module is configured to calculate multi-objective rewards based on queuing and delay data, thereby providing feedback signals for phase-time collaboration. A model training and online control module is configured to train the model using an epsilon-greedy strategy, receive real-time perception data after the training is completed, output phase-time collaborative control instructions, and guide fine-grained artificial intelligence traffic signal collaborative control.

[0012] In an embodiment, the state acquisition and tensor construction module acquires vehicle position and speed through a camera and a radar, and maps the vehicle physical coordinates to cell indexes through a coordinate mapping function The vehicle physical coordinates are mapped to cell indexes , thereby accurately realizing data association of fine-grained perception and providing data support for phase-time collaboration.

[0013] The application also relates to an electronic device, comprising at least one processor capable of completing traffic state tensor processing, Transformer encoding, D3QN calculation, and PER optimization, and at least one memory in communication connection with the processor, wherein the memory stores program instructions capable of being executed by the processor and contains model training and online control logic for phase-time collaborative control.

[0014] The application also relates to a computer readable storage medium, which stores program instructions, and the program instructions are executed by a processor to implement the phase-time coordinated fine-grained artificial intelligence traffic signal control method, and the storage and calling of traffic state data, model parameters and training result information.

[0015] In one embodiment, the application provides a phase-time coordinated fine-grained artificial intelligence traffic signal control method, which realizes accurate, stable and efficient signal intersection control and reduces traffic delay and queue length by constructing a closed-loop logic of "fine-grained perception-artificial intelligence cooperative decision-making-phase-time fine-grained control-feedback optimization".

[0016] In a first aspect, the present application provides a phase-time coordinated fine-grained artificial intelligence traffic signal control method, which takes a "phase-time coordinated artificial intelligence decision-making closed loop" as the core logic, and specifically comprises the following steps: Step 1: Collecting vehicle position and speed data of each entrance lane of the signal intersection, dividing each lane into 10 unit cells of gradually changing lengths (upstream to downstream lengths are 60m, 40m, 30m, 20m, 10m, 8m x 4 respectively), based on vehicle counting and average speed in the unit cell, constructing a 2-channel 3D traffic state tensor (channels are vehicle counting matrix and average speed matrix, row number corresponds to 12 lanes of 4 entrance directions, column number corresponds to 10 unit cells of each lane), providing accurate perception input for phase-time coordination, and then constructing fine-grained perception; Step 2: Using a Transformer encoder to flatten, linearly embed, learnable position encode and multi-attention layer calculate the traffic state tensor, output global traffic state representation, accurately capture spatial dependence between lanes, realize the transformation of perception information to coordinated decision data, link perception and phase-time coordinated decision-making, and then process the perception representation; Step 3: Based on the D3QN (dueling double DQN) artificial intelligence decision-making architecture, parallel computing state value function and advantage function, through the double Q network mechanism of "main network selecting coordinated action, target network evaluating target Q value", the phase-time coordinated action Q value is obtained to avoid Q value overestimation, improve the stability of the coordinated strategy, build an artificial intelligence decision-making bridge from perception to phase-time coordination, and then realize artificial intelligence coordinated decision-making; Step 4: Introducing the PER (priority experience replay) mechanism, calculating the priority of "perception-phase-time coordination" samples based on TD error and sampling, combining importance weight adjustment loss, optimizing the utilization efficiency of key traffic state samples, accelerating the convergence of artificial intelligence model, strengthening the closed loop iteration effect of phase-time coordination, and then forming a coordinated closed loop optimization; Step 5: Designing a "phase-time" coordinated action pair (phase is a fixed traffic direction combination of the intersection, and time length adjustment is a discrete value), restricting the maximum green time of the straight phase to 60s, the left turn phase to 30s, and the same phase to be continuously expanded at most 3 times, avoiding phase starvation, realizing coordinated control matching fine-grained perception, and then realizing phase-time fine-grained control; Step 6: Based on the number of queuing vehicles and the average delay of single vehicle, a multi-objective reward function is constructed (Q = 0.5), which provides an artificial intelligence feedback signal for phase-time coordination, guides the model to reduce intersection queuing and delay; Step 7: Using the ε-greedy exploration strategy to train the model (ε decays exponentially from 0.1 to 0.01 with a decay factor of 0.995), after training, input real-time traffic state data, output optimal signal phase and green time adjustment instructions, form a complete phase-time coordinated fine-grained artificial intelligence closed loop control, and then realize the accurate landing of the coordinated closed loop. , =0.5), which provides an artificial intelligence feedback signal for phase-time coordination, guides the model to reduce intersection queuing and delay; Step 7: Using the ε-greedy exploration strategy to train the model (ε decays exponentially from 0.1 to 0.01 with a decay factor of 0.995), after training, input real-time traffic state data, output optimal signal phase and green time adjustment instructions, form a complete phase-time coordinated fine-grained artificial intelligence closed loop control, and then realize the accurate landing of the coordinated closed loop.

[0017] In a second aspect, the present application provides a fine-grained artificial intelligence traffic signal control device oriented to phase sequence-time length cooperation, which sets functional modules around a "phase sequence-time length cooperation artificial intelligence closed loop", and each module cooperates to realize cooperative control, specifically including: 1. State acquisition and tensor construction module (fine-grained perception end), used to obtain signal intersection vehicle position and speed data, divide cells and construct a 2-channel 3D traffic state tensor, to provide accurate perception input for phase sequence-time length cooperation; 2. Transformer encoding module (perception processing end), used to flatten, embed, position encode and calculate multiple attention layers for the traffic state tensor, output global state representation, and realize the conversion of perception information to cooperative decision data; 3. D3QN value calculation module (decision core), used to calculate state value function and advantage function in parallel based on a duel double Q network architecture, output phase sequence-time length cooperation action Q value, generate cooperative decision basis, and link perception and cooperative action; 4. PER optimization training module (closed loop optimization end), used to realize priority experience replay, sample "perception-phase sequence-time length cooperation" samples based on TD error and adjust loss weight, and optimize cooperative closed loop training efficiency; 5. Action decision and execution module (fine-grained action end), used to generate a "phase sequence-time length" cooperative action pair, control signal lights combined with time length constraints, and execute cooperative closed loop action output; 6. Cooperative reward calculation module (closed loop feedback end), used to calculate multi-objective rewards based on queuing and delay data (perception results), and provide feedback signals for phase sequence-time length cooperation; 7. Artificial intelligence model training and online control module (cooperative center), used to train the model using an ε-greedy strategy, receive real-time perception data and output cooperative control instructions after training, and guide fine-grained artificial intelligence control operation of phase sequence-time length cooperation.

[0018] In a third aspect, the present application provides an electronic device including a memory and a processor, the memory storing program instructions executable by the processor, and the processor invoking the program instructions to execute the phase sequence-time length cooperation method as described in the first aspect: complete traffic state tensor construction (cooperative perception), Transformer encoding (perception processing), D3QN artificial intelligence value calculation (cooperative decision), PER cooperative closed loop optimization (training optimization), and phase sequence-time length cooperation decision (action output), to provide hardware support for phase sequence-time length fine-grained artificial intelligence control of signal intersections.

[0019] In a fourth aspect, the present application provides a computer-readable storage medium having stored thereon a computer program and experimental / training data related to the phase-time coordination. The computer program can perform the coordination method as described in the first aspect: relying on the input traffic state data (coordination perception source), completing artificial intelligence model training and real-time coordination control; the stored experimental / training data includes: traffic state tensor samples (coordination perception data), Transformer encoding parameters (perception processing parameters), D3QN model weights (coordination decision parameters), PER buffer "perception-phase-time coordination" samples (closed-loop samples), and rewards and performance indicators in the training process (coordination feedback data), providing data support for the reuse and optimization of the phase-time coordination method.

[0020] In a fifth aspect, the present application provides a display device and an input device, which provide interactive support for the phase-time coordination fine-grained artificial intelligence control. The display device can be used to visualize the traffic state tensor (coordination perception result), the phase-time coordination control instruction (action output), and the intersection operation performance (queue length, delay, etc., coordination effect), facilitating the monitoring of the coordination control operation state by the management personnel; the input device is used to input the cell division parameters (coordination perception configuration), the hyperparameters of the Transformer and D3QN (coordination decision configuration), and the key threshold values for traffic impact assessment (such as the maximum queue capacity and the maximum single vehicle delay, coordination feedback configuration), realizing the customized adaptation of the phase-time coordination control.

[0021] The present application integrates "precise state representation, artificial intelligence stable decision, efficient sample utilization, and phase-time coordination output" through the phase-time coordination fine-grained artificial intelligence control, significantly improving the convergence speed and control effect of the signal control at the signal intersection: compared with the traditional DQN, the model convergence speed is improved by 40%; compared with the induction control, the cumulative delay is reduced by 71% and the queue length is reduced by 85%, which has strong adaptability and practicality in complex urban traffic environment, effectively optimizing the traffic signal control efficiency and intersection passing capacity. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 Flowchart of the phase-time coordination fine-grained artificial intelligence traffic signal control method; Figure 2 Artificial intelligence signal control model architecture based on Transformer and D3QN; Figure 3 Traffic state tensor and Transformer encoding process schematic diagram; Figure 4 Signal intersection SUMO simulation scene schematic diagram; Figure 5 Structure diagram of phase sequence-time length coordinated fine-grained artificial intelligence traffic signal control device; Figure 6 Entity structure diagram for electronic device. DETAILED DESCRIPTION

[0023] To make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solution of the phase sequence-time length coordinated fine-grained artificial intelligence traffic signal control in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0024] Figure 1 The flowchart of the phase sequence-time length coordinated fine-grained artificial intelligence traffic signal control method is shown in FIG. 1. The method provided by the embodiments of the present application is applied to intelligent control of traffic signals at a signalized intersection. The core revolves around the closed-loop logic of “fine-grained perception data driving phase sequence-time length coordination adjustment, and feedback optimization of perception-coordination matching of the execution result”. Specifically, the method comprises the following steps: Figure 1 In the fine-grained phase sequence-time length coordination execution and closed-loop feedback stage, the system first retrieves the green light time length of the previous execution phase sequence from the time length table based on the phase sequence type data (such as the current execution phase sequence being straight or left turn) output by the fine-grained perception module, and then applies the adjustment amount of the current coordination action for fine-grained adjustment. This adjustment logic is directly related to the perceived phase sequence function difference. To adapt to the phase sequence type difference (such as straight phase sequence and left turn phase sequence), a differentiated maximum green light time length threshold is set: 60 seconds for straight phase sequence and 30 seconds for left turn phase sequence, to ensure that the adjusted time length matches the perceived phase sequence traffic demand and guarantees the operation safety and feasibility of the coordination strategy.

[0025] To maintain the closed-loop synchronization of “perception-phase sequence-time length coordination-simulation”, the system resets the updated green light time length at the beginning of each signal cycle based on the perceived phase sequence execution history data in the cycle, and only recalculates the effective green light time without a yellow light phase. Specifically, the cumulative green light time allocated to the current phase sequence in the current cycle is first calculated Then, the pre-defined maximum phase sequence time length upper limit (constraint value corresponding to the perceived phase sequence function) is combined to determine the remaining allowed time length . Given the agent output adjustment amount , the final executed green light time in this round is ​The minimum value of "Adjustment Amount + Cumulative Duration" and "Remaining Allowable Duration" ( ) Determined. If If the current phase has reached the maximum permissible duration, the system immediately terminates its execution and switches to the yellow light transition phase. Then, it selects a new phase from the candidate phase set (green light phases not currently in operation) based on the finely granular perception of vehicle queuing pressure at each phase. The system further evaluates whether the adjusted duration meets the agent's expectations based on the perceived state. When the actual duration is less than expected, it means the duration has been truncated by the remaining budget, and the yellow light phase is executed to complete the transition; if In other words, if the actual duration equals the expected duration, then based on two criteria, "whether the current lane group still maintains the highest queuing pressure (real-time perception data)" and "whether the number of consecutive extensions has not exceeded the limit (closed-loop historical records)," the subsequent rounds will be evaluated to determine whether to extend the phase. At the same time, a strict limit (three times) is introduced on the number of consecutive extensions to prevent closed-loop imbalance and deadlock caused by a single phase dominating.

[0026] Furthermore, the system records the execution order, duration, and corresponding fine-grained sensing data (such as vehicle counts and speeds in each cell) of each phase sequence within each signal cycle, and archives the logs at the end of the cycle. This data is not only used for subsequent performance evaluation and training result verification, but also provides feedback for the iterative optimization of the phase sequence-duration coordinated closed loop. For example, by analyzing the correlation between "sensing state - phase sequence - duration coordinated action - traffic efficiency" in the logs, the granularity of cell division or the step size of coordinated action adjustment can be adjusted. This action design based on phase sequence-duration fine-grained coordination, combined with dynamic feedback from sensing data, realizes a systematic transformation from traditional fixed-cycle, fixed-timing control to high-frequency adaptive signal control driven by "sensing-phase sequence-duration coordinated closed loop".

[0027] In the fine-grained phase-time coordinated action space design (closed-loop core component), the action space of the agent in the fine-grained signal timing environment constructed by this invention is designed around a dual-component coordinated mechanism of "phase sequence selection + duration adjustment". The action set is discrete and structured, forming a precisely matched coordinated response mechanism with fine-grained traffic state perception. Action decision-making is divided into two layers: .in The index representing the selected green light phase reflects a high-level collaborative decision on the priority direction of passage based on real-time fine-grained perception of the status (such as the queue density of each phase cell); To represent the fine-grained adjustment of the selected green phase sequence length, the agent is allowed to incrementally shorten, maintain, or lengthen the green light time in fixed steps — this design decouples phase sequence selection and green light time allocation, enabling the agent to select the currently most critical phase sequence based on the perception state and responsively adjust its length, improving the time resolution without significantly increasing the action dimension, supporting the precise matching and closed-loop stable operation of “perception-phase sequence-length coordination” in dynamic traffic environments, and facilitating more efficient artificial intelligence policy learning.

[0028] Figure 2 The phase sequence-length coordination fine-grained artificial intelligence traffic signal control model architecture of the present application is shown in Figure 2 The architecture takes “fine-grained artificial intelligence closed loop of phase sequence-length coordination” as the core and is built based on the agent-environment interaction paradigm of deep reinforcement learning, as follows: Within the architecture framework, the traffic environment evolves driven by SUMO, the agent as the controller periodically receives the 2-channel 3D traffic state tensor (including cell vehicle count, average speed) output by the fine-grained perception module, and relies on the artificial intelligence policy network to select the adaptive operation (such as phase sequence index, green light length ±1s adjustment) from the “phase sequence-length coordination” action set, realizing the precise matching of perception and coordination action. After the action is executed, the environment updates the traffic state (such as the change of cell vehicle distribution), and returns the immediate reward calculated based on the number of queued vehicles and the average delay per vehicle — this reward not only quantifies the effect of the coordination action, but also provides the basis for coordination feedback. Through repeated interaction of “perception → artificial intelligence coordination decision → execution → update → feedback → optimization”, the agent accumulates “perception-phase sequence-length coordination” experience and iterates the policy, and finally converges to a fine-grained coordination control strategy with strong generalization ability.

[0029] The complete phase sequence-length coordination closed loop control process can be formally described by the MDP five-tuple as shown in Figure 2 , where: represents the fine-grained traffic state space, which is characterized by a 2-channel 3D traffic state tensor, accurately reflecting the real-time running state of each cell at the intersection; represents the fine-grained “phase sequence-length” coordination action space, defining the “phase sequence selection + length adjustment” combined operation executable by the agent; P is the state transition function, where characterizes the probability distribution of transitioning from state s to under action ; R is the immediate reward function; is the discount factor, determining the weight of future rewards in the agent's policy update.

[0030] Based on this, the application develops a complete modeling framework around the phase-time collaborative closed loop: integrating fine-grained state perception (gradual cell, 3D tensor), "phase-time" collaborative action space, and perception-driven reward function; while integrating the Transformer state encoder (processing tensor output global representation) and the D3QN artificial intelligence architecture (stable calculation of collaborative action Q value to avoid overestimation), the authenticity and stability of the phase-time collaborative closed loop are ensured while optimizing the control strategy.

[0031] Figure 3 The traffic state tensor and the Transformer encoding process of the application are shown in the schematic diagram as Figure 3 The process is the core link of fine-grained perception of "phase-time collaboration", which provides high-quality perception input for subsequent phase-time collaborative decision-making and supports stable operation of the collaborative closed loop by accurately constructing tensors and efficiently encoding, as follows: First, the cell dimension design of the traffic state tensor (fine-grained perception basis). The cell dimension is designed according to the default vehicle length (5m) and the minimum distance (2.5m) of SUMO to achieve vehicle-level fine-grained perception. The downstream 8m small cell can accurately locate a single vehicle near the intersection (whose state directly affects the "phase-time" collaborative adjustment), and the upstream longest 60m large cell can accommodate about 8 vehicles in a vehicle queue, realizing global coverage of incoming vehicles. This hierarchical design balances high-granularity perception near the stop line and upstream traffic monitoring, providing "near-intersection vehicle precise position" data for collaborative decision-making, and laying the foundation for forward-looking adjustment of the phase-time collaborative closed loop through upstream vehicle queue prediction. Taking a 3x3 intersection (3 entry lanes in each direction) as an example, a total of 120 observation cells are formed, and the right side of the entry lane and the unit index are shown in Figure 3 Each unit is the minimum collection unit for fine-grained perception.

[0032] Second, the spatial discretization construction of the state space (fine-grained perception adaptation). To match the "phase-time" fine-grained collaborative decision-making requirements, the 200m road section upstream of the intersection center (each entry lane) is divided into 10 cells, using a "upstream to downstream gradually shortened" non-uniform strategy, with length configurations of 60m, 40m, 30m, 20m, 10m, followed by five 8m cells. This design focuses on the key areas of collaborative decision-making: the 8m cell near the stop line improves the perception resolution (vehicle queue / traffic dynamic decision green light duration adjustment range), and the upstream large cell captures the macro traffic trend (assists phase switching decision), laying the foundation for the phase-time collaborative closed loop.

[0033] Finally, there is the Transformer encoding process. Based on the fine-grained traffic state tensor, the Transformer encoding process sequentially executes the following steps: "Tensor Flattening → Linear Embedding → Learnable Position Encoding → Multi-Attention Layer Computation". First, flattening the 3D traffic state tensor into a 1D vector is called tensor flattening. Second, projecting the vector onto a high-dimensional semantic space using a weight matrix to obtain the embedding vector is called linear embedding. Third, adding learnable position vectors to capture the relative position information of cells (such as the spatial correlation between different cells in the traffic flow direction) is called learnable position encoding. Finally, the processed vector is input into a multi-layered stacked Transformer encoder (each layer containing multi-head self-attention, dropout, residual normalization, and a feedforward network), outputting a global traffic state representation that accurately depicts the spatial dependencies between vehicles in lanes. This provides input for subsequent phase-time collaborative decision-making in D3QN, effectively connecting fine-grained perception with collaborative decision-making. This step is called multi-attention layer computation.

[0034] Figure 4 This is a schematic diagram of a SUMO simulation scenario for a signalized intersection with fine-grained perception-action closed loop as described in this invention. Figure 4 As shown, the core of the simulation scenario constructed in this embodiment is used to verify the effectiveness of the "fine-grained artificial intelligence control of phase sequence-duration coordination" method. The practicality of the coordinated closed-loop logic is ensured by reproducing real traffic scenarios. Specific content includes: First, a framework is built around the phase-duration collaborative closed loop, based on the deep reinforcement learning agent-environment interaction paradigm: integrating fine-grained state perception (gradual cell division, 2-channel 3D tensor), "phase-duration" collaborative action space, and perception-driven reward function; at the same time, it integrates Transformer state encoder (processing perception data) and D3QN artificial intelligence architecture (stabilizing collaborative action decision-making), so as to restore the phase-duration collaborative closed loop logic of "perception-artificial intelligence collaborative decision-execution-feedback" while optimizing the strategy, ensuring that the verification scenario is consistent with the actual application.

[0035] Secondly, a four-way intersection scenario on a main urban road is used, with each entrance lane being 300 meters long (covering the upstream 200-meter cell range required for fine-grained perception). Each entrance direction is divided into three dedicated lanes: left turn, straight ahead, and a combined straight and right turn lane (see layout). Figure 4 — This configuration ensures that fine-grained perception can capture vehicle dynamics in different lanes, providing differentiated perception input for phase sequence-duration collaborative decision-making.

[0036] Furthermore, a fixed four-phase sequence scheme (main road protection for straight traffic, dedicated left-turn traffic, straight-right combination, and other conflict-free traffic flow combinations) is adopted. This not only avoids interference within the phase sequence but also provides the agent with a clear "phase sequence selection" dimension. It provides a standardized phase sequence basis for the fair verification of the phase sequence-duration coordinated closed-loop strategy and focuses on comparing the fine-grained control effect of "phase sequence-duration coordination".

[0037] Finally, the traffic environment evolves under SUMO-driven conditions, and the agent executes a collaborative closed-loop interaction: periodically receiving a 2-channel 3D traffic state tensor (including cell vehicle counts and speeds), and selecting "phase sequence-duration" collaborative actions (such as phase sequence switching and green light duration adjustment by ±1s) based on an AI policy network; after the action is executed, the environment updates the vehicle distribution status and returns an immediate reward calculated based on cell queue counts and single-vehicle delays (quantifying the impact of collaborative actions on closed-loop efficiency). Through repeated interactions of "perception → AI collaborative decision-making → execution → feedback → optimization," the agent iterates its strategy and converges to a fine-grained collaborative control strategy with strong generalization capabilities, verifying the effectiveness of the phase sequence-duration collaborative closed-loop method of this invention.

[0038] Figure 5 A schematic diagram of the fine-grained artificial intelligence traffic signal control device with phase sequence-duration coordination described in this invention is shown below. Figure 5 As shown, the device is structured around a closed-loop logic of "fine-grained perception → AI-assisted decision-making → phase sequence-duration control → feedback," with each module working together to achieve fine-grained intelligent control of intersection signals, as detailed below: The State Acquisition and Tensor Construction Module 501 (Fine-grained Perception Input): acquires real-time vehicle position and speed data for each entrance lane through devices such as cameras and radar; according to the rule of "dividing each lane into 10 cells of gradually varying lengths", it counts the number of vehicles and the average speed within each cell, generating a 2-channel 3D traffic state tensor (the channels are the vehicle count and average speed matrices, the number of rows corresponds to the 4 entrance directions, and the number of columns corresponds to 3 lanes × 10 cells), providing accurate perception input for phase sequence-time coordination.

[0039] Transformer Encoding Module 502 (Perception-Cooperative Decision-Making Connection): Receives 3D traffic state tensors and sequentially executes the process of "tensor flattening → linear embedding → learnable position encoding → multi-attention layer computation" to output a global traffic state representation that characterizes the spatial dependence between lanes. This transforms fine-grained perception data into features adapted to phase-sequence-time cooperative decision-making, thus achieving the connection between perception and phase-sequence-time cooperative decision-making.

[0040] D3QN Value Calculation Module 503 (Collaborative Decision Core): Based on the D3QN architecture, it calculates the state value function and advantage function in parallel to obtain the action Q value; it adopts a dual Q mechanism of "main network selects collaborative actions and target network evaluates values" to avoid overestimation of Q value and provides a stable decision basis for fine-grained phase sequence-duration collaborative action selection.

[0041] PER Optimization Training Module 504 (Collaborative Closed-Loop Training Optimization End): Calculates the TD error of the "perception-phase sequence-duration collaboration" experience samples through the PER mechanism and sets priority sampling. Combines importance weights to correct the loss, optimizes the utilization efficiency of key samples in phase sequence-duration collaboration, and accelerates the convergence of the artificial intelligence collaborative model.

[0042] Action Decision and Execution Module 505 (Fine-grained Cooperative Action Output): Generates "phase sequence-duration" cooperative action pairs (4 fixed phase sequences, {-1s, 0s, +1s} duration adjustment amounts), constrains the maximum green light duration and continuous extension number of the straight / left turn phase sequence, outputs the optimal control command, and realizes fine-grained phase sequence-duration cooperative execution that matches the perception.

[0043] Reward Calculation Module 506 (Collaborative Closed-Loop Feedback End): Based on perception data (number of vehicles in queue, average delay per vehicle), a multi-objective reward function is constructed to quantify the impact of phase sequence-duration collaborative actions on traffic efficiency, provide feedback signals for the artificial intelligence collaborative model, and guide the iterative optimization of the artificial intelligence collaborative strategy.

[0044] Model training and online control module 507 (cooperative closed-loop hub): It adopts the ε-greedy strategy to complete the training of artificial intelligence model; after training, it receives sensing data in real time, calls the model to output phase sequence-duration cooperative control commands, and leads the online cooperative closed-loop operation of "sensing-artificial intelligence cooperative decision-phase sequence-duration control-feedback".

[0045] Figure 6 This is a schematic diagram of the physical structure of the fine-grained artificial intelligence traffic signal control electronic device with phase sequence-duration coordination provided in an embodiment of the present invention. This device is a dedicated hardware platform for implementing the "fine-grained artificial intelligence control with phase sequence-duration coordination" method. Its core function is to run a collaborative algorithm process of "Transformer encoding + D3QN artificial intelligence decision-making + PER collaborative optimization" to achieve full-process intelligentization from fine-grained perception to phase sequence-duration coordinated execution, as detailed below: Reference Figure 6The electronic device includes a processor 601, a memory 602, and a bus 603. The processor 601 communicates with the memory 602 via the bus 603. The processor 601 calls program instructions from the memory 602 to execute the entire process of the phase-time collaborative method: dividing vehicle data collected by cameras and radar into gradually changing length cells to construct a 2-channel 3D traffic state tensor (fine-grained perception input); calling the Transformer encoder to output a global state representation (providing processed features for phase-time collaborative decision-making); calculating the Q-value of phase-time collaborative actions based on the D3QN architecture (collaborative decision-making); optimizing sample utilization through the PER mechanism (collaborative training); generating phase-time collaborative action pairs and executing instructions (fine-grained collaborative action output); calculating rewards based on queue count and delay (collaborative feedback); and completing the training and online control of the artificial intelligence model.

[0046] This invention also provides a computer-readable storage medium 604 (connected to the processor 601 via a bus 603), which stores core data supporting phase-sequence-duration collaborative multiplexing and optimization: Transformer encoder parameters, D3QN network weights, PER buffer "perception-phase-sequence-duration collaboration" samples, SUMO simulation configuration, etc. When the intersection structure matches historical scenarios with traffic flow characteristics, the pre-trained artificial intelligence collaborative model can be directly invoked without retraining, improving the response speed of phase-sequence-duration collaborative control; at the same time, the reward curve, convergence index, and historical control commands of the training process are stored, which are used for collaborative model performance evaluation and strategy tracing analysis, respectively.

[0047] In addition, a display device 605 and an input device 606 (both connected to the processor 601 via a bus 603) are configured: the display device 605 visualizes information about each stage of the collaboration, including 3D tensor heatmaps (fine-grained perception results), real-time "phase sequence-duration" collaboration commands (collaborative action output), and intersection operation indicators (collaboration effect), supporting managers to monitor the collaboration status and intervene manually; the input device 606 is used for customized parameter configuration of phase sequence-duration collaboration, including cell division rules, Transformer / D3QN hyperparameters, traffic assessment thresholds, etc., to adapt to the collaborative control needs of different intersections.

[0048] The electronic devices and other embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0049] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute the phase sequence-duration coordination method described in various embodiments or some parts of embodiments.

[0050] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A fine-grained artificial intelligence traffic signal control method based on phase sequence-duration coordination, characterized in that, Includes the following steps: Step 1: Obtain vehicle position and speed data for each entrance lane of the signalized intersection, divide each lane into cells of varying lengths, collect vehicle counts and average speeds within each cell, construct a 2-channel 3D traffic state tensor, and achieve fine-grained traffic state perception. Step 2: The traffic state tensor is flattened, linearly embedded, positionally encoded, and multi-attention layer calculated using a Transformer encoder to output a global traffic state representation, providing perceptual support for phase sequence-duration collaborative decision-making. Step 3: Based on the D3QN architecture, the state value function and the advantage function are computed in parallel. The action Q value is obtained through the dual Q network mechanism of the main network selecting actions and the target network evaluating the target Q value to avoid Q value overestimation, thus building an artificial intelligence decision bridge that perceives phase sequence-duration actions. Step 4: Introduce a priority experience replay mechanism, calculate sample priority based on TD error and sample, and adjust the loss in combination with importance weight to optimize model training efficiency and strengthen the closed-loop learning of phase sequence-duration collaboration. Step 5: Design a "phase sequence-duration" coordinated action pair, constraining the maximum green light duration of the straight-going phase sequence to 60s and the left-turn phase sequence to 30s, and the same phase sequence can be extended a maximum of 3 times consecutively to avoid phase sequence starvation and achieve fine-grained coordinated control of phase sequence and duration. Step 6: Construct a multi-objective reward function based on the number of vehicles in the queue and the average delay per vehicle to provide feedback signals for the phase sequence-duration collaborative closed loop and guide the model to reduce queuing and delay at intersections; Step 7: Train the model using the ε-greedy exploration strategy. After training, input real-time traffic status data and output the optimal signal phase sequence and green light duration adjustment instructions to form a fine-grained artificial intelligence traffic signal closed-loop control with phase sequence-duration coordination.

2. The method according to claim 1, characterized in that, Constructing a fine-grained perception core carrier for 2-channel 3D traffic state tensors with phase sequence-temporal coordination. Among them, the channel These are the vehicle counting matrix and the average speed matrix, respectively, with a number of rows. Corresponding to the 120 lanes at the four entrances (east, west, south, and north) of the intersection, the number of rows... Each lane has 10 cells; the lengths of the cells from upstream to downstream are 60m, 40m, 30m, 20m, 10m, 8m, 8m, 8m, 8m, 8m, and 8m respectively. Non-uniform division enables high-granularity perception near the intersection, providing accurate input for phase sequence-duration collaborative decision-making.

3. The method according to claim 1, characterized in that, The Transformer encoder is a key module for phase-time perception representation, containing an L-layer stacked structure. Each layer includes a multi-head self-attention module, a dropout module, a residual normalization module, and a feedforward network module. The position encoding is a learnable position vector, which is added to the tensor after linear embedding to capture the relative positional relationship of cells and improve the fine-grained perception of spatial correlation of phase-time collaboration.

4. The method according to claim 1, characterized in that, The priority experience replay mechanism is a phase-sequence-duration collaborative training optimization component, with sample priority. , 10 -6 Small constant, Priority coefficient, sampling probability Importance weight N is the playback buffer capacity; β is the correction coefficient and increases linearly. This represents the importance weights after normalization. It is the original importance weight. The purpose of normalizing the values ​​is to avoid excessive weights having an undue impact on the loss function, thus making training more stable. This represents the maximum value among all sample importance weights; by using the original importance weights... Dividing by this maximum value normalizes the importance weights, ensuring they are within a reasonable range, improving the effectiveness of the PER mechanism in optimizing training efficiency, and strengthening the stability of phase-duration collaborative closed-loop learning.

5. The method according to claim 1, characterized in that, The multi-objective reward function is key to phase-sequence-duration collaborative feedback, and its expression is: ,in This is the weighting coefficient and has a value of 0.

5. This represents the number of vehicles queuing within the cell range. The average delay per vehicle within the area. To the maximum queuing capacity of the area, For maximum single-vehicle delay, This represents the number of delayed vehicles within the area, i.e., the number of vehicles participating in the calculation of the average delay per vehicle. It is used to normalize the average delay per vehicle term, so that the calculation dimensions of the two objectives of queuing and delay in the reward function are consistent, thereby achieving a balanced optimization of multiple objectives. It transforms the fine-grained perception results into a feedback signal of phase sequence-duration coordination, guiding the model to reduce queuing and delay at intersections.

6. The method according to claim 1, characterized in that, The ε-greedy exploration strategy ensures phase-sequence-duration coordinated exploration, where ε decays exponentially from 0.1 to 0.01 by a decay factor of 0.995; the target network is updated every 20 steps through a soft update mechanism to balance the exploration and utilization of the phase-sequence-duration coordinated closed loop.

7. A fine-grained artificial intelligence traffic signal control device with phase sequence-duration coordination, characterized in that, include: State Acquisition and Tensor Construction Module: Used to acquire vehicle position and speed data at signalized intersections, divide cells and construct a 2-channel 3D traffic state tensor, providing fine-grained perception input for phase sequence-time coordination; Transformer encoding module: used to flatten, embed, position encode and multi-attention layer computation of traffic state tensors, output global state representation, and connect fine-grained perception with phase sequence-duration collaborative decision-making; D3QN Value Calculation Module: Based on a duel-style dual-Q network architecture, it calculates the state value function and advantage function in parallel, outputs the action Q value, and provides a basis for phase sequence-duration collaborative decision generation. PER Optimized Training Module: Used to implement priority experience replay, adjust loss weights based on TD error sampling, and optimize the training efficiency of the phase sequence-duration collaborative model; Phase sequence-duration collaborative decision-making and execution module: used to generate "phase sequence-duration" collaborative action pairs, combine them with duration constraint control traffic lights, and execute fine-grained collaborative actions; Reward calculation module: used to calculate multi-objective rewards based on queuing and delay data, providing feedback signals for phase sequence-duration coordination; Model training and online control module: Used to train the model using the ε-greedy strategy, and after training, receive real-time sensing data and output phase sequence-duration collaborative control commands to lead fine-grained artificial intelligence traffic signal collaborative control.

8. The apparatus according to claim 7, characterized in that, The state acquisition and tensor construction module obtains vehicle position and speed through cameras and radar, and uses coordinate mapping functions. Map vehicle physical coordinates to cell indexes It accurately realizes fine-grained data association and provides data support for phase sequence-duration collaboration.

9. An electronic device, characterized in that, include: At least one processor is capable of executing the fine-grained artificial intelligence traffic signal control method of phase sequence-duration coordination as described in any one of claims 1 to 6, and completing traffic state tensor processing, Transformer encoding, D3QN calculation and PER optimization; at least one memory is communicatively connected to the processor, the memory storing program instructions that can be executed by the processor, including model training and online control logic for phase sequence-duration coordination control.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions, which, when executed by a processor, implement the method as described in any one of claims 1 to 6, including the storage and retrieval of traffic state data, model parameters, and training result information.

Images