Autonomous traffic signal control subarea division method, device, medium and product

Abstract

The invention discloses an autonomous traffic signal control subarea division method and device, a medium and a product, and relates to the field of traffic signal control, and the method comprises the steps: constructing traffic signal control agent models of different sizes of a regional road network based on a deep Q network, and constructing a multi-intersection cooperation model network; dividing the multi-intersection collaborative model network into a plurality of signal control units, and determining different signal control unit combinations; constructing an optimization model by taking the minimum road network maximum queuing length of the credit control unit combination as a target and taking time complexity and space complexity constraints of the credit control unit combination as constraint conditions; and according to the optimization model, determining an optimal signal control unit combination by adopting a genetic algorithm, and carrying out traffic signal control sub-region division according to the optimal signal control unit combination. According to the invention, the traffic signal control sub-areas can be accurately divided, so that the accuracy and the control effect of traffic signal control are improved.

Description

Technical Field

[0001] This application relates to the field of traffic signal control, and in particular to an autonomous traffic signal control sub-area division method, device, medium and product. Background Technology

[0002] With rising living standards and advancements in the automotive industry, people's travel demands are constantly increasing, leading to growing urban traffic pressure and increasingly severe congestion. However, urban land resources are limited, especially in megacities and first-tier cities. Simply building new roads or widening existing ones is no longer an effective way to alleviate traffic pressure; in fact, it may attract even more traffic. The healthy development of transportation is fundamental to the rapid socio-economic development of cities. Timely solutions to traffic congestion can greatly contribute to urban development.

[0003] Optimizing traffic flow at intersections is an effective way to alleviate congestion. Urban intersections typically employ signal control, managing vehicle and pedestrian traffic through carefully designed signal timing schemes. However, signal control can disrupt traffic flow; improper timing or a surge in traffic volume can easily trigger congestion, especially at intersections in peak hours or at key locations such as scenic spots or city entrances. Once congestion occurs, it quickly spreads to surrounding areas, paralyzing local traffic. Therefore, coordinated control of key intersections and their surrounding areas is necessary to create key sub-regions. Currently, urban traffic signal control is trending towards regional coordinated control, or "area control." Quickly and accurately identifying the areas requiring coordinated control is crucial for improving the effectiveness of regional traffic control. The division of traffic control units integrates adjacent intersections with similar traffic characteristics into a single control unit for unified management, maximizing traffic efficiency and optimizing urban traffic management.

[0004] With the rapid development of artificial intelligence technology and the widespread adoption of the internet and wireless communication technologies, data acquisition has become more convenient. In intelligent traffic signal control, methods based on deep reinforcement learning are increasingly being widely applied. Traditional signal control methods are typically suitable for single-point, trunk line, or small-area control, characterized by simple equipment and convenient maintenance. However, traditional methods rely on precise mathematical modeling, and when applied to large-scale road networks, the accumulation of errors leads to a decline in regional control effectiveness. Therefore, how to achieve efficient signal control in complex, large-scale road networks has become a key research focus.

[0005] Reinforcement learning methods do not rely on precise model building, and are more adaptive and have stronger generalization capabilities. However, signal control methods based on reinforcement learning also have their own weaknesses: ① When the controlled traffic units become more extensive, the state-action space of a model will experience dimensionality explosion as the number of intersections increases; ② When transmitting global information, information loss is easily caused by high latency and failure rate, resulting in reduced control effectiveness; ③ Larger models require more resources, have higher costs, and the control accuracy will also decrease with larger models.

[0006] Traffic zoning is a preliminary step in urban traffic management and regional signal control. Researchers have proposed various evaluation indicators to assess the effectiveness of traffic sub-zone division in their studies of these methods. However, these evaluation methods focus on the division results, assessing the rationality and effectiveness of sub-zone division from the perspectives of homogeneity of the internal structure of the area, division quality, model optimization, spatiotemporal characteristics, correlation, independence, and overall stability. They do not consider the feasibility from the perspective of signal control.

[0007] Therefore, how to control the size of the signal control unit within a reasonable range, so as to use signal control methods based on deep reinforcement learning while ensuring a certain level of regional control performance, is a technical problem that urgently needs to be solved. Summary of the Invention

[0008] The purpose of this application is to provide an autonomous traffic signal control sub-zone division method, device, medium, and product that can accurately divide traffic signal control sub-zones, thereby improving the accuracy and effectiveness of traffic signal control.

[0009] To achieve the above objectives, this application provides the following solution:

[0010] Firstly, this application provides an autonomous traffic control sub-region partitioning method, which includes:

[0011] Based on deep Q-networks, traffic signal control agent models of different sizes are constructed for regional road networks, and a multi-intersection collaborative model network is also constructed. The traffic signal control agent models of different sizes are used to control different numbers of control intersections. The traffic signal control agent models use the combination of green phases executed by each signal controller as the action space, the maximum queue length corresponding to the green phase in each intersection as the state space, and the average queue length in the regional road network as the environmental reward.

[0012] The multi-intersection cooperative model network is divided into multiple signal control units, and different combinations of signal control units are determined; the signal control unit is the area where the traffic signal control intelligent agent model adjusts the traffic signal.

[0013] With the goal of minimizing the maximum queue length of the road network composed of signal control units (SCUs), and constrained by the time and space complexity of the SCU combination, an optimization model is constructed. The time complexity is the sum of the ratios of the training times of the traffic signal control agent models within each SCU. The space complexity is determined based on the number of parameters of the traffic signal control agent models within each SCU.

[0014] Based on the optimization model, a genetic algorithm is used to determine the optimal combination of signal control units, and traffic control sub-regions are divided according to the optimal combination of signal control units.

[0015] Optionally, the traffic signal control intelligent agent model specifically includes:

[0016] Using formula R t =r t +γR t+1 Determine the environmental reward R at time t t ;

[0017] Using formula Perform Q-function Q(s) t ,a t Iteration of )

[0018] Where, r t Let r be the average queue length in the regional road network at time t. t+1 Let R be the average queue length in the regional road network at time t+1. t+1 Let S be the environmental reward at time t+1, γ∈[0,1] be the discount factor, and S be the environmental reward at time t+1. t Let a be the state space at time t. t Let t be the action space. Let s be the state at time t+1. t+1 Below, all actions a t+1 The largest Q value in the range.

[0019] Optionally, the size of the signal control unit is determined based on the control intersection of the traffic signal control intelligent agent model.

[0020] Optionally, determining the corresponding signal control unit based on traffic signal control agent models of different sizes, and determining different combinations of signal control units, specifically includes:

[0021] Using formula The time complexity of determining the combination of signal control units n is O(n);

[0022] Using formula Determine the space complexity Ω(n) of the signal control unit combination n;

[0023] in, This represents the ratio of training time to training time for the traffic signal control agent model. t i Let N be the training time for the i-th traffic signal control agent model. a T represents the number of traffic signal control agent models in the signal control unit combination n. max This represents the maximum training time for a single traffic signal control agent in the signal control unit combination n. The ratio of parameters, ω i Ω represents the total number of actual model parameters for the i-th traffic signal control agent model. max This represents the maximum number of parameters for a single traffic signal control agent model within the signal control unit combination n.

[0024] Optionally, the optimization model, which aims to minimize the maximum queue length of the road network composed of signal control units (SCUs) and constrains the time and space complexity of SCU combinations, specifically includes:

[0025] Using formula Determine the optimization model;

[0026] Where f(x) is the objective function, max is the maximum value, O(n) is the time complexity of the signal control unit combination n, Ω(n) is the space complexity of the signal control unit combination n, O is the time complexity threshold, Ω is the space complexity threshold, and N a Let n be the number of traffic signal control agent models in the signal control unit combination n.

[0027] Secondly, this application provides an autonomous traffic control sub-area partitioning device, the autonomous traffic control sub-area partitioning device comprising:

[0028] The agent construction module is used to construct traffic signal control agent models of different sizes in the regional road network based on deep Q-networks, and to construct a multi-intersection collaborative model network. The traffic signal control agent models of different sizes are used to control different numbers of control intersections. The traffic signal control agent model uses the combination of green phases executed by each signal controller as the action space, the maximum queue length corresponding to the green phase in each intersection as the state space, and the average queue length in the regional road network as the environmental reward.

[0029] The signal control unit combination determination module is used to divide the multi-intersection cooperative model network into multiple signal control units and determine different combinations of signal control units; the signal control unit is the area where the traffic signal control intelligent agent model adjusts traffic signals;

[0030] The optimization model construction module is used to construct an optimization model with the objective of minimizing the maximum queue length of the road network composed of signal control units, and with constraints on the time and space complexity of the signal control unit combination. The time complexity is the sum of the ratios of the training times of the traffic signal control agent models in each signal control unit of the signal control unit combination; the space complexity is determined based on the number of parameters of the traffic signal control agent models in each signal control unit of the signal control unit combination.

[0031] The traffic control sub-region division module is used to determine the optimal combination of signal control units based on the optimization model and a genetic algorithm, and then divide the traffic control sub-regions according to the optimal combination of signal control units.

[0032] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the autonomous traffic control sub-region partitioning method.

[0033] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the autonomous traffic control sub-region partitioning method described above.

[0034] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the autonomous traffic control sub-region partitioning method.

[0035] According to the specific embodiments provided in this application, this application has the following technical effects:

[0036] This application provides an autonomous traffic signal control sub-zone division method, device, medium, and product. It constructs traffic signal control agent models (agents) of varying sizes within a regional road network based on Deep Q-Network (DQN), and builds a multi-intersection collaborative model network. The multi-intersection collaborative model network is then divided into multiple signal control units, and different combinations of signal control units are determined. The training time and number of parameters of the agent are used as metrics for algorithm complexity. Finally, a genetic algorithm is employed to solve the constrained optimal solution problem. This application can obtain traffic signal control sub-zone division results that ensure better signal control performance while satisfying complexity constraints. It can accurately divide traffic signal control sub-zones, thereby improving the accuracy and effectiveness of traffic signal control. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a schematic flowchart of an autonomous traffic control sub-region partitioning method in one embodiment of this application;

[0039] Figure 2 A schematic diagram of the basic network architecture for intelligent agents;

[0040] Figure 3 A diagram illustrating the training loss values ​​for the agent;

[0041] Figure 4 This is a schematic diagram of the genetic algorithm process;

[0042] Figure 5 This is a schematic diagram illustrating the principle of an autonomous traffic control sub-region partitioning method in one embodiment of this application;

[0043] Figure 6 This is a schematic diagram of the experimental environment;

[0044] Figure 7 This is a schematic diagram of the optimal fitness curve for a population.

[0045] Figure 8 This is a schematic diagram of the partitioning results based on the association degree model and the genetic algorithm. Detailed Implementation

[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] In an exemplary embodiment, Figure 1 As shown, an autonomous traffic control sub-region partitioning method is provided, which includes the following steps S101 to S104. Wherein:

[0049] S101, based on deep Q-networks, constructs traffic signal control agent models of varying sizes for regional road networks, and builds a multi-intersection collaborative model network; models and simulates traffic scenarios as a reinforcement learning environment, allowing agents to interact with the environment and learn control rules; traffic signal control agent models of varying sizes are used to control different numbers of intersections; for example, an agent controlling one intersection is called agent A, an agent controlling two intersections simultaneously is called agent B, and so on.

[0050] The traffic signal control agent model uses the combination of green phases executed by each signal controller as the action space, the maximum queue length corresponding to the green phase at each intersection as the state space, and the average queue length in the regional road network as the environmental reward.

[0051] The action space defines all possible actions an agent can take and is a crucial part of the output of deep reinforcement learning algorithms. In traffic signal control, the action space typically includes traffic light switching. In methods using deep Q-networks, actions are selected to choose the phase, setting up a classic four-phase traffic light system where the green light signals are: east-west straight, east-west left turn, north-south straight, and north-south left turn.

[0052] In this case, the action space is: A = {a1, a2, a1, ..., a n};

[0053] Where n is the number of intersections controlled by the intelligent agent, a n ∈[1,2,3,4] represents the green phase to be executed at the nth intersection. When executing the action, the action sequence number is decoded into the intersection signal phase sequence, and each phase selection is compared. If the current phase is the same as the previous selection, the green light duration of the current phase is extended. If the current phase is different from the previous one, the signal will wait for the preset yellow light time before switching to the next green light phase. For safety and practical considerations, the yellow light buffer time is set to 3 seconds.

[0054] The state space is used to describe the current environmental state of a traffic signal control system. The queue length of the approach lanes at an intersection is a common feature that effectively reflects the degree of congestion. In a cross-shaped, four-way, three-lane intersection, the queue length of the approach lanes in each direction is calculated. Then, at a certain time t, the environmental state of the current traffic scene area can be described as: S t =(q00,q01,q02,q03,...,qn m );S t It consists of the maximum queue length corresponding to the green phase at each intersection, where n represents the number of intersections controlled by the model, and the traffic signal phase is a classic four-phase system, qn mThis represents the maximum queue length of vehicles on the m-th phase approach lane at the nth intersection.

[0055] The reward mechanism is used to evaluate the agent's performance in traffic signal control tasks and guides the agent to optimize its strategy through feedback signals. A reward based on traffic congestion is set, calculating the average queue length in the regional road network, and using its negative value as the reward feedback to the agent; that is, the environmental reward R at time t. t For: R t =r t +γR t+1 ;

[0056] The goal of reinforcement learning is to maximize the rewards an agent can obtain. The agent's objective is to maximize its expected cumulative reward. The Q-function, Q(s), is defined as follows: t ,a t ), indicating that when in state s t The maximum discounted future reward when performing an action is determined by the iterative formula of the Q-function:

[0057] Where, r t Let r be the average queue length in the regional road network at time t. t+1 Let R be the average queue length in the regional road network at time t+1. t+1 Let S be the environmental reward at time t+1, γ∈[0,1] be the discount factor, and S be the environmental reward at time t+1. t Let a be the state space at time t. t Let t be the action space. Let s be the state at time t+1. t+1 Below, all actions a t+1 The largest Q value in the range.

[0058] Q-Learning in reinforcement learning uses Q-value tables to record the effectiveness of state-action pairs, where each state-action pair corresponds to a Q-value. This method of updating the value function using a table is suitable for discrete applications with low dimensionality. However, when the state or action space is large, storing and updating the Q-values ​​of all states and actions consumes enormous storage space. Essentially, a Q-value table is a mapping from a state-action pair to the maximum reward obtainable by performing an action in that state; this relationship can then be abstracted as an input-output mapping. The DeepMind team used deep neural networks to approximate the action decision network, replacing the Q-value table and solving the dimensionality limitation problem. Furthermore, deep neural networks can process state information extracted from the environment, allowing knowledge learned during training to generalize to unseen states.

[0059] When an agent needs to control multiple intersections simultaneously, the agent's input increases exponentially, and the signal controller operates on multiple traffic lights, resulting in an exponential increase in output actions. Therefore, different neural network architectures are designed for agents of varying sizes, primarily differing in the input dimension, the depth of the convolutional network, and the output dimension. The basic network architecture of a multi-intersection cooperative control agent based on DQN is as follows: Figure 2 As shown, from left to right, the network consists of an input layer, a convolutional layer, a pooling layer, a flattening layer, a fully connected layer, and an output layer. The input layer, composed of one-dimensional 4*N elements, receives observed traffic state data, where N represents the number of intersections under collaborative control. The convolutional layers extract features from the data; each agent's convolutional layer has a width of 256, and its depth varies with N, as shown in Table 1. The pooling layers summarize data features, with a depth of 1 and a width of 64. The flattening layer transforms the multi-dimensional input data into one dimension. The fully connected layer performs linear transformations, representing the relationship between data features and the output. The layer closest to the output layer has a width of 128, while the remaining layers have a width of 256, and their depths are shown in Table 1. The final layer is the output layer, essentially a Softmax activation function composed of one-dimensional 4*N elements, each representing the probability of the action combination. Finally, the action combination with the highest probability value is selected as the network's output i.

[0060] Table 1

[0061]

[0062] A typical scenario for a single intersection was built using the micro-traffic simulation software SUMO. Each intersection is equipped with a traffic signal controller, following common rules: green light indicates the corresponding direction is open, red light indicates the corresponding direction is blocked, and traffic flow is interrupted. Road detectors are installed on each approach lane to monitor environmental conditions. This scenario simulates a small at-grade intersection operating under normal conditions, with a total traffic volume set between 600 and 800 vehicles per hour. The proportion of through traffic is between 40% and 60%.

[0063] After multiple rounds of training and testing, a total of 5 agents were selected with a maximum of 5 control intersections. The model network architecture of the multi-intersection collaborative control agents is shown in Table 1. The training loss value for each agent (model A-model E) is as follows: Figure 3 As shown.

[0064] The average queue length and average vehicle speed in the signal control area were collected and calculated to evaluate the performance of different signal control methods, as shown in Table 2.

[0065] Table 2

[0066]

[0067] Compared to traditional fixed timing and adaptive signal control strategies, DQN-based traffic signal control agents excel in reducing queue length and increasing average speed at intersections.

[0068] S102, the multi-intersection cooperative model network is divided into multiple signal control units, and different combinations of signal control units are determined; the signal control unit is the area where the traffic signal control agent model adjusts the traffic signal; the size of the signal control unit is determined according to the control intersection of the traffic signal control agent model.

[0069] The above yielded five traffic signal control agent models. The area where the traffic signal is regulated by the traffic signal control agent model is called a signal control unit. The size of this signal control unit is the same as the number of intersections controlled collaboratively by the agent. A traffic signal area composed of multiple signal control units is called a signal control unit combination. Therefore, the signal control unit combination essentially describes the traffic sub-zone division result from the perspective of algorithm usage.

[0070] Deep learning models typically contain a large number of parameters and complex network structures, and the model itself also requires a certain amount of storage space closely related to the number of parameters. Furthermore, due to the special nature of reinforcement learning, the complexity of DQN-based agents is also reflected in their interaction with the environment.

[0071] Specifically, using formulas The time complexity of determining the combination of signal control units n is O(n);

[0072] Using formula Determine the space complexity Ω(n) of the signal control unit combination n;

[0073] in, This represents the ratio of training time to training time for the traffic signal control agent model. t i Let N be the training time for the i-th traffic signal control agent model. a T represents the number of traffic signal control agent models in the signal control unit combination n. max This represents the maximum training time for a single traffic signal control agent in the signal control unit combination n. The ratio of parameters, ω i Ω represents the total number of actual model parameters for the i-th traffic signal control agent model. max This represents the maximum number of parameters for a single traffic signal control agent model within the signal control unit combination n.

[0074] S103, with the objective of minimizing the maximum queue length of the road network in the signal control unit combination, and with constraints on the time complexity and space complexity of the signal control unit combination, an optimization model is constructed; the time complexity is the sum of the ratios of the training times of the traffic signal control agent models in each signal control unit in the signal control unit combination; the space complexity is determined based on the number of parameters of the traffic signal control agent models in each signal control unit in the signal control unit combination.

[0075] The sub-region partitioning results of an agent using multi-intersection cooperative signal control are described as signal control unit (SCU) combinations, and the complexity of these SCU combinations is defined. The problem of finding a balanced SCU combination with low complexity and good signal control performance among multiple SCU combinations can be described as finding the optimal SCU combination in terms of signal control performance while constraining its complexity within a certain range; that is, it is a constrained objective optimization problem.

[0076]

[0077] Where f(x) is the objective function, max is to maximize it, and the signal control effect of the signal control unit combination is set as the negative of the maximum queue length of the road network. O(n) is the time complexity of the signal control unit combination n, and Ω(n) is the space complexity of the signal control unit combination n, where O(n)∈[0,1], Ω(n)∈[0,1], O is the time complexity threshold, and Ω is the space complexity threshold. We set O = 0.7, Ω = 0.7, and N... a Let n be the number of traffic signal control agent models in the signal control unit combination n.

[0078] S104. Based on the optimization model, the optimal combination of signal control units is determined using a genetic algorithm, and traffic signal control sub-regions are divided based on the optimal combination of signal control units.

[0079] Constrained objective optimization problems can be solved using genetic algorithms. Signal control unit combinations are encoded as chromosomes, with time and space complexity as individual constraints and signal control effectiveness as individual fitness. After a certain number of iterations, the individual with the best fitness is decoded as a signal control unit combination, which represents the balanced optimal sub-region partitioning result. Taking a road network with 9 interconnected intersections as an example, the actual number of signal control unit combinations is finite. Since the agent can control a maximum of 5 intersections, the sample space is actually the set of all combinations with a maximum connected component count of 5, resulting in a solution space size of 798. Therefore, any signal control unit combination in the sample space is represented as a 10-bit binary chromosome, with upper and lower bound constraints on chromosome encoding. A signal control unit encoding and decoding mechanism is designed to ensure that any signal control unit combination corresponds to a unique individual in the population.

[0080] Generally, intersection connectivity primarily considers distance and traffic flow. The improved Winston connectivity model, combining discrete coefficients and queue length, effectively takes into account both distance and traffic flow principles, and can more accurately reflect the degree of need for coordinated intersection control. The improved Whitson connectivity model used is shown in the following formula:

[0081]

[0082] Where I represents the correlation between adjacent intersections, t represents the average travel time on the road segment, and Q represents the distance between the two intersections. max The maximum flow rate in the direction of inflow at the upstream intersection. I represents the average flow rate in the direction of inflow from the upstream intersection. In the experimental road network, the intersection spacing is 500m, the speed limit of the road section is 13km / h, and the flow input is uniform and stable. Based on experimental experience, I is approximately in the range of [0,2].

[0083] Existing signal control sub-region partitioning methods, whether traditional (e.g., static partitioning) or those based on correlation models, all focus on the traffic connectivity between intersections, resulting in overlapping partitioning principles based on static features. This can lead to similar final results across different partitioning methods. In the method proposed in this application, the randomly generated results in the solution space may differ significantly from the final result. Therefore, to avoid excessive deviation between the randomly generated initial population and individuals with optimal fitness, a basic partitioning result is first obtained based on the correlation model and used as the initial population. Based on the problem's scale, the population size is determined to be 50, and the number of iterations is set to 100.

[0084] In genetic algorithms, the design of crossover and mutation probabilities directly impacts the convergence process. The crossover probability determines the rate at which new individuals are generated. A low crossover probability makes the old population less susceptible to disruption, but it slows down the generation of new individuals, potentially leading to premature convergence. Conversely, an excessively high crossover probability can destroy superior individuals, slowing down convergence. Mutation also generates new individuals and is crucial for escaping local optima. A low mutation probability slows down the generation of new individuals, thus slowing the search for the optimal solution. Conversely, an excessively high mutation probability can destroy the optimal solution and degenerate the genetic algorithm into a random search. Therefore, choosing appropriate crossover and mutation probabilities is critical to the performance of genetic algorithms.

[0085] When chromosome fitness is high, it indicates that it is close to the optimal solution. In this case, the mutation probability should be increased and the crossover probability decreased to enhance local search capability without damaging the chromosome. Conversely, when chromosome fitness is low, the crossover probability should be increased and the mutation probability decreased to enhance global search capability and avoid the algorithm getting trapped in local optima. The designed crossover probability P_c and mutation probability P_m formulas are dynamically adjusted according to the number of population iterations and individual fitness, and the specific expressions are as follows.

[0086]

[0087] Where, p cmin p cmax Let f be the minimum and maximum crossover probabilities, f be the individual fitness, and f' be the maximum crossover probability between the two crossover pairs. p represents the average fitness of the population. mmax p mmin f represents the maximum and minimum mutation probabilities. max denoted as the maximum fitness of the population, g is the current iteration number, and G is the total number of iterations.

[0088] The evolutionary characteristics of a population differ at different evolutionary stages. In the early stages of evolution, the population needs to explore different locations in the solution space from its initial position; the main objective in this early stage is to preserve population diversity. In the middle stages of evolution, the superiority or inferiority of individuals emerges within the population, requiring increased attention to superior individuals. In the later stages of evolution, it is necessary to ensure that superior individuals are not eliminated and to improve the convergence speed of the algorithm.

[0089] Therefore, different genetic selection operations are employed depending on the evolutionary stage. In the early stage: a roulette wheel selection method is used, where the probability of an individual being selected is proportional to its fitness, creating a brutal survival environment of the fittest. In the middle stage: a combination of elite retention and random elimination is used. Individuals are ranked from highest to lowest fitness, and the population is divided into high, medium, and low fitness levels. High-fit individuals are duplicated, and half of the medium-fit individuals and half of the low-fit individuals are randomly eliminated. In the late stage: an elite retention system is used, with individuals ranked from highest to lowest fitness, and the population divided into high and low fitness levels. High-fit individuals are duplicated.

[0090] The constraints in this application arise in two places: firstly, the feasibility of the signal control unit combination analyzed in the previous chapter; and secondly, the constraints on individual complexity. When generating new individuals, they must be within the feasible solution space. During population updates, individuals with higher complexity are eliminated. The time complexity threshold and space complexity threshold are both set to 0.7 to ensure that the population can find individuals with optimal complexity and fitness that satisfy the constraints through evolution. The specific genetic algorithm process is as follows. Figure 4As shown.

[0091] like Figure 5 As shown, firstly, multiple multi-intersection cooperative control model networks are constructed. The complexity and signal control performance in the road network partition solution space are analyzed. The signal control unit partitioning result oriented towards the DQN algorithm's equilibrium is described as the optimal signal control performance solution with complexity constraints. A genetic algorithm is used to solve the problem of finding the constrained optimal solution. The genetic algorithm is designed, including chromosome encoding, mutation and crossover probabilities, selection operation strategies, etc. Experiments demonstrate that the above partitioning scheme steps can find the partitioning result with the optimal signal control performance that satisfies the constraints.

[0092] Signal control models based on reinforcement learning agents do not rely on precise mathematical modeling, resulting in smaller propagation errors and stronger generalization capabilities, which is beneficial for regional signal control. However, as the number of cooperative intersections increases, complexity and dimensionality explosion problems arise. Traffic sub-zone division is a prerequisite for regional signal control implementation. In traditional division methods, the evaluation of the division results mainly considers the degree of correlation. This application considers both division and signal control simultaneously, obtaining the optimal division result for regional signal control while satisfying the complexity constraints of the signal control algorithm.

[0093] To address the issue of large errors in regional signal control using traditional mathematical models, this application constructs a multi-intersection collaborative intelligent agent for signal control. To ensure the practical usability of the signal control algorithm, this application employs heuristic search, considering both algorithm complexity and signal control effectiveness, to solve the problem of finding the optimal signal control partitioning result under complexity constraints.

[0094] For a road network with a fixed structure, the number of signal control unit combinations is finite. Finding a set of signal control units with good performance but low complexity within a finite solution space can be viewed as a constrained optimization problem.

[0095] Genetic algorithms can effectively handle constrained optimization problems. In the intelligent search process, an elimination mechanism and individual validity assessment are introduced to address complexity constraints and feasible solutions. Signal control unit combinations are encoded as fixed-length chromosomes, with their fitness function value set as the signal control performance index. The validity and complexity indices of the signal control unit combinations are used as constraints, with time complexity not exceeding 0.7 and space complexity not exceeding 0.7, and are incorporated into the population selection operation.

[0096] The number of signal control unit combinations in a road network with 9 interconnected intersections is actually finite. Since the agent can control a maximum of 5 intersections, the sample space is actually the set of all combinations with a maximum connected number of 5, and the solution space size is 798. Therefore, any combination of signal control units in the sample space can be represented as a 10-bit binary chromosome.

[0097] To avoid excessive deviation between the randomly selected initial population and the individuals with optimal fitness, a basic partitioning result was first obtained based on the association model, and this result was used as the individuals in the initial population. Based on the scale of this problem, the population size was determined to be 50, and the number of population iterations was set to 100.

[0098] To more efficiently and quickly converge the optimal fitness value of the population to a stable value, adaptive crossover and mutation probabilities are used. The crossover and mutation probabilities are dynamically adjusted with the number of population iterations.

[0099] The flowchart of the genetic algorithm is shown below:

[0100] 1. Initialize parameters

[0101] Initialize the road network environment, set the number of iterations, and the population size;

[0102] 2. Initialize the population

[0103] The initial population is set as the partition result based on the association degree model;

[0104] 3. Start iteration

[0105] 4. Assess fitness

[0106] The negative value of the signal control effect of the corresponding combination for an individual is obtained as the fitness value;

[0107] 5. Select

[0108] Eliminate individuals with higher complexity ranking in the population;

[0109] Retain individuals with high fitness in the population;

[0110] 6. Cross

[0111] After calculating the crossover probability, random segment crossover is performed, and the effectiveness of the new individuals is determined.

[0112] 7. Mutation

[0113] After calculating the mutation probability, random point mutations are performed, and the validity of the new individuals is determined.

[0114] 8. Update the population

[0115] The new population replaces the old population, and the iteration process is repeated.

[0116] 9. Output the best individual and its fitness.

[0117] A simulation experiment was conducted using SUMO to build a road network. The correlation between intersections was then calculated based on a correlation model. The road network consists of nine three-lane crossroads with signal controllers. The structure of the road network is as follows: Figure 6As shown, the intersections have been numbered to facilitate the subsequent interpretation of the division results.

[0118] An adaptive genetic algorithm is used to search for the optimal partitioning result, obtaining the solution with the best fitness in the final population that satisfies both complexity and feasible solution constraints. The final partitioning result is obtained after chromosome decoding. After 100 iterations, the optimal fitness curve of the population is shown below. Figure 7 As shown.

[0119] like Figure 8 As shown in part (a), the road network is divided into 4 units, where unit 1 includes intersections 1, 2, and 3; unit 2 includes intersection 4; unit 3 includes intersections 5, 7, and 8; and unit 4 includes intersections 6 and 9. Figure 8 As shown in part (b), the road network is divided into four units: Unit 1 includes intersections 1, 2, and 3; Unit 2 includes intersections 4, 5, and 6; Unit 3 includes intersection 7; and Unit 4 includes intersections 6 and 9. The same DQN-based agent-based signal control scheme was applied to both division results. After a signal control simulation time of 3600 steps, the regional traffic state indicators and algorithm complexity are shown in Table 3.

[0120] Table 3

[0121]

[0122] The evaluation metrics show that the partitioning results of the equilibrium optimization method are basically the same in terms of the number of sub-regions, and the number of intersections in each unit of the optimized method is not much different from the partitioning results based on the correlation model. However, when using the same signal control agent and meeting the constraints of time complexity and space complexity, the optimized method achieves better signal control performance.

[0123] Firstly, relying on the open platform SUMO, a real-time control integrated simulation platform was built using Python, with Traci as the core for observation and feedback information transmission. On this platform, multiple multi-intersection cooperative control agents were built and trained, and various signal control schemes were tested and compared, providing underlying technical support for subsequent road network segmentation. For regional signal control problems, a multi-intersection cooperative control agent was built, and through multiple training and testing sessions, the network architecture that yielded the best signal control performance was found.

[0124] After training multiple multi-intersection cooperative control agents, to effectively represent the specific situation of sub-region division and the usage of DQN-based signal control agents, and to facilitate subsequent automatic optimization, the concept of signal control unit combination was first proposed. Furthermore, the shape of the signal control units and the complexity of signal control unit combination were investigated. Experiments showed that the shape of the signal control units has no impact on their control performance, stemming from the generalization performance of deep forced learning itself. Through multiple rounds of experimental testing, in the signal control unit combination sample dataset, space complexity increases with decreasing average number of signal control units, while time complexity decreases with decreasing average number of signal control units. Signal control performance deteriorates when the average number of signal control units is large, indicating the possibility of directly seeking a balance between algorithm complexity and control performance.

[0125] The subregion partitioning is transformed into a constrained objective optimization problem, which is solved using a genetic algorithm. First, based on the combination of signal control units (SCUs), encoding rules for chromosome individuals are designed, and the effective range of chromosome encoding is constrained. Simultaneously, to accelerate the search speed, the partitioning results based on association degree are directly used as the initial population. Then, the genetic operations are adaptively improved by adding constraints on the effectiveness of individuals during offspring generation and adding the elimination of inferior offspring during population updates. Experiments show that, after a finite number of iterations, this method can find a combination of SCUs that satisfies the complexity constraints and has optimal signal control performance.

[0126] To verify the effectiveness of the proposed method, it was compared with traditional partitioning methods based on the improved Winston model and dynamic partitioning methods based on clustering. Experimental results show that the proposed optimized partitioning method achieves a good balance between algorithm complexity and performance. In real-world traffic scenarios, due to limited hardware conditions of edge devices, training highly complex models and deploying highly complex signal control algorithms is often very difficult. The sub-region partitioning optimization method obtained by this study results in a lower complexity for using signal control algorithms, achieving optimal control performance while reducing resource usage, thus facilitating the practical deployment of intelligent transportation area partitioning and signal control schemes.

[0127] Based on the same inventive concept, this application also provides an autonomous traffic control sub-region partitioning device for implementing the autonomous traffic control sub-region partitioning method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more autonomous traffic control sub-region partitioning device embodiments provided below can be found in the limitations of the autonomous traffic control sub-region partitioning method described above, and will not be repeated here.

[0128] In one exemplary embodiment, an autonomous traffic control sub-area partitioning device is provided, comprising:

[0129] The agent construction module is used to construct traffic signal control agent models of different sizes in the regional road network based on deep Q-networks, and to construct a multi-intersection collaborative model network. The traffic signal control agent models of different sizes are used to control different numbers of control intersections. The traffic signal control agent model uses the combination of green phases executed by each signal controller as the action space, the maximum queue length corresponding to the green phase in each intersection as the state space, and the average queue length in the regional road network as the environmental reward.

[0130] The signal control unit combination determination module is used to divide the multi-intersection cooperative model network into multiple signal control units and determine different combinations of signal control units; the signal control unit is the area where the traffic signal control intelligent agent model adjusts traffic signals;

[0131] The optimization model construction module is used to construct an optimization model with the objective of minimizing the maximum queue length of the road network composed of signal control units, and with constraints on the time and space complexity of the signal control unit combination. The time complexity is the sum of the ratios of the training times of the traffic signal control agent models in each signal control unit of the signal control unit combination; the space complexity is determined based on the number of parameters of the traffic signal control agent models in each signal control unit of the signal control unit combination.

[0132] The traffic control sub-region division module is used to determine the optimal combination of signal control units based on the optimization model and a genetic algorithm, and then divide the traffic control sub-regions according to the optimal combination of signal control units.

[0133] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal. The computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is connected to the system bus via the I / O interfaces. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The I / O interfaces of the computer device are used for exchanging information between the processor and external devices. The communication interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an autonomous communication control sub-region partitioning method.

[0134] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0135] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0136] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0138] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0139] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0140] In this application, all actions to acquire signals, information, or data are carried out in compliance with the relevant data protection laws and policies of the country where the location is situated, and with the authorization granted by the owner of the relevant device.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. In summary, the content of this specification should not be construed as a limitation of this application.

Claims

1. An autonomous traffic control sub-region partitioning method, characterized in that, The autonomous traffic control sub-region division method includes: Based on deep Q-networks, traffic signal control agent models of different sizes are constructed for regional road networks, and a multi-intersection collaborative model network is also constructed. The traffic signal control agent models of different sizes are used to control different numbers of control intersections. The traffic signal control agent models use the combination of green phases executed by each signal controller as the action space, the maximum queue length corresponding to the green phase in each intersection as the state space, and the average queue length in the regional road network as the environmental reward. The multi-intersection cooperative model network is divided into multiple signal control units, and different combinations of signal control units are determined; the signal control unit is the area where the traffic signal control intelligent agent model adjusts the traffic signal. With the goal of minimizing the maximum queue length of the road network composed of signal control units (SCUs), and constrained by the time and space complexity of the SCU combination, an optimization model is constructed. The time complexity is the sum of the ratios of the training times of the traffic signal control agent models within each SCU. The space complexity is determined based on the number of parameters of the traffic signal control agent models within each SCU. Based on the optimization model, a genetic algorithm is used to determine the optimal combination of signal control units, and traffic control sub-regions are divided according to the optimal combination of signal control units.

2. The autonomous traffic control sub-region division method according to claim 1, characterized in that, The traffic signal control intelligent agent model specifically includes: Using formula R t =r t +γR t+1 Determine the environmental reward R at time t t ; Using formula Perform Q-function Q(s) t ,a t Iteration of ) Where, r t Let r be the average queue length in the regional road network at time t. t+1 Let R be the average queue length in the regional road network at time t+1. t+1 Let S be the environmental reward at time t+1, γ∈[0,1] be the discount factor, and S be the environmental reward at time t+1. t Let a be the state space at time t. t Let t be the action space. Let s be the state at time t+1. t+1 Below, all actions a t+1 The largest Q value in the range.

3. The autonomous traffic control sub-region division method according to claim 1, characterized in that, The size of the signal control unit is determined based on the control intersection of the traffic signal control intelligent agent model.

4. The autonomous traffic control sub-region division method according to claim 1, characterized in that, The process of determining corresponding signal control units based on traffic signal control agent models of different sizes, and determining different combinations of signal control units, specifically includes: Using formula The time complexity of determining the combination of signal control units n is O(n); Using formula Determine the space complexity Ω(n) of the signal control unit combination n; in, This represents the ratio of training time to training time for the traffic signal control agent model. t i Let N be the training time for the i-th traffic signal control agent model. a T represents the number of traffic signal control agent models in the signal control unit combination n. max This represents the maximum training time for a single traffic signal control agent in the signal control unit combination n. The ratio of parameters, ω i Ω represents the total number of actual model parameters for the i-th traffic signal control agent model. max This represents the maximum number of parameters for a single traffic signal control agent model within the signal control unit combination n.

5. The autonomous traffic control sub-region division method according to claim 1, characterized in that, The optimization model is constructed with the objective of minimizing the maximum queue length of the road network composed of signal control units (SCUs), and with constraints on the time and space complexity of SCU combinations. Specifically, it includes: Using formula Determine the optimization model; Where f(x) is the objective function, max is the maximum value, O(n) is the time complexity of the signal control unit combination n, Ω(n) is the space complexity of the signal control unit combination n, O is the time complexity threshold, Ω is the space complexity threshold, and N a Let n be the number of traffic signal control agent models in the signal control unit combination n.

6. An autonomous traffic control sub-area partitioning device, characterized in that, The autonomous traffic control sub-area partitioning device includes: The agent construction module is used to construct traffic signal control agent models of different sizes in the regional road network based on deep Q-networks, and to construct a multi-intersection collaborative model network. The traffic signal control agent models of different sizes are used to control different numbers of control intersections. The traffic signal control agent model uses the combination of green phases executed by each signal controller as the action space, the maximum queue length corresponding to the green phase in each intersection as the state space, and the average queue length in the regional road network as the environmental reward. The signal control unit combination determination module is used to divide the multi-intersection cooperative model network into multiple signal control units and determine different combinations of signal control units; the signal control unit is the area where the traffic signal control intelligent agent model adjusts traffic signals; The optimization model construction module is used to construct an optimization model with the objective of minimizing the maximum queue length of the road network composed of signal control units, and with constraints on the time and space complexity of the signal control unit combination. The time complexity is the sum of the ratios of the training times of the traffic signal control agent models in each signal control unit of the signal control unit combination; the space complexity is determined based on the number of parameters of the traffic signal control agent models in each signal control unit of the signal control unit combination. The traffic control sub-region division module is used to determine the optimal combination of signal control units based on the optimization model and a genetic algorithm, and then divide the traffic control sub-regions according to the optimal combination of signal control units.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the autonomous traffic control sub-region partitioning method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the autonomous traffic control sub-region partitioning method as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the autonomous traffic control sub-region partitioning method as described in any one of claims 1-5.

Images