A traffic signal lamp control method for multi-agent heterogeneous right-of-way allocation

Abstract

The application discloses a traffic signal lamp control method for multi-agent heterogeneous road right distribution, first extracts and encodes traffic agent features based on a graph neural network, then realizes road right distribution based on an Advantage Actor-Critic (A2C) model, and finally realizes signal lamp control based on multi-traffic-agent road right distribution. The application can process different types of agents, dynamically control signal lamps, reduce road congestion, and improve road network operation efficiency.

Description

Technical Field

[0001] This invention relates to the field of traffic control methods, specifically a traffic signal control method for multi-subject heterogeneous right-of-way allocation. Background Technology

[0002] With the development of autonomous transportation systems, the diversity and heterogeneity of traffic subjects are increasing, and various traffic subjects interfere with each other, seriously affecting the operational efficiency of the transportation system. In order to improve traffic operation efficiency, traffic light control oriented towards multi-subject right-of-way allocation has become a current research hotspot.

[0003] Traditional traffic light control methods based on right-of-way allocation often employ a single strategy for resource allocation, relying on fixed rules or preset strategies. However, the actual traffic environment is dynamic and easily affected by factors such as time of day, weather, and traffic conditions. For example, traffic flow and speed vary between rush hours and regular hours, holidays and weekdays, and rainy / snowy / sunny weather, making it difficult to flexibly respond to real-world traffic situations. Furthermore, existing methods often fail to adequately consider the heterogeneous characteristics of different traffic subjects. For instance, buses, cars, trucks, bicycles, and electric vehicles have different speeds and behavior patterns, resulting in traditional right-of-way allocation optimization results that do not accurately reflect actual needs. Therefore, it is necessary to define a multi-traffic subject characteristic model and construct a dynamic right-of-way allocation network for traffic light control to achieve dynamic right-of-way assignment for multiple traffic subjects and improve road network operational efficiency. Summary of the Invention

[0004] This invention provides a traffic signal control method for heterogeneous right-of-way allocation for multiple entities, in order to solve the problem that existing traffic signal control methods based on right-of-way allocation are not applicable to multiple types of traffic entities.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A traffic signal control method for heterogeneous right-of-way allocation among multiple stakeholders includes the following steps:

[0007] Step 1: Obtain the feature information of each traffic entity, and establish the feature vector of each traffic entity based on the feature information;

[0008] Each traffic entity is treated as a node, and the adjacency matrix of any pair of traffic entities is used as an edge. A weight matrix is ​​obtained by assigning weights to each edge, thereby constructing a traffic entity graph with a weight matrix.

[0009] The feature vector of each node is processed based on the weight matrix in the main traffic map to obtain the initial feature representation of each node;

[0010] Then, the feature representations of each node are input into the graph neural network for processing to obtain a low-dimensional feature representation of each node, i.e., the traffic subject.

[0011] Step 2: Obtain the right-of-way allocation strategy based on the A2C algorithm model. The process is as follows:

[0012] A state matrix is ​​constructed using the low-dimensional feature representations of each traffic subject obtained in step 1, and the choice of each traffic subject to the right-of-way allocation scheme is defined as a selection action.

[0013] The A2C algorithm model uses a policy network to estimate the state matrix, obtaining the probability distribution of various right-of-way allocation actions for each traffic subject. Each right-of-way allocation action represents a possible traffic behavior of the traffic subject, such as going straight, turning left, turning right, or stopping. Based on the probability distribution matrix, the right-of-way allocation action with the highest probability for each traffic subject can be obtained and selected to arrive at a right-of-way allocation strategy. The A2C algorithm model also uses a value network to estimate the cumulative reward value under the state matrix to evaluate the merits of the right-of-way allocation strategy.

[0014] The A2C algorithm model also constructs a reward function to represent the immediate reward obtained by the traffic subject after collecting and selecting actions under the state matrix, and constructs an advantage function based on the difference between the output of the policy network and the output of the value network.

[0015] The A2C algorithm model is trained and updated. During training and updates, the policy gradient is calculated based on the advantage function, and the gradient ascent method is used to process the policy gradient to update the parameters of the policy network. At the same time, the mean squared error of the value network is calculated using the cumulative reward and the output of the value network, and the mean squared error of the value network is processed using the gradient descent method to update the parameters of the value network. Finally, an overall loss function is established based on the gradient ascent result of the policy gradient and the gradient descent result of the mean squared error of the value network. Training ends when the overall loss function reaches the convergence condition. Thus, the right-of-way allocation strategy for each traffic subject is output through the trained A2C algorithm model.

[0016] Step 3: Based on the right-of-way allocation strategy for each traffic entity obtained in Step 2, obtain the right-of-way allocation weight for each traffic entity, and calculate the green light duration for each direction of the traffic signal based on the right-of-way allocation weight for each traffic entity.

[0017] In the further step 1, the weight of each edge is calculated based on the distance and interaction frequency between the traffic entities corresponding to that edge.

[0018] In the further step 1, after performing linear embedding and nonlinear mapping on the feature vector of each node based on the weight matrix, the initial feature representation of each node is obtained.

[0019] In the further step 1, the feature vector of each node is first standardized, and then linear embedding and nonlinear mapping are performed on the standardized feature vector of each node.

[0020] In the further step 1, the graph neural network uses multiple graph convolutional layers to process the feature representation of each node. Each graph convolutional layer combines the feature representation of each node with the feature representation of neighboring nodes to obtain a new feature representation. The output of each layer in the multiple graph convolutional layers is used as the input of the next layer, thereby obtaining the low-dimensional feature representation of each node.

[0021] Furthermore, in step 2, the immediate return is calculated using a return function, and then the immediate returns are summed to obtain the cumulative return.

[0022] Compared with the prior art, the advantages of the present invention are:

[0023] This invention targets diverse traffic entities, extracts and encodes traffic entity features based on graph neural networks, allocates right-of-way based on the Advantage Actor-Critic (A2C) model, and finally implements traffic light control based on right-of-way allocation for multiple traffic entities.

[0024] This invention can handle different types of entities, such as pedestrians, bicycles, cars, and autonomous vehicles, and control traffic lights at intersections based on right-of-way allocation strategies. It can dynamically control traffic lights under dynamic traffic environment conditions, reduce road congestion, and improve road network operation efficiency. Attached Figure Description

[0025] Figure 1 This is a flowchart of the method according to an embodiment of the present invention.

[0026] Figure 2 This is a flowchart illustrating the process of constructing a traffic entity map and obtaining feature representations in an embodiment of the present invention.

[0027] Figure 3 This is a flowchart of the neural network operation in an embodiment of the present invention.

[0028] Figure 4 This is a flowchart of the right-of-way allocation process based on the A2C algorithm model of this invention.

[0029] Figure 5 This is the intersection modeling diagram of the present invention. Detailed Implementation

[0030] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0031] This embodiment discloses a traffic signal control method for heterogeneous right-of-way allocation among multiple stakeholders, such as... Figure 1 The above includes the following steps:

[0032] Step 1: Subject feature extraction and encoding based on graph neural networks.

[0033] Traditional traffic light control methods based on right-of-way allocation often use uniform feature representations to handle different types of subjects. Graph neural networks, however, can learn different representations for each subject type, thus better capturing heterogeneous features. This helps to more accurately reflect the characteristics and behaviors of different subjects. Furthermore, the relationships between traffic subjects are often very complex, such as intersection conflicts and cooperative behaviors. Graph structures can more naturally characterize the relationships between subjects, enabling graph neural networks to handle these complex interactions more effectively. Moreover, traffic systems are often dynamic, and the relationships between subjects are constantly changing. Graph neural networks are adaptive, able to dynamically adjust feature representations according to different situations, better adapting to actual changes. Therefore, this embodiment utilizes graph neural network subject feature extraction and encoding to solve the problems of traditional methods having single-feature descriptions of traffic subjects, being unable to handle complex scenarios, and being unable to handle dynamic changes in relationships between subjects. The process is as follows:

[0034] (1.1) Obtain the feature information of each traffic entity, and establish the feature vector of each traffic entity based on the feature information. The feature vector of each traffic entity is xi = [x i1 ,x i2 ,…,x iD ], where x iD D represents the characteristic information of the traffic subject, which is the dimension of the traffic subject's characteristics. The dimension includes information such as the traffic subject's location, speed, and behavioral characteristics.

[0035] (1.2) such as Figure 2 As shown, each traffic entity is used as a node, the adjacency matrix of any pair of traffic entities is used as an edge, and a weight matrix is ​​obtained by assigning weights to each edge. Thus, a traffic entity graph with a weight matrix is ​​constructed.

[0036] The process of representing traffic entities and their relationships as a graph structure begins with node representation, using each traffic entity as a node, and then constructing a traffic entity graph based on the actual number of traffic entities in the traffic environment. Let there be N traffic entities, and define v... i Let represent the i-th traffic entity, where 1 ≤ i ≤ N. These entities constitute the node set V = {v1, v2, ..., v...} N}

[0037] Next, the edges of the traffic body graph are constructed. For each pair of traffic bodies v i and v j Based on the relationships and interaction levels between them, edges are established where related elements interact. Define an adjacency matrix A, where A... ij=1 indicates node v i and v j There exists an edge, A ij =0 indicates that there are no edges, and thus the edges of the main traffic graph are represented by an adjacency matrix.

[0038] Next, we need to establish the weights of the edges. For an existing edge A... ij Assign it a weight W ij , representing the strength of the relationship between the traffic entities. Weight W ij It is calculated based on the distance and interaction frequency information between the traffic entities, i.e., the nodes, in each edge.

[0039] First, define a weight function f(d) ij ,freq ij It takes the traffic entities in each edge, i.e., node v, as its core. i and v j The distance d between ij and interaction frequency freq ij As input, a weight value is output. The weight function can be designed according to the specific situation of the problem. A combination function is used to take into account both distance and interaction frequency, as shown below:

[0040] f(d ij ,freq ij )=α·g(d ij )+β·h(freq ij )

[0041] Among them, g(d ij h(freq) is a function of the distances between nodes in each edge. ij ) is a function of the frequency of interaction between nodes in each edge, and α and β are trade-off factors used to control the degree of influence of both.

[0042] The distance is represented using a reciprocal function, with closer distances having higher weights, as shown below:

[0043]

[0044] Wherein, ∈[0.0001,0.001] is used to avoid division by zero.

[0045] For the interaction frequency, it is normalized to a certain range and then represented by a linear function. The higher the interaction frequency, the greater the weight, as shown below:

[0046]

[0047] Here, minFreq and maxFreq are the minimum and maximum interaction frequencies, respectively.

[0048] Finally, the defined weight function is used to calculate the weight W of the edge. ij :

[0049] W ij =f(d ij ,freq ij )

[0050] Based on the weight W of each edge ij Obtain the weight matrix W representing the degree of connection between all nodes.

[0051] Therefore, by combining the node set V and the adjacency matrix A, we can construct the traffic entity graph G = (V, A). This graph represents the graph structure of traffic entities and their relationships.

[0052] (1.3) such as Figure 2 As shown, the feature vector of each node is processed based on the weight matrix in the traffic map to obtain the initial feature representation of each node.

[0053] To ensure consistent scaling across different features, the feature vector of each node is first standardized. Standardization can be achieved by subtracting the mean and dividing by the standard deviation, resulting in a feature distribution with zero mean and unit variance. The standardized feature vector of each node is represented as follows: medium μ D and σ D These are the mean and standard deviation of the Dth feature, respectively.

[0054] The standardized feature vector of each node is subjected to linear embedding and non-linear mapping to extract more meaningful node representations. In this embodiment, the ReLU function is used for linear embedding and non-linear mapping, as shown below:

[0055]

[0056] Where W is the weight matrix for the density between nodes, and b is the bias vector.

[0057] Finally, we obtain each node v i The initialization feature representation h i , which serves as the input to the graph neural network.

[0058] (1.4) such as Figure 3 As shown, each node v i The initialization feature representation h i The input is fed into a graph neural network for processing to obtain a low-dimensional feature representation of each node, i.e., the main traffic entity.

[0059] Graph neural networks consist of multiple graph convolutional layers, each of which stores nodes v i Feature representation hi By combining the feature representations of its neighboring nodes, a new node representation is computed.

[0060] Taking the first graph convolutional layer as an example, the calculation process is as follows:

[0061]

[0062] Where Ne(i) represents node v i The neighbor set of node v, where deg(i) represents node v i The degree, W (1) It is the parameter weight matrix of the first graph convolutional layer, b (l) σ is the bias vector of the first graph convolutional layer, and σ is the activation function. This represents the low-dimensional feature representation output by the first graph convolutional layer.

[0063] By stacking multiple graph convolutional layers, with the output of each layer serving as the input to the next, higher-level node representations are extracted progressively. Assuming the graph neural network has L graph convolutional layers, the computation process at the l-th layer is as follows:

[0064]

[0065] Among them, W (l) It is the weight matrix of the parameters of the l-th layer, b (l) It is the bias vector of the l-th layer. It is a low-dimensional feature representation of the output of the l-th layer.

[0066] After processing by a graph neural network with multiple graph convolutional layers (GNN), the feature representation h of each node is... i The feature will be mapped to a lower dimension to obtain a low-dimensional feature representation. Where l represents the final number of GNN layers.

[0067] Step 2: Obtain the right-of-way allocation strategy based on the A2C algorithm model.

[0068] Low-dimensional feature representations of multiple traffic entities were obtained through step 1. In addition to the dynamically changing relationships between traffic entities, step 2 uses these characteristic relationships as inputs and employs the Advantage Actor-Critic (A2C) algorithm model to optimize right-of-way allocation for multiple traffic entities. Compared to traditional methods that use fixed right-of-way allocation strategies in the face of dynamically changing traffic environments, the A2C algorithm model solves the problem that traditional methods cannot perform dynamic right-of-way allocation. It can better adapt to changes in the traffic environment and improve the operational efficiency of the road network. Figure 4 As shown, the process is as follows:

[0069] (2.1) Construct a state matrix using the low-dimensional feature representations of each traffic entity obtained in step 1.

[0070] In step 1, a traffic entity graph G = (V, A) has been constructed, where V is the set of nodes and A is the adjacency matrix representing the relationships between traffic entities. The aforementioned state representation involves representing the low-dimensional features of the graph's nodes. As part of the state. Assume there are N traffic entities (nodes) in the graph, and each node has a feature dimension of D. ′ Define state s t Representing the low-dimensional features of the graph nodes As part of the state.

[0071] After processing through multiple layers of graph neural networks, the low-dimensional features of each node are represented as follows: Where i represents the index of the node.

[0072] The feature representations of each node are stacked together to form a state matrix s. t as follows:

[0073]

[0074] In this state matrix, each row represents the low-dimensional feature vector of a node, and each column represents the dimension of the feature. This state matrix s t It contains low-dimensional feature representations of all traffic entities obtained after processing by a graph neural network.

[0075] Next, we define the action space, which has N traffic entities, each with M possible right-of-way allocation actions. The action space is represented as a discrete set of actions A, where each element a... i This represents the right-of-way allocation action chosen by the i-th traffic entity.

[0076] The action space is defined as follows:

[0077] A = {a1, a2, ..., a} N},a i ∈{1,2,…,M}

[0078] Among them, a i This represents the right-of-way allocation action chosen by the i-th subject. The right-of-way allocation action refers to the possible traffic behavior of that subject, such as going straight, turning left, turning right, or stopping. i The range is from 1 to M.

[0079] (2.2) In the A2C algorithm model, the reward function r is defined. i This represents the traffic entity in the state matrix s t Take action a i The immediate rewards obtained later can be used to calculate the cumulative reward. Consider the traffic vehicle in the state matrix s.t Choose action a i The subsequent traffic flow speed v i And the degree of congestion. The reward function is designed as follows:

[0080] r i =R(s) t ,a i )

[0081]

[0082] Among them, w1 and w2 are weighting factors used to balance the impact of speed and congestion. Used to reward high-speed traffic flow, with higher speeds resulting in higher rewards, and includes:

[0083] Congestion level = Standard deviation (v1, v2, ..., v) N )

[0084] (2.3) In the A2C algorithm model, a policy network and a value network are defined to optimize the right-of-way allocation strategy.

[0085] Policy networks are used to estimate the probability distribution of each traffic agent choosing multiple right-of-way allocation actions under a given state. Assuming the policy network parameter is θ, the policy network is represented as P(a i |s t ;θ), where a i This refers to the right-of-way allocation action of subject i. The softmax function is used to transform the output of the policy network into a probability distribution:

[0086] P(a i |s t ;θ)=softmax(W p ·h i +b p )

[0087] Among them, W p and b p These are the weights and biases of the policy network. Traffic subject i is in state s t The low-dimensional feature representation.

[0088] Value networks are used to estimate the cumulative reward value in a given state to evaluate the merits of right-of-way allocation strategies. Assuming the value network parameter is φ, the value network is represented as V(s) t ;φ), where V(s) t ;φ) is the state s t The value estimate below.

[0089] Value networks use neural networks to model and output a real number representing the estimated cumulative return.

[0090] V(s t ;φ)=W v ·h pool +b v

[0091] Among them, W v and b v It refers to the weights and biases of the value network, h pool It is the feature representation of the entire graph after aggregation by the aggregation layer, and it has:

[0092]

[0093] (2.4) In the A2C algorithm model, the advantage function A is defined. i (s t ,a i ) indicates that in state s t Choose action a i Compared to the advantage of average right-of-way allocation, it is used to adjust the direction of policy updates; the advantage function A i (s t ,a i As shown below:

[0094] A i (s t ,a i )=Q i (s t ,a i )-V i (s t ;φ)

[0095] Among them, Q i (s t ,a i ) represents state s t Take action a i The cumulative return afterward.

[0096] (2.5) Train and update the A2C algorithm model:

[0097] (2.5.1) State representation and initialization:

[0098] As can be seen from the above, the state matrix of the traffic entity is represented as s t The parameters for initializing the policy network are θ, and the parameters for initializing the value network are φ.

[0099] (2.5.2) Select Action:

[0100] At each time step t, a right-of-way allocation action a is selected for each traffic subject according to the policy network. i, where i represents the number of the traffic entity. Each traffic entity is determined according to the current state matrix s. t Given the network parameters θ, select the right-of-way allocation action a. i The probability is P(a) i |s t ;θ).

[0101] (2.5.3) Interacting with the environment.

[0102] Based on the selected right-of-way allocation action, interact with the environment and receive an immediate reward. t (i) and the next state s t+1 .

[0103] (2.5.4) Calculation of the dominance function.

[0104] Calculate the advantage function A i (s t ,a i ):

[0105] A i (s t ,a i )=Q i (s t ,a i )-V i (s t ;φ)

[0106] (2.5.5) Calculate the policy gradient based on the advantage function Update the policy network parameter θ as follows:

[0107]

[0108] in, Represents the gradient of the parameters. This indicates that the probability distribution "π" represents the probability distribution. θ "random variables obtained by sampling" t "Calculate the expected value of the function."

[0109] The gradient ascent method is used to update the policy network parameters θ based on the policy gradient, thereby increasing the probability of actions that are beneficial to cumulative rewards, as shown below:

[0110]

[0111] Where α is the learning rate.

[0112] (2.5.6) Calculate the mean square error of the value network and update the value network parameters φ using an optimization algorithm.

[0113] First, calculate the cumulative return. Based on the interaction process, calculate the future cumulative return starting from time step t. Specifically, the calculation uses the method of accumulating discount rewards, that is... Where γ is the discount factor.

[0114] Then, the mean squared error of the value network is calculated using cumulative returns. and the output V(s) of the value network t The mean square error of the value network is calculated as follows: φ).

[0115]

[0116] Update the value network parameters: Use gradient descent to update the value network parameters φ based on the mean squared error, in order to minimize the mean squared error.

[0117]

[0118] Where β is the learning rate. The gradient of the loss function

[0119] (2.5.7) Calculate the overall loss function.

[0120] Taking into account both policy gradient and value network update, the overall optimization is performed, and the overall loss function is as follows:

[0121]

[0122] Here, 'b' is a weighting factor used to balance the impact of policy gradients and value network updates. The overall loss is calculated based on the overall loss function.

[0123] (2.5.8) Iterative Loop:

[0124] Repeat steps (2.5.2) to (2.5.7) until the predetermined number of training rounds or the overall loss function converges, finally obtaining the optimized probabilistic model P(a) for the right-of-way allocation strategy. i |s t ;θ), which is the right-of-way allocation strategy.

[0125] Step 3: Based on the right-of-way allocation strategy for each traffic entity obtained in Step 2, obtain the right-of-way allocation weight for each traffic entity, and calculate the green light duration for each direction of the traffic signal based on the right-of-way allocation weight for each traffic entity.

[0126] This embodiment first models the intersection. For example... Figure 5 As shown, the intersection has four approach roads: north, south, east, and west. The traffic lights for each direction are S... N ,S S,S E ,S W There are N traffic entities, each with M possible actions. The actions of each traffic entity are represented as a discrete set A, where each element a... i The right-of-way allocation action chosen by the i-th subject is as follows:

[0127] A = {a1, a2, ..., a} N},a i ∈{1,2,…,M}

[0128] When the right-of-way allocation action a for each traffic entity is selected according to the right-of-way allocation strategy... i Then, the duration of traffic lights is adjusted based on these actions to control traffic flow.

[0129] Based on the right-of-way allocation strategy, select the right-of-way allocation action 'a' with the highest probability for each traffic entity. i , where i represents the subject number.

[0130] Based on the right-of-way allocation action selected by the traffic entity, a i It can calculate the right-of-way allocation weight w corresponding to each traffic entity. i This reflects the importance of traffic entities. The probability values ​​output by the policy network are used as the weights for allocating right-of-way to each traffic entity.

[0131] w i =P(a i |s t ;θ)

[0132] Based on the right-of-way allocation weights of the traffic entities w i Calculate the weight w of the path right-of-way in each direction. N w S ,w E, w W The right-of-way weights for east-west and north-south directions are determined based on the weighting percentages for each direction. Green light durations are then allocated for east-west and north-south directions according to these weighting percentages. The default yellow light duration is 3 seconds, and the full red light duration is 2 seconds.

[0133]

[0134]

[0135] Among them, T new,i T represents the adjusted green light signal duration. total This represents the total traffic light duration. The final east-west green light time is Tnew. ,NS The yellow light lasts for 3 seconds, and the red light lasts for Ttotal-Tnew. ,NS-3, the green light time for north-south traffic is Tnew ,EW The yellow light lasts for 3 seconds, and the red light lasts for Ttotal-Tnew. ,EW -3. Use this signal timing strategy to update the traffic light timing.

[0136] The preferred embodiments of the present invention have been described in detail above with reference to the accompanying drawings. These embodiments are merely descriptions of preferred embodiments and are not intended to limit the scope or concept of the invention. The specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. Such combinations, as long as they do not violate the spirit of the present invention, should also be considered as part of this disclosure. To avoid unnecessary repetition, the present invention will not further describe the various possible combinations.

[0137] This invention is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this invention and without departing from the design idea of ​​this invention, all modifications and improvements made by those skilled in the art to the technical solutions of this invention should fall within the protection scope of this invention. The technical content for which protection is sought in this invention has been fully described in the claims.

Claims

1. A traffic signal control method for heterogeneous right-of-way allocation among multiple stakeholders, characterized in that, Includes the following steps: Step 1: Obtain the feature information of each traffic entity, and establish the feature vector of each traffic entity based on the feature information; Each traffic entity is treated as a node, and the adjacency matrix of any pair of traffic entities is used as an edge. A weight matrix is ​​obtained by assigning weights to each edge, thereby constructing a traffic entity graph with a weight matrix. The feature vector of each node is processed based on the weight matrix in the main traffic map to obtain the initial feature representation of each node; Then, the feature representations of each node are input into the graph neural network for processing to obtain a low-dimensional feature representation of each node, i.e., the traffic subject. Step 2: Obtain the right-of-way allocation strategy based on the A2C algorithm model. The process is as follows: A state matrix is ​​constructed using the low-dimensional feature representations of each traffic subject obtained in step 1, and the choice of each traffic subject to the right-of-way allocation scheme is defined as a selection action. The A2C algorithm model uses a policy network to estimate the state matrix, obtaining the probability distribution of multiple right-of-way allocation actions for each traffic subject. The right-of-way allocation action is the possible traffic behavior of the traffic subject. Based on the probability distribution matrix, the right-of-way allocation action with the highest probability for each traffic subject can be obtained and selected to obtain the right-of-way allocation strategy. The A2C algorithm model also uses a value network to estimate the cumulative reward value under the state matrix to evaluate the merits of the right-of-way allocation strategy. The A2C algorithm model also constructs a reward function to represent the immediate reward obtained by the traffic subject after collecting and selecting actions under the state matrix, and constructs an advantage function based on the difference between the output of the policy network and the output of the value network. The A2C algorithm model is trained and updated. During training and updates, the policy gradient is calculated based on the advantage function, and the gradient ascent method is used to process the policy gradient to update the parameters of the policy network. At the same time, the mean squared error of the value network is calculated using the cumulative reward and the output of the value network, and the mean squared error of the value network is processed using the gradient descent method to update the parameters of the value network. Finally, an overall loss function is established based on the gradient ascent result of the policy gradient and the gradient descent result of the mean squared error of the value network. Training ends when the overall loss function reaches the convergence condition. Thus, the right-of-way allocation strategy for each traffic subject is output through the trained A2C algorithm model. Step 3: Based on the right-of-way allocation strategy for each traffic entity obtained in Step 2, obtain the right-of-way allocation weight for each traffic entity, and calculate the green light duration for each direction of the traffic signal based on the right-of-way allocation weight for each traffic entity.

2. The traffic signal control method for heterogeneous right-of-way allocation for multiple stakeholders according to claim 1, characterized in that, In step 1, the weight of each edge is calculated based on the distance and interaction frequency between the traffic entities corresponding to that edge.

3. The traffic signal control method for heterogeneous right-of-way allocation for multiple stakeholders as described in claim 1, characterized in that, In step 1, after performing linear embedding and nonlinear mapping on the feature vector of each node based on the weight matrix, the initial feature representation of each node is obtained.

4. A traffic signal control method for heterogeneous right-of-way allocation for multiple stakeholders as described in claim 3, characterized in that, In step 1, the feature vector of each node is first standardized, and then linear embedding and nonlinear mapping are performed on the standardized feature vector of each node.

5. A traffic signal control method for heterogeneous right-of-way allocation for multiple stakeholders according to claim 1, characterized in that, In step 1, the graph neural network uses multiple graph convolutional layers to process the feature representation of each node. Each graph convolutional layer combines the feature representation of each node with the feature representation of neighboring nodes to obtain a new feature representation. The output of each layer in the multiple graph convolutional layers is used as the input of the next layer, thereby obtaining the low-dimensional feature representation of each node.

6. A traffic signal control method for heterogeneous right-of-way allocation for multiple stakeholders according to claim 1, characterized in that, In step 2, the immediate return is calculated using a return function, and then the immediate returns are summed to obtain the cumulative return.

Images