A multi-intersection traffic signal control method and system based on multi-agent reinforcement learning

Abstract

This invention relates to a multi-intersection traffic signal control method and system based on multi-agent reinforcement learning, belonging to the field of intelligent traffic control and traffic signal management. The method includes the following steps: acquiring real-time traffic flow data from multiple intersections and performing feature encoding to obtain single-intersection feature vectors; aggregating global road network information through a dynamic graph convolutional network to generate intersection-level feature vectors; calculating attention weights through a collaborative selection module and using a learnable threshold generation sub-network to generate dynamic thresholds to filter and sparsify these weights, thereby explicitly and adaptively determining collaborative intersections and their weights; weighting and summing the obtained sparse attention weights with all intersection features to generate a comprehensive intersection feature representation; and finally, inputting this feature into a decision network to output real-time traffic signal control actions.

Description

Technical Field

[0001] This invention belongs to the field of intelligent traffic control and traffic signal management technology, and relates to a multi-intersection traffic signal control method and system based on multi-agent reinforcement learning. Background Technology

[0002] Traffic signal control systems, as a core component of intelligent transportation systems, play a crucial role in improving road network efficiency, alleviating traffic congestion, and ensuring road traffic safety. They are primarily used to coordinate and optimize signal phases at multiple intersections in an urban road network. By dynamically adjusting the green light duration and phase switching timing at each intersection, they balance the traffic demand from different directions, reduce queue lengths and delays, improve the overall road network capacity and operational efficiency, and mitigate the economic losses and environmental impacts of urban traffic congestion.

[0003] Current traffic signal control methods mostly adopt fixed coordination range or full road network coordination mode, which have problems such as high communication cost, high computational complexity, and insufficient real-time performance. In addition, they do not fully explore the potential correlation between non-adjacent intersections, resulting in poor robustness and scalability of control strategies, making it difficult to achieve the globally optimal traffic signal coordination control effect. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to provide a multi-intersection traffic signal control method and system based on multi-agent reinforcement learning.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A multi-intersection traffic signal control method and system based on multi-agent reinforcement learning includes the following steps: S1: Obtain real-time traffic flow data from multiple intersections; S2: Perform feature encoding on the real-time traffic flow data of each intersection to obtain the single-intersection feature vector corresponding to a single intersection; S3: Input all the single-intersection feature vectors into a dynamic graph convolutional network for interaction to obtain multiple single-intersection feature vectors that aggregate global road network information; S4: All single-intersection vectors of the aggregated global road network information are correlated through a two-layer multilayer perceptron to obtain the attention weight vectors of each intersection, which together form the attention weight matrix; at the same time, all single-intersection feature vectors are input into the threshold to generate a network to calculate a dynamic threshold, and the sparse attention weight matrix is ​​obtained by filtering with this threshold. S5: Weight the single-intersection feature representations of all intersections with the attention weight matrix to obtain a comprehensive intersection feature vector that integrates its own information, global information and collaborative intersection information; S6: Input all the comprehensive intersection features into the decision strategy network and output the real-time traffic signal control actions for all intersections.

[0006] Further, in step S2, the raw traffic flow data for each intersection includes multiple indicators of eight different traffic actions. This raw traffic flow data is input into a multilayer perceptron to obtain eight embedding vectors, each representing multiple indicators of a single traffic action at a single intersection. These embedding vectors are then added together according to phase to obtain eight phase-level feature vectors, each representing multiple indicators of a phase at an intersection. The eight phase-level feature vectors are then concatenated pairwise to obtain a tensor matrix representing the relationship between different phases, where each element corresponds to a phase pair feature vector. This tensor matrix is ​​then sequentially input into a depthwise separable convolution, a mask matrix, and another depthwise separable convolution for feature extraction and conflict phase filtering, resulting in a tensor matrix that aggregates all non-conflicting phases. Finally, the tensor matrix of non-conflicting phases is flattened and integrated and nonlinearly transformed using a multilayer perceptron to obtain multiple 64-dimensional single-intersection feature vectors.

[0007] Furthermore, the raw traffic flow data includes multiple indicators such as phase status, number of vehicles, queue length, road occupancy rate, and number of parked vehicles; the eight different traffic actions include east-straight, west-straight, east-left turn, west-left turn, south-straight, north-straight, south-left turn, and north-left turn; the eight phases correspond to four symmetrical release phases and four single-entry release phases.

[0008] Furthermore, step S3 specifically includes the following steps: All single-intersection feature vectors are input into a dynamic graph convolutional network. The dynamic graph convolutional network generates a network with trainable edge weights and calculates the dynamic connection weights between each pair of intersections based on the real-time traffic conditions to form a dynamic adjacency matrix. Then, based on the dynamic adjacency matrix, information is propagated and aggregated between intersections through graph convolution operations to obtain multiple single-intersection feature vectors that aggregate the dynamic dependencies of the road network.

[0009] Furthermore, step S4 specifically includes the following steps: All single-intersection feature vectors that aggregate the dynamic dependencies of the road network are input into a two-layer multi-perceptron to obtain an attention weight matrix. At the same time, all single-intersection feature vectors are input into a trainable threshold generation subnetwork to calculate a dynamic threshold. The attention weight matrix is ​​filtered using the dynamic threshold, and the scores below the dynamic threshold are set to zero. The filtered non-zero weight values ​​are then renormalized to obtain the final sparse attention weight matrix.

[0010] Furthermore, in step S5, the attention weight matrix is ​​weighted with the single-intersection feature vectors of all intersections to obtain multiple comprehensive intersection feature vectors that integrate their own information, global information, and collaborative intersection information.

[0011] Furthermore, in step S6, after inputting the feature vectors of multiple integrated intersections into the decision strategy network, the signal phase selection results of all intersections are output through multilayer perceptron feature extraction, Transformer capture of traffic state temporal dependence, multilayer perceptron and Softmax normalization processing, i.e. the real-time traffic signal control action.

[0012] Furthermore, the self-attention reinforcement constraint and the cooperative sparsity constraint are introduced to guide the attention weight moments to learn cooperative relationships that conform to the laws of traffic physics.

[0013] On the other hand, the present invention provides a multi-intersection traffic signal control system based on multi-agent reinforcement learning, comprising: Data acquisition and feature encoding module: acquires real-time traffic flow data from multiple intersections, performs feature encoding on the real-time traffic flow data, and obtains the feature vector of a single intersection; Global information aggregation module: The vector representation of the individual intersection is processed by a dynamic graph convolutional network to generate intersection feature vectors that aggregate key traffic information of the road network. Collaborative selection module: The attention weight matrix is ​​obtained through two-layer MLP calculation. The attention weight matrix and the single-intersection vector representation are weighted to generate a comprehensive intersection feature vector that integrates its own information, global information and collaborative intersection information. Signal control decision module: It is used to input the integrated intersection feature vector into the decision strategy network and output the real-time traffic signal control actions of the intersection.

[0014] The beneficial effects of this invention are as follows: This invention realizes explicit and adaptive decision-making on the number and weight of cooperative intersections through a learnable dynamic threshold mechanism, thereby improving the accuracy and interpretability of cooperative control. Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0015] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1This is a schematic diagram of a traffic signal control system according to an embodiment of the present invention; Figure 2 This is a flowchart of a traffic signal control method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the feature encoding module structure according to an embodiment of the present invention; Figure 4 This is a flowchart of the global information aggregation module and the collaborative selection module in an embodiment of the present invention; Figure 5 This is a schematic diagram of the decision strategy module structure according to an embodiment of the present invention. Detailed Implementation

[0016] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0017] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0018] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the invention. However, it will be apparent to those skilled in the art that embodiments of the invention may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the invention.

[0019] Example 1: Please see Figure 1 This embodiment provides a multi-intersection traffic signal control system based on multi-agent reinforcement learning, including a data acquisition and feature encoding module, a global information aggregation module, a collaborative selection module, and a signal decision module.

[0020] The data acquisition and feature encoding module acquires real-time traffic flow data from multiple intersections through sensors, performs feature encoding on the real-time traffic flow data, and obtains the feature vector of a single intersection. The global information aggregation module uses a dynamic graph convolutional network to aggregate the feature vectors of a single intersection, generating intersection feature vectors that aggregate key traffic information of the road network. The collaborative selection module calculates the attention weight matrix through a two-layer MLP and then weights it with the single-intersection vector representation to generate a comprehensive intersection feature representation that integrates its own information, global information and collaborative intersection information. The signal control decision module is used to input the comprehensive intersection feature representation into the decision strategy network and output the real-time traffic signal control actions of the intersection.

[0021] Example 2: like Figure 2 As shown, this embodiment, based on the system described in Embodiment 1, provides a multi-intersection traffic signal control method based on multi-agent reinforcement learning. The implementation steps of this method are as follows: Step 1: Collect real-time traffic flow data from multiple intersections using multi-source devices such as urban traffic display screens, intersection radar detectors, and camera sensors. Specifically, this includes multi-dimensional indicators such as phase status, number of vehicles, queue length, road occupancy rate, traffic flow, and number of parked vehicles at each intersection. Step 2: Extract multidimensional features of each traffic action at a single intersection, and input these multidimensional features into a multilayer perceptron (MLP) for embedding and concatenation to obtain an embedding vector for each traffic action. In this embodiment, a single traffic action corresponds to eight candidate signal phases (four symmetrical release phases and four single-way release phases) of a standard four-way intersection. The multidimensional features include phase state, number of vehicles, queue length, road occupancy rate, etc. After MLP mapping, a traffic action embedding vector with a dimension of 128 is obtained. ; Step 3: Input the traffic action embedding vector obtained in Step 2 into the feature encoding module for feature encoding, including the following steps: (1) such as Figure 3 As shown, feature encoding first assembles the embedding vectors of the eight traffic actions into a set. ; (2) Add the embedding vectors of two actions belonging to the same phase to obtain the phase embedding. This represents the aggregated representation of the phase. This operation elevates the analysis unit from 8 independent motions to 8 signal phases, thereby obtaining the embedding set of all phases. ; (3) Combine all phases pairwise to construct a phase relation matrix. Each element in this matrix corresponds to an eigenvector of a phase pair (p, q), which is formed by concatenating the embedding vectors of the two phases, and is defined as:

[0022] Where || denotes the vector concatenation operation, ultimately resulting in a vector of dimension . relation tensors

[0023] (4) The relation tensor is input into a depthwise separable convolution module to more efficiently fuse features between different phase pairs. This module first performs depthwise convolution to process the features of each channel independently and initially extract the interaction patterns between phase pairs; then it performs pointwise convolution to fuse information from different channels. This operation significantly reduces the number of parameters while maintaining the feature fusion capability, resulting in a parameter matrix. Defined as:

[0024] (5) Compare the parameter matrix with a predefined binary collision mask matrix. Perform multiplication if phase and If there is a conflict (both cannot be allowed simultaneously), the mask matrix has a value of 1; otherwise, it has a value of 0. This filters out information from non-conflicting phase pairs, allowing the model to focus on learning competing phase pairs. Defined as:

[0025] Here, ⊙ represents element-wise multiplication.

[0026] (6) To further refine the information, the masked feature matrix is... Applying a depthwise separable convolutional module with the same structure again outputs a phase competition representation matrix that aggregates the competition relationships between all non-conflicting phases. Defined as:

[0027] (7) Represent the phase competition matrix The vector is flattened and then integrated and nonlinearly transformed using a multilayer perceptron, compressing it into a 64-dimensional phase-level feature vector. Defined as:

[0028] Step 4: Take the result from Step 3 N The set of phase-level feature vectors of each intersection As input, perform intersection-level feature aggregation, such as Figure 3 As shown.

[0029] (1) Dynamic Adjacency Matrix Generation: The graph structure of the road network is not fixed. First, a network is generated using a lightweight edge weighting method, based on any two intersections. and The real-time state of the data is used to calculate the dynamic connection strength between them. The network takes the features of the preceding intersections as input:

[0030] in, W The weight matrix is ​​a learnable matrix. a Let be a learnable weight vector, and || denote the concatenation operation. To maintain stability, for each intersection... Connect it with all intersections Calculated Softmax normalization is performed to obtain the final dynamic adjacency matrix elements. :

[0031] Among the elements Indicates information from the intersection Spread to the intersection Normalized weights.

[0032] (2) Graph Convolution Aggregation: Using the dynamically generated adjacency matrix A, a graph convolution operation is performed to aggregate neighborhood information. A multi-layer propagation paradigm is adopted here:

[0033] in, It is composed of the stacked phase-level feature vectors of all intersections, i.e. , For the learnable parameters of the l-th layer, This is the activation function. After... L After layer propagation, the final intersection-level feature representation matrix, which aggregates global dynamic information, is obtained. This process integrates the features of each intersection with information from other intersections that are most relevant to its current traffic conditions.

[0034] Step 5: Convert the intersection-level feature representation matrix obtained in Step 4 into a single matrix. Input collaborative selection module, such as Figure 4 As shown: (1) Calculate the initial attention weights: the feature representation matrix after aggregating all intersections. Input a two-level MLP, for each intersection Calculate a weight vector for all intersections. Then, apply the weight vector... Sigmoid functions and L1 Norm normalization yields the initial attention weight vector. ,satisfy .

[0035] (2) Dynamic Threshold Generation: A threshold generation subnetwork composed of a lightweight MLP is designed. This subnetwork uses the characteristics of the current intersection. For input, through Sigmoid The function outputs a dynamic scalar threshold in the interval (0,1). , which represents the critical value for retaining effective collaborative weights.

[0036] (3) Threshold filtering and sparsification: using dynamic thresholds For the initial weights Perform filtering. Generate a binary mask vector. , among which when hour, Conversely, if the result is 0, element-wise multiplication is performed: Weights below the threshold are reset to zero. Then, for... Perform L1 renormalization. Obtain the final, sparse attention weight vector. ,satisfy At this point, the number of non-zero elements in the vector... This refers to the number of adaptive cooperative intersections at the current time.

[0037] Step 6: Weight the attention weights from Step 5 with the corresponding intersection vector representations to obtain the comprehensive intersection feature representation; defined as:

[0038] Step 7: Input the comprehensive intersection features into the decision strategy network Output the final signal to control the action. Its definition is:

[0039] The decision-making strategy network architecture is as follows: Figure 5 As shown, it is designed to MLP-Transformer-MLP The hybrid architecture is designed to capture both complex feature interactions and dynamic temporal dependencies of traffic states.

[0040] Step 8: To guide the attention weight matrix ATT To learn the cooperative relationships that conform to the laws of traffic physics, the following constraints are introduced into the training objectives: (1) Self-attention reinforcement constraint: This encourages each intersection to focus on its own state during decision-making, achieved by maximizing the mean of the diagonal elements of the attention matrix, defined as:

[0041] (2) Cooperative sparsity constraint: To avoid excessive dispersion of attention and improve the targeting of collaboration, the weight concentration is encouraged by minimizing the entropy of the attention distribution, defined as:

[0042] (3) The overall training loss function is the policy gradient loss of reinforcement learning. The weighted sum of the sum and constraint losses is defined as:

[0043] In the above embodiments, to obtain the traffic signal control strategy, a multi-intersection traffic signal control network based on multi-agent reinforcement learning can be trained. The dataset involved in the training process can be obtained through a traffic simulation platform or real intersection data acquisition equipment. Traffic flow data in the road network is collected through sensors or simulators. The traffic flow data mainly includes the phase status of each intersection, the number of vehicles, queue length, road occupancy rate, and traffic flow. Traffic control actions include the selection of signal phases and the setting of phase durations at each intersection. To address the problems of uneven distribution of dataset samples and insufficient samples in extreme scenarios, methods such as random road network topology generation, traffic flow disturbance, and sudden congestion injection are used to enhance the training data and improve the robustness and generalization ability of the control strategy. In the deep learning framework, this model architecture is used to build a collaborative control network. The prepared dataset is input into the multi-agent reinforcement learning control network. A batch size of the largest possible size is used to improve network performance and accelerate network training. A smaller learning rate is set to improve the network training effect, and an appropriate epoch is selected to ensure network convergence.

[0044] Example 3: An electronic device, comprising a memory and a processor; The memory is used to store computer programs; The processor is configured to implement the method described in Embodiment 1 when executing the computer program.

[0045] Example 4: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in Embodiment 1.

[0046] Example 5: A computer program product includes a computer program that, when executed by a processor, implements the method described in Example 1.

[0047] In the above embodiments, the reference to "this embodiment" in the specification indicates that a specific feature, structure, or characteristic described in connection with the embodiment is included in at least some embodiments, but not necessarily all embodiments. Multiple appearances of "this embodiment" do not necessarily refer to the same embodiment.

[0048] In the above embodiments, although the invention has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory structures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed. The embodiments of the invention are intended to cover all such substitutions, modifications, and variations falling within the broad scope of the appended claims.

[0049] As will be understood by those skilled in the art, the computer-readable storage medium described in this embodiment allows for the implementation of all or part of the steps in the above method embodiments by computer program-related hardware. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0050] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver, and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication between them. The memory is used to store computer programs, the communication interface is used to perform communication, and the processor and the transceiver are used to run the computer programs, so that the electronic terminal performs the steps of the above method.

[0051] In this embodiment, the memory may include random access memory (RAM) and may also include non-volatile memory, such as at least one disk storage device.

[0052] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0053] This invention can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0054] This invention can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-intersection traffic signal control method based on multi-agent reinforcement learning, characterized in that: Includes the following steps: S1: Obtain real-time traffic flow data from multiple intersections; S2: Perform feature encoding on the real-time traffic flow data of each intersection to obtain the single-intersection feature vector corresponding to a single intersection; S3: Input all the single-intersection feature vectors into a dynamic graph convolutional network for interaction to obtain multiple single-intersection feature vectors that aggregate global road network information; S4: All single-intersection vectors of the aggregated global road network information are correlated through a two-layer multilayer perceptron to obtain the attention weight vectors of each intersection, which together form the attention weight matrix; at the same time, all single-intersection feature vectors are input into the threshold to generate a network to calculate a dynamic threshold, and the sparse attention weight matrix is ​​obtained by filtering with this threshold. S5: Weight the single-intersection feature representations of all intersections with the attention weight matrix to obtain a comprehensive intersection feature vector that integrates its own information, global information and collaborative intersection information; S6: Input all the comprehensive intersection feature representations into the decision strategy network, and output the real-time traffic signal control actions for all intersections.

2. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 1, characterized in that: In step S2, the raw traffic flow data for each intersection includes multiple indicators of eight different traffic actions. This raw traffic flow data is input into a multilayer perceptron to obtain eight embedding vectors, each representing multiple indicators of a single traffic action at a single intersection. These embedding vectors are then added together according to phase to obtain eight phase-level feature vectors, each representing multiple indicators of a phase at an intersection. The eight phase-level feature vectors are concatenated pairwise to obtain a tensor matrix of relationships between different phases, where each element corresponds to a phase pair feature vector. This tensor matrix is ​​then sequentially input into a depthwise separable convolution, a mask matrix, and another depthwise separable convolution for feature extraction and conflict phase filtering, resulting in a tensor matrix that aggregates all non-conflicting phases. Finally, the tensor matrix of non-conflicting phases is flattened and integrated and nonlinearly transformed using a multilayer perceptron to obtain multiple 64-dimensional single-intersection feature vectors.

3. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 2, characterized in that: The raw traffic flow data includes multiple indicators such as phase status, number of vehicles, queue length, road occupancy rate, and number of parked vehicles; the eight different traffic actions include east-straight, west-straight, east-left turn, west-left turn, south-straight, north-straight, south-left turn, and north-left turn; the eight phases correspond to four symmetrical release phases and four single-entry release phases.

4. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 2, characterized in that: Step S3 specifically includes the following steps: All single-intersection feature vectors are input into a dynamic graph convolutional network. The dynamic graph convolutional network generates a network with trainable edge weights and calculates the dynamic connection weights between each pair of intersections based on the real-time traffic conditions to form a dynamic adjacency matrix. Then, based on the dynamic adjacency matrix, information is propagated and aggregated between intersections through graph convolution operations to obtain multiple single-intersection feature vectors that aggregate the dynamic dependencies of the road network.

5. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 4, characterized in that: Step S4 Specifically, the following steps are included: All single-intersection feature vectors that aggregate the dynamic dependencies of the road network are input into a two-layer multi-perceptron to obtain an attention weight matrix. At the same time, all single-intersection feature vectors are input into a trainable threshold generation subnetwork to calculate a dynamic threshold. The attention weight matrix is ​​filtered using the dynamic threshold, and scores below the dynamic threshold are set to zero. The filtered non-zero weight values ​​are renormalized to obtain the final sparse attention weight matrix.

6. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 5, characterized in that: In step S5, the attention weight matrix is ​​weighted with the single-intersection feature vectors of all intersections to obtain multiple comprehensive intersection feature vectors that integrate their own information, global information, and collaborative intersection information.

7. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 6, characterized in that: In step S6, after inputting the feature vectors of multiple integrated intersections into the decision strategy network, the signal phase selection results of all intersections are output through multilayer perceptron feature extraction, Transformer capture of traffic state temporal dependence, multilayer perceptron and Softmax normalization processing, i.e. the real-time traffic signal control action.

8. The multi-intersection traffic signal control method based on multi-agent reinforcement learning according to claim 1, characterized in that: By introducing self-attention reinforcement constraints and cooperative sparsity constraints, attention weight moments are guided to learn cooperative relationships that conform to the laws of traffic physics.

9. A multi-intersection traffic signal control system based on multi-agent reinforcement learning, characterized in that: include: Data acquisition and feature encoding module: acquires real-time traffic flow data from multiple intersections, performs feature encoding on the real-time traffic flow data, and obtains the feature vector of a single intersection; Global information aggregation module: The vector representation of the individual intersection is processed by a dynamic graph convolutional network to generate intersection feature vectors that aggregate key traffic information of the road network. Collaborative selection module: The attention weight matrix is ​​obtained through two-layer MLP calculation. The attention weight matrix and the single-intersection vector representation are weighted to generate a comprehensive intersection feature vector that integrates its own information, global information and collaborative intersection information. Signal control decision module: It is used to input the integrated intersection feature vector into the decision strategy network and output the real-time traffic signal control actions of the intersection.

Images