Urban traffic signal real-time collaborative optimization system and method based on space-time diagram convolutional network and reinforcement learning

Abstract

The invention discloses an urban traffic signal real-time collaborative optimization system and method based on a space-time diagram convolutional network and reinforcement learning, and relates to the technical field of intelligent traffic control. In order to overcome the defects of traffic signal fixed period control, the technical scheme adopted by the invention comprises edge computing equipment which is deployed beside an intersection camera and is used for acquiring video stream data in real time through a built-in local model, extracting traffic flow state characteristics and realizing dynamic phase timing optimization through cross-intersection collaborative decision, meanwhile, local model parameters are generated and uploaded to the cloud federated learning platform; the cloud federated learning platform is used for aggregating and optimizing the local model parameters of the edge computing devices, and regularly issuing global update parameters to the edge computing devices; and the traffic signal control equipment is deployed at the intersection and is used for adjusting the display state of the traffic signal lamp in real time according to the dynamic phase timing instruction. The traffic efficiency of the urban road network can be obviously improved, and the traffic control cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of intelligent traffic control technology, specifically to a real-time collaborative optimization system and method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning. Background Art

[0002] With the acceleration of urbanization, urban traffic congestion has become increasingly serious, bringing many negative impacts to people's travel and urban development. Traditional traffic signal control methods have significant shortcomings: fixed-cycle traffic lights cannot adjust according to real-time traffic flow changes, making it difficult to cope with dynamic traffic conditions, resulting in limited reduction in average delays, typically only around 15-20%; while centralized control systems have achieved coordinated control of traffic signals to some extent, communication delays can reach 2-5 minutes during peak hours, failing to meet the needs of real-time control. Therefore, there is an urgent need for a traffic signal control method that can adapt to the dynamic changes of urban road networks in real time and efficiently.

[0003] Traffic signal control technology has gone through three stages: traditional fixed-cycle control (based on classical theory to design timing systems, which suffer from defects such as the disconnect between static models and dynamic demands and the lack of road network coordination capabilities), inductive adaptive control (represented by SCOOT and SCATS, which can perceive traffic flow in real time but suffer from problems such as centralized architecture latency, limited scalability, and data privacy risks), and distributed reinforcement learning control (the rise of third-generation systems, but facing challenges such as insufficient spatiotemporal feature modeling, multi-agent coordination dilemmas, and lack of multi-objective trade-offs), as well as other technical attempts (such as combining genetic algorithms and fuzzy logic, which suffer from high computational complexity and difficulty in knowledge transfer). Each stage and method has its limitations.

[0004] To address the aforementioned pain points, a real-time collaborative optimization system and method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning is designed and proposed. This system achieves real-time adaptive optimization of traffic signals through edge-cloud collaborative computing, providing an innovative solution to alleviate urban traffic congestion. Summary of the Invention

[0005] This invention addresses the technical pain points of traditional fixed-period signal control, such as poor dynamic adaptability, high communication latency in centralized systems, and local optima caused by single-intersection optimization. It provides a real-time collaborative optimization system and method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning. By integrating advanced spatiotemporal feature extraction technology and distributed decision architecture, it achieves real-time collaborative control of urban traffic signals, effectively reducing vehicle delays, improving traffic efficiency, reducing exhaust emissions, and promoting green travel.

[0006] Firstly, the present invention provides a real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning. The technical solution adopted to solve the above-mentioned technical problems is as follows:

[0007] A real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning includes edge computing devices, a cloud-based federated learning platform, and traffic signal control equipment, wherein:

[0008] Edge computing devices are deployed next to intersection cameras. They have built-in local models and GPU acceleration modules to support the operation of local models. They are used to collect video stream data in real time, extract traffic flow status features, and achieve dynamic phase timing optimization through cross-intersection collaborative decision-making. At the same time, they generate local model parameters and upload them to the cloud federated learning platform.

[0009] The cloud-based federated learning platform is used to aggregate and optimize the local model parameters of various edge computing devices through the federated learning framework, and regularly send global update parameters to the edge computing devices to achieve collaborative upgrades of local models while protecting data privacy.

[0010] Traffic signal control equipment, deployed at intersections, is used to receive dynamic phase timing commands output by edge computing devices, adjust the display status of traffic lights in real time, and translate the cross-intersection collaborative decisions of edge computing devices into actual traffic control actions.

[0011] Optionally, the local models involved include a traffic detection model integrating YOLOv5 and DeepSORT, a spatiotemporal graph convolutional network model, and a reinforcement learning model employing a distributed decision architecture.

[0012] The edge computing device performs the following operations based on the local model:

[0013] The vehicle flow detection model integrates YOLOv5 and DeepSORT to collect vehicle data and track vehicle flow trajectories in real time.

[0014] By using a spatiotemporal graph convolutional network model to model the topological relationships of intersections and the spatiotemporal dependence of traffic flow, traffic flow state features can be accurately extracted.

[0015] Based on a reinforcement learning model with a distributed decision-making architecture, each intersection is treated as an independent intelligent agent. Cross-intersection collaborative decision-making is achieved through a multi-head attention mechanism, and phase timing is dynamically optimized based on a hybrid reward function that integrates queue length, delay time, carbon emissions, and fairness constraints.

[0016] Alternatively, the edge computing devices involved may use the NVIDIA Jetson AGX Xavier hardware platform;

[0017] The edge computing device has a built-in vehicle flow detection model, which adopts a combination of YOLOv5 target detection algorithm and DeepSORT target tracking algorithm: first, the YOLOv5 target detection algorithm is used to identify vehicle targets in video or image in real time and locate their positions, and then the DeepSORT target tracking algorithm is used to continuously track the vehicles across frames, thereby completing the vehicle flow detection task.

[0018] Further, optionally, the spatiotemporal graph convolutional network model involved includes an input part, an ST-Conv Block module, and an output part, wherein:

[0019] The input is time series data;

[0020] The ST-Conv Block module consists of three spatiotemporal convolutional layers, specifically:

[0021] Temporal gated convolution is used to perform temporal-dimensional gated convolution operations on input data. It adaptively captures temporal dynamic features through a gating mechanism and controls the flow of information to effectively extract temporal dependencies.

[0022] Spatial graph convolution is used to perform spatial graph convolution based on the output of temporally gated convolution, focusing on capturing the spatial dependencies between different intersections and mining the spatial correlation features of traffic flow.

[0023] The second temporal gated convolution is performed on the output data of the spatial graph convolution to further enhance the temporal feature extraction capability and integrate the performance of the features after spatial graph convolution in the temporal dimension.

[0024] The output is the data processed by the ST-Conv Block module, which is used for traffic flow prediction tasks.

[0025] Further, optionally, the reinforcement learning model involved includes an input processing layer, a feature fusion and memory layer, and a decision and evaluation layer, wherein:

[0026] The input processing layer is configured to receive three types of input information: traffic flow status, vehicle waiting status, and neighbor policies. Specifically: for the traffic flow status, a fully connected layer with a hidden layer dimension of 128 performs feature extraction and preliminary processing to convert the raw traffic flow status data into corresponding feature representations; for the vehicle waiting status, a fully connected layer with a hidden layer dimension of 32 processes the waiting-related status information to extract key features; and for the neighbor policies, which represent the policy information of neighboring agents, a fully connected layer with a hidden layer dimension of 64 processes the information to capture the effective features contained in the neighbor policies.

[0027] The feature fusion and memory layer is configured to receive three types of features processed by the fully connected layers in the above input processing layer, and input these three types of features together into a long short-term memory network with a hidden layer dimension of 64. The long short-term memory network can effectively process sequential data, capture the dependencies of data in the time dimension, fuse features from different input sources, and remember key information, thereby providing a more comprehensive and temporally sequential feature representation for subsequent decision-making and evaluation processes.

[0028] The decision and evaluation layer comprises a policy network and a value network. It is configured to receive features from the feature fusion and memory layer. Specifically, the policy network generates a specific action probability distribution using a softmax function. This distribution is used by the agent to select actions to be executed, thereby enabling control decisions regarding traffic signals. The value network evaluates the value of the current state or state-action pairs using a linear layer. The evaluation results serve as feedback information, providing a basis for optimizing the policy network and helping the reinforcement learning process to better learn the optimal policy.

[0029] Alternatively, the expression for the hybrid reward function involved is as follows:

[0030] R = ω1 * queue length + ω2 * delay time + ω3 * carbon emissions

[0031] Introducing fairness constraints, a compensation mechanism is added to the hybrid reward function: when an intersection fails to obtain a green light phase for 5 consecutive cycles, a fairness reward compensation term ΔR = β * number of waiting vehicles is superimposed on R;

[0032] Where ω1, ω2, ω3, and β are pre-set weighting coefficients.

[0033] Alternatively, during the local model operation, the edge computing device continuously updates the local model parameters and uploads the optimized local model parameters to the cloud-based federated learning platform in an encrypted manner. At the same time, it receives global optimization parameters from the cloud-based federated learning platform to iterate on the performance of the local model.

[0034] Further optional, the cloud-based federated learning platform aggregates local model parameters uploaded by edge computing devices through a federated learning framework. During the aggregation process, Paillier homomorphic encryption and differential privacy technology are used to encrypt and protect the local model parameters and traffic data, so as to achieve global knowledge sharing and local model collaborative optimization without disclosing the privacy of specific traffic data at intersections.

[0035] The optimized local model parameters are fed back to each edge computing device to update its local model.

[0036] Secondly, this invention provides a real-time collaborative optimization method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning. The technical solution adopted to solve the above-mentioned technical problems is as follows:

[0037] A real-time collaborative optimization method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning, which is based on the system described in the first aspect, includes the following steps:

[0038] S1. Deploy edge computing devices next to the intersection cameras to collect video stream data in real time and extract traffic flow status features.

[0039] S2. Edge computing devices rely on reinforcement learning models with distributed decision-making architecture to treat each intersection as an independent intelligent agent and achieve cross-intersection collaborative decision-making through multi-head attention mechanism to generate traffic signal control commands.

[0040] S3. Traffic signal control equipment deployed at intersections adjusts the display status of traffic lights in real time based on traffic signal control commands generated by edge computing devices.

[0041] S4. During the above operations, the cloud-based federated learning platform periodically collects the local model parameters of each edge computing device through the federated learning framework, updates them using the parameter aggregation method, and distributes the updated local model parameters to each edge computing device.

[0042] The present invention provides a real-time collaborative optimization system and method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning, which has the following advantages compared with the prior art:

[0043] 1. This invention can significantly improve the traffic efficiency of urban road networks, reduce traffic management costs, effectively cope with complex scenarios such as sudden congestion caused by traffic accidents and tidal traffic flow after large-scale events, and is suitable for real-time control of dynamic congestion in urban road networks.

[0044] 2. This invention is only applicable to urban road intersections, and the number of signal-controlled intersections in the road network is not less than 3; it does not support scenarios without phase control, such as highways and roundabouts. Attached Figure Description

[0045] Appendix Figure 1 This is a system architecture diagram of the present invention;

[0046] Appendix Figure 2 This is a schematic diagram of the spatiotemporal graph neural network model of the present invention;

[0047] Appendix Figure 3 This is a schematic diagram of the reinforcement learning model of the present invention. Detailed Implementation

[0048] To make the technical solution, the technical problem solved, and the technical effect of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with specific embodiments.

[0049] Example 1:

[0050] Combined with appendix Figure 1 This embodiment proposes a real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning, which includes edge computing devices, a cloud-based federated learning platform, and traffic signal control devices.

[0051] 1. Edge computing devices are deployed next to intersection cameras. They have built-in local models and GPU acceleration modules to support the operation of local models. They are used to collect video stream data in real time, extract traffic flow status features, and achieve dynamic phase timing optimization through cross-intersection collaborative decision-making. At the same time, they generate local model parameters and upload them to the cloud federated learning platform.

[0052] Specifically, the edge computing device uses the NVIDIA Jetson AGX Xavier hardware platform; the hardware requirements for the edge computing device are: it must be equipped with a heterogeneous computing architecture and include at least the following core components and performance indicators: GPU computing power of not less than 4 TFLOPS; VPU module supports INT8 quantization inference; memory capacity of not less than 16GB.

[0053] The local models include a traffic detection model integrating YOLOv5 and DeepSORT, a spatiotemporal graph convolutional network (STGCN) model, and a reinforcement learning model with a distributed decision architecture.

[0054] Edge computing devices perform the following operations based on the local model:

[0055] 1. A vehicle flow detection model integrating YOLOv5 and DeepSORT is used to collect vehicle data and track vehicle trajectories in real time. Specifically, the vehicle flow detection model adopts a combination of YOLOv5 object detection algorithm and DeepSORT object tracking algorithm. First, the YOLOv5 object detection algorithm is used to identify vehicle targets in video or images in real time and locate their positions. Then, the DeepSORT object tracking algorithm is used to continuously track vehicles across frames, thereby completing the vehicle flow detection task.

[0056] 2. The Spatiotemporal Graph Convolutional Network (STGCN) model is used to model the intersection topology and the spatiotemporal dependence of traffic flow, and to accurately extract traffic flow state features.

[0057] Reference Appendix Figure 2 The Spatiotemporal Graph Convolutional Network (STGCN) model involved includes an input part, an ST-Conv Block module, and an output part, wherein:

[0058] The input part is time series data (v t-M+1 ,…v t ), where v represents time-related features such as traffic flow, and M is the length of the time window. These data will be input into subsequent modules for processing.

[0059] The ST-Conv Block module consists of three spatiotemporal convolutional layers, specifically:

[0060] Temporal gated convolution is used to perform temporal-dimensional gated convolution operations on input data. It adaptively captures temporal dynamic features through a gating mechanism and controls the flow of information to effectively extract temporal dependencies. In this embodiment, the number of output channels is set to C=64.

[0061] Spatial Graph-Conv is used to perform spatial graph convolution based on the output of temporally gated convolution, focusing on capturing the spatial dependencies between different intersections and mining the spatial correlation features of traffic flow; in this embodiment, the number of output channels is set to C=16.

[0062] Temporal Gated-Conv: A second temporal gated convolution is performed on the output data of the spatial graph convolution to further enhance the temporal feature extraction capability and integrate the performance of the features after spatial graph convolution in the temporal dimension; in this embodiment, the number of output channels is set to C=64.

[0063] The output consists of data processed by the ST-Conv Block module. This data is then passed to the Output Layer, where the final output v^ is obtained. This output is used for predictive tasks such as traffic flow.

[0064] It should be noted that, Figure 2 The far right also shows a more detailed internal structure of the ST-Conv Block module, with input (v l t-M+1 ,…v l t First, it undergoes 1-D convolution, then combines temporal gated convolution and gated linear units (GLU) to obtain the output (v). l t-M+Kt ,…v l t This further illustrates the process of temporal feature extraction and processing, echoing the overall structure on the left, and together forming the complete architecture of the spatiotemporal graph neural network for processing spatiotemporal sequence data.

[0065] 3. Relying on a reinforcement learning model with a distributed decision-making architecture, each intersection is treated as an independent intelligent agent. Cross-intersection collaborative decision-making is achieved through a multi-head attention mechanism, and phase timing is dynamically optimized based on a hybrid reward function that integrates queue length, delay time, carbon emissions, and fairness constraints.

[0066] Reference Appendix Figure 3 The reinforcement learning model involved includes an input processing layer, a feature fusion and memory layer, and a decision and evaluation layer, wherein:

[0067] The input processing layer is configured to receive three types of input information: wave states, wait states, and neighbor policies. Specifically: for wave states, a fully connected layer with a hidden layer dimension of 128 performs feature extraction and preliminary processing to convert the raw wave state data into corresponding feature representations; for wait states, a fully connected layer with a hidden layer dimension of 32 processes the wait-related state information to extract key features; and for neighbor policies, which represent the policy information of neighboring agents, a fully connected layer with a hidden layer dimension of 64 processes the information to capture effective features contained within the neighbor policies.

[0068] The feature fusion and memory layer is configured to receive three types of features processed by each fully connected layer in the above input processing layer, and input these three types of features together into a Long Short-Term Memory (LSTM) network (64) with a hidden layer dimension of 64. The LSTM network (64) can effectively process sequence data, capture the dependencies of data in the time dimension, fuse features from different input sources, and memorize key information, thereby providing a more comprehensive and temporally relevant feature representation for subsequent decision-making and evaluation processes.

[0069] The decision and evaluation layer comprises a policy network (actor) and a value network (critic). It is configured to receive features from the feature fusion and memory layer outputs. Specifically, the policy network (actor) generates a specific action probability distribution using a softmax function. This distribution is used by the agent to select actions to execute, thereby enabling control decisions regarding traffic signals. The value network (critic) evaluates the value of the current state or state-action pairs using a linear layer. The evaluation results serve as feedback information, providing a basis for optimizing the policy network (actor) and helping the reinforcement learning process to better learn the optimal policy.

[0070] The expression for the hybrid reward function involved is as follows:

[0071] R = ω1 * queue length + ω2 * delay time + ω3 * carbon emissions

[0072] Introducing fairness constraints, a compensation mechanism is added to the hybrid reward function: when an intersection fails to obtain a green light phase for 5 consecutive cycles, a fairness reward compensation term ΔR = β * number of waiting vehicles is superimposed on R;

[0073] Where ω1, ω2, ω3, and β are pre-set weighting coefficients.

[0074] Second, the cloud-based federated learning platform aggregates and optimizes local model parameters from various edge computing devices using a federated learning framework. It periodically distributes globally updated parameters to these edge computing devices, enabling collaborative upgrades of local models while protecting data privacy. It's worth noting that Paillier homomorphic encryption and differential privacy technology are used during the aggregation process to encrypt and protect local model parameters and traffic data. This achieves global knowledge sharing and collaborative optimization of local models without disclosing the privacy of specific traffic data at intersections. The optimized local model parameters are then fed back to each edge computing device to update its local model.

[0075] 3. Traffic signal control equipment, deployed at intersections, is used to receive dynamic phase timing instructions output by edge computing devices, adjust the display status of traffic lights in real time, and transform the cross-intersection collaborative decisions of edge computing devices into actual traffic control actions.

[0076] In addition to this embodiment, during the local model operation, the edge computing device continuously updates the local model parameters and uploads the optimized local model parameters to the cloud federated learning platform in encryption. At the same time, it receives global optimization parameters from the cloud federated learning platform to iterate the performance of the local model.

[0077] Example 2:

[0078] Based on the system in Embodiment 1, this embodiment proposes a real-time collaborative optimization method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning, including the following steps:

[0079] S1. Deploy edge computing devices next to the intersection cameras to collect video stream data in real time and extract traffic flow status features.

[0080] S2. Edge computing devices rely on reinforcement learning models with distributed decision-making architecture to treat each intersection as an independent intelligent agent and achieve cross-intersection collaborative decision-making through multi-head attention mechanism to generate traffic signal control commands.

[0081] S3. Traffic signal control equipment deployed at intersections adjusts the display status of traffic lights in real time based on traffic signal control commands generated by edge computing devices.

[0082] S4. During the above operations, the cloud-based federated learning platform periodically collects the local model parameters of each edge computing device through the federated learning framework, updates them using the parameter aggregation method, and distributes the updated local model parameters to each edge computing device.

[0083] In summary, the real-time collaborative optimization system and method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning of this invention can significantly improve the traffic efficiency of urban road networks, reduce traffic management costs, and provide a scalable technical solution for dynamic traffic governance in smart cities.

[0084] The above specific examples illustrate the principles and implementation methods of the present invention in detail. These embodiments are merely for the purpose of helping to understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, any improvements and modifications made to the present invention by those skilled in the art without departing from the principles of the present invention should fall within the patent protection scope of the present invention.

Claims

1. A real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning, characterized in that, It includes edge computing devices, a cloud-based federated learning platform, and traffic signal control equipment, among which: Edge computing devices are deployed next to intersection cameras. They have built-in local models and GPU acceleration modules to support the operation of local models. They are used to collect video stream data in real time, extract traffic flow status features, and achieve dynamic phase timing optimization through cross-intersection collaborative decision-making. At the same time, they generate local model parameters and upload them to the cloud federated learning platform. The cloud-based federated learning platform is used to aggregate and optimize the local model parameters of various edge computing devices through the federated learning framework, and regularly send global update parameters to the edge computing devices to achieve collaborative upgrades of local models while protecting data privacy. Traffic signal control equipment, deployed at intersections, is used to receive dynamic phase timing commands output by edge computing devices, adjust the display status of traffic lights in real time, and translate the cross-intersection collaborative decisions of edge computing devices into actual traffic control actions.

2. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 1, characterized in that, The local model includes a traffic detection model integrating YOLOv5 and DeepSORT, a spatiotemporal graph convolutional network model, and a reinforcement learning model with a distributed decision architecture. The edge computing device performs the following operations based on the local model: The vehicle flow detection model integrates YOLOv5 and DeepSORT to collect vehicle data and track vehicle flow trajectories in real time. By using a spatiotemporal graph convolutional network model to model the topological relationships of intersections and the spatiotemporal dependence of traffic flow, traffic flow state features can be accurately extracted. Based on a reinforcement learning model with a distributed decision-making architecture, each intersection is treated as an independent intelligent agent. Cross-intersection collaborative decision-making is achieved through a multi-head attention mechanism, and phase timing is dynamically optimized based on a hybrid reward function that integrates queue length, delay time, carbon emissions, and fairness constraints.

3. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 2, characterized in that, The edge computing devices utilize the NVIDIA Jetson AGX Xavier hardware platform. The edge computing device has a built-in vehicle flow detection model, which adopts a combination of YOLOv5 target detection algorithm and DeepSORT target tracking algorithm: first, the YOLOv5 target detection algorithm is used to identify vehicle targets in video or image in real time and locate their positions, and then the DeepSORT target tracking algorithm is used to continuously track the vehicles across frames, thereby completing the vehicle flow detection task.

4. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 2, characterized in that, The spatiotemporal graph convolutional network model includes an input part, an ST-Conv Block module, and an output part, wherein: The input is time series data; The ST-Conv Block module consists of three spatiotemporal convolutional layers, specifically: Temporal gated convolution is used to perform temporal-dimensional gated convolution operations on input data. It adaptively captures temporal dynamic features through a gating mechanism and controls the flow of information to effectively extract temporal dependencies. Spatial graph convolution is used to perform spatial graph convolution based on the output of temporally gated convolution, focusing on capturing the spatial dependencies between different intersections and mining the spatial correlation features of traffic flow. The second temporal gated convolution is performed on the output data of the spatial graph convolution to further enhance the temporal feature extraction capability and integrate the performance of the features after spatial graph convolution in the temporal dimension. The output is the data processed by the ST-Conv Block module, which is used for traffic flow prediction tasks.

5. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 2, characterized in that, The reinforcement learning model includes an input processing layer, a feature fusion and memory layer, and a decision and evaluation layer, wherein: The input processing layer is configured to receive three types of input information: traffic flow status, vehicle waiting status, and neighbor policies. Specifically: for the traffic flow status, a fully connected layer with a hidden layer dimension of 128 performs feature extraction and preliminary processing to convert the raw traffic flow status data into corresponding feature representations; for the vehicle waiting status, a fully connected layer with a hidden layer dimension of 32 processes the waiting-related status information to extract key features; and for the neighbor policies, which represent the policy information of neighboring agents, a fully connected layer with a hidden layer dimension of 64 processes the information to capture the effective features contained in the neighbor policies. The feature fusion and memory layer is configured to receive three types of features processed by the fully connected layers in the above input processing layer, and input these three types of features together into a long short-term memory network with a hidden layer dimension of 64. The long short-term memory network can effectively process sequential data, capture the dependencies of data in the time dimension, fuse features from different input sources, and remember key information, thereby providing a more comprehensive and temporally sequential feature representation for subsequent decision-making and evaluation processes. The decision and evaluation layer comprises a policy network and a value network. It is configured to receive features from the feature fusion and memory layer. Specifically, the policy network generates a specific action probability distribution using a softmax function. This distribution is used by the agent to select actions to be executed, thereby enabling control decisions regarding traffic signals. The value network evaluates the value of the current state or state-action pairs using a linear layer. The evaluation results serve as feedback information, providing a basis for optimizing the policy network and helping the reinforcement learning process to better learn the optimal policy.

6. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 2, characterized in that, The expression for the hybrid reward function is as follows: R = ω1 * queue length + ω2 * delay time + ω3 * carbon emissions Introducing fairness constraints, a compensation mechanism is added to the hybrid reward function: when an intersection fails to obtain a green light phase for 5 consecutive cycles, a fairness reward compensation term ΔR = β * number of waiting vehicles is superimposed on R; Where ω1, ω2, ω3, and β are pre-set weighting coefficients.

7. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 2, characterized in that, During the local model operation, the edge computing device continuously updates the local model parameters and uploads the optimized local model parameters to the cloud federated learning platform in encryption. At the same time, it receives global optimization parameters from the cloud federated learning platform to iterate on the performance of the local model.

8. The real-time collaborative optimization system for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning according to claim 1, characterized in that, The cloud-based federated learning platform aggregates local model parameters uploaded by edge computing devices through a federated learning framework. During the aggregation process, Paillier homomorphic encryption and differential privacy technology are used to encrypt and protect the local model parameters and traffic data, so as to achieve global knowledge sharing and local model collaborative optimization without disclosing the privacy of specific traffic data at intersections. The optimized local model parameters are fed back to each edge computing device to update its local model.

9. A real-time collaborative optimization method for urban traffic signals based on spatiotemporal graph convolutional networks and reinforcement learning, characterized in that, Based on the system as described in any one of claims 1-8, it includes the following steps: S1. Deploy edge computing devices next to intersection cameras to collect video stream data in real time and extract traffic flow status features. S2. Edge computing devices rely on reinforcement learning models with distributed decision-making architecture to treat each intersection as an independent intelligent agent and achieve cross-intersection collaborative decision-making through multi-head attention mechanism to generate traffic signal control commands. S3. Traffic signal control equipment deployed at intersections adjusts the display status of traffic lights in real time based on traffic signal control commands generated by edge computing devices. S4. During the above operations, the cloud-based federated learning platform periodically collects the local model parameters of each edge computing device through the federated learning framework, updates them using the parameter aggregation method, and distributes the updated local model parameters to each edge computing device.

Images