Urban area traffic signal coordination optimization system based on deep reinforcement learning

Abstract

The application discloses a city area traffic signal cooperative optimization system based on deep reinforcement learning and belongs to the technical field of intelligent traffic Internet of Things application services.The application solves the problem of the real-time data collected by the existing system in the transmission and processing process, which faces the risk of eavesdropping, tampering, forgery or denial of service attack, leading to the problem of the error decision of the intelligent agent, through the calculation of the credit of the single-area intelligent agent, the quick response and isolation of the abnormal intelligent agent, the reliability and safety of the cooperative decision are ensured, the traffic flow in the future is predicted by using the traffic prediction model, the signal timing is dynamically adjusted to optimize the traffic flow, the local cooperative optimization is carried out for the congestion area, and the traffic congestion is quickly relieved, the serial number is bound for each control instruction, the instruction signature is generated, the instruction with the failed verification is refused to execute, and the safety of the instruction execution is ensured, and the instruction sending order is dynamically adjusted according to the real-time traffic flow and the signal machine state, and the efficiency of the instruction execution is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation Internet of Things (IoT) application service technology, specifically to an urban area traffic signal collaborative optimization system based on deep reinforcement learning. Background Technology

[0002] With the acceleration of urbanization, traffic congestion has become a global problem. Traditional timed signal control or adaptive control based on simple sensors is insufficient to cope with dynamic, high-dimensional, and nonlinear urban traffic flows. In recent years, deep reinforcement learning has been introduced into the field of traffic signal control, especially multi-agent deep reinforcement learning, which has shown great potential for optimizing regional traffic flow through collaborative decision-making among multiple intersection agents.

[0003] However, existing methods still have shortcomings when dealing with complex traffic scenarios. For example, the system is highly dependent on intelligent transportation IoT devices, such as cameras, radar, and vehicle-to-everything (V2X) terminals that collect real-time data. During the transmission, aggregation, and processing of this data, it is at risk of being eavesdropped on, tampered with, forged, or subjected to denial-of-service attacks. Malicious data injection can cause intelligent agents to make wrong decisions, leading to regional traffic chaos or even safety accidents.

[0004] Therefore, to meet existing needs, a collaborative optimization system for urban traffic signals based on deep reinforcement learning is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a collaborative optimization system for urban traffic signals based on deep reinforcement learning. This system calculates the reputation of agents in a single area, quickly responds to and isolates abnormal agents, ensuring the reliability and security of collaborative decision-making. It uses a traffic prediction model to predict future traffic flow and dynamically adjusts signal timing to optimize traffic flow. Furthermore, it performs local collaborative optimization for congested areas to quickly alleviate traffic congestion. Each control command is bound with a sequence number and a command signature is generated, rejecting commands that fail verification to ensure the security of command execution. Finally, it dynamically adjusts the command transmission order based on real-time traffic flow and signal status to improve command execution efficiency, thus solving the problems mentioned in the background section.

[0006] To achieve the above objectives, the present invention provides the following technical solution: A city area traffic signal collaborative optimization system based on deep reinforcement learning includes: The distributed sensing unit is configured to deploy a single-zone intelligent agent in each sub-region. Based on the authorization of the single-zone intelligent agent, it manages the intelligent transportation IoT devices in the jurisdiction. When using the intelligent transportation IoT devices to obtain real-time traffic data in the jurisdiction, it attaches a time-sensitive digital signature to verify that the data has not been tampered with during transmission. The collaborative optimization unit is configured to deploy collaborative intelligent agents in the cloud center, receive traffic status information uploaded by each single-zone intelligent agent, use a traffic prediction model to predict the traffic status information, and output signal timing schemes for each intersection or sub-area; and dynamically set reputation scores for each single-zone intelligent agent and collaborative intelligent agent as a quantitative indicator of their trustworthy behavior. The safety execution unit is configured to generate control commands based on the signal timing scheme, and assign command signatures and serial numbers to the control commands. Before the traffic signal controller executes the control commands, it verifies whether the command signature is valid and whether the serial number is fresh, determines whether the command has been hijacked, tampered with or maliciously replayed during transmission, and executes the verified control commands.

[0007] Furthermore, the collaborative optimization unit includes: The reputation assessment module is configured to calculate a reputation score based on the data quality, decision consistency, and contribution to global optimization provided by each single-zone agent. Reputation is used as a key factor in the calculation of attention weight. When a single-area agent needs to receive information from its neighbors to make decisions, the information weight of neighbors with high reputation is given higher weight, while the influence weight of neighbors with low reputation or whose reputation suddenly drops is reduced. A dynamic update mechanism is introduced to monitor the behavior and data performance of each agent in the single zone in real time, and to dynamically update the reputation based on the latest data quality, decision-making effect and global contribution. The system monitors changes in the reputation of agents in real time. If the reputation of any agent in a single zone suddenly drops sharply, a rapid response mechanism is triggered to isolate the agent in that zone and perform a detailed analysis of its behavior.

[0008] Furthermore, the collaborative optimization unit also includes: The traffic prediction module is configured to use a traffic prediction model to predict traffic status information. It is trained using historical traffic data to learn the spatiotemporal characteristics and changing patterns of traffic flow, predict future traffic flow trends, and dynamically adjust signal timing to optimize traffic flow and reduce congestion. Introducing meteorological data and event information enables the collaborative intelligent agent to adaptively adjust and optimize strategies based on real-time traffic conditions and environmental changes; The multi-level collaborative optimization module is configured to dynamically adjust the division of sub-regions based on real-time traffic conditions. In traffic-congested areas, it utilizes collaborative agents to conduct local collaboration with adjacent single-region agents to quickly adjust signal timing and alleviate congestion.

[0009] Furthermore, the collaborative optimization unit also includes: The data decryption module is configured to verify the legitimacy of the digital signature when the collaborative intelligent agent receives a data packet. The verification process includes: confirming that the data packet comes from a legitimate IoT device; and checking the timeliness of the data to ensure that the data was generated within a reasonable time frame. The data digest is recalculated using a hash algorithm and compared with the checksum in the digital signature to ensure that the data has not been tampered with. Use the public key to decrypt the digital fingerprint and verify the validity of the signature; During data transmission, sensitive data is encrypted to ensure that it is not stolen or tampered with during transmission; only authorized devices and users can access and process traffic data to prevent unauthorized access and data leakage.

[0010] Furthermore, the security execution unit includes: The instruction encryption module is configured to assign a unique sequence number to each control instruction based on the order in which the instructions are generated; hash the content of the control instruction to generate a fixed-length digest; and encrypt the hash digest using a private key to generate an instruction signature. The instruction verification module is configured to decrypt the instruction signature using the system's public key after the traffic signal controller receives the control instruction; check whether the instruction's sequence number is fresh and consecutive; check whether the instruction's timestamp is within a reasonable time range; if the instruction verification fails, the traffic signal controller will refuse to execute the instruction and trigger an exception handling mechanism to notify the system administrator for inspection and repair. The intelligent dispatch module is configured to dynamically adjust the order of instruction transmission based on traffic flow and signal execution status; and to dynamically optimize control instructions based on real-time traffic conditions and signal feedback information.

[0011] Furthermore, the distributed sensing unit includes: The data encryption module is configured to attach a time-sensitive digital signature to each data packet based on lightweight cryptography when the packet is generated; and to dynamically adjust the strength of the digital signature according to the importance of the data and the transmission environment. The data sharing module is configured to enable data sharing and collaborative control among the individual intelligent agents through a standardized communication protocol. Each individual intelligent agent sends traffic status information to neighboring individual intelligent agents and receives feedback data from neighboring individual intelligent agents.

[0012] Furthermore, the distributed sensing unit also includes: The adaptive optimization module is configured to automatically adjust the data collection strategy and anomaly detection algorithm based on historical data and current traffic conditions using machine learning algorithms; and adaptively adjust the sampling frequency of intelligent transportation IoT devices according to changes in traffic flow and environment. The adaptive optimization module is configured to monitor the operating status of intelligent transportation IoT devices in real time based on a single-zone intelligent agent, analyze the behavior patterns of intelligent transportation IoT devices, and optimize the configuration and parameter settings of intelligent transportation IoT devices using device behavior data.

[0013] Furthermore, the distributed sensing unit also includes: The data acquisition module is configured to acquire real-time traffic data within the jurisdiction using intelligent transportation IoT devices, including: capturing real-time images of vehicles through high-definition cameras for vehicle detection and traffic statistics; measuring vehicle speed and distance through radar sensors; communicating with vehicles through vehicle-to-everything (V2X) roadside units to obtain vehicle driving information; and detecting vehicle passage through geomagnetic coils. The data fusion module is configured to preprocess the collected multimodal traffic data, including data cleaning, normalization and standardization; and to fuse data from different devices to generate comprehensive traffic status information.

[0014] Furthermore, the data encryption module includes: The data encryption submodule is used to attach a time-sensitive digital signature to the data based on lightweight cryptography when each data packet is generated; The dynamic adjustment submodule is used for: Obtain a data importance score, which is determined based on a preset scoring rule for the impact scope of data leakage; Acquire real-time data of the transmission environment, including link encryption status, network latency fluctuation data, and the current reputation of a single-zone agent; Based on the real-time data of the transmission environment and the current reputation of the single-zone intelligent agent, the risk level of the transmission environment is determined by a preset risk assessment model. The signature strength level is calculated and output using a weighted algorithm. A multi-dimensional dynamic parameter pool is preset, which includes a basic layer, an environment layer, a device status layer, and a business layer; Based on the signature strength level, the corresponding parameters in the multi-dimensional dynamic parameter pool are selected, and dynamic encryption processing is performed on the selected parameters. Each parameter is independently subjected to cryptographic hash operation to obtain the encrypted parameters. A random sequence code is generated based on the signature strength level, and the encrypted parameters are rearranged according to the random sequence code; the random sequence code is used to restore the parameter order during subsequent signature verification. Preset encryption level configuration rules; Based on the encryption level configuration rules, basic encryption is performed on the encrypted parameters, and dynamic encryption is performed based on the signature strength level to generate the target encryption result; Assemble a signature structure, the signature structure including a signature header, a signature body, and a signature tail; A pre-defined hierarchical key system is established; keys for the corresponding levels are selected based on the strength levels. The keys at the corresponding level are rotated periodically based on the working key lifecycle, and the old keys are retained for a 2-minute buffer period; when the credit rating of a single agent in a single area is detected to have decreased by more than a preset threshold or the number of consecutive signature verification failures is ≥3, an event-driven rotation is triggered. The new key is synchronized to the terminal device through an encrypted channel. During the synchronization process, a two-signature verification is used: the new key is signed using the currently valid working key, and the terminal device updates its local key after verifying the signature.

[0015] Furthermore, the methods for constructing traffic prediction models include: Obtain the model training dataset; perform feature extraction and feature enhancement on the training dataset to obtain the enhanced feature matrix; Using intersections / sub-regions in the training dataset as nodes and road connectivity as edges, an initial topology is constructed, and a two-factor calculation of the spatial basis matrix is ​​introduced, which is the physical distance-flow correlation degree. The day is divided into 288 five-minute time slots. The feature vector of each time slot is learned through an embedding layer. Based on vehicle network event information, the event type, scope of impact and duration are encoded to obtain the event impact vector. The time slot feature vector and the event impact vector are fused through a gating mechanism to generate dynamic time weights. A topological memory module is constructed by introducing a long short-term memory unit. The historical adjacency matrix is ​​vectorized and input into an LSTM to store the adjacency matrix change patterns over nearly 10 time steps, and the memory weight matrix is ​​output. The spatial fundamental matrix, the temporal dynamic weight matrix, and the memory weight matrix are fused and normalized to generate a spatiotemporal dual-driven dynamic adjacency matrix. Based on a nested spatiotemporal deep coding network, feature mining is performed on the enhanced feature matrix and dynamic adjacency matrix to output encoded features; A Transformer decoder is used, with global features from the encoded features as input, to predict long-term traffic trends; a trend constraint loss is introduced to constrain the consistency between the predicted trend and the rate of change of the first derivative of traffic in the same period of history. Input long-term prediction results, mid-level features of encoded features, and historical enhancement features, and use gated recurrent units combined with graph attention mechanism to model the spatiotemporal smooth changes of mid-term traffic. Input the mid-term prediction results, fine-grained features of the encoded features, and historical multi-source data features. Use a lightweight gating fusion mechanism to dynamically adjust the feature weights. Input the weighted features into a multilayer perceptron and output fine-grained traffic prediction values. A gradient boosting tree model is trained using historical data to build a mapping relationship library between traffic density, vehicle type ratio, arrival rate, signal cycle, and green light ratio; for each intersection, the features corresponding to the short-term traffic forecast are input, and the initial signal timing scheme is output. Using intersections as nodes and road connectivity as edges, an intersection linkage graph is constructed, with the adjacency matrix weight set as the traffic flow impact coefficient between intersections. A graph attention network is used to input the initial timing scheme and short-term traffic flow forecast for each intersection, and output the linkage adjustment coefficient. The initial timing scheme and the linkage adjustment coefficient are fused to obtain the final timing scheme. The final timing scheme includes the cycle duration, the green light ratio of each phase, and the phase difference between adjacent intersections. The training dataset is input into the model in batches for training, resulting in a traffic prediction model.

[0016] Compared with the prior art, the beneficial effects of the present invention are: 1. In this invention, by calculating the reputation of each agent in a single zone, higher reputation neighbors are assigned higher information weights, while the influence weight of low reputation neighbors or those with plummeting reputation is reduced. Furthermore, agent behavior is monitored in real time, and abnormal agents are quickly responded to and isolated, ensuring the reliability and security of collaborative decision-making. Simultaneously, a traffic prediction model is used to predict future traffic flow and dynamically adjust signal timing to optimize traffic flow and reduce congestion. Local collaborative optimization is performed on congested areas to quickly alleviate traffic congestion and effectively improve the level of intelligence in traffic management.

[0017] 2. In this invention, by binding a unique serial number to each control command and generating a command signature, the legality of the command is verified at the traffic signal controller. Commands that fail verification are rejected and an exception handling mechanism is triggered, ensuring the security of command execution. Furthermore, the order of command transmission and the optimization of control commands are dynamically adjusted based on real-time traffic flow and signal controller status, improving the efficiency and adaptability of command execution. This significantly enhances the security, reliability, and execution efficiency of traffic signal control commands, effectively preventing commands from being tampered with, forged, or replay attacks. At the same time, it improves the level of intelligent transportation IoT application services and the ability to cope with complex traffic conditions, providing strong support for the smooth and safe operation of urban traffic. Attached Figure Description

[0018] Figure 1 This is a flowchart of the urban area traffic signal collaborative optimization system based on deep reinforcement learning according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] To address the technical challenges of existing systems heavily reliant on real-time data collected by intelligent transportation IoT devices, which faces risks of eavesdropping, tampering, forgery, or denial-of-service attacks during data transmission, aggregation, and processing, and where malicious data injection can lead to erroneous decisions by intelligent agents, please refer to [link to relevant documentation]. Figure 1 This embodiment provides the following technical solution: A city area traffic signal collaborative optimization system based on deep reinforcement learning includes: A distributed sensing unit is configured to deploy a single-zone intelligent agent in each sub-region. Based on the authorization of the single-zone intelligent agent, it manages the intelligent transportation IoT devices within its jurisdiction. For example, the urban road network is divided into multiple sub-regions according to traffic flow, road function, and geographical features. Each sub-region deploys a single-zone intelligent agent responsible for managing the intelligent transportation IoT devices within its jurisdiction. When acquiring real-time traffic data within the jurisdiction using intelligent transportation IoT devices, a time-sensitive digital signature is attached to verify that the data has not been tampered with during transmission. The distributed sensing unit includes: The data encryption module is configured to attach a time-sensitive digital signature to each data packet based on lightweight cryptography when it is generated. The digital signature includes: device identification, timestamp, data integrity check code, and digital fingerprint, ensuring that the data comes from a legitimate device and has not been tampered with. The strength of the digital signature is dynamically adjusted according to the importance of the data and the transmission environment. For example, the complexity and security of the signature are increased in high-risk areas or when transmitting sensitive data.

[0021] The data sharing module is configured to enable data sharing and collaborative control among the individual agents in each zone through a standardized communication protocol. Each individual agent in a zone sends traffic status information to neighboring individual agents in a zone and receives feedback data from neighboring individual agents in a zone, ensuring real-time sharing of traffic information within the zone and providing a foundation for collaborative optimization.

[0022] The adaptive optimization module is configured to automatically adjust data collection strategies and anomaly detection algorithms based on historical data and current traffic conditions using machine learning algorithms. For example, through machine learning algorithms, the agent can automatically identify new traffic patterns and abnormal features, improving the accuracy and efficiency of anomaly detection. It also adaptively adjusts the sampling frequency of intelligent transportation IoT devices according to changes in traffic flow and the environment. For instance, during peak traffic hours, the data collection frequency is increased to obtain more detailed traffic information; during off-peak hours, the sampling frequency is reduced to save equipment resources. In rainy or foggy weather, the agent in a single zone increases the sampling frequency of radar sensors within its jurisdiction, reducing reliance on camera image data to cope with the impact of severe weather on data collection. During road construction, the agent in a single zone adjusts the signal timing within its jurisdiction, prioritizing traffic flow around the construction area to reduce the impact of construction on traffic.

[0023] The adaptive optimization module is configured to monitor the operational status of intelligent transportation IoT devices in real time based on a single-zone intelligent agent, and analyze the behavioral patterns of these devices. For example, by analyzing the fault history and operational data of the intelligent transportation IoT devices, it can predict potential faults and perform maintenance in advance, reducing the impact of device faults on traffic data collection and improving system reliability. Furthermore, it utilizes device behavior data to optimize the configuration and parameter settings of the intelligent transportation IoT devices. For instance, based on device performance and traffic demand, it can dynamically adjust the camera's shooting angle and the radar sensor's detection range to improve the efficiency and accuracy of data collection.

[0024] The data acquisition module is configured to acquire real-time traffic data within the jurisdiction using intelligent transportation IoT devices, including: capturing real-time images of vehicles through high-definition cameras for vehicle detection and traffic flow statistics; measuring vehicle speed and distance through radar sensors; communicating with vehicles through vehicle-to-everything (V2X) roadside units to obtain vehicle driving information; and detecting vehicle passage through geomagnetic coils to provide basic data support for traffic signal optimization.

[0025] The data fusion module is configured to preprocess the collected multi-modal traffic data, including data cleaning, normalization, and standardization. For example, it converts vehicle speed data into a unified unit, smooths traffic flow data, and removes noise and outliers. The preprocessed data will be uploaded to the collaborative intelligent agent for global optimization decisions. It also fuses data from different devices to generate comprehensive traffic status information. For example, by combining image data from high-definition cameras and speed data from radar sensors, it generates more accurate traffic flow and speed information, improving data accuracy and reliability and providing a more comprehensive basis for subsequent optimization decisions.

[0026] The collaborative optimization unit is configured to deploy collaborative agents in the cloud center, receiving traffic status information uploaded by agents in each individual zone, including the traffic status of itself and adjacent areas. This information covers key indicators such as traffic flow, vehicle speed, queue length, and vehicle type, providing comprehensive data support for global optimization. It uses a traffic prediction model to predict traffic status information and outputs signal timing schemes for each intersection or sub-area. It also dynamically sets a reputation score for each agent in each individual zone and collaborative agent, serving as a quantitative indicator of their trustworthy behavior. The collaborative optimization unit includes: The reputation assessment module is configured to calculate a reputation score based on the data quality, decision consistency, and contribution to global optimization provided by each single-zone agent. Agents with high data quality and decisions consistent with the global optimization objective will receive higher reputation scores. Reputation score is used as a key factor in the calculation of attention weights; neighbors with high reputation scores have greater influence in collaborative decision-making, and their data and decision suggestions will be given priority. When a single-zone agent needs to receive information from neighbors for decision-making, higher-reputation neighbors are assigned higher information weights, while the influence weight of neighbors with low reputation scores or sudden drops in reputation scores is reduced. This can effectively isolate potential anomalies. To prevent malicious agents from negatively impacting global optimization, a dynamic update mechanism is introduced to monitor the behavior and data performance of agents in each zone in real time, dynamically updating their reputation based on the latest data quality, decision-making effectiveness, and global contribution. For example, if an agent performs well in multiple consecutive decisions, its reputation will gradually increase; conversely, if abnormal behavior occurs, its reputation will rapidly decline. Real-time monitoring of agent reputation changes is crucial; if the reputation of any agent in a zone suddenly plummets, a rapid response mechanism is triggered to isolate the agent and conduct a detailed analysis of its behavior, enabling timely detection and handling of potential security threats.

[0027] The traffic prediction module is configured to use a traffic prediction model to predict traffic status information. It is trained using historical traffic data to learn the spatiotemporal characteristics and changing patterns of traffic flow, predict future traffic flow trends, and dynamically adjust signal timing to optimize traffic flow and reduce congestion. It also incorporates meteorological data and event information, such as traffic accidents and road construction, enabling the collaborative intelligent agent to adaptively adjust optimization strategies based on real-time traffic conditions and environmental changes. For example, when a sudden traffic event is detected, the collaborative intelligent agent can quickly adjust signal timing to prioritize the diversion of traffic flow in the affected area.

[0028] The multi-level collaborative optimization module is configured to dynamically adjust the division of sub-regions based on real-time traffic conditions. For example, during peak traffic hours, congested areas are divided into smaller sub-regions for refined signal optimization. In congested areas, collaborative agents work with adjacent single-region agents to quickly adjust signal timing, alleviating congestion and effectively improving the system's response speed and flexibility. A real-time communication mechanism is established so that when an unreasonable signal timing is detected in a sub-region, the collaborative agent can immediately send adjustment suggestions to the single-region agents in that region. When any single-region agent successfully solves a traffic problem, its optimization strategy can be shared with adjacent single-region agents, improving the overall optimization level of the system.

[0029] The data decryption module is configured to verify the legitimacy of the digital signature when the collaborative agent receives a data packet. The verification process includes: confirming that the data packet comes from a legitimate IoT device; checking the timeliness of the data to ensure that the data was generated within a reasonable timeframe; recalculating the data digest using a hash algorithm and comparing it with the checksum in the digital signature to ensure that the data has not been tampered with; decrypting the digital fingerprint using a public key to verify the validity of the signature; encrypting sensitive data during data transmission to ensure that the data is not stolen or tampered with during transmission; allowing only authorized devices and users to access and process traffic data to prevent unauthorized access and data leakage; and monitoring abnormal situations during data transmission in real time. If a data signature verification failure or data integrity breach is detected, an alarm mechanism will be immediately triggered, and the system administrator and relevant maintenance personnel will be notified to take timely measures.

[0030] The beneficial effects achieved by the above are as follows: by calculating the reputation of each agent in a single area, higher reputation neighbors are assigned higher information weights, while the influence weight of low reputation neighbors or those with plummeting reputation is reduced; agent behavior is monitored in real time, and abnormal agents are quickly responded to and isolated, ensuring the reliability and security of collaborative decision-making; at the same time, traffic prediction models are used to predict future traffic flow and dynamically adjust signal timing to optimize traffic flow and reduce congestion; local collaborative optimization is carried out for congested areas to quickly alleviate traffic congestion and effectively improve the level of intelligence in traffic management.

[0031] The safety execution unit is configured to generate control commands based on the signal timing scheme. These commands include detailed parameters such as the phase duration of each traffic light, green light interval, and yellow light duration, ensuring that the traffic signal controller can accurately execute the optimized scheme. The unit assigns a command signature and binds a serial number to each control command. Before the traffic signal controller executes the control command, it verifies the validity of the command signature and the freshness of the serial number to prevent the command from being hijacked, tampered with, or maliciously replayed during transmission, thus preventing replay attacks. The unit then executes the verified control commands. The safety execution unit includes: The instruction encryption module is configured to assign a unique sequence number to each control instruction based on the order in which the instructions are generated; hash the content of the control instruction to generate a fixed-length digest; and encrypt the hash digest using a private key to generate an instruction signature.

[0032] The instruction verification module is configured to, after the traffic signal controller receives a control instruction, use the system's public key to decrypt the instruction signature, verifying the integrity and authenticity of the instruction; check whether the instruction's sequence number is fresh and continuous to prevent tampering; check whether the instruction's timestamp is within a reasonable time range to ensure the instruction's timeliness; if the instruction verification fails, such as due to an invalid signature, discontinuous sequence number, or expired timestamp, the traffic signal controller will refuse to execute the instruction and trigger an exception handling mechanism to notify the system administrator for inspection and repair.

[0033] The intelligent scheduling module is configured to dynamically adjust the order of instruction transmission based on traffic flow and signal controller execution status. For example, during peak traffic hours, it prioritizes sending signal timing instructions for key intersections. It also dynamically optimizes control instructions based on real-time traffic conditions and signal controller feedback. For instance, if the execution time of a signal controller is delayed, the system will automatically adjust the transmission time of subsequent instructions.

[0034] The beneficial effects achieved by the above are as follows: By binding a unique serial number to each control command and generating a command signature, the legality of the command is verified at the traffic signal controller, and commands that fail verification are rejected and exception handling mechanisms are triggered, ensuring the security of command execution; and the command sending order is dynamically adjusted and control commands are optimized according to real-time traffic flow and signal controller status, improving the efficiency and adaptability of command execution; the security, reliability, and execution efficiency of traffic signal control commands are significantly enhanced, effectively preventing commands from being tampered with, forged, or replay attacks, while improving the level of intelligent transportation IoT application services and the ability to cope with complex traffic conditions, providing strong support for the smooth and safe operation of urban traffic.

[0035] Working principle: By deploying single-zone intelligent agents in each sub-region, intelligent transportation IoT devices are managed, real-time traffic data is collected, and digital signatures are added to ensure data transmission security; collaborative intelligent agents are deployed in the cloud center to receive and process traffic data, predict traffic flow using traffic prediction models, dynamically adjust signal timing, and dynamically adjust the reputation of each intelligent agent through a reputation assessment module to optimize collaborative decision-making, generate and verify control commands, and ensure the secure execution of commands; the command sending order is dynamically adjusted and control commands are optimized based on real-time traffic conditions, achieving efficient, safe, and intelligent optimization of traffic signals, and improving the level of intelligence in urban traffic management.

[0036] The data encryption module includes: The data encryption submodule is used to attach a time-sensitive digital signature to the data based on lightweight cryptography when each data packet is generated; The dynamic adjustment submodule is used for: Obtain a data importance score, which is determined based on a preset scoring rule for the impact scope of data leakage; Acquire real-time data of the transmission environment, including link encryption status, network latency fluctuation data, and the current reputation of a single-zone agent; Based on the real-time data of the transmission environment and the current reputation of the single-zone intelligent agent, the risk level of the transmission environment is determined by a preset risk assessment model. The signature strength level is calculated and output using a weighted algorithm. A multi-dimensional dynamic parameter pool is preset, which includes a basic layer, an environment layer, a device status layer, and a business layer; Based on the signature strength level, the corresponding parameters in the multi-dimensional dynamic parameter pool are selected, and dynamic encryption processing is performed on the selected parameters. Each parameter is independently subjected to cryptographic hash operation to obtain the encrypted parameters. A random sequence code is generated based on the signature strength level, and the encrypted parameters are rearranged according to the random sequence code; the random sequence code is used to restore the parameter order during subsequent signature verification. Preset encryption level configuration rules; Based on the encryption level configuration rules, basic encryption is performed on the encrypted parameters, and dynamic encryption is performed based on the signature strength level to generate the target encryption result; Assemble a signature structure, the signature structure including a signature header, a signature body, and a signature tail; A pre-defined hierarchical key system is established; keys for the corresponding levels are selected based on the strength levels. The keys at the corresponding level are rotated periodically based on the working key lifecycle, and the old keys are retained for a 2-minute buffer period; when the credit rating of a single agent in a single area is detected to have decreased by more than a preset threshold or the number of consecutive signature verification failures is ≥3, an event-driven rotation is triggered. The new key is synchronized to the terminal device through an encrypted channel. During the synchronization process, a two-signature verification is used: the new key is signed using the currently valid working key, and the terminal device updates its local key after verifying the signature.

[0037] In this embodiment, the multi-dimensional dynamic parameter pool includes: a base layer containing device ID hash value, business scenario identifier, and timestamp.

[0038] In this embodiment, the environment layer includes the TLS version of the transmission link, network latency fluctuation value, and the current battery level of the device.

[0039] In this embodiment, the device status layer includes device operating health, historical communication success rate, and current reputation of a single-zone intelligent agent.

[0040] In this embodiment, the business layer includes a data integrity check code, a business operation permission identifier, and a data timeliness threshold.

[0041] In this embodiment, the signature strength level rules are as follows: Level 1 selects only the basic layer parameters; Level 2 selects the basic layer and environment layer parameters; Level 3 selects the basic layer, environment layer, and device state layer parameters; Level 4 selects the basic layer, environment layer, device state layer, and service layer parameters; Level 5 selects all parameters.

[0042] In this embodiment, the preset encryption level configuration rules are as follows: Level 1 is configured with SM2 asymmetric encryption; Level 2 is configured with SM2 asymmetric encryption and SM4 parameter-level symmetric encryption; Level 3 is configured with SM2 asymmetric encryption, SM4 parameter-level symmetric encryption and SHA-256 hash; Level 4 is configured with SM2 asymmetric encryption, SM4 overall-level symmetric encryption and SHA-256 hash; Level 5 is configured with SM2 asymmetric encryption, SM4 overall-level symmetric encryption, SHA-256 hash and timestamp encryption.

[0043] In this embodiment, the assembled signature structure includes: a signature header, a signature body, and a signature tail.

[0044] In this embodiment, the signature header includes a 1-byte strength level identifier, a 4-byte parameter sequence code, and a 2-byte key version.

[0045] In this embodiment, the signature body includes a concatenation of basic encryption results and dynamic encryption results in that order, with low-strength levels having no dynamic encryption field.

[0046] In this embodiment, the signature tail includes an 8-byte signature generation timestamp and a 4-byte anti-tampering verification code; wherein, the anti-tampering verification code is generated by performing SM3 hash on the signature body and is used to verify the legality of the digital signature when the collaborative intelligent agent receives the data packet.

[0047] In this embodiment, the hierarchical key system includes a root key, a master key, and working keys, with the working keys including ordinary working keys and high-strength working keys.

[0048] In this embodiment, Level 1-Level 3 are ordinary working keys derived from the master key; Level 4-Level 5 are high-strength working keys derived from the master key.

[0049] The working principle and beneficial effects of the above technical solution are as follows: Lightweight cryptography is used to attach a time-sensitive digital signature to data, effectively preventing data tampering and forgery, and ensuring the integrity and authenticity of data during transmission; by obtaining data importance scores and real-time transmission environment data, the risk level of the transmission environment is determined and the signature strength level is calculated, allowing for dynamic adjustment of encryption strategies according to different situations, improving the targeting and effectiveness of encryption; a multi-dimensional dynamic parameter pool is used to dynamically encrypt and rearrange parameters, increasing the complexity of encryption and the difficulty of cracking; basic encryption and dynamic encryption are performed based on encryption level configuration rules, providing different levels of encryption protection according to different signature strength levels; a hierarchical key system is adopted, with key rotation at regular intervals and a buffer period set. Event-driven rotation is triggered when credibility decreases or verification fails, and new keys are synchronized through an encryption channel for secondary signature verification, ensuring key security and reliable updates.

[0050] Methods for constructing traffic prediction models include: Obtain the model training dataset; perform feature extraction and feature enhancement on the training dataset to obtain the enhanced feature matrix; Using intersections / sub-regions in the training dataset as nodes and road connectivity as edges, an initial topology is constructed, and a two-factor calculation of the spatial basis matrix is ​​introduced, which is the physical distance-flow correlation degree. The day is divided into 288 five-minute time slots. The feature vector of each time slot is learned through an embedding layer. Based on vehicle network event information, the event type, scope of impact and duration are encoded to obtain the event impact vector. The time slot feature vector and the event impact vector are fused through a gating mechanism to generate dynamic time weights. A topological memory module is constructed by introducing a long short-term memory unit. The historical adjacency matrix is ​​vectorized and input into an LSTM to store the adjacency matrix change patterns over nearly 10 time steps, and the memory weight matrix is ​​output. The spatial fundamental matrix, the temporal dynamic weight matrix, and the memory weight matrix are fused and normalized to generate a spatiotemporal dual-driven dynamic adjacency matrix. Based on a nested spatiotemporal deep coding network, feature mining is performed on the enhanced feature matrix and dynamic adjacency matrix to output encoded features; A Transformer decoder is used, with global features from the encoded features as input, to predict long-term traffic trends; a trend constraint loss is introduced to constrain the consistency between the predicted trend and the rate of change of the first derivative of traffic in the same period of history. Input long-term prediction results, mid-level features of encoded features, and historical enhancement features, and use gated recurrent units combined with graph attention mechanism to model the spatiotemporal smooth changes of mid-term traffic. Input the mid-term prediction results, fine-grained features of the encoded features, and historical multi-source data features. Use a lightweight gating fusion mechanism to dynamically adjust the feature weights. Input the weighted features into a multilayer perceptron and output fine-grained traffic prediction values. A gradient boosting tree model is trained using historical data to build a mapping relationship library between traffic density, vehicle type ratio, arrival rate, signal cycle, and green light ratio; for each intersection, the features corresponding to the short-term traffic forecast are input, and the initial signal timing scheme is output. Using intersections as nodes and road connectivity as edges, an intersection linkage graph is constructed, with the adjacency matrix weight set as the traffic flow impact coefficient between intersections. A graph attention network is used to input the initial timing scheme and short-term traffic flow forecast for each intersection, and output the linkage adjustment coefficient. The initial timing scheme and the linkage adjustment coefficient are fused to obtain the final timing scheme. The final timing scheme includes the cycle duration, the green light ratio of each phase, and the phase difference between adjacent intersections. The training dataset is input into the model in batches for training, resulting in a traffic prediction model.

[0051] In this embodiment, a modal feature mapping layer is constructed based on a fully connected network to uniformly map each modal feature in the training dataset to 256 dimensions, resulting in a single modal feature vector. A Gaussian process model is introduced to dynamically calculate the reliability weight of each modality based on the correlation coefficient between each modal feature and the actual traffic flow in historical data. A two-level attention fusion mechanism is constructed to enhance key features of the single modal feature vector based on intra-modal attention, and to capture complementary information of different modal features through inter-modal attention, and to output an enhanced feature matrix by combining modal weights.

[0052] In this embodiment, ;in, This represents the enhanced feature of the i-th region at time t; For scaling dot product attention functions, This is a concatenated vector of all modal features; m is the modality index. For modal reliability weights; Let be the single-mode eigenvector of mode m at time t in region i.

[0053] In this embodiment, the two-level attention fusion mechanism includes intra-modal attention and inter-modal attention mechanisms: Intra-modal attention: Intermodal attention: The output within a modality is taken as the query, and the concatenated features of all modalities are taken as the key / value; Q, K, and V are the query, key, and value vectors of the attention mechanism, respectively. is the learnable parameter matrix; d is the feature dimension scaling factor; T represents the transpose.

[0054] In this embodiment, the physical distance is the shortest path distance of the road network; the traffic correlation degree is determined based on the absolute Pearson correlation coefficient of historical traffic during the same period, with a value range of [0,1].

[0055] In this embodiment, the attenuation coefficient is dynamically adjusted based on the fusion traffic correlation, and the spatial fundamental matrix is: Where a is the adaptive distance parameter; The standard deviation of traffic correlation; The shortest path distance in the road network; Traffic relevance; These are the elements of the spatial fundamental matrix.

[0056] In this embodiment, the time-dynamic weight matrix is: ;in, The weight between nodes i and j; t is the index of the current time period; These are bias parameters; This represents the feature vector for a given time period. Let i be the event influence vector of node i; This indicates vector concatenation; This is a learnable weight matrix.

[0057] In this embodiment, the spatiotemporal dual-drive dynamic adjacency matrix is ​​as follows: It is a spatiotemporal dual-driven dynamic adjacency matrix; For memorizing the weight matrix; It is the fundamental matrix of space; This is a time-dynamic weighting matrix; For Hadamard product; Norm is the normalization operation.

[0058] In this embodiment, a three-level nested coding unit is constructed, and six layers are stacked to form a coding network. Each coding unit includes a multi-scale gated spatiotemporal convolution submodule, a hierarchical attention submodule, and an adaptive residual fusion submodule in sequence, and the output of the previous level coding unit is used as the input of the next level coding unit.

[0059] In this embodiment, the multi-scale gated spatiotemporal convolution submodule fuses temporal convolution and graph convolution to capture spatiotemporal local features; and introduces update gate and reset gate to dynamically adjust the spatiotemporal feature fusion ratio. ; , These are graph convolution and temporal convolution operations, respectively. To update the door; To reset the door; This is the gate weight matrix; This is the gated bias vector; This represents a historical hidden state at the same scale. Output for submodules.

[0060] In this embodiment, the hierarchical attention submodule: calculates the feature correlation between the current time step and the last 5 time steps through local temporal attention; calculates the feature correlation between the current region and all regions through global spatial attention based on the dynamic adjacency matrix; and introduces an attention distillation mechanism, using the output of the previous level encoding unit as a reference to guide the learning of features at the current level. The vector obtained by projecting the high-level encoded features according to the dimensions; The projection matrix; This is the matrix for the current layer; These are high-level encoding features; d is a scaling factor. To standardize the process; This is the output for global spatial attention.

[0061] In this embodiment, the adaptive residual fusion submodule: designs multi-path residual connections, which respectively connect the original enhanced features, the output of the convolutional submodule, and the output of the attention submodule, using dynamically learnable parameters. Weighted fusion; employing a hybrid normalization method combining batch normalization and layer normalization: Dynamically select the normalization method based on the feature distribution; This is the final output of the submodule; These are dynamically learnable parameters; F represents the original enhancement feature.

[0062] In this embodiment, the lightweight gating fusion mechanism dynamically adjusts the feature weights: ;in, For medium-term prediction features, It is a fine-grained feature. Historical characteristics; G is the gate weight matrix; G is the gate vector; For gated bias; This is a feature of fusion.

[0063] In this embodiment, a mapping relationship model between "traffic density, vehicle type ratio, and arrival rate" and "signal cycle and green light ratio" is established by training with historical data, and a gradient boosting tree is adopted; for each intersection, short-term predicted traffic characteristics are input and an initial signal timing scheme is output.

[0064] In this embodiment, offline end-to-end training: the loss function is defined as: ;in, As task weight, ; It is a first-order difference operator. This is a historical traffic sequence for the same period. For long-term forecasting of flow sequences; Internal weights for predicting loss; Mean absolute error; Mean absolute percentage error; For trend constraint loss; This represents the average delay after the timing plan is implemented. Internal weights for signal loss; Predict traffic flow after the timing scheme is implemented; This refers to the actual traffic flow after the timing scheme is implemented; In this embodiment, a two-stage optimization strategy is adopted: Pre-training stage: Freeze the timing generation module and optimize only... Joint fine-tuning phase: Unfreeze all parameters, initialize with pre-trained weights, and optimize the entire process. The timing module gradient passes through The data is then sent back to the prediction module.

[0065] In this embodiment, the topological memory module: stores the adjacency matrix Flattened into vectors by row ,enter ; Output memory weight vector After being reshaped and Activation generates memory weight matrix .

[0066] In this embodiment, the event impact vector encoding is as follows: the event type is encoded using an embedding layer; the impact range is mapped to a binary vector; and the duration is normalized. Scalar; the three components are concatenated and then generated through a fully connected layer. .

[0067] In this embodiment, the training-inference distinction mechanism is as follows: During the training phase, the mid-to-short-term prediction module receives real historical data, and the feedback path... Inference phase: The mid-term / short-term prediction module receives the model's predicted values, and the feedback path is fully activated.

[0068] In this embodiment, the traffic efficiency index The average delay at intersections is calculated based on historical timing logs, with data sources including traffic simulation platforms or geomagnetic coil detectors.

[0069] In this embodiment, the spatiotemporal smoothing variation of the mid-term flow is modeled using the following method during the training phase: The mechanism involves feeding back the mid-term prediction results to the long-term prediction decoder during the inference phase, dynamically adjusting the long-term trend deviation.

[0070] In this embodiment, fine-grained flow prediction values ​​are output and used during the training phase. The strategy and reasoning phase feeds back the short-term forecast results to the medium-term forecast module to correct for medium-term forecast bias.

[0071] The working principle and beneficial effects of the above technical solution are as follows: Feature extraction and enhancement of the training dataset, combined with information from multiple aspects such as time-period features and event impacts, can more comprehensively reflect traffic conditions, providing a rich data foundation for accurate prediction; by constructing a topological structure, calculating the spatial basis matrix, and fusing temporal dynamic weights and memory weights, a spatiotemporal dual-driven dynamic adjacency matrix is ​​generated, effectively capturing the spatiotemporal characteristics of traffic data and improving the model's ability to perceive traffic changes; a nested spatiotemporal deep coding network is used to mine features, utilizing... The decoder predicts long-term trends, combines gated recurrent units and graph attention mechanisms to model mid-term changes, and a lightweight gated fusion mechanism outputs fine-grained predictions, achieving accurate traffic prediction in stages and at multiple levels. A nested spatiotemporal deep coding network is used to mine features, utilizing... The decoder predicts long-term trends, combines gated recurrent units and graph attention mechanisms to model mid-term changes, and uses a lightweight gated fusion mechanism to output fine-grained predictions, achieving accurate traffic flow prediction in stages and at multiple levels. The training dataset is input into the model in batches for training, enabling the model to fully learn data features and improve the accuracy and reliability of the traffic prediction model.

[0072] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or high-voltage switchgear that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or high-voltage switchgear.

[0073] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A collaborative optimization system for urban traffic signals based on deep reinforcement learning, characterized in that, include: The distributed sensing unit is configured to deploy a single-zone intelligent agent in each sub-region. Based on the authorization of the single-zone intelligent agent, it manages the intelligent transportation IoT devices in the jurisdiction. When using the intelligent transportation IoT devices to obtain real-time traffic data in the jurisdiction, it attaches a time-sensitive digital signature to verify that the data has not been tampered with during transmission. The collaborative optimization unit is configured to deploy collaborative intelligent agents in the cloud center, receive traffic status information uploaded by each single-zone intelligent agent, use a traffic prediction model to predict the traffic status information, and output signal timing schemes for each intersection or sub-area; and dynamically set reputation scores for each single-zone intelligent agent and collaborative intelligent agent as a quantitative indicator of their trustworthy behavior. The safety execution unit is configured to generate control commands based on a signal timing scheme, and to assign command signatures and sequence numbers to the control commands. Before a traffic signal controller executes a control command, it verifies whether the command signature is valid and whether the serial number is fresh to determine if the command has been hijacked, tampered with, or maliciously replayed during transmission, and then executes the verified control command. The distributed sensing unit includes: The data encryption module is configured to attach a time-sensitive digital signature to each data packet based on lightweight cryptography when the packet is generated; and to dynamically adjust the strength of the digital signature according to the importance of the data and the transmission environment. The data sharing module is configured to enable data sharing and collaborative control between individual intelligent agents through a standardized communication protocol. Each individual intelligent agent sends traffic status information to neighboring individual intelligent agents and receives feedback data from neighboring individual intelligent agents. The data encryption module includes: The data encryption submodule is used to attach a time-sensitive digital signature to the data based on lightweight cryptography when each data packet is generated; A dynamic adjustment submodule is used to obtain a data importance score, which is determined based on a preset scoring rule for the impact range of data leakage. Acquire real-time data of the transmission environment, including link encryption status, network latency fluctuation data, and the current reputation of the agent in a single zone; based on the real-time data of the transmission environment and the current reputation of the agent in a single zone, determine the risk level of the transmission environment through a preset risk assessment model; and calculate and output the signature strength level through a weighted algorithm. A multi-dimensional dynamic parameter pool is preset, which includes a basic layer, an environment layer, a device status layer, and a business layer; Based on the signature strength level, parameters corresponding to the multi-dimensional dynamic parameter pool are selected, and dynamic encryption processing is performed on the selected parameters. Each parameter is independently subjected to a cryptographic hash operation to obtain the encrypted parameter. A random sequence code is generated based on the signature strength level, and the encrypted parameters are rearranged according to the random sequence code. The random sequence code is used to restore the parameter order during subsequent signature verification. A preset encryption level configuration rule is established; based on the encryption level configuration rule, basic encryption is performed on the encrypted parameters, and dynamic encryption is performed based on the signature strength level to generate the target encryption result; Assemble a signature structure, which includes a signature header, a signature body, and a signature footer; pre-set a hierarchical key system; and select a key of the corresponding level according to the strength level. The keys at the corresponding level are rotated periodically based on the working key lifecycle, and the old keys are retained for a 2-minute buffer period; when the credit rating of a single agent in a single area is detected to have decreased by more than a preset threshold or the number of consecutive signature verification failures is ≥3, an event-driven rotation is triggered. The new key is synchronized to the terminal device through an encrypted channel. During the synchronization process, a two-signature verification is used: the new key is signed using the currently valid working key, and the terminal device updates its local key after verifying the signature.

2. The urban area traffic signal collaborative optimization system based on deep reinforcement learning according to claim 1, characterized in that, The collaborative optimization unit includes: The reputation assessment module is configured to calculate a reputation score based on the data quality, decision consistency, and contribution to global optimization provided by each single-zone agent. Reputation is used as a key factor in the calculation of attention weight. When a single-area agent needs to receive information from its neighbors to make decisions, the information weight of neighbors with high reputation is given higher weight, while the influence weight of neighbors with low reputation or whose reputation suddenly drops is reduced. A dynamic update mechanism is introduced to monitor the behavior and data performance of each agent in the single zone in real time, and to dynamically update the reputation based on the latest data quality, decision-making effect and global contribution. The system monitors changes in the reputation of agents in real time. If the reputation of any agent in a single zone suddenly drops sharply, a rapid response mechanism is triggered to isolate the agent in that zone and perform a detailed analysis of its behavior.

3. The urban area traffic signal collaborative optimization system based on deep reinforcement learning according to claim 2, characterized in that, The collaborative optimization unit further includes: The traffic prediction module is configured to use a traffic prediction model to predict traffic status information. It is trained using historical traffic data to learn the spatiotemporal characteristics and changing patterns of traffic flow, predict future traffic flow trends, and dynamically adjust signal timing to optimize traffic flow and reduce congestion. Introducing meteorological data and event information enables the collaborative intelligent agent to adaptively adjust and optimize strategies based on real-time traffic conditions and environmental changes; The multi-level collaborative optimization module is configured to dynamically adjust the division of sub-regions based on real-time traffic conditions. In traffic-congested areas, it utilizes collaborative agents to conduct local collaboration with adjacent single-region agents to quickly adjust signal timing and alleviate congestion.

4. The urban area traffic signal cooperative optimization system based on deep reinforcement learning according to claim 3, characterized in that, The collaborative optimization unit further includes: The data decryption module is configured to verify the legitimacy of the digital signature when the collaborative intelligent agent receives a data packet. The verification process includes: confirming that the data packet comes from a legitimate IoT device; and checking the timeliness of the data to ensure that the data was generated within a reasonable time frame. The data digest is recalculated using a hash algorithm and compared with the checksum in the digital signature to ensure that the data has not been tampered with. Use the public key to decrypt the digital fingerprint and verify the validity of the signature; During data transmission, sensitive data is encrypted to ensure that it is not stolen or tampered with during transmission; only authorized devices and users can access and process traffic data to prevent unauthorized access and data leakage.

5. The urban area traffic signal collaborative optimization system based on deep reinforcement learning according to claim 1, characterized in that, The security execution unit includes: The instruction encryption module is configured to assign a unique sequence number to each control instruction based on the order in which the instructions are generated; hash the content of the control instruction to generate a fixed-length digest; and encrypt the hash digest using a private key to generate an instruction signature. The instruction verification module is configured to decrypt the instruction signature using the system's public key after the traffic signal controller receives the control instruction; check whether the instruction's sequence number is fresh and consecutive; check whether the instruction's timestamp is within a reasonable time range; if the instruction verification fails, the traffic signal controller will refuse to execute the instruction and trigger an exception handling mechanism to notify the system administrator for inspection and repair. The intelligent dispatch module is configured to dynamically adjust the order of instruction transmission based on traffic flow and signal execution status; and to dynamically optimize control instructions based on real-time traffic conditions and signal feedback information.

6. The urban area traffic signal cooperative optimization system based on deep reinforcement learning according to claim 1, characterized in that, The distributed sensing unit further includes: The adaptive optimization module is configured to automatically adjust the data collection strategy and anomaly detection algorithm based on historical data and current traffic conditions using machine learning algorithms; and adaptively adjust the sampling frequency of intelligent transportation IoT devices according to changes in traffic flow and environment. The adaptive optimization module is configured to monitor the operating status of intelligent transportation IoT devices in real time based on a single-zone intelligent agent, analyze the behavior patterns of intelligent transportation IoT devices, and optimize the configuration and parameter settings of intelligent transportation IoT devices using device behavior data.

7. The urban area traffic signal cooperative optimization system based on deep reinforcement learning according to claim 6, characterized in that, The distributed sensing unit further includes: The data acquisition module is configured to acquire real-time traffic data within the jurisdiction using intelligent transportation IoT devices, including: capturing real-time images of vehicles through high-definition cameras for vehicle detection and traffic statistics; measuring vehicle speed and distance through radar sensors; communicating with vehicles through vehicle-to-everything (V2X) roadside units to obtain vehicle driving information; and detecting vehicle passage through geomagnetic coils. The data fusion module is configured to preprocess the collected multimodal traffic data, including data cleaning, normalization and standardization; and to fuse data from different devices to generate comprehensive traffic status information.

8. The urban area traffic signal cooperative optimization system based on deep reinforcement learning according to claim 1, characterized in that, Methods for constructing traffic prediction models include: Obtain the model training dataset; perform feature extraction and feature enhancement on the training dataset to obtain the enhanced feature matrix; Using intersections / sub-regions in the training dataset as nodes and road connectivity as edges, an initial topology is constructed, and a two-factor calculation of the spatial basis matrix is ​​introduced, which is the physical distance-flow correlation degree. The day is divided into 288 five-minute time slots. The feature vector of each time slot is learned through an embedding layer. Based on vehicle network event information, the event type, scope of impact and duration are encoded to obtain the event impact vector. The time slot feature vector and the event impact vector are fused through a gating mechanism to generate dynamic time weights. A topological memory module is constructed by introducing a long short-term memory unit. The historical adjacency matrix is ​​vectorized and input into an LSTM to store the adjacency matrix change patterns over nearly 10 time steps, and the memory weight matrix is ​​output. The spatial fundamental matrix, the temporal dynamic weight matrix, and the memory weight matrix are fused and normalized to generate a spatiotemporal dual-driven dynamic adjacency matrix. Based on a nested spatiotemporal deep coding network, feature mining is performed on the enhanced feature matrix and dynamic adjacency matrix to output encoded features; A Transformer decoder is used, with global features from the encoded features as input, to predict long-term traffic trends; a trend constraint loss is introduced to constrain the consistency between the predicted trend and the rate of change of the first derivative of traffic in the same period of history. Input long-term prediction results, mid-level features of encoded features, and historical enhancement features, and use gated recurrent units combined with graph attention mechanism to model the spatiotemporal smooth changes of mid-term traffic. Input the mid-term prediction results, fine-grained features of the encoded features, and historical multi-source data features. Use a lightweight gating fusion mechanism to dynamically adjust the feature weights. Input the weighted features into a multilayer perceptron and output fine-grained traffic prediction values. A gradient boosting tree model is trained using historical data to build a mapping relationship library between traffic density, vehicle type ratio, arrival rate, signal cycle, and green light ratio; for each intersection, the features corresponding to the short-term traffic forecast are input, and the initial signal timing scheme is output. Using intersections as nodes and road connectivity as edges, an intersection linkage graph is constructed, with the adjacency matrix weight set as the traffic flow impact coefficient between intersections. A graph attention network is used to input the initial timing scheme and short-term traffic flow forecast for each intersection, and output the linkage adjustment coefficient. The initial timing scheme and the linkage adjustment coefficient are fused to obtain the final timing scheme. The final timing scheme includes the cycle duration, the green light ratio of each phase, and the phase difference between adjacent intersections. The training dataset is input into the model in batches for training, resulting in a traffic prediction model.

Images