Traffic signal control system based on multi-mode perception and collaborative optimization

Abstract

The invention relates to the technical field of traffic signal control, in particular to a traffic signal control system based on multi-modal sensing and collaborative optimization, which comprises a multispectral sensing unit, a heterogeneous sensing fusion module, an edge calculation unit, a multi-target game decision engine, a redundancy control module and a signal lamp group, the multispectral sensing unit adopts a visible light and infrared dual-mode camera combination, the heterogeneous sensing fusion module adopts a 77GHz millimeter wave radar and 905nm laser radar combination, and the multi-target game decision engine adopts an MARL algorithm; according to the method, global traffic flow optimization is realized through a hierarchical control architecture, so that the severe weather detection precision is remarkably improved, the detection precision is greater than or equal to 92.5%, the emergency vehicle priority passing efficiency is also improved, and the response time is less than or equal to 25 seconds.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic signal control, and particularly to a traffic signal control system based on multi-modal perception and collaborative optimization. Background Art

[0002] With the continuous expansion of the urban scale, the ownership of motor vehicles has increased rapidly, and the congestion of urban traffic has become increasingly serious. Improving traffic management capabilities is an issue that every city must face.

[0003] The traffic signal control system for intersections has the following problems: traditional systems usually adopt fixed timing or time-division adjustment, and cannot respond to the dynamic changes of traffic flow in real time, resulting in waste of green light time or vehicle detention. Single-intersection independent control leads to unbalanced regional traffic flow (regional traffic efficiency loss ≥ 15%); at the same time, there is a lack of real-time identification and priority passage mechanism for emergency vehicles such as ambulances and fire trucks, resulting in low traffic efficiency in case of emergencies and too long priority passage response time for emergency vehicles. In addition, the existing systems do not meet the requirement of Article 5.3.2 of GB / T 20839-2023 regarding the priority response time of emergency vehicles ≤ 30 seconds, and rely on a single sensor, resulting in a significant decrease in detection accuracy under bad weather or complex lighting conditions, and the detection system fails under bad weather (target loss rate in rain and fog weather ≥ 37%). Moreover, the early image processing algorithms have a large amount of calculation and are difficult to run in real time on embedded devices, resulting in system response delay; for independent intersection control, there is a lack of multi-intersection collaborative optimization ability, which easily leads to unbalanced regional traffic flow.

[0004] Therefore, in view of the above problems, the present invention proposes a traffic signal control system based on multi-modal perception and collaborative optimization. Summary of the Invention

[0005] In order to overcome the problems that traditional systems usually adopt fixed timing or time-division adjustment, cannot respond to the dynamic changes of traffic flow in real time, and result in low traffic efficiency in case of emergencies, the present invention proposes a traffic signal control system based on multi-modal perception and collaborative optimization.

[0006] The technical solution of the present invention is: a traffic signal control system based on multi-modal perception and collaborative optimization, including a multi-spectral perception unit, a heterogeneous sensor fusion module, an edge computing unit, a spatio-temporal alignment subsystem, a multi-objective game decision-making engine, a redundant control module, and a signal light group. The multi-spectral perception unit is connected to the heterogeneous sensor fusion module, the heterogeneous sensor fusion module is connected to the edge computing unit, the edge computing unit is connected to the spatio-temporal alignment subsystem, the spatio-temporal alignment subsystem is connected to the multi-objective game decision-making engine, the multi-objective game decision-making engine is connected to the redundant control module, and the redundant control module is connected to the signal light group.

[0007] Preferably, the multi - spectral sensing unit includes a visible - light camera and an infrared thermal imager. The visible - light camera uses a Sony IMX585 sensor with a dynamic range of ≥120dB, which can cover 6 lanes. The infrared thermal imager has a working wavelength of 8 - 14μm and a temperature measurement accuracy of ±0.5°C.

[0008] Preferably, the heterogeneous sensing fusion module integrates a 77GHz millimeter - wave radar and a 905nm lidar, and realizes target tracking through an improved JPDA algorithm, where the ID switching rate ≤5% / s.

[0009] Preferably, the edge computing unit is equipped with an NVIDIA Jetson Orin platform with a computing power of 275 TOPS. And a YOLOv8 model accelerated by TensorRT and a MARL algorithm are deployed to dynamically generate phase - switching instructions. The YOLOv8 model takes an input of 320×320 and has a throughput of 150 FPS.

[0010] Preferably, the time - space alignment subsystem supports dual - mode timing of Beidou + 5G with a synchronization error ≤1μs, and completes the conversion between the LiDAR and visual coordinate systems by combining the ICP algorithm.

[0011] Preferably, the multi - target game - decision engine trains multi - agent game strategies based on the IPPO algorithm, and the reward function is R = α * traffic efficiency - β * safety risk.

[0012] Preferably, the redundant control module adopts a dual - channel CAN bus and a heartbeat detection mechanism. The bus uses a TJA1454 transceiver with a bandwidth of 5Mbps, and the switching time of the heartbeat detection mechanism ≤10ms.

[0013] Preferably, the signal lamp group drives a solid - state relay through a Siemens ST950 PLC, supports countdown display and sound - light prompt, and automatically switches to the yellow - flash mode when a fault occurs.

[0014] Preferably, the system combines a priority marking strategy and a forced - passage mechanism, and dynamically interrupts the signal cycle through V2X communication to implement a "green channel" for emergency vehicles, with a response time ≤25 seconds.

[0015] Preferably, the time - space alignment subsystem realizes sub - meter - level fusion accuracy of multi - source data through dual - mode timing of Beidou + 5G, with a coordinate error ≤5cm.

[0016] Advantages of the present invention:

[0017] 1. The present invention realizes the optimization of the global traffic flow through a hierarchical control architecture, thereby significantly improving the detection accuracy of bad weather, and the detection accuracy ≥ 92.5%.

[0018] 2. High-precision all-weather traffic target detection is achieved through the multi-spectral perception unit and the heterogeneous sensing fusion module. Based on the edge computing unit and the MARL multi-objective game decision-making engine, the signal timing is dynamically optimized, improving the traffic efficiency and reducing the conflict probability.

[0019] 3. By combining the Beidou + 5G time-space alignment subsystem and the dual-channel CAN redundant control module, it is ensured that the priority response time of emergency vehicles ≤ 25 seconds, overall solving the problems of traditional system response delay, single-sensor limitation, and lack of multi-intersection coordination, and significantly optimizing the safety and efficiency in complex traffic scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The system structure diagram of the present invention is shown. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0022] Please refer to Figure 1 , the present invention provides an embodiment: a traffic signal control system based on multi-modal perception and collaborative optimization, including a multi-spectral perception unit, a heterogeneous sensing fusion module, an edge computing unit, a time-space alignment subsystem, a multi-objective game decision-making engine, a redundant control module, and a signal light group. The multi-spectral perception unit is connected to the heterogeneous sensing fusion module, the heterogeneous sensing fusion module is connected to the edge computing unit, the edge computing unit is connected to the time-space alignment subsystem, the time-space alignment subsystem is connected to the multi-objective game decision-making engine, the multi-objective game decision-making engine is connected to the redundant control module, and the redundant control module is connected to the signal light group.

[0023] The above multi-modal perception unit is used to collect the traffic scenarios in multiple directions at the intersection in real time, including:

[0024] Visible light camera: The visible light camera: is a multi-spectral imaging device with a dynamic range ≥ 120dB and covering 6 lanes, supporting WDR technology.

[0025] Infrared thermal imager: Operating wavelength 8 - 14μm, temperature measurement accuracy ±0.5°C;

[0026] LiDAR: Velodyne VLP-16, vertical resolution 0.4°, ranging accuracy ±2cm;

[0027] Preferably, the above heterogeneous sensing fusion module integrates the EKF-PF hybrid filtering algorithm, processes the linear motion model through extended Kalman filtering, and processes the non-linear scenario through particle filtering, so as to achieve a multi-target tracking ID switching rate ≤5% / s; target height measurement accuracy ±5cm; data fusion delay ≤15ms.

[0028] Furthermore, the operation process of the multi-modal perception unit (including visible light / infrared dual-mode camera) is described in detail:

[0029] S1, Multi-spectral collaborative acquisition. The visible light camera (Sony IMX585 sensor) captures RGB images in four directions of the intersection at a frame rate of 30fps, uses a 120° wide-angle to cover 6 lanes, and has built-in WDR technology to dynamically adjust the exposure, with a dynamic range ≥120dB; the infrared thermal imager synchronously acquires the heat source distribution data, with a temperature measurement accuracy of ±0.5°C, and realizes all-weather target detection through a non-cooled focal plane array (UCPA); this step uses a dual-channel asynchronous acquisition circuit to achieve image stream time alignment through FPGA, with a jitter <1μs.

[0030] S2, Cross-modal data fusion. The visible light image passes through an ISP chip and is based on the non-local means algorithm (NLM) to eliminate low-light noise, and at the same time uses adaptive logarithmic transformation to fuse multi-exposure frames. The infrared image extracts the heat source contour through the YOLOv7-Tiny model, performs IoU matching (threshold >0.6) with the visible light detection frame, and finally outputs the fused feature map to the edge computing unit, with a delay ≤20ms.

[0031] Furthermore, the operation process of (integrated 77GHz millimeter-wave radar + 905nm LiDAR) is described in detail:

[0032] S1, Millimeter-wave radar data preprocessing. First, the 77GHz millimeter-wave radar emits a Chirp signal, calculates the range-Doppler matrix through FFT, separates the multi-target point cloud through the DBSCAN clustering algorithm to filter static obstacles, and uses Kalman filtering to predict the motion state of the target, with a position error of ±0.1m and a speed error of ±0.1m / s.

[0033] S2, LiDAR point cloud processing. The 905nm LiDAR outputs point cloud data at a frequency of 20Hz, with a vertical resolution of 0.4°. After VoxelGrid filtering, the sampling is reduced to 1000 points / frame. At the same time, the ground plane is extracted by the RANSAC algorithm, and semantic features are extracted by the PointNet++ network to classify traffic targets, with an accuracy > 92%.

[0034] S3, Multi-source data fusion. The spatio-temporal alignment subsystem associates the millimeter-wave radar target ID with the LiDAR point cloud coordinates through an improved JPDA algorithm, and combines the EKF-PF hybrid filter (extended Kalman filter for linear motion processing and particle filter for non-linear scenario processing) to output the fused trajectory for multi-target tracking and sub-meter positioning. Finally, the data is transmitted to the edge computing unit with a delay ≤ 15ms.

[0035] Furthermore, the operation process of the edge computing unit (equipped with the NVIDIA Jetson Orin platform) is described in detail:

[0036] S1, Receive multi-source data from the heterogeneous sensor fusion module. The multi-source data includes video streams, LiDAR point clouds, and millimeter-wave radar trajectories, and is processed in real time through the CUDA parallel computing framework. Among them, the YOLOv8 model performs vehicle and pedestrian detection with a resolution of 320×320 and a throughput of 150FPS. The 3DU-Net network segments the point cloud semantics with an accuracy > 92%. This process deploys the TensorRT acceleration engine and INT8 quantization, with an accuracy loss < 1%. Among them, the model is optimized. YOLOv8-tiny is used to reduce the computational volume by 30% through channel pruning technology, and the LiDAR point cloud processing adopts the VoxelGrid+PointNet++ combined architecture. Among them, the TensorRT engine is optimized, and the optimization content is to fuse the operators (convolution + BN + ReLU → composite operator), reducing the video memory occupancy of dynamic memory allocation by 40%.

[0037] S2, Dynamic decision generation. Build a spatio-temporal graph model (ST-GCN) to represent traffic flow dynamics, where nodes represent vehicles / pedestrians and edge weights are spatio-temporal distances. Based on the multi-agent reinforcement learning (MARL) algorithm, a signal light phase control strategy is generated, comprehensively weighing the traffic efficiency and safety risks (reward function R = α * efficiency - β * safety). Finally, the optimized instructions are output to the redundant control module, and the decision delay ≤ 25ms, thus ensuring real-time response.

[0038] Furthermore, the operation process of the spatio-temporal alignment subsystem (supporting Beidou + 5G dual-mode time synchronization) is described in detail:

[0039] S1. Use the Beidou receiver to capture the B1I frequency signal and calculate the pseudo-range observation value (accuracy ±10 ns). At the same time, the 5G base station synchronizes the GPSTime and transmits it via the PTP protocol, and applies the clock deviation compensation algorithm (error <1 μs) to achieve sub-microsecond-level unified time reference.

[0040] S2. Based on the LiDAR point cloud, construct a three-dimensional coordinate system with the geometric center of the intersection as the origin. Use the ICP algorithm to perform spatial registration on the polar coordinate data of the millimeter-wave radar and the visual coordinate system (error <5 cm) to complete the spatial alignment of multi-sensor data.

[0041] S3. In the timestamp alignment stage, the system attaches Lamport logical clock tags to mark the timing relationship for each frame of data, and solves the timing conflict problem of cross-device data acquisition through the vector clock. Finally, it outputs a globally unified spatio-temporal tag to the edge computing unit to ensure that the spatio-temporal consistency of heterogeneous sensing data meets the fusion accuracy requirements, providing an accurate spatio-temporal reference for subsequent multi-object tracking and decision-making.

[0042] Furthermore, the action process of the multi-object game decision-making engine (based on the MARL algorithm) is described in detail:

[0043] S1. Starting from the construction of the state space, first define the state vector S_t that includes vehicle position, speed, acceleration, and signal light phase, and use the LSTM network to model the traffic flow timing characteristics to predict the traffic flow changes in the next three cycles.

[0044] S2. Subsequently, design the strategy space. Treat each intersection as an independent agent, and its action space A_t includes two operations: extending the green light by Δt and phase switching. At the same time, introduce communication constraints (C-COM) to ensure that the decisions of adjacent intersections satisfy the V2X communication delay less than 50 ms.

[0045] In the design of the reward mechanism, the safety reward R_safe is calculated by the reciprocal of the conflict probability, the efficiency reward R_eff is determined by the product of the number of passing vehicles and the average speed, and the comprehensive reward R dynamically adjusts the weight coefficient α (initial value 0.7) to balance efficiency and safety (R = αR_eff + (1 - α)R_safe); finally, use the distributed training framework to optimize the multi-agent strategy based on the IPPO algorithm (Independent Proximal Policy Optimization), and set the capacity of the experience replay pool to 10^6 and the PPO clipping parameter ε = 0.3.

[0046] Furthermore, the action process of the redundant control module (dual-channel CAN bus + heartbeat detection mechanism) is described in detail:

[0047] The main channel uses a CAN FD bus (5 Mbps) to transmit real-time control instructions, and the backup channel uses a traditional CAN bus (1 Mbps) to transmit status monitoring data. The two are physically isolated to ensure no interference with each other. The main control unit sends a heartbeat packet containing CRC checksum and sequence number to the backup unit every 10 ms. The latter continuously monitors the heartbeat signal. If the signal is not received or the check fails for three consecutive times, the fault switching logic is triggered. Hardware faults are detected in real time through GPIO pins for abnormal chip temperature / voltage, and software faults are monitored by a watchdog timer (timeout threshold: 300 ms). Any fault will initiate a switching sequence: first, the main control unit disables the output driver to prevent instruction conflicts, then the backup unit takes over the bus arbitration right and retransmits the last three valid frames to ensure the continuity of the signal lamp status. At the same time, the redundant power supply uses dual 24V DC inputs in combination with an isolated DC-DC converter to ensure seamless switching in case of power interruption, and the HMAC-SHA256 algorithm performs integrity verification on all control instructions to prevent data tampering. After fault recovery, the system automatically executes a self-check program, preferentially restoring communication on the main channel, and the backup channel remains in a standby state until the next heartbeat detection cycle, forming a closed-loop fault-tolerant control.

[0048] The system dynamically manages the energy consumption of sensors through event-triggered perception: usually, a low-power millimeter-wave radar continuously scans the road. When a vehicle or pedestrian is detected (triggered by Doppler signals or micro-motion features), a high-precision lidar and an infrared camera are immediately activated for detailed perception, and automatically switch back to the sleep mode 100 ms after the target disappears. Combined with adaptive sampling rate adjustment, the overall energy consumption of the sensors can be reduced by 60%.

[0049] Furthermore, the present invention has the following innovative points:

[0050] Multi-modal spatio-temporal alignment:

[0051] The first three-dimensional spatio-temporal calibration system of Beidou + 5G + LiDAR, with a positioning accuracy of ±5 cm, improving the positioning reliability by 300% compared with traditional methods (only GPS + IMU).

[0052] Edge-cloud collaborative decision-making:

[0053] The edge side processes real-time tasks with a latency <100 ms;

[0054] The cloud trains a globally optimized model and updates the strategy once every 24 hours.

[0055] Secure redundant architecture:

[0056] The two CAN buses are physically isolated, with an electromagnetic shielding distance >3 mm;

[0057] The fault switching time <50 ms, meeting the ISO 26262 ASIL-D standard.

[0058] Furthermore, the output relationships of each module and the details of hardware / software interaction are described in detail:

[0059] For the multi-modal perception unit:

[0060] Input: Optical signal (visible light / infrared light) → Original image of the environmental scene.

[0061] Output: Multi-modal fusion feature map (including target position, heat source distribution, timestamp).

[0062] Hardware interaction:

[0063] Visible light camera: Sony IMX585 sensor (1 / 1.2" CMOS, 2.9μm pixel, 120° wide angle) + 6mm focal length lens.

[0064] Infrared thermal imager: Uncooled focal plane array (UCPA, 8 - 14μm wavelength, temperature measurement accuracy ±0.5°C).

[0065] FPGA chip: Xilinx Artix-7 for asynchronous data acquisition and clock alignment (jitter < 1μs).

[0066] Software interaction:

[0067] OpenCV: Fisheye distortion correction (cv2.fisheye.initUndistortRectifyMap in OpenCV).

[0068] Heat source segmentation: YOLOv7-Tiny model (YOLO architecture, input resolution 416×416).

[0069] Multi-modal fusion: Cross-modal feature alignment based on attention mechanism (Transformer architecture).

[0070] Interface:

[0071] Gigabit Ethernet (RJ45) transmits the original data to the edge computing unit.

[0072] LVDS interface outputs the time synchronization signal to the spatio-temporal alignment subsystem.

[0073] The visible light camera and the infrared thermal imager adopt polarization multiplexing technology:

[0074] Visible light channel: Sony IMX585 sensor (quantum efficiency ≥ 80%);

[0075] Infrared channel: Uncooled focal plane array (NETD ≤ 40mK);

[0076] Polarizer dynamic switching (extinction ratio ≥ 1000:1).

[0077] For the heterogeneous sensing fusion module:

[0078] Input:

[0079] Millimeter-wave radar: 77GHz FMCW signal reflected wave.

[0080] LiDAR: 360° rotating point cloud data (vertical resolution 0.4°).

[0081] Output: Spatio-temporally aligned fusion trajectory (coordinate error ≤ 5cm, velocity error ±0.2m / s).

[0082] Hardware interaction:

[0083] Millimeter-wave radar: TI AWR2243 chip (4 transmit and 4 receive channels, Chirp slope 1.5GS / s).

[0084] LiDAR: Velodyne VLP-16 (16 lines, maximum ranging 200m).

[0085] DSP processor: TI TMS320C6748 executes FFT and clustering algorithms.

[0086] Software interaction:

[0087] Radar processing: MTT algorithm (multi-target tracking) + DBSCAN clustering.

[0088] Point cloud processing: PointNet++ network (implemented in PyTorch, classification accuracy 92%).

[0089] Fusion algorithm: Improved JPDA algorithm (joint probabilistic data association).

[0090] Interface:

[0091] CAN bus (ISO 11898-2) transmits fusion data to the heterogeneous sensing fusion module.

[0092] Ethernet (TSN) synchronizes spatio-temporal coordinates to the edge computing unit.

[0093] For the edge computing unit:

[0094] Input: Multi-source data streams (video, point cloud, radar trajectory).

[0095] Output: Dynamic decision instructions (phase switching, priority marking).

[0096] Hardware interaction:

[0097] GPU: NVIDIA Orin (275 TOPS computing power, supporting FP16 / INT8 mixed precision).

[0098] Memory: LPDDR5 16GB (bandwidth 51.2GB / s, dual-channel mode).

[0099] Storage: UFS 3.1 1TB (supporting RAID 0 acceleration).

[0100] Software interaction:

[0101] Deep learning: TensorRT accelerates YOLOv8 (input 320×320, throughput 150FPS).

[0102] Real-time scheduling: ROS2 Galactic framework (QoS policy to ensure the priority of critical tasks).

[0103] Edge inference: ONNX Runtime optimizes the model inference latency (<20ms / frame).

[0104] Interfaces:

[0105] PCIe 4.0×4 connects the NVIDIA GPU and CPU.

[0106] USB 3.2 Gen2 transfers debug logs to the PC side.

[0107] The improved JPDA algorithm achieves three major innovations:

[0108] First, a dynamic probability association matrix (update frequency ≥20Hz); second, the fusion of multiple hypothesis tracking (MHT) and probability data association (PDA); third, a prediction correction mechanism based on Kalman filtering.

[0109] Technical details of model optimization:

[0110] The YOLOv8 model adopts INT8 quantization (precision loss ≤1.2%);

[0111] TensorRT engine optimization: operator fusion (convolution + BN + ReLU → composite operator) - dynamic memory allocation (video memory occupancy reduced by 40%).

[0112] For the spatio-temporal alignment subsystem:

[0113] Input: original timestamp (device local clock).

[0114] Output: globally unified timestamp (error ≤1μs).

[0115] Hardware interaction:

[0116] BeiDou module: UBLOX F9P receiver (B1I frequency band, positioning accuracy ±10 cm).

[0117] 5G module: Quectel RM500Q (Sub-6GHz frequency band, timing accuracy ±30 ns).

[0118] Atomic clock: OCXO oven-controlled crystal oscillator (frequency stability ±5×10 -11 ).

[0119] Software interaction:

[0120] PTP protocol: IEEE 1588-2019 for sub-microsecond clock synchronization.

[0121] Coordinate transformation: ICP algorithm (Iterative Closest Point) to align LiDAR and vision coordinate systems.

[0122] Timestamp encoding: NTP timestamp format (64-bit integer + 32-bit fraction).

[0123] Interface:

[0124] PPS signal (1PPS) to synchronize the edge computing unit and the signal lamp controller.

[0125] NTP protocol to upload the alignment status to the cloud server.

[0126] For the multi-objective game decision-making engine:

[0127] Input: fused trajectory, signal lamp status, emergency vehicle priority.

[0128] Output: optimal phase control strategy (green light duration, switching timing).

[0129] Hardware interaction:

[0130] GPU: Jetson Orin for parallel game simulation (768 CUDA cores).

[0131] Memory: HBM2E 24GB (bandwidth 1TB / s, reducing data transfer latency).

[0132] Software interaction:

[0133] MARL framework: IPPO algorithm (Independent Proximal Policy Optimization). Reward function: R = α · traffic efficiency - β · safety risk (α = 0.65, dynamically adjusted).

[0134] Conflict detection: spatio-temporal conflict detection algorithm (SCOOT model).

[0135] Interface:

[0136] The ZeroMQ message queue communicates with the edge computing unit.

[0137] The RESTful API reports the decision results to the traffic management center.

[0138] For the redundant control module:

[0139] Inputs: master control instructions, heartbeat signals, fault codes.

[0140] Outputs: safety control signals (signal lamp drive, degraded mode).

[0141] Hardware interaction:

[0142] CAN transceiver: TJA1454 (supporting CAN FD, rate 5Mbps).

[0143] Watchdog: MAX706 watchdog timer (timeout threshold 300ms).

[0144] Redundant power supply: dual 24V DC inputs (isolated DC-DC conversion).

[0145] Software interaction:

[0146] Fault diagnosis: CANopen protocol parses error frames (error counter threshold > 127).

[0147] Fault-tolerant switching: state machine control (normal → degraded → emergency mode).

[0148] Secure communication: HMAC-SHA256 verifies the integrity of instructions.

[0149] Interfaces:

[0150] Dual CAN buses connect the ECU to the signal lamp controller.

[0151] GPIO interrupt triggers hardware emergency braking.

[0152] Parameter optimization:

[0153] Discount factor γ = 0.98 (determined through Q-learning convergence testing).

[0154] Exploration rate ε = 0.1 (ε-greedy strategy).

[0155] State space dimensionality reduction: uses PCA to reduce 256-dimensional features to 32 dimensions.

[0156] Furthermore, to verify the technical effects of the present invention, a crossroads in the core business district of a certain city (average daily traffic flow of 8,000 vehicles) was selected for on-site measurement, and the comparison data is as follows:

[0157]

[0158] Description of the test environment:

[0159] Test period: March - May 2025 (covering morning / evening rush hours);

[0160] Device configuration: Dual - mode camera (Sony IMX585 + UCPA thermal imager), Velodyne VLP - 16 lidar;

[0161] Data source: Cross - verification between the traffic flow monitoring system of the traffic police command center and in - vehicle GPS trajectory data.

[0162] Furthermore, to verify the technical effects of the present invention, a crossroads in the core business district of a certain city (with a daily average traffic volume of 8,000 vehicles) was selected for on - site measurement, and the comparison data is as follows:

[0163] Regarding the single - sensor failure tolerance experiment:

[0164] Test scenario: Simulate the failure of the millimeter - wave radar: Shield the radar data input through software;

[0165] Simulate the failure of the lidar: Physically block the lidar field of view;

[0166] Test results:

[0167]

[0168]

[0169] Regarding the multi - intersection cooperative control experiment:

[0170] Test area: A typical urban intersection group composed of 3 adjacent intersections;

[0171] Test method: Control group: Traditional single - intersection independent control (fixed timing);

[0172] Experimental group: The multi - intersection cooperative optimization system of the present invention;

[0173] Test results:

[0174]

[0175] Description of the test environment:

[0176] Test period: March - May 2025 (covering morning / evening rush hours);

[0177] Device Configuration: Dual-mode camera (Sony IMX585 + UCPA thermal imager), Velodyne VLP-16 lidar;

[0178] Data Source: Cross-validation of traffic monitoring system in traffic police command center and vehicle-mounted GPS trajectory data.

[0179] Furthermore, for Time-Sensitive Networking (TSN):

[0180] Hardware: Switch supporting IEEE 802.1AS (such as Renesas RZ / N2L).

[0181] Software: Linux kernel PTP daemon (ptp4l), clock synchronization accuracy ≤ 1 μs.

[0182] For the fault tolerance mechanism:

[0183] Hardware: Dual-power redundant power supply module (main and standby switching time ≤ 10 ms).

[0184] Software: Heartbeat packet detection (ECU sends status to CPU every 500 ms), timeout triggers degradation mode.

[0185] Legal Compliance:

[0186] Hardware: Yellow light drive circuit is forced to last for 3 seconds, in line with Article 5.2.3 of GB 14887-2011.

[0187] Software: Traffic rule library locks the yellow light time, prohibits AI strategy modification, in line with Article 7.1.2 of JT / T 1049-2017.

[0188] Furthermore, for implementation verification:

[0189] Input Verification: Camera and radar data time alignment test (PTP synchronization error ≤ 1 ms).

[0190] Output Verification: Signal lamp switching delay test (from CPU instruction to lamp color change ≤ 200 ms).

[0191] Hardware Compatibility: RS485 bus load test (supports 32 nodes in parallel, bit error rate ≤ 1e-9).

[0192] Software Robustness: mAP ≥ 0.85 for YOLOv5 in rainy and foggy weather (evaluated using COCO dataset).

Claims

1. A traffic signal control system based on multi-modal perception and collaborative optimization, characterized in that: It includes a multispectral sensing unit, a heterogeneous sensing fusion module, an edge computing unit, a spatio-temporal alignment subsystem, a multi-object game decision-making engine, a redundancy control module, and a signal light group. The multispectral sensing unit is connected to the heterogeneous sensing fusion module, the heterogeneous sensing fusion module is connected to the edge computing unit, the edge computing unit is connected to the spatio-temporal alignment subsystem, the spatio-temporal alignment subsystem is connected to the multi-object game decision-making engine, the multi-object game decision-making engine is connected to the redundancy control module, and the redundancy control module is connected to the signal light group.

2. The traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, characterized in that: The multispectral sensing unit includes a visible light camera and an infrared thermal imager. The visible light camera uses a Sony IMX585 sensor with a dynamic range ≥ 120dB and can cover 6 lanes. The infrared thermal imager has a working wavelength of 8 - 14μm and a temperature measurement accuracy of ±0.5°C.

3. The traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, wherein: The heterogeneous sensing fusion module integrates a 77GHz millimeter-wave radar and a 905nm lidar, and realizes target tracking through an improved JPDA algorithm, where the ID switching rate ≤ 5% / s.

4. A traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, characterized in that: The edge computing unit is equipped with an NVIDIA Jetson Orin platform with a computing power of 275 TOPS. It deploys a TensorRT-accelerated YOLOv8 model and a MARL algorithm to dynamically generate phase switching instructions. The YOLOv8 model takes 320×320 as input and has a throughput of 150 FPS.

5. The traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, characterized in that: The spatio-temporal alignment subsystem supports Beidou + 5G dual-mode time synchronization with a synchronization error ≤ 1μs, and completes the LiDAR and visual coordinate system conversion by combining the ICP algorithm.

6. The traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, characterized in that: The multi-object game decision-making engine trains multi-agent game strategies based on the IPPO algorithm, and the reward function is R = α * traffic efficiency - β * safety risk.

7. A traffic signal control system based on multimodal perception and collaborative optimization according to claim 1, characterized in that: The redundancy control module adopts a dual-channel CAN bus and a heartbeat detection mechanism. The bus uses a TJA1454 transceiver with a bandwidth of 5Mbps, and the switching time of the heartbeat detection mechanism ≤ 10ms.

8. The traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, wherein: The signal light group drives a solid-state relay through a Siemens ST950 PLC, supports countdown display and sound and light prompts, and automatically switches to the yellow flash mode when a fault occurs.

9. The traffic signal control system based on multi-modal perception and collaborative optimization according to claim 1, characterized in that: The system realizes an "emergency vehicle green channel" with a response time ≤ 25 seconds by combining a priority marking strategy and a forced passage mechanism, and dynamically interrupting the signal cycle through V2X communication.

10. A traffic signal control system based on multimodal perception and collaborative optimization according to claim 1, characterized in that: The spatio-temporal alignment subsystem realizes multi-source data sub-meter-level fusion accuracy through Beidou + 5G dual-mode time synchronization, with a coordinate error ≤ 5cm.

Images