A highway safety warning system and method based on road condition information lights

Abstract

The application relates to the technical field of traffic safety and intelligent traffic system, in particular to a highway safety early warning system and method based on road condition information lights, which comprises a front-end acquisition subsystem, a deep learning analysis subsystem, a communication and control subsystem and an external service linkage subsystem. The front-end acquisition subsystem obtains real-time traffic data, weather information and video streams through solar road condition information lights, high-speed cameras, weather sensors and vehicle detectors and the like. The deep learning analysis subsystem adopts multi-source data fusion, a multi-layer space-time feature extraction network and a dynamic risk assessment module to realize accurate identification and scoring of traffic risks and generate corresponding flow distribution strategies. The communication and control subsystem realizes real-time control of the front-end warning devices through a background platform and a data server, and pushes early warning information to a navigation platform, a video monitoring platform and a traffic management department. The application effectively reduces the probability of secondary accidents and improves the safety and traffic efficiency of highways.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of traffic safety and intelligent transportation system, in particular to a highway safety warning system and method based on road condition information light, which is applied to traffic flow monitoring, traffic incident detection, severe weather warning and intelligent induction and distribution and the like. BACKGROUND

[0002] With the rapid development of highways in modern transportation system, the highway traffic flow and vehicle density are increasing, how to effectively monitor the operation of the highway and quickly warn potential dangers has become a key issue of traffic safety and management. The existing highway monitoring usually adopts video monitoring, vehicle detector, meteorological sensor and other facilities, but they work independently and lack perfect data fusion and intelligent analysis capability. When traffic accidents occur or severe weather (such as fog, snowstorm, rainstorm, etc.) is encountered, there is a lag problem in obtaining the dangerous information of the front road, and there is a lack of efficient distribution indication means, which easily leads to emergency braking or congestion of vehicles in front of the accident or dangerous section, and greatly increases the probability of secondary accidents.

[0003] In recent years, with the development of artificial intelligence and big data technology, traffic incident detection and prediction methods based on deep learning have been applied to intelligent transportation system. However, due to the characteristics of high speed, large flow, and variable environment (night, tunnel, mountainous area, extreme weather, etc.) of highway scene, traditional algorithms based on rules or shallow machine learning are difficult to obtain sufficient accuracy and real-time performance. At the same time, part of the existing solutions still adopt wired mode in device power supply, which is difficult to realize effective coverage of the whole road section, especially in remote mountainous areas or around tunnels. In addition, most of the current systems lack deep fusion and self-learning mechanism of multi-source data (video stream, traffic flow, meteorological data, navigation feedback, etc.), and cannot dynamically adjust according to real-time scene and historical accident mode, resulting in insufficient flexibility of warning accuracy and coping strategy. SUMMARY

[0004] Based on this, in order to improve the accuracy of highway accident identification and shunting, reduce the risk of secondary accidents, the application provides a highway safety warning system based on road condition information light, which uses a deep learning model to analyze multi-source data, and uses a solar road condition information light and wireless communication means to realize real-time linkage warning. By deploying a multi-layer spatiotemporal feature extraction network and a dynamic risk assessment and prediction module in the cloud or background center, the system can realize all-around and all-weather traffic safety warning and shunting induction. The scheme can not only effectively improve the identification accuracy of accidents or congestion, but also can give corresponding risk scores and shunting suggestions for different road sections and different time periods in combination with weather factors and historical accident distribution. After the accident or anomaly is resolved, the model can be automatically corrected according to the feedback information to realize self-learning optimization, thereby further enhancing the adaptability and intelligent level of the system.

[0005] In order to achieve the above-mentioned purpose, the application adopts the following technical scheme:

[0006] A highway safety warning system based on road condition information light, the system comprises:

[0007] 1) Front-end acquisition and warning subsystem, comprising a road condition information light powered by solar energy and a front-end acquisition module; a) the road condition information light is equipped with double high-brightness flashing warning lights, a high-pitched loudspeaker and a dot matrix LED display screen, and receives background instructions through a wireless communication module;

[0008] b) the front-end acquisition module comprises a high-speed camera, a weather sensor, a vehicle detector and other sensing devices for collecting road traffic state, weather information and vehicle operation data;

[0009] 2) Deep learning analysis subsystem, deployed in the cloud or background center, comprising:

[0010] a) multi-source data fusion module, for receiving real-time data feedback from high-speed cameras, weather sensors, vehicle detectors and navigation platforms, and performing multi-modal data preprocessing and alignment;

[0011] b) multi-layer spatiotemporal feature extraction network, based on convolutional neural network (CNN) and self-attention mechanism (Self-Attention) fusion structure, for feature extraction of video frames and time series traffic data to focus on potential accident occurrence areas or abnormal vehicle trajectories; c) dynamic risk assessment and prediction module, based on long short-term memory network (LSTM) or Transformer structure, combining weather changes, traffic flow fluctuations and historical accident data, for risk scoring of possible accidents or abnormalities on the highway, and automatically generating warning levels and recommended shunting strategies;

[0012] 3) Communication and control subsystem, including data exchange server and background control platform, for receiving risk assessment results of deep learning analysis subsystem, and issuing control instructions to front-end acquisition and warning subsystem to automatically or semi-automatically adjust the flashing mode of road condition information light, LED display content, and the form and content of voice broadcast;

[0013] 4) External service linkage subsystem, including bidirectional data interaction module with high-speed video monitoring platform and navigation platform, which automatically triggers early warning when identifying abnormal information such as traffic congestion, bad weather, traffic accidents or road construction, and synchronously pushes corresponding shunting scheme or safe driving prompt to the navigation platform;

[0014] 5) Self-learning optimization subsystem, including:

[0015] a) Historical data incremental learning module, which records and labels previous accidents, early warning information and early warning response results, and dynamically updates the deep learning model offline through incremental learning method;

[0016] b) Feedback improvement module, which evaluates the early warning effect in real time according to the feedback of on-site monitoring personnel, patrol police and navigation platform users, automatically corrects model parameters or alarm threshold, and improves the identification accuracy and timeliness of abnormal scenes;

[0017] 6) Accident and bad weather integrated early warning strategy, which, when identifying target events such as accidents, ice and snow, fog, and water accumulation, combines real-time traffic flow prediction results and shunting strategy to issue warning prompts to road condition information lights and play voice through loudspeakers, and pushes corresponding guiding shunting information on the navigation platform to avoid secondary accidents and optimize traffic organization.

[0018] The highway safety warning system according to claim 1, wherein the front-end acquisition and warning subsystem further comprises a large-capacity energy storage unit and an intelligent power management module for maintaining normal power supply of the system during long-time rainy weather or at night, and monitoring the power state in real time through remote diagnosis function, and sending fault early warning to the background control platform when the power is insufficient.

[0019] As a preferred embodiment, the multi-layer spatio-temporal feature extraction network further adopts a multi-head self-attention mechanism to cross-correlate data of different road sections, different camera perspectives and different time periods, thereby improving the identification ability of accidents in special scenes such as tunnel entrances, sharp bends and service area entrances.

[0020] As preferred, the external service linkage subsystem will send real-time traffic data and prediction results to relevant emergency management departments or traffic command centers when detecting abnormal information, triggering emergency plans, including police car dispatch, accident site control, and highway entrance closure operations, to achieve multi-department coordination and linkage.

[0021] As preferred, the system is based on dynamic neighborhood mapping and adaptive time window interpolation algorithm, enhancing the detection sensitivity of high-risk section data and improving the spatio-temporal consistency of multi-modal data.

[0022] Further, the present application also discloses a highway safety warning method based on traffic information lights, applied to the highway safety warning system, which comprises the following steps:

[0023] 1) Data acquisition: using high-speed cameras, weather sensors, vehicle detectors and other multi-source sensing devices of the front-end acquisition module to obtain vehicle flow, speed, weather conditions and road video data;

[0024] 2) Data preprocessing: time alignment, noise filtering and format conversion are performed on multi-source data, and the preprocessed data is input into the deep learning analysis subsystem;

[0025] 3) Accident identification: using a multi-layer spatio-temporal feature extraction network to perform fusion analysis on video frames and time series traffic data to identify possible accidents or abnormal scenes;

[0026] 4) Risk scoring and shunting strategy generation: according to the identification results, the risk score is calculated through the dynamic risk assessment and prediction module, and the historical accident distribution, real-time traffic flow and weather changes are combined to generate warning levels and corresponding shunting strategies;

[0027] 5) Information push: the warning information, shunting strategy and driving prompt are sent to:

[0028] a) Traffic information lights: control the flashing frequency of the flashing light, LED text display and voice prompt content;

[0029] b) Navigation platform: real-time update of accident diversion information to guide rear drivers to avoid or detour;

[0030] c) High-speed video monitoring platform and related management departments: assist in traffic diversion or emergency disposal;

[0031] 6) Self-learning optimization: continuously monitor and record the execution results, and correct the deep learning model parameters or alarm thresholds through the self-learning optimization subsystem to improve the identification accuracy and response speed for special road sections or extreme weather.

[0032] As preferred, in the data preprocessing process, the heterogeneous data collected by multiple cameras and sensors are dynamically weighted and fused, different spatio-temporal resolutions are set for weather sensor data, vehicle detector data and camera video stream, and the weighting coefficients are adaptively adjusted according to the latest traffic flow and weather changes during the fusion process.

[0033] As preferred, in the risk score calculation process, a multi-objective loss function is used to simultaneously consider the influence of traffic congestion degree, abnormal vehicle driving behavior and extreme weather on highway safety, and online optimization is used to reduce false positives and false negatives.

[0034] As preferred, after information pushing, the system automatically performs statistical analysis on the subsequent vehicle deceleration or lane change behavior based on an event-driven mechanism, and returns the analysis result to the deep learning model to update the risk assessment threshold in similar scenarios, thereby improving the response speed of the system to similar abnormal scenarios.

[0035] As preferred, the executed warning process and results are post-evaluated, including:

[0036] a) Warning accuracy evaluation: comparing the accident warning level issued by the system with the actual accident severity;

[0037] b) Shunting effect evaluation: comparing the vehicle delay time before and after the implementation of the shunting strategy based on traffic flow data and congestion index;

[0038] c) Model incremental training: labeling the false positives or warning delay cases according to the evaluation results, and incorporating them into the historical data incremental learning module to continuously improve the prediction ability of the deep learning model in extreme road conditions.

[0039] The present application has the following technical effects due to the adoption of the above technical solutions:

[0040] 1. Significantly improve the accuracy and timeliness of accident detection and warning: by combining multi-source data fusion (including video, weather, vehicle detector and navigation platform feedback information, etc.) with deep learning algorithms, the present application can realize real-time analysis and early detection of traffic accidents, abnormal vehicle driving, adverse weather influence, etc. on highways. Compared with traditional schemes mainly relying on manual inspection or simple threshold detection, the present application can more accurately and quickly identify potential risks and issue warnings, thereby reducing the probability of secondary accidents.

[0041] 2. Deep fusion and self-learning optimization of multi-modal information: The present application adopts a multi-layer spatio-temporal feature extraction network and a dynamic risk assessment and prediction module to fully exploit the associated features in video frames, traffic flow, meteorological data, and historical accident records. This scheme can collect and fuse newly acquired data at any time after initial deployment, continuously optimize the deep learning model using incremental learning, online transfer learning, and other methods, thereby maintaining high prediction accuracy and robustness in varying environments such as extreme weather, holiday peaks, and special road sections in mountainous areas.

[0042] 3. Intelligent diversion and induction scheme to optimize traffic flow organization: The dynamic risk assessment and prediction module not only scores the probability or severity of accidents, but also automatically generates recommended diversion strategies based on road traffic and weather conditions. Through real-time linkage with navigation platforms, vehicles can be guided to bypass or disperse traffic in advance, effectively alleviating main line congestion, improving road traffic efficiency, and reducing the risk of queuing collisions and secondary accidents.

[0043] 4. Continuous monitoring and feedback mechanism to improve early warning strategies: After an accident or anomaly occurs, the system described in the present application feeds back the handling and execution effects (such as actual diversion efficiency, congestion, accident loss, etc.) to the self-learning optimization subsystem through external service linkage subsystems, dynamically adjusts the deep learning model and alarm thresholds in real time or offline, continuously improves the accuracy of early warning and decision-making quality in the iteration process, and enhances the maintainability and adaptability of the system.

[0044] 5. Realization of integrated highway safety management: The present application can seamlessly interface with existing highway video surveillance platforms and traffic management department information systems, forming a full-process linkage system from anomaly identification, information release to emergency disposal, not only improving single-point accident handling efficiency, but also providing reliable data support for subsequent road safety statistical analysis, traffic planning and management decision-making, and further improving the overall architecture of intelligent highway safety management. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 The system block diagram of the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments will be described clearly and completely below. Obviously, the described embodiments are only a part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0047] As shown in Figure 1 a highway safety warning system based on road condition information lights, the system comprises:

[0048] 1) Front-end acquisition and warning subsystem, including a road condition information light powered by solar energy and a front-end acquisition module; the road condition information light is equipped with double high-brightness flashing warning lights, a high-pitched loudspeaker and a dot matrix LED display screen, and receives background instructions through a wireless communication module; the front-end acquisition module includes various sensing devices such as high-speed cameras, weather sensors and vehicle detectors, for acquiring road traffic state, weather information and vehicle operation data;

[0049] 2) Deep learning analysis subsystem, deployed in the cloud or the background center, including:

[0050] Multi-source data fusion module: receiving real-time data from feedback data of high-speed cameras, weather sensors, vehicle detectors and navigation platforms, and performing multi-modal data preprocessing and alignment;

[0051] Multi-layer spatiotemporal feature extraction network: using a fusion structure based on convolutional neural network (CNN) and self-attention mechanism (Self-Attention) to extract features from video frames and time series traffic data; the self-attention mechanism can focus on potential accident occurrence areas or abnormal vehicle trajectories under multi-time and multi-view conditions;

[0052] Dynamic risk assessment and prediction module: based on long short-term memory network (LSTM) or Transformer structure, combining factors such as weather changes, traffic flow fluctuations and historical accident distribution to score the risk of possible accidents or abnormalities on the expressway, and automatically generating warning levels and recommended diversion strategies for different time periods and different road sections;

[0053] 3) Communication and control subsystem, including a data exchange server and a background control platform, for receiving risk assessment results from the deep learning analysis subsystem, issuing control instructions to the front-end acquisition and warning subsystem according to the assessment results, and automatically or semi-automatically adjusting the flashing mode of the road condition information light, the content of the LED display and the form and content of the voice broadcast;

[0054] 4) External service linkage subsystem, including a bidirectional data interaction module with a high-speed video monitoring platform and a navigation platform, for automatically triggering an alarm when abnormal information such as traffic congestion, severe weather, traffic accidents or road construction is identified, and synchronously pushing the corresponding diversion scheme or safety driving prompt to the navigation platform;

[0055] 5) Feedback improvement module: based on feedback from on-site monitoring personnel, patrol police and navigation platform users, real-time evaluation of warning effect, automatic correction of model parameters or alarm thresholds, and improvement of identification accuracy and timeliness of abnormal scenes;

[0056] 6) Accident and bad weather integrated early warning strategy: when a target event is identified, the target event includes an accident, ice and snow, fog, water accumulation, deep learning analysis subsystem combines real-time traffic flow prediction results and shunting strategy, sends warning prompts to road condition information lights, and plays voice through a loudspeaker, while pushing corresponding induced shunting information on the navigation platform to effectively avoid secondary accidents and optimize traffic organization.

[0057] I. System hardware architecture and deployment details

[0058] 1. Solar road condition information light

[0059] 1) Hardware specifications:

[0060] Solar panels: power generation power not less than 50W, conversion rate ≥ 17%;

[0061] Battery: according to different regions of sunshine duration, the capacity is usually selected 20Ah-60Ah;

[0062] Environmental adaptation: meet IP65 or above protection level, can work stably in-20℃-+60℃ environment;

[0063] Intelligent power supply management: use MPPT (Maximum Power Point Tracking) algorithm to maximize the power generation efficiency of solar panels.

[0064] 2) Information display:

[0065] Flashing warning light: composed of high-power LED modules, light intensity adjustable, with multiple flashing modes (such as 0.5Hz, 1Hz, 2Hz, etc.);

[0066] Loudspeaker: programmable sound source chip, supports multiple languages and multiple warning voice modes, and can adjust the volume according to the noise environment;

[0067] Dot matrix LED display screen: such as 16x16 or 32x32 dot matrix, supports text, symbols and simple animation scrolling, and has a brightness sensor for automatically reducing brightness to prevent glare in night or tunnel environment.

[0068] 3) Wireless communication: 4G / 5G module or NB-IoT module, used for data interaction with the background control platform, supporting TCP / IP, MQTT or other protocols;

[0069] Remote firmware upgrade (FOTA): can upgrade the firmware of the road condition light, configure parameters and diagnose faults in the background, reducing on-site maintenance cost.

[0070] 2. Front-end acquisition module

[0071] High-speed camera: resolution not less than 720p, support low-illumination or infrared imaging, frame rate ≥25FPS, optional automatic aperture / optical zoom, adapt to day and night and complex lighting environment;

[0072] Vehicle detector: based on microwave, radar, geomagnetic or video detection technology, real-time measurement of lane occupancy, vehicle speed, headway, vehicle type classification, etc.

[0073] Weather sensor: including visibility sensor, temperature and humidity sensor, rainfall sensor, anemometer, etc., to realize early monitoring of severe weather such as heavy rain, heavy snow, fog, etc.

[0074] Data acquisition gateway: multiple front-end devices can be aggregated through CAN bus, RS485 or Ethernet to nearby edge computing unit or gateway, and transmitted to the back-end through cellular network or optical fiber, realizing distributed acquisition and edge preprocessing (such as video encoding, data encryption, etc.).

[0075] 3. Back-end control center / cloud deployment

[0076] Server configuration: GPU server or high-performance CPU cluster can be used to support large-scale video / sequential data deep learning inference; if cooperating with cloud vendors, AI inference instances on the cloud can be used.

[0077] Database and storage: distributed database (HBase) or time series database (InfluxDB) is used to store sensor and vehicle detector data; video frames or recordings can be stored in a distributed file system (such as HDFS) for subsequent offline training or post-analysis.

[0078] Business logic middleware: Kafka, RabbitMQ, etc. message queue can be used to realize high-speed data channel, asynchronous processing or real-time stream processing (Stream Processing) of multi-channel video, sensor data and navigation feedback.

[0079] 4. External service linkage system

[0080] Navigation platform integration: use unified API interface (RESTful) to interface with Gaode Map or other mainstream navigation services, exchange road condition information, accident location, congestion index, etc.

[0081] Video monitoring platform interface: through ONVIF protocol or special SDK, it can be connected with existing monitoring platform, pop-up window prompt of accident event, danger warning on monitoring host, so as to make emergency dispatch for on-duty personnel.

[0082] Traffic management department / police command system: Establish a secure connection through VPN or dedicated line to realize emergency situation sharing, such as remotely closing a ramp, remotely calling a police car or rescue vehicle, etc.

[0083] II. Multi-source data fusion module

[0084] For multi-modal preprocessing and alignment of feedback data of high-speed cameras, weather sensors, vehicle detectors and navigation platforms, an innovative algorithm combining dynamic neighborhood mapping and adaptive time-window interpolation is proposed.

[0085] Among them, dynamic neighborhood mapping: makes the system pay more attention to high-risk section data, and improves the detection sensitivity of accident impact. Adaptive time-window interpolation: solves the problem of multi-frequency and multi-modal data alignment, and ensures that the fused features have high spatio-temporal consistency and robustness.

[0086] 1. Dynamic neighborhood mapping

[0087] 1) Highway logical section division

[0088] The entire highway network is divided into several sections {S1, S2, …, Sn} according to the lane and geographical location, and each section corresponds to one or more monitoring cameras and vehicle detector sensors.

[0089] For the spatial adjacency relationship of sections S i and S j , define the initial adjacency matrix A∈R n×n :

[0090]

[0091] 2) Dynamic neighborhood weight update

[0092] When an accident or abnormal traffic occurs in a section, the neighbor weight of the corresponding section is temporarily increased. In matrix A, a dynamic weight factor α i，j (t) can be set, so that:

[0093]

[0094] Among them, is the initial adjacency relationship, and β i，j (t) is calculated according to real-time traffic fluctuations, historical accident probabilities and other factors, which can be calculated in the following way:

[0095]

[0096] where ΔFlow i，j (t) represents the rate of change of the flow of section Si and its neighbor Sj at the latest time, Θ flow is the threshold value, and γ is the adjustment coefficient.

[0097] 3) Use of adjacency matrix during fusion

[0098] When performing data alignment or subsequent feature extraction, the updated A(t) is used to determine the neighbor section data that needs to be focused on during fusion, improving the sensitivity to nearby accidents or congestion spread.

[0099] 2. Adaptive Time-Window Interpolation

[0100] 1) Multi-source data buffering and timestamp collection

[0101] A sliding buffer queue is maintained for each section / camera / sensor dimension, and the elements in the queue are sorted by timestamp;

[0102] The timestamp precision can be up to seconds or even finer (milliseconds), ensuring that high-frequency data (video frames) and low-frequency data (weather, vehicle detectors) can be reasonably aligned.

[0103] 2) Sliding time window

[0104] Set a basic time window ΔT (5 seconds) to extract data for the interval [t-ΔT, t] before time t;

[0105] If individual sensor data is missing or sampling is inaccurate within the window, linear interpolation or Kalman filter-based prediction compensation is used:

[0106]

[0107] where, x miss (t): the missing data point at time t, which needs to be estimated by interpolation;

[0108] x obs (t-δ): the actually observed data point at time t-δ;

[0109] x obs (t-2δ): the actually observed data point at time t-2δ;

[0110] δ: the standard sampling interval of data. For example, if a sensor samples once every 5 seconds, then δ = 5s;

[0111] δ eff : interpolation step, representing the time offset of the missing data, satisfying 0 ≤ δeff ≤ δ;

[0112] If δ eff = δ, it means standard equidistant interpolation.

[0113] If δ eff < δ, it means the actual interpolation point is closer to the previous observation point.

[0114] 3) Noise detection and weight assignment

[0115] Anomaly detection is performed on sensor data (such as three times standard deviation method, DBSCAN clustering, etc.), if noise value is detected, its weight in the final fusion is reduced; a weight function w i (t) ∈ [0, 1] can be defined to represent the credibility.

[0116] 4) Fusion data output

[0117] Finally, in each time window, the aligned multi-modal feature set {X video , X flow , X weather , X nav} is obtained, and according to:

[0118]

[0119] The definitions of each parameter are as follows:

[0120] X fusion (t): the fusion feature vector calculated at time t, representing the weighted comprehensive result of different data sources;

[0121] D: data source set, containing all data sources that need to be fused (such as cameras, vehicle detectors, weather sensors, navigation platforms, etc.).

[0122] X i (t): the original feature vector from data source i at time t. For example:

[0123] If i represents a high-speed camera, X i (t) is the feature extracted from the video frame;

[0124] If i represents a weather sensor, X i (t) is the environmental feature such as temperature, humidity, wind speed, etc.

[0125] If i represents a vehicle detector, X i (t) is the index such as flow, lane occupancy, average speed, etc.

[0126] w i(t): the dynamic weight given to data source i at time t, used to adjust the influence degree of each data source. The weight usually satisfies:

[0127]

[0128] The fusion feature is input into a next-stage spatio-temporal feature extraction network.

[0129] Three, multi-layer spatio-temporal feature extraction network

[0130] The present application proposes a structure of "CNN+Multi-Head Self-Attention+Spatio-Temporal Fusion Gate", to simultaneously mine spatio-temporal dependence and semantic information from video frames and numerical features such as sensors. It can comprehensively use video and vehicle detectors and other data in complex lighting and extreme weather environments, and deeply mine accident clues; the spatio-temporal fusion gate effectively integrates sensor / navigation platform feedback and video features, and is particularly suitable for improving recognition accuracy in night or blocked camera view conditions.

[0131] 1. CNN convolution feature extraction

[0132] 1) Video sequence input

[0133] Frame the video frames in the time window Standardize (such as subtract the mean and divide by the standard deviation) and input into CNN; if ResNet is used as the backbone, 3x3 convolution + BN + ReLU can be performed in the front section, repeated for several layers, and residual blocks are inserted at the feature map size reduction.

[0134] 2) Feature map output

[0135] The output shape can be [B, T, C', H', W'], where B is the batch size, T is the time dimension, C' is the channel number, H', W' is the spatial resolution after downsampling;

[0136] In the night or bad weather, the attention auxiliary module (CBAM) can be inserted to enhance the regional significant features.

[0137] 2. Multi-Head Self-Attention

[0138] 1) Sequence mapping

[0139] The CNN output is unfolded in the spatio-temporal dimension as a sequence Z∈R (T×H′×W′)×C′ , and is concatenated or passed through an MLP to upgrade / downgrade dimensions with sensor fusion features X sensor (t) to obtain Z'∈R L×d(Where L = T x H' x W', d is the feature dimension)

[0140] 2) Multi-head attention formula

[0141] Multiply Z' by trainable weight matrix W Q ,W K ,W V Get Q, K, V:

[0142] Q = Z'W Q , K = Z'W K , V = Z'W V ,

[0143] Attention calculation of each head:

[0144]

[0145] Where d k is the number of columns of Q or K; after concatenation or weighting of multi-head output:

[0146] Z attn = Concat(head1,..., head h )W O ,

[0147] h is the number of multi-heads, W O is the output projection matrix.

[0148] 3) Spatio-Temporal Fusion Gate

[0149] Introduce a gating mechanism to re-fuse the video attention output and sensor features. Let the gate function G be:

[0150] G = σ(W g [Z attn ; M] + b g ),

[0151] G ∈ R d : Gate weight vector, each element is between [0, 1], controlling the weight of attention feature Z attn and numerical feature M when fusing.

[0152] σ(.): Sigmoid activation function, mapping the calculated gate value to [0, 1], formula: So that G can play a dynamic weighting role.

[0153] W g ∈ R d×2d : Trainable weight matrix, mapping the concatenated features to the same dimension as G.

[0154] b g ∈R d : trainable bias vector.

[0155] Final fusion:

[0156] Z fused = G☉Z attn + (1-G)☉M.

[0157] Z fused ∈R d : fused final feature, combining video information and sensor information.

[0158] ⊙\: element-wise multiplication (Hadamard product), representing gating:

[0159] When G≈1, Zattn dominates;

[0160] When G≈0, M dominates;

[0161] When G is between 0 and 1, it represents the fusion of both.

[0162] 3. Feature normalization and residual connection

[0163] After multi-head attention, Layer Normalization or Batch Normalization is usually done and connected with the input sequence in residual connection, the formula is as follows:

[0164] Z final = LayerNorm(Z fused + Z').

[0165] Z final ∈R d : final spatio-temporal feature representation, used for subsequent dynamic risk assessment and prediction modules;

[0166] Z' ∈R d : original input feature, added to Z fused in residual connection to preserve original information;

[0167] LayerNorm (Layer Normalization): normalize each channel of the feature vector:

[0168]

[0169] where:

[0170]

[0171] γ, β are learnable parameters to adjust the scale and translation after normalization.

[0172] The output tensor is the result of multi-layer spatiotemporal feature extraction, which is used in the subsequent dynamic risk assessment and prediction module.

[0173] IV. Dynamic Risk Assessment and Prediction Module

[0174] Based on Long Short-Term Memory (LSTM) or Transformer architectures, this system incorporates factors such as weather changes, traffic flow fluctuations, and historical accident templates to score potential accidents or anomalies and generate warning levels and diversion strategies for different times and road segments. Each road segment maintains its own hidden state and supports neighborhood information exchange, enabling a more flexible and detailed depiction of the local dynamics of the highway network. Accident similarity matching incorporates historical accident experience into risk calculations, providing better identification capabilities for rare / complex accident types, and can continuously learn and update the template library based on new accidents.

[0175] 1. Partitioned Memory Cell

[0176] 1) Segmented initialization

[0177] Divide the highway into n segments (corresponding to the aforementioned neighborhood mapping), and maintain an LTMCell or Transformer hidden state h for each segment. i , i∈{1,2,…,n}.

[0178] Each segment has an independent memory unit, which can be represented as:

[0179] (h i (t), c i (t))=LSTM i (Z final (t),h i (t-1),c i (t-1));

[0180] If the Transformer is used, it combines multi-head attention with the previous hidden state for computation. The structure is different, but the idea is similar.

[0181] 2) Neighborhood interaction

[0182] To address the impact of cross-segment accidents or congestion, after each update, limited information exchange of hidden states is performed between adjacent segments based on the dynamic neighborhood mapping matrix A(t). This can be defined as follows:

[0183]

[0184] in Indicates segment S i The set of neighbors, α i，j(t) comes from the aforementioned neighborhood dynamic weights. This allows adjacent road segments to quickly coordinate in response to major accidents.

[0185] 3) Merge output

[0186] After all segments have been updated, concatenation or weighting yields the overall representation:

[0187] H(t)=[h′1(t);h′2(t);...;h′ n (t)].

[0188] This representation includes both the current moment's video and sensor features, and also reflects the interactions between different segments.

[0189] 2. Accident Similarity Matching

[0190] 1) Historical accident template library

[0191] Beforehand, collect various common accidents (such as rear-end collisions, rollovers, multi-vehicle collisions, extreme weather accidents, etc.) to form a template vector set T = {t1, t2, ..., tm}, and record their typical spatiotemporal patterns and traffic parameter statistics (such as visibility, traffic flow thresholds, etc.).

[0192] 2) Similarity calculation

[0193] After obtaining H(t), calculate its relationship with each accident template t. k Similarity:

[0194]

[0195] The parameters are defined as follows:

[0196] sim(H(t),t k ): Vectors H(t) and t k The cosine similarity between the two traffic patterns ranges from -1 to 1. It is used to measure the similarity between the current traffic state H(t) and the historical accident patterns t. k The similarity between them.

[0197] If sim(H(t),t k If )≈1, it means that the current state is very similar to the accident k, and a similar accident may be happening.

[0198] If sim(H(t),t k The value ≈ 0 indicates that the current state and the accident k have a low correlation.

[0199] If sim(H(t),t k<0, indicating the current state has an opposite pattern to incident k.

[0200] H(t) ∈ R d : Traffic state feature vector at current time t, containing fused features from video, vehicle detectors, weather sensors, etc. Its dimension d depends on the output dimension of the deep learning model.

[0201] tk ∈ R d : k-th historical incident template vector, representing the feature pattern of a typical incident (e.g., rear-end collision, rollover, multi-vehicle pileup), usually based on historical data statistics or deep learning clustering.

[0202] H(t).t k : Vector dot product, calculating the similarity between the current state and the historical incident template:

[0203]

[0204] If the vector angle is small, the dot product is large, indicating high similarity between the two features.

[0205] If the vector angle is close to 90°, the dot product is close to 0, indicating no correlation between the two.

[0206] ||H(t)|| and ||t k ||: L2 norm (i.e., Euclidean norm) of the vector, used for normalization to make similarity calculation independent of feature vector length:

[0207]

[0208] 3) Risk Correction

[0209] To enhance awareness of similar scenarios, a weighted term can be added to the final risk score:

[0210]

[0211] r(t) ∈ [0,1]: Risk score at time t, indicating the likelihood of an accident occurring on the current road segment. The larger the value, the higher the risk of an accident, which is usually used to determine whether to trigger a warning or take traffic control measures.

[0212] W r ∈ R 1×d : Trainable weight vector used to map high-dimensional features H(t) to a scalar risk value.

[0213] H(t) ∈ R d : Traffic state feature vector at current time t, containing fused features extracted from multi-modal data (video, vehicle detectors, weather data, etc.).

[0214] b r ∈R b : trainable bias term to adjust the baseline output of the model.

[0215] σ(.) : Sigmoid activation function to map the risk score to [0,1]:

[0216]

[0217] This ensures that r(t) is always within a reasonable risk score range.

[0218] 3. Multi-task output and diversion strategy

[0219] 1) Multi-task learning

[0220] Define multiple output heads:

[0221] Risk score r(t) ∈ [0,1];

[0222] Congestion / accident level c(t) ∈ {0,1,2,3};

[0223] Diversion strategy d(t) is a classification result based on risk score r(t), traffic flow state f(t), accident severity level c(t), etc. It represents the diversion strategy recommended by the system at time t.

[0224] ① Input feature vector

[0225] X(t) = [r(t), f(t), c(t), X sensor (t)],

[0226] r(t): risk score, range [0,1];

[0227] f(t): traffic flow state (such as flow, speed, lane occupancy, etc.);

[0228] c(t): accident severity level, value {0,1,2,3};

[0229] X sensor (t): other related features, such as weather data, historical accident similarity, etc.

[0230] ② Diversion strategy classification model uses a classification model (Softmax) to map X(t) to get the probability of each strategy:

[0231]

[0232] W k and b k are trainable weights and biases of the classifier, corresponding to the kth strategy respectively.

[0233] The output P(d(t) = k) represents the probability of selecting the kth policy.

[0234] ③ Strategy determination: based on the classification probability P(d(t) = k), the policy with the highest probability is selected as the current diversion decision:

[0235] ④ Strategy definition

[0236] Assume that the policy corresponds to the following categories:

[0237] k = 1: NONE (no special measures)

[0238] k = 2: LIMIT_SPEED (speed limit)

[0239] k = 3: DIVERSION (diversion)

[0240] k = 4: CLOSE_RAMP (close ramp)

[0241] The final strategy d(t) can be specifically represented as:

[0242]

[0243] Joint loss function:

[0244]

[0245] Using historical data with labeled diversion strategies to train the classifier, optimize the weights by minimizing the cross-entropy loss function:

[0246]

[0247] where is the true policy category of the ith sample (One-Hot encoding).

[0248] Dynamic threshold adjustment: according to the priority of different strategies in actual operation, set the minimum activation threshold. For example:

[0249]

[0250] Weighted decision: to reduce the false positive rate, introduce a weight coefficient to weight and correct the strategy probability:

[0251] P adjusted (d(t) = k) = w k ·· P(d(t) = k),

[0252] where wk represents the priority weight of strategy k, for example, give higher weight to severe strategies (such as closing ramps).

[0253] 2) Threshold decision

[0254] Set multi-level thresholds θ1<θ2<θ3 according to risk score r(t), trigger corresponding level of warning or implement diversion strategy when r(t) exceeds a certain threshold.

[0255] 3) Adaptation and online update

[0256] If there are a large number of false positives or false negatives in the actual operation of the system, automatically adjust the template library or threshold θi according to the feedback information, and optimize the model parameters in the next round of offline or online training to ensure long-term stability and accuracy.

[0257] Five, warning execution and linkage

[0258] 1, Road condition information light linkage

[0259] Flash mode setting:

[0260] Low risk warning: intermittent flashing light 0.5Hz;

[0261] Medium risk warning: 1Hz flashing, and play a short voice prompt "congestion ahead, please slow down";

[0262] High risk warning: 2Hz fast flashing, LED screen large font prompt "accident! Please drive carefully", high-pitched loudspeaker cycle play warning words;

[0263] Light and volume control: If the system detects night or poor weather visibility is too low, can automatically improve the LED brightness, increase the voice volume limit, to enhance the visibility and audibility; avoid misbreak accident scene.

[0264] 2, Navigation platform push push mechanism: once the risk score exceeds the threshold, the background generates an event package (location, time, event type, severity), and pushes it to the navigation platform through the API interface; if there are multiple alternative routes, you can carry "suggested diversion route" for platform analysis;

[0265] Flight reminder: the navigation platform analyzes the received diversion or accident information, if the rear vehicle is about to enter the incident section, the navigation APP triggers an emergency voice prompt on the user side and automatically refreshes the route, so that the driver can complete the speed reduction or bypass within a safe distance.

[0266] 3, High-speed video monitoring platform cooperative integration display: the system can automatically pop up the accident picture on the monitoring center main screen, mark the accident location, occurrence time, estimated severity, etc.

[0267] Linkage emergency handling: On-duty personnel can issue emergency commands remotely through manual intervention interfaces, such as closing upstream ramps, dispatching rescue vehicles or police cars to the scene; the system automatically records this manual intervention operation and incorporates it into subsequent model learning.

[0268] VI. Self-learning optimization and post-evaluation

[0269] 1. Multi-angle post-evaluation

[0270] Accuracy / recall rate: Evaluate the accuracy (Precision) and recall (Recall) of accident identification within a certain time period, as well as the false alarm rate (False Alarm Rate);

[0271] Shunt effect: Use high-speed cameras / vehicle detectors to analyze actual congestion duration, queue length, vehicle delay, etc. after the event, compare the possible losses if the shunt is not executed, and quantify the improvement of traffic efficiency of the invention.

[0272] User feedback survey: Collect drivers' satisfaction with warning information, opinions on false alarms or delays through navigation platforms or traffic management departments, and provide references for improving algorithm thresholds and strategies.

[0273] 2. Incremental learning and online transfer learning

[0274] Edge computing and pre-labeling: Detecting suspicious accident segments in video streams can automatically clip and pre-label videos (such as vehicle collisions, abnormal parking) and send them to the background to speed up the efficiency of manual labeling;

[0275] Regular batch training: The background calls historical data to train CNN, LSTM or Transformer models in batches during off-peak hours; if new road segments are connected or new weather patterns appear, the main model is used for small sample fine-tuning at the beginning.

[0276] Model stability maintenance: Use distillation learning, regularization and other methods to avoid "catastrophic forgetting" (Catastrophic Forgetting) to ensure that the system maintains high recognition ability for common and new accident types.

[0277] 3. System health monitoring and fault detection: If a road condition light frequently goes offline or has abnormal power, the system will automatically alert and notify maintenance personnel;

[0278] Performance monitoring: Through the Dash Board in the background, visualize the recent accident detection latency (average <1 second or <2 seconds) and data transmission delay, and timely identify bottlenecks or abnormalities;

[0279] Version management: For deep learning model, version number can be used for management. If the new version has obvious performance decline in real traffic environment, the last stable version can be rolled back to ensure the quality of early warning.

[0280] The above description of the embodiments of the present application has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the application to the precise form described, and many modifications, variations, and alterations are possible in light of the above teachings. It was described what is intended as the general principles of the application, and the illustrated embodiments were meant only to exemplify the application.

Claims

1. A highway safety warning system based on road information lights, characterized by, The system comprises: 1) a front-end acquisition and warning subsystem, comprising a road condition information light powered by solar energy and a front-end acquisition module; a) the road condition information light is equipped with double high-brightness flashing warning lights, a high-pitched loudspeaker and a dot matrix LED display screen, and receives background instructions through a wireless communication module; b) the front-end acquisition module comprises a high-speed camera, a weather sensor, a vehicle detector and various sensing devices for acquiring road traffic state, weather information and vehicle operation data; 2) a deep learning analysis subsystem deployed in the cloud or a background center, comprising: a) a multi-source data fusion module for receiving real-time data fed back by the high-speed camera, the weather sensor, the vehicle detector and the navigation platform, and performing multi-modal data preprocessing and alignment; b) a multi-layer spatiotemporal feature extraction network based on a convolutional neural network (CNN) and a self-attention mechanism (Self-Attention) fusion structure for feature extraction of video frames and time series traffic data to focus on potential accident occurrence areas or abnormal vehicle trajectories; c) a dynamic risk assessment and prediction module based on a long short-term memory network (LSTM) or a Transformer structure, combining weather changes, traffic flow fluctuations and historical accident data to score the risk of possible accidents or abnormalities on the expressway, and automatically generate warning levels and recommended diversion strategies; 3) a communication and control subsystem comprising a data exchange server and a background control platform for receiving risk assessment results from the deep learning analysis subsystem and issuing control instructions to the front-end acquisition and warning subsystem to automatically or semi-automatically adjust the flashing mode of the road condition information light, the LED display content and the form and content of the voice broadcast; 4) an external service linkage subsystem comprising a bidirectional data interaction module with the high-speed video monitoring platform and the navigation platform, which automatically triggers warnings and synchronously pushes corresponding diversion schemes or safe driving prompts to the navigation platform when identifying abnormal information such as traffic congestion, severe weather, traffic accidents or road construction; 5) a self-learning optimization subsystem comprising: a) a historical data incremental learning module for recording and labeling previous accidents, warning information and warning response results, and dynamically updating the deep learning model offline through incremental learning methods; b) a feedback improvement module for real-time evaluation of warning effectiveness based on feedback from on-site monitoring personnel, patrol police and navigation platform users, automatically correcting model parameters or alarm thresholds to improve the identification accuracy and timeliness of abnormal scenarios; 6) an integrated accident and severe weather warning strategy that, when identifying accident, ice and snow, fog and water accumulation target events, combines real-time traffic flow prediction results and diversion strategies to issue warning prompts to the road condition information light and play voice through the high-pitched loudspeaker, while pushing corresponding diversion information on the navigation platform to avoid secondary accidents and optimize traffic organization; The multi-source data fusion module adopts a dynamic neighborhood mapping processing method, which comprises the following steps: a) division of expressway logical sections The entire highway network is divided into several sections {S1, S2, …, Sn} according to lanes and geographical positions, each section corresponding to one or more monitoring cameras and vehicle detector sensors; For the spatial adjacency relation of segments S i and S j , define an initial adjacency matrix A∈R n×n : ； b) Neighborhood dynamic weight update When an accident or abnormal flow occurs in a section, the neighbor weight of the corresponding section is temporarily increased; a dynamic weight factor is set in matrix A So that: ， wherein, is the initial adjacency relation, then according to the real-time flow fluctuation, historical accident probability factors, the calculation is as follows: ， wherein denotes the rate of fast change of the flow at the most recent time instant of the section Si with its neighbor Sj, is a threshold value and γ is a tuning factor. c) Use adjacency matrix when fusing In performing data alignment or subsequent feature extraction, the updated A(t) neighbor segment data that needs to be focused on when fusing a certain segment, improving the sensitivity to the impact of nearby accidents or congestion.

2. The highway safety warning system of claim 1, wherein The front-end acquisition and warning subsystem further comprises a large-capacity energy storage unit and an intelligent power management module, which are used to maintain normal power supply of the system during long-time rainy weather or at night, and to monitor the power state in real time through a remote diagnosis function, and send a failure warning to the background control platform when the power is insufficient.

3. The highway safety warning system of claim 1, wherein The multi-layer spatio-temporal feature extraction network further adopts a multi-head self-attention mechanism to cross-correlate and learn the data of different road sections, different camera perspectives and different time periods, thereby improving the identification capability of accidents in special scenes such as tunnel entrances, sharp turns and service area entrances.

4. The highway safety warning system of claim 1, wherein When detecting abnormal information, the external service linkage subsystem sends real-time road condition data and prediction results to relevant emergency management departments or traffic command centers, triggers emergency plans, including police car dispatching, accident site control and highway entrance closure operation, and realizes multi-department coordination and linkage.

5. The highway safety warning system of claim 1, wherein The system is based on dynamic neighborhood mapping and adaptive time window interpolation algorithm, which enhances the detection sensitivity of high-risk section data and improves the spatio-temporal consistency of multi-modal data.

6. A highway safety warning method based on road condition information lights, applied to the highway safety warning system of claim 1, characterized in that, The method comprises the following steps: 1) Data acquisition: vehicle flow, speed, weather conditions and road video data are obtained by using high-speed cameras, weather sensors and vehicle detector multi-source sensing devices of the front-end acquisition module; 2) Data preprocessing: time alignment, noise filtering and format conversion are performed on the multi-source data, and the preprocessed data is input into the deep learning analysis subsystem; 3) Accident identification: the multi-layer spatio-temporal feature extraction network is used to fuse and analyze video frames and time series traffic data to identify possible accidents or abnormal scenes; 4) Risk scoring and diversion strategy generation: according to the identification result, the risk score is calculated through the dynamic risk assessment and prediction module, and the historical accident distribution, real-time traffic flow and weather change are combined to generate warning levels and corresponding diversion strategies; 5) Information pushing: the warning information, diversion strategy and driving prompt are sent to: a) Road condition information light: control the flashing frequency of the flashing light, the content of the LED text display and the voice prompt; b) Navigation platform: real-time update of accident induction information to guide the rear driver to avoid or detour; c) Highway video monitoring platform and related management departments: assist in traffic induction or emergency disposal; 6) Self-learning optimization: continuously monitor and record the execution results, and correct the deep learning model parameters or alarm thresholds through the self-learning optimization subsystem to improve the identification accuracy and response speed for special road sections or extreme weather.

7. The method of claim 6, wherein, In the data preprocessing process, the heterogeneous data collected by multiple cameras and sensors are dynamically weighted and fused, different spatio-temporal resolutions are set for weather sensor data, vehicle detector data and camera video stream, and the weighting coefficients are adaptively adjusted according to the latest traffic flow and weather changes during the fusion process.

8. The method of claim 6, wherein, In the risk score calculation process, a multi-objective loss function is used to focus on the impact of traffic congestion, abnormal vehicle driving behavior and extreme weather on highway safety, and online optimization is used to reduce false positives and false negatives.

9. The method of claim 6, wherein, After information pushing, the system automatically analyzes the subsequent vehicle deceleration or lane change behavior based on the event-driven mechanism, and returns the analysis results to the deep learning model to update the risk assessment threshold in similar scenarios, improving the response speed of the system to similar abnormal scenarios.

10. The method of claim 6, wherein, Post-event evaluation of the executed warning process and results, including: a) Warning accuracy evaluation: compare the accident warning level issued by the system with the actual accident severity; b) Diversion effect evaluation: compare the vehicle delay time before and after the implementation of the diversion strategy based on traffic flow data and congestion index; c) Model incremental training: label false positives or delayed warning cases based on evaluation results and include them in the historical data incremental learning module to continuously improve the predictive ability of the deep learning model in extreme weather conditions.

Images