Highway safety early warning system and method based on road condition information lamp

Abstract

The invention relates to the technical field of traffic safety and intelligent traffic systems, in particular to a highway safety early warning system and method based on road condition information lamps, and the system comprises a front-end acquisition subsystem, a deep learning analysis subsystem, a communication and control subsystem and an external service linkage subsystem. The front-end acquisition subsystem acquires real-time traffic data, weather information and video streams through equipment such as a solar road condition information lamp, a high-speed camera, a meteorological sensor and a vehicle detector. And the deep learning analysis subsystem adopts multi-source data fusion, a multi-layer spatial-temporal feature extraction network and a dynamic risk assessment module to realize accurate identification and scoring of traffic risks and generate a corresponding shunting strategy. And the communication and control subsystem realizes real-time control on the front-end warning equipment through the background platform and the data server, and pushes early warning information to a navigation platform, a video monitoring platform and a traffic management department. According to the invention, the probability of secondary accidents is effectively reduced, and the safety and traffic efficiency of the expressway are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic safety and intelligent transportation systems, and in particular to a highway safety warning system and method based on road condition information lights, which is applied to various scenarios such as traffic flow monitoring, traffic incident detection, severe weather warning, and intelligent induced diversion. Background Art

[0002] With the rapid development of highways in the modern transportation system, highway traffic flow and vehicle density are increasing. How to effectively monitor the operation of highways and quickly warn of potential dangers has become a key issue in traffic safety and management. Existing highway monitoring usually uses a variety of facilities such as video monitoring, vehicle inspection devices, and meteorological sensors, but most of them work independently and lack perfect data fusion and intelligent analysis capabilities. When a traffic accident occurs or encounters severe weather (such as fog, heavy snow, heavy rain, etc.), there is often a lag in obtaining information about road hazards ahead, and there is a lack of efficient diversion indication means, which can easily lead to emergency braking or congestion of vehicles before accidents or dangerous sections, causing a significant increase in the probability of secondary accidents.

[0003] In recent years, with the development of artificial intelligence and big data technology, traffic incident detection and prediction methods based on deep learning have begun to be applied to intelligent transportation systems. However, due to the characteristics of high speed, large traffic volume, and changeable environment (night, tunnels, mountainous areas, extreme weather, etc.) in highway scenes, traditional rule-based or shallow machine learning algorithms are difficult to obtain sufficient accuracy and real-time performance. At the same time, some existing solutions still use wired mode for equipment power supply, which makes it difficult to achieve effective coverage of the entire road section, especially in remote mountainous areas or around tunnels. In addition, most current systems lack deep fusion and self-learning mechanisms for multi-source data (video streams, traffic flow, meteorological data, navigation feedback, etc.), and cannot be dynamically adjusted according to real-time scenarios and historical accident patterns, resulting in insufficient flexibility in warning accuracy and response strategies. Summary of the invention

[0004] Based on this, in order to improve the accuracy of highway accident identification and diversion and reduce the risk of secondary accidents, the present invention proposes a highway safety warning system based on road condition information lights. The system uses a deep learning model to integrate and analyze multi-source data, and uses solar road condition information lights and wireless communication means to perform real-time linkage warning. Technical solutions. By deploying a multi-layer spatiotemporal feature extraction network and a dynamic risk assessment and prediction module in the cloud or backend center, all-round and all-weather traffic safety warnings and diversion induction on highways can be achieved. This solution can not only effectively improve the accuracy of accident or congestion identification, but also combine weather factors and historical accident distribution to give corresponding risk scores and diversion suggestions for different sections and time periods. After the accident or abnormality is resolved, the model can also be automatically corrected according to the feedback information to achieve self-learning optimization, thereby further enhancing the adaptability and intelligence level of the system.

[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions: A highway safety warning system based on road condition information lights, the system comprising: 1) Front-end collection and warning subsystem, including a road condition information light powered by solar energy and a front-end collection module; a) The road condition information light is equipped with dual high-brightness flashing warning lights, a tweeter and a dot matrix LED display screen, and receives background instructions through a wireless communication module; b) The front-end acquisition module includes a variety of sensor devices such as high-speed cameras, meteorological sensors, and vehicle inspection devices, which are used to collect road traffic conditions, weather information, and vehicle operation data; 2) Deep learning analysis subsystem, deployed in the cloud or backend center, including: a) Multi-source data fusion module, which is used to receive real-time data from high-speed cameras, meteorological sensors, vehicle inspection devices and navigation platforms, and perform multi-modal data preprocessing and alignment; b) A multi-layer spatiotemporal feature extraction network, based on a fusion structure of a convolutional neural network (CNN) and a self-attention mechanism, extracts features from video frames and time-series traffic data to focus on potential accident areas or abnormal vehicle trajectories; c) A dynamic risk assessment and prediction module, based on a long short-term memory network (LSTM) or Transformer structure, combines weather changes, traffic flow fluctuations, and historical accident data to score the risks of possible accidents or anomalies on the highway, and automatically generates warning levels and recommended diversion strategies; 3) Communication and control subsystem, including data exchange server and background control platform, used to receive risk assessment results of deep learning analysis subsystem and issue control instructions to front-end acquisition and warning subsystem to automatically or semi-automatically adjust the flashing mode of traffic information lights, LED display content and voice broadcast form and content; 4) External service linkage subsystem, including a two-way data interaction module with the high-speed video surveillance platform and the navigation platform. When abnormal information such as traffic congestion, bad weather, traffic accidents or road construction is identified, it automatically triggers an early warning and simultaneously pushes the corresponding diversion plan or safe driving tips to the navigation platform; 5) Self-learning optimization subsystem, including: a) Historical data incremental learning module, which records and labels all previous accidents, warning information and warning response results, and dynamically updates the deep learning model through incremental learning method when offline; b) Feedback improvement module, which evaluates the warning effect in real time based on the feedback from on-site monitoring personnel, patrol traffic police and navigation platform users, automatically corrects model parameters or alarm thresholds, and improves the recognition accuracy and timeliness of abnormal scenes; 6) Integrated early warning strategy for accidents and severe weather. When target events such as accidents, ice and snow, fog, and water accumulation are identified, the real-time traffic flow forecast results and diversion strategies are combined to issue warning prompts to the road condition information lights, and voice broadcasts are made through the tweeters. At the same time, the corresponding induced diversion information is pushed on the navigation platform to avoid secondary accidents and optimize traffic organization.

[0006] The highway safety warning system according to claim 1 is characterized in that the front-end acquisition and warning subsystem further includes a large-capacity energy storage unit and an intelligent power management module, which are used to maintain normal power supply to the system during long periods of rainy weather or at night, and monitor the power status in real time through a remote diagnosis function, and send a fault warning to the background control platform when the power is insufficient.

[0007] Preferably, the multi-layer spatiotemporal feature extraction network further adopts a multi-head self-attention mechanism to perform cross-correlation learning on data from different road sections, different camera perspectives and different time periods, so as to improve the ability to recognize accidents occurring in special scenarios such as tunnel entrances, sharp bends, and service area entrances and exits.

[0008] Preferably, when the external service linkage subsystem detects abnormal information, it will send the real-time traffic data and prediction results to the relevant emergency management department or traffic command center, triggering the emergency plan, including police car dispatch, accident scene control and highway entrance closure operation, to achieve multi-department coordination and linkage.

[0009] Preferably, the system is based on dynamic neighborhood mapping and adaptive time window interpolation algorithm to enhance the detection sensitivity of high-risk section data and improve the spatiotemporal consistency of multimodal data.

[0010] Furthermore, the present invention also discloses a highway safety warning method based on road condition information lights, which is applied to the highway safety warning system. The method comprises the following steps: 1) Data acquisition: Use the high-speed camera, meteorological sensor, vehicle inspection device and other multi-source sensing equipment of the front-end acquisition module to obtain vehicle flow, speed, weather conditions and road video data; 2) Data preprocessing: Time alignment, noise filtering and format conversion of multi-source data, and input the preprocessed data into the deep learning analysis subsystem; 3) Accident identification: Use a multi-layer spatiotemporal feature extraction network to fuse and analyze video frames and time-series traffic data to identify possible accidents or abnormal scenes; 4) Risk scoring and diversion strategy generation: Based on the identification results, the risk score is calculated through the dynamic risk assessment and prediction module, and the warning level and corresponding diversion strategy are generated in combination with the historical accident distribution, real-time traffic flow and weather changes; 5) Information push: Send warning information, diversion strategies and driving tips to: a) Traffic information light: control the flashing frequency of the flashing light, LED text display and voice prompt content; b) Navigation platform: update accident guidance information in real time to guide the rear drivers to avoid or detour; c) High-speed video surveillance platform and relevant management departments: assist in traffic guidance or emergency response; 6) Self-learning optimization: Continuously monitor and record the execution results, and correct the deep learning model parameters or alarm thresholds through the self-learning optimization subsystem to improve the recognition accuracy and response speed for special road sections or extreme weather.

[0011] Preferably, in the data preprocessing process, the heterogeneous data collected by multiple cameras and sensors are dynamically weighted and fused, different spatiotemporal resolutions are set for the weather sensor data, vehicle inspection data and camera video stream, and the weighting coefficients are adaptively adjusted according to the latest traffic flow and weather changes during the fusion process.

[0012] Preferably, in the risk score calculation process, a multi-objective loss function is used to simultaneously focus on the degree of traffic congestion, abnormal vehicle driving behavior, and the impact of extreme weather on highway safety, and to reduce false positives and false negatives through online optimization.

[0013] Preferably, after the information is pushed, the system automatically performs statistical analysis on subsequent vehicle deceleration or lane change behaviors based on an event-driven mechanism, and feeds the analysis results back to the deep learning model to update the risk assessment thresholds in similar scenarios, thereby improving the system's response speed to similar abnormal scenarios.

[0014] As a preference, conduct a post-evaluation of the completed early warning process and results, including: a) Warning accuracy assessment: Compare the accident warning level issued by the system with the actual accident severity; b) Diversion effect evaluation: Based on traffic flow data and congestion index, compare the vehicle delay time before and after the diversion strategy is implemented; c) Model incremental training: Mark false alarms or warning delay cases based on the evaluation results, incorporate them into the historical data incremental learning module, and continuously improve the deep learning model's predictive capabilities under extreme road conditions.

[0015] The present invention has the following technical effects due to the adoption of the above technical solution: 1. Significantly improve the accuracy and timeliness of accident detection and early warning: Through the combination of multi-source data fusion (including video, weather, vehicle inspection and navigation platform feedback information, etc.) and deep learning algorithms, the present invention can achieve real-time analysis and early detection of traffic accidents, abnormal vehicle driving, bad weather and other conditions on highways. Compared with traditional solutions based on manual inspection or simple threshold detection, it can more accurately and quickly identify potential risks and issue early warnings, reducing the probability of secondary accidents.

[0016] 2. Realize deep fusion and self-learning optimization of multimodal information: This invention uses a multi-layer spatiotemporal feature extraction network and a dynamic risk assessment and prediction module to fully explore the correlation features in video frames, traffic flow, meteorological data, and historical accident records. After the initial deployment, this solution can collect and integrate newly acquired data at any time, and continuously optimize the deep learning model using incremental learning, online transfer learning, etc., so as to maintain high prediction accuracy and robustness in changing environments such as extreme weather, holiday peaks, and special sections of mountainous roads.

[0017] 3. Provide intelligent diversion and induction solutions to optimize traffic flow organization: The dynamic risk assessment and prediction module can not only score the probability or severity of accidents, but also automatically generate recommended diversion strategies based on road traffic and meteorological conditions. Through real-time linkage with the navigation platform, it guides vehicles to detour or disperse traffic in advance, thereby effectively alleviating main line congestion, improving road traffic efficiency, and reducing the risk of possible queue collisions and secondary accidents.

[0018] 4. Continuous monitoring and feedback mechanism to improve early warning strategy: After an accident or abnormality occurs, the system described in the present invention will feed back the disposal and execution effects (such as actual diversion efficiency, congestion situation, accident loss, etc.) to the self-learning optimization subsystem through the external service linkage subsystem, and dynamically adjust the deep learning model and alarm threshold in real time or offline, continuously improve the early warning accuracy and decision-making quality in the iterative process, and enhance the maintainability and adaptability of the system.

[0019] 5. Implementation of integrated highway safety management: The present invention can be seamlessly connected with the existing highway video surveillance platform and the information system of the traffic management department to form a full-process linkage system from abnormality identification, information release to emergency response. It not only improves the efficiency of single-point accident handling, but also provides reliable data support for subsequent road safety statistical analysis, traffic planning and management decisions, further improving the overall architecture of intelligent highway safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0021] The following is a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, a highway safety warning system based on road condition information lights, the system includes: 1) Front-end acquisition and warning subsystem, including a road condition information light powered by solar energy and a front-end acquisition module; the road condition information light is equipped with dual high-brightness flashing warning lights, a tweeter and a dot matrix LED display screen, and receives background instructions through a wireless communication module; the front-end acquisition module includes a variety of sensor equipment such as high-speed cameras, meteorological sensors, and vehicle inspection devices, which are used to collect road traffic status, weather information and vehicle operation data; 2) Deep learning analysis subsystem, deployed in the cloud or backend center, including: Multi-source data fusion module: receives real-time data from high-speed cameras, meteorological sensors, vehicle inspection devices and their navigation platform feedback data, and performs multi-modal data preprocessing and alignment; Multi-layer spatiotemporal feature extraction network: A fusion structure based on convolutional neural network (CNN) and self-attention mechanism is used to extract features from video frames and time-series traffic data. The self-attention mechanism can focus on potential accident areas or abnormal vehicle driving trajectories under multi-time and multi-view conditions; Dynamic risk assessment and prediction module: Based on the long short-term memory network (LSTM) or Transformer structure, combined with factors such as weather changes, traffic flow fluctuations and historical accident distribution, it can score the risks of possible accidents or anomalies on the highway, and automatically generate warning levels and recommended diversion strategies for different times and sections; 3) Communication and control subsystem, including data exchange server and background control platform, is used to receive risk assessment results of deep learning analysis subsystem, issue control instructions to front-end acquisition and warning subsystem according to the assessment results, and automatically or semi-automatically adjust the flashing mode of traffic information lights, LED display content and the form and content of voice broadcast; 4) External service linkage subsystem, including a two-way data interaction module with the high-speed video surveillance platform and the navigation platform, which is used to automatically trigger an early warning when abnormal information such as traffic congestion, bad weather, traffic accidents or road construction is identified, and the corresponding diversion plan or safe driving prompts are pushed to the navigation platform simultaneously; 5) Feedback improvement module: Based on the feedback from on-site monitoring personnel, patrol police and navigation platform users, the module evaluates the warning effect in real time, automatically corrects the model parameters or alarm thresholds, and improves the recognition accuracy and timeliness of abnormal scenes; 6) Integrated early warning strategy for accidents and severe weather: When target events are identified, including accidents, ice and snow, fog, and waterlogging, the deep learning analysis subsystem combines real-time traffic flow prediction results with diversion strategies, issues warning prompts to road condition information lights, and plays voice through tweeters. At the same time, it pushes corresponding induced diversion information on the navigation platform to effectively avoid secondary accidents and optimize traffic organization.

[0023] 1. System hardware architecture and deployment details 1. Solar traffic information lights 1) Hardware specifications: Solar panels: power generation not less than 50W, conversion rate ≥ 17%; Battery: Depending on the sunshine duration in different regions, the capacity is usually 20Ah to 60Ah; Environmental adaptability: Meet IP65 or above protection level, can work stably in -20℃~+60℃ environment; Intelligent power supply management: Use the MPPT (Maximum Power Point Tracking) algorithm to maximize the power generation efficiency of solar panels.

[0024] 2) Information display: Flashing warning light: It is composed of high-power LED modules with adjustable luminous intensity and multiple flashing modes (such as 0.5Hz, 1Hz, 2Hz, etc.); Tweeter: Programmable sound source chip, supports multiple languages ​​and multiple warning voice modes, and can adaptively adjust the volume according to the noise environment; Dot matrix LED display: such as 16×16 or 32×32 dot matrix, supports text, symbols and simple animation scrolling, and has a brightness sensor to automatically reduce the brightness to prevent glare at night or in tunnel environments.

[0025] 3) Wireless communication: 4G / 5G module or NB-IoT module, used for data interaction with the background control platform, supporting TCP / IP, MQTT or other protocols; Remote firmware upgrade (FOTA): The road condition lights can be upgraded, parameters configured and fault diagnosed in the background, reducing on-site maintenance costs.

[0026] 2. Front-end acquisition module High-speed camera: resolution no less than 720p, support for low-light or infrared imaging, frame rate ≥ 25FPS, optional automatic aperture / optical zoom, adaptable to daytime, nighttime and complex lighting environments; Vehicle detector: Based on microwave, radar, geomagnetic or video detection technology, it can measure lane occupancy, vehicle speed, headway, vehicle type classification, etc. in real time; Meteorological sensors: including visibility sensors, temperature and humidity sensors, rainfall sensors, wind speed and direction sensors, etc., to achieve early monitoring of severe weather conditions such as heavy rain, heavy snow, and fog; Data acquisition gateway: Multiple front-end devices can be aggregated to a nearby edge computing unit or gateway via CAN bus, RS485 or Ethernet, and transmitted to the backend via cellular network or optical fiber to achieve distributed acquisition and edge preprocessing (such as video encoding, data encryption, etc.).

[0027] 3. Backend control center / cloud deployment Server configuration: GPU servers or high-performance CPU clusters can be used to support deep learning reasoning of large-scale video / time series data; if working with cloud vendors, cloud-based AI reasoning instances can be used; Database and storage: Distributed databases (HBase) or time series databases (InfluxDB) are used to store sensor and vehicle inspection data; video frames or recordings can be stored in distributed file systems (such as HDFS) for subsequent offline training or post-analysis; Business logic middleware: Kafka, RabbitMQ and other message queues can be used to implement high-speed data channels, and asynchronously process or perform real-time stream processing (Stream Processing) on ​​multiple videos, sensor data and navigation feedback.

[0028] 4. External service linkage system Navigation platform integration: Use a unified API interface (RESTful) to connect to Amap or other mainstream navigation services, and exchange traffic information, accident locations, congestion index, etc. in both directions; Video surveillance platform connection: It can be connected to the existing monitoring platform through ONVIF protocol or dedicated SDK, and pop-up windows will be displayed on the monitoring host to prompt accidents and danger warnings, so that the on-duty personnel can make emergency dispatches; Traffic management department / traffic police command system: Establish a secure connection through VPN or dedicated line to achieve emergency situation sharing, such as remote closure of ramps, remote calling of police cars or rescue vehicles, etc.

[0029] 2. Multi-source data fusion module For multimodal preprocessing and alignment of feedback data from high-speed cameras, meteorological sensors, vehicle inspection devices and navigation platforms, an innovative algorithm combining dynamic neighborhood mapping and adaptive time-window interpolation is proposed.

[0030] Among them, dynamic neighborhood mapping: enables the system to pay more attention to high-risk section data and improve the detection sensitivity of accident impact. Adaptive time window interpolation: solves the problem of multi-frequency and multi-modal data alignment and ensures that the fusion features have high temporal and spatial consistency and robustness.

[0031] 1. Dynamic Neighborhood Mapping 1) Logical section division of expressways The entire highway network is divided into several sections {S1, S2, …, Sn} according to lanes and geographical locations. Each section corresponds to one or more surveillance cameras and vehicle inspection sensors.

[0032] For segment S i and S j The spatial adjacency relationship of , defines the initial adjacency matrix A∈R n×n : 2) Dynamic weight update of neighborhood When an accident or traffic anomaly occurs in a certain section, the neighbor weight of the corresponding section is temporarily increased. A dynamic weight factor α can be set in the matrix A i，j (t), such that: in, is the initial adjacency relationship, β i，j (t) can be calculated based on factors such as real-time traffic fluctuations and historical accident probabilities as follows: Where ΔFlow i，j(t) represents the rapid change rate of the traffic between segment Si and its neighbor Sj at the latest moment, Θ flow is the threshold, and γ is the adjustment coefficient.

[0033] 3) Using the adjacency matrix when merging When performing data alignment or subsequent feature extraction, the neighboring segment data that needs to be focused on during fusion of a certain segment will be determined based on the updated A(t), thereby increasing sensitivity to nearby accidents or congestion.

[0034] 2. Adaptive Time-Window Interpolation 1) Multi-source data buffering and timestamp collection A sliding buffer queue is maintained for each segment / camera / sensor dimension, and the elements in the queue are sorted by timestamp; The timestamp accuracy can be down to the second level or even finer (milliseconds), ensuring that high-frequency data (video frames) and low-frequency data (weather, vehicle inspection devices) are properly aligned.

[0035] 2) Sliding time window Set a basic time window ΔT (5 seconds) and extract data from the interval [t-ΔT, t] before time t; If individual sensor data within the window is missing or sampled inaccurately, linear interpolation or prediction compensation based on Kalman filtering is used: Among them, x miss (t): the missing data point at time t, the value to be estimated by interpolation; x obs (t-δ): the data point actually observed at time t-δ; x obs (t-2δ): the data point actually observed at time t-2δ; δ: standard sampling interval of data. For example, if a sensor samples once every 5 seconds, then δ = 5s; δ eff : interpolation step, indicating the time offset of the current missing data, satisfying 0≤δ eff ≤δ; If δ eff =δ, it means standard equal-interval interpolation; If δ eff <δ, it means that the actual interpolation point is closer to the previous observation point.

[0036] 3) Noise detection and weight allocation Perform anomaly detection on sensor data (such as triple standard deviation method, DBSCAN clustering, etc.). If a noise value is detected, reduce its weight in the final fusion; a weight function w can be defined i (t)∈[0,1] to represent the credibility.

[0037] 4) Fusion data output Finally, the aligned multimodal feature set {X video , X flow , X weather , X nav}, and can be based on: The definitions of the parameters are as follows: X fusion (t): fusion feature vector calculated at time t, representing the weighted comprehensive result of different data sources; D: Data source collection, including all data sources that need to be fused (such as cameras, vehicle inspection devices, weather sensors, navigation platforms, etc.).

[0038] X i (t): The original feature vector from data source i at time t. For example: If i represents a high-speed camera, then X i (t) is the feature extracted from the video frame; If i represents a meteorological sensor, then X i (t) is environmental characteristics such as temperature, humidity, and wind speed; If i represents a vehicle detector, then X i (t) is the flow rate, lane occupancy, average speed and other indicators; w i (t): The dynamic weight assigned to data source i at time t, which is used to adjust the influence of each data source. This weight usually satisfies: The fused features are input into the spatiotemporal feature extraction network of the next stage.

[0039] 3. Multi-layer spatiotemporal feature extraction network The present invention proposes a "CNN+Multi-Head Self-Attention+Spatio-Temporal Fusion Gate" structure to simultaneously mine spatio-temporal dependency and semantic information from numerical features such as video frames and sensors. It can comprehensively use a variety of data such as videos and vehicle inspection devices to deeply mine accident clues in complex lighting and extreme weather environments; the spatio-temporal fusion gate effectively integrates sensor / navigation platform feedback and video features, and is particularly suitable for improving recognition accuracy at night or when the camera's field of view is blocked.

[0040] 1. CNN convolution feature extraction 1) Video sequence input Extract the video frames within the time window Normalize (such as subtracting the mean, dividing by the standard deviation) and send it to CNN; if ResNet is used as the backbone, 3×3 convolution + BN + ReLU can be performed in the front stage, repeated several layers, and a residual block can be inserted where the feature map size is halved.

[0041] 2) Feature map output The output shape can be [B, T, C′, H′, W′], where B is the batch size, T is the temporal dimension, C′ is the number of channels, and H′, W′ are the spatial resolutions after downsampling; At night or in bad weather, a CBAM can be inserted to enhance area salient features.

[0042] 2. Multi-Head Self-Attention 1) Sequence Mapping Expand the CNN output in the spatiotemporal dimension into a sequence Z∈R (T×H′×W′)×C′ , and then fused with the sensor feature X sensor (t) Do concatenation or MLP dimensionality increase / reduction to obtain Z′∈R L×d (where L = T × H′ × W′, and d is the feature dimension).

[0043] 2) Multi-Head Attention Formula Multiply Z′ by the trainable weight matrix W Q ,W K ,W V Get Q, K, V: Q=Z′W Q , K=Z′W K , V=Z′W V , Attention calculation for each head: where d kis the number of columns of Q or K; after concatenating or weighting multiple outputs, we get: Z attn =Concat(head1, ..., head h )W O , h is the number of long positions, W O is the output projection matrix.

[0044] 3) Spatio-Temporal Fusion Gate The gating mechanism is introduced to fuse the video attention output with the sensor features again. Let the gating function G be: G=σ(W g [Z attn ;M]+b g ), G∈R d : Gate weight vector, each element is between [0,1], controlling the attention feature Z attn and the weight of the numerical feature M during fusion.

[0045] σ(.): Sigmoid activation function, which maps the calculated gate value to [0,1]. Formula: This allows G to play a dynamic weighting role.

[0046] W g ∈R d×2d : A trainable weight matrix that maps the concatenated features to the same dimension as G.

[0047] b g ∈R d : Trainable bias vector.

[0048] Final Fusion: Z fused =G☉Z attn +(1-G)☉M.; Z fused ∈R d : The final feature after fusion contains both video information and sensor information.

[0049] ⊙\: Element-wise multiplication (Hadamard product), representing the gating effect: When G ≈ 1, Zattn dominates; When G≈0, M dominates; When G is between 0 and 1, it indicates that the two are fused.

[0050] 3. Feature Normalization and Residual Connection After multi-head attention, Layer Normalization or Batch Normalization is usually performed and connected with the input sequence residual. The formula is as follows: Z final =LayerNorm(Z fused + Z′). Z final ∈R d : The final spatiotemporal feature representation is used in the subsequent dynamic risk assessment and prediction module; Z′∈R d : Original input features, connected with Z in the residual connection fused Add to preserve the original information; LayerNorm (layer normalization): Normalize each channel of the feature vector: in: γ, β are learnable parameters used to adjust the scale and translation after normalization.

[0051] The output tensor is the result of multi-layer spatiotemporal feature extraction, which is used in the subsequent dynamic risk assessment and prediction module.

[0052] 4. Dynamic Risk Assessment and Prediction Module Based on the long short-term memory network (LSTM) or Transformer structure, factors such as weather changes, traffic flow fluctuations, and historical accident templates are taken into consideration to score possible accidents or anomalies and generate warning levels and diversion strategies at different times and sections. Each section maintains its hidden state separately and supports neighborhood information interaction, which can more flexibly and finely depict the local dynamics of the highway network; accident similarity matching incorporates historical accident experience into risk calculations, has better recognition capabilities for rare / complex accident types, and can continuously self-learn and update the template library based on new accidents.

[0053] 1. Partitioned Memory Cell 1) Segment initialization Divide the highway into n sections (corresponding to the aforementioned neighborhood mapping), and maintain an LSTMCell or Transformer hidden state h for each section i , i∈{1,2,…,n}.

[0054] Each segment has an independent memory unit, which can be expressed as: (h i (t), c i (t)) = LSTMi (Z final (t),h i (t-1),c i (t-1)); If Transformer is used, multi-head attention is combined with the hidden state of the previous step for comprehensive calculation. The structure is different but the idea is similar.

[0055] 2) Neighborhood Interaction In order to deal with cross-section accidents or congestion, after each update, limited information exchange of hidden states is performed between adjacent sections according to the dynamic neighborhood mapping matrix A(t). It can be defined as: in Indicates segment S i The neighbor set of α i，j (t) The dynamic weight of the neighborhood mentioned above allows adjacent road sections to quickly coordinate with each other in the event of a major accident.

[0056] 3) Merge output After all segments are updated, they are concatenated or weighted to get the overall representation: H(t)=[h′1(t);h′2(t);...;h′ n (t)]. This representation not only includes the summary of video and sensor features at the current moment, but also reflects the mutual influence between each segment.

[0057] 2. Accident Similarity Matching 1) Historical accident template library Various common accidents (such as rear-end collisions, rollovers, multi-vehicle collisions, extreme weather accidents, etc.) are collected in advance to form a template vector set T = {t1, t2, ..., tm}, and their typical spatiotemporal patterns and traffic parameter statistics (such as visibility, traffic flow threshold, etc.) are recorded.

[0058] 2) Similarity calculation After obtaining H(t), calculate its difference with each accident template t k Similarity: The definitions of the parameters are as follows: sim(H(t),t k ): vector H(t) and t k The cosine similarity between the two is in the range of [-1,1]. It is used to measure the current traffic state H(t) and the historical accident mode t k The similarities between.

[0059] If sim(H(t),t k )≈1, indicating that the current state is very similar to accident k and a similar accident may be occurring.

[0060] If sim(H(t),t k )≈0, indicating that the current state has a low correlation with accident k.

[0061] If sim(H(t),t k )<0, indicating that the current state and accident k have opposite patterns.

[0062] H(t)∈R d : The traffic state feature vector at the current time t, which contains the fusion features from video, vehicle detector, meteorological sensor and other data. Its dimension d depends on the output dimension of the deep learning model.

[0063] tk∈R d : The kth historical accident template vector represents the characteristic pattern of a typical accident (such as rear-end collision, rollover, and multi-vehicle collision), which is usually obtained based on historical data statistics or deep learning clustering.

[0064] H(t).t k :Vector dot product (Dot Product), calculate the similarity between the current state and the historical accident template: If the vector angle is small, the dot product is large, indicating that the two features are more similar.

[0065] If the angle between the vectors is close to 90°, the dot product is close to 0, indicating that the two are unrelated.

[0066] ||H(t)|| and ||t k ||: L2 norm of the vector (i.e. Euclidean norm), used for normalization so that the similarity calculation is not affected by the length of the feature vector:

[0067] 3) Risk Correction To enhance awareness of similar scenarios, weighted items can be added to the final risk score: r(t)∈[0,1]: Risk score at time t, indicating the possibility of an accident on the current road section. The larger the value, the higher the accident risk, which is usually used to decide whether to trigger an early warning or take traffic control measures.

[0068] W r ∈R 1×d: A trainable weight vector used to map the high-dimensional feature H(t) to a scalar risk value.

[0069] H(t)∈R d : The traffic state feature vector at the current time t, which contains fusion features extracted from multimodal data (video, vehicle detector, meteorological data, etc.).

[0070] b r ∈R b : A trainable bias term used to adjust the baseline output of the model.

[0071] σ(.): Sigmoid activation function, which maps the risk score to between [0,1]: This ensures that r(t) is always within a reasonable risk score range.

[0072] 3. Multi-task output and diversion strategy 1) Multi-task learning To define multiple output headers: Risk score r(t)∈[0,1]; Congestion / accident level c(t)∈{0,1,2,3}; The diversion strategy d(t) is the classification result calculated based on factors such as risk score r(t), traffic flow state f(t), accident severity level c(t), etc., which represents the diversion strategy recommended by the system at time t.

[0073] ① Input feature vector X(t)=[r(t),f(t),c(t),X sensor (t)], r(t): risk score, ranging from [0,1]; f(t): traffic flow status (such as flow, vehicle speed, lane occupancy, etc.); c(t): accident severity level, ranging from {0,1,2,3}; X sensor (t): Other relevant features, such as weather data, historical accident similarity, etc.

[0074] ② The diversion strategy classification model uses a classification model (Softmax) to map X(t) to obtain the probability of each strategy: W k and b k are the trainable weights and biases of the classifier, corresponding to the k-th strategy respectively.

[0075] The output P(d(t)=k) represents the probability of selecting the kth strategy.

[0076] ③ Strategy determination Based on the classification probability P(d(t)=k), the strategy with the highest probability is selected as the current diversion decision: ④Strategy definition Assume that the policies correspond to the following categories: k=1:NONE No special measures) k = 2: LIMIT_SPEED (speed limit) k = 3: DIVERSION k = 4: CLOSE_RAMP (close ramp) The final strategy d(t) can be specifically expressed as: Joint loss function: Use historical data with labeled diversion strategies to train the classifier and optimize the weights by minimizing the cross entropy loss function: in is the true strategy category of the i-th sample (One-Hot encoding).

[0077] Dynamic threshold adjustment: The minimum activation threshold can be set according to the priority of different strategies in actual operation. For example: Weighted decision: To reduce the false alarm rate, a weight coefficient can be introduced to perform weighted correction on the strategy probability: P adjusted (d(t)=k)=w k P(d(t)=k), Where wk represents the priority weight of strategy k, for example, a higher weight is given to severe strategies (such as ramp closure).

[0078] 2) Threshold determination According to the risk score r(t), multi-level thresholds θ1<θ2<θ3 are set. When r(t) exceeds a certain threshold, the corresponding level of warning is triggered or the diversion strategy is executed.

[0079] 3) Adaptation and online updating If the system has a large number of false alarms or missed alarms in actual operation, the template library or threshold θi will be automatically adjusted according to the feedback information, and the model parameters will be optimized in the next round of offline or online training to ensure long-term stability and accuracy.

[0080] V. Warning Execution and Linkage 1. Traffic information light linkage Flashing mode setting: Low risk warning: the light flashes intermittently at 0.5Hz; Medium risk warning: flashes at 1Hz and plays a short voice prompt "Congestion ahead, please slow down"; High-risk warning: 2Hz fast flashing, LED screen prompts "Accident! Please drive carefully" in large letters, and the tweeter plays the warning message in a loop; Light and volume control: If the system detects that visibility is too low at night or in bad weather, it can automatically increase the LED brightness and raise the upper limit of voice volume to enhance visibility and audibility, avoiding accidentally entering the accident scene.

[0081] 2. Navigation platform push mechanism: Once the risk score exceeds the threshold, the background generates an event package (location, time, event type, severity) and pushes it to the navigation platform through the API interface; if there are multiple alternative routes, the "suggested diversion route" can be carried for platform analysis; Flight-style reminder: The navigation platform analyzes the diversion or accident information it receives. If the vehicle behind is about to enter the accident section, the navigation APP triggers an emergency voice prompt on the user side and automatically refreshes the route, allowing the driver to slow down or detour within a safe distance.

[0082] 3. High-speed video surveillance platform collaborative integrated display: The system can automatically pop up the accident screen on the main screen of the monitoring center, marking the accident location, occurrence time, estimated severity and other information; Joint emergency response: The on-duty personnel can remotely issue emergency commands through the manual intervention interface, such as closing the upstream ramp, dispatching a rescue vehicle or police car to the scene; the system automatically records this manual intervention operation and incorporates it into subsequent model learning.

[0083] 6. Self-learning optimization and post-evaluation 1. Post-event evaluation from multiple angles Precision / Recall: Evaluate the accuracy (Precision) and recall (Recall) of accident identification within a specific time period, as well as the false alarm rate (False Alarm Rate); Diversion effect: Utilize post-process data analysis of high-speed cameras / vehicle inspection devices to analyze actual congestion duration, queue length, vehicle delays and other indicators, compare the potential losses if diversion is not performed, and quantify the improvement in traffic efficiency brought about by the present invention.

[0084] User feedback survey: Collect drivers’ satisfaction with warning information and their opinions on false alarms or delays through navigation platforms or traffic management departments, and provide reference for improving algorithm thresholds and strategies.

[0085] 2. Incremental learning and online transfer learning Edge computing and pre-labeling: If a suspected accident segment is detected in the video stream, the video can be automatically edited and pre-labeled (such as vehicle collision, abnormal parking) and sent to the backend to speed up manual labeling efficiency; Regular batch training: The backend uses historical data to perform batch training on CNN, LSTM or Transformer models during off-peak hours. If new road sections are added or new weather patterns appear, the backbone model is used in the initial stage to perform small sample fine-tuning.

[0086] Model stability maintenance: Use distillation learning, regularization and other means to avoid "catastrophic forgetting" and ensure that the system maintains high recognition capabilities for both common and new accident types.

[0087] 3. System health monitoring fault detection: If a road condition information light is frequently offline or the power is abnormal, the system will automatically alarm and notify the maintenance personnel; Performance monitoring: Dash Board is used to visualize recent accident detection delays (average <1 second or <2 seconds) and data transmission delays in the background, and bottlenecks or anomalies are discovered in a timely manner. Version management: For deep learning models, version numbers can be used for management. If the new version shows obvious performance degradation in real traffic conditions, it can be rolled back to the previous stable version to ensure the quality of the warning.

[0088] The above is a description of the embodiments of the present invention. Through the above description of the disclosed embodiments, professionals and technicians in the field can implement or use the present invention. Various modifications to these embodiments will be apparent to professionals and technicians in the field. The general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will conform to the widest range consistent with the principles and novelties disclosed herein.

Claims

1. A highway safety warning system based on road condition information lights, characterized in that: The system includes: 1) Front-end collection and warning subsystem, including a road condition information light powered by solar energy and a front-end collection module; a) The road condition information light is equipped with dual bright flashing warning lights, a tweeter and a dot matrix LED display screen, and receives background instructions through a wireless communication module; b) The front-end acquisition module includes a variety of sensor devices such as high-speed cameras, meteorological sensors, and vehicle inspection devices, which are used to collect road traffic conditions, weather information, and vehicle operation data; 2) Deep learning analysis subsystem, deployed in the cloud or backend center, including: a) Multi-source data fusion module, which is used to receive real-time data from high-speed cameras, meteorological sensors, vehicle inspection devices and navigation platforms, and perform multi-modal data preprocessing and alignment; b) A multi-layer spatiotemporal feature extraction network, based on the fusion structure of convolutional neural network (CNN) and self-attention mechanism, extracts features from video frames and time-series traffic data to focus on potential accident areas or abnormal vehicle trajectories; c) Dynamic risk assessment and prediction module, based on long short-term memory network (LSTM) or Transformer structure, combines weather changes, traffic flow fluctuations and historical accident data to score the risks of possible accidents or anomalies on the highway, and automatically generates warning levels and recommended diversion strategies; 3) Communication and control subsystem, including data exchange server and background control platform, is used to receive risk assessment results of deep learning analysis subsystem and issue control instructions to front-end acquisition and warning subsystem to automatically or semi-automatically adjust the flashing mode of traffic information lights, LED display content and voice broadcast form and content; 4) External service linkage subsystem, including a two-way data interaction module with the high-speed video surveillance platform and the navigation platform. When abnormal information such as traffic congestion, bad weather, traffic accidents or road construction is identified, it automatically triggers an early warning and simultaneously pushes the corresponding diversion plan or safe driving tips to the navigation platform; 5) Self-learning optimization subsystem, including: a) Historical data incremental learning module, which records and labels all previous accidents, warning information and warning response results, and dynamically updates the deep learning model through incremental learning method when offline; b) Feedback improvement module, which evaluates the warning effect in real time based on the feedback from on-site monitoring personnel, patrol traffic police and navigation platform users, automatically corrects model parameters or alarm thresholds, and improves the recognition accuracy and timeliness of abnormal scenes; 6) Integrated early warning strategy for accidents and severe weather. When target events such as accidents, ice and snow, fog, and water accumulation are identified, the real-time traffic flow forecast results and diversion strategies are combined to issue warning prompts to the road condition information lights, and voice broadcasts are made through the tweeters. At the same time, the corresponding induced diversion information is pushed on the navigation platform to avoid secondary accidents and optimize traffic organization.

2. The highway safety warning system according to claim 1, characterized in that: The front-end acquisition and warning subsystem further includes a large-capacity energy storage unit and an intelligent power management module, which are used to maintain normal power supply of the system during long periods of rainy weather or at night, and monitor the power status in real time through the remote diagnosis function, and send a fault warning to the background control platform when the power is low.

3. The highway safety warning system according to claim 1, characterized in that: The multi-layer spatiotemporal feature extraction network further adopts a multi-head self-attention mechanism to perform cross-correlation learning on data from different road sections, different camera perspectives and different time periods, so as to improve the ability to identify accidents occurring in special scenarios such as tunnel entrances, sharp bends, and service area entrances and exits.

4. The highway safety warning system according to claim 1, characterized in that: When the external service linkage subsystem detects abnormal information, it will send real-time traffic data and prediction results to the relevant emergency management department or traffic command center, triggering an emergency plan, including police car dispatch, accident scene control and highway entrance closure operations, to achieve multi-department coordination and linkage.

5. The highway safety warning system according to claim 1, characterized in that: The system is based on dynamic neighborhood mapping and adaptive time window interpolation algorithm to enhance the detection sensitivity of high-risk section data and improve the spatiotemporal consistency of multimodal data.

6. A highway safety warning method based on road condition information lights, applied to the highway safety warning system according to claim 1, characterized in that: The method comprises the following steps: 1) Data acquisition: Use the high-speed camera, meteorological sensor, vehicle inspection device and other multi-source sensing equipment of the front-end acquisition module to obtain vehicle flow, speed, weather conditions and road video data; 2) Data preprocessing: Time alignment, noise filtering and format conversion of multi-source data, and input the preprocessed data into the deep learning analysis subsystem; 3) Accident identification: Use a multi-layer spatiotemporal feature extraction network to fuse and analyze video frames and time-series traffic data to identify possible accidents or abnormal scenes; 4) Risk scoring and diversion strategy generation: Based on the identification results, the risk score is calculated through the dynamic risk assessment and prediction module, and the warning level and corresponding diversion strategy are generated in combination with the historical accident distribution, real-time traffic flow and weather changes; 5) Information push: Send warning information, diversion strategies and driving tips to: a) Traffic information light: control the flashing frequency of the flashing light, LED text display and voice prompt content; b) Navigation platform: update accident guidance information in real time to guide the rear drivers to avoid or detour; c) High-speed video surveillance platform and relevant management departments: assist in traffic guidance or emergency response; 6) Self-learning optimization: Continuously monitor and record the execution results, and correct the deep learning model parameters or alarm thresholds through the self-learning optimization subsystem to improve the recognition accuracy and response speed for special road sections or extreme weather.

7. The method according to claim 6, characterized in that In the data preprocessing process, the heterogeneous data collected by multiple cameras and sensors are dynamically weighted and fused, different spatiotemporal resolutions are set for weather sensor data, vehicle inspection data and camera video streams, and the weighting coefficients are adaptively adjusted according to the latest traffic flow and weather changes during the fusion process.

8. The method according to claim 6, characterized in that In the process of risk score calculation, a multi-objective loss function is used to simultaneously focus on the degree of traffic congestion, abnormal vehicle driving behavior, and the impact of extreme weather on highway safety, and to reduce false positives and false negatives through online optimization.

9. The method according to claim 6, characterized in that After the information is pushed, the system automatically conducts statistical analysis on subsequent vehicle deceleration or lane change behaviors based on an event-driven mechanism, and sends the analysis results back to the deep learning model to update the risk assessment thresholds in similar scenarios and improve the system's response speed to similar abnormal scenarios.

10. The method according to claim 6, characterized in that Conduct post-evaluation of the completed early warning process and results, including: a) Warning accuracy assessment: Compare the accident warning level issued by the system with the actual accident severity; b) Diversion effect evaluation: Based on traffic flow data and congestion index, compare the vehicle delay time before and after the diversion strategy is implemented; c) Model incremental training: Mark false alarms or warning delay cases based on the evaluation results, incorporate them into the historical data incremental learning module, and continuously improve the deep learning model's predictive capabilities under extreme road conditions.

Images