Intelligent sensing and early warning system for highway safety events

Abstract

This invention relates to an intelligent perception and early warning system for highway safety incidents. Its architecture includes a device access layer, a data computing layer, a business capability layer, an application service layer, and a user interaction layer. Its core modules include a unified access and management module for intelligent sensing devices, a multimodal data intelligent analysis engine module, a safety incident assessment and early warning generation module, a hierarchical distribution and linkage control module for early warning information, a real-time monitoring and visualization module for traffic operation status, and a closed-loop incident handling and statistical analysis module. This system integrates or interfaces with front-end intelligent sensing devices to perform real-time intelligent analysis of acquired multimodal data, accurately identify safety risks and operational status, generate standardized early warning information, drive a closed-loop management system, and achieve controllable safety risks, quantifiable operational status, and intelligent management processes.

Description

Technical Field

[0001] This invention relates to an intelligent perception and early warning system for highway safety incidents, belonging to the field of intelligent highway operation platforms. Background Technology

[0002] As highway operation and management deepen towards digitalization and intelligence, toll areas (including entrance plazas, exit plazas, tollbooth lanes, and surrounding connecting roads) are crucial nodes for vehicle convergence and diversion, making their safety and traffic order paramount. Traditional management models primarily rely on manual video patrols and post-event review, which inherently suffer from limitations such as "invisibility, incomplete coverage, and slow response." In particular, they cannot proactively and quickly perceive and intervene in sudden safety incidents such as pedestrians and non-motorized vehicles illegally entering, vehicles driving in the wrong direction, abnormal parking, and traffic accidents, resulting in significant safety hazards. Furthermore, management personnel lack precise quantitative perception of the real-time operational status of toll areas, such as traffic density and queue length, making it difficult to make scientific and efficient traffic management decisions. The applicant has conducted research and development to address these issues and has achieved results, which are now being applied for in this patent application. Summary of the Invention

[0003] The main objective of this invention is to address the problems existing in the prior art by providing an intelligent perception and early warning system for highway safety incidents. Based on precise perception technology and intelligent analysis algorithms, it enables all-weather, all-element, and intelligent safety monitoring and operational status perception of the physical space of the toll area, transforming passive response into proactive early warning and experience-based judgment into data-driven decision-making, thereby significantly improving the safety control level and operational management efficiency of highway toll areas.

[0004] The technical solution of this invention to solve its technical problem is as follows:

[0005] A smart perception and early warning system for highway safety incidents includes an architecture layer and a core module; the architecture layer includes:

[0006] The device access layer is used to access the various intelligent sensing devices deployed in the toll area, receive data from each intelligent sensing device, and perform unified management and scheduling; the intelligent sensing devices include high-definition network cameras and Raivision fusion sensing all-in-one machines.

[0007] The data computing layer is used to acquire data from the device access layer and perform intelligent analysis; the intelligent analysis includes video intelligent analysis, radar-visual data fusion analysis, and traffic parameter calculation.

[0008] The business capability layer is used to encapsulate the core business logic of the system; the core business logic includes event analysis and early warning generation service, early warning classification and routing service, traffic situation aggregation service, and early warning distribution adapter;

[0009] The application service layer is used to provide functional services to users.

[0010] The user interaction layer provides a direct interface for users to operate and obtain information.

[0011] The core modules include: a unified access and management module for intelligent sensing devices, located at the device access layer; a multimodal data intelligent analysis engine module, located at the data computing layer; a security event assessment and early warning generation module, located at the business capability layer, used to provide event assessment and early warning generation services; a hierarchical distribution and linkage control module for early warning information, located at the business capability layer, used to provide early warning hierarchical and routing services; a real-time monitoring and visualization module for traffic operation status, located at the business capability layer, used to provide traffic status aggregation services; and a closed-loop event handling and statistical analysis module, located at the user interaction layer.

[0012] The further improved technical solution of this invention is as follows:

[0013] Preferably, the device access layer performs device management, status monitoring, and receives and performs initial parsing of video streams or radar data streams for each intelligent sensing device, converting the original bitstream data into standard data packets for upper-layer processing.

[0014] The intelligent analysis results of the data computing layer include structured event descriptions and traffic parameters, and are output in a standardized JSON format message.

[0015] The warning distribution adapter of the business capability layer is used to distribute warning information to downstream systems through different channels (such as message queues, HTTP interfaces, and signaling systems);

[0016] The application service layer provides functional services including early warning event subscription and query services, real-time video viewing services, traffic situation data services, and system configuration management services.

[0017] The user interaction layer includes a comprehensive monitoring and command screen for panoramic display of toll area dynamics, early warning event distribution, and traffic situation heat map; an early warning and handling workbench for providing on-duty personnel with functions such as early warning information list, details viewing, handling instructions issuance, and handling feedback input; and a system management backend for configuring equipment, managing rules, managing user permissions, and viewing system logs and operation reports.

[0018] Preferably, the workflow of the unified access and management module for the intelligent sensing devices is as follows:

[0019] S1. Supports automatic discovery, batch import, and online registration of smart sensing devices deployed in toll areas;

[0020] S2. Create an independent file for each device, recording its model, IP address, installation location, region, and status; the installation location includes GPS coordinates and orientation angle; the status is one of online, offline, or faulty;

[0021] S3. Provide device configuration templates for batch parameter settings of devices; the parameters include resolution, frame rate, and ROI drawing.

[0022] S4. Continuously monitor the device heartbeat, issue alarms for offline or abnormal devices, and support device maintenance operations; the maintenance operations include remote restart and firmware upgrade.

[0023] Preferably, the multimodal data intelligent analysis engine module is configured with various deep learning models optimized for highway scenarios, specifically including a video analysis model, a radar-visual fusion analysis model, and a traffic parameter calculation model. The video analysis model, based on a convolutional neural network, is used to detect and track various targets in the video footage in real time. These targets include pedestrians, non-motorized vehicles, and various types of motor vehicles. Through temporal behavior analysis, it identifies abnormal behavior patterns of the targets (such as crossing barriers, walking or riding against traffic on the lane, or vehicles remaining stationary in non-stop areas for extended periods). The radar-visual fusion analysis model employs pre-fusion or post-fusion processing strategies to correlate and match radar point cloud targets with video image targets, improving target detection rate and trajectory tracking stability under adverse weather conditions (such as at night, in backlight, rain, snow, or fog). The traffic parameter calculation model, based on tracked vehicle trajectory data, is used to count the number of vehicles passing through a virtual detection line or detection area in real time, calculate the time-averaged speed and spatial-averaged speed, identify the tail position of the queue through the spatial distribution of vehicle queues, and calculate the queue length. All parameters are aggregated and output at preset time intervals (such as 1 minute or 5 minutes).

[0024] More preferably, the multimodal data intelligent analysis engine module adopts deep learning-based multi-target tracking and behavior recognition technology, including: using a tracking algorithm (such as DeepSORT or a better algorithm) to stably and continuously track each target in the complex scene of a toll plaza, and assigning a unique ID to each target; combining the trajectory sequence obtained by tracking, using a spatiotemporal convolutional network or Transformer model to model the trajectory sequence, learning the pattern differences between normal passage and abnormal behavior, thereby achieving high-accuracy recognition of target behavior (such as intrusion, wrong-way movement, loitering, etc.) based on trajectory semantics.

[0025] More preferably, the multimodal data intelligent analysis engine module adopts a front-end fusion perception technology of radar and video, including: adopting a pre-fusion processing strategy to perform spatiotemporal alignment and feature association between radar point clouds and video pixels at the raw data level, using the radar's accurate ranging and velocity information to guide the key areas of video analysis, and using the texture information of the video to classify radar targets, and finally outputting fused target information with both high-precision spatial attributes and accurate category labels.

[0026] Preferably, the security event analysis and early warning generation module is configured with a rule engine. The rules configured in the rule engine are logical combinations based on multiple dimensions, including target type, motion trajectory, speed, location area, and duration. The security event analysis and early warning generation module receives and analyzes the raw target data and behavioral signals output by the multimodal data intelligent analysis engine module, performs advanced semantic understanding according to business rules, determines whether a security event requiring early warning is constituted, and generates corresponding early warning information if it is. The early warning information is stored in a structured manner and specifically includes: a unique event ID, event type, event level, occurrence time, occurrence location, associated evidence snapshots or short video clips, and confidence score. The occurrence location includes device ID and pixel coordinates or geographic coordinates. The security event analysis and early warning generation module also supports compound judgment of continuous and related primary events to generate higher-level events.

[0027] More preferably, the security incident analysis and early warning generation module adopts spatial rule engine technology based on geographic information system, including: digitizing the physical layout of the toll area (such as lane lines, square boundaries, isolation facilities) to construct an electronic fence and logical rule library; when analyzing the type and behavior of the target, comparing it with GIS spatial information in real time and making a judgment.

[0028] Preferably, the workflow of the early warning information hierarchical distribution and linkage control module includes:

[0029] In this module, managers can pre-set different warning levels (such as high, medium, and low) and corresponding response plans for different event types and different occurrence areas (such as the main entrance and ordinary squares). The plan defines the method of pushing the warning: for high-risk warnings (such as pedestrians entering the driving lane), in addition to pop-up windows and strong sound and light prompts on the system interface, the warnings are automatically pushed to the nearest road patrol personnel via SMS or mobile application, and the on-site sound and light alarm devices are automatically controlled to provide voice warnings and flashing lights.

[0030] Preferably, the real-time traffic operation status monitoring and visualization module aggregates traffic parameter data from the multimodal data intelligent analysis engine module in real time, performs regional-level aggregation calculations in the background, and generates intuitive status indicators. The core functions of the real-time traffic operation status monitoring and visualization module include: panoramic traffic flow status display: displaying the traffic flow density of each lane in real time on an electronic map in the form of heat map or color coding; key indicator dashboard: displaying queue length curves, average speed, and lane occupancy rate of key areas in real time; saturation index calculation: calculating the "regional saturation index" (e.g., 0-100%) by combining multiple parameters such as queue length, vehicle density, and traffic speed through an algorithm model, providing managers with a simple basis for judging the degree of congestion; historical status review: supporting the playback of historical status changes over time for post-event evaluation and pattern analysis.

[0031] Preferably, the workflow of the event handling closed-loop and statistical analysis module is as follows:

[0032] After receiving an early warning at the early warning and response workstation, on-duty personnel confirm the warning, assign response tasks (i.e., designate personnel or teams for response), and provide remote guidance via voice intercom or video link. Response personnel provide feedback on the on-site situation and response results via mobile devices. The event response closed-loop and statistical analysis module tracks the entire process of the event from occurrence, early warning, task assignment, response to closure, and records the time points of each stage, automatically calculating efficiency indicators. These efficiency indicators include "early warning response time," "on-site arrival time," and "event response duration." Simultaneously, the event response closed-loop and statistical analysis module provides statistical analysis functions, generating reports by time, region, and event type, analyzing high-incidence periods and locations to optimize deployment strategies and conduct targeted governance.

[0033] This invention, the Intelligent Perception and Early Warning System for Highway Safety Incidents, focuses on the specific scenario of toll areas. By integrating or connecting with front-end intelligent sensing devices (such as radar-visual fusion sensing units and high-definition video cameras), it performs real-time intelligent analysis on the acquired multimodal data, accurately identifies safety risks and operational status, and generates standardized early warning information to drive a closed-loop management system. The core value of this system is reflected in: (1) Controllable safety risks: Through automated and intelligent identification methods, it achieves "second-level discovery and real-time alarm" for various safety incidents, compressing the response time from the occurrence of an incident to the initiation of disposal, and strengthening the safety defense line. (2) Quantifiable operational status: It transforms the traffic operation status of toll areas from vague qualitative descriptions into precise quantitative indicators (such as saturation index), providing objective and real-time data support for management decisions. (3) Intelligent management process: It constructs a complete data closed loop of "perception-analysis-early warning-disposal-evaluation", promoting the upgrade of safety management and operation command from the traditional model that relies on human experience to a modern model based on data intelligence. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall situation in Embodiment 1 of the present invention. Detailed Implementation

[0035] The present invention will be further described in detail below with reference to the embodiments. However, the present invention is not limited to the examples given.

[0036] Example 1

[0037] The intelligent perception and early warning system for highway safety incidents in this embodiment adopts a layered, decoupled, and microservice-based overall architecture to ensure that the system's functional modules are clear, easy to maintain, and expandable; for example... Figure 1As shown, the overall architecture is divided into the device access layer, data computing layer, business capability layer and application service layer from bottom to top. Among them, (1) Device access layer: As the boundary between the system and the physical world, this layer accesses various intelligent sensing devices deployed in the toll area through standard protocols (such as GB / T28181, ONVIF) or dedicated SDKs, mainly including high-definition network cameras, radar-visual fusion sensing all-in-one machines, etc.; and on this basis, it performs device management, status monitoring, video stream / radar data stream reception and primary parsing, converts the original bit stream data into standard data packets that can be processed by the upper layer, and performs unified management and scheduling of multiple video streams. (2) Data Computation Layer: As the "brain" of the system for intelligent analysis, this layer is the core computing unit of the system, deploying multiple parallel AI analysis engines, specifically including: a video intelligent analysis engine, used to decode and compute the incoming video stream in real time, run deep learning models, and realize target detection, tracking, and behavior recognition; a radar-visual data fusion engine, used to process radar point cloud data and perform spatiotemporal fusion with video analysis results to improve the perception accuracy and reliability in complex environments (such as nighttime, rainy and foggy weather); and a traffic parameter calculation engine, used to calculate micro-traffic parameters such as traffic flow, density, speed, and queue length in real time based on the tracked vehicle trajectory data. In this layer, the analysis results of all engines, including structured event descriptions and traffic parameters, are output as standardized JSON format messages. (3) Business Capability Layer: This layer encapsulates the core business logic of the system. Specifically, it includes: Event Analysis and Early Warning Generation Service, which receives the raw analysis results output by the data calculation layer, performs logical judgment based on the preset event judgment rule base (e.g., if a pedestrian's coordinates continuously move from the square to the lane and the speed is below the threshold, it is judged as "pedestrian intrusion"), confirms the event, and generates a standardized early warning event containing information such as event type, location, time, image / video evidence link, and confidence level; Early Warning Grading and Routing Service, which assigns appropriate early warning levels (such as Level 1, Level 2, and Level 3) and specified push routes to early warning events based on the event type and contingency plan; Traffic Situation Aggregation Service, which aggregates and statistically analyzes traffic parameters to generate regional-level situation indicators and visualization data; and Early Warning Distribution Adapter, which distributes the formatted early warning information to downstream systems through different channels (such as message queues, HTTP interfaces, and signaling systems). (4) Application Service Layer: This layer provides functional services to end users. It provides services such as early warning event subscription and query, real-time video access, traffic situation data, and system configuration management in the form of API interfaces and Web services. These services support the user interface and external system integration of the upper layer.(5) User interaction layer: serving as the direct interface for users to operate and obtain information; mainly including: a comprehensive monitoring and command screen, used to display the dynamics of the toll area, the distribution of early warning events and the traffic situation heat map in a panoramic view; an early warning and handling workbench, used to provide on-duty personnel with functions such as early warning information list, details viewing, handling instructions issuance and handling feedback input; and a system management backend, used to configure equipment, manage rules, manage user permissions, view system logs and operation reports.

[0038] The system adopts a lightweight model and edge collaborative computing architecture technology: considering the wide distribution of toll stations, the system's data computing layer supports lightweight model deployment, which can run on edge computing devices, realizing "where the data is, the analysis is there", reducing the occupation and dependence on central bandwidth and improving the overall response speed; at the same time, the central platform retains the ability to perform complex models, global situation analysis and big data mining, forming a collaborative computing system of "real-time edge perception and global central intelligence".

[0039] The core modules of the intelligent perception and early warning system for highway safety incidents in this embodiment include: a unified access and management module for intelligent sensing devices, a multimodal data intelligent analysis engine module, a safety incident judgment and early warning generation module, a hierarchical distribution and linkage control module for early warning information, a real-time monitoring and visualization module for traffic operation status, and a closed-loop and statistical analysis module for incident handling. Details are as follows.

[0040] (1) Unified access and management module for intelligent sensing devices:

[0041] This module, located in the device access layer, is responsible for managing all front-end sensing devices. Specifically, it supports automatic device discovery, batch import, and online registration; it creates an independent profile for each device, recording its model, IP address, installation location (accurate to GPS coordinates and orientation angle), region, and status (online / offline / faulty); it provides device configuration templates for batch parameter settings (such as resolution, frame rate, and ROI drawing); it continuously monitors device heartbeats, issues alarms for offline or abnormal devices, and supports remote device restarts, firmware upgrades, and other maintenance operations.

[0042] (2) Multimodal data intelligent analysis engine module:

[0043] This module, located in the data computation layer, is the core algorithm module for realizing "intelligent perception." It incorporates several deep learning models optimized for highway scenarios, including:

[0044] The video analytics model, based on a convolutional neural network, is used to detect and track pedestrians, non-motorized vehicles, and various types of motorized vehicles in video footage in real time. Through temporal behavior analysis, it identifies abnormal behavior patterns of the targets, such as crossing barriers, walking or riding against traffic in the lane, and vehicles remaining stationary in non-stop areas for extended periods.

[0045] The radar-visual fusion analysis model serves as an important supplement and enhancement to the video analysis model. It utilizes radar to provide accurate distance and velocity information unaffected by lighting conditions. The model employs pre-fusion or post-fusion processing strategies to correlate and match radar point cloud targets with video image targets, significantly improving target detection rate and trajectory tracking stability under adverse weather conditions such as nighttime, backlight, rain, snow, and fog, and effectively reducing false alarms and missed alarms.

[0046] The traffic parameter calculation model is based on stably tracked vehicle trajectory data. It is used to count the number of vehicles passing through in real time within a virtual detection line or detection area, calculate the time average speed and spatial average speed, identify the position of the tail of the queue and calculate the queue length through the spatial distribution of vehicle queues, and aggregate all parameters at configurable time intervals (such as 1 minute or 5 minutes).

[0047] Specifically, this module employs deep learning-based multi-target tracking and behavior recognition technology: using advanced DeepSORT or better tracking algorithms, it achieves stable and continuous tracking of targets such as pedestrians and vehicles in complex toll plaza scenarios, and assigns a unique ID to each target; combining the track sequences obtained from tracking, it uses spatiotemporal convolutional networks or Transformer models to model the track sequences, learns the pattern differences between normal passage and abnormal behavior, and thus achieves high-accuracy recognition of behaviors such as intrusion, wrong-way driving, and loitering based on trajectory semantics.

[0048] This module employs front-end fusion perception technology combining radar and video: to overcome the limitations of pure video perception, this module supports deep integration with the radar-video fusion all-in-one machine; it adopts a pre-fusion processing strategy, that is, at the raw data level, it performs spatiotemporal alignment and feature association between radar point clouds and video pixels, uses the radar's precise ranging and velocity measurement information to guide the key areas of video analysis, and uses the rich texture information of the video to classify radar targets, and finally outputs fused target information with both high-precision spatial attributes and accurate category labels, which greatly improves the robustness of the system under all-weather conditions.

[0049] (3) Security Incident Analysis and Early Warning Generation Module:

[0050] This module, located in the business capability layer, receives and analyzes raw target data and behavioral signals output by the multimodal data intelligent analysis engine module. It performs advanced semantic understanding based on business rules to determine whether a security event warrants a warning. The module contains a flexibly configurable rule engine. Rules configured within the engine are logical combinations based on multiple dimensions such as target type, movement trajectory, speed, location area, and duration. For example, a "pedestrian intrusion" rule might be defined as: "target type is pedestrian," "enters the lane area from the plaza area," and "moves continuously for more than 10 meters within the lane area." When the conditions are met, the module immediately generates a warning message. This warning message is stored in a structured manner and must include: a unique event ID, event type, event level, occurrence time, occurrence location (device ID and pixel coordinates / geographic coordinates), associated evidence snapshots or short video clips, and a confidence score. Furthermore, the module supports compound judgments on consecutive and related primary events to generate higher-level events (such as inferring "congestion risk" from multiple instances of "slow traffic" and "increased density").

[0051] Specifically, this module employs a spatial rule engine technology based on Geographic Information System (GIS): it digitizes the physical layout of the toll area (lane lines, plaza boundaries, and isolation facilities) and constructs it into an electronic fence and a logical rule base; the module not only analyzes the type and behavior of targets, but also compares them with precise GIS spatial information in real time. For example, the core basis for determining "intrusion" is the spatial relationship between the target's trajectory and the electronic fence of the "prohibited area"; this makes event determination more accurate, rule configuration more intuitive (configurable by drawing fences on a map), and supports more complex spatial logic judgments.

[0052] (4) Early warning information hierarchical distribution and linkage control module:

[0053] This module resides in the business capability layer. To ensure appropriate responses to warnings, it implements a tiered distribution strategy. Administrators can pre-set different warning levels (high, medium, low) and corresponding contingency plans for different event types and occurrence areas (e.g., main entrance and general plaza). The contingency plan defines the warning push method: for high-risk warnings (e.g., pedestrians entering the driving lane), in addition to pop-ups and strong audio-visual alerts on the software interface, it automatically pushes warnings to the nearest road patrol personnel via SMS or mobile application; simultaneously, it automatically controls on-site audio-visual alarm devices to issue voice warnings and flashing lights. This module provides a standard API interface, enabling real-time push of warning information to the provincial road network command platform (TOCC) and traffic police department systems, facilitating cross-departmental collaboration; and all distribution records are log-traceable.

[0054] (5) Real-time monitoring and visualization module for traffic operation status:

[0055] This module resides in the business capability layer. It focuses on the quantitative perception of operational efficiency within toll areas. This module aggregates traffic parameter data from the multimodal data intelligent analysis engine module in real time, performs regional-level aggregation calculations in the background, and generates intuitive situation indicators. The core functions of this module specifically include:

[0056] Panoramic traffic flow display: The traffic density of each lane is displayed in real time on an electronic map using heat maps or color coding.

[0057] Key performance indicator dashboard: Real-time display of queue length curves, average speed, lane occupancy rate, etc. in key areas.

[0058] Saturation index calculation: By taking into account multiple parameters such as queue length, vehicle density, and traffic speed, an easy-to-understand "regional saturation index" (e.g., 0-100%) is calculated through an algorithm model, providing managers with an easy-to-understand basis for judging the degree of congestion.

[0059] Historical situation retrospective: Supports replaying historical situation changes over time for post-event assessment and pattern analysis.

[0060] (6) Event handling closed-loop and statistical analysis module

[0061] This module is located in the user interaction layer. It transforms early warning information into effective management actions and accumulates management knowledge. After receiving an early warning at the early warning response platform, on-duty personnel can confirm it, assign response tasks (designating personnel or teams), and provide remote guidance via voice intercom or video link. Response personnel can provide feedback on the scene and response results via mobile devices. This module tracks the entire process of the event, from occurrence, early warning, task assignment, response, to closure, recording the time points of each stage and automatically calculating efficiency indicators such as "early warning response time," "on-site arrival time," and "event response duration." This module provides rich statistical analysis functions, generating reports by time, region, event type, and other dimensions, analyzing high-incidence periods and locations, and providing data support for optimizing deployment strategies and carrying out targeted governance.

[0062] According to the specific implementation methods represented by the above embodiments, the intelligent perception and early warning system for highway safety incidents of the present invention has the following characteristics.

[0063] (1) Accurate perception of events in all scenarios: Based on multi-source data such as video streams and radar point clouds, it can accurately identify various safety events such as pedestrians entering the toll area, non-motorized vehicles entering the toll area, vehicles driving in the wrong direction, vehicles parking abnormally, traffic accidents, and littering. The accuracy rate of identification is no less than the advanced level in the industry, and the false alarm rate is controlled within an acceptable threshold.

[0064] (2) Real-time traffic situation quantitative assessment: While achieving safety monitoring, it simultaneously analyzes and calculates key traffic operation parameters within the toll area, such as cross-sectional traffic flow, vehicle density, queue length, average vehicle speed, etc., and can integrate them to generate a comprehensive "regional operation saturation index".

[0065] (3) Hierarchical and Classified Intelligent Early Warning: The identified events and abnormal states can be automatically graded and classified according to predefined rules (such as event type, location of occurrence, severity) and structured early warning information can be generated.

[0066] (4) Multi-channel early warning information distribution: It can distribute early warning information in real time to designated duty personnel, on-site mobile posts or superior command platforms (such as TOCC) through various means such as software interface pop-up windows, sound and light prompts, mobile application push, and standardized interface messages, ensuring that the early warning reaches the target audience in a timely and effective manner.

[0067] (5) Closed-loop management of incident handling: It provides functions such as visualization of early warning events, assignment of handling tasks, tracking of handling process, feedback and review of results, forming a complete closed loop of incident management, and supports analysis and statistics of incident handling efficiency.

[0068] (6) High availability and easy expansion of the system: It adopts a stable and reliable system architecture and supports 24 / 7 uninterrupted operation; the system design has good scalability and can flexibly connect to new types of sensing devices, define new event recognition rules, and connect to new external systems.

[0069] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation fall within the protection scope claimed by the present invention.

Claims

1. A smart perception and early warning system for highway safety incidents, comprising an architecture layer and a core module; characterized in that, The architecture layer includes: The device access layer is used to access the various intelligent sensing devices deployed in the toll area, receive data from each intelligent sensing device, and perform unified management and scheduling; the intelligent sensing devices include high-definition network cameras and Rayvision fusion sensing all-in-one machines. The data computing layer is used to acquire data from the device access layer and perform intelligent analysis; the intelligent analysis includes video intelligent analysis, radar-visual data fusion analysis, and traffic parameter calculation. The business capability layer is used to encapsulate the core business logic of the system; the core business logic includes event analysis and early warning generation service, early warning classification and routing service, traffic situation aggregation service, and early warning distribution adapter; The application service layer is used to provide functional services to users. The user interaction layer provides a direct interface for users to operate and obtain information. The core modules include: a unified access and management module for intelligent sensing devices, located at the device access layer; a multimodal data intelligent analysis engine module, located at the data computing layer; a security event assessment and early warning generation module, located at the business capability layer, used to provide event assessment and early warning generation services; a hierarchical distribution and linkage control module for early warning information, located at the business capability layer, used to provide early warning hierarchical and routing services; a real-time monitoring and visualization module for traffic operation status, located at the business capability layer, used to provide traffic status aggregation services; and a closed-loop event handling and statistical analysis module, located at the user interaction layer.

2. The intelligent perception and early warning system for highway safety incidents according to claim 1, characterized in that, The device access layer manages and monitors the status of each intelligent sensing device, receives and performs preliminary parsing of video streams or radar data streams, and converts the raw bitstream data into standard data packets for upper-layer processing. The intelligent analysis results of the data computing layer include structured event descriptions and traffic parameters, and are output in a standardized JSON format. The early warning distribution adapter of the business capability layer is used to distribute early warning information to downstream systems through different channels. The application service layer provides functional services including early warning event subscription and query services, real-time video viewing services, traffic situation data services, and system configuration management services. The user interaction layer includes a comprehensive monitoring and command screen for panoramic display of toll area dynamics, early warning event distribution, and traffic situation heatmaps. The early warning and response workbench provides on-duty personnel with functions such as early warning information list, detailed viewing, issuance of response instructions, and input of response feedback; the system management backend is used to configure equipment, manage rules, manage user permissions, and view system logs and operation reports.

3. The intelligent perception and early warning system for highway safety incidents according to claim 1, characterized in that, The workflow of the unified access and management module for intelligent sensing devices is as follows: S1. Supports automatic discovery, batch import, and online registration of smart sensing devices deployed in toll areas; S2. Create an independent file for each device, recording its model, IP address, installation location, region, and status; the installation location includes GPS coordinates and orientation angle; the status is one of online, offline, or faulty; S3. Provide device configuration templates for batch parameter settings of devices; the parameters include resolution, frame rate, and ROI drawing. S4. Continuously monitor the device heartbeat, issue alarms for offline or abnormal devices, and support device maintenance operations; the maintenance operations include remote restart and firmware upgrade.

4. The intelligent perception and early warning system for highway safety incidents according to claim 3, characterized in that, The multimodal data intelligent analysis engine module is equipped with various deep learning models optimized for highway scenarios, including a video analysis model, a radar-visual fusion analysis model, and a traffic parameter calculation model. The video analysis model, based on a convolutional neural network, is used to detect and track various targets in video footage in real time. These targets include pedestrians, non-motorized vehicles, and various types of motor vehicles, and identify abnormal behavior patterns of the targets through temporal behavior analysis. The radar-visual fusion analysis model employs pre-fusion or post-fusion processing strategies to correlate and match radar point cloud targets with video image targets, improving target detection rate and trajectory tracking stability under adverse weather conditions. The traffic parameter calculation model, based on tracked vehicle trajectory data, is used to count the number of vehicles passing through a virtual detection line or detection area in real time, calculate the time-averaged and spatial-averaged speeds, identify the tail position of the queue based on the spatial distribution of vehicle queues, calculate the queue length, and aggregate and output all parameters at preset time intervals.

5. The intelligent perception and early warning system for highway safety incidents according to claim 4, characterized in that, The multimodal data intelligent analysis engine module adopts deep learning-based multi-target tracking and behavior recognition technology, including: using a tracking algorithm to stably and continuously track each target in the complex scene of a toll plaza, and assigning a unique ID to each target; combining the trajectory sequence obtained by tracking, using a spatiotemporal convolutional network or Transformer model to model the trajectory sequence, learning the pattern differences between normal passage and abnormal behavior, thereby achieving high-accuracy recognition of target behavior based on trajectory semantics.

6. The intelligent perception and early warning system for highway safety incidents according to claim 4, characterized in that, The multimodal data intelligent analysis engine module adopts a front-end fusion perception technology of radar and video, including: adopting a pre-fusion processing strategy to perform spatiotemporal alignment and feature association between radar point clouds and video pixels at the raw data level, using the radar's accurate ranging and velocity information to guide the key areas of video analysis, and using the texture information of the video to classify radar targets, and finally outputting fused target information with both high-precision spatial attributes and accurate category labels.

7. The intelligent perception and early warning system for highway safety incidents according to claim 4, characterized in that, The security incident analysis and early warning generation module is equipped with a rule engine. The rules configured in the rule engine are logical combination rules based on multiple dimensions, including target type, motion trajectory, speed, location area, and duration. The security event analysis and early warning generation module receives and analyzes the raw target data and behavioral signals output by the multimodal data intelligent analysis engine module. Based on business rules, it performs advanced semantic understanding to determine whether a security event warrants an early warning. If so, it generates corresponding early warning information. This early warning information is stored in a structured manner and includes: a unique event ID, event type, event level, occurrence time, occurrence location, associated evidence snapshots or short video clips, and a confidence score. The occurrence location includes the device ID and pixel coordinates or geographic coordinates. The security event analysis and early warning generation module also supports compound judgments on consecutive and related primary events to generate higher-level events.

8. The intelligent perception and early warning system for highway safety incidents according to claim 7, characterized in that, The security incident analysis and early warning generation module adopts spatial rule engine technology based on geographic information system, including: digitizing the physical layout of the toll area and constructing it into an electronic fence and logical rule base; when analyzing the type and behavior of the target, comparing it with GIS spatial information in real time and making a judgment.

9. The intelligent perception and early warning system for highway safety incidents according to claim 7, characterized in that, The workflow of the hierarchical distribution and linkage control module for early warning information includes: In this module, managers can pre-set different warning levels and corresponding response plans for different event types and different occurrence areas. The plan defines the method of pushing warnings: for high-risk warnings, in addition to pop-up windows and strong sound and light prompts on the system interface, they are automatically pushed to the nearest patrol personnel via SMS or mobile application, and the on-site sound and light alarm devices are automatically controlled to provide voice warnings and flashing lights.

10. The intelligent perception and early warning system for highway safety incidents according to claim 9, characterized in that, The real-time traffic operation status monitoring and visualization module gathers traffic parameter data from the multimodal data intelligent analysis engine module in real time, performs regional-level aggregation calculations in the background, and generates intuitive status indicators. The core functions of the real-time traffic operation monitoring and visualization module include: panoramic traffic flow display: displaying the traffic density of each lane in real time on an electronic map using heat maps or color coding; key indicator dashboard: displaying queue length curves, average speed, and lane occupancy rates in key areas in real time; saturation index calculation: calculating the "regional saturation index" by combining multiple parameters such as queue length, vehicle density, and traffic speed through an algorithm model, providing managers with a simple basis for judging the degree of congestion; historical situation review: supporting the playback of historical situation changes over time for post-event evaluation and pattern analysis.

11. The intelligent perception and early warning system for highway safety incidents according to claim 10, characterized in that, The workflow of the event handling closed-loop and statistical analysis module is as follows: After receiving an early warning at the early warning and response workstation, on-duty personnel confirm the warning, assign response tasks, and provide remote guidance via voice intercom or video link. Response personnel provide feedback on the on-site situation and response results via mobile devices. The event response closed-loop and statistical analysis module tracks the entire process of the event from occurrence, early warning, task assignment, response to closure, and records the time points of each stage, automatically calculating efficiency indicators. The efficiency indicators include "early warning response time," "on-site arrival time," and "event response duration." At the same time, the event response closed-loop and statistical analysis module provides statistical analysis functions, which can generate reports in multiple dimensions by time, region, and event type, and analyze high-incidence periods and locations of events to optimize deployment strategies and carry out targeted governance.

Images