Intelligent road warning and early warning patrol and rescue linkage system and method thereof

Abstract

The application discloses an intelligent road alarm and early warning patrol and rescue linkage system and a method thereof, relates to the technical field of road safety, and uses intelligent hundred-meter plates, unmanned aerial vehicle clusters and mobile terminals to collect accident alarm information and road multi-source data; edge calculation is used to perform accident target detection classification and abnormal situation classification based on preprocessed data, and the data is sent to the cloud; real-time road multi-source data is imported into a digital twin model to obtain traffic situation space-time characteristics, and an accident risk heat map is obtained by combining the accident target detection classification and abnormal situation classification analysis to make an emergency decision; alarm and early warning information of different accident types and different risk levels is generated according to the accident risk heat map, different department linkage mechanisms are generated according to the emergency decision, and the alarm and early warning information is sent to different department systems to realize disposal process automation. The application constructs a whole-chain linkage system of 'perception-analysis-warning-linkage', realizes rapid response and rescue, reduces management cost and optimizes traffic efficiency.

Description

Technical Field

[0001] This invention relates to the field of road safety technology, and more specifically, to an intelligent road alarm, early warning, patrol, and rescue linkage system and method. Background Technology

[0002] The traditional accident reporting process on highways and other roads involves the caller contacting highway patrol via telephone. The patrol officer, upon receiving the call and determining the caller's location, notifies the relevant highway management department, which then notifies the rescue department. This process is cumbersome and time-consuming. Furthermore, callers may not be able to accurately describe their location to the patrol officer, hindering rapid response and potentially causing secondary accidents. While QR code-based emergency reporting technology has reduced reporting time, the inability of road management departments to immediately obtain information about the accident scene and quickly handle the situation still poses a threat to the safety of callers.

[0003] In recent years, with the rapid development of technologies such as the Internet of Things (IoT), Artificial Intelligence (AI), 5G communication, and big data analytics, intelligent road monitoring and emergency response systems have gradually become a research hotspot. For example, computer vision-based smart cameras can identify traffic accidents, violations, or road anomalies in real time; distributed sensor networks can monitor environmental parameters such as road icing, water accumulation, and visibility; and vehicle-to-everything (V2X) technology can enable information interaction between vehicles and infrastructure, providing early warnings of potential hazards. Therefore, building a complete "perception-analysis-early warning-coordination" solution by integrating multimodal perception technologies (such as radar, video, and infrared sensing), edge computing, cloud computing, and intelligent decision-making algorithms is an urgent problem to be solved. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes an intelligent road alarm, early warning, patrol, and rescue linkage system and method, constructing a full-chain linkage system of "perception-analysis-early warning-linkage" to achieve rapid response and rescue.

[0005] The first aspect of this invention provides an intelligent road alarm, early warning, patrol and rescue linkage system, including a multi-source sensing and acquisition module, an edge computing processing module, a cloud-based decision analysis module, an early warning information release module, an emergency linkage response module, and a management and maintenance module;

[0006] The multi-source sensing and acquisition module uses smart 100-meter markers, drone clusters, and mobile terminals to collect accident alarm information and multi-source road data.

[0007] The edge computing processing module preprocesses multi-source road data and performs accident target detection and classification and abnormal situation classification based on the preprocessed data.

[0008] The cloud-based decision analysis module imports real-time multi-source road data into a digital twin model to obtain the spatiotemporal characteristics of traffic conditions, obtains an accident risk heat map based on the accident target detection classification and abnormal situation classification analysis, and makes emergency decisions based on the accident risk heat map.

[0009] The early warning information release module generates alarm and early warning information for different accident types and risk levels based on the accident risk heat map, and visualizes the alarm and early warning information in a multimodal manner.

[0010] The emergency response module generates inter-departmental coordination mechanisms based on emergency decisions, and automates the response process based on these mechanisms.

[0011] The management and maintenance module is responsible for the management and operation of system equipment, performs three-dimensional visual monitoring of equipment status, and generates early warnings of abnormal equipment status.

[0012] In this solution, the multi-source sensing and acquisition module uses smart 100-meter markers, drone swarms, and mobile terminals to collect accident alarm information and multi-source road data, specifically:

[0013] The intelligent 100-meter sign uses its sensing unit to acquire vehicle status data, road surface status data, environmental data, and user alarm information, and uses an in-vehicle mobile terminal to acquire high-precision positioning data and collision detection data of road vehicles.

[0014] The drone swarm acquires road monitoring video streams along a preset route. When a user alarm message from a smart 100-meter sign is collected, the location information of the user's mobile terminal and the mileage information of the smart 100-meter sign are acquired. The location information and mileage information are returned to the drone swarm, and the nearest drone is selected to read the accident scene monitoring video stream.

[0015] The system acquires multi-source road data collected from smart 100-meter signs, drone clusters, and mobile terminals, performs spatiotemporal alignment, and sends user alarm information and multi-source road data to the edge computing processing module.

[0016] In this solution, the edge computing processing module performs data preprocessing on multi-source road data, and then performs accident target detection and classification, as well as anomaly classification, based on the preprocessed data. Specifically:

[0017] The road multi-source data is divided according to data category. Based on the radar performance indicators in the smart 100-meter sign, the neighborhood radius and minimum number of neighborhood points of the DBSCAN clustering algorithm are dynamically set, and the DBSCAN clustering algorithm is used to perform density clustering on the radar data.

[0018] Find neighboring points within the neighborhood radius. When the number of points in the neighborhood of a point cloud reaches the minimum number of neighboring points, the point cloud is marked as a core point and the points in the neighborhood are grouped into the same cluster. Isolated points that cannot meet the density requirements are marked as noise. After continuous verification, the points marked as noise are removed.

[0019] Adaptive histogram equalization is used to process visual data. Video frames are divided into blocks, each block has its own grayscale histogram calculated and contrast limits set. Standard histogram equalization is performed on each block to enhance the visual data.

[0020] The system obtains fault description data from user alarm information, extracts word vectors from the fault description data, generates corresponding semantic features, constructs a detection and classification network, and obtains preprocessed multi-source road data as model input.

[0021] Local point cloud features and local feature maps are obtained by using multi-layer perception and multi-layer convolution, respectively. The local distribution features of the local point cloud features and local feature maps are obtained by using average pooling. The local point cloud features, local feature maps and corresponding local distribution features are concatenated to obtain the local features of radar data and visual data, respectively.

[0022] Attention weighting is introduced to weight the local features, max pooling is used for feature aggregation, semantic features are used to weight and enhance the classifier, and the weighted and enhanced classifier is used to obtain the accident target detection and classification results. When there is no user alarm information, the detection and classification network is used to identify the abnormal road conditions and classification results.

[0023] In this solution, real-time multi-source road data is imported into a digital twin model in the cloud-based decision analysis module to obtain the spatiotemporal characteristics of traffic conditions. Specifically:

[0024] The system acquires preprocessed multi-source road data output by the edge computing processing module, obtains static road network data through high-precision maps and road design parameters, generates a road network spatial topology based on the static road network data, constructs a three-dimensional scene based on the road network spatial topology and road three-dimensional model, and performs data mapping of the three-dimensional scene using the multi-source road data.

[0025] By using mapping relationships to drive traffic flow simulation with multi-source data from actual roads, a dynamic digital twin of the road is generated. After continuous iterative optimization, traffic flow twin data of the digital twin model is obtained.

[0026] The road is gridded, and the node feature matrix is ​​obtained based on the spatial coordinates, velocity characteristics, acceleration distribution characteristics and environmental coupling characteristics of the grid. Physical edges are constructed through the topological relationship between grids, and interactive edges are established based on the velocity correlation between grids.

[0027] A spatiotemporal graph is constructed using grid nodes, physical edges, and interactive edges. The grid containing the accident target or abnormal situation target is taken as the central unit. Neighboring units are selected based on the central unit and the preset influence radiation distance. A neighborhood matrix is ​​constructed based on the spatiotemporal graph.

[0028] The neighborhood matrix is ​​represented and learned by a spatiotemporal graph attention network. In the spatial attention layer, the feature vector of each node is linearly transformed. Attention coefficients are obtained through graph attention. The node feature vector is updated by executing a multi-head attention mechanism in parallel. Neighbor features are aggregated by weighted summation to obtain spatial features.

[0029] In the temporal convolutional layer, gated convolution is performed through dilated causal convolution and gating mechanism to extract the temporal dependence of node feature vectors and obtain temporal features. Spatial and temporal features are fused. Spatiotemporal attention blocks are constructed based on spatial attention layer and temporal convolutional layer. Multiple spatiotemporal attention blocks are stacked and residual connections are made. Spatiotemporal features of traffic situation are obtained through stacked spatiotemporal attention blocks.

[0030] In this solution, the cloud-based decision analysis module obtains an accident risk heatmap based on the accident target detection classification and abnormal situation classification analysis, specifically as follows:

[0031] The traffic situation spatiotemporal characteristics of the area surrounding the accident target category or abnormal situation category target in the road are obtained, and the traffic accident domain knowledge graph is accessed. The accident target category or abnormal situation category is used to perform entity localization and marking in the domain knowledge graph.

[0032] The interaction relationships between labeled entities and different accident entities are extracted from the domain knowledge graph. The interaction frequency between accidents or anomalies is obtained through historical traffic accident instances. The degree of association is assigned to the interaction relationship based on the interaction frequency. In addition, the number of edge structures of the accident entity that has an interaction relationship with the labeled entity is counted. The number of edge structures and the maximum possible number of connections are used to characterize the importance of the accident entity.

[0033] Based on the degree of correlation and importance, the accident entities are sorted, and a preset number of accident entities are selected according to the sorting results as the associated accidents corresponding to the accident target category or abnormal situation category in the road, and the corresponding multi-source accident features are extracted.

[0034] In the digital twin model of a road, the spatiotemporal characteristics of the road's traffic situation and the multi-source accident characteristics of related accidents are calculated in a gridded fine-grained similarity. When the similarity is greater than a preset threshold, an associated accident label is generated in the corresponding grid, and an accident risk is assigned according to the deviation value of the similarity. Finally, an accident risk heat map with associated accident labels corresponding to accident targets or abnormal situations in the road is obtained.

[0035] In this plan, emergency decision-making is based on the aforementioned accident risk heat map, specifically as follows:

[0036] Accidents and potential accidents are identified based on the real-time updated accident risk heat map, and dynamic feature vectors are constructed based on the spatiotemporal characteristics of traffic conditions corresponding to the grids where accidents and potential accidents are located. Multidimensional similarity retrieval is then performed using the category labels of accidents and potential accidents and their corresponding dynamic feature vectors.

[0037] Using Euclidean distance, Manhattan distance, Hausdorff distance and Fraser distance as metric functions, hierarchical filtering is performed to obtain similar contingency plans. From the similar contingency plans, the set of emergency measures corresponding to accidents and potential accidents is read.

[0038] The firefly algorithm is used to optimize emergency decision-making based on the emergency measures set corresponding to accidents and potential accidents. The emergency measures set is encoded, and emergency measures combinations are generated by mutation of the historical best emergency plan corresponding to different accidents and potential accidents. The emergency measures combinations are generated by random sampling, and the emergency measures combinations are aggregated to initialize the firefly population.

[0039] The fitness function is associated with the objective function, and the brightness of individual fireflies at different positions is calculated. Individual fireflies move closer to fireflies with higher brightness based on the distance they move. During the movement, the position is updated through an adaptive step size mechanism. When the new position is reached, the brightness is calculated. Crossover and mutation operations are introduced to carry out emergency measures recombination.

[0040] An elite retention strategy is used to select a preset number of firefly individuals to enter the next generation. The best firefly individuals in history are subjected to Gaussian perturbation to generate new individuals. After iterative updates, the optimal combination of emergency measures corresponding to accidents and potential accidents is obtained based on the best position of the fireflies, and emergency decisions are output.

[0041] In this solution, the emergency response module generates inter-departmental coordination mechanisms based on emergency decisions, and automates the response process based on these mechanisms. Specifically:

[0042] The system acquires emergency decisions regarding accidents and potential accidents on the road and implements multi-departmental collaborative control. It constructs collaborative linkage relationships based on the core handling departments and functions involved in the emergency decisions and generates linkage commands from these relationships for multi-channel real-time communication.

[0043] Intelligent supervision enables emergency response based on real-time monitoring indicators. In the event of an anomaly, the linkage command is automatically resent. If a secondary anomaly occurs, neighboring departments are notified for coordination.

[0044] A second aspect of this invention provides an intelligent road alarm, early warning, patrol, and rescue linkage method, applied to an intelligent road alarm, early warning, patrol, and rescue linkage system, comprising the following steps:

[0045] Use smart 100-meter markers, drone swarms, and mobile terminals to collect accident alarm information and multi-source road data;

[0046] The road multi-source data is preprocessed, and edge computing is used to perform accident target detection and classification and abnormal situation classification based on the preprocessed data, which is then sent to the cloud.

[0047] Real-time multi-source road data is imported into a digital twin model in the cloud to obtain the spatiotemporal characteristics of traffic conditions. Accident risk heatmaps are obtained by combining the accident target detection and classification and abnormal situation classification analysis. Emergency decisions are made based on the accident risk heatmaps.

[0048] Based on the accident risk heat map, alarm and early warning information for different accident types and risk levels are generated. Based on emergency decisions, different inter-departmental linkage mechanisms are generated and sent to different departmental systems to automate the handling process.

[0049] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0050] This invention utilizes multi-source sensing devices such as intelligent 100-meter markers, drones, and GPS satellite positioning systems, combined with edge computing, cloud computing, and AI decision-making technologies, to construct a full-chain solution of "perception-analysis-early warning-coordination," bringing significant improvements to road safety, emergency response, and management efficiency.

[0051] Rapid location tracking and efficient multi-department collaboration significantly shorten emergency response times; automated patrols replace manual labor, improving road management efficiency and reducing maintenance costs. AI-driven decision-making for traffic control and precise information delivery improves traffic flow. Applicable to various scenarios including highways, urban expressways, and treacherous mountain roads, it provides crucial support for smart transportation and smart city development. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments or examples of the present invention, the drawings used in the embodiments or examples will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained according to these drawings without creative effort.

[0053] Figure 1 A block diagram of an intelligent road alarm, early warning, patrol, and rescue linkage system is shown.

[0054] Figure 2 A flowchart for acquiring the spatiotemporal characteristics of traffic conditions is shown;

[0055] Figure 3A flowchart illustrating emergency decision-making based on an accident risk heatmap is shown.

[0056] Figure 4 A flowchart of a smart road alarm, early warning, patrol, and rescue linkage method is shown. Detailed Implementation

[0057] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0058] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0059] Figure 1 A block diagram of an intelligent road alarm, early warning, patrol, and rescue linkage system is shown.

[0060] The first embodiment of the present invention provides an intelligent road alarm, early warning, patrol and rescue linkage system, including a multi-source sensing and acquisition module 101, an edge computing processing module 102, a cloud decision analysis module 103, an early warning information release module 104, an emergency linkage response module 105, and a management and maintenance module 106;

[0061] The multi-source sensing and acquisition module 101 uses smart 100-meter markers, drone clusters, and mobile terminals to collect accident alarm information and multi-source road data.

[0062] The edge computing processing module 102 performs data preprocessing on multi-source road data, and performs accident target detection and classification and abnormal situation classification based on the preprocessed data;

[0063] The cloud-based decision analysis module 103 imports real-time multi-source road data into a digital twin model to obtain the spatiotemporal characteristics of traffic conditions, obtains an accident risk heat map based on the accident target detection classification and abnormal situation classification analysis, and makes emergency decisions based on the accident risk heat map.

[0064] The early warning information release module 104 generates alarm and early warning information for different accident types and different risk levels based on the accident risk heat map, and visualizes the alarm and early warning information in a multimodal manner.

[0065] The emergency response module 105 generates inter-departmental coordination mechanisms based on emergency decisions, and automates the response process based on these mechanisms.

[0066] The management and maintenance module 106 is responsible for the management and operation of system equipment, performs three-dimensional visual monitoring of equipment status, and generates early warnings of abnormal equipment status.

[0067] It should be noted that the multi-source sensing and acquisition module uses smart 100-meter markers, drone swarms, and mobile terminals to collect accident alarm information and multi-source road data. The smart 100-meter marker has built-in GPS / BeiDou positioning for accurate road location identification, and millimeter-wave radar to detect vehicle speed and distance within a preset range. It integrates environmental sensors (temperature, humidity, visibility, and road icing detection), and is equipped with RFID / NFC tags, a panoramic camera, and a thermal infrared imager to support rapid vehicle or rescue equipment identification. It also features a low-power LoRa / NB-IoT communication module, enabling real-time data uploads to edge nodes. The drone swarm is equipped with a sensing suite including a three-light pod (visible light / infrared / laser ranging), synthetic aperture radar (SAR), and a gas detector. It also features a multispectral camera and infrared thermal imaging for all-weather monitoring of road anomalies. The drone swarm is based on a mesh self-organizing network and supports collaborative operations of up to 30 drones. Mobile terminals are divided into vehicle terminals and user terminals. They integrate communication units for real-time communication and utilize high-precision positioning modules and collision detection sensors for high-precision positioning and collision detection. User terminals include smartphones, smart bracelets, AR glasses, etc. For example, smart bracelets can provide users with vital sign data to a certain extent, monitor heart rate and blood oxygen, and automatically alarm in case of danger.

[0068] The intelligent 100-meter marker uses a sensing unit to acquire vehicle status data, road surface status data, environmental data, and user alarm information. It also uses an in-vehicle mobile terminal to acquire high-precision vehicle positioning data and collision detection data. The intelligent 100-meter marker supports user-submitted alarms by scanning a QR code on the marker. Simultaneously, the alarm is triggered by a road warning device installed behind the accident site, alerting oncoming vehicles. After receiving the alarm information, the road management department determines the marker's location and real-time positioning information. In a preferred embodiment, the number of alarms triggered by scanning the intelligent QR code markers along the entire road is statistically analyzed. The markers along the road are plotted on the horizontal axis, and the number of scans is plotted on the vertical axis, providing a clear view of the alarm status for the entire road segment. This allows for safety control of accident-prone sections (curved roads, steep slopes, foggy areas, etc.). The system can be set with the date on the horizontal axis and the number of scans on the vertical axis, which provides a clear view of the scan alarm status at each time point. This allows for appropriate safety control during periods when accidents are more frequent (such as holidays and the Spring Festival travel rush).

[0069] The drone swarm acquires road monitoring video streams along a preset flight path. When a user alarm message from a smart 100-meter marker is received, the location information of the user's mobile terminal and the marker number are obtained. The location and marker information are then returned to the drone swarm. The nearest drone is selected to read the accident scene monitoring video stream, and safety guidance is provided to personnel through the loudspeaker function of the smart patrol drone. The multi-source road data collected by the smart 100-meter marker, drone swarm, and mobile terminal are spatiotemporally aligned, and the user alarm message and multi-source road data are sent to the edge computing processing module.

[0070] It should be noted that the edge computing processing module performs data preprocessing on multi-source road data, and classifies accident targets and anomalies based on the preprocessed data. Edge computing nodes are deployed on roadside units or 5G base stations, and execute low-latency decisions, such as automatically triggering early warning signals and dispatching nearby drones, in response to notifications for data preprocessing and accident / anomaly filtering.

[0071] The multi-source road data is categorized according to data type. Based on the radar performance indicators in the smart 100-meter marker, the neighborhood radius and minimum number of neighborhood points for the DBSCAN clustering algorithm are dynamically set. The DBSCAN clustering algorithm is used to perform density clustering on the radar data. It does not require pre-setting the number of clusters, adapts to the dynamically changing number of targets in road scenarios, and effectively separates real targets from radar clutter. The raw polar coordinate data acquired by the radar (including distance, azimuth, Doppler velocity, etc.) is converted into three-dimensional point cloud data in a Cartesian coordinate system. Each converted data point contains three dimensions: lateral position, longitudinal position, and radial velocity. Considering the differences in dimensions of different coordinate axes, the spatial coordinates need to be standardized. The standardized point cloud data is then input into the DBSCAN algorithm for density clustering. The algorithm scans each data point, finding its neighboring points within its neighborhood radius. When a point's neighborhood contains a sufficient number of other points, that point is marked as a core point, and the points within its neighborhood are grouped into the same cluster. Isolated points that do not meet the density requirements are marked as noise. After initial clustering, velocity consistency is checked by calculating the velocity variance of each point within the same cluster. If the variance exceeds a threshold (e.g., 1.5 m / s), the entire cluster is considered dynamic noise. After multiple consecutive frames of verification, only points marked as noise for multiple consecutive frames (e.g., 3 frames) are ultimately removed to avoid misjudgments caused by transient interference. For points located at the edge of a cluster but not meeting the requirements of a core point, their retention or removal is determined based on their velocity similarity to that of the core point.

[0072] Adaptive histogram equalization is used to process visual data. Video frames are divided into blocks, and each block independently calculates its grayscale histogram to ensure local contrast optimization. A contrast limit is set, and histogram bars exceeding the threshold are cropped. Excess pixels are evenly distributed across grayscale levels. Standard histogram equalization is performed on each block, and bilinear interpolation is used to smooth block boundaries, eliminate block artifacts, and enhance visual data. In rainy or foggy weather, homomorphic filtering preprocessing is combined, and in strong light glare conditions, a halo detection algorithm is used to suppress local glare.

[0073] The system acquires fault description data from user alarm information, extracts word vectors from the fault description data using the BERT model, generates corresponding semantic features, constructs a detection and classification network, and uses preprocessed multi-source road data as model input. It employs multi-layer perceptron (5-layer MLP (256-128-64-32-16 dimensions)) and multi-layer convolution (4 convolutional blocks, 16-32-64-128 channels) to acquire local point cloud features and local feature maps, respectively. Average pooling is then used to obtain the local distribution features of the local point cloud features and local feature maps. Finally, 1×1 convolutions are used to upscale the point cloud features to 128 dimensions while maintaining the same visual feature dimensions. After spatial alignment, the local point cloud features, local feature maps, and corresponding local distribution features are concatenated to obtain local features from radar and visual data. Attention weighting is introduced to weight these local features, and max pooling is used for feature aggregation. The aggregated features are then imported into a classifier network. This network trains the classifier using large datasets of various road accidents and road anomalies, such as vehicle accidents (rear-end collisions, rollovers, etc.), vehicle anomalies (sudden braking, wrong-way driving, illegal parking, etc.), road anomalies (potholes, water accumulation, icing, etc.), environmental anomalies (heavy fog, heavy rain, strong winds), and facility anomalies (damaged guardrails, missing signs, etc.). Semantic features of the accidents or anomalies described in user alarm messages are obtained. The most relevant classifier is selected based on similarity, and the classifier is weighted and enhanced using these semantic features. The weighted and enhanced classifier is then used to obtain the accident target detection classification result. When there are no user alarm messages, the detection and classification network is used to identify the road anomaly classification result.

[0074] Figure 2 A flowchart for obtaining the spatiotemporal characteristics of traffic conditions is shown.

[0075] According to an embodiment of the present invention, in the cloud-based decision analysis module, real-time multi-source road data is imported into a digital twin model to obtain the spatiotemporal characteristics of traffic conditions, specifically as follows:

[0076] S202, obtain the preprocessed multi-source road data output by the edge computing processing module, obtain static road network data through high-precision map and road design parameters, generate road network spatial topology based on the static road network data, construct a three-dimensional scene based on the road network spatial topology and road three-dimensional model, and use the multi-source road data to perform data mapping of the three-dimensional scene;

[0077] S204 uses mapping relationships to drive traffic flow simulation with multi-source data from actual roads, generating a dynamic digital twin of the road. After continuous iterative optimization, the traffic flow twin data of the digital twin model is obtained.

[0078] S206 involves gridding the road, obtaining node feature matrices based on the grid's spatial coordinates, velocity characteristics, acceleration distribution characteristics, and environmental coupling characteristics, constructing physical edges through the topological relationships between grids, and establishing interactive edges based on the velocity correlation between grids.

[0079] S208, construct a spatiotemporal graph using grid nodes, physical edges, and interactive edges, take the grid where the accident target or abnormal situation target is located as the central unit, select neighboring units according to the central unit and the preset influence radiation distance, and construct a neighborhood matrix based on the spatiotemporal graph;

[0080] S210, the neighborhood matrix is ​​represented and learned through a spatiotemporal graph attention network. In the spatial attention layer, the feature vector of each node is linearly transformed. Attention coefficients are obtained through graph attention. The node feature vector is updated by executing a multi-head attention mechanism in parallel. Neighbor features are aggregated using weighted summation to obtain spatial features.

[0081] In S212, in the temporal convolutional layer, gated convolution is performed through dilated causal convolution and gating mechanism to extract the temporal dependence of node feature vectors, obtain temporal features, perform feature fusion of spatial and temporal features, construct spatiotemporal attention blocks based on spatial attention layer and temporal convolutional layer, stack multiple spatiotemporal attention blocks and perform residual connection, and obtain spatiotemporal features of traffic situation through stacked spatiotemporal attention blocks.

[0082] It should be noted that static elements such as roads, bridges, and traffic lights are loaded from the BIM library. Based on trajectory data, a 3D model with dynamic attributes is generated, dynamic entities are injected, and environmental and weather effects such as rain, snow, and fog are rendered in real time to generate a dynamic digital twin of the road for real-time traffic flow simulation. A spatiotemporal graph is constructed using grid nodes, physical edges, and interactive edges, and characterized by node feature matrices, adjacency matrices, and dynamic attention matrices. A spatiotemporal graph attention network is constructed using spatial attention layers and temporal convolutional layers, stacking four spatiotemporal attention blocks. Each block contains: a spatial attention sublayer, a temporal convolutional sublayer, residual connections, and layer normalization. Spatiotemporal features of road traffic flow in future time steps are extracted. In the temporal convolutional layer, a one-dimensional convolutional kernel with a width of 3 is used, sliding along the time axis. Gated dilated convolutions are performed according to a preset collision rate, and left padding is used to ensure no leakage of future information and to guarantee causality. Road traffic flow and road changes are monitored in real time. When a sudden change occurs in road traffic flow, a spatiotemporal graph update is triggered.

[0083] It should be noted that in the cloud-based decision analysis module, the spatiotemporal characteristics of traffic conditions around accident target categories or abnormal situation categories on the road are obtained, and connected to the traffic accident domain knowledge graph. The accident target category or abnormal situation category is used to perform entity localization and labeling in the domain knowledge graph. The interaction relationships between labeled entities and different accident entities are extracted from the domain knowledge graph. The interaction frequency between accidents or abnormalities is obtained through historical traffic accident instances. The degree of association is assigned to the interaction relationship based on the interaction frequency. Additionally, the number of edge structures of accident entities that have interaction relationships with labeled entities is counted, and the number of edge structures and the maximum possible number of connections are used to characterize... The importance of accident entities is assessed; based on the degree of correlation and importance, accident entities are ranked, and a predetermined number of accident entities are selected according to the ranking results as associated accidents corresponding to accident target categories or abnormal situation categories in the road, and corresponding multi-source accident features are extracted; in the digital twin model of the road, the spatiotemporal features of the road's traffic situation and the multi-source accident features of associated accidents are calculated using a gridded fine-grained similarity method. When the similarity is greater than a predetermined threshold, associated accident labels are generated in the corresponding grid, and accident risks are assigned according to the deviation value of the similarity. Finally, an accident risk heatmap with associated accident labels corresponding to accident targets or abnormal situations in the road is obtained. For existing accident targets or abnormal situations in the road, corresponding risks are generated according to a predetermined accident type-risk level matrix. For example, in the event of a vehicle collision, a multi-vehicle chain collision is a level 1 risk, a single vehicle with severe deformation is a level 2 risk, and a minor scratch is a level 3 risk.

[0084] The accident risk heatmap, containing associated accident tags, corresponding to accident targets or abnormal situations on the road, is imported into the early warning information release module. Based on preset graded early warning standards, accident levels and corresponding release channels are generated. Channels are adaptively allocated for different receiving scenarios; for example, a navigation app pop-up is generated for drivers selecting the in-vehicle AR-HUD mode, while a multi-screen control panel is generated for traffic management centers selecting a 3D sand table. In the event of a network outage, the LoRa self-organizing network broadcasts basic text and switches to a pre-stored scheme based on roadside equipment, using drones as temporary communication relays.

[0085] According to an embodiment of the present invention, emergency decision-making based on the accident risk heat map specifically includes:

[0086] S302, based on the real-time updated accident risk heat map, identify accidents and potential accidents, and construct dynamic feature vectors based on the spatiotemporal features of traffic conditions corresponding to the grids where accidents and potential accidents are located. Multidimensional similarity retrieval is performed through the category labels of accidents and potential accidents and the corresponding dynamic feature vectors.

[0087] S304, use Euclidean distance, Manhattan distance, Hausdorf distance and Fraser distance as metric functions to perform hierarchical filtering, obtain similar contingency plans, and read the set of emergency measures corresponding to accidents and potential accidents from the similar contingency plans;

[0088] S306. Based on the emergency measures set corresponding to accidents and potential accidents, the firefly algorithm is used to optimize emergency decision-making. The emergency measures set is encoded, and emergency measures combinations are generated according to the mutation of the historical best emergency plan corresponding to different accidents and potential accidents. Emergency measures combinations are generated through random sampling, and the emergency measures combinations are aggregated to initialize the firefly population.

[0089] S308 associates the fitness function with the objective function, calculates the brightness of individual fireflies at different positions, and moves towards the brighter fireflies based on the distance traveled. During the movement, the position is updated through an adaptive step size mechanism. When the new position is reached, the brightness is calculated. Crossover and mutation operations are introduced to carry out emergency measures recombination.

[0090] S310 uses an elite retention strategy to select a preset number of firefly individuals to enter the next generation. It generates new individuals by performing Gaussian perturbation on the historical best firefly individuals. After iterative updates, it obtains the best combination of emergency measures corresponding to accidents and potential accidents based on the best positions of the fireflies, and outputs emergency decisions.

[0091] It should be noted that Euclidean distance, Manhattan distance, Hausdorff distance, and Fraser distance are used as metrics for multi-dimensional hierarchical filtering. Euclidean distance represents spatial similarity, Manhattan distance represents dynamic feature differences, and Hausdorff distance and Fraser distance represent spatiotemporal trajectory matching. This enables hierarchical initial screening and fine screening to retrieve similar cases, obtain the handling effects of similar historical cases, and select a preset number of contingency plans to extract an emergency response set based on the handling effects. By fusing multiple metrics, the utilization rate of historical cases is improved, emergency decision-making time is reduced, and the scientific nature of the handling plan is enhanced. Based on the emergency response set corresponding to accidents and potential accidents, the Firefly algorithm is used for emergency decision optimization. A multi-objective optimization objective function is constructed by weighted aggregation based on timeliness, economy, and safety, and constraints are set, such as a 200m buffer zone including the accident point in the road closure area, and resource over-allocation penalties.

[0092] In the firefly algorithm, each firefly moves towards a firefly with higher fitness based on the distance traveled. During the movement, a non-linear decay step size based on the iterative process is introduced for position updates. Furthermore, perturbation is dynamically adjusted based on fitness; high-fitness individuals experience reduced perturbation, while low-fitness individuals experience increased exploration. Cross-recombining emergency response measures with high fitness values ​​facilitates the merging of important features, thereby accelerating the optimization process of emergency response measure combinations. The optimal emergency response measure combination corresponding to the accident and potential accidents is obtained based on the firefly's best position as the primary solution. Alternative solutions can also be selected based on suboptimal solutions. By optimizing emergency decision-making through biomimetic intelligence, the efficiency of emergency response plan generation, multi-objective coordinated optimization capabilities, and adaptability to complex scenarios are improved.

[0093] It should be noted that the emergency response module acquires emergency decisions regarding accidents and potential accidents on the road for multi-departmental collaborative control. It obtains a role-capability matrix for each department, such as: traffic police (traffic control / accident investigation), fire department (professional rescue / hazardous materials handling), medical personnel (emergency treatment / patient transfer), and road administration (road clearing / facility repair). Based on the core departments and functions involved in the emergency decisions, collaborative relationships are constructed. For example, traffic police and fire departments coordinate at the accident scene. These collaborative relationships generate linkage commands for multi-channel real-time communication. For instance, mobile devices are used for personnel dispatch, and IoT devices are used for traffic light control. This involves advance reconnaissance via drone swarms → fire monitoring → lane guidance, followed by intelligent traffic light control (red wave → green wave → return to normal), and optimal route navigation and danger zone warnings via vehicle terminals. Intelligent supervision of emergency response is performed based on real-time monitoring indicators. In case of anomalies, linkage commands are automatically resent. If a secondary anomaly occurs, neighboring departments are notified for coordination.

[0094] The management and maintenance module sets up electronic asset tags for equipment management, performs system equipment management and maintenance and remote firmware upgrades based on health status monitoring results, ensures encrypted data transmission, builds a security protection system, performs three-dimensional visual monitoring of equipment status, generates equipment status anomaly warnings, and automatically dispatches work orders based on the anomaly warnings and equipment status.

[0095] Figure 4 A flowchart of a smart road alarm, early warning, patrol, and rescue linkage method is shown.

[0096] The second embodiment of the present invention provides an intelligent road alarm, early warning, patrol and rescue linkage method, applied to an intelligent road alarm, early warning, patrol and rescue linkage system, including the following steps:

[0097] S402 uses smart 100-meter markers, drone swarms, and mobile terminals to collect accident alarm information and multi-source road data;

[0098] S404 preprocesses multi-source road data, uses edge computing to perform accident target detection and classification and abnormal situation classification based on the preprocessed data, and sends the data to the cloud.

[0099] S406. Real-time multi-source road data is imported into a digital twin model in the cloud to obtain the spatiotemporal characteristics of traffic conditions. Accident risk heat map is obtained by combining the accident target detection and classification and abnormal situation classification analysis. Emergency decision-making is made based on the accident risk heat map.

[0100] S408 generates alarm and early warning information for different accident types and risk levels based on the accident risk heat map, generates inter-departmental linkage mechanisms based on emergency decisions, and sends them to different departmental systems to automate the handling process.

[0101] The third embodiment of the present invention provides a computer-readable storage medium, which includes a program for a method of intelligent road alarm, early warning, patrol and rescue linkage. When the program is executed by a processor, it implements the steps of the method of intelligent road alarm, early warning, patrol and rescue linkage.

[0102] In the embodiments provided in this application, it should be understood that the disclosed methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, and can be electrical, mechanical, or other forms.

[0103] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0104] Alternatively, if the integrated modules of this invention are implemented as software functional units and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0105] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. An intelligent road alarm, early warning, patrol, and rescue linkage system, characterized in that, The system includes a multi-source sensing and acquisition module, an edge computing processing module, a cloud-based decision analysis module, an early warning information dissemination module, an emergency response module, and a management and maintenance module. The multi-source sensing and acquisition module uses smart 100-meter markers, drone clusters, and mobile terminals to collect accident alarm information and multi-source road data. The edge computing processing module preprocesses multi-source road data and performs accident target detection and classification and abnormal situation classification based on the preprocessed data. The cloud-based decision analysis module imports real-time multi-source road data into a digital twin model to obtain the spatiotemporal characteristics of traffic conditions, obtains an accident risk heat map based on the accident target detection classification and abnormal situation classification analysis, and makes emergency decisions based on the accident risk heat map. The early warning information release module generates alarm and early warning information for different accident types and risk levels based on the accident risk heat map, and visualizes the alarm and early warning information in a multimodal manner. The emergency response module generates inter-departmental coordination mechanisms based on emergency decisions, and automates the response process based on these mechanisms. The management and maintenance module is responsible for the management and operation of system equipment, performs three-dimensional visual monitoring of equipment status, and generates early warnings of abnormal equipment status. In the cloud-based decision analysis module, real-time multi-source road data is imported into a digital twin model to obtain the spatiotemporal characteristics of traffic conditions, specifically: The system acquires preprocessed multi-source road data output by the edge computing processing module, obtains static road network data through high-precision maps and road design parameters, generates a road network spatial topology based on the static road network data, constructs a three-dimensional scene based on the road network spatial topology and road three-dimensional model, and performs data mapping of the three-dimensional scene using the multi-source road data. By using mapping relationships to drive traffic flow simulation with multi-source data from actual roads, a dynamic digital twin of the road is generated. After continuous iterative optimization, traffic flow twin data of the digital twin model is obtained. The road is gridded, and the node feature matrix is ​​obtained based on the spatial coordinates, velocity characteristics, acceleration distribution characteristics and environmental coupling characteristics of the grid. Physical edges are constructed through the topological relationship between grids, and interactive edges are established based on the velocity correlation between grids. A spatiotemporal graph is constructed using grid nodes, physical edges, and interactive edges. The grid containing the accident target or abnormal situation target is taken as the central unit. Neighboring units are selected based on the central unit and the preset influence radiation distance. A neighborhood matrix is ​​constructed based on the spatiotemporal graph. The neighborhood matrix is ​​represented and learned by a spatiotemporal graph attention network. In the spatial attention layer, the feature vector of each node is linearly transformed. Attention coefficients are obtained through graph attention. The node feature vector is updated by executing a multi-head attention mechanism in parallel. Neighbor features are aggregated by weighted summation to obtain spatial features. In the temporal convolutional layer, gated convolution is performed through dilated causal convolution and gating mechanism to extract the temporal dependence of node feature vectors and obtain temporal features. Spatial and temporal features are fused. Spatiotemporal attention blocks are constructed based on spatial attention layer and temporal convolutional layer. Multiple spatiotemporal attention blocks are stacked and residual connections are made. Spatiotemporal features of traffic situation are obtained through stacked spatiotemporal attention blocks.

2. The intelligent road alarm, early warning, patrol, and rescue linkage system according to claim 1, characterized in that, In the multi-source sensing and acquisition module, intelligent 100-meter markers, drone swarms, and mobile terminals are used to collect accident alarm information and multi-source road data, specifically: The intelligent 100-meter sign uses its sensing unit to acquire vehicle status data, road surface status data, environmental data, and user alarm information, and uses an in-vehicle mobile terminal to acquire high-precision positioning data and collision detection data of road vehicles. The drone swarm acquires road monitoring video streams along a preset route. When a user alarm message from a smart 100-meter sign is collected, the location information of the user's mobile terminal and the mileage information of the smart 100-meter sign are acquired. The location information and mileage information are returned to the drone swarm, and the nearest drone is selected to read the accident scene monitoring video stream. The system acquires multi-source road data collected from smart 100-meter signs, drone clusters, and mobile terminals, performs spatiotemporal alignment, and sends user alarm information and multi-source road data to the edge computing processing module.

3. The intelligent road alarm, early warning, patrol, and rescue linkage system according to claim 1, characterized in that, In the edge computing processing module, multi-source road data is preprocessed, and accident target detection and classification, as well as anomaly classification, are performed based on the preprocessed data. Specifically: The road multi-source data is divided according to data category. Based on the radar performance indicators in the smart 100-meter sign, the neighborhood radius and minimum number of neighborhood points of the DBSCAN clustering algorithm are dynamically set, and the DBSCAN clustering algorithm is used to perform density clustering on the radar data. Find neighboring points within the neighborhood radius. When the number of points in the neighborhood of a point cloud reaches the minimum number of neighboring points, the point cloud is marked as a core point and the points in the neighborhood are grouped into the same cluster. Isolated points that cannot meet the density requirements are marked as noise. After continuous verification, the points marked as noise are removed. Adaptive histogram equalization is used to process visual data. Video frames are divided into blocks, each block has its own grayscale histogram calculated and contrast limits set. Standard histogram equalization is performed on each block to enhance the visual data. The system obtains fault description data from user alarm information, extracts word vectors from the fault description data, generates corresponding semantic features, constructs a detection and classification network, and obtains preprocessed multi-source road data as model input. Local point cloud features and local feature maps are obtained by using multi-layer perception and multi-layer convolution, respectively. The local distribution features of the local point cloud features and local feature maps are obtained by using average pooling. The local point cloud features, local feature maps and corresponding local distribution features are concatenated to obtain the local features of radar data and visual data, respectively. Attention weighting is introduced to weight the local features, max pooling is used for feature aggregation, semantic features are used to weight and enhance the classifier, and the weighted and enhanced classifier is used to obtain the accident target detection and classification results. When there is no user alarm information, the detection and classification network is used to identify the abnormal road conditions and classification results.

4. The intelligent road alarm, early warning, patrol, and rescue linkage system according to claim 1, characterized in that, In the cloud-based decision analysis module, an accident risk heatmap is obtained based on the accident target detection classification and abnormal situation classification analysis, specifically as follows: The traffic situation spatiotemporal characteristics of the area surrounding the accident target category or abnormal situation category target in the road are obtained, and the traffic accident domain knowledge graph is accessed. The accident target category or abnormal situation category is used to perform entity localization and marking in the domain knowledge graph. The interaction relationships between labeled entities and different accident entities are extracted from the domain knowledge graph. The interaction frequency between accidents or anomalies is obtained through historical traffic accident instances. The degree of association is assigned to the interaction relationship based on the interaction frequency. In addition, the number of edge structures of the accident entity that has an interaction relationship with the labeled entity is counted. The number of edge structures and the maximum possible number of connections are used to characterize the importance of the accident entity. Based on the degree of correlation and importance, the accident entities are sorted, and a preset number of accident entities are selected according to the sorting results as the associated accidents corresponding to the accident target category or abnormal situation category in the road, and the corresponding multi-source accident features are extracted. In the digital twin model of a road, the spatiotemporal characteristics of the road's traffic situation and the multi-source accident characteristics of related accidents are calculated in a gridded fine-grained similarity. When the similarity is greater than a preset threshold, an associated accident label is generated in the corresponding grid, and an accident risk is assigned according to the deviation value of the similarity. Finally, an accident risk heat map with associated accident labels corresponding to accident targets or abnormal situations in the road is obtained.

5. The intelligent road alarm, early warning, patrol, and rescue linkage system according to claim 4, characterized in that, Emergency decision-making is based on the aforementioned accident risk heat map, specifically as follows: Accidents and potential accidents are identified based on the real-time updated accident risk heat map, and dynamic feature vectors are constructed based on the spatiotemporal characteristics of traffic conditions corresponding to the grids where accidents and potential accidents are located. Multidimensional similarity retrieval is then performed using the category labels of accidents and potential accidents and their corresponding dynamic feature vectors. Using Euclidean distance, Manhattan distance, Hausdorff distance and Fraser distance as metric functions, hierarchical filtering is performed to obtain similar contingency plans. From the similar contingency plans, the set of emergency measures corresponding to accidents and potential accidents is read. The firefly algorithm is used to optimize emergency decision-making based on the emergency measures set corresponding to accidents and potential accidents. The emergency measures set is encoded, and emergency measures combinations are generated by mutation of the historical best emergency plan corresponding to different accidents and potential accidents. The emergency measures combinations are generated by random sampling, and the emergency measures combinations are aggregated to initialize the firefly population. The fitness function is associated with the objective function, and the brightness of individual fireflies at different positions is calculated. Individual fireflies move closer to fireflies with higher brightness based on the distance they move. During the movement, the position is updated through an adaptive step size mechanism. When the new position is reached, the brightness is calculated. Crossover and mutation operations are introduced to carry out emergency measures recombination. An elite retention strategy is used to select a preset number of firefly individuals to enter the next generation. The best firefly individuals in history are subjected to Gaussian perturbation to generate new individuals. After iterative updates, the optimal combination of emergency measures corresponding to accidents and potential accidents is obtained based on the best positions of the fireflies, and emergency decisions are output.

6. The intelligent road alarm, early warning, patrol, and rescue linkage system according to claim 1, characterized in that, In the emergency response module, inter-departmental coordination mechanisms are generated based on emergency decisions. These mechanisms then automate the response process. Specifically: The system acquires emergency decisions regarding accidents and potential accidents on the road, enabling multi-departmental collaborative control. It establishes collaborative linkage relationships based on the core departments and functions involved in the emergency decisions, and generates linkage commands from these relationships for multi-channel real-time communication. Intelligent supervision enables emergency response based on real-time monitoring indicators. In the event of an anomaly, the system automatically resends the linkage command. If a secondary anomaly occurs, it notifies neighboring departments for coordination.

7. A method for intelligent road alarm, early warning, patrol, and rescue linkage, characterized in that, The system applied to the intelligent road alarm, early warning, patrol, and rescue linkage system as described in any one of claims 1-6 includes the following steps: Use smart 100-meter markers, drone swarms, and mobile terminals to collect accident alarm information and multi-source road data; The road multi-source data is preprocessed, and edge computing is used to perform accident target detection and classification and abnormal situation classification based on the preprocessed data, which is then sent to the cloud. Real-time multi-source road data is imported into a digital twin model in the cloud to obtain the spatiotemporal characteristics of traffic conditions. Accident risk heatmaps are obtained by combining the accident target detection and classification and abnormal situation classification analysis. Emergency decisions are made based on the accident risk heatmaps. Based on the accident risk heat map, alarm and early warning information for different accident types and risk levels are generated. Based on emergency decisions, different inter-departmental linkage mechanisms are generated and sent to different departmental systems to automate the handling process.

Images