Intelligent road alarm early warning patrol rescue linkage system and method thereof

Abstract

The invention discloses an intelligent road alarm early warning patrol rescue linkage system and a method thereof, and relates to the technical field of road safety. An intelligent hectometer board, an unmanned aerial vehicle cluster and a mobile terminal are used for collecting accident alarm information and road multi-source data; performing accident target detection classification and abnormal condition classification based on the preprocessed data by utilizing edge calculation, and sending to the cloud; importing real-time road multi-source data into a digital twinborn model to obtain traffic situation spatial-temporal characteristics, and combining accident target detection classification and abnormal condition classification analysis to obtain an accident risk thermodynamic diagram for emergency decision making; and according to the accident risk thermodynamic diagram, generating alarm early warning information of different accident types and different risk levels, according to an emergency decision, generating a linkage mechanism of different departments, and sending the linkage mechanism to different department systems to realize automation of a disposal process. According to the invention, a full-chain linkage system of perception-analysis-early warning-linkage is constructed, rapid response and rescue are realized, the management cost is reduced, and the traffic efficiency is optimized.

Description

Technical Field

[0001] The present invention relates to the field of road safety technology, and more specifically, to an intelligent road alarm, early warning, inspection, rescue and linkage system and its method. Background Art

[0002] The original accident alarm process for roads such as highways: The alarm caller contacts the highway traffic police through the alarm call. After the highway traffic police receive the alarm and determine the location of the alarm caller, they notify the highway management department of the corresponding highway, and the highway management department then notifies the rescue department to carry out the rescue. This process is relatively cumbersome and time-consuming, and there will also be a situation where the alarm caller cannot accurately describe their current location to the highway traffic police, resulting in the traffic police being unable to quickly handle the accident, affecting the rescue efficiency, and even causing secondary accidents. Currently, there is a QR code scanning alarm technology in society. Although the alarm time is reduced, the road management department cannot obtain the accident scene situation in the first time and cannot quickly handle the accident, which still poses a threat to the safety of the alarm caller.

[0003] In recent years, with the rapid development of technologies such as the Internet of Things (IoT), artificial intelligence (AI), 5G communication, and big data analysis, intelligent road monitoring and emergency linkage systems have gradually become a research hotspot. For example, intelligent cameras based on computer vision can real-time identify traffic accidents, illegal behaviors, or road anomalies; distributed sensor networks can monitor environmental parameters such as road icing, water accumulation, and visibility; vehicle-to-everything (V2X) technology can achieve information interaction between vehicles and infrastructure, and early warn of potential dangers. Therefore, it is an urgent problem to be solved to construct a full-chain solution of "perception - analysis - early warning - linkage" by integrating multi-modal perception technologies (such as radar, video, infrared sensing), edge computing, cloud computing, and intelligent decision-making algorithms. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention proposes an intelligent road alarm, early warning, inspection, rescue and linkage system and its method, constructs a full-chain linkage system of "perception - analysis - early warning - linkage", and realizes rapid response and rescue.

[0005] The first aspect of the present invention provides an intelligent road alarm, early warning, inspection, rescue and linkage system, including a multi-source perception and acquisition module, an edge computing and processing module, a cloud decision-making and analysis module, an early warning information publishing module, an emergency linkage and disposal module, and a management and maintenance module; The multi-source perception and acquisition module uses intelligent hundred-meter signs, unmanned aerial vehicle clusters, and mobile terminals to collect accident alarm information and road multi-source data; The edge computing and processing module performs data preprocessing on the road multi-source data, and conducts accident target detection and classification and abnormal situation classification based on the preprocessed data; The cloud decision - making and analysis module imports real - time road multi - source data into the digital twin model to obtain the spatio - temporal characteristics of traffic situations, analyzes the accident risk heat map based on the accident target detection classification and abnormal situation classification, and makes emergency decisions according to the accident risk heat map; The early warning information release module generates alarm and early warning information of different accident types and different risk levels according to the accident risk heat map, and visually displays the alarm and early warning information through a multi - modal method; The emergency linkage and disposal module generates different department linkage mechanisms according to the emergency decision, and realizes the automation of the disposal process based on different department linkage mechanisms; The management and maintenance module is responsible for the management, operation and maintenance of system equipment, conducts three - dimensional visual monitoring of equipment status, and generates early warnings for equipment status anomalies.

[0006] In this solution, in the multi - source perception and collection module, intelligent hundred - meter signs, UAV clusters and mobile terminals are used to collect accident alarm information and road multi - source data. Specifically: The sensing unit of the intelligent hundred - meter sign is used to obtain vehicle status data, road surface status data, environmental data and user alarm information, and the in - vehicle mobile terminal is used to obtain high - precision positioning data and collision detection data of road vehicles; The UAV cluster is used to obtain road monitoring video streams according to a preset route. When the user alarm information of the intelligent hundred - meter sign is collected, the location information of the user mobile terminal and the stake number information of the intelligent hundred - meter sign are obtained, and the location information and stake number information are returned to the UAV cluster. The nearest UAV is selected to read the accident scene monitoring video stream; The spatio - temporal alignment of the road multi - source data collected by the intelligent hundred - meter sign, UAV cluster and mobile terminal is obtained, and the user alarm information and road multi - source data are sent to the edge computing and processing module.

[0007] In this solution, in the edge computing and processing module, data pre - processing is performed on the road multi - source data, and accident target detection classification and abnormal situation classification are performed based on the pre - processed data. Specifically: The road multi - source data is divided according to data categories, the neighborhood radius and the minimum number of neighborhood points of the DBSCAN clustering algorithm are dynamically set according to the radar performance index in the intelligent hundred - meter sign, and the DBSCAN clustering algorithm is used to perform density clustering on the radar data; Adjacent points within the neighborhood radius are searched. When the number of point clouds within the neighborhood of the point cloud reaches the minimum number of neighborhood points, the point cloud is marked as a core point, and the point clouds within the neighborhood are grouped into the same cluster, while the isolated points that do not meet the density requirements are marked as noise. After continuous verification, the points marked as noise are removed; Process visual data using adaptive histogram equalization, divide video frames into blocks, independently calculate the grayscale histogram for each block and set the contrast limit, perform standard histogram equalization on each block, and enhance the visual data; Obtain the fault description data in the user alarm information, extract the word vectors of the fault description data, generate corresponding semantic features, construct a detection and classification network, and obtain the preprocessed multi-source road data as the model input; Use multi-layer perception and multi-layer convolution to obtain local point cloud features and local feature maps respectively, and use average pooling to obtain the local distribution features of the local point cloud features and local feature maps. Concatenate the local point cloud features, local feature maps and their corresponding local distribution features to obtain the local features of radar data and visual data respectively; Introduce attention weighting to weight the local features, use max pooling for feature aggregation, use semantic features to weight and enhance the classifier, and use the weighted enhanced classifier to obtain the accident target detection and classification results. When there is no user alarm information, use the detection and classification network to identify the abnormal situation classification results of the road.

[0008] In this solution, in the cloud decision analysis module, import the real-time multi-source road data into the digital twin model to obtain the spatio-temporal characteristics of the traffic situation. Specifically: Obtain the preprocessed multi-source road data output by the edge computing processing module, and obtain the road network static data through the high-precision map and road design parameters. Generate the road network spatial topology according to the road network static data, construct a three-dimensional scene based on the road network spatial topology and the road three-dimensional model, and use the multi-source road data for data mapping of the three-dimensional scene; Drive the traffic flow simulation with the actual multi-source road data through the mapping relationship to generate the dynamic digital twin of the road, and obtain the traffic flow twin data of the digital twin model after continuous iterative optimization; Grid the road, obtain the node feature matrix based on the spatial coordinates, speed characteristics, acceleration distribution characteristics, and environmental coupling characteristics of the grid, construct physical edges through the topological relationship between the grids, and establish interaction edges according to the speed correlation between the grids; Use grid nodes, physical edges, and interaction edges to construct a spatio-temporal graph. Take the grid where the accident target or abnormal situation target is located as the central unit, select neighborhood units according to the central unit and the preset influence radiation distance, and construct a neighborhood matrix based on the spatio-temporal graph; Perform representation learning on the neighborhood matrix through the spatio-temporal graph attention network. Perform a linear transformation on each node feature vector in the spatial attention layer, obtain the attention coefficient through graph attention, parallelly execute the multi-head attention mechanism to update the node feature vector, and use weighted summation for neighbor feature aggregation to obtain spatial features; In the temporal convolutional layer, gated convolution is performed through dilated causal convolution and a gating mechanism to extract the temporal dependencies of the node feature vectors, obtain temporal features, fuse the spatial and temporal features, construct a spatio-temporal attention block based on the spatial attention layer and the temporal convolutional layer, stack multiple spatio-temporal attention blocks and perform residual connections, and obtain the spatio-temporal features of the traffic situation through the stacked spatio-temporal attention blocks.

[0009] In this solution, in the cloud decision analysis module, an accident risk heat map is obtained according to the accident target detection classification and abnormal situation classification analysis, specifically: Obtain the spatio-temporal features of the traffic situation in the area around the accident target category or abnormal situation category target on the road, access the accident domain knowledge graph, and use the accident target category or abnormal situation category to perform entity positioning and marking in the domain knowledge graph; Extract the interaction relationships between the marked entities and different accident entities in the domain knowledge graph, obtain the interaction frequencies between accidents or abnormalities through historical traffic accident instances, assign an association degree to the interaction relationships according to the interaction frequencies, and additionally count the number of edge structures of the accident entities themselves that have interaction relationships with the marked entities, and use the number of edge structures and the maximum possible connection number to characterize the importance of the accident entities; Sort the accident entities based on the association degree and importance, screen a preset number of accident entities according to the sorting results as the associated accidents corresponding to the accident target category or abnormal situation category on the road and extract the corresponding multi-source accident features; In the digital twin model of the road, perform a grid-based fine-grained similarity calculation on the spatio-temporal features of the traffic situation on the road and the multi-source accident features of the associated accidents. When the similarity is greater than the preset threshold, an associated accident label is generated in the corresponding grid, and an accident risk is assigned according to the deviation value of the similarity, and finally an accident risk heat map with an associated accident label corresponding to the accident target or abnormal situation on the road is obtained.

[0010] In this solution, emergency decision-making is performed according to the accident risk heat map, specifically: Determine accidents and potential accidents according to the real-time updated accident risk heat map, construct dynamic feature vectors for the spatio-temporal features of the traffic situation corresponding to the grids where the accidents and potential accidents are located, and perform multi-dimensional similarity retrieval through the category labels of the accidents and potential accidents and the corresponding dynamic feature vectors; Use the Euclidean distance, Manhattan distance, Hausdorff distance, and Fréchet distance as metric functions for hierarchical filtering to obtain similar plans, and read the set of emergency measures corresponding to the accidents and potential accidents in the similar plans; Use the firefly algorithm to optimize the emergency decision-making for the set of emergency measures corresponding to accidents and potential accidents. Encode the set of emergency measures, generate emergency measure combinations by mutating the historical optimal emergency plans corresponding to different accidents and potential accidents, and generate emergency measure combinations through random sampling. Aggregate and initialize the firefly population with the emergency measure combinations. Associate the fitness function with the objective function, calculate the brightness of firefly individuals at different positions. The firefly individuals move closer to the fireflies with higher brightness according to the moving distance, and update their positions through an adaptive step size mechanism during the movement. Calculate the brightness when reaching the new position, and introduce crossover and mutation operations for emergency measure recombination. Use the elitist retention strategy to select a preset number of firefly individuals to enter the next generation. Generate new individuals by performing Gaussian perturbation on the historical optimal firefly individuals. After iterative update, obtain the best emergency measure combination corresponding to the accident and potential accident according to the best position of the fireflies, and output the emergency decision.

[0011] In this solution, in the emergency response linkage disposal module, different department linkage mechanisms are generated according to the emergency decision, and the disposal process is automated based on different department linkage mechanisms. Specifically: Obtain the emergency decisions of accidents and potential accidents on the road for multi-department collaborative control. Construct a collaborative linkage relationship according to the core disposal departments and functions involved in the emergency decision, and generate linkage instructions from the collaborative linkage relationship for multi-channel instant communication. Conduct intelligent supervision of emergency disposal according to real-time monitoring indicators. When an abnormal situation occurs, automatically resend the linkage instruction. If a secondary abnormality occurs, notify adjacent departments for collaboration.

[0012] The second aspect of the present invention provides an intelligent road alarm, early warning, inspection, rescue and linkage method, which is applied to an intelligent road alarm, early warning, inspection, rescue and linkage system, and includes the following steps: Use intelligent hundred-meter signs, unmanned aerial vehicle clusters and mobile terminals to collect accident alarm information and multi-source road data. Perform data preprocessing on the multi-source road data, use edge computing to detect and classify accident targets and abnormal situations based on the preprocessed data, and send them to the cloud. Import the real-time multi-source road data into the digital twin model in the cloud to obtain the spatio-temporal characteristics of the traffic situation. Combine the accident target detection classification and abnormal situation classification analysis to obtain an accident risk heat map, and make an emergency decision according to the accident risk heat map. Generate alarm and early warning information for different accident types and different risk levels according to the accident risk heat map, generate different department linkage mechanisms according to the emergency decision, and send them to different department systems to automate the disposal process.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: The present invention constructs a full-chain solution of "perception - analysis - early warning - linkage" through multi-source perception devices such as intelligent 100-meter signs, drones, and GPS satellite positioning systems, combined with technologies such as edge computing, cloud computing, and AI decision-making, bringing significant improvements in aspects such as road safety, emergency response, and management efficiency.

[0014] By quickly positioning and highly efficient cooperation among multiple departments, the emergency response time is greatly shortened. Automated inspections replace manual work, improving road management efficiency and reducing operation and maintenance costs. Using AI decision-making for traffic control and precise information push improves traffic passing efficiency. It is applicable to various scenarios such as highways, urban expressways, and mountainous dangerous roads, providing important support for the construction of intelligent transportation and smart cities. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments or exemplifications of the present invention, the following will briefly introduce the drawings required for use in the description of the embodiments or exemplifications. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the drawings shown.

[0016] Figure 1 Shows a block diagram of an intelligent road alarm, early warning, inspection, rescue, and linkage system.

[0017] Figure 2 Shows a flowchart for obtaining spatio-temporal characteristics of traffic situations; Figure 3 Shows a flowchart for making emergency decisions based on an accident risk heat map; Figure 4 Shows a flowchart of an intelligent road alarm, early warning, inspection, rescue, and linkage method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] In order to be able to more clearly understand the above-mentioned objects, features, and advantages of the present invention, the following will further describe the present invention in detail with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0019] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0020] Figure 1 Shows a block diagram of an intelligent road alarm, early warning, inspection, rescue, and linkage system.

[0021] The first embodiment of the present invention provides an intelligent road alarm, early warning, inspection, rescue and linkage system, which includes a multi-source perception and acquisition module 101, an edge computing and processing module 102, a cloud decision-making and analysis module 103, an early warning information release module 104, an emergency linkage and disposal module 105, and a management and maintenance module 106; The multi-source perception and acquisition module 101 uses intelligent hundred-meter signs, unmanned aerial vehicle (UAV) clusters and mobile terminals to collect accident alarm information and multi-source road data; The edge computing and processing module 102 performs data preprocessing on the multi-source road data, and conducts accident target detection and classification and abnormal situation classification based on the preprocessed data; The cloud decision-making and analysis module 103 imports real-time multi-source road data into a digital twin model to obtain spatio-temporal characteristics of traffic conditions, analyzes and obtains an accident risk heat map according to the accident target detection and classification and abnormal situation classification, and makes emergency decisions according to the accident risk heat map; The early warning information release module 104 generates alarm and early warning information of different accident types and different risk levels according to the accident risk heat map, and visually displays the alarm and early warning information through a multi-modal method; The emergency linkage and disposal module 105 generates different department linkage mechanisms according to emergency decisions, and realizes the automation of the disposal process based on different department linkage mechanisms; The management and maintenance module 106 is responsible for the management, operation and maintenance of system equipment, conducts three-dimensional visual monitoring of the equipment status, and generates early warnings for abnormal equipment status.

[0022] It should be noted that in the multi-source perception and acquisition module, intelligent hundred-meter signs, UAV clusters and mobile terminals are used to collect accident alarm information and multi-source road data. The intelligent hundred-meter sign is built-in with GPS / Beidou positioning to accurately identify the road position, and millimeter-wave radar is used to detect data such as the speed and distance of vehicles within a preset range. It integrates environmental sensors (temperature and humidity, visibility, road icing detection), is equipped with RFID / NFC tags, panoramic cameras and thermal infrared imagers, supports quick identification of vehicles or rescue equipment, and is equipped with a low-power LoRa / NB-IoT communication module, which can upload data to the edge node in real time. The UAV cluster is equipped with a perception kit including a three-light pod (visible light / infrared / laser ranging), synthetic aperture radar (SAR), gas detector, etc., and is equipped with a multi-spectral camera + infrared thermal imager to monitor road anomalies all-weather. The UAV cluster is based on Mesh self-organizing network and supports the collaborative operation of 30 UAVs. The mobile terminal is divided into in-vehicle terminal and user terminal, integrates a communication unit for real-time communication, and uses a high-precision positioning module and a collision detection sensor for high-precision positioning and collision detection. The user terminal includes smart phones, smart bracelets, AR glasses, etc. For example, the smart bracelet can provide user vital sign data to a certain extent, monitor heart rate and blood oxygen, and automatically alarm in case of danger.

[0023] Use the sensing unit of the intelligent 100-meter sign to obtain vehicle status data, road surface status data, environmental data and user alarm information, and use the in-vehicle mobile terminal to obtain high-precision positioning data and collision detection data of road vehicles; the intelligent 100-meter sign supports users to scan and alarm, and submit alarm information by scanning the QR code on the 100-meter sign. When submitting the alarm information, the road warning device installed behind the accident is triggered synchronously to warn the oncoming vehicles behind. After receiving the alarm information submitted by the alarm person in the background, the road management department determines the stake number position and real-time positioning information of the alarm person. In a preferred embodiment of the present application, the number of times of scanning and alarm of the intelligent QR code 100-meter signs on the whole road can be counted. The stake numbers of the whole road can be used as the abscissa, and the number of scanning times can be used as the ordinate to intuitively see the scanning and alarm situation of the whole section of the road, and safety control can be carried out on accident-prone sections (curved road sections, steep slope sections, fog-prone sections, etc.). The date can be used as the abscissa and the number of scanning times can be used as the ordinate to intuitively see the scanning and alarm situation at each time point, and corresponding safety control can be carried out on accident-prone time periods (holidays, Spring Festival travel rush, etc.).

[0024] Use the drone cluster to obtain the road monitoring video stream according to the preset route. When the user alarm information of the intelligent 100-meter sign is collected, obtain the location information of the user mobile terminal and the stake number information of the intelligent 100-meter sign, return the location information and stake number information to the drone cluster, select the nearest drone to read the monitoring video stream of the accident scene, and conduct safety guidance for personnel through the voice function of the intelligent inspection drone; obtain the spatio-temporal alignment of the multi-source road data collected by the intelligent 100-meter sign, the drone cluster and the mobile terminal, and send the user alarm information and the multi-source road data to the edge computing processing module.

[0025] It should be noted that in the edge computing processing module, data preprocessing is performed on the multi-source road data, and accident target detection classification and abnormal situation classification are performed based on the preprocessed data. The edge computing nodes are deployed in the roadside unit or the 5G base station. When notifying data preprocessing and accident anomaly filtering, low-latency decisions are executed, such as automatically triggering warning signals and dispatching the nearest drones, etc.

[0026] Divide the multi-source road data according to the data category, dynamically set the neighborhood radius and the minimum number of neighborhood points of the DBSCAN clustering algorithm according to the radar performance indicators in the intelligent 100-meter sign, and use the DBSCAN clustering algorithm to perform density clustering on the radar data. The DBSCAN clustering algorithm does not require presetting the number of clusters, adapts to the dynamically changing number of targets in the road scene, and can effectively separate real targets from radar clutter. Convert the original polar coordinate data collected by the radar (including information such as distance, azimuth angle, and Doppler velocity) into three-dimensional point cloud data in the Cartesian coordinate system. Each converted data point contains three dimensions: lateral position, longitudinal position, and radial velocity. Considering the dimensional differences of different coordinate axes, it is necessary to standardize the spatial coordinates. Input the standardized point cloud data into the DBSCAN algorithm for density clustering. The algorithm will scan each data point and find adjacent points within its neighborhood radius. When a point's neighborhood contains a sufficient number of other points, the point is marked as a core point, and the points within the neighborhood are grouped into the same cluster. Isolated points that do not meet the density requirements are marked as noise. After the preliminary clustering is completed, perform a velocity consistency check, calculate the velocity variance of each point within the same cluster. If it exceeds the threshold (such as 1.5 m / s), the entire cluster is regarded as dynamic noise. After continuous verification of multiple frames, only points marked as noise for multiple consecutive frames (such as 3 frames) are finally removed to avoid misjudgment caused by instantaneous interference; for points located at the cluster edge but not meeting the core point requirements, their retention or removal is determined according to their velocity similarity to the core point.

[0027] Process the visual data using adaptive histogram equalization processing. Divide the video frames into blocks, independently calculate the grayscale histogram for each block to ensure local contrast optimization, set the contrast limit, clip the histogram bars that exceed the threshold, and evenly distribute the extra pixels to each gray level. Perform standard histogram equalization on each block, smooth the block boundaries through bilinear interpolation to eliminate blocky artifacts, and enhance the visual data; when rainy or foggy weather occurs, combine with homomorphic filtering preprocessing, and in the case of strong light glare, cooperate with the halo detection algorithm for local suppression.

[0028] Obtain the fault description data in the user's alarm information, use the BERT model to extract the word vectors of the fault description data, generate corresponding semantic features, construct a detection and classification network, and obtain the pre-processed multi-source road data as the model input; use a multi-layer perceptron (5-layer MLP (256-128-64-32-16 dimensions)) and multi-layer convolution (4 convolutional blocks, channel numbers 16-32-64-128) to obtain local point cloud features and local feature maps respectively, and use average pooling to obtain the local distribution features of the local point cloud features and local feature maps. The point cloud features are raised to 128 dimensions through 1×1 convolution, keeping the visual feature dimensions unchanged. After spatial alignment, the local point cloud features, local feature maps, and corresponding local distribution features are feature concatenated to obtain the local features of radar data and visual data respectively; introduce attention weighting to weight the local features, use max pooling for feature aggregation, import the aggregated features into the classifier network, and in the classifier network, obtain different road accident and road anomaly data through big data to train the classifier, such as vehicle accidents (rear-end collisions, rollovers, etc.), vehicle anomalies (sudden braking, reverse parking violations, etc.), road anomalies (potholes, water accumulation, icing, etc.), environmental anomalies (heavy fog, heavy rain, strong wind), and facility anomalies (guardrail damage, sign loss, etc.). Obtain the semantic features of the accident or anomaly described in the user's alarm information, use similarity to obtain the most relevant classifier, use the semantic features to weight and enhance the classifier, and use the weighted and enhanced classifier to obtain the accident target detection and classification results. When there is no user alarm information, the detection and classification network is used to identify the road anomaly classification results.

[0029] Figure 2 The flowchart of obtaining the spatio-temporal features of the traffic situation is shown.

[0030] According to an embodiment of the present invention, in the cloud decision-making analysis module, the real-time multi-source road data is imported into the digital twin model to obtain the spatio-temporal features of the traffic situation, specifically: S202, obtain the pre-processed multi-source road data output by the edge computing processing module, and obtain the road network static data through the high-precision map and road design parameters. Generate a road network spatial topology according to the road network static data, construct a three-dimensional scene based on the road network spatial topology and the road three-dimensional model, and use the multi-source road data for data mapping of the three-dimensional scene; S204, drive the traffic flow simulation with the actual multi-source road data through the mapping relationship to generate a dynamic digital twin of the road, and obtain the traffic flow twin data of the digital twin model after continuous iterative optimization; S206, perform grid processing on the road, obtain a node feature matrix based on the spatial coordinates, speed features, acceleration distribution features, and environmental coupling features of the grid, construct physical edges through the topological relationship between the grids, and establish interaction edges according to the speed correlation between the grids; S208. Use grid nodes, physical edges, and interaction edges to construct a spatio-temporal graph. Take the grid where the accident target or abnormal situation target is located as the central unit, select neighborhood units according to the central unit and a preset influence radiation distance, and construct a neighborhood matrix based on the spatio-temporal graph. S210. Perform representation learning on the neighborhood matrix through a spatio-temporal graph attention network. In the spatial attention layer, perform a linear transformation on each node feature vector, obtain attention coefficients through graph attention, parallelly execute the multi-head attention mechanism to update the node feature vectors, and use weighted summation for neighbor feature aggregation to obtain spatial features. S212. In the temporal convolutional layer, perform gated convolution through dilated causal convolution and a gating mechanism to extract the temporal dependencies of the node feature vectors, obtain temporal features, fuse the spatial features and temporal features, construct a spatio-temporal attention block according to the spatial attention layer and the temporal convolutional layer, stack multiple spatio-temporal attention blocks and perform residual connections, and obtain the spatio-temporal features of the traffic situation through the stacked spatio-temporal attention blocks.

[0031] It should be noted that models such as roads, bridges, and traffic lights are called from the BIM library for static element loading, a three-dimensional model with dynamic attributes is generated according to the trajectory data, dynamic entity injection is performed, and the environment and weather effects such as rain, snow, and fog are rendered in real time to generate a dynamic digital twin of the road for real-time traffic flow simulation. Use grid nodes, physical edges, and interaction edges to construct a spatio-temporal graph, and represent the spatio-temporal graph through a node feature matrix, an adjacency matrix, and a dynamic attention matrix. Utilize a spatial attention layer and a temporal convolutional layer to construct a spatio-temporal graph attention network, stack 4 spatio-temporal attention blocks, each block containing: a spatial attention sub-layer, a temporal convolutional sub-layer, a residual connection, and layer normalization, extract the spatio-temporal features of the road traffic flow in future time steps, use a one-dimensional convolutional kernel with a width of 3 in the temporal convolutional layer, slide along the time axis, perform gated dilated convolution according to a preset collision rate, and use left padding to ensure that future information is not leaked to ensure causality. Monitor the road traffic flow and road changes in real time. When the road traffic flow changes suddenly, it is necessary to trigger the update of the spatio-temporal graph.

[0032] It should be noted that in the cloud decision-making and analysis module, the spatio-temporal characteristics of the traffic situation in the area around the accident target category or abnormal situation category target on the road are obtained, the domain knowledge graph of traffic accidents is accessed, and the accident target category or abnormal situation category is used to perform entity positioning and marking in the domain knowledge graph; the interaction relationships between the marked entities and different accident entities are extracted from the domain knowledge graph, the interaction frequencies between accidents or abnormalities are obtained through historical traffic accident instances, and the association degree is assigned to the interaction relationships according to the interaction frequencies. In addition, the number of edge structures of the accident entities themselves that have interaction relationships with the marked entities is counted, and the importance of the accident entities is characterized by using the number of edge structures and the maximum possible connection number; the accident entities are sorted based on the association degree and importance, and a preset number of accident entities are selected according to the sorting results as the associated accidents corresponding to the accident target category or abnormal situation category on the road, and the corresponding multi-source accident characteristics are extracted; in the digital twin model of the road, the spatio-temporal characteristics of the traffic situation on the road and the multi-source accident characteristics of the associated accidents are calculated for the fine-grained similarity of the grid. When the similarity is greater than the preset threshold, an associated accident label is generated in the corresponding grid, and an accident risk is assigned according to the deviation value of the similarity. Finally, an accident risk heat map with an associated accident label corresponding to the accident target or abnormal situation on the road is obtained. For the existing accident targets or abnormal situations on the road, corresponding risks are generated according to the preset accident type-risk level matrix. For example, in the case of a vehicle collision, a multi-vehicle chain collision is a first-level risk, a single vehicle with serious deformation is a second-level risk, and a minor scratch is a third-level risk.

[0033] The accident risk heat map with an associated accident label corresponding to the accident target or abnormal situation on the road is imported into the early warning information release module, and the accident level and the corresponding release channels are generated according to the preset hierarchical early warning standard. For different receivers, channel adaptive allocation is performed accordingly. For example, for drivers, the in-vehicle AR-HUD mode is selected to generate a navigation APP pop-up window, and for the traffic management center, a three-dimensional sand table is selected to generate multi-screen joint control. When the network is interrupted, the LoRa self-organizing network broadcast basic text is started, and based on the pre-stored scheme for switching roadside devices, a drone is used as a temporary communication relay.

[0034] According to the embodiment of the present invention, emergency decision-making is performed according to the accident risk heat map, specifically: S302, determine accidents and potential accidents according to the accident risk heat map updated in real time, and construct a dynamic feature vector for the spatio-temporal characteristics of the traffic situation corresponding to the grid where the accidents and potential accidents are located, and perform multi-dimensional similarity retrieval through the category labels of the accidents and potential accidents and the corresponding dynamic feature vectors; S304, use the Euclidean distance, Manhattan distance, Hausdorff distance, and Fréchet distance as metric functions for hierarchical filtering to obtain similar plans, and read the set of emergency measures corresponding to the accidents and potential accidents in the similar plans; S306. Use the firefly algorithm to optimize the emergency decision-making for the set of emergency measures corresponding to accidents and potential accidents. Encode the set of emergency measures, generate emergency measure combinations by mutating the historical optimal emergency plans corresponding to different accidents and potential accidents, and generate emergency measure combinations through random sampling. Aggregate and initialize the firefly population with the emergency measure combinations. S308. Associate the fitness function with the objective function, calculate the brightness of firefly individuals at different positions. The firefly individuals move closer to the fireflies with higher brightness according to the moving distance, and update their positions through an adaptive step-size mechanism during the movement. Calculate the brightness when reaching the new position, and introduce crossover and mutation operations for emergency measure recombination. S310. Use the elitist retention strategy to select a preset number of firefly individuals to enter the next generation. Generate new individuals by performing Gaussian perturbation on the historical optimal firefly individuals. After iterative update, obtain the best emergency measure combination corresponding to the accident and potential accident according to the best position of the fireflies, and output the emergency decision.

[0035] It should be noted that the Euclidean distance, Manhattan distance, Hausdorff distance, and Fréchet distance are used as metric functions for multi-dimensional hierarchical filtering. The Euclidean distance represents spatial similarity, the Manhattan distance represents dynamic feature differences, and the Hausdorff distance and Fréchet distance represent spatio-temporal trajectory matching. Hierarchical pre-screening and fine-screening are implemented to retrieve similar cases, obtain the disposal effects of the retrieved similar historical cases, and select a preset number of plans according to the disposal effects to extract the set of emergency measures. Through multi-metric fusion, the utilization rate of historical cases is improved, the emergency decision-making time is reduced, and the scientific nature of the disposal plan is enhanced. Use the firefly algorithm to optimize the emergency decision-making for the set of emergency measures corresponding to accidents and potential accidents, construct an objective function for multi-objective optimization through weighted aggregation according to timeliness, economy, and safety, and set constraint conditions, such as the road closure range includes a 200m buffer zone around the accident point, resource over-allocation penalty, etc.

[0036] In the firefly algorithm, for each firefly, it moves closer to the firefly with higher fitness according to the moving distance, and introduces a non-linear decay step size based on the iterative process for position update during the movement. In addition, the perturbation is dynamically adjusted according to the fitness, with smaller perturbation for high-quality individuals and larger exploration for low-quality individuals. Cross-recombining the emergency measure combinations with high fitness values is beneficial for the merger of important features, thus accelerating the optimization selection process of the emergency measure combinations. Obtain the best emergency measure combination corresponding to the accident and potential accident according to the best position of the fireflies as the main promotion plan, and alternative plans can be selected according to the sub-optimal solutions. Optimize the emergency decision-making through bionic intelligence, improve the generation efficiency of emergency plans, multi-objective coordination optimization ability, and adaptability to complex scenarios.

[0037] It should be noted that in the emergency response linkage disposal module, emergency decisions on accidents and potential accidents on the road are obtained for multi-department collaborative control, and the role-capability matrix of each department is obtained. For example: traffic police - traffic control / accident investigation, fire department - professional rescue / hazardous chemical disposal, medical - emergency treatment / wounded transfer, road administration - road clearance / facility repair. Based on the core disposal departments and functions involved in the emergency decision, a collaborative linkage relationship is constructed. For example, the traffic police and the fire department conduct on-site collaboration at the accident scene. The collaborative linkage relationship is generated into a linkage instruction for multi-channel instant communication. For example, mobile devices are used for personnel dispatching, Internet of Things devices are used for traffic signal control, and an unmanned aerial vehicle (UAV) cluster is used for advanced reconnaissance → fire monitoring → channel guidance. When calling intelligent traffic signals for red waveband control → green wave channel → return to normal state, the in-vehicle terminal is used to achieve optimal path navigation → warning of dangerous areas. Intelligent supervision of emergency disposal is carried out according to real-time monitoring indicators. When abnormal situations occur, the linkage instruction is automatically resent. If a secondary anomaly occurs, adjacent departments are notified for collaboration.

[0038] The management and maintenance module sets asset electronic tags for equipment management, and conducts system equipment management, operation and maintenance, and remote firmware upgrade according to the health status monitoring results. At the same time, it ensures encrypted data transmission, constructs a security protection system, conducts three-dimensional visual monitoring of the equipment status, generates early warnings of abnormal equipment status, and automatically dispatches work orders based on the equipment location according to the early warnings.

[0039] Figure 4 The flowchart of the intelligent road alarm, early warning, inspection, rescue and linkage method is shown.

[0040] The second embodiment of the present invention provides an intelligent road alarm, early warning, inspection, rescue and linkage method, which is applied to an intelligent road alarm, early warning, inspection, rescue and linkage system, and includes the following steps: S402, using intelligent hundred-meter signs, UAV clusters and mobile terminals to collect accident alarm information and multi-source road data; S404, preprocessing the multi-source road data, using edge computing to detect and classify accident targets and classify abnormal situations based on the preprocessed data, and sending them to the cloud; S406, importing the real-time multi-source road data into the digital twin model in the cloud to obtain the spatio-temporal characteristics of the traffic situation, combining the accident target detection classification and abnormal situation classification analysis to obtain an accident risk heat map, and making an emergency decision according to the accident risk heat map; S408, generating alarm and early warning information of different accident types and different risk levels according to the accident risk heat map, generating different department linkage mechanisms according to the emergency decision, and sending them to different department systems to realize the automation of the disposal process.

[0041] The third embodiment of the present invention provides a computer-readable storage medium, which includes a program for the intelligent road alarm, early warning, inspection, rescue and linkage method. When the program for the intelligent road alarm, early warning, inspection, rescue and linkage method is executed by a processor, the steps of the intelligent road alarm, early warning, inspection, rescue and linkage method are implemented.

[0042] In several embodiments provided by the present application, it should be understood that the disclosed method can be implemented in other ways. The above-described embodiments are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or modules may be electrical, mechanical, or other forms.

[0043] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other various media that can store program codes.

[0044] Alternatively, if the above-mentioned integrated module of the present invention is implemented in the form of a software functional unit and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present invention essentially or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: removable storage devices, ROM, RAM, magnetic disks or optical disks and other various media that can store program codes.

[0045] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, and all of them should be covered by the protection scope of the present invention.

Claims

1. An intelligent road alarm, early warning, inspection, rescue and linkage system, characterized in that, The system includes a multi-source perception and acquisition module, an edge computing and processing module, a cloud decision-making and analysis module, a warning information publishing module, an emergency linkage and disposal module, and a management and maintenance module; The multi-source perception and acquisition module uses intelligent hundred-meter signs, unmanned aerial vehicle (UAV) clusters, and mobile terminals to collect accident alarm information and multi-source road data; The edge computing and processing module performs data preprocessing on the multi-source road data, and conducts accident target detection and classification, as well as abnormal situation classification, based on the preprocessed data; The cloud decision-making and analysis module imports real-time multi-source road data into a digital twin model to obtain spatio-temporal characteristics of the traffic situation, analyzes the accident risk heat map based on the accident target detection and classification and abnormal situation classification, and makes emergency decisions according to the accident risk heat map; The warning information publishing module generates alarm and warning information of different accident types and different risk levels based on the accident risk heat map, and visually displays the alarm and warning information through a multi-modal method; The emergency linkage and disposal module generates different department linkage mechanisms according to the emergency decision, and realizes the automation of the disposal process based on different department linkage mechanisms; The management and maintenance module is responsible for the management, operation, and maintenance of system equipment, conducts three-dimensional visual monitoring of the equipment status, and generates early warnings for abnormal equipment status.

2. The intelligent road alarm, early warning, inspection, rescue and linkage system according to claim 1, characterized in that In the multi-source perception and acquisition module, intelligent hundred-meter signs, UAV clusters, and mobile terminals are used to collect accident alarm information and multi-source road data. Specifically: The perception unit of the intelligent hundred-meter sign is used to obtain vehicle status data, road surface status data, environmental data, and user alarm information, and the in-vehicle mobile terminal is used to obtain high-precision road vehicle positioning data and collision detection data; The UAV cluster is used to obtain road monitoring video streams according to a preset route. When the user alarm information of the intelligent hundred-meter sign is collected, the location information of the user mobile terminal and the stake number information of the intelligent hundred-meter sign are obtained, and the location information and stake number information are returned to the UAV cluster. The nearest UAV is selected to read the monitoring video stream of the accident scene; The multi-source road data collected by the intelligent hundred-meter sign, UAV cluster, and mobile terminal are aligned in time and space, and the user alarm information and multi-source road data are sent to the edge computing and processing module.

3. The intelligent road alarm, early warning, inspection, rescue and linkage system according to claim 1, characterized in that In the edge computing and processing module, data preprocessing is performed on the multi-source road data, and accident target detection and classification, as well as abnormal situation classification, are conducted based on the preprocessed data. Specifically: The multi-source road data are divided according to data categories, the neighborhood radius and minimum neighborhood point number of the DBSCAN clustering algorithm are dynamically set according to the radar performance index in the intelligent hundred-meter sign, and the DBSCAN clustering algorithm is used to perform density clustering on the radar data; Adjacent points within the neighborhood radius are searched. When the number of point clouds within the neighborhood of the point cloud reaches the minimum neighborhood point number, the point cloud is marked as a core point, and the point clouds within the neighborhood are grouped into the same cluster. Isolated points that do not meet the density requirements are marked as noise. After continuous verification, the points marked as noise are removed; Process visual data using adaptive histogram equalization, divide video frames into blocks, independently calculate the grayscale histogram for each block and set the contrast limit, perform standard histogram equalization on each block, and enhance the visual data; Obtain the fault description data in the user alarm information, extract the word vectors of the fault description data, generate corresponding semantic features, construct a detection and classification network, and obtain the pre-processed multi-source road data as the model input; Use multi-layer perception and multi-layer convolution to obtain local point cloud features and local feature maps respectively, and use average pooling to obtain the local distribution features of the local point cloud features and local feature maps. Concatenate the local point cloud features, local feature maps and corresponding local distribution features to obtain the local features of radar data and visual data respectively; Introduce attention weighting to weight the local features, use max pooling for feature aggregation, use semantic features to weight and enhance the classifier, and use the weighted and enhanced classifier to obtain the accident target detection and classification results. When there is no user alarm information, use the detection and classification network to identify the classification results of road anomalies; 4. An intelligent road alarm, early warning, inspection, rescue and linkage system according to claim 1, characterized in that, In the cloud decision analysis module, import the real-time multi-source road data into the digital twin model to obtain the spatio-temporal characteristics of the traffic situation. Specifically: Obtain the pre-processed multi-source road data output by the edge computing processing module, and obtain the road network static data through the high-precision map and road design parameters. Generate the road network spatial topology according to the road network static data, construct a three-dimensional scene based on the road network spatial topology and the road three-dimensional model, and use the multi-source road data for data mapping of the three-dimensional scene; Drive the traffic flow simulation with the actual multi-source road data through the mapping relationship to generate the dynamic digital twin of the road, and obtain the traffic flow twin data of the digital twin model after continuous iterative optimization; Divide the road into grids, obtain the node feature matrix based on the spatial coordinates, speed characteristics, acceleration distribution characteristics, and environmental coupling characteristics of the grids, construct physical edges through the topological relationship between the grids, and establish interaction edges according to the speed correlation between the grids; Use grid nodes, physical edges, and interaction edges to construct a spatio-temporal graph, take the grid where the accident target or abnormal situation target is located as the central unit, select neighborhood units according to the central unit and the preset influence radiation distance, and construct a neighborhood matrix based on the spatio-temporal graph; Perform representation learning on the neighborhood matrix through the spatio-temporal graph attention network. In the spatial attention layer, perform a linear transformation on each node feature vector, obtain the attention coefficient through graph attention, parallelly execute the multi-head attention mechanism to update the node feature vector, and use weighted summation for neighbor feature aggregation to obtain spatial features; In the temporal convolution layer, perform gated convolution through dilated causal convolution and gated mechanism to extract the temporal dependence of the node feature vector, obtain temporal features, fuse the spatial features and temporal features, construct a spatio-temporal attention block according to the spatial attention layer and the temporal convolution layer, stack multiple spatio-temporal attention blocks and perform residual connection, and obtain the spatio-temporal characteristics of the traffic situation through the stacked spatio-temporal attention blocks.

5. An intelligent road alarm, early warning, inspection, rescue and linkage system according to claim 4, characterized in that, In the cloud decision-making and analysis module, an accident risk heat map is obtained according to the accident target detection classification and abnormal situation classification analysis, specifically as follows: Obtain the spatio-temporal characteristics of the traffic situation in the area around the accident target category or abnormal situation category target on the road, access the accident domain knowledge graph, and use the accident target category or abnormal situation category to perform entity positioning and marking in the domain knowledge graph; Extract the interaction relationships between the marked entities and different accident entities in the domain knowledge graph, obtain the interaction frequencies between accidents or abnormalities through historical traffic accident instances, assign an association degree to the interaction relationships according to the interaction frequencies, and additionally count the number of edge structures of the accident entities themselves that have interaction relationships with the marked entities. Use the number of edge structures and the maximum possible connection number to characterize the importance of the accident entities; Sort the accident entities based on the association degree and importance, select a preset number of accident entities according to the sorting results as the associated accidents corresponding to the accident target category or abnormal situation category on the road, and extract the corresponding multi-source accident characteristics; In the digital twin model of the road, perform a grid-based fine-grained similarity calculation between the spatio-temporal characteristics of the traffic situation on the road and the multi-source accident characteristics of the associated accidents. When the similarity is greater than the preset threshold, an associated accident label is generated in the corresponding grid, and an accident risk is assigned according to the deviation value of the similarity. Finally, an accident risk heat map with associated accident labels corresponding to the accident targets or abnormal situations on the road is obtained.

6. The intelligent road alarm and early warning inspection and rescue linkage system according to claim 5, wherein Perform emergency decision-making according to the accident risk heat map, specifically as follows: Determine accidents and potential accidents based on the real-time updated accident risk heat map, construct a dynamic feature vector for the spatio-temporal characteristics of the traffic situation in the grid corresponding to the accidents and potential accidents, and perform multi-dimensional similarity retrieval through the category labels of the accidents and potential accidents and the corresponding dynamic feature vectors; Use the Euclidean distance, Manhattan distance, Hausdorff distance, and Fréchet distance as metric functions for hierarchical filtering to obtain similar plans, and read the set of emergency measures corresponding to the accidents and potential accidents in the similar plans; Based on the set of emergency measures corresponding to the accidents and potential accidents, use the firefly algorithm to optimize the emergency decision-making. Encode the set of emergency measures, generate emergency measure combinations by mutating according to the historical optimal emergency plans corresponding to different accidents and potential accidents, and generate emergency measure combinations through random sampling. Aggregate and initialize the firefly population with the emergency measure combinations; Associate the fitness function with the objective function, calculate the brightness of firefly individuals at different positions, and the firefly individuals move closer to the firefly with higher brightness according to the moving distance. During the movement, the position is updated through an adaptive step mechanism, and the brightness is calculated when reaching the new position. Introduce crossover and mutation operations for emergency measure recombination; Use the elitist retention strategy to select a preset number of firefly individuals to enter the next generation, perform Gaussian perturbation on the historical optimal firefly individuals to generate new individuals. After iterative update, obtain the best emergency measure combination corresponding to the accidents and potential accidents according to the best positions of the fireflies, and output the emergency decision.

7. An intelligent road alarm, early warning, inspection, rescue and linkage system according to claim 1, characterized in that In the emergency response linkage disposal module, different department linkage mechanisms are generated according to emergency decisions, and the disposal process is automated based on different department linkage mechanisms. Specifically: Obtain emergency decisions on accidents and potential accidents on the road for multi-department collaborative control, construct a collaborative linkage relationship based on the core disposal departments and functions involved in the emergency decisions, and generate linkage instructions from the collaborative linkage relationship for multi-channel instant communication; Conduct intelligent supervision of emergency disposal according to real-time monitoring indicators. When an abnormal situation occurs, automatically resend the linkage instructions. If a secondary abnormality occurs, notify adjacent departments for collaboration.

8. An intelligent road alarm, early warning, inspection, rescue and linkage method, characterized in that, Applied to the intelligent road alarm warning inspection and rescue linkage system according to any one of claims 1-7, including the following steps: Use intelligent hundred-meter signs, drone swarms, and mobile terminals to collect accident alarm information and multi-source road data; Perform data preprocessing on the multi-source road data, use edge computing to detect and classify accident targets and abnormal situations based on the preprocessed data, and send them to the cloud; Import real-time multi-source road data into the digital twin model in the cloud to obtain spatio-temporal characteristics of the traffic situation, combine the accident target detection classification and abnormal situation classification analysis to obtain an accident risk heat map, and make emergency decisions based on the accident risk heat map; Generate alarm warning information for different accident types and different risk levels according to the accident risk heat map, generate different department linkage mechanisms according to emergency decisions, and send them to different department systems to automate the disposal process.

Images