Early warning method and device for pavement collapse
By obtaining real-time and historical pavement data, using collapse prediction models for data comparison and risk level assessment, and combining image data for early warning processing, the problems of lag monitoring of hidden dangers on urban roads and lack of multi-department coordination mechanisms are solved, and timely warning of road collapse is achieved, reducing losses caused by accidents.
Patent Information
- Application Number
- CN202510516524.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-23
- Publication Date
- 2025-08-26
AI Technical Summary
The monitoring technology for hidden dangers for road collapse in urban roads is lagging behind, the coordination mechanism of multiple departments is lacking, and early warning response standards are lacking, resulting in frequent road collapse accidents and losses of life and property of pedestrians and vehicles.
By obtaining real-time and historical pavement data, using collapse prediction models for data comparison and risk level assessment, combining image data for early warning processing, and issuing warning or prompt information to relevant departments.
Timely warning of urban road collapse has been achieved, losses caused by collapse have been reduced, and the safety of pedestrians and vehicles have been ensured.
Smart Images
Figure CN120544339A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of road engineering technology, and in particular to a method and device for early warning of road collapse. Background Art
[0002] Urban road pavement collapses are a hidden threat to modern urban safety. These disasters often occur without warning, causing traffic disruptions and vehicle damage at best, and casualties and significant property losses at worst. The current response system suffers from significant shortcomings: First, hazard monitoring technology lags behind, with over 80% of cities still relying on manual inspections. Second, a multi-departmental coordination mechanism is lacking. Underground pipelines involve management departments such as water supply, gas, and telecommunications, resulting in overlapping responsibilities and inefficient risk assessment. Third, early warning and response standards are lacking, and a tiered early warning system for collapse risk, similar to earthquake levels, has yet to be established. Consequently, rescue and road repair efforts are often delayed until a collapse occurs, but the incident has already occurred, causing damage to the lives and property of pedestrians and vehicles.
[0003] Therefore, there is an urgent need for a method for early warning of road subsidence on urban roads, so as to avoid the loss of life and property safety of pedestrians and vehicles caused by road subsidence. Summary of the Invention
[0004] The purpose of this application is to solve at least one of the above technical deficiencies.
[0005] In one aspect, an embodiment of the present application provides a road collapse early warning method, the method comprising:
[0006] Acquire real-time road surface data of the monitored road surface and historical road surface data corresponding to the monitored road surface, and determine a change in the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data;
[0007] Obtain the data standard value corresponding to the monitored road surface, and compare the real-time data change with the data standard value to determine whether the real-time road surface data is abnormal road surface data;
[0008] If the real-time road surface data is determined to be abnormal road surface data, image data corresponding to the monitored road surface is obtained, and the abnormal road surface data and image data are input into the road collapse prediction model to obtain the collapse risk level;
[0009] Carry out collapse warning processing on the monitored road surface according to the collapse risk level.
[0010] Optionally, the type of real-time road surface data includes at least one of real-time deformation, real-time stress, real-time temperature, and real-time settlement. Determining a change in real-time data of the monitored road surface based on the real-time road surface data and historical road surface data includes:
[0011] Performing alignment and standardization processing on various types of data included in the real-time road surface data to obtain processed real-time road surface data;
[0012] Obtaining a weight corresponding to each type of data, and performing weighted processing on the processed real-time road surface data based on the weight corresponding to each type of data to obtain target real-time road surface data;
[0013] Based on the target real-time road surface data and historical road surface data, the real-time data change of the monitored road surface is determined.
[0014] Optionally, each type of data included in the real-time road surface data is aligned and standardized to obtain processed real-time road surface data, including:
[0015] Performing time alignment processing on various types of data included in the real-time road surface data through a sliding window to obtain aligned real-time road surface data;
[0016] Performing spatial registration processing on the aligned real-time road surface data to obtain registered real-time road surface data;
[0017] The registered real-time road surface data is normalized to obtain processed real-time road surface data.
[0018] Optionally, the real-time data variation is compared with the data standard value to determine whether the real-time road surface data is abnormal road surface data, including:
[0019] Determine the difference between the real-time data change and the data standard value, and compare the difference with the preset threshold;
[0020] If the difference is greater than a preset threshold, the real-time road surface data is determined to be abnormal road surface data;
[0021] If the difference is not greater than the preset threshold, it is determined that the real-time road surface data is not abnormal road surface data.
[0022] Optionally, the collapse prediction model includes a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer, and a risk level output layer. Abnormal road surface data and image data are input into the road collapse prediction model to obtain a collapse risk level, including:
[0023] Input the abnormal road surface data and image data into the input layer for feature extraction to obtain the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data;
[0024] Input the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector;
[0025] The spatiotemporal feature vector is input into the cross-modal attention fusion layer for attention calculation to obtain the spatiotemporal fusion feature;
[0026] The spatiotemporal fusion features and the feature vectors corresponding to the image data are input into the risk level output layer for risk decision fusion processing to obtain the collapse risk level.
[0027] Optionally, the spatiotemporal feature extraction layer includes a time series module and a spatial feature module, and the spatiotemporal feature vector includes a time feature vector and a spatial feature vector;
[0028] The feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data are input into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector, including:
[0029] The time series module captures the temporal dependency of the feature vector corresponding to the abnormal road surface data and performs temporal attention processing to obtain the temporal feature vector;
[0030] Based on the spatial feature module, the feature vector corresponding to the image data is processed with graph features to obtain the spatial feature vector.
[0031] Optionally, the spatial feature module includes a convolutional layer, a graph attention network, and a global pooling layer. Based on the spatial feature module, the feature vector corresponding to the image data is processed with graph features to obtain a spatial feature vector, including:
[0032] The feature vector corresponding to the image data is input into the convolution layer for spatiotemporal joint feature extraction to obtain spatiotemporal joint features;
[0033] Input the spatiotemporal joint features into the graph attention network to obtain graph features;
[0034] The graph features are compressed based on the global pooling layer to obtain the spatial feature vector.
[0035] Optionally, collapse warning processing can be performed on the monitored road surface according to the collapse risk level, including:
[0036] If the collapse risk level is level 1, a collapse warning message will be sent to the relevant road departments so that they can take preventive measures in a timely manner;
[0037] If the collapse risk level is the second level, a collapse reminder message and real-time road surface data will be sent to the road-related departments so that the road-related departments can carry out traffic control in a timely manner.
[0038] Optionally, after determining that the real-time road surface data is abnormal road surface data, the method further includes:
[0039] Obtain real-time image data corresponding to the monitored road surface;
[0040] The cause of the abnormality is determined based on real-time image data and fed back to the road-related departments so that they can take corresponding measures.
[0041] On the other hand, an embodiment of the present application provides a road collapse warning device, comprising:
[0042] a change amount determination module, configured to obtain real-time road surface data of the monitored road surface and historical road surface data corresponding to the monitored road surface, and determine a change amount of the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data;
[0043] The abnormal data determination module is used to obtain the data standard value corresponding to the monitored road surface, and compare the real-time data change with the data standard value to determine whether the real-time road surface data is abnormal road surface data;
[0044] A risk level determination module is used to obtain image data corresponding to the monitored road surface when the real-time road surface data is determined to be abnormal road surface data, and input the abnormal road surface data and the image data into the road collapse prediction model to obtain the collapse risk level;
[0045] The early warning processing module is used to perform collapse early warning processing on the monitored road surface according to the collapse risk level.
[0046] Optionally, the type of the real-time road surface data includes at least one of real-time deformation, real-time stress, real-time temperature, and real-time settlement. When the change amount determination module determines the change amount of the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data, it is specifically configured to:
[0047] Performing alignment and standardization processing on various types of data included in the real-time road surface data to obtain processed real-time road surface data;
[0048] Obtaining a weight corresponding to each type of data, and performing weighted processing on the processed real-time road surface data based on the weight corresponding to each type of data to obtain target real-time road surface data;
[0049] Based on the target real-time road surface data and historical road surface data, the real-time data change of the monitored road surface is determined.
[0050] Optionally, when the variation determination module aligns and standardizes each type of data included in the real-time road surface data to obtain the processed real-time road surface data, it is specifically configured to:
[0051] Performing time alignment processing on various types of data included in the real-time road surface data through a sliding window to obtain aligned real-time road surface data;
[0052] Performing spatial registration processing on the aligned real-time road surface data to obtain registered real-time road surface data;
[0053] The registered real-time road surface data is normalized to obtain processed real-time road surface data.
[0054] Optionally, when the data determination module compares the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data, it is specifically used to:
[0055] Determine the difference between the real-time data change and the data standard value, and compare the difference with the preset threshold;
[0056] If the difference is greater than a preset threshold, the real-time road surface data is determined to be abnormal road surface data;
[0057] If the difference is not greater than the preset threshold, it is determined that the real-time road surface data is not abnormal road surface data.
[0058] Optionally, the collapse prediction model includes a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer, and a risk level output layer. When the risk level determination module inputs abnormal road surface data and image data into the road collapse prediction model to obtain the collapse risk level, it is used to:
[0059] Input the abnormal road surface data and image data into the input layer for feature extraction to obtain the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data;
[0060] Input the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector;
[0061] The spatiotemporal feature vector is input into the cross-modal attention fusion layer for attention calculation to obtain the spatiotemporal fusion feature;
[0062] The spatiotemporal fusion features and the feature vectors corresponding to the image data are input into the risk level output layer for risk decision fusion processing to obtain the collapse risk level.
[0063] Optionally, the spatiotemporal feature extraction layer includes a time series module and a spatial feature module, and the spatiotemporal feature vector includes a time feature vector and a spatial feature vector;
[0064] The risk level determination module inputs the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector, specifically for:
[0065] The time series module captures the temporal dependency of the feature vector corresponding to the abnormal road surface data and performs temporal attention processing to obtain the temporal feature vector;
[0066] Based on the spatial feature module, the feature vector corresponding to the image data is processed with graph features to obtain the spatial feature vector.
[0067] Optionally, the spatial feature module includes a convolutional layer, a graph attention network, and a global pooling layer. When the risk level determination module performs graph feature processing on the feature vector corresponding to the image data based on the spatial feature module to obtain the spatial feature vector, it is specifically used to:
[0068] The feature vector corresponding to the image data is input into the convolution layer for spatiotemporal joint feature extraction to obtain spatiotemporal joint features;
[0069] Input the spatiotemporal joint features into the graph attention network to obtain graph features;
[0070] The graph features are compressed based on the global pooling layer to obtain the spatial feature vector.
[0071] Optionally, when performing collapse warning processing on the monitored road surface according to the collapse risk level, the early warning processing module is specifically used to:
[0072] If the collapse risk level is level 1, a collapse warning message will be sent to the relevant road departments so that they can take preventive measures in a timely manner;
[0073] If the collapse risk level is the second level, a collapse reminder message and real-time road surface data will be sent to the road-related departments so that the road-related departments can carry out traffic control in a timely manner.
[0074] Optionally, after determining that the real-time road surface data is abnormal road surface data, the abnormal data determination module is further configured to:
[0075] Obtain real-time image data corresponding to the monitored road surface;
[0076] The cause of the abnormality is determined based on real-time image data and fed back to the road-related departments so that they can take corresponding measures.
[0077] In another aspect, an embodiment of the present application provides an electronic device, including a processor and a memory:
[0078] The memory is configured to store machine-readable instructions, which, when executed by the processor, cause the processor to perform any one of the methods for early warning of road collapse.
[0079] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:
[0080] In an embodiment of the present application, by obtaining real-time road surface data of the monitored road surface and comparing the real-time road surface data with historical road surface data, after determining that there is abnormal data, the collapse risk level is obtained based on the collapse prediction model. At this time, the road surface collapse can be promptly handled based on the collapse risk level, thereby preventing the road surface collapse from causing losses to the life and property safety of pedestrians and vehicles. In the case of collapse, warning information can be provided to the road and traffic authorities in a timely manner, so that timely measures can be taken to reduce the losses caused by the collapse.
[0081] In addition, in an embodiment of the present application, after real-time road surface data is acquired, the real-time road surface data can be time-aligned. This ensures that the lengths of various types of data in the real-time road surface data are consistent with the lengths of historical data from the same period. The data can also be spatially registered and normalized, which can further eliminate data layout deviations and dimensional differences, thereby ensuring data accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0082] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0083] Figure 1 A schematic diagram of a process flow of a road collapse early warning method provided in an embodiment of the present application;
[0084] Figure 2 A schematic diagram of the structure of the collapse prediction model provided in the embodiment of the present application;
[0085] Figure 3 A schematic structural diagram of a road collapse warning device provided in an embodiment of the present application;
[0086] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0087] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and are not to be construed as limiting the present invention.
[0088] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.
[0089] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0090] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0091] Specifically, such as Figure 1 As shown, the method may include:
[0092] Step S101 : acquiring real-time road surface data of a monitored road surface and historical road surface data corresponding to the monitored road surface, and determining a change in the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data.
[0093] Optionally, the monitored road surface refers to a road that may collapse. In this application, a timed period can be set to obtain real-time road surface data of the monitored road surface. The specific time period of the historical road surface data can be set according to the road surface conditions, such as road surface data within a week or road surface data within 3 days. This embodiment of the present application does not limit this.
[0094] Optionally, the method for acquiring real-time road surface data can be customized. For example, a high-precision static level, fiber Bragg grating sensor, or satellite positioning (GNSS) device can be used to acquire real-time road surface data, thereby adapting to complex urban environments (such as anti-electromagnetic interference, waterproof and dustproof, etc.). Accordingly, sensor equipment for acquiring real-time road surface data can be densely deployed (with spacing of 50-100 meters) in high-settlement areas such as road intersections, areas with dense underground pipelines, and construction-affected areas. At the same time, it can be deployed along the centerline of the road and at intervals on both sides of the lane line to form a grid monitoring network. Sensor equipment can be installed using a drilled buried installation method to ensure close contact with the roadbed, and GNSS equipment can be installed on roadside poles to ensure stable signal reception. The sensor equipment transmits real-time data to the data center via the LoRa / NB-IoT low-power wide area network, and can be configured with an edge computing module to perform pre-processing such as filtering and denoising on the acquired raw data to ensure data accuracy.
[0095] Furthermore, after obtaining the real-time road surface data of the monitored road surface and the historical road surface data corresponding to the monitored road surface, the real-time road surface data and the historical road surface data can be compared to determine the real-time data change of the monitored road surface, and the data change can reflect the current changes in the monitored road surface.
[0096] In an optional embodiment of the present application, the type of real-time road surface data includes at least one of real-time deformation, real-time stress, real-time temperature, and real-time settlement. Determining a change in the real-time data of the monitored road surface based on the real-time road surface data and historical road surface data includes:
[0097] Performing alignment and standardization processing on various types of data included in the real-time road surface data to obtain processed real-time road surface data;
[0098] Obtaining a weight corresponding to each type of data, and performing weighted processing on the processed real-time road surface data based on the weight corresponding to each type of data to obtain target real-time road surface data;
[0099] Based on the target real-time road surface data and historical road surface data, the real-time data change of the monitored road surface is determined.
[0100] Optionally, to better understand the specific conditions of the monitored road surface, the acquired real-time road surface data may include multiple types of data, such as at least one of real-time deformation, real-time stress, real-time temperature, and real-time settlement. Since the acquired real-time road surface data varies in type, the data lengths of different types of real-time road surface data may differ. Therefore, when determining the change in real-time data of the monitored road surface, each acquired data type may be aligned and standardized to obtain processed real-time road surface data. In this case, the lengths of each type of data in the processed real-time road surface data are consistent.
[0101] Furthermore, since different types of data have different effects on road collapse, different weights are set according to the effects of each type of data on road collapse. After obtaining the processed real-time road data, the processed real-time road data can be weighted based on the weight corresponding to each type of data to obtain the target real-time road data. Then, based on the target real-time road data and the historical road data, the real-time data change of the monitored road surface can be determined.
[0102] In an optional embodiment of the present application, each type of data included in the real-time road surface data is aligned and standardized to obtain processed real-time road surface data, including:
[0103] Performing time alignment processing on various types of data included in the real-time road surface data through a sliding window to obtain aligned real-time road surface data;
[0104] Performing spatial registration processing on the aligned real-time road surface data to obtain registered real-time road surface data;
[0105] The registered real-time road surface data is normalized to obtain processed real-time road surface data.
[0106] Optionally, for each type of real-time road surface data, a sliding window (e.g., a 5-minute window) can be used to align the real-time data with historical data from the same period (e.g., the same season or time period) to obtain the aligned real-time road surface data. This aligned real-time road surface data can then be spatially aligned, such as using a GIS coordinate system to spatially configure sensor positions to eliminate layout deviations and obtain the aligned real-time road surface data. Furthermore, since the acquired real-time road surface data may have dimensional differences, the aligned real-time road surface data can be normalized to obtain the processed real-time road surface data.
[0107] In an embodiment of the present application, after real-time road surface data is acquired, the real-time road surface data can be time-aligned. This ensures that the lengths of various types of data in the real-time road surface data are consistent with the lengths of historical data from the same period. The data is also spatially registered and normalized, which can further eliminate data layout deviations and dimensional differences, thereby ensuring data accuracy.
[0108] Step S102: obtaining a data standard value corresponding to the monitored road surface, and comparing the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data.
[0109] Optionally, the data standard value corresponding to the monitored road surface can be obtained. The data standard value is the data value when the monitored road surface does not collapse and is in normal driving. Specifically, it can be obtained by building a dynamic reference baseline based on historical data (such as an ARIMA prediction value). Accordingly, the real-time data change can be compared with the data standard value to determine whether the real-time road data is abnormal road data.
[0110] In an optional embodiment of the present application, comparing the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data includes:
[0111] Determine the difference between the real-time data change and the data standard value, and compare the difference with the preset threshold;
[0112] If the difference is greater than a preset threshold, the real-time road surface data is determined to be abnormal road surface data;
[0113] If the difference is not greater than the preset threshold, it is determined that the real-time road surface data is not abnormal road surface data.
[0114] Optionally, when comparing the real-time data change and the data standard value, the difference between the real-time data change and the data standard value can be determined first, and then a preset threshold is obtained, and the obtained difference is compared with the preset threshold. If the obtained difference is greater than the preset threshold, it means that the data fluctuation of the current monitored road surface is relatively large. At this time, it can be determined that the real-time road surface data is abnormal road surface data. On the contrary, if the difference is not greater than the preset threshold, it means that the data fluctuation of the current monitored road surface is relatively small, and the possibility of road collapse is small. At this time, the real-time road surface data is not abnormal road surface data.
[0115] In an optional embodiment of the present application, after determining that the real-time road surface data is abnormal road surface data, the method further includes:
[0116] Obtain real-time image data corresponding to the monitored road surface;
[0117] The cause of the abnormality is determined based on real-time image data and fed back to the road-related departments so that they can take corresponding measures.
[0118] Optionally, after determining that the acquired real-time road surface data is abnormal road surface data, in order to better determine the cause of the data abnormality, real-time image data corresponding to the monitored road surface can be obtained (such as taking photos and recording using cameras along the road to obtain real-time image data), and then the cause of the abnormality can be determined based on the real-time image data, and the cause of the abnormality can be fed back to the road-related departments so that the road-related departments can take corresponding measures based on the cause of the abnormality.
[0119] Step S103: If it is determined that the real-time road surface data is abnormal road surface data, image data corresponding to the monitored road surface is obtained, and the abnormal road surface data and the real-time image data are input into a road collapse prediction model to obtain a collapse risk level.
[0120] Optionally, after determining that the real-time road surface data is abnormal road surface data, it means that there is a risk of collapse of the monitored road surface. At this time, image data corresponding to the monitored road surface can be obtained, and then the abnormal road surface data and image data can be input into the road surface collapse prediction model to obtain the collapse risk level, which reflects the probability of collapse of the monitored road surface.
[0121] In an optional embodiment of the present application, the collapse prediction model includes a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer, and a risk level output layer. Abnormal road surface data and image data are input into the road collapse prediction model to obtain a collapse risk level, including:
[0122] Input the abnormal road surface data and image data into the input layer for feature extraction to obtain the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data;
[0123] Input the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector;
[0124] The spatiotemporal feature vector is input into the cross-modal attention fusion layer for attention calculation to obtain the spatiotemporal fusion feature;
[0125] The spatiotemporal fusion features and the feature vectors corresponding to the image data are input into the risk level output layer for risk decision fusion processing to obtain the collapse risk level.
[0126] Optionally, the collapse prediction model can be trained based on a multimodal spatiotemporal fusion network (MSTF-Net), including a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer and a risk level output layer connected in sequence.
[0127] Correspondingly, after obtaining the abnormal road surface data (i.e., real-time road surface data) and image data, the abnormal road surface data and image data are input into the input layer. The fully connected layer in the input layer maps the abnormal road surface data to the feature space to obtain the feature vector corresponding to the abnormal road surface data, and the ConvNeXt convolutional neural network in the input layer extracts the multi-scale visual features of the image data to obtain the feature vector corresponding to the image data.
[0128] Furthermore, the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data are input into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector, which is then input into the cross-modal attention fusion layer. The cross-modal attention fusion layer uses the temporal features in the spatiotemporal feature vector as the query, and the spatial features as the key and value. The similarity matrix between the query and the key is then calculated, and the weight is obtained through Softmax normalization. Finally, a weighted sum value vector is performed based on the obtained weight to generate the spatiotemporal fusion feature.
[0129] Furthermore, the feature vectors corresponding to the spatiotemporal fusion features and image data are input into the risk level output layer. The risk level output layer uses the XGBoost algorithm to screen key features and output the risk probability distribution. Then, the sensor node information is aggregated through GraphSAGE to generate graph features, and the graph features are averaged and pooled to obtain sample-level features. Finally, the two types of feature scores are weighted and combined to obtain the collapse risk level.
[0130] In an optional embodiment of the present application, the spatiotemporal feature extraction layer includes a time series module and a spatial feature module, and the spatiotemporal feature vector includes a time feature vector and a spatial feature vector;
[0131] The feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data are input into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector, including:
[0132] The time series module captures the temporal dependency of the feature vector corresponding to the abnormal road surface data and performs temporal attention processing to obtain the temporal feature vector;
[0133] Based on the spatial feature module, the feature vector corresponding to the image data is processed with graph features to obtain the spatial feature vector.
[0134] Optionally, the spatiotemporal feature extraction layer includes a time series module and a spatial feature module. In this case, the feature vector corresponding to the abnormal road surface data is input into the time series module. The time series module captures the forward / backward temporal dependency based on the bidirectional LSTM algorithm, and then calculates the importance weight of each time step based on the temporal attention mechanism, and performs weighted aggregation to obtain the time feature vector. At the same time, the feature vector corresponding to the image data is input into the spatial feature module. The spatial feature module performs graph feature processing on the feature vector corresponding to the image data to obtain the spatial feature vector.
[0135] In an optional embodiment of the present application, the spatial feature module includes a convolutional layer, a graph attention network, and a global pooling layer. The spatial feature module performs graph feature processing on the feature vector corresponding to the image data to obtain a spatial feature vector, including:
[0136] The feature vector corresponding to the image data is input into the convolution layer for spatiotemporal joint feature extraction to obtain spatiotemporal joint features;
[0137] Input the spatiotemporal joint features into the graph attention network to obtain graph features;
[0138] The graph features are compressed based on the global pooling layer to obtain the spatial feature vector.
[0139] Optionally, the spatial feature module includes a convolutional layer, a graph attention network and a global pooling layer. The convolutional layer extracts the spatiotemporal joint features (time × space dimension convolution) of the feature vector corresponding to the image data. Then the graph attention network (GAT) converts the spatiotemporal joint features (image grid form) into a graph structure (node = feature area, edge = spatial adjacency), and aggregates neighborhood information to generate graph features. Finally, the graph features are compressed based on the global pooling layer to obtain the spatial feature vector.
[0140] Optionally, in order to better understand the collapse prediction model in this application, such as Figure 2 As shown, a structural schematic diagram of a collapse prediction model is provided. The collapse prediction model includes a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer and a risk level output layer connected in sequence. The spatiotemporal feature extraction layer includes a connected time series module and a spatial feature module, and the spatial feature module includes a convolutional layer, a graph attention network and a global pooling layer connected in sequence.
[0141] Step S104: Perform collapse warning processing on the monitored road surface according to the collapse risk level.
[0142] Optionally, after the collapse risk level is obtained, different collapse warning processes are performed according to different risk levels.
[0143] In an optional embodiment of the present application, collapse warning processing is performed on the monitored road surface according to the collapse risk level, including:
[0144] If the collapse risk level is level 1, a collapse warning message will be sent to the relevant road departments so that they can take preventive measures in a timely manner;
[0145] If the collapse risk level is the second level, a collapse reminder message and real-time road surface data will be sent to the road-related departments so that the road-related departments can carry out traffic control in a timely manner.
[0146] Optionally, the output collapse risk level is divided into the first level and the second level. The first level indicates that the monitored road surface currently has a high risk of collapse and the possibility of collapse is high. At this time, a collapse warning reminder message can be sent to the road-related departments to enable the road-related departments to conduct further detection and timely processing, which can prevent possible collapse sections.
[0147] The second level indicates that the monitored road surface has collapsed. At this time, a collapse prompt message will be sent to the road-related departments, and real-time road surface data will be promptly fed back to the traffic and road-related departments so that the road-related departments can promptly fence off the road sections and control traffic, thereby reducing losses caused by the collapse.
[0148] In an embodiment of the present application, by obtaining real-time road surface data of the monitored road surface and comparing the real-time road surface data with historical road surface data, after determining that there is abnormal data, the collapse risk level is obtained based on the collapse prediction model. At this time, the road surface collapse can be promptly handled based on the collapse risk level, thereby preventing the road surface collapse from causing losses to the life and property safety of pedestrians and vehicles. In the case of collapse, warning information can be provided to the road and traffic authorities in a timely manner, so that timely measures can be taken to reduce the losses caused by the collapse.
[0149] The embodiment of the present application provides a road collapse warning device, such as Figure 3 As shown, the device 30 may include: a variation determination module 301, an abnormal data determination module 302, a risk level determination module 303 and an early warning processing module 304, wherein:
[0150] a change amount determination module, configured to obtain real-time road surface data of the monitored road surface and historical road surface data corresponding to the monitored road surface, and determine a change amount of the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data;
[0151] The abnormal data determination module is used to obtain the data standard value corresponding to the monitored road surface, and compare the real-time data change with the data standard value to determine whether the real-time road surface data is abnormal road surface data;
[0152] A risk level determination module is used to obtain image data corresponding to the monitored road surface when the real-time road surface data is determined to be abnormal road surface data, and input the abnormal road surface data and the image data into the road collapse prediction model to obtain the collapse risk level;
[0153] The early warning processing module is used to perform collapse early warning processing on the monitored road surface according to the collapse risk level.
[0154] Optionally, the type of the real-time road surface data includes at least one of real-time deformation, real-time stress, real-time temperature, and real-time settlement. When the change amount determination module determines the change amount of the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data, it is specifically configured to:
[0155] Performing alignment and standardization processing on various types of data included in the real-time road surface data to obtain processed real-time road surface data;
[0156] Obtaining a weight corresponding to each type of data, and performing weighted processing on the processed real-time road surface data based on the weight corresponding to each type of data to obtain target real-time road surface data;
[0157] Based on the target real-time road surface data and historical road surface data, the real-time data change of the monitored road surface is determined.
[0158] Optionally, when the variation determination module aligns and standardizes each type of data included in the real-time road surface data to obtain the processed real-time road surface data, it is specifically configured to:
[0159] Performing time alignment processing on various types of data included in the real-time road surface data through a sliding window to obtain aligned real-time road surface data;
[0160] Performing spatial registration processing on the aligned real-time road surface data to obtain registered real-time road surface data;
[0161] The registered real-time road surface data is normalized to obtain processed real-time road surface data.
[0162] Optionally, when the data determination module compares the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data, it is specifically used to:
[0163] Determine the difference between the real-time data change and the data standard value, and compare the difference with the preset threshold;
[0164] If the difference is greater than a preset threshold, the real-time road surface data is determined to be abnormal road surface data;
[0165] If the difference is not greater than the preset threshold, it is determined that the real-time road surface data is not abnormal road surface data.
[0166] Optionally, the collapse prediction model includes a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer, and a risk level output layer. When the risk level determination module inputs abnormal road surface data and image data into the road collapse prediction model to obtain the collapse risk level, it is used to:
[0167] Input the abnormal road surface data and image data into the input layer for feature extraction to obtain the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data;
[0168] Input the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector;
[0169] The spatiotemporal feature vector is input into the cross-modal attention fusion layer for attention calculation to obtain the spatiotemporal fusion feature;
[0170] The spatiotemporal fusion features and the feature vectors corresponding to the image data are input into the risk level output layer for risk decision fusion processing to obtain the collapse risk level.
[0171] Optionally, the spatiotemporal feature extraction layer includes a time series module and a spatial feature module, and the spatiotemporal feature vector includes a time feature vector and a spatial feature vector;
[0172] The risk level determination module inputs the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector, specifically for:
[0173] The time series module captures the temporal dependency of the feature vector corresponding to the abnormal road surface data and performs temporal attention processing to obtain the temporal feature vector;
[0174] Based on the spatial feature module, the feature vector corresponding to the image data is processed with graph features to obtain the spatial feature vector.
[0175] Optionally, the spatial feature module includes a convolutional layer, a graph attention network, and a global pooling layer. When the risk level determination module performs graph feature processing on the feature vector corresponding to the image data based on the spatial feature module to obtain the spatial feature vector, it is specifically used to:
[0176] The feature vector corresponding to the image data is input into the convolution layer for spatiotemporal joint feature extraction to obtain spatiotemporal joint features;
[0177] Input the spatiotemporal joint features into the graph attention network to obtain graph features;
[0178] The graph features are compressed based on the global pooling layer to obtain the spatial feature vector.
[0179] Optionally, when performing collapse warning processing on the monitored road surface according to the collapse risk level, the early warning processing module is specifically used to:
[0180] If the collapse risk level is level 1, a collapse warning message will be sent to the relevant road departments so that they can take preventive measures in a timely manner;
[0181] If the collapse risk level is the second level, a collapse reminder message and real-time road surface data will be sent to the road-related departments so that the road-related departments can carry out traffic control in a timely manner.
[0182] Optionally, after determining that the real-time road surface data is abnormal road surface data, the abnormal data determination module is further configured to:
[0183] Obtain real-time image data corresponding to the monitored road surface;
[0184] The cause of the abnormality is determined based on real-time image data and fed back to the road-related departments so that they can take corresponding measures.
[0185] A road collapse warning device of this embodiment can execute a road collapse warning method shown in the embodiment of this application. The implementation principle is similar and will not be repeated here.
[0186] An embodiment of the present application provides an electronic device, which includes: a processor; and a memory, wherein the memory is configured to store machine-readable instructions, which, when executed by the processor, causes the processor to execute a road collapse warning method.
[0187] The present application embodiment provides an electronic device, such as Figure 4 As shown, Figure 4 The electronic device 2000 shown includes a processor 2001 and a memory 2003. The processor 2001 and the memory 2003 are connected, for example, via a bus 2002. Optionally, the electronic device 2000 may further include a transceiver 2004. It should be noted that in actual applications, the number of transceivers 2004 is not limited to one, and the structure of the electronic device 2000 does not constitute a limitation on the embodiments of the present application.
[0188] Processor 2001 may be a CPU, a general-purpose processor, a DSP, an ASIC, an FPGA, or other programmable logic device, a transistor logic device, a hardware component, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 2001 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.
[0189] The bus 2002 may include a path for transmitting information between the above components. The bus 2002 may be a PCI bus or an EISA bus, etc. The bus 2002 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.
[0190] The memory 2003 may be a ROM or other type of static storage device that can store static information and instructions, a RAM or other type of dynamic storage device that can store information and instructions, or an EEPROM, a CD-ROM or other optical disk storage, an optical disc storage (including a compact disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.
[0191] The memory 2003 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 2001. The processor 2001 is used to execute the application code stored in the memory 2003 to implement Figure 3 The illustrated embodiment provides an operation of a road collapse warning device.
[0192] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0193] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A road collapse early warning method, characterized in that: include: Acquiring real-time road surface data of a monitored road surface and historical road surface data corresponding to the monitored road surface, and determining a change in the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data; Obtaining a data standard value corresponding to the monitored road surface, and comparing the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data; If it is determined that the real-time road surface data is abnormal road surface data, image data corresponding to the monitored road surface is obtained, and the abnormal road surface data and the image data are input into a road collapse prediction model to obtain a collapse risk level; Perform collapse warning processing on the monitored road surface according to the collapse risk level.
2. The method according to claim 1, characterized in that The type of the real-time road surface data includes at least one of real-time deformation, real-time stress, real-time temperature, and real-time settlement. The determining of the real-time data change of the monitored road surface based on the real-time road surface data and the historical road surface data includes: performing alignment and standardization processing on each type of data included in the real-time road surface data to obtain processed real-time road surface data; Obtaining a weight corresponding to each type of data, and performing weighted processing on the processed real-time road surface data based on the weight corresponding to each type of data to obtain target real-time road surface data; Based on the target real-time road surface data and the historical road surface data, a real-time data change amount of the monitored road surface is determined.
3. The method according to claim 2, characterized in that The aligning and standardizing each type of data included in the real-time road surface data to obtain processed real-time road surface data includes: Performing time alignment processing on various types of data included in the real-time road surface data through a sliding window to obtain aligned real-time road surface data; Performing spatial registration processing on the aligned real-time road surface data to obtain registered real-time road surface data; Normalization processing is performed on the registered real-time road surface data to obtain the processed real-time road surface data.
4. The method according to claim 1, wherein The comparing the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data includes: Determine the difference between the real-time data change and the data standard value, and compare the difference with a preset threshold; If the difference is greater than the preset threshold, determining that the real-time road surface data is abnormal road surface data; If the difference is not greater than the preset threshold, it is determined that the real-time road surface data is not abnormal road surface data.
5. The method according to claim 1, wherein The collapse prediction model includes a multimodal input layer, a spatiotemporal feature extraction layer, a cross-modal attention fusion layer, and a risk level output layer. The abnormal road surface data and the image data are input into the road collapse prediction model to obtain the collapse risk level, including: Inputting the abnormal road surface data and the image data into the input layer for feature extraction to obtain a feature vector corresponding to the abnormal road surface data and a feature vector corresponding to the image data; Inputting the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain a spatiotemporal feature vector; Inputting the spatiotemporal feature vector into the cross-modal attention fusion layer for attention calculation to obtain spatiotemporal fusion features; The spatiotemporal fusion features and the feature vector corresponding to the image data are input into the risk level output layer for risk decision fusion processing to obtain the collapse risk level.
6. The method according to claim 2, characterized in that The spatiotemporal feature extraction layer includes a time series module and a spatial feature module, and the spatiotemporal feature vector includes a time feature vector and a spatial feature vector; The step of inputting the feature vector corresponding to the abnormal road surface data and the feature vector corresponding to the image data into the spatiotemporal feature extraction layer for spatiotemporal feature processing to obtain the spatiotemporal feature vector includes: Capturing the temporal dependency of the feature vector corresponding to the abnormal road surface data based on the time series module and performing time attention processing to obtain the temporal feature vector; Based on the spatial feature module, the feature vector corresponding to the image data is processed with graph features to obtain the spatial feature vector.
7. The method according to claim 6, characterized in that The spatial feature module includes a convolution layer, a graph attention network and a global pooling layer. The spatial feature module performs graph feature processing on the feature vector corresponding to the image data to obtain the spatial feature vector, including: Inputting the feature vector corresponding to the image data into the convolution layer to perform spatiotemporal joint feature extraction processing to obtain spatiotemporal joint features; Inputting the spatiotemporal joint features into the graph attention network to obtain graph features; The graph features are compressed based on the global pooling layer to obtain the spatial feature vector.
8. The method according to claim 1, characterized in that The performing collapse warning processing on the monitored road surface according to the collapse risk level includes: If the collapse risk level is level 1, a collapse warning message is sent to the road-related departments so that the road-related departments can take preventive measures in a timely manner; If the collapse risk level is the second level, collapse prompt information and the real-time road surface data are sent to the road-related departments so that the road-related departments can carry out traffic control in a timely manner.
9. The method according to claim 1, characterized in that After determining that the real-time road surface data is abnormal road surface data, the method further includes: Acquiring real-time image data corresponding to the monitored road surface; The cause of the abnormality is determined based on the real-time image data, and the cause of the abnormality is fed back to the road-related department so that the road-related department can take corresponding measures.
10. A road collapse warning device, characterized in that: include: a change amount determination module, configured to obtain real-time road surface data of a monitored road surface and historical road surface data corresponding to the monitored road surface, and determine a change amount of the real-time data of the monitored road surface based on the real-time road surface data and the historical road surface data; an abnormal data determination module, configured to obtain a data standard value corresponding to the monitored road surface, and compare the real-time data variation with the data standard value to determine whether the real-time road surface data is abnormal road surface data; a risk level determination module, configured to, when determining that the real-time road surface data is abnormal road surface data, obtain image data corresponding to the monitored road surface, and input the abnormal road surface data and the image data into a road collapse prediction model to obtain a collapse risk level; The early warning processing module is used to perform collapse early warning processing on the monitored road surface according to the collapse risk level.