A road collapse risk prediction method based on deep learning multi-source data fusion

Abstract

This invention relates to the field of road safety monitoring and risk early warning, specifically disclosing a road collapse risk prediction method based on deep learning multi-source data fusion. The method includes: collecting multi-source data related to road collapse and constructing a road collapse risk dataset; establishing a multi-source data fusion prediction model based on deep learning, including image coding branch, underground coding branch, deformation coding branch, feature fusion layer, and prediction output layer; inputting the fused features into the prediction network to output a road collapse risk index or probability of occurrence; inputting the risk index or probability of occurrence into a cloud model risk grader to output the road collapse risk level and corresponding membership degree or confidence level, and generating a risk heatmap and early warning information. This invention can achieve robust fusion and risk prediction under conditions of incomplete or fluctuating multi-source data, and improves the usability of engineering applications through interpretable risk grading results, providing technical support for road inspection, maintenance decision-making, and emergency response.

Description

Technical Field

[0001] This invention relates to the field of road safety monitoring and risk early warning technology, specifically to a method for predicting road collapse risk based on deep learning multi-source data fusion. Background Technology

[0002] Road collapses are characterized by their suddenness and high hazard. Their formation and development are often related to factors such as the evolution of pavement defects, underground anomalies such as underground cavities or pipeline leaks, surface subsidence and deformation, and external disturbances. Existing methods for assessing or warning of road collapse risks mostly rely on a single data source or a limited number of indicators, making them susceptible to noise, differences in sampling scales, and missing information, resulting in insufficient accuracy and stability in risk identification.

[0003] Meanwhile, data from road surface images, underground surveys, and surface deformation monitoring differ significantly in data format, spatial reference frame, and time scale. Without effective preprocessing and spatiotemporal alignment mechanisms, it is difficult to achieve collaborative analysis and fusion modeling of the same road location. Furthermore, in engineering practice, multi-source data often suffers from missing data or uneven quality, further reducing the reliability of the fusion results and making the risk level output lack interpretable evidence.

[0004] Therefore, there is an urgent need for a method that can preprocess and spatiotemporally align multi-source data such as road surface images, underground detection, and surface deformation monitoring, and achieve robust fusion under conditions of missing or quality fluctuations, outputting a prediction method for road collapse risk level with confidence characterization, so as to improve the accuracy, robustness, and engineering applicability of road collapse risk prediction. Summary of the Invention

[0005] To address the problems in existing technologies for predicting road collapse risks, such as strong heterogeneity of multi-source data, unreliable fusion results due to missing or unstable data sources, and lack of interpretable classification and confidence expression for risk output, this invention provides a road collapse risk prediction method based on deep learning multi-source data fusion. This method enables the prediction of collapse risk index, risk level determination, and early warning interpretation for target roads, providing technical support for road maintenance management and emergency response, and solving the problems mentioned in the background.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a road collapse risk prediction method based on deep learning multi-source data fusion, comprising the following steps: S1. Collect multi-source data related to road collapse, including road surface image data, underground detection data, and surface deformation monitoring data; S2. Preprocess and align the collected multi-source data spatiotemporally to form a sample dataset and divide it into a training set, a validation set, and a test set. S3. Train and validate the deep learning multi-source fusion prediction network using training and validation set data to obtain the optimal deep learning multi-source fusion prediction network. S4. Input the test set data into the optimal deep learning multi-source fusion prediction network for feature extraction and fusion, and output the probability of road collapse or risk index. S5. Generate road collapse risk levels based on the probability of risk occurrence or risk index, and output risk distribution maps and early warning information.

[0007] Preferably, the road surface image data includes visible road surface images and hyperspectral road surface images; the underground detection data includes ground penetrating radar data or underground cavity detection data; and the surface deformation monitoring data includes subsidence monitoring data or interferometric synthetic aperture radar deformation monitoring data.

[0008] Preferably, the preprocessing includes anomaly data processing, missing data processing, noise reduction processing, and normalization processing; the spatiotemporal alignment includes spatial registration and temporal matching.

[0009] Preferably, the spatial registration includes: Mapping data from different sources to a unified road space reference system; Road surface image data, underground detection data, and surface deformation monitoring data are linked according to the road location index to form multi-source data records corresponding to the same road location.

[0010] Preferably, the time matching includes: Time alignment is performed on multi-source data with different sampling frequencies according to a preset time window; Statistical aggregation or interpolation alignment is used to form time window samples from multi-source data within the same time window, so that the samples contain multi-source input features corresponding to road locations and time windows.

[0011] Preferably, the deep learning multi-source fusion prediction network includes: The image coding branch is used to extract features from road image data and output image feature vectors; The underground coding branch is used to extract features from underground exploration data and output underground feature vectors; The deformation coding branch is used to extract features from surface deformation monitoring data and output deformation feature vectors. The feature fusion layer is used to fuse image feature vectors, underground feature vectors, and deformation feature vectors and output fused features. The prediction output layer is used to output the probability of road collapse or risk index based on the fusion features.

[0012] Preferably, the feature fusion layer includes a missing labeling mechanism and a gating weighting mechanism; the missing labeling mechanism is used to characterize whether each data source is missing or invalid; the gating weighting mechanism is used to reduce the weight of the corresponding feature in the fusion when the data source is missing or its quality is reduced, so as to improve the robustness of risk prediction.

[0013] Preferably, the road collapse risk level is generated by a cloud model risk classifier. The cloud model risk classifier maps the probability of risk occurrence or risk index to a preset risk level and outputs the corresponding membership degree or confidence degree for interpretable expression of the risk level results. The preset risk level includes one or more of low risk, medium risk, and high risk.

[0014] Preferably, the risk distribution map includes a road collapse risk heat map, and the early warning information includes early warning location identification information or early warning threshold triggering information.

[0015] Preferably, in the training and validation of the deep learning multi-source fusion prediction network, the training set is used to iteratively train the deep learning multi-source fusion prediction network; the validation set is used to optimize the network hyperparameters, obtain model evaluation indicators, and determine the optimal model.

[0016] Preferably, the model evaluation metrics include one or more of the following: accuracy, recall, F1 score, mean squared error, and mean absolute error, used to measure the effectiveness and stability of risk prediction.

[0017] Preferably, the multi-source data may further include one or more of meteorological rainfall data, traffic load data, water-related pipeline information, and construction activity information. The further multi-source data is then used for feature extraction through a structured data coding branch and then participates in feature fusion to improve the completeness and applicability of risk prediction.

[0018] The beneficial effects of this invention are: 1) This invention introduces three types of data—road surface images, underground detection, and surface deformation monitoring—for joint modeling, which can more comprehensively characterize the appearance of road collapse, underground anomalies, and disaster-causing conditions, thereby improving the accuracy and applicability of risk prediction. 2) This invention achieves adaptive processing of missing data and quality fluctuations from multiple sources through a missing data labeling mechanism and a gated weighting mechanism, thereby improving the stability and robustness of data fusion and reducing the risk of misjudgment due to the unavailability of a single data source. 3) This invention realizes risk level determination and membership expression through the hierarchical output layer of the cloud model, so that the prediction results have interpretability and uncertainty expression capabilities, which facilitates engineers to carry out hierarchical disposal and early warning decision-making. Attached Figure Description

[0019] Figure 1This is a schematic diagram of a road collapse risk prediction method based on deep learning and multi-source data fusion. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] This invention provides a technical solution: a road collapse risk prediction method based on deep learning multi-source data fusion, such as... Figure 1 As shown, it includes the following steps: S1. Collect multi-source data related to road collapse.

[0022] Multi-source data related to road collapse includes, but is not limited to, road surface image data, underground exploration data, surface deformation monitoring data, and other auxiliary data. The methods for collecting and organizing these data types are as follows: (1) Road surface image data Visible light road surface photo acquisition: High-resolution road surface photos of the target road are acquired using vehicle-mounted cameras, handheld cameras, or fixed monitoring equipment to obtain appearance information of defects such as cracks, potholes, and repairs; the road location and time information corresponding to the image are recorded during acquisition to achieve spatial correspondence and temporal matching with other data.

[0023] Hyperspectral road surface image acquisition: Carrier-mounted, UAV-based, or ground-based hyperspectral imaging equipment is used to acquire cubic data of hyperspectral images of the road surface to characterize the road material state and subtle surface changes. During acquisition, the road location and time information corresponding to the image are recorded to achieve spatial correspondence and temporal matching with other data.

[0024] Defect labeling and sample construction: Defect category and location are labeled on the collected road surface image samples. The labeling forms include one or more of the following: target box labeling and pixel-level segmentation labeling, which provides supervision information for subsequent extraction of defect severity indicators.

[0025] Hyperspectral road surface image acquisition: Carrier-mounted, UAV-based, or ground-based hyperspectral imaging equipment is used to acquire cubic data of hyperspectral images of the road surface to characterize the road material state and subtle surface changes. During acquisition, the road location and time information corresponding to the image are recorded to achieve spatial correspondence and temporal matching with other data.

[0026] Hyperspectral preprocessing and feature construction: Radiometric correction and reflectance normalization are performed on the hyperspectral images, and bands are selected or dimensionality reduced to extract spectral features that characterize water content, aging or material changes as supplementary feature inputs to the road image data.

[0027] (2) Underground exploration data Ground penetrating radar data acquisition: Ground penetrating radar equipment is used to scan along the road longitudinally or laterally to obtain underground medium reflection profile data, which is used to characterize the hidden dangers such as underground cavities, loose areas, voids and abnormal water content; during the acquisition, the scanning trajectory, mileage station number or positioning information is recorded simultaneously to achieve spatial correspondence with the road location unit.

[0028] Sample construction: The ground-penetrating radar profile is preprocessed and sliced, and the profile segment corresponding to the road location unit is used as the sample input; the label of underground anomaly can be determined based on on-site verification, existing hazard records or expert interpretation results, and used for subsequent underground feature coding and risk learning.

[0029] (3) Surface deformation monitoring data LiDAR data acquisition: The three-dimensional point cloud data of the target road area is acquired by using vehicle-mounted LiDAR, UAV LiDAR or ground LiDAR scanning, and digital elevation model or road surface elevation profile is generated to characterize the micro-topographic features and morphological changes of the road and surrounding surface.

[0030] Sample construction: Multiple LiDAR data collections were conducted on the same area at different time periods. The point cloud or elevation model was registered and differentially calculated to obtain deformation characteristics such as settlement, uplift, or elevation change rate. The deformation characteristics were mapped to the road spatial reference system, and the deformation statistics within the road location unit were used as the deformation characteristics input for that road location unit to characterize the impact of uneven settlement and surface deformation trends along the road on the risk of collapse.

[0031] (4) Other auxiliary data Meteorological rainfall data: Information such as rainfall amount and duration of continuous rainfall in the target area is collected to characterize the impact of rainfall infiltration and groundwater level changes on the risk of collapse.

[0032] Traffic load data: Collect information such as traffic flow, proportion of loaded vehicles or equivalent axle load to characterize the impact of traffic load on roadbed structure and underground cavity expansion.

[0033] Data on water-related pipelines and construction activities: Collect information on pipeline type, burial depth, leakage records, and construction excavation activities to characterize the impact of human disturbance and pipeline leakage on the risk of road collapse.

[0034] This embodiment organizes the samples using "road location units". Road location units can be divided into equal-length segments according to road mileage direction. Let the first segment be...i The mileage range corresponding to each road location unit is [s] i ,s i +Δs]. A sample record is created by combining road surface images, underground survey data, and surface deformation data within the same road location unit and the same time window.

[0035] The dataset is divided into training, validation, and test sets according to a set ratio for subsequent model training, tuning, and testing.

[0036] S2. Preprocess and align the collected multi-source data spatiotemporally to form a sample dataset, which is then divided into a training set, a validation set, and a test set.

[0037] S21. Anomaly and Missing Data Handling. Perform validity checks on each data source and remove obviously abnormal records; retain missing data records with missing identifiers to provide input for subsequent missing data labeling mechanisms.

[0038] S22. Denoising and Normalization. Denoising and normalization are performed on numerical data. Normalization can be achieved using minimum-maximum normalization, the expression of which is: ; in, x The original value, x min and x max These are the minimum and maximum values ​​of the sample statistics, respectively. x' This is the result of normalization.

[0039] Furthermore, radiometric correction and reflectance calculation are performed on the close-range hyperspectral images. Let the original digital quantity of the hyperspectral image be... I Dark reference is D White is the reference. W Then reflectivity R Calculate as follows: ; in epsilon To prevent extremely small positive numbers with a denominator of zero.

[0040] Furthermore, the hyperspectral images are subjected to band processing and feature construction, including band selection, noise reduction or dimensionality reduction; and spectral statistical features are extracted within the road location unit as input for the pavement material state features.

[0041] S23. Spatial Registration. Mapping data from different sources to a unified road spatial reference system and indexing it using road location units. i By associating multi-source data, each data source can be spatially mapped to the same road segment.

[0042] S24. Time Matching. Align data from different sampling frequencies to a unified time window. Let the time window length be Δ. t , No. k The time window is [ t k , t k + Δ t Within the same time window, statistical aggregation can be used to form representative values, such as the mean: ; in, n k This represents the number of samples taken within the time window. x k,j For the first k Within the first time window j One observation value.

[0043] Furthermore, a road collapse risk sample dataset was constructed from the preprocessed and spatiotemporally aligned data, and divided into training, validation, and test sets.

[0044] S3. Train and validate the deep learning multi-source fusion prediction network using training and validation set data to obtain the optimal deep learning multi-source fusion prediction network.

[0045] The multi-source fusion prediction network was iteratively trained using the training set, and the training parameters were optimized using the validation set to obtain the optimal model. The optimal model was then tested using the test set.

[0046] In this embodiment, the training objective function may include risk prediction loss and gating constraint loss. Taking risk occurrence probability supervision as an example, the risk prediction loss can use binary cross-entropy: ; in, y ∈{0,1} represents the risk label. p To predict probabilities.

[0047] To suppress anomalies in missing mode weights, a missing mode gate constraint loss can be added: ; The total loss function can be expressed as: ; in, lambda This is a weighting factor.

[0048] The model testing phase outputs evaluation metrics, including one or more of the following: accuracy, recall, F1 score, mean squared error, and mean absolute error. When the model output is the probability or level of road collapse risk, accuracy, precision, recall, and F1 score are used for evaluation; when the model output is a continuous risk index, mean squared error and mean absolute error are used for evaluation.

[0049] Accuracy is expressed as: ; Precision and recall are expressed as follows: ; The F1 value is expressed as: ; The mean square error and the mean absolute error are expressed as follows: , ; in, TP , TN , FP , FN These represent the number of true positive, true negative, false positive, and false negative samples, respectively. y i For the true value, For predicted values, n This represents the number of samples.

[0050] S4. Using a deep learning multi-source fusion prediction network, feature encoding is performed on road surface images, underground detection data, and surface deformation data respectively, providing input features for subsequent fusion. The test set data is input into the optimal deep learning multi-source fusion prediction network for feature extraction and fusion, outputting the probability of road collapse or risk index.

[0051] The deep learning multi-source fusion prediction network includes a road image feature extraction branch (image coding branch), an underground detection feature extraction branch (underground coding branch), a surface deformation feature extraction branch (deformation coding branch), a feature fusion layer, and a risk prediction output layer. Each branch outputs a corresponding modal feature vector, denoted as z. img z gpr z def .

[0052] S41. Road Surface Image Feature Extraction. Road surface image feature extraction involves extracting features from the input road surface image data, which includes visible light images and near-field hyperspectral images.

[0053] The visible light image sub-branch outputs the damage detection and segmentation results. The detection output includes the bounding box and category, while the segmentation output includes a pixel-level mask. The hyperspectral sub-branch extracts the spectral features of the pavement material state and outputs a hyperspectral feature vector.

[0054] S42. Calculation of Disease Severity Indicators. Calculate indicators such as the percentage of diseased area based on segmented masks. Let a certain type of disease mask be... M ( x, y )∈{0,1}, the total number of valid pixels within the ROI is N The percentage of diseased area can then be expressed as: ; For crack-type defects, the length density can be further calculated. The crack mask is refined to obtain a skeleton set Ω. Let the number of skeleton pixels be |Ω|, and the actual length corresponding to the pixel resolution be... r The actual area covered by the image is S Then the crack length density can be expressed as: ; Construction of hyperspectral material state characteristics: Statistical features of hyperspectral reflectance are extracted from the disease mask area or road location unit area, and spectral indices are constructed. Let the selected band reflectance be... and Then the spectral index I Represented as: ; in epsilon It is a very small positive number.

[0055] Image branch internal fusion: The image feature vector z output from the visible photon branch... rgb The eigenvector z of the hyperspectral sub-branch output hsi By fusing the data, the branch feature vector z of the road surface image is obtained. img The fusion method can be either concatenation mapping or weighted fusion; for example, the weighted fusion expression is: ; in β rgb and β hsi Let be the weight coefficient, and satisfy... β rgb + β hsi =1.

[0056] The above-mentioned disease severity indicators, together with deep image features, form the image branch feature vector z. img .

[0057] S43. Subsurface Detection Feature Extraction. Ground-penetrating radar profile data is treated as a two-dimensional tensor input to the subsurface coding network, outputting a subsurface feature vector z. gpr If the radar data has multiple input channels, it can be stacked along the channel dimension before being input into the encoding network.

[0058] S44. Surface Deformation Feature Extraction. Feature encoding is performed on surface deformation monitoring data to output a deformation feature vector z. def When the deformed data is a time series { d 1 , d 2 ,…, d m When}, sedimentation rate characteristics can be constructed: ; Where, Δ t j The interval between adjacent observations is defined as follows. The settlement and velocity sequences are input together into the deformation coding network to obtain z. def .

[0059] S45. Feature Fusion: Construct a missing labeling mechanism and a gated weighted fusion mechanism to achieve adaptive fusion of multi-source features.

[0060] S451. Missing Data Label Vector Construction. The availability of three types of data sources is labeled to construct a missing data label vector: ; in, m img , m gpr , m def ∈{0,1}, takes the value 1 when the corresponding data source is available, and takes the value 0 when it is unavailable or invalid.

[0061] S452. Gating score calculation. Calculate the gating score for each modality's eigenvector. s i In this embodiment, the gate scoring can be implemented using a small gate network, as shown in the example expression: ; Among them, z i Indicates the first i One modal feature vector, [• •] indicates vector concatenation, w i and b i These are learnable parameters.

[0062] S453, Gating Weight Normalization. Normalize the gating scores to obtain the modal weights: ; S454, Missing Mode Constraints and Weighted Fusion. To avoid missing modes affecting the fusion, missing mode markers are introduced into the weight constraints: ; in, epsilon To prevent extremely small positive numbers with a denominator of zero, the final fusion feature is: ; Among them, z i ∈{z img ,z gpr ,z def}

[0063] By using the missing data marker and gating weighting mechanism described above, the contribution of a certain modality to the fusion features can be automatically reduced when a data source is missing or its quality fluctuates, thereby improving the robustness of risk prediction.

[0064] S46. Output the probability of road collapse risk or risk index based on fusion features.

[0065] The risk prediction layer can use a fully connected network to map fused features into a risk index. R Example expression: ; in, f (•) is a mapping function consisting of multiple fully connected layers and nonlinear activations.

[0066] When the output is the probability of risk occurrence p In this case, a Sigmoid mapping can be used: ; S5. Generate road collapse risk levels based on the probability of risk occurrence or risk index, and output risk distribution maps and early warning information.

[0067] Furthermore, the probability of risk occurrence or risk index is input into the cloud model risk classifier, which outputs the risk level and membership degree or confidence level.

[0068] This embodiment uses a three-level risk system as an example: low risk, medium risk, and high risk. For the third... k Each risk level sets cloud model parameters ( E xk , E nk , H ek ),in E xk For the expectation, E nk For entropy, Hek It is hyperentropy.

[0069] During the cloud model generation process, random entropy is first generated: ; For a given risk index R Its impact on the first k The membership degree of a level can be represented as: ; Risk level can be determined by the maximum membership degree: ; And It serves as a confidence level indicator for this risk level and is used for engineering interpretation and early warning output.

[0070] Furthermore, the risk level and confidence level of each location unit on the road can be mapped to a road risk heat map, which can be used to visually display high-risk locations and key areas for early warning. Early warning information includes warning location identification information or warning threshold trigger information.

[0071] This invention proposes a road collapse risk prediction method based on deep learning and multi-source data fusion. Through joint modeling of road surface image data, underground detection data, and surface deformation monitoring data, it outputs the probability or risk index of road collapse risk and further realizes risk level classification and visualization. Compared with existing technologies, this invention can fully explore the complementary information between multi-source data, improving the accuracy, robustness, and timeliness of risk prediction. It provides a scientific basis for road maintenance management and emergency response, and is applicable to high-risk road collapse scenarios such as urban roads, underground engineering construction areas, and mining areas.

[0072] This invention introduces a missing data labeling mechanism and a gated weighting mechanism in the process of multi-source data fusion: when some modal data is missing or of unstable quality, the fusion weight of the missing modality can be adaptively suppressed, thereby reducing the negative impact of missing modalities on fusion features and improving the stability and reliability of risk prediction results.

[0073] This invention employs a cloud model risk grading mechanism to map risk indices or risk probabilities to risk levels and output corresponding membership or confidence levels, thereby enabling interpretable expression of risk grading results. Simultaneously, it can map the risk level or risk index of road units to a risk heat map, providing intuitive support for risk area identification, inspection priority ranking, and treatment decisions.

[0074] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0075] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0076] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0077] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0078] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for predicting road collapse risk based on deep learning multi-source data fusion, characterized in that, Includes the following steps: S1. Collect multi-source data related to road collapse, including road surface image data, underground detection data, and surface deformation monitoring data; S2. Preprocess and align the collected multi-source data spatiotemporally to form a sample dataset and divide it into a training set, a validation set, and a test set. S3. Train and validate the deep learning multi-source fusion prediction network using training and validation set data to obtain the optimal deep learning multi-source fusion prediction network. S4. Input the test set data into the optimal deep learning multi-source fusion prediction network for feature extraction and fusion, and output the probability of road collapse or risk index. S5. Generate road collapse risk levels based on the probability of risk occurrence or risk index, and output risk distribution maps and early warning information.

2. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 1, characterized in that: The road surface image data includes visible road surface images and hyperspectral road surface images; the underground detection data includes ground penetrating radar data or underground cavity detection data; the surface deformation monitoring data includes subsidence monitoring data or interferometric synthetic aperture radar deformation monitoring data.

3. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 1, characterized in that: The preprocessing includes abnormal data processing, missing data processing, noise reduction processing, and normalization processing; the spatiotemporal alignment includes spatial registration and temporal matching.

4. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 3, characterized in that: The spatial registration includes: Mapping data from different sources to a unified road space reference system; Road surface image data, underground detection data, and surface deformation monitoring data are linked according to the road location index to form multi-source data records corresponding to the same road location.

5. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 3, characterized in that: The time matching includes: Time alignment is performed on multi-source data with different sampling frequencies according to a preset time window; Statistical aggregation or interpolation alignment is used to form time window samples from multi-source data within the same time window, so that the samples contain multi-source input features corresponding to road locations and time windows.

6. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 1, characterized in that: The deep learning multi-source fusion prediction network includes: The image coding branch is used to extract features from road image data and output image feature vectors; The underground coding branch is used to extract features from underground exploration data and output underground feature vectors; The deformation coding branch is used to extract features from surface deformation monitoring data and output deformation feature vectors. The feature fusion layer is used to fuse image feature vectors, underground feature vectors, and deformation feature vectors and output fused features. The prediction output layer is used to output the probability of road collapse or risk index based on the fusion features.

7. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 6, characterized in that: The feature fusion layer includes a missing labeling mechanism and a gating weighting mechanism; the missing labeling mechanism is used to characterize whether each data source is missing or invalid; the gating weighting mechanism is used to reduce the weight of the corresponding feature in the fusion when the data source is missing or its quality is reduced, so as to improve the robustness of risk prediction.

8. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 1, characterized in that: The road collapse risk level is generated by the cloud model risk classifier, which maps the probability of risk occurrence or risk index to a preset risk level and outputs the corresponding membership degree or confidence degree for interpretable expression of the risk level results.

9. The road collapse risk prediction method based on deep learning multi-source data fusion according to claim 1, characterized in that: The risk distribution map includes a heat map of road collapse risk, and the early warning information includes early warning location identification information or early warning threshold trigger information.

Images