Urban pavement crack or collapse three-dimensional detection and evolution monitoring system based on binocular camera

Abstract

The application discloses a kind of based on binocular camera's city road surface crack or collapse three-dimensional detection and evolution monitoring system, comprising: binocular camera unit, for obtaining the double field of view image of road monitoring area, preliminary generation with regular grid as carrier ground DEM, output initial parallax diagram;Environment sensing unit, for real-time acquisition of environmental parameters, constructs environmental noise compensation model, generates environmental compensation amount;Edge computing unit, for extracting road surface real deformation based on calculated DEM time series data and excluding environmental interference, obtain time series difference data;Then the time series difference data is combined with environmental compensation amount, and significance determination is carried out;Cloud analysis module is used for integrating heterogeneous monitoring data, constructing multi-source time series database, predicting road surface deformation trend based on machine learning time series model and triggering graded early warning.The application can realize the intelligent monitoring and disaster warning of city road structure, and is suitable for the field of intelligent city infrastructure safety monitoring.

Description

Technical Field

[0001] This invention belongs to the field of smart city infrastructure monitoring technology, specifically relating to a three-dimensional detection and evolution monitoring system for urban road surface cracks or collapses based on binocular cameras. Background Technology

[0002] Urban roads, bridges, and subway entrances are subject to multiple factors during long-term service, including traffic loads, rainwater erosion, groundwater level fluctuations, and foundation settlement, making them highly susceptible to early-stage defects such as micro-cracks or collapses.

[0003] Existing monitoring methods include manual inspection, laser scanning, and monocular video detection. Among these, manual inspection is inefficient and highly subjective; laser scanning, while highly accurate, is expensive and complex; and monocular video detection is limited by lighting and viewing angle, making it difficult to obtain true three-dimensional deformation information.

[0004] Therefore, there is an urgent need for a low-cost, high-precision, and long-term automatic intelligent visual monitoring system to achieve real-time three-dimensional detection and evolution trend analysis of cracks or collapses in urban roads. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a three-dimensional detection and evolution monitoring system for urban road surface cracks or collapses based on binocular vision. This system establishes a digital elevation model (DEM) of the land surface through long-term fixed-point binocular monitoring, achieving automatic detection and trend prediction of cracks or collapses with millimeter-level accuracy. The system includes:

[0006] The binocular camera unit is used to acquire dual-field-of-view images of the road monitoring area, initially generate a surface DEM with a regular grid as the carrier, and output the initial disparity map to the edge calculation unit;

[0007] An environmental sensing unit is used to collect environmental parameters in real time, build an environmental noise compensation model, generate environmental compensation quantities, and transmit them to the edge computing unit.

[0008] The edge computing unit is used to extract the actual road surface deformation and eliminate environmental interference based on the calculated DEM time-series data to obtain time-series differential data; then, the time-series differential data is combined with the environmental compensation amount to determine the significance.

[0009] The cloud-based analytics module is used to integrate the heterogeneous monitoring data from the edge computing unit and the environmental sensing unit, construct a multi-source time-series database, predict road surface deformation trends based on machine learning time-series models, and trigger graded early warnings.

[0010] The present invention produces the following beneficial effects:

[0011] This invention establishes a digital elevation model (DEM) of the earth's surface using binocular vision, enabling continuous monitoring of road surface elevation. Employing a time-series elevation difference and significance determination algorithm, combined with a temperature and load compensation model, it can achieve millimeter-level crack or collapse detection and dynamic evolution analysis. Connecting the cloud-based analysis module to a road inspection robot or drone monitoring platform via an IoT interface enables remote, continuous monitoring of urban road structures.

[0012] This invention features a compact structure, high detection accuracy, low cost, and stable operation. It enables intelligent monitoring and disaster early warning of urban road structures and is applicable to the field of smart city infrastructure safety monitoring. Attached Figure Description

[0013] Figure 1 This is an overall architecture diagram of a binocular vision-based three-dimensional detection and evolution monitoring system for urban road surface cracks or collapses, according to an embodiment of this application.

[0014] Figure 2 This is a schematic diagram of the physical installation of a test example for this application.

[0015] Figure 3 This is a flowchart of the binocular elevation modeling algorithm used in the test case of this application.

[0016] Figure 4 This is a flowchart for determining and screening the significance of diseases in the test cases of this application.

[0017] Figure 5 This is a schematic diagram of the LSTM evolution prediction model used in the test case of this application. Detailed Implementation

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] like Figure 1 As shown, this application provides a three-dimensional detection and evolution monitoring system for urban road surface cracks or collapses based on binocular vision. The system follows a main workflow: "binocular camera acquisition of left and right field-of-view images—parallax calculation—triangulation to obtain millimeter-level DEM—introduction of temperature, vibration, and load compensation—time series differencing to obtain instantaneous and cumulative deformation—statistical thresholding and morphological screening to obtain diseased areas—cloud-based fusion prediction and graded early warning." This system aims to achieve three-dimensional detection and evolution monitoring of urban road surface cracks or collapses. It includes:

[0020] (1) A binocular camera unit is used to acquire dual-field-of-view (left and right fields of view) images of the road monitoring area, generate surface depth information through binocular vision reconstruction technology, initially generate a digital elevation model (DEM) of the surface based on a regular grid, and output an initial disparity map to the edge computing unit. Among them, the DEM is the core data foundation reflecting the micro-deformation of the road surface, and each pixel in its grid corresponds to the absolute elevation value of the actual position of the road surface.

[0021] Furthermore, the working mode of the binocular camera unit is as follows:

[0022] Step 1: Install two high-resolution industrial cameras (resolution ≥ 5 megapixels, frame rate ≥ 15fps) on a fixed bracket above the road to form a fixed baseline binocular structure. The baseline length is calibrated using a laser rangefinder. The matching calibration module periodically calibrates the camera's intrinsic parameters (focal length, distortion coefficient) and extrinsic parameters (relative attitude, baseline distance) using a fixed reference point on the road surface to eliminate attitude drift errors during long-term operation.

[0023] Step 2: Perform basic hardware optimization on the acquired original images of the left and right fields of view: perform radial and tangential distortion correction based on calibration parameters to correct image distortion caused by lens manufacturing errors; solve the problem of uneven illumination by grayscale normalization; and use Gaussian filtering to smooth high-frequency noise and preserve road surface texture details.

[0024] Step 3: The pixel-level SAD (Sum of Absolute Differences) matching algorithm is used to calculate the grayscale difference within the search window of the left and right images, output the initial disparity map, and directly transmit it to the edge computing unit as the core input data for elevation modeling.

[0025] (2) An environmental sensing unit is used to collect environmental parameters in real time, build an environmental noise compensation model, generate environmental compensation amount and transmit it to the edge computing unit.

[0026] Furthermore, the environmental sensing unit integrates a temperature sensor, a vibration sensor, and a traffic load detection module at the hardware level to collect environmental parameters in real time, thereby constructing an environmental noise compensation model and providing accurate environmental compensation parameters for the edge computing unit.

[0027] The temperature sensor is a high-precision thermistor temperature sensor (measurement accuracy ±0.1℃, sampling frequency 1Hz), used to collect the road surface temperature and the ambient temperature around the equipment in real time, and calculate the temperature change to provide a compensation benchmark for elevation drift caused by thermal expansion and contraction.

[0028] The vibration sensor is a triaxial accelerometer (measurement range 0-20m / s², sampling frequency 10Hz) used to monitor the vibration acceleration characteristics caused by heavy vehicle traffic or wind in real time.

[0029] The traffic load detection module uses an embedded strain gauge load sensor (measurement accuracy ±0.1kN, sampling frequency 0.1Hz) to acquire the road surface bearing pressure characteristics. In this embodiment, the traffic load detection module uses a piezoelectric thin film sensor pre-embedded under the road surface to monitor the dynamic load of passing vehicles in real time and converts it into quantified load parameters.

[0030] Furthermore, the environmental noise compensation model is a mathematical model constructed based on a multivariate regression algorithm. Its core is to establish a quantitative mapping relationship between "environmental parameters and spurious elevation changes," retaining only the real deformation data caused by cracks or collapses. It can be divided into a training phase and a real-time correction phase.

[0031] ① Training Phase: Completed before system deployment, this phase is used for comparative experiments on pavement without defects using different materials. Model coefficients are obtained based on a multivariate regression model and embedded into the environmental sensing unit's embedded program. The specific workflow is as follows:

[0032] Three standard road sections without defects were selected, each with different materials (including asphalt, concrete, and composite pavement). Sensors consistent with the monitoring system were deployed and environmental parameters and corresponding elevation deviation data were continuously collected for 30 days.

[0033] The collected sample data were divided into training and validation sets in an 8:2 ratio and substituted into the fitting coefficients of the multivariate regression model.

[0034] Once the validation set accuracy meets the standard, the final model coefficients are determined and embedded into the embedded program of the environmental sensing unit.

[0035] ② Real-time correction stage: This can be divided into two parts: online operation and dynamic compensation. Its workflow includes:

[0036] During system operation, temperature, vibration, and traffic load data are collected in real time, and the corresponding compensation amount is output. At the same time, the compensation amount and the original environmental parameters are transmitted to the edge computing unit in real time, and the edge computing unit performs specific elevation data correction.

[0037] (3) Edge computing unit, used to extract the actual deformation of the road surface and eliminate environmental interference based on the calculated DEM time series data to obtain time series difference data; then combine the time series difference data with the environmental compensation amount to make a significance determination.

[0038] Furthermore, the edge computing unit includes:

[0039] ① The elevation modeling module addresses the issues of insufficient accuracy and lack of absolute coordinates in the original depth information. Its operation includes:

[0040] Step 1: Perform algorithm-level optimization on the initial disparity map output by the binocular camera unit, and calculate the sub-pixel correction amount using a quadratic interpolation method;

[0041] Step 2: Combining the pre-calibrated binocular hardware parameters (including: absolute elevation of the camera optical center, baseline length, and focal length), the optimized parallax is converted into absolute elevation based on the principle of triangulation.

[0042] Step 3: Perform weighted fusion on the initial elevation data of N consecutive frames (N≥5, frame interval 10 minutes), and simultaneously access the temperature and vibration data of the environmental sensing unit to correct for thermal expansion and micro-vibration interference.

[0043] ② The time series analysis module is used to extract the true road surface deformation and eliminate environmental interference from the DEM time series data output by the elevation modeling module through "spatial registration - differential calculation - anomaly filtering". Its working method is as follows:

[0044] Step 1: Using the fixed reference point on the road surface as the anchor point, calculate the affine transformation matrix between the current DEM and the initial DEM to ensure that the same pixel of the DEM corresponds to the same physical position on the road surface at different times, thus eliminating spatial misalignment caused by the micro-displacement of the support.

[0045] Step 2: Combine the temperature change, vibration and traffic load of the environmental sensing unit, and calculate the compensation amount through a multivariate regression model;

[0046] Step 3: Accumulate the instantaneous deformation over T consecutive time intervals;

[0047] Step 4: Based on the 3σ statistical threshold and morphological filtering (opening operation for noise reduction + closing operation for hole filling), filter regions with a connected region area ≥ 50 mm² and an aspect ratio ≥ 4 to determine suspected areas of cracks or collapses.

[0048] ③ The saliency determination module receives the time-series difference data output by the time-series analysis module and combines it with the environmental compensation amount generated by the environmental sensing unit. Through a combined algorithm of "statistical threshold + morphological optimization," it reduces misjudgments caused by environmental interference. Its working principle is as follows:

[0049] Step 1, directional input data, the data of which all come from the edge computing unit and the processed data of the collaborative module, including:

[0050] The time series analysis module outputs corrected instantaneous elevation difference and cumulative elevation difference;

[0051] The environmental sensing unit has pre-processed compensation parameters;

[0052] The DEM pixel-physical coordinate mapping relationship output by the elevation modeling module (e.g., 1 pixel corresponds to an actual area of ​​1mm×1mm) provides a basis for subsequent morphological feature calculations.

[0053] Step 2: To accurately define the "normal fluctuation range", a static benchmark area with no road surface defects is selected. A square area of ​​200×200 pixels (corresponding to an actual area of ​​200mm×200mm) is defined with a fixed reference point laid out on the road surface as the center. This area has been manually verified in the early stage to be free of cracks or collapses and is uniformly affected by environmental interference. The cumulative elevation difference standard deviation of this benchmark area in the historical N time series periods is calculated.

[0054] Step 3: Construct a dynamic threshold model by combining environmental compensation parameters to ensure consistent judgment accuracy under different environments:

[0055] Step 4: For suspected abnormal pixel regions (binary image, suspected points are 1, non-suspected points are 0) where the "cumulative elevation difference absolute value is ≥ dynamic threshold", perform a combined filtering of "opening operation + closing operation" to eliminate isolated noise points and micro-holes.

[0056] Step 5: Since both road surface cracks and collapses have clear morphological characteristics (cracks extend linearly, and collapses are irregularly blocky), they are screened by verifying the area of ​​connected components and the aspect ratio. The area of ​​the connected components verified is not less than a preset threshold, and the aspect ratio verification must match the morphological characteristics of the cracks or collapses.

[0057] (4) Cloud analysis module, used to integrate the heterogeneous monitoring data of the edge computing unit and the environmental sensing unit, construct a multi-source time series database, predict the road surface deformation trend based on machine learning time series model and trigger graded early warning.

[0058] Furthermore, the cloud-based analysis module incorporates a data fusion submodule and a trend prediction submodule to enable intelligent early warning.

[0059] ① Data Fusion Submodule: This submodule integrates heterogeneous monitoring data from edge computing units and environmental sensing units, transforming it into a unified, spatiotemporally consistent, and non-redundant multi-source time-series database to provide high-quality input data for the trend prediction submodule. Its specific operation is as follows:

[0060] Step 1: Connect the output of the edge computing unit and the output of the environmental sensing unit, and convert all data into a structured format of "timestamp + feature dimension + value".

[0061] Step 2, spatiotemporal alignment, as follows:

[0062] Step 2.1: Using the image acquisition timestamps of the binocular camera unit as the reference timestamp sequence, downsampling is performed on data with frequencies higher than the reference frequency (taking the peak value within a 1-second window corresponding to each reference timestamp), and linear interpolation is used to complete the data with frequencies lower than the reference frequency, ensuring that the timestamps of all data completely match the reference timestamp sequence.

[0063] Step 2.2: Based on the coordinate system of fixed calibration reference points on the road surface, map the point monitoring data of the environmental sensing unit to the area grid coordinates of the DEM.

[0064] Step 3, data cleaning, as follows:

[0065] Step 3.1: For missing values ​​caused by data transmission interruptions, use linear interpolation to fill them in to ensure temporal continuity;

[0066] Step 3.2: Based on the 3σ criterion, calculate the mean and standard deviation of each feature dimension, remove extreme noise values, and replace them with the linear interpolation results of adjacent valid data;

[0067] Step 3.3: Delete redundant data with duplicate timestamps and duplicate feature dimensions.

[0068] Step 4: Bind the spatiotemporally aligned elevation deformation features and environmental impact features into a unified feature vector; store the fused feature vector in the time series database according to the "time dimension + spatial grid dimension".

[0069] ② Trend Prediction Submodule: Built on a machine learning time series model, this module predicts the deformation increment and rate of change within a specific future period, compares them with a preset safety threshold, and automatically triggers tiered early warnings. The specific steps are as follows:

[0070] Step 1: Extract multi-source time-series data of the target monitoring area from the unified time-series database, and perform two preprocessing steps on the data:

[0071] Map all features to the [0,1] interval to eliminate the interference of magnitude differences on model training;

[0072] For occasional missing values ​​during database retrieval, a forward imputation method (taking the same feature value from the previous time step) is used to fill in the missing values, ensuring the integrity of time series data.

[0073] Step 2: Extract the instantaneous rate of change, the cumulative mean deformation, and the load correlation coefficient from the preprocessed data. Concatenate the three types of features with the original normalized data to form the model input feature vector.

[0074] Step 3: Use LSTM (Long Short-Term Memory) network as the core prediction model:

[0075] Using the input feature vectors from the past 30 days as the training set, and the "cumulative deformation in the next 24 hours" of the corresponding time period as the label, an LSTM network (containing 1 input layer, 2 hidden layers (128 neurons each), and 1 output layer) is constructed. The mean squared error (MSE) loss function is minimized through the Adam optimizer and trained until the loss on the validation set converges (e.g., MSE < 0.001 on the validation set).

[0076] Input the feature vectors from the last 7 days, and the model will output the cumulative deformation prediction and instantaneous rate of change prediction for the next 72 hours.

[0077] Step 4: Based on the cumulative deformation prediction value for the next 72 hours, and combined with the urban road maintenance standards, set a three-level safety threshold and automatically trigger the corresponding warning.

[0078] Specifically, the cloud-based analytics module connects to a road inspection robot or drone monitoring platform via an Internet of Things (IoT) interface to enable remote and continuous monitoring of urban road structures.

[0079] like Figure 2 As shown in the example, the test case of this application installs the system proposed in the above embodiments on a monitoring bracket structure above the road, wherein... Figure 2 The meanings of each number are as follows: fixed bracket 1, vibration-resistant base 2, binocular camera unit (including left field-of-view camera lens 3-1 and right field-of-view camera lens 3-2), temperature sensor 4, vibration sensor 5, traffic load detection module 6, and fixed calibration reference points (including first point 7-1, second point 7-2, and third point 7-3). The binocular camera unit is mounted on the fixed bracket 1, the edge computing unit is mounted inside or near the vibration-resistant base 2, the environmental sensing unit is mounted on the bracket column and includes temperature sensor 4 and vibration sensor 5, the traffic load detection module 6 is mounted below the road surface, and the reference points are laid out on the road surface for periodic self-calibration.

[0080] The above structural arrangement enables the binocular camera unit to stably cover the monitoring area and obtain continuous time-series data. The vibration-resistant base 2 reduces the disturbance of vehicle load and wind vibration on the camera's external parameters, and the reference point enables the correction of external parameter drift during long-term operation.

[0081] like Figure 3 As shown, during system operation, the binocular camera unit performs the following operations:

[0082] Images of the left and right fields of view of the monitoring area are collected synchronously at preset time intervals, and the images and collection timestamps are transmitted to the edge computing unit.

[0083] To improve matching stability, preprocessing was performed on the original left and right images: ① Radial and tangential distortion correction was performed based on calibration parameters; ② Gray-level normalization was performed, mapping the gray levels to [0, 255] to reduce uneven illumination; ③ A 3×3 Gaussian filter was applied ( =1.0) Smooths high-frequency noise and preserves crack edge texture.

[0084] After preprocessing, a pixel-level SAD matching algorithm is used to calculate the grayscale difference within an 11×11 search window of the left and right images to obtain an initial disparity map (disparity accuracy ±1 pixel). This initial disparity map is used as the "original surface depth information" input into the edge calculation unit for elevation modeling.

[0085] like Figure 4 As shown, after receiving the initial disparity map, the edge computing unit first performs sub-pixel level disparity optimization. The sub-pixel correction amount is calculated using a quadratic interpolation method. This is used to improve parallax accuracy from ±1 pixel to ±0.1 pixel. The formula is:

[0086]

[0087] in, The SAD cost sequence is located near the initial disparity.

[0088] After obtaining the sub-pixel correction, the final disparity is calculated using the following formula:

[0089]

[0090] in, , These are the column coordinates of the matching points in the left and right images.

[0091] Next, combining the pre-calibrated binocular hardware parameters: absolute elevation of the camera optical center. Baseline length and focal length Based on the principle of triangulation, the optimized parallax is converted into the absolute elevation of the target point on the road surface: in, Obtained through total station calibration, it is used to assign "absolute coordinate attributes" to depth information.

[0092] To reduce random errors and enhance stability, a weighted fusion is performed on the DEMs of N consecutive frames (N≥5, frame interval 10 minutes), using the following formula:

[0093]

[0094] Among them, weight , Let be the standard deviation of the elevation in the k-th frame.

[0095] To further mitigate the effects of thermal expansion and contraction and micro-vibrations, the system incorporates environmental sensor data and calculates pre-compensation amounts. ,in, Let t be the temperature. For reference temperature, The characteristic value is the vibration acceleration; this pre-compensation is used to reduce the disturbance of elevation by thermal expansion and vibration residuals at the DEM output level.

[0096] To ensure that the compensation amount has a traceable quantitative basis, the test cases in this application use multivariate regression to construct a mapping model of "environmental parameters - spurious elevation changes", and distinguish between the training phase and the real-time correction phase.

[0097] During the training phase, standard road sections without defects were selected. The criteria for determining standard road sections without defects include: (1) visual integrity: no visible cracks, potholes, or looseness on the surface; (2) structural stability: after at least 48 hours of comparative observation using a total station or level, the elevation fluctuation of the road section under no external load interference is confirmed to be less than 0.1 mm; (3) smoothness requirements: the standard deviation of the initial digital elevation model in the area. .

[0098] Environmental parameters were collected continuously for 30 days during this phase. , , ) and corresponding elevation deviation The samples were divided into training and validation sets in an 8:2 ratio, and the coefficients were fitted using the least squares objective function. and with goodness of fit ( () serves as a condition for model solidification.

[0099] During online operation: The environmental sensing unit outputs compensation values ​​in real time. ,in, The elevation change is spurious due to environmental noise, and a, b, and c are the regression coefficients for temperature, vibration, and load, respectively. Here, is the load parameter, and d is a constant term used to correct the system baseline error.

[0100] To obtain the "true elevation," the edge computing unit performs online correction on the original elevation: ,in, The original elevation is uncompensated. The true elevation after eliminating environmental interference.

[0101] To eliminate spatial misalignment caused by micro-displacement of the support, the system uses reference points 7-1, 7-2, and 7-3 as anchor points to calculate the affine transformation matrix between the current DEM and the initial DEM, thereby achieving temporal spatial registration of the DEM and ensuring that the same pixel corresponds to the same physical location on the road surface.

[0102] After spatial registration is completed, the corrected instantaneous deformation is calculated. By subtracting spurious environmental factors from elevation changes, only the true deformation caused by cracks or collapses is retained:

[0103]

[0104]

[0105] The cumulative deformation is obtained by summing the instantaneous deformation over T consecutive time periods. Among them, T can be 72, corresponding to a monitoring period of 3 days, which is used to reflect the evolution trend and cumulative effect of the disease.

[0106] To accurately define the "normal fluctuation range," a 200×200 pixel reference area was delineated centered on reference point 7, and its cumulative standard deviation over N historical time periods was calculated:

[0107]

[0108] Where M is the total number of pixels in the reference region, and R is the set of pixels in the reference region. It reflects the normal fluctuation level of road surfaces without defects under environmental influences, providing an objective basis for threshold setting.

[0109] Based on 3 Criteria set basic threshold Furthermore, temperature and vibration are introduced as dual parameters for dynamic correction to obtain a dynamic judgment threshold. .in, This is the temperature correction factor. This is the vibration correction factor. The peak value of the vibration is measured; this threshold is used to maintain consistency in judgment under different environments and reduce false positives and false negatives.

[0110] Will satisfy The pixels marked as suspected abnormal regions (binary image A) are subjected to a combined "opening + closing" filter to eliminate isolated noise and fill holes:

[0111] Opening operation:

[0112] Closing operation:

[0113] Where B is a rectangular structuring element; the opening operation is used to remove isolated noise blocks and retain the main body of the crack, and the closing operation is used to fill the tiny holes in the crack area and restore the edge accuracy.

[0114] Perform morphological feature verification, including connected component area verification and aspect ratio verification:

[0115] (1) Area of ​​connected domain

[0116] Where C is the set of connected pixels. The actual area corresponding to each pixel (determined by the DEM pixel-physical coordinate mapping relationship); only those satisfying the condition are retained. Connected components.

[0117] (2) Aspect ratio L / W:

[0118] For the linear characteristic setting of the crack Setting the blocky features of the collapse This is to exclude false areas that "meet the area standard but do not conform to the disease pattern".

[0119] like Figure 5 As shown, the cloud-based analysis module receives significant deformation data and synchronized environmental time-series data transmitted from the edge computing unit, and utilizes the high computing power resources of the cloud to perform long-term trend analysis across days and cycles. Specifically:

[0120] Binocular acquisition of timestamp sequences Using the reference frequency as a baseline, interpolation is performed on the environmental data to achieve time alignment; when the frequency of the environmental data is lower than the reference frequency, linear interpolation is used.

[0121]

[0122] in, X represents the temperature, vibration, and load parameters.

[0123] To eliminate differences in feature magnitude, min-max normalization is performed on the input features: And construct a fused feature vector. .in, The weights can be determined based on the Pearson correlation coefficient, so that more critical influencing factors can be given higher weights.

[0124] In the trend prediction phase, the instantaneous rate of change is extracted from time-series data. Cumulative deformation mean Load correlation coefficient :

[0125]

[0126]

[0127]

[0128] It is then concatenated with the original normalized data to form the model input vector.

[0129] Using LSTM as the prediction model, its gating and state update formulas are as follows:

[0130]

[0131]

[0132]

[0133]

[0134]

[0135] in, For the input feature vector, For output of the hidden layer, In cellular state, This is an element-wise product.

[0136] Tiered warning trigger: Based on the predicted maximum cumulative deformation over the next 72 hours. Set a three-level threshold, with the following triggering rules:

[0137]

[0138] Threshold example: =5mm =10mm =20mm, the warning information is pushed to the traffic management platform through the IoT interface, and can be linked with inspection robots or drones for verification, forming a closed loop of "prediction-warning-verification".

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A three-dimensional detection and evolution monitoring system for urban road surface cracks or collapses based on binocular vision, characterized in that, include: The binocular camera unit is used to acquire dual-field-of-view images of the road monitoring area, initially generate a surface DEM with a regular grid as the carrier, and output the initial disparity map to the edge calculation unit; An environmental sensing unit is used to collect environmental parameters in real time, build an environmental noise compensation model, generate environmental compensation quantities, and transmit them to the edge computing unit. Edge computing units are used to extract the true road surface deformation and eliminate environmental interference based on the calculated DEM time-series data to obtain time-series differential data; The time-series difference data is then combined with the environmental compensation amount to determine significance. The cloud-based analytics module is used to integrate the heterogeneous monitoring data from the edge computing unit and the environmental sensing unit, construct a multi-source time-series database, predict road surface deformation trends based on machine learning time-series models, and trigger graded early warnings.

2. The system according to claim 1, characterized in that, The binocular camera unit operates as follows: A fixed baseline binocular structure was constructed, and a matching calibration module was used to periodically calibrate the camera's intrinsic and extrinsic parameters through fixed reference points on the road surface in order to eliminate attitude drift errors during long-term operation. Hardware-based optimizations are performed on the acquired dual-field-of-view raw images, including distortion correction, grayscale normalization, and noise smoothing. A pixel-level SAD matching algorithm is used to calculate grayscale differences based on optimized dual-field images, output an initial disparity map, and directly transmit it to the edge computing unit.

3. The system according to claim 1 or 2, characterized in that, The environmental parameters include: road surface temperature, ambient temperature around the equipment, vibration acceleration of the equipment, and average traffic load per unit time.

4. The system according to claim 3, characterized in that, The environmental noise compensation model includes: The training phase is completed before system deployment and is used to conduct comparative experiments on pavement without defects of different materials. The model coefficients are obtained based on the multivariate regression model and solidified into the embedded program of the environmental sensing unit. During the real-time correction phase, the system inputs and collects data in real time during operation and outputs the compensation amount at the corresponding moment; at the same time, the compensation amount and the original environmental parameters are transmitted to the edge computing unit in real time.

5. The system according to claim 4, characterized in that, The edge computing unit includes: The elevation modeling module is used to optimize the initial disparity map and calculate the DEM time series data; The time series analysis module is used to extract the actual road surface deformation and eliminate environmental interference from the DEM time series data, and output time series difference data. The saliency determination module is used to receive the time-series difference data, combine it with the environmental compensation amount, and combine statistical thresholds with morphological optimization to reduce misjudgments caused by environmental interference.

6. The system according to claim 5, characterized in that, The salience determination module operates as follows: The directional input data includes the instantaneous elevation difference and cumulative elevation difference corrected by the time series analysis module, the compensation parameters preprocessed by the environmental sensing unit, and the DEM pixel-physical coordinate mapping relationship output by the elevation modeling module; Select a static reference area with no pavement defects and calculate its cumulative elevation difference standard deviation over N historical time periods; A dynamic threshold model is constructed by combining environmental compensation parameters; For suspected abnormal pixel regions where the absolute value of the cumulative elevation difference is greater than or equal to the dynamic threshold, a combined filtering operation of opening and closing is performed. The actual diseased areas were screened by verifying the area of ​​the connected domain and the aspect ratio.

7. The system according to claim 1, characterized in that, The cloud analytics module has the following built-in features: The data fusion submodule is used to integrate the heterogeneous monitoring data from the edge computing unit and the environmental sensing unit to construct a multi-source time-series database. The trend prediction submodule is used to predict road surface deformation trends based on machine learning time series models and trigger graded early warnings.

8. The system according to claim 7, characterized in that, The data fusion submodule operates as follows: The corrected DEM data output by the edge computing unit, information on significantly deformed areas, and environmental parameters output by the environmental sensing unit are all converted into a structured format of timestamp + feature dimension + numerical value. Using the image acquisition timestamps of the binocular camera unit as the reference sequence, downsampling is performed on data with frequencies higher than the reference frequency, and data with frequencies lower than the reference frequency is completed; based on the coordinate system of the fixed calibration reference points on the road surface, the monitoring data of the environmental sensing unit is mapped to the grid coordinates of the DEM; The data is cleaned, missing values ​​are filled, extreme noise values ​​are removed and replaced based on the 3σ criterion, and redundant data is deleted. The spatiotemporally aligned elevation deformation features and environmental impact features are bound into a unified feature vector; the fused feature vector is then stored in a time-series database in the form of a combination of time dimension and spatial grid dimension.

9. The system according to claim 7 or 8, characterized in that, The trend prediction submodule performs the following steps: Multi-source time-series data of the target monitoring area are extracted, all features are normalized, and the missing values ​​are filled in using the forward imputation method. Extract core features, including instantaneous rate of change, cumulative average deformation, and load correlation coefficient, and concatenate the core features with the original normalized data to form the model input feature vector; An LSTM network is constructed as the core prediction model. It is trained by minimizing the mean square error loss function through the Adam optimizer to obtain the cumulative deformation prediction value and the instantaneous change rate prediction value for the next n hours. Based on the obtained cumulative deformation prediction values, and combined with urban road maintenance standards, safety thresholds are set, and corresponding warning levels are automatically triggered.

10. The system according to claim 1, characterized in that, The cloud-based analysis module connects to a road inspection robot or drone monitoring platform via an IoT interface to achieve remote and continuous monitoring of urban road structures.

Images