Landslide deformation identification and early warning method based on video monitoring

Abstract

The invention provides a landslide deformation identification and early warning method based on video monitoring. The method comprises the following steps of S1, data acquisition, S2, centroid extraction, S3, deformation judgment, construction of a sliding window range envelope line based on a feature point centroid coordinate sequence, and statistics of coordinate extreme values and standard deviations in a window to expand a confidence interval boundary; and when continuous multi-frame centroid coordinates exceed the boundary, a deformation early warning signal is triggered, and multi-modal feature verification is carried out in combination with the change rate of the length-width ratio of the external rectangle and the contour Fourier descriptor difference. And high-precision real-time monitoring and reliability verification of slope deformation are realized. According to the scheme, aiming at the problems of high cost, poor real-time performance, weak anti-interference capability and the like of a traditional monitoring technology, a set of complete software and hardware collaborative solution is provided, and the accuracy and adaptability of landslide early warning are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of long-term monitoring of large road slopes, and in particular to a landslide deformation identification and early warning method based on video monitoring. Background Art

[0002] Landslides are a frequent occurrence, and traditional monitoring methods (such as GPS and inclinometers) have significant drawbacks: high hardware deployment costs, infrequent data updates, and difficulty capturing sudden landslide deformations. Furthermore, equipment is susceptible to damage from inclement weather, further increasing maintenance costs. While existing video monitoring technologies can capture images in real time, they rely on inefficient manual interpretation, and their algorithms are not adaptable to complex environments. Strong light or shadows blur feature points, rain and fog introduce significant noise, and swaying vegetation can easily lead to false detections. Furthermore, existing indoor test models can only simulate single-directional sliding, making it difficult to replicate the multi-directional, complex deformations found in actual landslides. Furthermore, they lack simulation of equipment performance fluctuations, leading to significant discrepancies between algorithm validation results and actual scenarios. Therefore, a high-precision monitoring solution integrating automated image processing, dynamic threshold adjustment, and multimodal feature verification is urgently needed. This solution, coupled with an indoor test model capable of realistically simulating multi-directional sliding, sudden changes in illumination, and noise interference, can address the low sensitivity, high false alarm rate, and inadequate validation of existing technologies. Summary of the Invention

[0003] To solve the above problems, the present invention proposes a landslide deformation identification and early warning method based on video monitoring, which includes the following steps:

[0004] S1: Data acquisition: A high-definition camera uses the RTSP protocol to transmit a real-time video stream. Data preprocessing is then performed. The video stream is parsed using a Matlab script, and key frames are captured at 2-second intervals and converted into JPG image sequences. Histogram equalization is performed on the images to enhance the contrast between feature points and the background by adjusting the grayscale distribution.

[0005] S2: Centroid extraction, using a fixed threshold method to perform binary segmentation on the enhanced image, combined with connected area threshold analysis to filter out isolated noise points, and separate the adhesion areas through morphological corrosion and expansion operations to retain stable feature points;

[0006] S3: Deformation judgment: construct a sliding window range envelope based on the centroid coordinate sequence of feature points, and count the extreme values ​​and standard deviations of the coordinates within the window to expand the confidence interval boundaries. When the centroid coordinates of multiple consecutive frames exceed the boundaries, a deformation warning signal is triggered. At the same time, multimodal feature verification is performed by combining the change rate of the aspect ratio of the circumscribed rectangle and the difference in the contour Fourier descriptor.

[0007] Preferably, in S2, the fixed threshold method in image binarization is used to enhance the image data after histogram equalization. This method classifies the pixels in the image into two categories, foreground and background, by setting a threshold.

[0008] If the pixel value I(x,y) ≥ T, then it is set to 1, which is the foreground.

[0009] If the pixel value I(x,y) < T, then it is set to 0, which is the background.

[0010] Preferably, the image after binarization includes feature points, background, and noise. A second threshold is set for control to remove noise. Based on the adjacency relationship of pixels in the binary image, connected regions are marked, and the number of pixels in each region is counted as the area. A minimum area threshold is set to filter out isolated noise with an area smaller than the threshold. Combining morphological erosion and dilation operations separates the adhered regions and smooths the boundaries, retaining the effective feature points.

[0011] Preferably, the connected region means that the gray values of two pixels are the same and adjacent.

[0012] Preferably, in S3, the sliding window range envelope algorithm statistically calculates the extreme values and standard deviations of the centroid coordinates of feature points through a dynamic reference window, and expands the boundary range of the confidence interval. When the centroid coordinates exceed the boundary for multiple consecutive frames, a deformation alarm is triggered.

[0013] Preferably, for the sliding window range envelope method, a dynamic reference window Wk={ck-N,...,ck-1} with a length of N = 15 is defined, and the extreme values of the coordinates within the window are calculated and the confidence interval is expanded:

[0014]

[0015] where is the standard deviation of the coordinates within the window, and the coefficient 1.96 corresponds to the 95% confidence interval of the normal distribution. The out-of-bounds criterion for the current centroid ck is defined as: , when it satisfies (i = 0,...,4) for Th = 5 consecutive frames, a deformation alarm is triggered.

[0016] Preferably, the aspect ratio change rate of the rectangle is .

[0017] Preferably, the difference of the contour Fourier descriptor is .

[0018] The proposed method for landslide deformation identification and early warning based on video monitoring has the following beneficial effects: By combining automated image processing algorithms with high-fidelity indoor scaled physical model testing, it enables high-precision, real-time monitoring of slope deformation and reliability verification. This approach addresses the high cost, poor real-time performance, and weak anti-interference capabilities of traditional monitoring technologies by providing a comprehensive hardware and software collaborative solution, significantly improving the accuracy and adaptability of landslide early warning systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments.

[0020] Figure 1 Schematic diagram of the landslide deformation identification and early warning method based on video monitoring of the present invention;

[0021] Figure 2 A schematic diagram of a specific interface for data acquisition according to the present invention;

[0022] Figure 3 This is a schematic diagram of a specific implementation of the centroid extraction of the present invention;

[0023] Figure 4 Schematic diagram of deformation judgment;

[0024] Figure 5 Provide a schematic diagram for field layout;

[0025] Figure 6 Build a schematic diagram for the model;

[0026] Figure 7 Schematic diagram before and after noise removal;

[0027] Figure 8 Obtain schematic diagrams for image recognition of indoor test models;

[0028] Among them, 1. Movable bracket; 2. Steel tilting table; 3. Anti-slip piles; 4. Feature area; 5. Obstruction; 6. Highway; 7. Monitoring equipment. DETAILED DESCRIPTION

[0029] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0030] Example

[0031] The landslide deformation identification and early warning method based on video monitoring disclosed in this embodiment specifically includes the following steps: As shown in the figure,

[0032] S1: Data acquisition, including video capture and data preprocessing, uses a high-definition camera to transmit real-time video streams via the RTSP protocol. A Hikvision HST-IPD24D1-MBT / GLSE / N high-definition camera is secured with a corrosion-resistant bracket. 3D modeling optimizes mounting locations to ensure coverage of key landslide deformation zones, such as the main sliding direction, trailing edge cracks, and leading edge shear outlets. Before camera installation, slope deformation trends are analyzed using 3D point cloud data to optimize viewing angles and coverage to avoid blind spots. The device supports dual-mode 4G / fiber transmission and transmits real-time video streams via the RTSP protocol. It is equipped with a solar power system and a backup lithium battery pack to ensure continuous operation for over 72 hours in harsh field environments. In the event of a network interruption, the system automatically resumes downloads, prioritizing key frames and retransmitting the complete video stream upon network recovery to ensure data integrity. The video stream is encrypted by AES and compressed in H.265 format for transmission. After receiving it, the local server automatically generates a timestamp index file, recording the start time, resolution (1920×1080), frame rate (25 frames / second) and storage path of each video, facilitating rapid retrieval and analysis.

[0033] Data preprocessing and video stream parsing are implemented using a Matlab script. Keyframes are captured at 2-second intervals and converted into JPG image sequences, generating approximately 1,800 frames per hour (data volume 3.6GB). Preprocessing latency is strictly controlled within 20 seconds, meeting real-time requirements. To address image blur, water vapor interference, and uneven lighting caused by long-distance shooting, the system uses histogram equalization to enhance contrast. The cumulative distribution function is calculated using the normalized grayscale histogram, redistributing pixel grayscale levels and stretching the grayscale difference between feature points and background. This processing expands the image's dynamic range by over 30%, significantly improving feature point recognition. The enhanced image is then binarized and segmented, with a fixed threshold set to distinguish foreground from background. Connected region area thresholding is then used to filter out isolated noise points (such as gravel reflections or bird occlusions) with an area smaller than the threshold. The detailed process includes connected component labeling, area counting and screening, and morphological erosion and dilation operations. The remaining stable feature points are recorded using their centroid coordinates to form a time series dataset. The morphological operation uses a 3×3 circular kernel to separate the adhesion area and smooth the boundary contour to ensure the integrity and consistency of feature point extraction.

[0034] The present invention performs a simple and effective contrast enhancement on the acquired image data, namely, Histogram Equalization (HE). This method is a contrast enhancement technique based on the image's grayscale histogram. By redistributing the image's grayscale levels, the output image's grayscale distribution becomes more uniform, thereby expanding the image's dynamic range and improving visual quality. The core idea of ​​this method is to use the image's cumulative distribution function (CDF) to perform a nonlinear mapping of grayscale levels. This stretches the more concentrated grayscale levels in the original image, while compressing the more sparsely distributed grayscale levels, ultimately achieving a uniform grayscale distribution.

[0035] For a grayscale image, its grayscale range is [0, L−1], where L is the total number of grayscale levels (usually 256). Indicates grayscale The number of pixels, its normalized form is:

[0036]

[0037] in Grayscale N is the total number of pixels in the image.

[0038] The cumulative distribution function (CDF) is defined as:

[0039]

[0040] CDF reflects the gray level The ratio of the number of pixels with gray levels below 0 to the total number of pixels.

[0041] Map the grayscale through CDF to get the new grayscale , in order to map the result to discrete gray levels, it is usually rounded:

[0042] After converting the data enhancement results into spatial height and color, it can be seen that there is an obvious height difference between the feature points and the background. In order to extract the feature points for subsequent processing, the invention uses image binarization to extract the feature points.

[0043] That is, as described in S2: centroid collection, i.e., feature point extraction, binary segmentation of the enhanced image using a fixed threshold method, filtering out isolated noise points combined with connected region area threshold analysis, and separating adhesion areas through morphological erosion and dilation operations to retain stable feature points;

[0044] The present invention uses the fixed threshold method in image binarization to enhance the image data after histogram equalization. This method divides the pixels in the image into two categories, foreground and background, by setting a threshold, as follows.

[0045] If the pixel value I(x,y) ≥ T, then it is set to 1 (foreground).

[0046] If the pixel value I(x,y) < T, then it is set to 0 (background).

[0047]

[0048] However, although the features and background are clearly distinguishable in the binarized image, there are still many noise points, such as Figure 4 , which is caused by the reflection of a large number of gravels with different particle sizes on the slope surface under natural light. In order to further extract more stable and refined feature points, a second threshold is set here for control to remove the redundant noise points.

[0049] In binary image processing, noise usually appears as isolated pixel points or small connected areas, such as the white small connected areas scattered in the above figure. The noise removal method based on connectivity analysis calculates the area of the connected region and sets a threshold to distinguish noise from the effective target, thereby achieving noise filtering. The core of this method lies in combining connected region labeling, area statistics, and threshold screening techniques, which can effectively separate the noise and the target region and improve the accuracy of image processing.

[0050] First, connectivity refers to the adjacency relationship between pixels in an image. In a binary image, if the gray values of two pixels are the same and adjacent (4-neighborhood or 8-neighborhood), they are regarded as belonging to the same connected region. Noise usually appears as small isolated regions, while effective targets (such as anti-slide piles, landslide feature points, etc.) have larger connected areas. By setting an area threshold T, the connected regions with an area smaller than T can be determined as noise and deleted. Specifically, for a binary image I(x,y) ∈ {0,1}, the area Ai of the connected region Ri is defined as the total number of pixels in that region. If Ai < T, then Ri is marked as a noise region and set to the background value (0).

[0051] In practical applications, binarized images often require morphological operations to eliminate minor noise and enhance the integrity of connected regions. This method uses structuring elements (such as circular or rectangular kernels) to perform erosion and dilation operations to separate connected regions and smooth edges. Subsequently, a connected component labeling algorithm (regionprops) is used to traverse the image, marking all connected regions and calculating their properties (such as area, centroid, and bounding box). These properties provide a basis for subsequent noise filtering and object extraction.

[0052] Taking the on-site video frame image data processing as an example, 280 feature points with connected areas were detected in the image. However, according to the actual distribution of anti-slide piles and the spatial distribution pattern of landslide feature points, such a large number of feature points do not exist on site. Obviously, a large number of areas with low connectivity are noise points. Therefore, the present invention sets the area threshold T=50 and only retains areas with connectivity greater than 50 as stable feature points, thereby achieving noise reduction processing for feature point extraction. The experimental results show that the noise reduction rate is as high as 98.13%, and it can even be further increased to 100% after manual selection to remove noise points, indicating that this method can effectively remove noise and significantly improve the accuracy and reliability of feature point extraction.

[0053] Through the above method, the present invention not only achieves efficient noise filtering in binary images, but also provides a high-quality data foundation for subsequent feature point analysis. The algorithm performs well in feature extraction tasks in complex backgrounds and has high robustness and generalization capabilities.

[0054] After efficiently filtering out the noise in the binary image, deformation calculation is performed in this case. The sliding window range envelope is constructed based on the centroid coordinate sequence of the feature points. The extreme values ​​and standard deviations of the coordinates within the window are counted to expand the confidence interval boundaries. When the centroid coordinates of multiple consecutive frames exceed the boundaries, a deformation warning signal is triggered. At the same time, multimodal feature verification is performed by combining the change rate of the aspect ratio of the circumscribed rectangle and the difference in the contour Fourier descriptor. Specifically:

[0055] By building a spatiotemporal statistical model, real-time monitoring and early warning of structural deformation are achieved. The algorithm first obtains the centroid coordinates of the target area through image moment calculation, defining ct = (xt, yt) as the centroid position of the t-th frame, where the coordinates are accurately calculated using the zero-order moment m00 and the first-order moments m10 and m01:

[0056]

[0057] This method improves upon the moment invariance theory in digital image processing (Gonzalez & Woods, 2018), enhancing segmentation robustness through morphological preprocessing. The centroid sequence C = {c1, c2, ..., cT} forms a temporal evolution process, and its statistical properties reflect changes in the target's motion pattern. The core detection mechanism uses a sliding window range envelope method, defining a dynamic reference window Wk = {ck-N, ..., ck-1} of length N = 15. The coordinate extremes within the window are calculated and the confidence intervals are expanded:

[0058]

[0059] in is the standard deviation of the coordinates within the window, and the coefficient 1.96 corresponds to the 95% confidence interval of the normal distribution. The out-of-bounds criterion for the current centroid ck is defined as:

[0060]

[0061] When Th=5 consecutive frames satisfy (i=0,...,4) triggers a deformation alarm. This mechanism combines spatial anomaly detection with temporal persistence verification, conforming to the principle of sequential probability ratio testing in fault detection. Preliminary experiments have shown that structural deformation can cause abrupt changes in the standard deviation of the center of mass trajectory. This allows for enhanced detection sensitivity using an improved CUSUM control chart:

[0062]

[0063] in is the baseline displacement, and δ is the minimum detectable change. To further improve environmental adaptability, the algorithm introduces an adaptive boundary adjustment factor:

[0064]

[0065] Include noise estimation and the instantaneous velocity term , α and β are adjustment parameters, and PFA is the preset false alarm probability. This design draws on the constant false alarm rate processing technology in radar detection (Richards, 2014). Experimental verification shows that the change rate of the aspect ratio of the joint bounding rectangle is Differences from the contour Fourier descriptor Multimodal features such as CNN can increase the detection probability PD to 96.8% and reduce the false alarm rate PFA to below 1.2% (N=15, Th=5).

[0066] The deformation calculation module constructs a sliding window range envelope (window length 10 frames) based on dynamic centroid trajectory analysis. The extreme values ​​and standard deviations of the coordinates within the window are counted, and the 95% confidence interval boundaries (extreme value ±1.96 times the standard deviation) are expanded. If the centroid coordinates exceed the bounds for five consecutive frames, the system triggers a deformation warning. Multimodal validation is performed using the ratio of change in the bounding rectangle's aspect ratio and the difference in the contour Fourier descriptor to ensure the reliability of the warning. The ratio of change in the bounding rectangle detects shape distortion by calculating the change in the aspect ratio of the bounding rectangle of the target area in real time. The difference in the contour Fourier descriptor quantifies the deformation magnitude by comparing the Fourier coefficients of the current contour with those of the reference contour. Experiments demonstrate that this mechanism achieves a false alarm rate of less than 1.2% and a response time of 0.12 seconds, improving efficiency by over 90% compared to traditional manual interpretation. It effectively captures the complete evolution of a landslide, from creep to sudden change.

[0067] In this embodiment, a verification model is also established to verify the robustness of the system in complex interference scenarios by conducting multiple sets of tests under dark light (brightness -50%), high noise (100% density) and single feature point conditions.

[0068] This case is used to verify the indoor test model of the landslide deformation identification system.

[0069] An indoor scaled landslide model was constructed to simulate multi-directional sliding and complex environmental interference. Figure 3 As shown, the model consists of a steel tilting table (2m × 1m). The slope is continuously adjustable (0°–90°) via a movable bracket mechanism. The surface is coated with a coating that closely matches the color of the actual soil. One end of the table is fixed to the wall to simulate bedrock constraints, while the free end is supported by a movable bracket to control the unloading effect of the free surface. The anti-slide pile model uses white hardwood cubes (4×4×4cm³) that resemble characteristic areas of the field slope. These cubes are arranged in an orderly manner on the table surface and marked with fine lines to ensure that the positions of the anti-slide piles remain essentially the same throughout each test.

[0070] The multi-directional sliding control system uses a thin wire to pull the top of the anti-slide pile. Force is applied in a range of directions, including vertical sliding, 15° leftward and 15° rightward sliding, combined leftward and rightward sliding, and 45° downward and leftward sliding, covering typical displacement patterns in actual landslides. The sliding initiation device uses a manually triggered, semi-automatic pressing mechanism to simulate multi-directional deformation. The sliding initiation process is divided into two stages: an initial 20-second simulation of sliding body creep (center of mass fluctuation range ±2 pixels), followed by the triggering of high-speed sliding (a physical process of 0.5 seconds). A frame interpolation algorithm is used to expand the high-speed displacement into a 2-second video stream of progressive deformation to match the detection algorithm's requirements. Environmental interference simulation uses image processing software to adjust brightness and grain noise. A removable feature point module is also included (the number is gradually reduced from 8 to 1 to simulate local monitoring failures). The monitoring camera uses a DJI Action4, capturing undistorted video at 4K resolution and 120 frames per second. It is fixed 1.5 meters in front of the model and adjusted to simulate non-ideal operating conditions of field equipment.

[0071] Experimental results show that by dynamically adjusting the feature point extraction threshold (±5% to 10%) and sliding window parameters, the false detection rate can be controlled within 2%, and the missed detection rate is less than 0.5%. The time difference between multi-directional sliding trigger warnings is less than 0.08 seconds, and the direction recognition accuracy exceeds 95%, verifying the algorithm's strong robustness to complex interference scenarios. In addition, the particle noise and light and dark variation data injected into the model test provide a high-fidelity experimental foundation for algorithm optimization, significantly improving the system's adaptability in practical engineering applications.

Claims

1. A landslide deformation identification and early warning method based on video monitoring, characterized in that: It includes the following steps: S1: Data acquisition: The video stream is transmitted in real time via the RTSP protocol by a high-definition camera, and then data preprocessing is carried out. The video stream is parsed based on the Matlab script, key frames are intercepted at an interval of 2 seconds and converted into a JPG image sequence. Histogram equalization enhancement processing is performed on the images to improve the contrast between feature points and the background by adjusting the gray distribution; S2: Centroid extraction: The fixed threshold method is used to perform binary segmentation on the enhanced image. Combining the analysis of the connected region area threshold to filter out isolated noise points, and separating the adhesion regions through morphological erosion and dilation operations to retain stable feature points; S3: Deformation judgment: A sliding window range envelope is constructed based on the centroid coordinate sequence of feature points, and the coordinate extreme values and standard deviations within the window are statistically analyzed to expand the confidence interval boundary; When the centroid coordinates of multiple consecutive frames exceed the boundary, a deformation warning signal is triggered, and at the same time, multi-modal feature verification is carried out by combining the aspect ratio change rate of the circumscribed rectangle and the difference of the contour Fourier descriptor.

2. The landslide deformation identification and early warning method based on video monitoring according to claim 1 is characterized in that: In S2, the fixed threshold method in image binarization is used to enhance the image data after histogram equalization. This method divides the pixels in the image into two categories, foreground and background, by setting a threshold. If the pixel value I(x,y)≥T, then it is set to 1, which is the foreground; if the pixel value I(x,y)<T, then it is set to 0, which is the background.

3. The landslide deformation identification and early warning method based on video monitoring according to claim 2 is characterized in that: The image after binarization includes feature points, background, and noise points. A second threshold is set for control to remove the noise points. The connected regions are marked based on the adjacency relationship of pixels in the binary image, and the number of pixels in each region is statistically analyzed as the area; A minimum area threshold is set to filter out isolated noise points with an area smaller than the threshold; Combining morphological erosion and dilation operations to separate the adhesion regions and smooth the boundaries to retain valid feature points.

4. The landslide deformation identification and early warning method based on video monitoring according to claim 1 or 3 is characterized in that: The connected region means that the gray values of two pixels are the same and adjacent.

5. The landslide deformation identification and early warning method based on video monitoring according to claim 1 is characterized in that: In S3, the sliding window range envelope algorithm statistically analyzes the extreme values and standard deviations of the centroid coordinates of feature points through a dynamic reference window to expand the range of the confidence interval boundary; When the centroid coordinates of multiple consecutive frames exceed the boundary, a deformation alarm is triggered.

6. The landslide deformation identification and early warning method based on video monitoring according to claim 5 is characterized in that: For the sliding window range envelope method, a dynamic reference window Wk={ck-N,...,ck-1} with a length of N = 15 is defined, and the coordinate extreme values within the window are calculated and the confidence interval is expanded:

7. Among them is the standard deviation of the coordinates within the window, and the coefficient of 1.96 corresponds to the 95% confidence interval of the normal distribution; The out-of-bounds criterion of the current centroid ck is defined as: , when Th=5 consecutive frames satisfy (i=0,...,4) triggers the deformation alarm.

8. The landslide deformation identification and early warning method based on video monitoring according to claim 1 is characterized in that: The change rate of the aspect ratio of the rectangle is .

9. The landslide deformation identification and early warning method based on video monitoring according to claim 1 is characterized in that: The difference of the contour Fourier descriptor is .

Images