A landslide deformation identification and early warning method based on video monitoring

Abstract

The application provides a landslide deformation identification and early warning method based on video monitoring, which comprises the following steps: S1: data acquisition, S2: centroid extraction, S3: deformation judgment, a sliding window range envelope is constructed based on a feature point centroid coordinate sequence, and the coordinate extreme value and standard deviation in the window are counted to expand the confidence interval boundary; when the centroid coordinates of continuous multiple frames exceed the boundary, a deformation early warning signal is triggered, and multi-modal feature verification is carried out in combination with the change rate of the length-width ratio of the circumscribed rectangle and the difference of the Fourier descriptor of the contour. The application realizes high-precision real-time monitoring and reliability verification of the slope deformation by combining an automatic image processing algorithm with a high-fidelity indoor scale physical model test. The scheme provides a complete software and hardware cooperative solution scheme for the problems of high cost, poor real-time performance and weak anti-interference ability of the traditional monitoring technology, and significantly improves the accuracy and adaptability of the landslide early warning.

Description

Technical Field

[0001] This invention relates to the field of long-term monitoring of large road slopes, and in particular to a method for landslide deformation identification and early warning based on video monitoring. Background Technology

[0002] Landslides are frequent, and traditional monitoring methods (such as GPS and inclinometers) have significant drawbacks: high hardware deployment costs, low data update frequency, and difficulty in capturing sudden landslide deformations; furthermore, equipment is susceptible to damage from severe weather, further increasing maintenance costs. While existing video monitoring technologies can acquire images in real time, their reliance on manual interpretation is inefficient, and their algorithms lack adaptability to complex environments—strong light or shadows blur feature points, rain and fog introduce significant noise, and vegetation movement easily leads to false detections. Moreover, existing indoor test models can only simulate unidirectional sliding, failing to reproduce the multi-directional complex deformations of actual landslides, and lack simulation of equipment performance fluctuations, resulting in serious discrepancies between algorithm verification results and real-world scenarios. Therefore, there is an urgent need for a high-precision monitoring solution that integrates automated image processing, dynamic threshold adjustment, and multi-modal feature verification, coupled with an indoor test model capable of realistically simulating multi-directional sliding, sudden changes in illumination, and noise interference, to address the problems of low sensitivity, high false alarm rates, and insufficient verification in existing technologies. Summary of the Invention

[0003] To address the above problems, this invention proposes a landslide deformation identification and early warning method based on video monitoring, comprising the following steps:

[0004] S1: Data Acquisition: The video stream is transmitted back in real time via the RTSP protocol through a high-definition camera. Then, the data is preprocessed. The video stream is parsed based on Matlab scripts. Keyframes are captured at 2-second intervals and converted into JPG image sequences. Histogram equalization enhancement processing is performed on the images. The contrast between feature points and background is improved by adjusting the grayscale distribution.

[0005] S2: Centroid extraction. The enhanced image is binarized using a fixed threshold method. Isolated noise points are filtered out by combining connected region area threshold analysis. Adhesive regions are separated by morphological erosion and dilation operations to preserve stable feature points.

[0006] S3: Deformation judgment. Based on the centroid coordinate sequence of feature points, a sliding window range envelope is constructed. The extreme values ​​and standard deviations of the coordinates within the window are statistically analyzed to expand the confidence interval boundary. When the centroid coordinates of multiple consecutive frames exceed the boundary, a deformation warning signal is triggered. At the same time, multimodal feature verification is performed by combining the aspect ratio change rate of the circumscribed rectangle with the difference between the contour Fourier descriptor.

[0007] Preferably, in S2, the fixed threshold method in image binarization is used to enhance the image data after histogram equalization. This method classifies the pixels in the image into two categories, foreground and background, by setting a threshold.

[0008] If the pixel value I(x,y) ≥ T, then it is set to 1, which is the foreground.

[0009] If the pixel value I(x,y) < T, then it is set to 0, which is the background.

[0010] Preferably, the image after binarization includes feature points, background, and noise. A second threshold is set for control to remove noise. Based on the adjacency relationship of pixels in the binary image, connected regions are marked, and the number of pixels in each region is counted as the area. A minimum area threshold is set to filter out isolated noise points with an area smaller than the threshold. Combining morphological erosion and dilation operations separates the adhesion regions and smooths the boundaries, retaining the effective feature points.

[0011] Preferably, the connected region means that the gray values of two adjacent pixels are the same.

[0012] Preferably, in S3, the sliding window range envelope algorithm statistically calculates the extreme values and standard deviations of the centroid coordinates of feature points through a dynamic reference window, expanding the boundary range of the confidence interval. When the centroid coordinates exceed the boundary for multiple consecutive frames, a deformation alarm is triggered.

[0013] Preferably, for the sliding window range envelope method, a dynamic reference window Wk = {ck - N,..., ck - 1} with a length of N = 15 is defined, and the coordinate extreme values within the window are calculated and the confidence interval is expanded:

[0014]

[0015] where is the standard deviation of the coordinates within the window, and the coefficient 1.96 corresponds to the 95% confidence interval of the normal distribution. The out-of-bounds criterion for the current centroid ck is defined as: , when Th = 5 consecutive frames satisfy (i = 0,..., 4), a deformation alarm is triggered.

[0016] Preferably, the aspect ratio change rate of the rectangle is .

[0017] Preferably, the difference of the contour Fourier descriptor is .

[0018] The landslide deformation identification and early warning method based on video monitoring proposed in this invention has the following beneficial effects: By combining automated image processing algorithms with high-fidelity indoor scaled physical model experiments, high-precision real-time monitoring and reliability verification of slope deformation are achieved. Addressing the problems of high cost, poor real-time performance, and weak anti-interference capability of traditional monitoring technologies, this solution proposes a complete hardware and software collaborative solution, significantly improving the accuracy and adaptability of landslide early warning. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0020] Figure 1 This is a schematic diagram of the landslide deformation identification and early warning method based on video monitoring according to the present invention;

[0021] Figure 2 This is a schematic diagram of the specific interface for data acquisition in this invention;

[0022] Figure 3 This is a schematic diagram illustrating a specific implementation of centroid extraction according to the present invention;

[0023] Figure 4 A schematic diagram for determining deformation;

[0024] Figure 5 This is a schematic diagram of the field layout;

[0025] Figure 6 Provide a schematic diagram for the model construction;

[0026] Figure 7 These are before and after diagrams showing the process of noise removal.

[0027] Figure 8 A schematic diagram illustrating image recognition and acquisition for an indoor experimental model;

[0028] Among them, 1. movable support; 2. steel inclined table; 3. anti-slip piles; 4. characteristic area; 5. obstruction; 6. highway; 7. monitoring equipment. Detailed Implementation

[0029] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0030] Example

[0031] The landslide deformation identification and early warning method based on video monitoring disclosed in this embodiment specifically includes the following steps: as shown in the figure.

[0032] S1: Data acquisition, including video capture and data preprocessing, utilizes a high-definition camera to transmit video streams in real-time via the RTSP protocol. A corrosion-resistant bracket is used to secure the Hikvision HST-IPD24D1-MBT / GLSE / N high-definition camera. Installation points are optimized through 3D modeling to ensure coverage of key landslide deformation areas, such as the main sliding direction, rear edge tensile cracks, and leading edge shear exits. Before camera installation, 3D point cloud data analysis of slope deformation trends is conducted to rationally plan the line-of-sight angle and coverage area, avoiding blind spots. The equipment supports 4G / fiber optic dual-mode transmission, transmitting video streams in real-time via the RTSP protocol. It is equipped with a solar power system and a backup lithium battery pack, ensuring continuous operation for over 72 hours in harsh outdoor environments. In the event of a network interruption, the system automatically enables the resume function, prioritizing the caching of key frame data and transmitting the complete video stream once the network is restored, ensuring data integrity. The video stream is encrypted with AES and then compressed in H.265 format before being transmitted. After receiving the video, the local server automatically generates a timestamp index file, which records the start time, resolution (1920×1080), frame rate (25 frames / second), and storage path of each video segment, facilitating quick retrieval and analysis.

[0033] Data preprocessing and video stream parsing are implemented using Matlab scripts. Keyframes are captured at 2-second intervals and converted into JPG image sequences, generating approximately 1800 frames per hour (3.6GB of data). Preprocessing latency is strictly controlled within 20 seconds to meet real-time requirements. To address image blurring, moisture interference, and uneven lighting caused by long-distance shooting, the system employs histogram equalization to enhance contrast. The cumulative distribution function is calculated using a normalized grayscale histogram, redistributing pixel gray levels and stretching the grayscale difference between feature points and the background. The processed image's dynamic range is expanded by over 30%, significantly improving feature point recognition. Further binarization segmentation is performed on the enhanced image, setting a fixed threshold to distinguish foreground from background. A connected component area thresholding method is used to filter out isolated noise points (such as gravel reflections and bird occlusion) with areas smaller than the threshold limit. The specific process includes connected component labeling, area statistics and filtering, morphological erosion and dilation operations, and finally, the retained stable feature points are recorded using centroid coordinates to form a time-series dataset. Morphological operations employ a 3×3 circular kernel to separate adhered regions and smooth boundary contours, ensuring the completeness and consistency of feature point extraction.

[0034] This invention provides a simple and effective contrast enhancement for the acquired image data: Histogram Equalization (HE). This method is a contrast enhancement technique based on the image's gray-level histogram. By redistributing the gray levels of the image, it makes the gray-level distribution of the output image more uniform, thereby expanding the dynamic range of the image and improving visual quality. The core idea of ​​the method is to use the cumulative distribution function (CDF) of the image to perform a non-linear mapping on the gray levels. This stretches the more concentrated gray levels in the original image, while compressing the sparser gray levels, ultimately achieving a uniform distribution of gray levels.

[0035] For a grayscale image, its gray level range is [0, L−1], where L is the total number of gray levels (usually 256). The image's histogram... Represents grayscale level The number of pixels, in its normalized form, is:

[0036]

[0037] in grayscale The number of pixels, where N is the total number of pixels in the image.

[0038] The cumulative distribution function (CDF) is defined as follows:

[0039]

[0040] CDF reflects the gray level. The proportion of pixels at gray levels 7 and below to the total number of pixels.

[0041] The gray levels are mapped using CDF to obtain new gray levels. To map the result to discrete gray levels, it is usually rounded:

[0042] After converting the data augmentation results into spatial height and color, it can be seen that there is a significant height difference between the feature points and the background. In order to extract feature points for subsequent processing, the invention uses image binarization to extract feature points.

[0043] As described in S2: centroid acquisition, i.e. feature point extraction, uses a fixed threshold method to perform binarization segmentation on the enhanced image, combines connected region area threshold analysis to filter out isolated noise points, and separates the adhered regions through morphological erosion and dilation operations to retain stable feature points;

[0044] The present invention uses the fixed threshold method in image binarization to enhance the image data after histogram equalization. This method divides the pixels in the image into two categories, foreground and background, by setting a threshold, as follows.

[0045] If the pixel value I(x,y)≥TI(x,y)≥T, it is set to 1 (foreground).

[0046] If the pixel value I(x,y)<TI(x,y)<T, it is set to 0 (background).

[0047]

[0048] However, although the features and the background are clearly distinguished in the binarized image, there are still many noise points, such as Figure 4 , which is caused by the reflection of a large number of gravels with different particle sizes on the slope surface under natural light. In order to further extract more stable and refined feature points, a second threshold is set here for control to remove the redundant noise points.

[0049] In binary image processing, noise usually appears as isolated pixel points or small connected areas, such as the white small connected areas scattered in the above figure. The noise removal method based on connectivity analysis calculates the area of the connected region and sets a threshold to distinguish noise from the effective target, thereby achieving noise filtering. The core of this method lies in combining connected region labeling, area statistics, and threshold screening techniques, which can effectively separate the noise and the target region and improve the accuracy of image processing.

[0050] First of all, connectivity refers to the adjacency relationship between pixels in an image. In a binary image, if the gray values of two pixels are the same and they are adjacent (4-neighborhood or 8-neighborhood), they are regarded as the same connected region. Noise usually appears as small isolated regions, while effective targets (such as anti-slide piles, landslide feature points, etc.) have larger connected areas. By setting an area threshold T, the connected regions with an area smaller than T can be determined as noise and deleted. Specifically, for the binary image I(x,y)∈{0,1}, the area Ai of the connected region Ri is defined as the total number of pixels in this region. If Ai<T, then Ri is marked as a noise region and set to the background value (0).

[0051] In practical applications, binarized images often require morphological operations to eliminate minute noise and enhance the integrity of connected regions. This invention uses structuring elements (such as circular or rectangular kernels) to perform erosion and dilation operations, separating adhered regions and smoothing edges. Subsequently, the image is traversed using a regionprops algorithm to label all connected regions and statistically analyze their properties (such as area, centroid, and bounding box). These properties are then used to provide a basis for subsequent noise filtering and target extraction.

[0052] Taking on-site video frame image data processing as an example, 280 feature points with connected regions were detected in the image. However, based on the actual distribution of anti-slide piles and the spatial distribution pattern of landslide feature points, such a large number of feature points do not exist on-site. Obviously, a large number of regions with low connectivity are noise points. Therefore, this invention sets an area threshold T=50, retaining only regions with connectivity greater than 50 as stable feature points, thereby achieving noise reduction processing for feature point extraction. Experimental results show a noise reduction rate as high as 98.13%, which can be further improved to 100% after manual selection and noise removal, indicating that this method can effectively remove noise and significantly improve the accuracy and reliability of feature point extraction.

[0053] Through the above method, this invention not only achieves efficient noise removal from binary images but also provides a high-quality data foundation for subsequent feature point analysis. The algorithm performs excellently in feature extraction tasks under complex backgrounds, exhibiting high robustness and generalization ability.

[0054] After efficient noise removal from the binary image, deformation calculation is performed as described in this case. A sliding window range envelope is constructed based on the centroid coordinate sequence of feature points, and the extreme values ​​and standard deviations of the coordinates within the window are statistically analyzed to expand the confidence interval boundary. When the centroid coordinates of multiple consecutive frames exceed the boundary, a deformation warning signal is triggered. Simultaneously, multimodal feature verification is performed by combining the aspect ratio change rate of the circumscribed rectangle with the difference between the contour Fourier descriptor. Specifically:

[0055] Real-time monitoring and early warning of structural deformation are achieved by constructing a spatiotemporal statistical model. The algorithm first obtains the centroid coordinates of the target region through image moment calculation, defining ct=(xt,yt) as the centroid position of frame t, where the coordinates are precisely calculated from the zeroth moment m00 and the first moments m10,m01.

[0056]

[0057] This method improves upon the moment invariance theory in digital image processing (Gonzalez & Woods, 2018) by enhancing segmentation robustness through morphological preprocessing. The centroid sequence C={c1,c2,...,cT} constitutes a temporal evolution process, and its statistical properties reflect changes in the target's motion pattern. The core detection mechanism employs the sliding window range envelope method, defining a dynamic reference window Wk={ck-N,...,ck-1} of length N=15, calculating the coordinate extrema within the window, and expanding the confidence interval.

[0058]

[0059] in Let be the standard deviation of the coordinates within the window, with a coefficient of 1.96 corresponding to the 95% confidence interval of a normal distribution. The out-of-bounds criterion for the current centroid ck is defined as:

[0060]

[0061] When Th=5 consecutive frames satisfy A deformation alarm is triggered when (i=0,...,4). This mechanism integrates spatial anomaly detection and temporal persistence verification, conforming to the sequence probability ratio test principle in fault detection. Preliminary experiments show that structural deformation causes a jump in the standard deviation of the centroid trajectory. The detection sensitivity can be enhanced by using an improved CUSUM control chart.

[0062]

[0063] in Let δ be the baseline displacement, and δ be the minimum detectable change. To further improve environmental adaptability, the algorithm introduces an adaptive boundary adjustment factor:

[0064]

[0065] Includes noise estimation term With instantaneous velocity term α and β are adjustable parameters, and PFA is the preset false alarm probability. This design draws inspiration from the constant false alarm rate (CFAR) processing technique in radar detection (Richards, 2014). Experimental verification shows that the aspect ratio change rate of the combined circumscribed rectangle... Difference from contour Fourier descriptor Multimodal features can increase the detection probability (PD) to 96.8% and reduce the false alarm rate (PFA) to below 1.2% (N=15, Th=5).

[0066] The deformation calculation module, based on dynamic centroid trajectory analysis, constructs a sliding window range envelope (window length 10 frames), statistically analyzes the extreme values ​​and standard deviations of coordinates within the window, and expands the 95% confidence interval boundary (extreme value ± 1.96 times the standard deviation). When the centroid coordinates exceed the boundary for 5 consecutive frames, the system triggers a deformation warning. Simultaneously, it performs multimodal verification by combining the aspect ratio change rate of the circumscribed rectangle and the difference in the Fourier descriptor of the contour, ensuring the reliability of the warning results. The aspect ratio change rate of the circumscribed rectangle detects shape distortion by calculating the aspect ratio change of the circumscribed rectangle of the target area in real time; the difference in the Fourier descriptor of the contour quantifies the deformation amplitude by comparing the Fourier coefficients of the current contour and the reference contour. Experiments show that this mechanism has a false alarm rate of less than 1.2% and a response time of 0.12 seconds, improving efficiency by more than 90% compared to traditional manual interpretation, and can effectively capture the complete evolution process of landslides from creep to abrupt change.

[0067] In this embodiment, a verification model was also established to verify the robustness of the system under multiple sets of tests in complex interference scenarios, including low light (brightness -50%), high noise (100% density) and single feature point conditions.

[0068] This case study presents an indoor test model for validating a landslide deformation identification system.

[0069] An indoor scaled-down landslide model capable of simulating multidirectional sliding and complex environmental disturbances was constructed. For example... Figure 3 As shown, the main body of the model consists of a steel inclined table (2m×1m), the slope of which is continuously adjustable (0°~90°) via a movable support mechanism. The surface is covered with a coating that is approximately the same color as the real soil. One end of the table is fixed to the wall to simulate bedrock constraint, while the free end is controlled by a movable support to control the unloading effect on the free face. The anti-slide pile model uses white hardwood cubes (4×4×4cm3) that approximate the characteristics of the field slope area, arranged in an orderly manner on the table surface and marked with thin lines to ensure that the position of the anti-slide piles is basically the same in each test.

[0070] The multi-directional sliding control system uses thin wires to pull the top of the anti-slide piles, applying force in directions including vertical sliding, 15° leftward deviation, 15° rightward deviation, combined left-right sliding, and 45° downward leftward sliding, covering typical displacement patterns of actual landslides. The initiation device uses a manually triggered semi-automatic pressing device to simulate multi-directional deformation. The initiation process is divided into two stages: the initial 20 seconds simulate the creep of the landslide body (centroid fluctuation range ±2 pixels), followed by high-speed sliding (physical process 0.5 seconds). The high-speed displacement is extended into a 2-second progressive deformation video stream using a frame interpolation algorithm to match the algorithm's detection requirements. Environmental interference simulation uses image processing software to adjust brightness and grain noise, and is equipped with detachable feature point modules (the number decreases from 8 to 1 to simulate local monitoring failure). The monitoring camera uses a DJI Action4 to capture undistorted video at 4K resolution and 120 frames per second, fixed 1.5 meters in front of the model, and adjusted to simulate non-ideal working conditions of field equipment.

[0071] Experimental results show that by dynamically adjusting the feature point extraction threshold (±5%~10%) and sliding window parameters, the false detection rate can be controlled within 2%, and the missed detection rate is less than 0.5%. The time difference for triggering warnings through multi-directional sliding is less than 0.08 seconds, and the direction recognition accuracy exceeds 95%, verifying the algorithm's strong robustness to complex interference scenarios. Furthermore, the granular noise and light / darkness variation data injected into the model experiments provide a high-fidelity experimental basis for algorithm optimization, significantly improving the system's adaptability in practical engineering.

Claims

1. A method for landslide deformation identification and early warning based on video monitoring, characterized in that, It includes the following steps: S1: Data acquisition: The video stream is transmitted in real time through an RTSP protocol by a high-definition camera, and then data preprocessing is performed. The video stream is parsed based on a Matlab script, key frames are intercepted at 2-second intervals and converted into a JPG image sequence. Histogram equalization enhancement processing is performed on the images to improve the contrast between feature points and the background by adjusting the gray distribution; S2: Centroid extraction. The fixed threshold method is used to perform binary segmentation on the enhanced image. Isolated noise points are filtered out by combining the analysis of the connected region area threshold, and the morphological erosion and dilation operations are used to separate the adhesion regions and retain stable feature points; S3: Deformation judgment. A sliding window range envelope is constructed based on the centroid coordinate sequence of the feature points, and the coordinate extreme values and standard deviation within the window are statistically analyzed to expand the confidence interval boundary; When the centroid coordinates of multiple consecutive frames exceed the boundary, a deformation warning signal is triggered. At the same time, multi-modal feature verification is performed by combining the aspect ratio change rate of the circumscribed rectangle and the difference of the contour Fourier descriptor; For the sliding window range envelope method, a dynamic reference window Wk={ck-N,...,ck-1} with a length of N = 15 is defined, and the coordinate extreme values within the window are calculated and the confidence interval is expanded: in The standard deviation of the coordinates within the window is given, and a coefficient of 1.96 corresponds to the 95% confidence interval of a normal distribution. The current boundary crossing criterion for the centroid ck is defined as follows: When Th=5 consecutive frames satisfy Deformation alarm is triggered at time, i=0,...,4.

2. The landslide deformation identification and early warning method based on video monitoring according to claim 1, characterized in that, In S2, the fixed threshold method in image binarization is used to enhance the image data after histogram equalization. This method divides the pixels in the image into two categories by setting a threshold: foreground and background, If the pixel value I(x,y)≥TI(x,y)≥T, it is set to 1, which is the foreground; If the pixel value I(x,y)<TI(x,y)<T, it is set to 0, which is the background.

3. The landslide deformation identification and early warning method based on video monitoring according to claim 2, characterized in that, The image after binarization includes feature points, background, and noise points. A second threshold is set for control to remove noise points. The connected regions are marked based on the adjacency relationship of the pixels in the binary image, and the number of pixels in each region is counted as the area; A minimum area threshold is set to filter out isolated noise points with an area smaller than the threshold; The morphological erosion and dilation operations are combined to separate the adhesion regions and smooth the boundaries, and the effective feature points are retained.

4. The landslide deformation identification and early warning method based on video monitoring according to claim 1 or 3, characterized in that, The connected region means that the gray values of two pixels are the same and adjacent.

5. The landslide deformation identification and early warning method based on video monitoring according to claim 1, characterized in that, The ratio of the rectangle's length to width changes by a certain percentage. .

6. The landslide deformation identification and early warning method based on video monitoring according to claim 1, characterized in that, Contour Fourier descriptor differences are .

Images