A landslide warning method combining slope radar vision perception technology

Abstract

The present invention discloses a landslide warning method combining slope radar vision perception technology, which periodically monitors a target slope to obtain surface abnormal deformation data and optical images of the target slope; identifies local cracking areas and seepage areas in the optical images of the target slope based on deep learning methods, and quantifies the characteristic parameters of the local cracking areas and seepage areas; performs spatio-temporal fusion on the surface abnormal deformation data of the target slope and the characteristic data in the optical images, and constructs a slope landslide monitoring and warning method based on the fused surface abnormal deformation data and optical images; the present invention combines synthetic aperture radar and video surveillance device vision monitoring to comprehensively improve the accuracy and reliability of slope instability monitoring. By analyzing optical images through deep learning, risk characteristics such as local cracking and seepage are timely identified, and spatio-temporal fusion of deformation amounts and visual characteristics is achieved, significantly improving the sensitivity of the warning mechanism and reducing the false alarm rate.

Description

Technical Field

[0001] The present invention relates to the technical field of slope landslide early warning, and specifically relates to a landslide early warning method combining slope radar vision perception technology. Background Art

[0002] The method for monitoring and early warning of slope instability is an important research topic in the field of safety production. The surface abnormal deformation generated before slope instability is the most easily captured and monitored feature. Synthetic aperture slope radar (S-SAR) and image recognition technology have been widely used in landslide disaster monitoring. Synthetic aperture slope radar (S-SAR) senses the minute deformation of the slope surface (along the monitoring line of sight) by transmitting microwaves and receiving reflected signals, and image recognition technology is also used to identify the cracking and water seepage problems existing in the slope.

[0003] However, S-SAR can only monitor the deformation amount, and for other visual potential precursors of slope instability, such as local cracking and water seepage, it usually cannot effectively capture them. The early warning ability of the landslide monitoring and early warning model established through a single precursor feature (deformation amount / deformation rate) is limited. Especially for hard rock slopes, their collapse or instability occurs suddenly and rapidly, and usually does not show obvious deformation before instability, making it difficult to reach the set early warning threshold, and it is very difficult to monitor and early warn such landslides. Summary of the Invention

[0004] The purpose of the present invention is to provide a landslide early warning method combining slope radar vision perception technology to solve the technical problem of the limited early warning ability of the landslide monitoring and early warning model established through a single precursor feature (deformation amount / deformation rate) in the prior art.

[0005] To solve the above technical problems, the present invention specifically provides the following technical solutions:

[0006] A landslide early warning method combining slope radar vision perception technology includes the following steps:

[0007] Step 100: Periodically monitor the target slope using a radar and a video monitoring device respectively to obtain the surface abnormal deformation data of the target slope from the radar monitoring data and obtain the optical image of the target slope;

[0008] Step 200: Identify the local cracking area and water seepage area in the optical image of the target slope based on the deep learning method, and quantify the characteristic parameters of the local cracking area and water seepage area;

[0009] Step 300: Perform temporal fusion and spatial fusion on the surface abnormal deformation data of the target slope and the characteristic parameters of the local cracking area and the water seepage area, and respectively determine the temporal coordinate values and spatial coordinate values of the abnormal deformation data on the surface with respect to the local cracking area and the water seepage area. Based on the surface abnormal deformation data, and the temporal and spatial coordinate values of the local cracking area and the water seepage area, construct a slope landslide monitoring and early warning method.

[0010] As a preferred embodiment of the present invention, in the step 100, the monitoring periods of the S-SAR radar and the video monitoring device are respectively and , and the starting time points of the monitoring are respectively and ;

[0011] Obtain radar monitoring data and slope optical images under different time series, and obtain the surface abnormal deformation amount of the target slope from the radar monitoring data R. As a preferred embodiment of the present invention, in the step 200, when implementing the method for identifying the local cracking area and the water seepage area in the optical image of the target slope based on the deep learning method, first train a deep learning model with historical images with slope cracking areas and water seepage areas until the deep learning model can identify optical images with slope cracking and water seepage. The specific implementation method is as follows:

[0012] Collect historical photo samples with slope cracking areas and water seepage areas, manually mark the slope cracking areas and water seepage areas in each image of the historical photo samples, and convert the historical photo samples into binary images retaining the slope cracking areas and water seepage areas;

[0013] Use the original historical photo samples and their corresponding binary images as the input and output of the convolutional neural network, and obtain a deep learning model that can be used to identify slope cracking areas and water seepage areas through training and optimization;

[0014] Input the optical image of the target slope into the deep learning model to obtain the binary image dataset of the target slope, as well as the slope cracking areas and water seepage areas in each binary image.

[0015] As a preferred embodiment of the present invention, the implementation method for quantifying the characteristic parameters of the local cracking area and the water seepage area is:

[0016] Traverse the binary image dataset For the pixel values in the binary image, detect the pixel points in the slope cracking area and the water seepage area. If they exist, it indicates that the deep learning model has recognized the slope cracking area and the water seepage area;

[0017] Adopt the morphological analysis method to quantify the cracking length and the maximum cracking width of the slope cracking area, and quantify the water seepage area of the water seepage area.

[0018] As a preferred solution of the present invention, the implementation method of using the morphological analysis method for the slope cracking area and the water seepage area is:

[0019] Use the contour search number and area calculation function in the OpenCV software library of Python to calculate the water seepage area in the binary image;

[0020] Use the skeleton morphology function in the skimage software library of Python to calculate the single-pixel-wide skeleton of the slope cracking area in the binary image. The distance between the two farthest endpoints on the single-pixel-wide skeleton line is the cracking length;

[0021] Use the skeleton morphology function in the skimage software library of Python to calculate the distance between the two black pixel points with the farthest distance in the normal direction of the skeleton line in the binary image, which is the maximum cracking width.

[0022] As a preferred solution of the present invention, in the step 300, perform temporal fusion and spatial fusion on the surface abnormal deformation data of the target slope and the characteristic parameters in the optical image, so as to map the surface abnormal deformation data of the target slope and the characteristic data in the optical image to a unified time axis and a three-dimensional space coordinate system. The specific implementation method is:

[0023] Map the radar monitoring data and the optical images taken by the video monitoring device to the same fusion time series, and use the linear interpolation method to calculate the surface abnormal deformation amount of the radar monitoring data corresponding to the time when the video monitoring device takes pictures;

[0024] Construct a three-dimensional coordinate system with the radar monitoring data, perform three-dimensional fusion on the radar monitoring data and the slope cracking area and the water seepage area in the optical image within the same slope area, use the collinear equation to calculate the three-dimensional coordinate values of the slope cracking area and the water seepage area in the three-dimensional coordinate system, and determine the three-dimensional coordinate values of each pixel point in the optical image;

[0025] Determine the surface abnormal deformation amount in the radar monitoring data, and the three-dimensional coordinate values corresponding to the slope cracking area and the water seepage area of the optical image respectively, and perform spatial fusion on the surface abnormal deformation amount in the radar monitoring data and the slope cracking area and the water seepage area of the optical image with the same three-dimensional coordinate values.

[0026] As a preferred embodiment of the present invention, when performing time fusion on the surface abnormal deformation data of the target slope and the characteristic parameters in the optical image, the patrol shooting period of the video monitoring device is set to , and the radar monitoring period is . If the nth patrol shooting moment of the video monitoring device is t, then: ; where is the time point when the video monitoring device starts patrol shooting;

[0027] The monitoring periods of the S-SAR radar and the video monitoring device are and respectively, and the starting monitoring time points are and Let ; where t0 is the time point when the S-SAR radar starts monitoring; there is ; then the radar monitoring data at time t is: .

[0028] As a preferred embodiment of the present invention, the slope cracking area and the water seepage area in the optical image are in a two-dimensional plane coordinate system. The collinearity equation is used to establish the mapping relationship between the two-dimensional plane coordinate system and the three-dimensional coordinate system, and the collinearity equation is established and solved to determine the three-dimensional coordinate values of the slope cracking area and the water seepage area in the three-dimensional coordinate system; ; ; In the formula: is the image plane coordinate value of each pixel point in the optical image; is the internal orientation element of the optical image; are the three outer orientation line elements of the optical image; is the three-dimensional coordinate value of each pixel point in the optical image in the three-dimensional coordinate system; are the nine direction cosines composed of the three outer orientation angle elements of the image.

[0029] As a preferred embodiment of the present invention, when the time point corresponding to the surface abnormal deformation amount in the radar monitoring data matches the time point when the slope cracking area and the water seepage area appear in the optical image in the fusion time series, and the three-dimensional coordinate value corresponding to the surface abnormal deformation amount in the radar monitoring data is the same as the three-dimensional coordinate value corresponding to the slope cracking area and the water seepage area in the optical image, based on the standardized data of the surface abnormal deformation amount, the slope cracking area and the water seepage area, the comprehensive risk index of this monitoring point is set, and the warning value of this monitoring point on the target slope is determined based on the comprehensive risk index;

[0030] When the time points corresponding to the surface abnormal deformation amounts in the radar monitoring data do not match the time points of the slope cracking areas and seepage areas in the optical images on the fused time series, or when the three-dimensional coordinate values corresponding to the surface abnormal deformation amounts in the radar monitoring data are different from the three-dimensional coordinate values of the slope cracking areas and seepage areas in the optical images, the surface abnormal deformation amounts are used as single parameters, and the slope cracking areas and seepage areas are used as single parameters to independently determine the warning values of the monitoring points where abnormalities occur.

[0031] As a preferred embodiment of the present invention, the method for setting the comprehensive risk index of the monitoring point based on the standardized data of the surface abnormal deformation amount, the slope cracking area and the seepage area is as follows:

[0032] Standardize the quantitative characteristic data of the surface abnormal deformation amount in the radar monitoring data of the monitoring point, the slope cracking area and the seepage area in the optical image;

[0033] Assign weights to the surface abnormal deformation amount, the slope cracking area and the seepage area of the monitoring point. Multiply the standardized characteristic data by their corresponding weights to obtain weighted characteristic values, and calculate the comprehensive risk index of the monitoring point through the summation of the weighted characteristic values. By setting the threshold of the comprehensive risk index, the comprehensive risk index is divided into different monitoring risk levels; the comprehensive risk index ; where R is the comprehensive risk index, D is the standardized surface abnormal deformation amount, C is the standardized cracking length, W is the standardized maximum cracking width, S is the standardized seepage area, w 1 、w 2 、w 3 and w 4 are the assigned weights corresponding to the surface abnormal deformation amount, the cracking length, the maximum cracking width and the seepage area, respectively.

[0034] The present invention has the following beneficial effects compared with the prior art:

[0035] The present invention combines two slope monitoring methods, namely, the visual monitoring of synthetic aperture radar and video surveillance equipment. By analyzing optical images through deep learning, risk features such as local cracking and water seepage are timely identified. The surface abnormal deformation amount identified by radar monitoring is spatially and temporally fused with the local cracking and water seepage of visual features to determine whether there are problems of local cracking, water seepage, and surface abnormal deformation simultaneously at the same monitoring point at a certain monitoring time point. Moreover, for the same monitoring point with simultaneous local cracking, water seepage, and surface abnormal deformation problems, feature fusion is performed, and a monitoring risk level is set, which can significantly improve the sensitivity of the early warning mechanism and reduce the false alarm rate. This multi-feature fusion method is not only applicable to hard rock slopes but also can be widely applied to the monitoring of other types of slopes, providing effective guarantee for safe production. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only exemplary, and for those of ordinary skill in the art, without creative efforts, other implementation drawings can be obtained according to the provided drawings.

[0037] Figure 1 It is a flowchart of the landslide early warning method according to an embodiment of the present invention;

[0038] Figure 2 It is an effect diagram of manually marked cracking areas in the landslide early warning method according to an embodiment of the present invention;

[0039] Figure 3 It is an effect diagram of manually marked water seepage areas in the landslide early warning method according to an embodiment of the present invention;

[0040] Figure 4 It is a flowchart of constructing a convolutional neural network in the landslide early warning method according to an embodiment of the present invention;

[0041] Figure 5 It is a schematic diagram of quantifying cracking areas in the landslide early warning method according to an embodiment of the present invention;

[0042] Figure 6 It is a schematic diagram of quantifying water seepage areas in the landslide early warning method according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0044] As shown Figure 1 in the figure, the present invention provides a landslide warning method combining slope radar vision perception technology. By combining the monitoring results of slope radar with the vision perception parameters formed by optical image recognition technology, the slope radar can capture the precursor characteristics of slope instability more comprehensively and accurately, and establish a slope instability monitoring and warning method considering multi-feature fusion, including the following steps:

[0045] Step 100: Use radar and video monitoring devices to periodically monitor the target slope, so as to obtain the surface abnormal deformation data of the target slope from the radar monitoring data, and obtain the optical image of the target slope.

[0046] In step 100, use S-SAR radar and video monitoring devices to periodically monitor the target slope. Among them, the monitoring periods of the S-SAR radar and the video monitoring device are respectively and , and the starting monitoring points are respectively and ;

[0047] Obtain radar monitoring data and slope optical images at different time series, and obtain the surface abnormal deformation amount of the target slope from the radar monitoring data R. Step 200: Based on the deep learning method, identify the local cracking area and water seepage area in the optical image of the target slope, and quantify the characteristic parameters of the local cracking area and water seepage area.

[0048] In step 200, when implementing the method for identifying the local slope cracking area and water seepage area in the optical image of the target slope based on the deep learning method, first train the deep learning model with historical images with slope cracking and water seepage until the deep learning model can identify the optical images with slope cracking and water seepage. The specific implementation method is as follows:

[0049] (1) Collect historical photo samples with slope cracking areas and water seepage areas, and manually mark the slope cracking areas and water seepage areas in each image of the historical photo samples, and convert the historical photo samples into binary images retaining the slope cracking areas and water seepage areas. Among them, generate the binary image of the photo sample (a black-and-white image), the crack and water seepage areas are used as the foreground (black), and other areas are used as the background (white). That is, when converting the historical photo sample into a binary image retaining the slope cracking area and water seepage area, set the pixel values of the slope cracking area and water seepage area to 0, and set the pixel values of the non-slope cracking area and water seepage area to 255.

[0050] As Figure 2 shown, whereFigure 2 Figure (a) shows a partial cracking diagram of the rock mass in the historical photo sample. Figure 2 Figure (b) shows the binary image of the partial cracking diagram of the rock mass, with the slope cracking area manually marked.

[0051] As Figure 3 shown, where Figure 3 Figure (a) shows a partial water seepage diagram of the rock mass in the historical photo sample. Figure 3 Figure (b) shows the binary image of the partial water seepage diagram of the rock mass, with the water seepage area manually marked.

[0052] (2) Use the original historical photo sample and its corresponding binary image as the input and output of the convolutional neural network, and obtain a deep learning model that can be used to identify the slope cracking area and water seepage area through training and optimization.

[0053] (3) Input the optical image of the target slope into the deep learning model to obtain the binary image dataset of the target slope , as well as the slope cracking area and water seepage area in each binary image. The training process of the deep learning model and the binarization process of the optical image of the target slope are as Figure 3 shown.

[0054] The implementation method for quantifying the characteristic parameters of the local cracking area and water seepage area is as follows:

[0055] Traverse the pixel values in the binary images in the binary image dataset , and detect whether there are pixel points with a pixel value of 0. If so, it means that the deep learning model has identified the slope cracking area and water seepage area.

[0056] Use the morphological analysis method to quantify the cracking length and maximum cracking width of the slope cracking area, and quantify the water seepage area of the water seepage area.

[0057] The implementation method for using the morphological analysis method for the slope cracking area and water seepage area is as follows:

[0058] Use the contour finding function and area calculation function in the OpenCV software library of Python to calculate the water seepage area in the binary image, as Figure 6 shown;

[0059] Use the skeleton morphological function in the skimage software library of Python to calculate the single-pixel-wide skeleton of the slope cracking area in the binary image. The distance between the two farthest endpoints on the single-pixel-wide skeleton line is the cracking length;

[0060] Calculate the distance between the two black pixels with the farthest normal direction from the skeleton line in the binary image using the skeleton morphology function in the skimage library of Python, which is the maximum cracking width, as Figure 5 shown.

[0061] Step 300: Perform time fusion and spatial fusion on the surface abnormal deformation data of the target slope and the characteristic parameters of the local cracking area and the seepage area, and respectively determine the time coordinate values and spatial coordinate values of the surface abnormal deformation data, the local cracking area, and the seepage area. Based on the time coordinate values and spatial coordinate values of the surface abnormal deformation data, the local cracking area, and the seepage area, construct a slope landslide monitoring and early warning method.

[0062] In step 300, perform time fusion and spatial fusion on the surface abnormal deformation data of the target slope and the characteristic parameters in the optical image, so as to map the surface abnormal deformation data of the target slope and the characteristic data in the optical image to the unified time axis and the three-dimensional space coordinate system. The specific implementation method is as follows:

[0063] Time fusion: Map the radar monitoring data and the optical images captured by the video monitoring device to the same fused time series, and use the linear interpolation method to calculate the surface abnormal deformation amount of the radar monitoring data corresponding to the time when the video monitoring device captures images.

[0064] When performing time fusion on the surface abnormal deformation data of the target slope and the characteristic parameters in the optical image, set the image capture period of the video monitoring device as , and the radar monitoring period as . If the nth image capture time of the video monitoring device is t, then: ; where is the starting time point of the video monitoring device to capture images;

[0065] The monitoring periods of the S-SAR radar and the video monitoring device are and respectively, and the starting monitoring time points are and respectively;

[0066] Let ; where t0 is the starting time point of the S-SAR radar to start monitoring. There is ; then the radar monitoring data at time t is: .

[0067] In this embodiment, when the S-SAR radar scans the target slope once, the video monitoring device captures an image corresponding to the target slope, constructs a virtual fused time series, and uses the monitoring period of the video monitoring device as the reference value to establish the image capture period of the video monitoring device , the radar monitoring period The matching relationship between them is used to determine the optical image captured by the video monitoring device at time t and the radar monitoring data corresponding to time t' that matches time t based on this matching relationship. By integrating the optical image corresponding to each shooting time point and the radar monitoring data formed by the radar scanning time point that matches this shooting time point into the same fusion time series, it is possible to mark the slope area where surface abnormal deformation, cracks, and water seepage occur simultaneously.

[0068] Spatial fusion: A three-dimensional coordinate system is constructed with the radar monitoring data. The radar monitoring data within the same slope area is three-dimensionally fused with the slope cracking area and water seepage area in the optical image. The collinearity equation is used to calculate the three-dimensional coordinate values of the slope cracking area and water seepage area in the three-dimensional coordinate system, and the three-dimensional coordinate values of each pixel point in the optical image are determined.

[0069] Determine the surface abnormal deformation amount in the radar monitoring data and the three-dimensional coordinate values corresponding to the slope cracking area and water seepage area of the optical image respectively, and spatially fuse the surface abnormal deformation amount in the radar monitoring data with the same three-dimensional coordinate values with the slope cracking area and water seepage area of the optical image.

[0070] The slope cracking area and water seepage area in the optical image are in a two-dimensional plane coordinate system. The mapping relationship between the two-dimensional plane coordinate system and the three-dimensional coordinate system is established using the collinearity equation. The collinearity equation is established and solved to determine the three-dimensional coordinate values of the slope cracking area and water seepage area in the three-dimensional coordinate system; ; ; Where: is the image plane coordinate value of each pixel point in the optical image; is the internal orientation element of the optical image; are the three external orientation line elements of the optical image; is the three-dimensional coordinate value of each pixel point in the optical image; are the nine direction cosines composed of the three external orientation angle elements of the image.

[0071] To facilitate the understanding of this equation, it is also necessary to additionally introduce the three internal orientation elements of the image. f is the focal length of the camera. Since the optical axis center of the camera cannot be exactly centered on the center of the photo (i.e., the sensor center) during the manufacturing process, there will be a slight deviation, which is represented by x0 and y0, generally at the sub-millimeter level. x0, y0, and f can be easily obtained through the camera calibration program, which means that these three parameters are also known values in the collinearity equation. Observing the equation, it can be found that in order to solve for X, Y, and Z, it is also necessary to know the three external orientation line elements and the three external orientation angle elements The line element represents the position of the camera relative to the earth coordinate system, and the angle element represents the spatial posture of the camera. When drones on the market take photos, the photos will have POS information, which automatically stores the exterior orientation line element and angle element values. Therefore, the three exterior orientation line elements and the three exterior azimuth elements Is a known number.

[0072] The purpose of spatial fusion is to test whether the abnormal surface deformation of radar monitoring data and the slope cracking and water seepage characteristics identified in the optical image can correspond to the same area. According to the model of video surveillance equipment and the position sensor information of the video surveillance equipment, the internal and external orientation elements of each photo can be determined. By taking at least two photos taken at different angles of the same slope cracking area and water seepage area, the collinear equation can be established and solved to determine the position of the slope cracking area and water seepage area in three-dimensional space.

[0073] When the time point corresponding to the appearance of abnormal surface deformation in the radar monitoring data matches the time point of the appearance of slope cracking area and water seepage area in the optical image in the fused time series, and the three-dimensional coordinate value corresponding to the abnormal surface deformation in the radar monitoring data is the same as the three-dimensional coordinate value corresponding to the slope cracking area and water seepage area in the optical image, a comprehensive risk index of the monitoring point is set based on the standardized data of the abnormal surface deformation, the slope cracking area and the water seepage area, and the early warning value of the monitoring point on the target slope is determined based on the comprehensive risk index.

[0074] When the time point corresponding to the appearance of abnormal surface deformation in the radar monitoring data does not match the time point when the slope cracking area and the water seepage area appear in the optical image in the fused time series, or when the three-dimensional coordinate value corresponding to the abnormal surface deformation in the radar monitoring data is different from the three-dimensional coordinate value corresponding to the slope cracking area and the water seepage area in the optical image, the abnormal surface deformation is taken as a single parameter, and the slope cracking area and the water seepage area are taken as single parameters to independently determine the warning value of the monitoring point where the abnormality occurs.

[0075] The implementation method of setting the comprehensive risk index of the monitoring point based on the standardized data of abnormal surface deformation, slope cracking area and seepage area is as follows:

[0076] The abnormal surface deformation in the radar monitoring data of the monitoring point, and the quantitative characteristic data of the slope crack area and the water seepage area in the optical image are standardized. In this embodiment, the quantitative characteristic data of the abnormal surface deformation, the slope crack area and the water seepage area are subjected to Z-score standardization to form quantity-free tempered data.

[0077] Assign weights to the surface abnormal deformation amount, slope cracking area, and seepage area of the monitoring point. Multiply the standardized characteristic data by their corresponding weights to obtain weighted characteristic values. Calculate the comprehensive risk index of the monitoring point through the summation of the weighted characteristic values. Set the threshold of the comprehensive risk index and divide the comprehensive risk index into different monitoring risk levels; Comprehensive risk index ; where R is the comprehensive risk index, D is the standardized surface abnormal deformation amount, C is the standardized cracking length, W is the standardized maximum cracking width, S is the standardized seepage area, w 1 、w 2 、w 3 and w 4 are the assigned weights corresponding to the surface abnormal deformation amount, cracking length, maximum cracking width, and seepage area, respectively. In actual monitoring, when surface abnormal deformation, cracking, and seepage occur simultaneously at the same monitoring point, it means that the risk level of the monitoring point is relatively high. The defect of the existing method of separately warning multiple monitoring results is that when the surface abnormal deformation, cracking, and seepage occurring simultaneously at the same monitoring point do not reach the warning values respectively, no warning will be issued. Moreover, the radar principle only monitors the slope displacement by emitting electromagnetic waves and cannot obtain any other precursor characteristics of slope instability. If there is a relatively dangerous situation, that is, the slope has cracked / seeped, but the radar fails to issue a warning or cannot issue a warning.

[0078] However, the processing method of this embodiment can endow the slope radar with visual perception ability. When surface abnormal deformation, cracking, and seepage occur simultaneously at the same monitoring point, a comprehensive risk index is reconstructed for hierarchical warning. Even when the surface abnormal deformation, cracking, and seepage do not reach the warning values respectively, the comprehensive risk index formed by their fusion will issue an alarm according to the specific value. By calculating the comprehensive risk index in real time, potential risks can be detected in a timely manner. Regularly adjust the weights and thresholds according to new monitoring data and actual events to improve the accuracy of warning.

[0079] This embodiment combines two slope monitoring methods, namely, the visual monitoring of synthetic aperture radar and video surveillance equipment. By analyzing optical images through deep learning, it can timely identify risk features such as local cracking and water seepage, and fuse the surface abnormal deformation amount identified by radar monitoring with the local cracking and water seepage of visual features in terms of time and space to determine whether there are problems of local cracking, water seepage, and surface abnormal deformation at the same monitoring point at a certain monitoring time point. Moreover, it fuses the features of the same monitoring point with simultaneous local cracking, water seepage, and surface abnormal deformation, sets the monitoring risk level, which can significantly improve the sensitivity of the early warning mechanism and reduce the false alarm rate. This multi-feature fusion method is not only applicable to hard rock slopes but also can be widely applied to the monitoring of other types of slopes, providing effective guarantee for safe production.

[0080] The above embodiments are only exemplary embodiments of the present application and are not used to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art can make various modifications or equivalent replacements to the present application within the essence and protection scope of the present application, and such modifications or equivalent replacements should also be regarded as falling within the protection scope of the present application.

Claims

1. A landslide early warning method combined with slope radar visual perception technology, characterized in that: The following steps are involved: Step 100: Periodically monitor the target slope using radar and video monitoring equipment respectively, so as to obtain abnormal surface deformation data of the target slope from radar monitoring data, and obtain an optical image of the target slope; Step 200: identifying local cracked areas and water seepage areas in the optical image of the target slope based on a deep learning method, and quantifying characteristic parameters of the local cracked areas and water seepage areas; Step 300: Temporally and spatially fuse the surface abnormal deformation data of the target slope and the characteristic parameters of the local cracking area and the water seepage area, and respectively determine the time coordinate values ​​and space coordinate values ​​of the surface abnormal deformation data and the local cracking area and the water seepage area, and construct a slope landslide monitoring and early warning method based on the surface abnormal deformation data, the time coordinate values ​​and space coordinate values ​​of the local cracking area and the water seepage area; In step 300, the abnormal surface deformation data of the target slope and the characteristic parameters in the optical image are temporally fused and spatially fused, so that the abnormal surface deformation data of the target slope and the characteristic data in the optical image are mapped to a unified time axis and a three-dimensional space coordinate system. The specific implementation method is: The radar monitoring data and the optical images taken by the video surveillance equipment are mapped to the same fusion time series, and the surface abnormal deformation of the radar monitoring data corresponding to the video surveillance equipment patrol time is calculated by linear interpolation method; A three-dimensional coordinate system is constructed using radar monitoring data, the radar monitoring data in the same slope area is three-dimensionally integrated with the slope crack area and the water seepage area in the optical image, the three-dimensional coordinate values ​​of the slope crack area and the water seepage area in the three-dimensional coordinate system are calculated using a collinear equation, and the three-dimensional coordinate value of each pixel point in the optical image is determined; Determine the abnormal surface deformation in the radar monitoring data and the three-dimensional coordinate values ​​corresponding to the slope cracking area and the water seepage area of ​​the optical image, and spatially fuse the abnormal surface deformation in the radar monitoring data with the slope cracking area and the water seepage area of ​​the optical image having the same three-dimensional coordinate value.

2. The landslide early warning method combined with slope radar visual perception technology according to claim 1 is characterized in that: In step 100, the monitoring cycles of the S-SAR radar and the video surveillance equipment are respectively and The monitoring start time points are and ; Obtain radar monitoring data in different time series and slope optical images , the surface abnormal deformation of the target slope is obtained from the radar monitoring data R.

3. The landslide early warning method combined with slope radar visual perception technology according to claim 1 is characterized in that: In step 200, when implementing the method of identifying the local cracked area and the water seepage area of ​​the slope in the optical image of the target slope based on the deep learning method, the deep learning model is first trained by using historical images with the cracked area and the water seepage area of ​​the slope until the deep learning model can identify the optical image with the cracked area and the water seepage of the slope. The specific implementation method is: Collect historical photo samples with slope cracking areas and water seepage areas, manually mark the slope cracking areas and water seepage areas in each image of the historical photo samples, and convert the historical photo samples into binary images that retain the slope cracking areas and water seepage areas; The original historical photo samples and their corresponding binary images were used as the input and output of the convolutional neural network, and a deep learning model that can be used to identify slope cracking areas and seepage areas was obtained through training and optimization. The optical image of the target slope is input into the deep learning model to obtain a binary image dataset of the target slope. , as well as the slope cracking area and water seepage area in each binary image.

4. The landslide early warning method combined with slope radar visual perception technology according to claim 3 is characterized in that: The implementation method of quantifying the characteristic parameters of the local cracking area and the water seepage area is: Traverse the binary image dataset Detect the pixel points where the slope crack area and the water seepage area appear based on the pixel values ​​in the binary image in , if they exist, it means that the deep learning model has identified the slope crack area and the water seepage area; The morphological analysis method was used to quantify the crack length and maximum crack width of the slope cracking area, and the seepage area of ​​the seepage area was quantified.

5. The landslide early warning method combined with slope radar visual perception technology according to claim 4 is characterized in that: The implementation method of using morphological analysis method for slope cracking area and seepage area is as follows: Calculate the area of ​​the water seepage region in the binary image using the contour search and area calculation functions in the OpenCV software library of Python; The skeleton morphology function in Python's skimage software library is used to calculate the single-pixel wide skeleton of the slope cracking area in the binary image. The distance between the two farthest endpoints on the single-pixel wide skeleton line is the crack length. The skeleton morphology function in Python's skimage software library is used to calculate the distance between the two black pixels with the farthest distance in the normal direction of the skeleton line in the binary image, which is the maximum crack width.

6. The landslide early warning method combined with slope radar visual perception technology according to claim 1 is characterized in that: When the surface abnormal deformation data of the target slope and the characteristic parameters in the optical image are time-fused, the patrol cycle of the video monitoring device is set to , the radar monitoring cycle is , let the time of the nth patrol of the video surveillance device be t, then: ;in is the time point when the video surveillance equipment starts patrolling; the monitoring cycles of S-SAR radar and video surveillance equipment are and The monitoring start points are and ; make ; t0 is the time when the S-SAR radar starts monitoring; ; Then the radar monitoring data at time t is: .

7. The landslide early warning method combined with slope radar visual perception technology according to claim 1 is characterized in that: The cracked area and water seepage area of ​​the slope in the optical image are two-dimensional plane coordinate systems. The mapping relationship between the two-dimensional plane coordinate system and the three-dimensional coordinate system is established using the collinear equation. The collinear equation is established and solved to determine the three-dimensional coordinate values ​​of the cracked area and water seepage area of ​​the slope in the three-dimensional coordinate system. ; ; Where: , is the image plane coordinate value of each pixel in the optical image; , , is the internal orientation element of the optical image; , , are the three exterior orientation line elements of the optical image; is the three-dimensional coordinate value of each pixel point of the optical image in the three-dimensional coordinate system; The nine direction cosines are composed of the three exterior azimuth elements of the image.

8. The landslide early warning method combined with slope radar visual perception technology according to claim 1 is characterized in that: When the time point corresponding to the appearance of abnormal surface deformation in the radar monitoring data matches the time point of the appearance of the slope cracking area and the water seepage area in the optical image in the fused time series, and the three-dimensional coordinate value corresponding to the abnormal surface deformation in the radar monitoring data is the same as the three-dimensional coordinate value corresponding to the slope cracking area and the water seepage area in the optical image, a comprehensive risk index of the monitoring point is set based on the standardized data of the abnormal surface deformation, the slope cracking area and the water seepage area, and the early warning value of the monitoring point on the target slope is determined based on the comprehensive risk index; When the time point corresponding to the appearance of abnormal surface deformation in the radar monitoring data does not match the time point at which the slope crack area and the water seepage area appear in the optical image in the fused time series, or when the three-dimensional coordinate value corresponding to the abnormal surface deformation in the radar monitoring data is different from the three-dimensional coordinate value corresponding to the slope crack area and the water seepage area appearing in the optical image, the abnormal surface deformation is used as a single parameter, and the slope crack area and the water seepage area are used as single parameters to independently determine the warning value of the monitoring point where the abnormality occurs.

9. The landslide early warning method combined with slope radar visual perception technology according to claim 8 is characterized in that: The implementation method of setting the comprehensive risk index of the monitoring point based on the standardized data of abnormal surface deformation, slope cracking area and seepage area is as follows: Standardize the surface abnormal deformation in the radar monitoring data of the monitoring point, and the quantitative characteristic data of the slope crack area and the water seepage area in the optical image; Assign weights to the abnormal surface deformation, slope cracking area and water seepage area of ​​the monitoring point, multiply the standardized characteristic data by its corresponding weight to obtain weighted characteristic values, and calculate the comprehensive risk index of the monitoring point by adding the weighted characteristic values. Set the threshold of the comprehensive risk index and divide the comprehensive risk index into different monitoring risk levels; Comprehensive risk index ; Among them, R is the comprehensive risk index, D is the normalized surface abnormal deformation, C is the normalized crack length, W is the normalized maximum crack width, S is the standardized seepage area, w 1 、w 2 、w 3 and w 4 They are the distribution weights corresponding to the abnormal surface deformation, crack length, maximum crack width and water seepage area.

Images