Deformation monitoring method of foundation pit slope based on multi-source image recognition

Abstract

The present invention relates to the technical field of foundation pit slope monitoring, and in particular, to a foundation pit slope deformation monitoring method based on multi-source image recognition. The method comprises the following steps: S1, installing an artificial target at a monitoring reference point, and using a drone to fly in five directions to shoot the monitoring reference point and the artificial target; S2, collecting multi-source image data of the area to be monitored, and pre-processing the collected multi-source image data; S3, extracting the contour information of the slope in the image from the fused image data based on the Canny edge detection algorithm; S4, analyzing the fused multi-source image data based on an optical flow model, and tracking the motion trajectory of the feature points on the surface of the slope; S5, quantifying the fused multi-source image data; S6, based on the deformation quantification result, when the monitored deformation exceeds a threshold, immediately starting an early warning mechanism. The information layer fusion algorithm after introducing the time-space fusion constraint can effectively analyze the changes in the continuous image sequence and monitor the movement and deformation of the slope.

Description

Technical Field

[0001] The invention relates to the technical field of foundation pit slope monitoring, and in particular to a foundation pit slope deformation monitoring method based on multi-source image recognition. Background Art

[0002] Foundation pit slope deformation monitoring is the process of regularly or continuously observing, recording and analyzing the changes in the morphology, structure or physical properties of the slopes around the edges of natural or artificially excavated foundation pits. It aims to identify potential signs of instability at an early stage, predict possible landslides or other geological disasters, ensure the safety of personnel, protect infrastructure and the ecological environment, and provide a scientific basis for geological disaster prevention and risk management.

[0003] For the deformation monitoring of the slopes around the foundation pit with multi-source image recognition, it is necessary to use a multi-source data fusion method to integrate data at the spatial level, that is, the complementary use of different sensor data at the same time point. However, the evolution law of slope deformation over time is lacking, and the details of the evolution of slope deformation over time cannot be accurately captured, which affects the comprehensive understanding and accurate assessment of the slope stability status. In addition, the traditional slope monitoring image processing method has limited processing capabilities for occlusion, shadows and other problems that may arise under complex terrain conditions, and may not be able to effectively identify and distinguish real slope deformation signals from interference factors. In view of this, a foundation pit slope deformation monitoring method based on multi-source image recognition is provided. Summary of the invention

[0004] The purpose of the present invention is to provide a foundation pit slope deformation monitoring method based on multi-source image recognition to solve the problems proposed in the above background technology, such as the lack of evolution law of slope deformation over time, the inability to accurately capture the details of the evolution of slope deformation over time, and the limited ability of traditional slope monitoring image processing methods to handle problems such as occlusion and shadows that may arise under complex terrain conditions, and the inability to effectively identify and distinguish real slope deformation signals from interference factors, which affects the comprehensive understanding and accurate assessment of the slope stability status.

[0005] To achieve the above object, the present invention aims to provide a method for monitoring deformation of foundation pit slope based on multi-source image recognition, comprising the following steps:

[0006] S1. Set up monitoring benchmarks in the area to be monitored, install artificial targets at the monitoring benchmarks, and use the drone to fly in five directions to shoot the monitoring benchmarks and artificial targets;

[0007] S2, collecting multi-source image data of the area to be monitored, preprocessing the collected multi-source image data, and fusing the multi-source image data using multimodal fusion technology;

[0008] Among them, the multimodal fusion technology is based on the information layer fusion algorithm and introduces time constraints to fuse the features of different modal images. The multi-source image data includes modal data of visible light image V, infrared image IR and radar image R. The information layer fusion algorithm after introducing time and space fusion constraints can better quantify the degree, speed, direction and displacement parameters of slope deformation. The introduction of time constraints can effectively analyze the changes in continuous image sequences and monitor the movement and deformation of the slope.

[0009] S3, extracting the contour information of the slope in the image from the fused image data based on the Canny edge detection algorithm, and identifying the slope boundary;

[0010] And use image feature detection technology to identify the cracks and sliding surface feature data of the slope;

[0011] S4. Analyze the fused multi-source image data based on the optical flow model, track the motion trajectory of the feature points on the slope surface, introduce spatiotemporal continuity constraints into the optical flow model, calculate the displacement changes of each part of the slope in time and space, so as to monitor the movement and deformation of the slope and identify potential deformation areas;

[0012] In order to accurately evaluate the motion state and deformation degree of the slope, the spatial-temporal continuity constraint is introduced into the optical flow model. The specific expression of the optical flow model after the spatial-temporal continuity constraint is optimized is:

[0013] ;

[0014] In the formula, Indicates that the image is The gradient of the axis; Indicates that the image is The gradient of the axis; It represents the function to be minimized by the optical flow model, reflecting the combined cost of the two criteria of image brightness consistency and smoothness; represents the integral operator, which is used to calculate the energy functional of the entire image; express spatial gradient of express spatial gradient of A trade-off parameter that represents the balance between brightness consistency and smoothness; represents brightness consistency constraint; Represents the smoothness constraint of the optical flow field; Indicates the rate of change of image grayscale over time, reflecting the change of image content over time; Indicates that the optical flow field is Component in the axial direction; Indicates that the optical flow field is Component in the axial direction;

[0015] Slope deformation is often a gradual accumulation process, which may include very small but important displacement changes. Considering the spatiotemporal continuity constraints can help capture these small displacements, thereby discovering potential unstable areas early. In addition, the slope terrain is complex, and there may be problems such as occlusion and shadows. The advanced optical flow model can utilize the spatiotemporal continuity constraints and combine the complementary advantages of multi-source image information to meet the needs of deformation monitoring in complex environments.

[0016] Solving the expression of the above optical flow model, we get the following set of equations:

[0017] ;

[0018] ;

[0019] In the formula, represents the Laplace operator, which is used to characterize the second-order derivative in space;

[0020] Solving the above equations gives an optical flow field with spatiotemporal continuity and , and then construct the motion trajectory of the characteristic points on the slope surface, and calculate the displacement changes of each part of the slope in time and space based on it, so as to monitor the movement and deformation of the slope and identify the potential deformation area;

[0021] S5, quantifying the fused multi-source image data, and calculating the deformation speed, direction, and displacement parameters of each part of the slope;

[0022] The expressions involved in quantifying multi-source image data are:

[0023] The coordinates of the feature points in two consecutive frames are and , and introduce elevation changes , then the displacement parameter is:

[0024] ;

[0025] In the formula, Represents a three-dimensional displacement vector, that is, the displacement change of a feature point between two consecutive observations, which contains the displacement information of the feature point in three-dimensional space; Indicates the elevation information of the feature point corresponding to two consecutive frames of images, that is, the height change in the vertical direction; represents the elevation change, that is ;

[0026] By introducing elevation changes , which can more accurately describe the actual displacement of the characteristic points on the slope surface in three-dimensional space. In slope monitoring, especially when there is a significant vertical displacement, relying solely on horizontal displacement is not enough to fully describe the deformation characteristics of the slope. After introducing elevation changes, the displacement changes in the up and down directions caused by landslides and landslides can be accurately captured;

[0027] Deformation direction of the slope:

[0028] ;

[0029] In the formula, Angle indicating the direction of deformation; represents the inverse tangent function;

[0030] Slope deformation speed:

[0031] The time interval between two consecutive observations is , due to the existence of noise and uneven time intervals, the Kalman filter is introduced to optimize the deformation speed. The deformation speed of the slope after optimization is:

[0032] ;

[0033] In the formula, It represents the deformation speed of the optimized slope; The Kalman filter algorithm is used to represent the function. The Kalman filter algorithm is a recursive Bayesian filter used to estimate the state of a system containing noise.

[0034] S6. Based on the deformation quantification results, combined with the geomechanical model and historical disaster data, the stability of the slope is evaluated and a safety threshold is set. Based on the threshold judgment rule, when the monitored deformation exceeds the threshold, the early warning mechanism is immediately activated and early warning information is issued.

[0035] As a further improvement of the technical solution, in S2, the specific steps involved in the multimodal fusion technology are:

[0036] S1.1, extract image features from multi-source image data of each modality;

[0037] The image features include texture features, color features, shape features and depth information, and the multi-source image data includes modal data of visible light image V, infrared image IR and radar image R;

[0038] S1.2, feature matching and temporal registration of image features from different sources to ensure that they are compared and fused in the same reference frame;

[0039] S1.3, assign fusion weights according to the importance of different image modalities;

[0040] S1.4. Use information layer fusion algorithm to fuse multi-source image data and introduce time constraints to calculate the displacement changes of the monitored area in time and space, so as to effectively monitor and locate the slight deformation and potential unstable areas of the slope.

[0041] As a further improvement of the technical solution, in S1.4, the specific expression of the information layer fusion algorithm is:

[0042] ;

[0043] In the formula, Represents the fused information layer feature vector; represents the feature vector extracted from the visible light image; represents the feature vector extracted from the infrared image; represents the feature vector extracted from the radar image; represents the weight assigned to the feature vector of the visible light image; represents the weight assigned to the infrared image feature vector; represents the weight assigned to the radar image feature vector;

[0044] In order to quantify the deformation dynamics of the slope in continuous time series images, a time-constrained optimization information layer fusion algorithm is introduced, and the optimized expression is:

[0045] ;

[0046] In the formula, Indicates at time Feature vector of the information layer after fusion; Indicates at time Feature vectors extracted from visible light images; Indicates at time Feature vectors extracted from infrared images; Indicates at time Feature vectors extracted from radar images; Indicates at time Weights assigned to the feature vectors of visible light images Indicates time The weights assigned to the infrared image feature vectors; Indicates time The weights assigned to the radar image feature vectors; represents the time series continuity function; Represents the spatial continuity function, which is used to consider the continuity of the image in space; Indicates the current frame features; Indicates the next frame features; Represents the features of the previous frame; Indicates the current frame Neighborhood pixels.

[0047] In time Feature vector of the fused information layer The feature information extracted from visible light images, infrared images and radar images at time t is integrated. The feature information includes texture features, color features, shape features and depth information, reflecting the morphological and structural characteristics of the slope under different physical characteristics of illumination conditions;

[0048] The time-constrained optimization information layer fusion algorithm is introduced to compare and analyze the image information of the past, present and future moments to identify and track subtle changes in the slope; the image information at different time points is dynamically adjusted according to its importance and reliability, thereby enhancing the sensitivity and responsiveness to the deformation characteristics of the slope that evolve over time; the continuity in the time dimension is fully utilized to improve the spatial and temporal resolution of deformation monitoring, which helps to timely discover and locate minor deformations and signs of instability of the slope and predict possible deformation development trends;

[0049] in, ;

[0050] In the formula, and Both represent weight factors;

[0051] ;

[0052] In the formula, Represents the convolution kernel weight; represents the bias term; Represents the convolution operation; Indicates the number of convolution kernels.

[0053] As a further improvement of the technical solution, in S3, the Canny edge detection algorithm is used to perform edge detection on the preprocessed image, and the calculation steps involved in the Canny edge detection algorithm are:

[0054] Gaussian filtering is performed on the preprocessed image to reduce noise and smooth the image;

[0055] ;

[0056] Adaptive Gaussian filter is introduced to optimize the above expression:

[0057] ;

[0058] In the formula, Represents the position of each pixel in the filter relative to the center pixel; represents the standard deviation of the Gaussian kernel; and Represents the pixel relative to the center pixel within the Gaussian kernel range Other pixel positions of ;

[0059] Calculate the gradient magnitude of an image and direction , get the contour information:

[0060] ;

[0061] ;

[0062] In the formula, is the gradient magnitude, which indicates the edge strength of each pixel in the image; is the direction, indicating the edge direction of each pixel; Represents the horizontal gradient image; Represents the vertical gradient image;

[0063] For different types of images, differential operators are introduced to calculate horizontal and vertical gradients;

[0064] For horizontal gradient images :

[0065] ;

[0066] For vertical gradient images :

[0067] ;

[0068] In the formula, Representing images Gradient in the horizontal direction; Representing images Gradient in the vertical direction; Represents the position of each pixel within the filter relative to the center pixel.

[0069] According to the calculated gradient amplitude and direction , which can represent the contour information of the image:

[0070] Specifically, the gradient amplitude Describes the edge strength of each pixel in the image. Higher values ​​usually correspond to edges in the image. Gradient direction Describes the edge direction of each pixel, that is, the direction of the edge in the image; through the gradient amplitude and direction The combination of can describe the contour information in the image, usually the gradient amplitude The corresponding pixel point will be considered as the edge point in the image, and the gradient direction Can be used to detect edge continuity and direction.

[0071] As a further improvement of the technical solution, the specific steps involved in the image feature detection technology are:

[0072] Crack monitoring:

[0073] Use linear Hough transform to find cracks:

[0074] ;

[0075] In the formula, Represents the coordinates of each pixel in the image; In Hough space, it represents the vertical distance from the origin to the line; In Hough space, it represents the inclination angle of a straight line; represents the Hough transform function; express is the coordinate of any pixel in the image;

[0076] Sliding surface identification:

[0077] ;

[0078] In the formula, Shown in The sliding surface area identified at the first iteration; It means that after one iteration, the enlarged sliding surface area contains more neighboring pixels similar to the original area; Represents the neighborhood pixel set around the current sliding surface area;

[0079] Characteristic parameter calculation:

[0080] The expression for crack length is:

[0081] ;

[0082] The expression for the sliding surface area is:

[0083] ;

[0084] In the formula, Indicates the crack length; represents the index among a series of discrete points at the crack edge; Indicates The horizontal coordinate of the point on the image plane; Indicates The horizontal and vertical coordinates of the points on the image plane; Represents the coordinates of pixels within the sliding surface area; Indicates the pixel area where the sliding surface is located; The coordinates of the pixels in the sliding surface area all belong to the pixel area where the sliding surface is located; represents the area of ​​the sliding surface; Indicates The horizontal coordinate of a point; Indicates The vertical coordinate of a point; Represents the number of line segments between discrete points.

[0085] As a further improvement of the technical solution, in S6, the threshold judgment rule is specifically:

[0086] ;

[0087] In the formula, Indicates the absolute value of elevation change; It indicates the elevation change threshold, which is used to measure whether the slope elevation change exceeds the safety range; Represents the modulus of the three-dimensional displacement vector; Indicates the displacement vector change threshold; Represents the absolute value of the optimized deformation speed; Indicates the threshold value of deformation speed change;

[0088] When any of the conditions is met, it means that the monitored deformation exceeds the set safety threshold, and the system will activate the early warning mechanism;

[0089] Among them, if the elevation change of the slope feature point during the continuous observation period (i.e. elevation difference) is greater than , it indicates potential instability or ongoing geological disasters, such as landslides and landslides. In slope deformation monitoring, by comparing and It can be determined whether the vertical displacement of the slope reaches the warning level. When it exceeds this threshold, the system will issue a warning signal.

[0090] Compared with the prior art, the present invention has the following beneficial effects:

[0091] 1. In the foundation pit slope deformation monitoring method based on multi-source image recognition, time constraints are introduced into the information layer fusion algorithm, and the multi-source data are deeply fused by combining time series and spatial correlation, thereby improving the accuracy and integrity of the information. The information layer fusion algorithm after introducing time and space fusion constraints can better quantify the degree, speed, direction and displacement parameters of slope deformation, and effectively analyze the changes in continuous image sequences to monitor the movement and deformation of the slope;

[0092] At the same time, Gaussian filtering preprocessing can smooth the image and reduce the impact of noise. Then the differential operator is applied to make the calculated gradient more representative of the real image edge. For different types of images, differential operators are introduced to calculate the horizontal and vertical gradients, which can reduce the impact of noise while extracting useful edge information and accurately determine the edge position and direction in the image.

[0093] 2. In the foundation pit slope deformation monitoring method based on multi-source image recognition, by introducing spatiotemporal continuity constraints in the optical flow model, the optimized optical flow model can accurately evaluate the motion state and deformation degree of the slope. For the complex slope terrain, there may be problems such as occlusion and shadow. The optical flow model can utilize spatiotemporal continuity constraints and combine the complementary advantages of multi-source image information to adapt to the deformation monitoring needs in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0094] Figure 1 is a flow chart of the overall method of the present invention;

[0095] Figure 2 It is a schematic diagram of the on-site implementation of the slope of the present invention. DETAILED DESCRIPTION

[0096] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] Example:

[0098] See also Figure 1-2 As shown, this embodiment provides a foundation pit slope deformation monitoring method based on multi-source image recognition, comprising the following steps:

[0099] S1. Set up monitoring benchmarks in the area to be monitored, install artificial targets at the monitoring benchmarks, and use the drone to fly in five directions to shoot the monitoring benchmarks and artificial targets;

[0100] Specifically, the setting of monitoring benchmarks provides a unified spatial reference frame for the entire monitoring system, ensuring accurate determination and comparison of target positions during subsequent image data acquisition, processing, and analysis;

[0101] Moreover, installing artificial targets can not only provide a clear visual focus for the five-way flight shooting of the UAV and improve the image quality, but also the artificial targets are easy to identify in the image, which is helpful for the registration, correction and feature extraction in the later image processing; by shooting the benchmarks and artificial targets from different angles (five-way flight) by the UAV, it is possible to obtain multi-perspective and multi-level information of the slope, and increase the three-dimensionality and integrity of the data. This multi-perspective data helps to more comprehensively capture the details of the slope structure and potential deformation characteristics, and provide richer original materials for subsequent multi-source data fusion and analysis;

[0102] S2. Collect multi-source image data of the area to be monitored, pre-process the collected multi-source image data, and fuse the multi-source image data using multi-modal fusion technology; multi-source image data pre-processing includes image denoising, image correction, image enhancement, image cropping and resampling, radiation correction and data format conversion; after the pre-processing steps, the multi-source image data can be effectively fused together to extract more valuable comprehensive information for slope deformation monitoring;

[0103] Among them, the multimodal fusion technology is based on the information layer fusion algorithm and introduces time constraints to fuse the features of different modal images. The information layer fusion algorithm after introducing time and space fusion constraints can better quantify the degree, speed, direction and displacement parameters of slope deformation, and effectively analyze the changes in continuous image sequences to monitor the movement and deformation of the slope.

[0104] In this embodiment, the specific steps involved in the multimodal fusion technology are:

[0105] S1.1, extract image features from multi-source image data of each modality;

[0106] Among them, image features include texture features, color features, shape features and depth information, and multi-source image data include modal data of visible light image V, infrared image IR and radar image R. Among them, the shooting of reference points and artificial targets by UAV five-direction flight is part of multi-source image data acquisition. The purpose of using UAV five-direction flight to shoot monitoring reference points and artificial targets is to obtain high-quality, multi-perspective specific target image data, so as to achieve complementarity, coordination and integration with multi-source image data of the monitored area, and ensure accurate spatial positioning, reliable monitoring accuracy, accurate feature recognition and sensitive change detection.

[0107] S1.2, feature matching and temporal registration of image features from different sources to ensure that they are compared and fused in the same reference frame;

[0108] Extract key points or feature points (such as SIFT, SURF, ORB and other feature descriptors) from images of different modalities, and then use matching algorithms (such as BFMatcher, FLANN matcher) to find corresponding feature point pairs between images of different modalities to achieve feature matching;

[0109] For continuous time series images, the time offset of each frame of image relative to a reference frame is estimated through inter-frame difference, optical flow method or other time series analysis techniques to achieve time registration; synchronize image data of two images from different sources after feature matching and time registration;

[0110] S1.3, assign fusion weights according to the importance of different image modalities;

[0111] S1.4. Use information layer fusion algorithm to fuse multi-source image data and introduce time constraints to calculate the displacement changes of the monitored area in time and space, so as to effectively monitor and locate the slight deformation and potential unstable areas of the slope;

[0112] Specifically, the specific expression of the information layer fusion algorithm is:

[0113] ;

[0114] In the formula, Represents the fused information layer feature vector; represents the feature vector extracted from the visible light image; represents the feature vector extracted from the infrared image; represents the feature vector extracted from the radar image; represents the weight assigned to the feature vector of the visible light image; represents the weight assigned to the infrared image feature vector; represents the weight assigned to the radar image feature vector;

[0115] In order to quantify the deformation dynamics of the slope in continuous time series images, a time-constrained optimization information layer fusion algorithm is introduced, and the optimized expression is:

[0116] ;

[0117] In the formula, Indicates at time Feature vector of the information layer after fusion; Indicates at time Feature vectors extracted from visible light images; Indicates at time Feature vectors extracted from infrared images; Indicates at time Feature vectors extracted from radar images; Indicates at time Weights assigned to the feature vectors of visible light images Indicates time The weights assigned to the infrared image feature vectors; Indicates time The weights assigned to the radar image feature vectors; represents the time series continuity function; Represents the spatial continuity function, which is used to consider the continuity of the image in space; Indicates the current frame features; Indicates the next frame features; Represents the features of the previous frame; Indicates the current frame Neighborhood pixels.

[0118] In time Feature vector of the fused information layer The feature information extracted from visible light images, infrared images and radar images at time t is integrated. The feature information includes texture features, color features, shape features and depth information, reflecting the morphological and structural characteristics of the slope under different physical characteristics of illumination conditions;

[0119] The time-constrained optimization information layer fusion algorithm is introduced to compare and analyze the image information of the past, present and future moments to identify and track subtle changes in the slope; the image information at different time points is dynamically adjusted according to its importance and reliability, thereby enhancing the sensitivity and responsiveness to the deformation characteristics of the slope that evolve over time; the continuity in the time dimension is fully utilized to improve the spatial and temporal resolution of deformation monitoring, which helps to timely discover and locate minor deformations and signs of instability of the slope and predict possible deformation development trends;

[0120] in, ;

[0121] In the formula, and Both represent weight factors;

[0122] ;

[0123] In the formula, Represents the convolution kernel weight; represents the bias term; Represents the convolution operation; Indicates the number of convolution kernels;

[0124] S3, extracting the contour information of the slope in the image from the fused image data based on the Canny edge detection algorithm, and identifying the slope boundary;

[0125] Image feature detection technology is used to identify the cracks and sliding surface feature data of the slope; the cracks and sliding surface feature data are closely related to the deformation activity of the slope;

[0126] Specifically, the Canny edge detection algorithm is used to perform edge detection on the preprocessed image, and the calculation steps involved in the Canny edge detection algorithm are:

[0127] Gaussian filtering is performed on the preprocessed image to reduce noise and smooth the image.

[0128] ;

[0129] Since different types of images have different noise characteristics, different Gaussian filter standard deviations are set according to the noise characteristics of each type of image. , for each image patch, calculate an appropriate local standard deviation , using adaptive standard deviation Perform Gaussian filtering on the image block by block, and for each position , using the corresponding adaptive standard deviation Calculate the Gaussian kernel weights and optimize the above expression by introducing an adaptive Gaussian filter:

[0130] ;

[0131] In the formula, Represents the position of each pixel in the filter relative to the center pixel; represents the standard deviation of the Gaussian kernel; and Represents the pixel relative to the center pixel within the Gaussian kernel range Other pixel positions of ;

[0132] The adaptive Gaussian filter can ensure that different areas of the image are smoothed to different degrees while maintaining edge clarity, further improving the quality of edge detection; different Gaussian filter standard deviations are set according to the noise characteristics of each type of image , images can be analyzed at different scales to obtain edge features of different sizes and thicknesses;

[0133] Calculate the gradient magnitude of an image and direction , get the contour information:

[0134] ;

[0135] ;

[0136] In the formula, is the gradient magnitude, which indicates the edge strength of each pixel in the image; is the direction, indicating the edge direction of each pixel; Represents the horizontal gradient image; Represents the vertical gradient image;

[0137] Differentiated operators are introduced to calculate horizontal and vertical gradients for different types of images. Differentiated operators can effectively detect edges and places with dramatic changes in images. The grayscale or color intensity of an image will suddenly change at the edge. By performing differential operations on the image, this change is quantified, and the position of the edge is determined by finding the maximum value of the gradient. By using differential operators to calculate the gradient of the smoothed image, useful edge information can be extracted while reducing the influence of noise.

[0138] For horizontal gradient images :

[0139] ;

[0140] For vertical gradient images :

[0141] ;

[0142] In the formula, Representing images Gradient in the horizontal direction; Representing images Gradient in the vertical direction; Represents the position of each pixel within the filter relative to the center pixel.

[0143] According to the calculated gradient amplitude and direction , which can represent the contour information of the image:

[0144] Specifically, the gradient amplitude Describes the edge strength of each pixel in the image. Higher values ​​usually correspond to edges in the image. Gradient direction Describes the edge direction of each pixel, that is, the direction of the edge in the image; through the gradient amplitude and direction The combination of can describe the contour information in the image, usually the gradient amplitude The corresponding pixel point will be considered as the edge point in the image, and the gradient direction Can be used to detect edge continuity and direction.

[0145] In this embodiment, the specific steps involved in the image feature detection technology are:

[0146] Crack monitoring:

[0147] Use linear Hough transform to find cracks:

[0148] ;

[0149] In the formula, Represents the coordinates of each pixel in the image; In Hough space, it represents the vertical distance from the origin to the line; In Hough space, it represents the inclination angle of a straight line; represents the Hough transform function; express is the coordinate of any pixel in the image;

[0150] Since slope cracks usually appear as linear features in images, Hough transform can effectively detect straight line features from images and can stably detect cracks even in the presence of noise, fractures or partial occlusions.

[0151] Sliding surface identification:

[0152] ;

[0153] In the formula, Shown in The sliding surface area identified at the first iteration; It means that after one iteration, the enlarged sliding surface area contains more neighboring pixels similar to the original area; Represents the neighborhood pixel set around the current sliding surface area;

[0154] Characteristic parameter calculation:

[0155] The expression for crack length is:

[0156] ;

[0157] The expression for the sliding surface area is:

[0158] ;

[0159] In the formula, Indicates the crack length; represents the index among a series of discrete points at the crack edge; Indicates The horizontal coordinate of the point on the image plane; Indicates The horizontal and vertical coordinates of the points on the image plane; Represents the coordinates of pixels within the sliding surface area; Indicates the pixel area where the sliding surface is located; The coordinates of the pixels in the sliding surface area all belong to the pixel area where the sliding surface is located; represents the area of ​​the sliding surface; Indicates The horizontal coordinate of a point; Indicates The vertical coordinate of a point; Indicates the number of line segments between discrete points;

[0160] S4. Analyze the fused multi-source image data based on the optical flow model, track the motion trajectory of the feature points on the slope surface, introduce spatiotemporal continuity constraints into the optical flow model, calculate the displacement changes of each part of the slope in time and space, so as to monitor the movement and deformation of the slope and identify potential deformation areas;

[0161] In this embodiment, in order to accurately evaluate the motion state and deformation degree of the slope, the spatiotemporal continuity constraint is introduced into the optical flow model. The specific expression of the optical flow model after the spatiotemporal continuity constraint is introduced is:

[0162] ;

[0163] In the formula, Indicates that the image is The gradient of the axis; Indicates that the image is The gradient of the axis; It represents the function to be minimized by the optical flow model, reflecting the combined cost of the two criteria of image brightness consistency and smoothness; represents the integral operator, which is used to calculate the energy functional of the entire image; express spatial gradient of express spatial gradient of A trade-off parameter that represents the balance between brightness consistency and smoothness; represents brightness consistency constraint; Represents the smoothness constraint of the optical flow field; Indicates the rate of change of image grayscale over time, reflecting the change of image content over time; Indicates that the optical flow field is Component in the axial direction; Indicates that the optical flow field is Component in the axial direction;

[0164] Slope deformation is often a gradual accumulation process, which may include very small but important displacement changes. Considering the spatiotemporal continuity constraint can help capture these small displacements, thereby discovering potential unstable areas early. In addition, the slope terrain is complex, and there may be problems such as occlusion and shadow. The optical flow model can utilize the spatiotemporal continuity constraint and combine the complementary advantages of multi-source image information to adapt to the deformation monitoring needs in complex environments.

[0165] Solving the expression of the above optical flow model, we get the following set of equations:

[0166] ;

[0167] ;

[0168] In the formula, represents the Laplace operator, which is used to characterize the second-order derivative in space;

[0169] Solving the above equations gives an optical flow field with spatiotemporal continuity and , and then construct the motion trajectory of the characteristic points on the slope surface, and calculate the displacement changes of each part of the slope in time and space, including parameters such as displacement amount, displacement speed, and displacement direction, to monitor the movement and deformation of the slope, and identify potential deformation areas;

[0170] Displacement :

[0171] ;

[0172] Displacement speed :

[0173] ;

[0174] In the formula, Indicates a time interval;

[0175] Displacement direction :

[0176] ;

[0177] Specifically, for the slope surface A feature point of which the motion between two or more consecutive frames is represented by the corresponding Accurately depict the value, optical flow field and Together they form a two-dimensional vector field, which intuitively reflects the instantaneous motion state of the slope surface feature points on the image plane;

[0178] By calculating the optical flow vector corresponding to the same feature point in a series of continuous images By accumulating, the motion trajectory of the feature point in the time and space domain can be constructed; this trajectory reflects the displacement evolution of the feature points on the slope surface over time, and is the key basis for analyzing the movement and deformation of the slope; by analyzing the motion trajectories of a large number of feature points, the displacement mode, speed, direction and other characteristics of different parts of the slope can be identified, thereby revealing the overall deformation trend of the slope and the local unstable area;

[0179] S5, quantifying the fused multi-source image data, and calculating the deformation speed, direction, and displacement parameters of each part of the slope;

[0180] In this embodiment, the expression involved in quantizing multi-source image data is:

[0181] The coordinates of the feature points in two consecutive frames are and , and introduce elevation changes , then the displacement parameter is:

[0182] ;

[0183] In the formula, Represents a three-dimensional displacement vector, that is, the displacement change of a feature point between two consecutive observations, which contains the displacement information of the feature point in three-dimensional space; Indicates the elevation information of the feature point corresponding to two consecutive frames of images, that is, the height change in the vertical direction; represents the elevation change, that is ;

[0184] By introducing elevation changes , which can more accurately describe the actual displacement of the characteristic points on the slope surface in three-dimensional space. In slope monitoring, especially when there is a significant vertical displacement, relying solely on horizontal displacement is not enough to fully describe the deformation characteristics of the slope. After introducing elevation changes, the displacement changes in the up and down directions caused by landslides and landslides can be accurately captured;

[0185] Deformation direction of the slope:

[0186] ;

[0187] In the formula, Angle indicating the direction of deformation; represents the inverse tangent function;

[0188] Slope deformation speed:

[0189] The time interval between two consecutive observations is , due to the existence of noise and uneven time intervals, the Kalman filter is introduced to optimize the deformation speed. The deformation speed of the slope after optimization is:

[0190] ;

[0191] In the formula, It represents the deformation speed of the optimized slope; The Kalman filter algorithm is used to represent the function. The Kalman filter algorithm is a recursive Bayesian filter used to estimate the state of a system containing noise.

[0192] S6. Based on the deformation quantification results, combined with the geomechanical model and historical disaster data, the stability of the slope is evaluated and a safety threshold is set. Based on the threshold judgment rule, when the monitored deformation exceeds the threshold, the early warning mechanism is immediately activated and early warning information is issued.

[0193] In this embodiment, the threshold determination rule is specifically:

[0194] ;

[0195] In the formula, Indicates the absolute value of elevation change; It indicates the elevation change threshold, which is used to measure whether the slope elevation change exceeds the safety range; Represents the modulus of the three-dimensional displacement vector; Indicates the displacement vector change threshold; Represents the absolute value of the optimized deformation speed; Indicates the threshold value of deformation speed change;

[0196] When any of the conditions is met, it means that the monitored deformation exceeds the set safety threshold, and the system will activate the early warning mechanism;

[0197] Among them, if the elevation change of the slope feature point during the continuous observation period (i.e. elevation difference) is greater than , it indicates potential instability or ongoing geological disasters, such as landslides and landslides. In slope deformation monitoring, by comparing and It can be determined whether the vertical displacement of the slope reaches the warning level. When it exceeds this threshold, the system will issue a warning signal.

[0198] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A method for monitoring deformation of foundation pit slope based on multi-source image recognition, characterized in that: The following steps are involved: S1. Set up monitoring benchmarks in the area to be monitored, install artificial targets at the monitoring benchmarks, and use the drone to fly in five directions to shoot the monitoring benchmarks and artificial targets; S2, collecting multi-source image data of the area to be monitored, preprocessing the collected multi-source image data, and fusing the multi-source image data using multimodal fusion technology; Among them, the multimodal fusion technology is based on the information layer fusion algorithm and introduces time constraints to fuse the features of different modal images, and the multi-source image data includes the modal data of visible light image V, infrared image IR and radar image R; S3, extracting the contour information of the slope in the image from the fused image data based on the Canny edge detection algorithm, and identifying the slope boundary; And use image feature detection technology to identify the cracks and sliding surface feature data of the slope; S4. Analyze the fused multi-source image data based on the optical flow model, track the motion trajectory of the feature points on the slope surface, introduce spatiotemporal continuity constraints into the optical flow model, calculate the displacement changes of each part of the slope in time and space, so as to monitor the movement and deformation of the slope and identify potential deformation areas; The spatial-temporal continuity constraint is introduced into the optical flow model. The specific expression of the optical flow model after the spatial-temporal continuity constraint is introduced is: ； In the formula, Indicates that the image is The gradient of the axis; Indicates that the image is The gradient of the axis; Represents the function to be minimized by the optical flow model; represents the integral operator; express spatial gradient of express spatial gradient of A trade-off parameter that represents the balance between brightness consistency and smoothness; represents brightness consistency constraint; Represents the smoothness constraint of the optical flow field; Indicates the rate of change of image grayscale over time; Indicates that the optical flow field is Component in the axial direction; Indicates that the optical flow field is Component in the axial direction; Solving the expression of the above optical flow model, we get the following set of equations: ； ； In the formula, represents the Laplace operator; S5, quantifying the fused multi-source image data, and calculating the deformation speed, direction, and displacement parameters of each part of the slope; The expressions involved in quantifying multi-source image data are: The coordinates of the feature points in two consecutive frames are and , and introduce elevation changes , then the displacement parameter is: ； In the formula, represents a three-dimensional displacement vector; Indicates the elevation information corresponding to the feature point in two consecutive frames of images; Indicates the amount of elevation change; Deformation direction of the slope: ； In the formula, Angle indicating the direction of deformation; represents the inverse tangent function; Slope deformation speed: The time interval between two consecutive observations is , the Kalman filter is introduced to optimize the deformation speed, and the deformation speed of the slope after optimization is: ； In the formula, It represents the deformation speed of the optimized slope; Represents a function; S6. Based on the deformation quantification results, a safety threshold is set. Based on the threshold judgment rule, when the monitored deformation exceeds the threshold, the early warning mechanism is immediately activated and early warning information is issued.

2. The method for monitoring deformation of foundation pit slope based on multi-source image recognition according to claim 1 is characterized in that: In S2, the specific steps involved in the multimodal fusion technology are: S1.1, extract image features from multi-source image data of each modality; The image features include texture features, color features, shape features and depth information, and the multi-source image data includes modal data of visible light image V, infrared image IR and radar image R; S1.2, feature matching and time registration of image features from different sources; S1.3, assign fusion weights according to the importance of different image modalities; S1.

4. Use information layer fusion algorithm to fuse multi-source image data and introduce time constraints to calculate the displacement changes of the monitored area in time and space.

3. The method for monitoring deformation of foundation pit slope based on multi-source image recognition according to claim 2 is characterized in that: In S1.4, the specific expression of the information layer fusion algorithm is: ； In the formula, Represents the fused information layer feature vector; represents the feature vector extracted from the visible light image; represents the feature vector extracted from the infrared image; represents the feature vector extracted from the radar image; represents the weight assigned to the feature vector of the visible light image; represents the weight assigned to the infrared image feature vector; represents the weight assigned to the radar image feature vector; The time constraint optimization information layer fusion algorithm is introduced, and the optimized expression is: ； In the formula, Indicates at time Feature vector of the fused information layer; Indicates at time Feature vectors extracted from visible light images; Indicates at time Feature vectors extracted from infrared images; Indicates at time Feature vectors extracted from radar images; Indicates time The weights assigned to the feature vectors of the visible light image; Indicates time The weights assigned to the infrared image feature vectors; Indicates time The weights assigned to the radar image feature vectors; represents a time series continuity function; Represents the spatial continuity function, which is used to consider the continuity of the image in space; Indicates the current frame features; Indicates the next frame features; Represents the features of the previous frame; Indicates the current frame Neighborhood pixels.

4. The method for monitoring deformation of foundation pit slope based on multi-source image recognition according to claim 1 is characterized in that: In S3, the Canny edge detection algorithm is used to perform edge detection on the preprocessed image, and the calculation steps involved in the Canny edge detection algorithm are: Perform Gaussian filtering on the preprocessed image: ； Adaptive Gaussian filter is introduced to optimize the above expression: ； In the formula, Represents the position of each pixel in the filter relative to the center pixel; represents the standard deviation of the Gaussian kernel; and Represents the pixel relative to the center pixel within the Gaussian kernel range Other pixel positions of ; Calculate the gradient magnitude of an image and direction , get the contour information: ； ； In the formula, is the gradient amplitude; For direction; Represents the horizontal gradient image; Represents the vertical gradient image; For different types of images, differential operators are introduced to calculate horizontal and vertical gradients; For horizontal gradient images : ； For vertical gradient images : ； In the formula, Representing images Gradient in the horizontal direction; Representing images Gradient in the vertical direction; Represents the position of each pixel within the filter relative to the center pixel.

5. The method for monitoring deformation of foundation pit slope based on multi-source image recognition according to claim 3 is characterized in that: The specific steps involved in the image feature detection technology are: Crack monitoring: Use linear Hough transform to find cracks: ； In the formula, Represents the coordinates of each pixel in the image; In Hough space, it represents the vertical distance from the origin to the line; In Hough space, it represents the inclination angle of a straight line; represents the Hough transform function; express is the coordinate of any pixel in the image; Sliding surface identification: ； In the formula, Indicated in The sliding surface area identified at the first iteration; It indicates the enlarged sliding surface area after one iteration; Represents the neighborhood pixel set around the current sliding surface area; Characteristic parameter calculation: The expression for crack length is: ； The expression for the sliding surface area is: ； In the formula, Indicates the crack length; represents the index among a series of discrete points at the crack edge; Indicates The horizontal coordinate of the point on the image plane; Indicates The horizontal and vertical coordinates of the points on the image plane; Represents the coordinates of pixels within the sliding surface area; Indicates the pixel area where the sliding surface is located; represents the area of ​​the sliding surface; Indicates The horizontal coordinate of a point; Indicates The vertical coordinate of a point; Represents the number of line segments between discrete points.

6. The method for monitoring deformation of foundation pit slope based on multi-source image recognition according to claim 1, characterized in that: In S6, the threshold determination rule is specifically: ； In the formula, Indicates the absolute value of elevation change; Indicates the elevation change threshold; Represents the modulus of the three-dimensional displacement vector; Indicates the displacement vector change threshold; Represents the absolute value of the optimized deformation speed; Indicates the deformation speed change threshold.

Images