A lightweight rock-soil mass deformation and movement monitoring system and method

Abstract

A kind of lightweight rock-soil mass deformation and motion monitoring system and method belong to geological disaster monitoring technical field.The technical scheme: image acquisition module acquires the image sequence of target area;Data storage and transmission module is used for image buffering and remote transmission;Data processing and analysis module includes point tracking unit, depth calculation and correction unit, coordinate conversion and displacement calculation unit, respectively for tracking the motion trajectory of pixel in image, calculate and correct the depth information of monitoring object, and convert depth into three-dimensional coordinate and calculate real displacement;Early warning and display module realizes the visualization of data and multi-stage alarm.Affirmative effect: the present application has the advantages of lightweight, low cost, convenient deployment etc., can realize the quantitative monitoring of rock-soil mass real displacement and motion, has good robustness and real-time performance, adapts to complex environment such as illumination change, camera slight shaking etc., can also resist the interference of rain, fog weather, shadow etc. to image features to a certain extent.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of geological disaster monitoring, and specifically relates to a rock-soil mass deformation and displacement monitoring system and method based on image point tracking and depth calculation, which is suitable for rock-soil mass stability monitoring and evaluation, geological disaster early warning, and engineering safety management, etc. BACKGROUND

[0002] Rock-soil mass deformation and movement displacement monitoring is an important link in geological disaster prevention and engineering safety management, and is widely used in geological, water conservancy, hydropower, transportation, mining and other engineering fields. Existing rock-soil mass monitoring methods mainly include two categories: contact type and non-contact type.

[0003] Although the contact type monitoring method (such as inclinometer, strain gauge, GPS displacement meter, etc.) has high precision, it generally has problems such as high cost, complex construction, limited point density, poor flexibility of monitoring points, etc. This method usually requires a large number of sensors and power supply communication lines, which is difficult to meet the needs of large-scale, long-term and continuous field monitoring. At the same time, once the deployment is completed, it is difficult to adjust the monitoring points in the later stage, and it is also difficult to actively find potential high-risk areas, which has poor adaptability and expansibility.

[0004] The non-contact type monitoring method (such as total station measurement, LiDAR laser scanning, InSAR radar interferometric measurement and unmanned aerial vehicle oblique photography, etc.) has remote monitoring and dense sampling capability, and can realize a certain degree of spatial wide coverage. However, such systems have high equipment cost and complex data processing, and are usually periodic sampling, which can only realize periodic collection but not real-time dynamic perception, making it difficult to meet the actual engineering needs of rapid deployment and long-term normalization monitoring.

[0005] With the development of machine vision and image processing technology, visual monitoring methods for rock-soil mass surface deformation or overall movement have gradually emerged. This kind of technology often uses visual systems to detect the displacement of targets on the surface of the monitoring object to infer the deformation or overall movement of the rock-soil mass. However, this technology still has the following key bottlenecks:

[0006] (1) Unable to quantitatively restore the real spatial displacement: Most methods can only obtain the displacement of the pixels in the target, lack depth information support, and are difficult to accurately reconstruct the real displacement in three-dimensional space;

[0007] (2) Limited target deployment: The terrain of some rock-soil mass areas is complex and the surface is loose, making it difficult to stably install the target, limiting the deployment density, and the displacement of the target does not always fully reflect the actual deformation of the rock-soil mass;

[0008] (3) Poor durability of target: affected by environmental factors such as weathering, rain, construction, etc., the target is prone to aging, displacement or falling off, making it difficult to achieve long-term continuous monitoring;

[0009] (4) System complexity, high cost: the existing system depends on high-precision optical or laser equipment, algorithm calculation is complex, software and hardware integration is high, the overall cost is high;

[0010] (5) Weak response ability, insufficient flexibility: the monitoring system is difficult to adjust quickly after deployment, cannot adapt to the monitoring needs of dynamic changes in risk areas, and is difficult to achieve early active identification and early warning.

[0011] Therefore, it is urgent to build a lightweight, low-cost, easy-to-deploy, dense monitoring point, real three-dimensional space displacement measurement capability, strong robustness and real-time geotechnical deformation and movement monitoring system and method. Based on this demand, the present application proposes an innovative monitoring system combining image point tracking algorithm and depth calculation system, which realizes continuous, efficient and intelligent perception of the deformation state of geotechnical body in complex environment, and provides solid technical support for geological disaster warning and engineering safety monitoring by building a widely adaptable and generalizable monitoring system and method. SUMMARY

[0012] In order to solve the technical problems existing in the prior art, the present application aims to overcome the limitations of existing geotechnical deformation and movement monitoring technology in deployment cost, monitoring accuracy, adaptability and real-time performance, and provides a lightweight, low-cost, high-precision, three-dimensional displacement calculation capable geotechnical deformation and movement monitoring system and method to realize continuous, stable and automatic perception and early warning of the deformation behavior of geotechnical body in geological disaster risk area and engineering site.

[0013] To achieve the above purpose, the present application provides the following technical solutions:

[0014] A lightweight geotechnical deformation and movement monitoring system, comprising:

[0015] An image acquisition module arranged in a stable reference area for acquiring image sequences or video frames of the monitoring area;

[0016] A data storage and transmission module for local caching and remote transmission of image data;

[0017] A data processing and analysis module, comprising:

[0018] A point tracking unit for tracking the pixel displacement of feature points in the image sequence;

[0019] A depth calculation and correction unit for generating and correcting the depth information of the pixel points in the image;

[0020] A coordinate conversion and displacement calculation unit for converting the pixel coordinates and depth information into three-dimensional coordinates and calculating the real displacement;

[0021] Early warning and display module for visual display and multi-level early warning of displacement data.

[0022] Further, the image acquisition module includes a camera device equipped with a controllable gimbal and an electric zoom lens, supporting adaptive cruise and local magnification monitoring.

[0023] Further, the point tracking unit uses optical flow method, feature matching, deep learning tracking method or their combination to realize pixel-level or sub-pixel-level displacement tracking, and has the ability of occlusion discrimination and motion consistency analysis.

[0024] Further, the depth calculation and correction unit generates a depth map using monocular, binocular, active light source, laser radar, millimeter wave radar or multi-source fusion method, and combines laser point cloud, radar data or known reference depth for depth correction.

[0025] Further, the coordinate conversion and displacement calculation unit converts pixel coordinates and depth values into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters and distortion parameters, and calculates the real displacement vector by comparing three-dimensional coordinates at different time points.

[0026] Further, the early warning and display module generates displacement-time curve, displacement vector field, deformation cloud chart or three-dimensional dynamic model, and triggers multi-level early warning based on preset threshold or intelligent judgment algorithm.

[0027] The present application also includes a lightweight rock-soil deformation and motion monitoring method, the steps are as follows:

[0028] S1: System layout and calibration, install camera equipment and complete camera parameter calibration;

[0029] S2: Image data acquisition, obtain image sequence of the monitoring area;

[0030] S3: Feature point tracking and pixel displacement calculation, track feature points and obtain pixel displacement vector;

[0031] S4: Depth calculation and correction, generate and correct depth map;

[0032] S5: Three-dimensional displacement vector calculation, combine pixel coordinates and depth information to calculate three-dimensional real displacement;

[0033] S6: Data analysis and early warning, statistical analysis and visualization of displacement data, and triggering early warning.

[0034] Further, in step S4, the depth correction includes:

[0035] Obtain high-precision depth reference through laser or radar data;

[0036] Scale and bias calibration is performed using a RANSAC fitting method;

[0037] An interpolation method with image-space joint weight is used to propagate and fuse the depth residual, to generate the corrected depth map.

[0038] Further, in step S2, the image acquisition supports edge computing and dynamic uploading mechanism, and the sampling frequency is adaptively adjusted according to the risk level of the monitoring area, and high-precision local cruise monitoring is started when an anomaly is identified.

[0039] Further, in step S6, the warning information is published through SMS, email or platform push, and the warning threshold includes multiple thresholds of prompt level, alarm level and emergency level.

[0040] The beneficial effects of the present application are:

[0041] Compared with the prior art, the lightweight rock-soil mass deformation and motion monitoring system and method has the following technical characteristics and beneficial effects:

[0042] The present application collects continuous images on the surface of the rock-soil mass, combines pixel point cross-frame tracking and depth calculation algorithm, realizes direct quantization of feature point space displacement, and overcomes the limitation that the existing image monitoring method can only obtain pixel-level relative displacement. Compared with the traditional method, the present application has the following obvious advantages:

[0043] (1) Non-contact, continuous dynamic monitoring: without direct contact with the rock-soil mass, large-scale, continuous and dynamic deformation and motion monitoring can be realized, avoiding interference with the structure of the rock-soil mass.

[0044] (2) Low cost, convenient to deploy: visual imaging or multi-modal sensing devices can be used for data acquisition, and the hardware type is not limited, and the function is realized as continuous or timed acquisition of monitoring target image sequence.

[0045] (3) High safety: the monitoring equipment can be arranged in stable bedrock or away from high-risk areas to avoid exposing personnel to potential dangers.

[0046] (4) Dense monitoring points, flexible and expandable: a single camera device can monitor thousands of pixels simultaneously, and the number of monitoring points can be increased or decreased at any time according to actual conditions, especially for high-risk areas, the monitoring density can be dynamically increased to realize large-scale risk survey and key prevention and control.

[0047] (5) High resource utilization, flexible data transmission: by introducing an edge computing unit, preliminary processing and intelligent filtering are performed at the data acquisition end, and high-risk data is uploaded preferentially. This greatly reduces the data transmission and storage load, and improves the response speed to dangerous situations, achieving an optimal balance between monitoring efficiency and cost-effectiveness.

[0048] (6) Strong real-time performance and efficient data processing: The monitoring system supports near-real-time or real-time output of displacement results, enabling rapid response to abnormal deformation of rock-soil mass and supporting rapid decision-making on the construction site.

[0049] (7) High compatibility of algorithms and sensors: Point tracking, depth calculation, and correction algorithms support multiple implementation methods, including traditional image processing methods, deep learning, or multi-sensor fusion strategies. The system can combine monocular / dual-camera, LiDAR, millimeter-wave radar, and other multi-source data to improve monitoring accuracy and stability.

[0050] (8) Strong environmental adaptability: It can handle complex environments such as light changes, shadows, rain and fog, and slight camera shaking, while maintaining monitoring accuracy and robustness.

[0051] (9) High value of engineering application and promotion: It can be widely applied to slope, foundation pit, tunnel, dam, underground space, and other types of geotechnical engineering scenes, and has potential for secondary development, providing reliable technical support for geological disaster warning, engineering safety management, and long-term monitoring.

[0052] In summary, the present application provides a lightweight, low-cost, convenient, flexible, robust, and high-density real-time monitoring system and method for rock-soil mass deformation and movement monitoring, significantly improving the applicability and promotion value of existing rock-soil mass monitoring technology in practical engineering applications.

[0053] The present application not only covers system structure and functional flow, but also covers all software and hardware combination methods and algorithm paths that can achieve equivalent functions, ensuring that the scope of patent protection is not limited by specific implementation methods. BRIEF DESCRIPTION OF DRAWINGS

[0054] To more clearly illustrate the technical solutions of the embodiments of the present application, the present application will be described in detail below with reference to the drawings and detailed embodiments. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0055] Figure 1 is the structure flowchart of the present application;

[0056] Figure 2 is the laser-camera joint calibration diagram of embodiment 1 of the present application;

[0057] Figure 3 is the coordinate transformation schematic diagram of embodiment 1 of the present application;

[0058] Figure 4 is the laser-pixel matching schematic diagram of embodiment 1 of the present application;

[0059] Figure 5 Matching point pair RANSAC fitting case graph for embodiment 1 of the present application;

[0060] Figure 6 Monitoring point continuous motion monitoring trajectory graph for embodiment 1 of the present application;

[0061] Figure 2 Reference signs: 1-chessboard calibration plate, 2-tripod, 3-three-dimensional laser scanner, 4-camera device. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical scheme and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application. The present application will be described below in combination with the drawings and examples. Figures 1-6 The lightweight geotechnical deformation and motion monitoring system and method are further described.

[0063] Embodiment 1

[0064] The present application is suitable for continuous monitoring of geotechnical deformation and motion trajectory in scenes such as slope and underground space, for judging the stability of geotechnical body, and the specific scheme is as follows:

[0065] I. System hardware layout

[0066] In this embodiment, the rock along the highway is taken as the monitoring object.

[0067] Image acquisition: In this embodiment, an 800 million pixel (4K resolution) gun-ball integrated high-definition network camera is selected as the camera device. The gun-shaped camera is used for global monitoring, and the spherical camera is used for intelligent cruising and local key monitoring. The camera is equipped with a rain and dustproof shell, zoom and remote controllable gimbal. It is installed on a stable bedrock on the opposite mountain of the slope, fixed by expansion bolts, to ensure that its field of view can completely cover the entire slope area to be monitored. The camera is connected to the nearby power supply system and wireless data transmission module through power line and network cable. The camera device collects image sequences in a dynamic adaptive acquisition mode.

[0068] Data processing: In this embodiment, a cloud server is used as the data processing center for running the core algorithm of the system, including image point tracking, depth calculation, three-dimensional displacement vector calculation and early warning analysis. Through centralized calculation and management, the powerful computing power of the cloud can be fully utilized to support accurate calculation of large-scale and high-density monitoring points, thereby improving the monitoring accuracy and system response capability.

[0069] Further, the present application introduces edge computing and dynamic data uploading mechanism:

[0070] At the edge of the monitoring site, preliminary calculations can be performed on some representative monitoring points without uploading complete images to the cloud, achieving lightweight processing.

[0071] The system can determine the stability of the rock-soil mass according to the edge computing results: when the monitoring indicators show an increased risk or abnormal deformation, automatically start high-frequency data upload, and the cloud server performs comprehensive and high-precision monitoring point calculation and analysis.

[0072] Combined with the adaptive sampling strategy, the sampling frequency is increased in high-risk monitoring areas and reduced in low-risk areas, thereby balancing monitoring accuracy, real-time response capability, and data transmission cost.

[0073] Through the above mechanism, the application significantly reduces data transmission volume and computing pressure while ensuring high-density, multi-point, and high-precision monitoring, achieving a flexible and efficient monitoring mode with cloud and edge collaboration, and enhancing the system's engineering applicability in large-scale, long-term, and complex environments.

[0074] Further, the application introduces a local magnification and adaptive cruise mechanism. This mechanism is driven by the data processing and analysis module. When the system analyzes and identifies that the displacement or displacement rate of a monitoring area exceeds the preset threshold or meets the intelligent algorithm, it automatically controls the gimbal to rotate and align to the high-risk area, adjusts the focal length for local magnification collection, and regards the area as a new high-precision cruise monitoring point. This significantly improves the spatio-temporal resolution and monitoring accuracy of key hidden danger areas.

[0075] II. Detailed steps of the monitoring process

[0076] Reference Figure 1 The monitoring method of the present embodiment is executed according to the following steps:

[0077] Preferably, step S1 system deployment and initialization:

[0078] S101, before the start of monitoring, install the image acquisition module (camera) on a stable area (such as bedrock or a stable structure), ensuring that the camera does not displace during the entire monitoring period, thereby minimizing the occurrence of monitoring errors.

[0079] S102, camera intrinsic parameter calibration, as shown in Figure 2 The camera intrinsic parameter calibration includes a checkerboard calibration board 1, a tripod 2, a three-dimensional laser scanner 3, and a camera device 4. The checkerboard calibration board 1 is used to calibrate the camera device 4 to obtain the intrinsic matrix K and distortion coefficient of the camera device 4. In this embodiment, the intrinsic matrix K obtained by calibration is:

[0080] (1)

[0081] wherein, , f is the equivalent focal length of the camera, , is the principal point coordinate. By calibration, the accuracy of subsequent pixel point tracking and depth calculation can be ensured.

[0082] Preferably, step S2 image data acquisition:

[0083] S201: Control the camera to collect images of the monitoring area at a preset time interval, supporting timed, continuous or dynamic video frame acquisition.

[0084] S202: Image data is transmitted to the data processing center. This embodiment uses a cloud server, which can use its powerful computing power to process large-scale, high-density monitoring point data. At the same time, in order to reduce data transmission volume and cost, the system supports: (1) Adaptive adjustment of sampling rate according to slope risk level: increase sampling frequency in high-risk areas, and reduce sampling frequency in low-risk areas; (2) Edge computing mode: preliminary calculation is performed on part of the representative monitoring points on site, and only when an anomaly or an increased risk is found, large-scale image uploading to the cloud for comprehensive high-precision calculation is started. (3) Intelligent cruise triggered transmission mechanism: using wide-area general monitoring-key precision monitoring method, when the system analyzes and identifies that the displacement or displacement rate of a monitoring area exceeds the preset threshold or meets the intelligent algorithm, the image acquisition module is automatically controlled to rotate and align the high-risk area, and the focal length is adjusted for local magnification collection. This area is regarded as a new high-precision cruise monitoring point.

[0085] Preferably, step S3 feature point tracking and pixel displacement calculation:

[0086] S301: Process the continuously collected image sequence, evenly arrange monitoring points in the monitoring area, and encrypt points in key or high-risk areas.

[0087] S302: Cross-frame matching is performed using pixel-level or sub-pixel-level point tracking algorithm, and the pixel position change of each feature point in the image sequence is recorded, providing basic data for subsequent depth calculation and three-dimensional displacement analysis.

[0088] Preferably, step S4 depth calculation and correction:

[0089] S401: Select a reference image, input a pre-trained depth calculation model to generate an initial depth map, and obtain the relative depth information of the pixel points.

[0090] S402: Laser-camera joint calibration to obtain the camera extrinsic parameter matrix. Required equipment includes: 1 3D laser scanner, 2 tripods, 1 camera, and 1 checkerboard calibration board (12×9 grids, each grid side length 81 mm). By acquiring multiple calibration images and synchronous point cloud data, the camera extrinsic parameter matrix is ​​calculated to achieve registration between the point cloud coordinates and the image coordinate system.

[0091] S403: Establish the spatial correspondence between monocular depth and laser depth. By defining a set of pixels with high-precision laser measurement values ​​and their depth mapping, it provides accurate matching data pairs for subsequent scale and bias correction.

[0092] S404: Performs scale and offset correction on depth, and combines laser depth or other high-precision sensor data with RANSAC fitting and residual interpolation methods to generate an accurate corrected depth map, providing a reliable basis for subsequent three-dimensional displacement calculations.

[0093] Preferably, step S401, as follows: Figure 2 As shown, a customized checkerboard calibration board 1 is stably placed at different positions and angles in the calibration area. Multiple images containing the checkerboard pattern are captured by a camera device 4, and point cloud data from a 3D laser scanner 3 is acquired simultaneously. Then, a checkerboard corner extraction algorithm is used to identify the corner pixel coordinates in each image. The camera's intrinsic and extrinsic parameter matrices are calculated using a camera calibration method, such as... Figure 3 As shown. Then, the point cloud data acquired by the 3D laser scanner 3 is registered with the image coordinate system to obtain the extrinsic transformation matrix between the laser coordinate system and the camera coordinate system. , Figure 3 middle, This represents the coordinates of an observation point. , , These represent the three mutually perpendicular coordinate axes of the local coordinate system. Indicates the camera's focal length. This represents the transformation matrix between the camera coordinate system and the world coordinate system.

[0094] Further, in step S402, the camera extrinsic parameter matrix is ​​obtained. In step S1, the camera intrinsic parameter matrix K has already been obtained. Using the camera's extrinsic and intrinsic parameter matrices, the point cloud coordinates in the 3D laser scanner's 3D coordinate system are transformed to the camera coordinate system. The coordinates in the camera coordinate system are:

[0095] (2)

[0096] in This represents the point cloud coordinates obtained by laser. Let these be the coordinates of the point cloud in the camera coordinate system. Represents the rotation matrix in the transformation matrix. represents the translation matrix in the transformation matrix. It is then projected to the pixel plane:

[0097] (3)

[0098] wherein represents the pixel coordinate of the space point projected on the image plane, is the coordinate of the space point in the camera coordinate system; wherein is the actual depth value, i.e. the distance of the point to the camera plane. In the modeling process, the Z-axis direction of the camera coordinate system is the shooting direction. Therefore, only part is reserved. In addition, due to the limitation of image resolution, the reservation condition of the projection point is:

[0099] (4)

[0100] wherein , are the width and height of the image pixel respectively. Therefore, the real depth (z in the camera coordinate system) corresponding to the laser projection to the pixel (u, v) is

[0101] (5)

[0102] For the same pixel, if there are multiple laser points projected on the pixel, the nearest point is taken wherein represents the depth map (matrix) obtained using the radar, represents the depth value of the laser point; in this embodiment, the schematic of the point cloud projected on the image is shown as Figure 4 .

[0103] Further, in step S403, the correspondence between the monocular depth and the laser depth. Assuming that the initial depth given by the monocular model at the pixel is . The pixel set with the corresponding laser depth is , the corresponding laser depth is , and the monocular depth is .

[0104] Further, in step S404, the global scale and bias calibration. To correct the scale drift of the monocular depth, this embodiment adopts a linear model:

[0105] (6)

[0106] wherein is a global scale factor, used to eliminate the proportional deviation of the monocular depth map; is a global bias, used to eliminate the depth zero point error; denotes the corrected depth value, denotes the monocular depth.

[0107] The minimum sample size m = 2 for each random sampling, two different indices are randomly selected from the corresponding point set The candidate model is constructed with the two samples, and the analytical solution is given by the two points:

[0108] (7)

[0109] where , are randomly selected sample indices, denotes the laser depth value of the point , denotes the laser depth value of the point , denotes the monocular depth value of the point , denotes the monocular depth value of the point , . , denote the candidate scale and bias, respectively.

[0110] For each pixel , the residual under the candidate model is calculated:

[0111] (8)

[0112] The residual is compared with a threshold to determine the inlier point set:

[0113] (9)

[0114] where denotes the absolute residual of pixel under the candidate model, is the residual threshold for inlier determination, is the inlier index set corresponding to the candidate model. Then the candidate model is scored based on the inlier ratio, and the candidate model with the highest score is selected as the current optimal model, obtaining the scale and bias of the optimal model . The residual of the corrected point is calculated using the parameters:

[0115] (10)

[0116] is the residual of each point under the final model, ​​scaling factor of the optimal model, bias of the optimal model, the residual sequence will be used in the next step for residual interpolation and depth map correction. The matching point pair RANSAC fitting case in this embodiment is shown as Figure 5 .

[0117] As shown in Figure 4 , the pixels covered by laser observations have values, but many pixels have no laser observations (holes). The weighted interpolation (bilateral) with image-space joint weights is used to estimate the residual of any pixel :

[0118] (11)

[0119] The purpose of this formula is to propagate sparse residuals to the hole region while preserving the image edges, is the weighted coefficient for pixel , is the residual of the th laser observed pixel. The weight is defined as:

[0120] (12)

[0121] In the formula, is the target pixel coordinate, is the pixel coordinate with laser observed residual ; is the spatial weight scale (control the decay with distance); is the color / intensity weight scale, used to control the edge preserving ability; is the image gray or color vector; is the laser or observation point confidence (can be 1); is the set of laser covered pixels.

[0122] The corrected depth of each pixel is obtained by combining the interpolated residual and the model correction:

[0123] (13)

[0124] where is the original monocular depth value of the target pixel ;

[0125] If a pixel has both laser depth observation , the laser depth and the corrected depth can be fused according to the fusion weight :

[0126] (14)

[0127] Fusion weights The method for calculating the difference between laser confidence level and the two is as follows:

[0128] (15)

[0129] in For monocular models at pixels The original depth; The corrected depth (including residual compensation); Indicates the depth at which the laser can be directly observed (if applicable); To optimize weighting, laser measurement is preferred when the result is close to 1. Laser depth confidence (which can be defined by point cloud density, echo intensity, etc.); , To adjust the constants of weight sensitivity, based on empirical values ​​or verification, in this embodiment, they are taken as 0.5 and 2 respectively.

[0130] Finally, a corrected depth map is output for subsequent spatial displacement calculation steps, thereby ensuring that the calculated three-dimensional displacement is more consistent with the actual deformation or movement of the real slope.

[0131] Further, step S5 involves the calculation of the three-dimensional displacement vector:

[0132] The pixel coordinates of the feature points obtained in step S3 are combined with the depth values ​​of the corresponding positions in the depth map corrected in step S4. Using formula (3) and the camera intrinsic parameter matrix, each feature point is converted into three-dimensional spatial coordinates in the camera coordinate system:

[0133] (16)

[0134] Where (u,v) are the pixel coordinates of the feature point. For the corresponding pixel depth, =(Xc,Yc,Zc) represents the three-dimensional spatial coordinates.

[0135] By comparing the three-dimensional coordinates at different time points, the true displacement vector of each feature point within the monitoring period is calculated:

[0136] (17)

[0137] in and These are the three-dimensional coordinates of the same feature point at different times. In this way, the displacement magnitude, direction, and trend of change of each monitoring point can be obtained. Figure 6 The example shows the continuous deformation-motion trajectory of some monitoring points in this embodiment.

[0138] Further, step S6 data analysis and early warning:

[0139] Smooth the displacement sequence of each feature point (such as moving average or Kalman filtering), and suppress the influence of noise on the monitoring results. Then calculate the regional average displacement, maximum displacement, displacement growth rate and other statistical indicators to judge the overall deformation trend of the slope.

[0140] Based on the design safety standard of the slope, historical monitoring data or expert experience, set multiple threshold values, for example:

[0141] First threshold T1: prompt level threshold, used to prompt early displacement signs;

[0142] Second threshold T2: alarm level threshold, indicating that the slope may have a risk of instability;

[0143] Third threshold T3: emergency threshold, indicating that emergency measures need to be taken immediately.

[0144] Superimpose the displacements of all feature points on the slope image in different colors or arrow lengths to generate a displacement field distribution map, and draw time-displacement curves, displacement heat maps, cumulative displacement bar charts and other visualization charts to intuitively observe the deformation-motion evolution process.

[0145] When the displacement of any monitoring point or area exceeds the preset threshold, the system automatically generates early warning information, including the number of points exceeding the limit, the displacement, the level of exceeding the limit, the timestamp, etc., and is published through multiple channels.

[0146] Through the above process, the embodiment realizes high-precision three-dimensional displacement monitoring through depth calculation and laser depth correction, while fully utilizing the cloud and edge collaborative mechanism to reduce data transmission and computing costs while ensuring accuracy, improving real-time, reliability and scalability of monitoring.

[0147] Embodiment 2

[0148] A lightweight geotechnical deformation and motion monitoring system, comprising:

[0149] Part 1 Image acquisition module: controllable camera device arranged at a stable position (such as stable bedrock, support structure, structure, etc.) opposite or above the monitoring area, to acquire image sequences or video frames of the monitoring object (such as slope, foundation pit, tunnel, rock wall, ground subsidence area, etc.). This module supports, including but not limited to: ordinary high-definition camera, industrial camera, low-illumination imaging device, thermal imager, multi-spectral camera system, or mobile terminal with visual acquisition capability; the core of this module is not only a data acquisition end, but also an intelligent unit with adaptive cruise and dynamic sensing capability.

[0150] Hardware configuration: This module supports including but not limited to: ordinary high-definition camera, industrial camera, low-illumination imaging device, thermal imager, multi-spectral camera system, or mobile terminal with visual acquisition capability. To realize adaptive monitoring, the core device needs to be equipped with a controllable pan-tilt and an electric zoom lens network camera.

[0151] Intelligent cruise mechanism: This module introduces optical zoom-local magnification and adaptive cruise mechanism. This mechanism is driven by the data processing and analysis module: when the system identifies displacement abnormalities in a specific area, it can automatically control the rotation of the pan-tilt and the zoom of the lens, and perform local magnification tracking and monitoring on the area, thereby realizing dynamic adaptive adjustment from wide-area survey to detailed investigation.

[0152] Data storage and transmission module: used for local caching of collected image data, and sending to the data processing platform through wired (such as optical fiber) or wireless means. The platform can be deployed on a local server, edge computing terminal, or cloud environment to ensure the stability and security of data transmission. This module can be combined with an edge computing box to optimize data transmission and resource utilization efficiency. Through a low-cost edge computing core, a low-time complexity algorithm is used to preliminarily screen and densely upload monitoring data of high-risk areas or large deformation time periods, realizing dynamic data transmission and computing analysis.

[0153] Part 2 Data processing and analysis module: the core module of the system, mainly completes image point tracking, depth calculation and correction, displacement calculation, etc., including:

[0154] Point tracking unit: used to track the pixel position changes of natural texture feature points or artificial marker points in image sequences, and calculate pixel-level or sub-pixel-level displacement vectors. This unit can use methods including but not limited to the following and combinations, as well as other equivalent algorithms: any algorithm or method that can be used to realize feature point cross-frame pixel or sub-pixel displacement detection, image interpolation, motion recovery, and noise suppression, including optical flow method, feature matching, deep learning tracking method, or other equivalent methods. This unit combines a robust matching mechanism for occlusion discrimination and motion consistency analysis to solve stable tracking under complex lighting, occlusion, shadow, camera jitter, etc. All equivalent algorithms that can realize cross-frame pixel displacement detection, sub-pixel interpolation, occlusion recovery, and noise suppression are within the scope of this invention.

[0155] Depth calculation and correction unit: used for generating depth information for pixels in the monitoring image, realizing the conversion from two-dimensional pixel motion to three-dimensional coordinate motion. This unit can adopt methods including but not limited to monocular, binocular, active light source, laser radar, millimeter wave radar or multi-source fusion method, or deep learning model, graph optimization method, scale recovery method, etc. Depth correction can be combined with laser, radar data for depth post-processing correction, known calibration points or measured reference depth for scaling, multi-frame sliding window consistency adjustment, or through model fusion, post-processing filtering and the like to optimize the depth map confidence. All equivalent technical solutions capable of inferring, completing or optimizing the depth information of the pixels in the monitoring range belong to the protection scope of the present application.

[0156] Coordinate conversion and calculation unit: based on camera internal participation distortion parameters, the pixel points and their depth values in each frame of image are converted into three-dimensional coordinates in the camera coordinate system. By comparing the three-dimensional coordinates at different time points, the real displacement calculation of the monitoring points in the three-dimensional space is realized, and multi-frame fusion, filtering and denoising processing are supported to enhance the accuracy and robustness.

[0157] Part 3 warning and display module: used for visualizing the calculated displacement information to generate: displacement-time curve, displacement vector field, rock-soil mass overall deformation cloud chart, three-dimensional dynamic model or video stream. Different alarm thresholds or intelligent judgment algorithms are set to multi-level warn abnormal deformation rate or cumulative deformation (such as early warning, primary warning, emergency warning, etc.), and the warning information can be sent through short message, email, platform push and the like.

[0158] A lightweight rock-soil mass deformation and motion monitoring system and method, comprising the following steps:

[0159] S1: system layout and calibration: selecting a stable reference area to install a camera device, completing camera internal parameter, distortion parameter and (if necessary) external parameter calibration.

[0160] S2: image data acquisition: controlling the image acquisition module to acquire monitoring area images at regular time intervals or in real time, providing edge computing or uploading to a data processing platform.

[0161] S3: feature point tracking and pixel displacement calculation: by selecting natural feature points or laying out marker points, cross-frame point tracking in image sequences is performed to obtain pixel displacement vectors.

[0162] S4: depth calculation and correction: applying monocular or multi-modal depth calculation algorithm to image frames, combining with external sensor data (such as laser radar, radar point cloud) for correction to obtain a reliable depth map.

[0163] S5: Three-dimensional displacement vector calculation: Combine the pixel coordinates of the feature points obtained in S3 with the depth values of the corresponding positions in the depth map in S4, and use the camera intrinsic parameters to calculate the three-dimensional coordinates of each feature point in the camera coordinate system. By comparing the three-dimensional coordinates at different time points, the three-dimensional real displacement vector of each feature point is calculated.

[0164] S6: Data analysis and early warning: Statistical analysis is performed on the displacement data of all feature points, and a displacement chart is drawn. Once the monitoring value exceeds the preset safety threshold or meets the intelligent judgment algorithm, an early warning information is generated and published immediately.

[0165] Example 3

[0166] The monitoring method of this embodiment includes the following steps:

[0167] S1: System layout and calibration: Select stable reference positions (such as stable bedrock, support structure or structure) to layout the camera device in the monitoring area. Complete the intrinsic parameter calibration and distortion correction of the camera device, and realize the space coordinate mapping by combining the extrinsic parameter calibration or known reference depth. The calibration method is not limited to the specific implementation, including manual calibration, automatic calibration, feature point or calibration board based method, and joint optimization strategy combined with multiple source sensors.

[0168] S2: Image data acquisition: Control the image acquisition module to acquire images of the monitoring area in real time, at a preset time interval or in a dynamic adaptive acquisition mode. It can be combined with variable focus or long focus camera unit, combined with intelligent cruise mode, to improve the monitoring accuracy and monitoring frequency of key monitoring areas. Image data can be transmitted to the data processing platform through wired or wireless mode, and the platform includes local server, edge computing terminal or cloud. The data transmission method is not limited to a specific protocol or communication technology.

[0169] S3: Feature point tracking and pixel displacement calculation: Select natural texture feature points or artificial layout marker points in the monitoring area in the acquired image sequence, and perform cross-frame pixel-level or sub-pixel-level tracking to obtain the displacement vector of each feature point. The point tracking method can use optical flow method, feature matching algorithm, deep learning tracking model or other equivalent technology, and can be combined with sub-pixel interpolation, occlusion recovery, motion consistency analysis and robust filtering to realize stable tracking in complex light, shadow, rain, fog, camera shaking and other environments.

[0170] S4: Depth calculation and rectification: Use monocular, binocular, active light source or multi-source sensors to generate an initial depth map, and combine laser point cloud, radar point cloud or other external reference data for depth rectification. Rectification methods can include laser, radar depth post-processing rectification, multi-frame sliding window consistency adjustment, scaling calibration, model fusion or post-processing filtering, etc. to ensure the integrity and accuracy of the depth map. The depth calculation and rectification algorithm is not limited to a specific implementation, and all equivalent solutions that can achieve two-dimensional pixel displacement to three-dimensional coordinate mapping are within the scope of the invention.

[0171] S5: Three-dimensional displacement vector calculation: Combine the feature point pixel coordinates obtained in S3 with the depth values of the corresponding pixels in the depth map in S4, and convert them to three-dimensional space coordinates in the camera coordinate system through camera parameters and distortion parameters. Compare the three-dimensional coordinates at different time points to calculate the real displacement vector of each feature point in three-dimensional space. Multi-frame fusion, filtering, denoising and other methods can be used to improve the calculation accuracy and robustness.

[0172] S6: Data analysis and early warning: Statistical analysis and visual display of three-dimensional displacement data of all feature points, including displacement / motion-time curve, displacement vector field, overall deformation cloud chart and dynamic three-dimensional model, etc. Multiple alarm thresholds or early warning models can be set, and when the monitoring data exceeds the threshold or meets the conditions of the intelligent algorithm, the system automatically generates early warning information and publishes it through SMS, email or platform push. The analysis and early warning algorithm is not limited to a specific implementation, and all methods that achieve equivalent functions are within the scope of the invention.

[0173] The entire process of the monitoring method of the invention can flexibly adjust the monitoring point density, acquisition frequency and algorithm strategy to adapt to different geological environments, monitoring area size and risk level requirements, and realize high-density, scalable and real-time rock and soil deformation and movement monitoring. The monitoring method of the invention is shown in Figure 1

[0174] Obviously, the above embodiments are only examples for clarity and do not limit the implementation. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, it is not necessary or possible to exhaust all implementations. The obvious changes or variations derived therefrom are still within the protection scope of the invention.​

Claims

1. A lightweight soil and rock deformation and motion monitoring system, characterized in that, include: The image acquisition module is deployed in a stable reference area to acquire image sequences or video frames of the monitored area. The data storage and transmission module is used for local caching and remote transmission of image data; The data processing and analysis module includes: A point tracking unit is used to track the pixel displacement of feature points in an image sequence. The depth calculation and correction unit is used to generate and correct the depth information of pixels in the image; The coordinate transformation and displacement calculation unit is used to convert pixel coordinates and depth information into three-dimensional coordinates and calculate the actual displacement; The early warning and display module is used for the visualization of displacement data and multi-level early warning. Perform the following steps: S1: System deployment and calibration, installation of camera equipment and completion of camera parameter calibration; Obtain the intrinsic parameter matrix K and distortion coefficients of the camera device; the intrinsic parameter matrix K is: Where fx and fy are the equivalent focal lengths of the camera, and cx and cy are the coordinates of the principal point; S2: Image data acquisition, obtaining image sequences of the monitored area; S3: Feature point tracking and pixel displacement calculation, tracking feature points and obtaining pixel displacement vectors; S4: Depth Calculation and Correction, generating and correcting the depth map, the steps are as follows: S401: The corner point extraction algorithm is used to identify the pixel coordinates of corner points in each image. The intrinsic and extrinsic parameters of the camera are calculated by the camera calibration method. The point cloud data is registered with the image coordinate system to obtain the extrinsic parameter transformation matrix (R,T) between the laser coordinate system and the camera coordinate system. S402: Obtain the camera extrinsic parameter matrix: Using the camera's extrinsic and extrinsic parameter matrices, transform the point cloud coordinates to the camera coordinate system. The coordinates in the camera coordinate system are: P c =RP L +t (2) Where P L P represents the point cloud coordinates obtained by laser. c Let R represent the coordinates of the point cloud in the camera coordinate system, R represent the rotation matrix in the transformation matrix, and t represent the translation matrix in the transformation matrix; then it is projected onto the pixel plane: Where (u,v) represents the pixel coordinates of the spatial point projected onto the image plane, (X... c ,Y c Z c Z represents the coordinates of the point in the camera coordinate system; where Z is the coordinate of the point in the camera coordinate system. c Given the actual depth value, the conditions for retaining the projected points are: 0≤u<W,0≤v<H (4) Where W and H are the width and height of the image pixels, respectively; therefore, the true depth corresponding to the laser projection onto pixel (u,v) is: D L (u,v)=Z c (5) If multiple laser points are projected onto the same pixel, then the nearest point D is selected. L (u,v)=minZ c D L This indicates a depth map acquired using radar, Z. c This represents the depth value of the laser point; S403. Correspondence between monocular depth and laser depth: Let the initial depth of the monocular model at pixel (u,v) be D. m (u,v); where the set of pixels corresponding to the laser depth is... The corresponding laser depth is L i :=D L (u i ,v i The monocular depth is M. i :=D m (u i ,v i ); S404, Global Scale and Bias Calibration: Using a linear model: Where s is the global scale factor, used to eliminate the scale deviation of the monocular depth map; b is the global offset, used to eliminate the depth zero point error; M represents the corrected depth value. i Indicates monocular depth; Each random sampling requires a minimum sample size of m = 2, starting from the corresponding point set. Two different indices j and k are randomly selected from the samples; a candidate model is constructed using these two samples, and the analytical solution is given from two points: Where j, k are randomly selected sample indices, L j L represents the laser depth value at point j. k M represents the laser depth value at point k. j M represents the monocular depth value at point j. k Represents the monocular depth value at point k, requiring M to be... j ≠M k ;s (cand) ,b (cand) These represent the candidate scale and bias, respectively. For each pixel i, calculate the residual under the candidate model: The residuals are compared with a threshold T to determine the set of interior points: in This represents the absolute residual of pixel i under the candidate model, where τ is the residual threshold used for interior point determination. Let be the set of interior point indices corresponding to the candidate model; then, score the candidate models based on the proportion of interior points, select the candidate model with the highest score as the current optimal model, and obtain the scale and bias s of the optimal model. ref ,b ref Use this parameter to calculate the residual at the correction point: r i =L i -(s ref M i +b ref ),i=1,…,N (10) r i For the residuals at each point in the final model, s ref b represents the scaling factor of the optimal model. ref This represents the bias of the optimal model; The residual δ(p) of any pixel p is estimated using weighted interpolation with joint image-space weights: w i (p) represents the weighting coefficient for pixel p, r i This represents the residual of the i-th laser observation pixel; where the weights are defined as follows: In the formula, p is the target pixel coordinate. i σ represents the pixel coordinates with laser observation residual ri; s σ is the spatial weighting scale used to control the decay with distance; r Represents the color / intensity weight scale, used to control edge preservation capability; I is the image grayscale or color vector; c i Ω represents the confidence level of the laser or observation point; Ω represents the set of pixels covered by the laser. The correction depth per pixel is obtained by combining the interpolated residuals with the model correction. D c (p)=s ref D m (p)+b ref +δ(p) (13) Where D m (p) represents the original monocular depth value of the target pixel p; If a pixel p simultaneously has laser depth observation D L (p), the laser depth and the correction depth are fused according to the fusion weight a(p): D out (p)=α(p)D L (p)+(1-α(p))D c (p) (14) The weight a(p) is calculated based on the laser confidence level and the difference between the two as follows: Where D m (p) represents the original depth of the monocular model at pixel p; D c (p) represents the corrected depth; D L (p) represents the depth directly observed by laser; α(p) is the fusion weight, with laser measurements being preferred when the value is close to 1; c L (p) represents the laser depth confidence level; γ, σ d The constant for adjusting the weighted sensitivity is determined through empirical values ​​or verification. S405: Outputs a corrected depth map for subsequent spatial displacement calculation steps; S5: 3D displacement vector calculation, combining pixel coordinates with depth information to calculate the true 3D displacement, the steps are as follows: The pixel coordinates of the feature points obtained in step S3 are combined with the depth values ​​of the corresponding positions in the depth map corrected in step S4. Using formula (3) and the camera intrinsic parameter matrix, each feature point is converted into three-dimensional spatial coordinates in the camera coordinate system: Where (u, v) are the pixel coordinates of the feature point, Zc is the corresponding pixel depth, and Pc = (Xc, Yc, Zc) are the three-dimensional spatial coordinates; By comparing the three-dimensional coordinates at different time points, the true displacement vector of each feature point within the monitoring period is calculated: ΔP=P t2 -P t1 (17) Where Pt1 and Pt2 are the three-dimensional coordinates of the same feature point at different times; S6: Data Analysis and Early Warning. Performs statistical analysis and visualization of displacement data and triggers early warnings.

2. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, The image acquisition module includes a camera device equipped with a controllable gimbal and a motorized zoom lens, supporting adaptive cruise and local magnification monitoring.

3. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, The point tracking unit employs optical flow, feature matching, deep learning tracking methods, or combinations thereof, to achieve pixel-level or sub-pixel-level displacement tracking, and possesses occlusion discrimination and motion consistency analysis capabilities.

4. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, The depth calculation and correction unit generates a depth map using monocular, binocular, active light source, lidar, millimeter-wave radar, or multi-source fusion methods, and performs depth correction by combining laser point cloud, radar data, or known reference depth.

5. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, The coordinate transformation and displacement calculation unit converts pixel coordinates and depth values ​​into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters and distortion parameters, and calculates the real displacement vector by comparing the three-dimensional coordinates at different time points.

6. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, The early warning and display module generates displacement-time curves, displacement vector fields, deformation cloud maps or three-dimensional dynamic models, and triggers multi-level early warnings based on preset thresholds or intelligent judgment algorithms.

7. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, In step S4, depth correction includes: High-precision depth references are obtained using laser or radar data; Scale and bias calibration were performed using the RANSAC fitting method. An image-spatial joint weighted interpolation method is used to propagate and fuse depth residuals to generate a corrected depth map.

8. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, In step S2, the image acquisition supports edge computing and dynamic uploading mechanisms, adaptively adjusts the sampling frequency according to the risk level of the monitoring area, and starts high-precision local cruise monitoring when an anomaly is detected.

9. The lightweight soil and rock deformation and motion monitoring system according to claim 1, characterized in that, In step S6, the warning information is released via SMS, email or platform push, and the warning thresholds include multiple levels such as prompt level, alarm level and emergency level.

Images