Multi-source data fused non-contact ground fracture monitoring method, device and system and storage medium

Abstract

The invention provides a non-contact ground fracture monitoring method, device and system fusing multi-source data, and a storage medium. The method comprises the steps of obtaining a two-dimensional visible light image sequence and a three-dimensional depth image sequence of a to-be-monitored ground fracture region; obtaining a two-dimensional plane time sequence deformation vector and a three-dimensional deformation feature sequence of the edge key points of the ground fracture; obtaining a ranging data sequence of ground fracture edge key points; dynamically determining weights of the two-dimensional plane time sequence deformation vector, the three-dimensional deformation feature sequence and the ranging data sequence; establishing a transfer function between the fusion data matrix and a ground fracture target deformation result; and forming an observation data conditional probability based on the sensor measurement error and the corresponding weights of the two-dimensional plane time sequence deformation vector, the three-dimensional deformation feature sequence and the ranging data sequence, and performing maximum likelihood estimation on the observation data conditional probability to obtain a ground fracture target deformation result. By adopting the monitoring method provided by the invention, the ground fracture deformation can be monitored more accurately.

Description

Technical Field

[0001] This application relates to the field of ground fissure monitoring technology, and in particular to a non-contact ground fissure monitoring method, device, system and storage medium that integrates multi-source data. Background Technology

[0002] Ground fissure monitoring is the continuous observation and analysis of the deformation, activity, and impact range of surface fissures through various technical means.

[0003] In the process of developing this application, the inventors discovered at least the following problems in the prior art: Although modern ground fissure monitoring technology systems have taken initial shape, some technical bottlenecks and problems still exist in practical applications, mainly including the following aspects: (1) Insufficient equipment accuracy and timeliness Existing monitoring equipment still has certain shortcomings in terms of accuracy and timeliness: Contact sensors are significantly affected by environmental factors and have a certain degree of measurement error, especially under extreme weather conditions where the error may be further amplified. For example, fiber optic sensors are susceptible to interference from changes in ambient temperature, requiring phase compensation algorithms to eliminate the effects of thermal strain; ultrasonic detection has a near-field blind zone and insufficient sensitivity to minute cracks.

[0004] InSAR monitoring is affected by land cover (such as vegetation and buildings), resulting in some blind spots, which leads to a decrease in the density of effective monitoring points, and the long data update cycle makes it difficult to capture sudden deformations.

[0005] Manual monitoring is infrequent and struggles to capture sudden ground fissure deformations, leading to the loss of some crucial data. Visual monitoring is limited by lighting conditions; images are easily blurred at night and in rainy or foggy weather, affecting the accuracy of fissure identification.

[0006] (2) Limitations on adaptability to deployment conditions Traditional monitoring devices have certain adaptability issues when deployed in complex environments: InSAR monitoring requires a wide field of view, but its effectiveness is limited in densely built-up areas or areas with complex terrain. Unmanned aerial vehicle (UAV) remote sensing suffers from electromagnetic interference and complex airflow, resulting in insufficient flight stability and increasing the difficulty of acquiring high-precision data.

[0007] GNSS base stations have high requirements for bedrock foundations. In soft soil areas or unstable geological conditions, insufficient stability may lead to reference value drift. Fiber optic sensors must be strictly deployed along the principal stress direction of the structure, which is difficult to construct and susceptible to damage during secondary construction.

[0008] Contact sensors require regular maintenance, have a high failure rate, and incur significant maintenance costs, impacting the long-term stability of the monitoring system. Laser scanning equipment is significantly affected by dust and water mist, necessitating frequent cleaning and maintenance of the optical lenses.

[0009] (3) Weak ability to fuse multi-source data Current monitoring systems still face certain technical bottlenecks in multi-source data fusion: Data from different monitoring methods differ in spatiotemporal resolution (such as continuous monitoring data from optical fibers versus periodic images from drones), making data matching and fusion difficult and hindering the unification of spatiotemporal benchmarks.

[0010] The lack of an effective mathematical correlation model between visual monitoring (such as human interpretation) and physical sensor data affects the accuracy of data fusion. For example, it is difficult to establish a quantitative mapping relationship between the crack width of image recognition and the internal strain field of fiber optic sensing.

[0011] Data assimilation algorithms are relatively outdated, and existing algorithms are not adaptable enough to handle complex deformations, resulting in large fusion errors in monitoring data. The joint inversion of multi-source heterogeneous data (such as InSAR deformation fields, GNSS three-dimensional displacement, and ultrasonic internal defects) still suffers from multiple solutions problem.

[0012] (4) Bottlenecks in the engineering application of new technologies Emerging monitoring technologies face new challenges in practical applications: Image processing technology relies on high-quality image acquisition; crack recognition algorithms are susceptible to noise interference; existing deep learning models lack generalization ability, and recognition accuracy drops sharply with small sample data. Infrared thermal imaging is easily affected by environmental temperature differences, requiring the establishment of complex thermodynamic compensation models.

[0013] Unmanned aerial vehicle (UAV) remote sensing technology suffers from a "dammed lake" effect in data processing. The terabytes of image data generated by a single flight cause processing delays, making it difficult to meet real-time early warning requirements. Furthermore, technologies for massive storage and rapid retrieval of laser scanning point cloud data are not yet mature.

[0014] The low standardization of new monitoring technologies, such as the lack of a unified evaluation system for strain demodulation accuracy in distributed optical fiber monitoring and defect location algorithms in ultrasonic testing, hinders the promotion of these technologies and data interoperability.

[0015] Therefore, there is a need for a non-contact ground fissure monitoring method, device, system, and storage medium that integrates multi-source data to at least partially solve the above-mentioned technical problems. Summary of the Invention

[0016] In view of this, embodiments of this application provide a non-contact ground fissure monitoring method, apparatus, system, and storage medium that integrates multi-source data, so as to at least solve one of the problems in the prior art.

[0017] In a first aspect, embodiments of this application provide a non-contact ground fissure monitoring method that integrates multi-source data, the monitoring method comprising: Acquire continuous visual images of the ground fissure area to be monitored, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence; Two-dimensional planar temporal deformation vectors of key points at the edge of ground fissures are obtained based on two-dimensional visible light image sequences, and three-dimensional solid deformation feature sequences of key points at the edge of ground fissures are obtained based on three-dimensional depth image sequences; the key points at the edge of ground fissures are preset targets at the edge of ground fissures. Obtain a continuous sequence of distance measurement data for key points at the edge of the ground fissure in the area to be monitored. The weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared with historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data. A transfer function between the fused data matrix and the deformation results of the ground fissure target is established by constructing a fused data matrix using two-dimensional planar temporal deformation vectors, three-dimensional solid deformation feature sequences, and ranging data sequences. Based on the sensor measurement errors used to acquire visual images and wireless ranging data, and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence, the observation data conditional probability is formed. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

[0018] Secondly, embodiments of this application also provide a non-contact ground fissure monitoring device that integrates multi-source data, the monitoring device comprising: Memory is used to store executable instructions for a computer; A processor, used to execute computer-executable instructions stored in the memory, implements the monitoring method of the above-described technical solution.

[0019] Thirdly, embodiments of this application also provide a storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the monitoring method of the above-described technical solution.

[0020] Fourthly, embodiments of this application also provide a non-contact ground fissure monitoring system that integrates multi-source data, the monitoring system comprising: Remote terminal units are deployed around the ground fissure area to be monitored according to the location of the monitoring points; each remote terminal unit integrates a main control module and a wireless data transmission module, and is equipped with connection ports for a visible light camera, an RGBD camera and a wireless ranging module. A visible light camera is placed on a remote terminal unit around the ground fissure area to be monitored, with monitoring points set at the locations, to acquire a two-dimensional visible light image sequence of the ground fissure area to be monitored. An RGBD camera on a remote terminal unit, with monitoring points set around the area of ​​the ground fissure to be monitored, is used to acquire a sequence of three-dimensional depth images of the area of ​​the ground fissure to be monitored. A wireless ranging module on a remote terminal unit, which is set with monitoring point locations around the ground fissure area to be monitored, is used to acquire a continuous ranging data sequence of key points on the edge of the ground fissure in the ground fissure area to be monitored. The monitoring center receives two-dimensional visible light image sequences, three-dimensional depth image sequences, and ranging data sequences, and performs the following processing: Two-dimensional planar temporal deformation vectors of key points at the edge of ground fissures are obtained based on two-dimensional visible light image sequences, and three-dimensional solid deformation feature sequences of key points at the edge of ground fissures are obtained based on three-dimensional depth image sequences. The weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared with historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data. A transfer function between the fused data matrix and the deformation results of the ground fissure target is established by constructing a fused data matrix using two-dimensional planar temporal deformation vectors, three-dimensional solid deformation feature sequences, and ranging data sequences. Based on the measurement errors of the visible light camera, RGBD camera, and wireless ranging module used to acquire visual images and wireless ranging data, as well as the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence, the observation data conditional probability is formed. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

[0021] The monitoring method according to the embodiments of this application constructs a multimodal monitoring system of "visible light visual recognition + wireless ranging + RGBD visual recognition", opens up the fusion channel of multi-source heterogeneous data, constructs a three-dimensional deformation mathematical model, breaks through the limitations of traditional monitoring technology in terms of accuracy, timeliness and adaptability, effectively improves the accuracy of ground fissure deformation monitoring, and provides solid technical support for geological disaster prevention and control.

[0022] Additional advantages, objectives, and features of this application will be set forth in part in the description which follows, and will in part become apparent to those skilled in the art upon review of the following description, or may be learned by practice of the application. The objectives and other advantages of this application can be realized and obtained by means of the structures specifically pointed out in the specification and drawings.

[0023] Those skilled in the art will understand that the purposes and advantages that can be achieved with this application are not limited to those specifically described above, and that the above and other purposes that this application can achieve will be more clearly understood from the following detailed description. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, do not constitute a limitation thereof. The components in the drawings are not drawn to scale but are merely for illustrating the principles of this application. For ease of illustration and description of certain parts of this application, corresponding portions in the drawings may be enlarged, i.e., may appear larger relative to other components in an exemplary device actually manufactured according to this application. In the drawings: Figure 1 This is a flowchart of a monitoring method according to an embodiment of this application; Figure 2 This is a schematic diagram of a monitoring device according to an embodiment of this application; Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present application; Figure 4 This is a schematic diagram of a monitoring system according to an embodiment of this application; Figure 5 This is a schematic diagram showing the location distribution of a monitoring system according to an embodiment of this application; Figure 6 This is a schematic diagram of a remote terminal unit in a monitoring system according to an embodiment of this application; Figure 7 This is a schematic diagram of the installation of a remote terminal unit in a monitoring system according to an embodiment of this application; Figure 8 This is an exploded view of the mounting components of a remote terminal unit in a monitoring system according to an embodiment of this application.

[0025] Explanation of reference numerals in the attached figures: 400. Monitoring system; 410. Remote terminal unit; 411. Host; 412. N-type interface; 413. First connection port; 414. Second connection port; 415. Fixture; 416. U-shaped clamp; 417. Buckle; 420. Visible light camera; 430, RGBD camera; 440. Wireless ranging module; 450. Monitoring Center. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and their descriptions are used to explain this application, but are not intended to limit it.

[0027] It should also be noted that, in order to avoid obscuring this application with unnecessary details, only the structures and / or processing steps closely related to the solution according to this application are shown in the accompanying drawings, while other details that are not closely related to this application are omitted.

[0028] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0029] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0030] In the following description, embodiments of the present application will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0031] First, refer to Figure 1 This application describes a non-contact ground fissure monitoring method 100 that integrates multi-source data according to an embodiment of the present application. For example... Figure 1 As shown, the determination method 100 may include steps S110 to S160, as detailed below: In step S110, continuous visual images of the ground fissure area to be monitored are acquired, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence.

[0032] In step S120, a two-dimensional planar temporal deformation vector of the key points at the edge of the ground fissure is obtained based on a two-dimensional visible light image sequence, and a three-dimensional solid deformation feature sequence of the key points at the edge of the ground fissure is obtained based on a three-dimensional depth image sequence; the key points at the edge of the ground fissure are preset targets at the edge of the ground fissure.

[0033] In step S130, a sequence of distance measurement data of key points at the edge of the ground fissure in the area to be monitored is obtained.

[0034] In step S140, the weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data.

[0035] In step S150, a transfer function is established between the fused data matrix and the ground fissure target deformation result, using the fused data matrix constructed from the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence.

[0036] In step S160, based on the sensor measurement error used to acquire visual images and wireless ranging data, and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence and the ranging data sequence, the observation data conditional probability is formed. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

[0037] In the embodiments of this application, firstly, continuous visual images of the ground fissure area to be monitored are acquired, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence; based on the two-dimensional visible light image sequence, a two-dimensional planar temporal deformation vector of key points at the edge of the ground fissure is acquired, and based on the three-dimensional depth image sequence, a three-dimensional solid deformation feature sequence of key points at the edge of the ground fissure is acquired; the key points at the edge of the ground fissure are preset targets at the edge of the ground fissure; next, a continuous ranging data sequence of key points at the edge of the ground fissure area to be monitored is acquired; then, based on the visual image feature matching rate, visual image clarity, number of feature points per frame of the visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, and wireless... The weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence are dynamically determined based on the stability of the ranging data and the signal strength of the wireless ranging data. Then, using the fused data matrix constructed from the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence, a transfer function is established between the fused data matrix and the deformation result of the ground fissure target. Finally, based on the sensor measurement errors used to acquire visual images and wireless ranging data, and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence, the conditional probability of the observation data is formed. Maximum likelihood estimation is performed on the conditional probability of the observation data to obtain the deformation result of the ground fissure target.

[0038] As described above, the monitoring method 100 according to the embodiments of this application integrates multiple visual monitoring technologies to form a comprehensive and multi-dimensional ground fissure monitoring system. At the same time, high-precision wireless ranging is introduced as an auxiliary supplement to provide data support when visual monitoring is affected by environmental interference. Finally, a multi-source data fusion algorithm is used to achieve high-precision and high-reliability monitoring of ground fissure deformation, forming a high-precision monitoring technology method that takes into account both wide coverage and anti-interference capabilities, thus solving the bottleneck problems of weak environmental adaptability and single data in traditional monitoring.

[0039] The following will combine Figure 1 The specific description includes the above-described steps of the monitoring method 100 according to the embodiments of this application.

[0040] In the embodiments of this application, step S110 involves acquiring continuous visual images of the ground fissure area to be monitored, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence.

[0041] Specifically, before proceeding to step S110, a target (receiving end) can be preset at the edge of the ground fissure in the area to be monitored, serving as a key point at the edge of the ground fissure. Ground fissure deformation monitoring is achieved by monitoring the displacement and deformation of the key point at the edge of the ground fissure.

[0042] Specifically, a two-dimensional visible light image sequence can be obtained using high-resolution visible light cameras positioned at monitoring points around the area of ​​the ground fissure to be monitored. A three-dimensional depth image sequence can also be obtained by acquiring image data using RGBD cameras positioned at monitoring points around the area of ​​the ground fissure to be monitored. The three-dimensional depth image sequence acquired by the RGBD cameras includes color images (RGB) and depth maps.

[0043] In the embodiments of this application, in step S120, a two-dimensional planar temporal deformation vector of the key points at the edge of the ground fissure is obtained based on a two-dimensional visible light image sequence, and a three-dimensional solid deformation feature sequence of the key points at the edge of the ground fissure is obtained based on a three-dimensional depth image sequence; the key points at the edge of the ground fissure are preset targets at the edge of the ground fissure.

[0044] Specifically, obtaining the two-dimensional planar temporal deformation vector of key points at the edge of a ground fissure based on a two-dimensional visible light image sequence may include the following steps: a. Image Grayscale Conversion: A weighted average method is used to process the pixel values ​​of all channels to obtain a single-channel grayscale image. The pixel values ​​range from [0, 255]. For crack detection, the grayscale image retains the image features that reveal crack information and significantly speeds up the detection process. This eliminates redundant image information while improving algorithm processing efficiency and enhancing the real-time performance of the entire crack detection algorithm. b. Gray-scale transformation: A logarithmic transformation is performed on the image to enhance the low gray-scale components and compress the high gray-scale components, thereby improving the contrast of dark areas in the gray-scale image. While reducing the gray-scale values ​​in bright areas of the image, crack information is effectively highlighted. c. Histogram equalization: The image is processed using a restricted adaptive histogram equalization algorithm to reduce the uneven lighting in the original image caused by differences in shooting time and lighting. This makes the gray values ​​of the pixels evenly distributed within a larger gray range, thereby improving the contrast of the image. Ultimately, the crack information in the transformed image is more obvious and easier to identify. d. Feature extraction and deformation calculation: SIFT / SURF / ORB feature matching and YOLO model can be combined. First, YOLO is used to quickly segment the crack area. Then, edge key points are extracted for the segmented area. Based on the key points in the historical image, the key points in the current detection image are matched, the displacement vector is calculated, and the two-dimensional planar temporal deformation vector of the key points at the edge of the ground crack is obtained.

[0045] Furthermore, the accuracy of the obtained two-dimensional planar temporal deformation vectors of key points at the edge of ground fissures can be improved to the millimeter level through cubic spline subpixel interpolation.

[0046] Obtaining the three-dimensional deformation feature sequence of key points at the edge of a ground fissure based on a three-dimensional depth image sequence may include the following steps: a. Depth Information Acquisition and Preprocessing: The RGBD camera simultaneously acquires RGB images and depth data of the scene using infrared structured light or Time-of-Flight (ToF) technology, where each pixel of the depth map corresponds to a physical distance (raw accuracy ≤ 1mm). An improved algorithm based on bilateral filtering is adopted to suppress random noise while preserving the abrupt depth changes at the crack edges. Weighted calculation using Gaussian kernel functions in the spatial and depth domains avoids detail loss due to excessive smoothing. b. 3D Point Cloud Reconstruction and Registration: The preprocessed depth map is pixel-level aligned with the corresponding RGB image. Using camera intrinsic parameters (focal length, principal point coordinates), the (u,v) coordinates and depth values ​​of each pixel are converted into 3D spatial coordinates (X, Y, Z), generating 3D point cloud data for the crack region. For point cloud sequences acquired at different times, an improved ICP (Iterative Closest Point) algorithm is used for registration. c. Extraction of 3D deformation features of cracks: Based on the registered point cloud data, combined with the crack information in the RGB image and the normal vector and curvature features of the point cloud, a region growing algorithm is used to segment the crack region from the point cloud, ensuring a segmentation accuracy of ≥90%; the segmented crack point cloud is compared point by point, and the spatial displacement vector (ΔX, ΔY, ΔZ) of the point cloud at adjacent time points is calculated. Through statistical analysis, the uplift / settlement of the crack in the vertical direction (Z axis), the displacement in the horizontal direction (X, Y axis), and the change in crack volume are obtained.

[0047] This step integrates multiple visual technologies such as high-resolution camera image analysis and RGBD camera depth sensing to construct a composite visual monitoring system of "two-dimensional image + three-dimensional depth," achieving full-process coverage of cracks from surface deformation to three-dimensional feature analysis.

[0048] By introducing an RGBD camera to simultaneously acquire color images (RGB) and depth maps of the crack area, a three-dimensional monitoring dimension of "color texture + spatial depth" is constructed. Through multi-step processing, high-precision analysis of the crack's three-dimensional deformation is achieved, supplementing the spatial dimension information of the two-dimensional image.

[0049] In the embodiments of this application, step S130 involves obtaining a sequence of ranging data of key points at the edge of a continuous ground fissure in the area to be monitored.

[0050] Specifically, the high-precision wireless ranging module serves as a supplement to visual monitoring, focusing on high-precision distance monitoring of key crack locations and providing stable deformation reference data in extreme scenarios such as visual obstruction and drastic changes in lighting.

[0051] High-precision wireless ranging modules primarily utilize ultra-wideband (UWB) technology for ranging. This technology is based on Time-of-Flight (TOF) technology, specifically calculating distance by measuring the flight time of a radio signal from the transmitting device (wireless ranging module) to the receiving device (the receiver at the key point). TOF technology is existing and will not be elaborated upon here.

[0052] The above measurement method has errors, and these errors cannot be eliminated, but can only be minimized. The best way is to send the message out as quickly as possible after receiving it to prevent the error from accumulating. Also, since there may be obstacles or interference between the two devices, the distance measurement may vary greatly, so it is best to take the average value multiple times.

[0053] In the embodiments of this application, in step S140, the weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image clarity, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data.

[0054] Specifically, when the visual image feature matching rate (the ratio of the number of matched feature points to the total number of feature points, a common concept in the industry) is greater than a set value, such as 85%, the weights of the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence are both configured to the first range, such as 40%-45%. The weight of the ranging data sequence is 10%-20%, mainly used to verify the consistency of the visual data.

[0055] Specifically, when the visual image feature matching rate is greater than a set value, the system also determines the displacement deviation between the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence in the wireless ranging direction and the ranging data sequence. If the deviation is less than a set value (e.g., 5%), a fused data vector is constructed based on the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence. If the deviation is greater than a set value, a data anomaly warning is triggered, prompting manual verification of the equipment status.

[0056] When the visual image feature matching rate is less than a set value, the stability of the wireless ranging data is less than a set value (e.g., standard deviation ≤ set value, such as 2mm), and the signal strength of the wireless ranging data is greater than a set value (e.g., RSSI ≥ set value, such as -85dBm), the weights of the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence are both configured to the second range, such as 15%-25%. The weight of the ranging data sequence is 50%-70%.

[0057] Furthermore, when the visual image feature matching rate is less than the set value, the stability of the wireless ranging data is less than the set value, and the signal strength of the wireless ranging data is greater than the set value, the ranging data sequence is interpolated to the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence through the spatial correlation algorithm to fill the visual blind spot and ensure the continuity of visual deformation monitoring.

[0058] The specific weight values ​​(or fluctuations) of the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence, regardless of whether they are assigned to the first or second range, need to be determined based on a comprehensive evaluation of the visual image sharpness (entropy value ≥ preset value, e.g., 5.0), the number of feature points per frame of the visual image (SIFT / SURF / ORB feature points per frame ≥ preset value, e.g., 50), and the repetition rate of the visual image matching point pairs compared to historical matching point pairs (≥ preset value, e.g., 90%). The contributions of the above visual indicators to the weights all exhibit a monotonically increasing relationship. Common calculation methods include direct addition, sum of squares, and applying various monotonic kernel functions followed by summation.

[0059] In the embodiments of this application, in step S150, a fusion data matrix is ​​constructed using a two-dimensional planar temporal deformation vector, a three-dimensional solid deformation feature sequence, and a ranging data sequence, and a transfer function is established between the fusion data matrix and the deformation result of the ground fissure target.

[0060] Specifically, before completing step S140 and proceeding to step S150, the visually extracted features of the ground fissure edge can be spatially matched with the coordinates of key points in high-precision wireless ranging to construct an association network of "edge line - feature point" to ensure the consistency of data in physical space.

[0061] First, a fused data matrix is ​​constructed based on two-dimensional planar temporal deformation vectors, three-dimensional solid deformation feature sequences, and ranging data sequences. in, To merge the data matrix, It is a three-dimensional deformation feature sequence. It is a two-dimensional planar temporal deformation vector. This is a distance measurement data sequence.

[0062] Establish a fused data matrix Deformation results of ground fissure target Transfer function between The specific form is, in, The transformation relationship between the deformation results of the ground fissure target and the three-dimensional point cloud coordinate system formed by the three-dimensional deformation feature sequence is given. The relationship between the deformation results of the ground fissure target and the projection of the camera focal plane used to acquire the two-dimensional visible light image sequence is shown. This represents the projection relationship between the deformation result of the ground fissure target and the wireless ranging direction.

[0063] In the embodiments of this application, in step S160, the observation data conditional probability is formed based on the sensor measurement error used to acquire visual images and wireless ranging data, as well as the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence and the ranging data sequence. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

[0064] Specifically, the measurement errors of sensors used to acquire visual images and wireless ranging data, such as the measurement errors of visible light cameras, RGBD cameras, and wireless ranging modules, can be determined through prior measurements or based on historical measurement results.

[0065] Utilizing sensor measurement errors for acquiring visual images and wireless ranging data (assuming a Gaussian noise model) ), and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence. Correction of standard deviation of each measurement This forms the conditional probability of the observed data. Maximum likelihood estimation is performed on the conditional probabilities of the observed data to obtain the deformation results of the ground fissure target. The deformation results of the ground fissure target include the change in ground fissure width ΔW (mm), vertical subsidence / uplift ΔZ (mm), and horizontal displacement (ΔX, ΔY) (mm).

[0066] The specific solution process involves solving nonlinear problems, and the methods include, but are not limited to, least squares method, Newton iteration, Levenberg-Marquardt method, etc.

[0067] To obtain the deformation results of ground fissure targets, an improved Kalman filter algorithm can also be used. Specifically, time-series deformation data (such as ΔW and ΔL sequences) from visual and high-precision wireless ranging are fused in the time domain using Kalman filtering. This leverages the high sampling rate advantage of visual data and the stability advantage of high-precision wireless ranging to generate a smooth deformation time series.

[0068] In addition, after obtaining the target deformation results of the ground fissures, the target deformation results of the ground fissures can be input into the ARIMA model to obtain the ground fissure deformation prediction results, predict the future deformation trend of the ground fissures, and provide forward-looking decision support for disaster prevention and mitigation.

[0069] A spatiotemporal distribution map of ground fissure deformation can also be constructed based on the target deformation results of ground fissures, with different colors representing different ground fissure deformation intensities. For example, red areas represent high deformation zones (≥ set values, such as 2 mm / day), and blue areas represent stable zones (< set values, such as 1 mm / day).

[0070] Based on the above description, the monitoring method according to the embodiments of this application has the following beneficial effects: ① The efficiency and comprehensiveness of vision-driven systems: By integrating the advantages of high-resolution visible light cameras and RGBD cameras, it can achieve wide-range (120° field of view per camera) coverage through two-dimensional images and capture three-dimensional deformation through three-dimensional depth information. A single vision system can meet more than 80% of routine monitoring needs, and the cost is lower than the traditional multi-device independent deployment solution. ② Anti-interference capability assisted by high-precision wireless ranging module: In scenarios where vision is obstructed (such as vegetation cover) or where lighting changes drastically (such as rainstorms or strong light), the penetration (can penetrate rain, fog, and vegetation) and ranging stability of the high-precision wireless ranging module can effectively fill monitoring blind spots and improve system reliability. ③ High precision guarantee of fusion model: Through sub-pixel interpolation, 3D point cloud alignment and other technologies, the deformation monitoring accuracy can reach the millimeter level in vision-dominated scenarios; combined with the auxiliary verification of high-precision wireless ranging module, the accuracy can still be maintained within ±2mm in extreme scenarios, meeting the high precision requirements of ground fissure monitoring.

[0071] refer to Figure 2A monitoring apparatus 200 for implementing the monitoring method according to an embodiment of this application includes a processor 210 and a memory 220. The monitoring apparatus 200 may include one or more processors 210 and one or more memories 220. The memory 220 stores an executable program that is run by the processor 210. When the executable program is run by the processor 210, it causes the processor 210 to execute the monitoring method 100 described above according to an embodiment of this application.

[0072] The processor 210 may be a central processing unit (CPU) or other processing units with data processing capabilities and / or instruction execution capabilities.

[0073] The memory 220 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 210 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of this application described herein, and / or other desired functions. Various applications and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the applications.

[0074] The monitoring device 200 may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 2 The components and structure of the monitoring device 200 shown are merely exemplary and not limiting; the monitoring device 200 may also have other components and structures as needed.

[0075] The input device can be a device used by a user to input commands, and can include one or more of a keyboard, mouse, microphone, and touchscreen. Furthermore, the input device can also be any interface for receiving information.

[0076] The output device can output various information (e.g., images or sounds) to the outside (e.g., a user), and may include one or more of a display, speaker, etc. Furthermore, the output device can also be any other device with output functionality.

[0077] For example, the example monitoring device 200 for implementing the monitoring method 100 according to the embodiments of this application can be applied to terminal devices (such as mobile phones), tablet computers, laptop computers, ultra-mobile personal computers (UMPCs), handheld computers, netbooks, personal digital assistants (PDAs), wearable devices (such as smartwatches, smart glasses, or smart helmets), augmented reality (AR), virtual reality (VR) devices, smart home devices, in-vehicle computers, and other electronic devices. The embodiments of this application do not impose any limitations on this.

[0078] Those skilled in the art can understand the specific operation of the monitoring device 200 for implementing the monitoring method 100 according to the embodiments of this application in conjunction with the content described above. For the sake of brevity, the specific details will not be repeated here, but only some main operations of the processor 210 will be described.

[0079] In one embodiment of this application, when the executable program is run by the processor 210, the processor 210 performs the following steps: Acquire continuous visual images of the ground fissure area to be monitored, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence; obtain two-dimensional planar temporal deformation vectors of key points at the ground fissure edge based on the two-dimensional visible light image sequence, and obtain a three-dimensional deformation feature sequence of key points at the ground fissure edge based on the three-dimensional depth image sequence; the key points at the ground fissure edge are pre-defined targets at the ground fissure edge; acquire continuous ranging data sequences of key points at the ground fissure edge in the area to be monitored; and evaluate the results based on visual image feature matching rate, visual image sharpness, number of feature points per frame of visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, and stability of wireless ranging data. The weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence are dynamically determined based on the signal strength of the wireless ranging data. A transfer function is established between the fused data matrix constructed from the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence and the deformation result of the ground fissure target. Based on the sensor measurement errors used to acquire visual images and wireless ranging data, and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence, the conditional probability of the observation data is formed. Maximum likelihood estimation is then performed on the conditional probability of the observation data to obtain the deformation result of the ground fissure target.

[0080] The above exemplarily illustrates a monitoring method 100 according to an embodiment of this application. The following, in conjunction with... Figure 3 This application describes an electronic device 300 provided in another aspect of its embodiments.

[0081] Reference Figure 3 This describes an example electronic device 300 used to implement the monitoring method of the embodiments of this application. The electronic device 300 may include a visual image acquisition module 310, a visual deformation feature acquisition module 320, a ranging data sequence acquisition module 330, a dynamic weight determination module 340, a transfer function construction module 350, and a deformation result acquisition module 360. Wherein: The visual image acquisition module 310 is used to acquire continuous visual images of the ground fissure area to be monitored, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence.

[0082] The visual deformation feature acquisition module 320 is used to: acquire the two-dimensional planar temporal deformation vector of the key points of the ground fissure edge based on the two-dimensional visible light image sequence, and acquire the three-dimensional solid deformation feature sequence of the key points of the ground fissure edge based on the three-dimensional depth image sequence; the key points of the ground fissure edge are preset targets of the ground fissure edge.

[0083] The ranging data sequence acquisition module 330 is used to acquire a continuous ranging data sequence of key points on the edge of the ground fissure in the area to be monitored.

[0084] The dynamic weight determination module 340 is used to dynamically determine the weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data.

[0085] The transfer function construction module 350 is used to: establish a transfer function between the fused data matrix and the ground fissure target deformation result using a fused data matrix constructed from a two-dimensional planar temporal deformation vector, a three-dimensional solid deformation feature sequence, and a ranging data sequence.

[0086] The deformation result acquisition module 360 ​​is used to: form the observation data conditional probability based on the sensor measurement error used to acquire visual images and wireless ranging data, as well as the corresponding weights of the two-dimensional planar time-series deformation vector, the three-dimensional solid deformation feature sequence and the ranging data sequence, perform maximum likelihood estimation on the observation data conditional probability, and obtain the deformation result of the ground fissure target.

[0087] The electronic device 300 proposed in this application embodiment can more accurately monitor ground fissure deformation.

[0088] In addition, the following combination Figure 4 and Figure 5The monitoring system 400 provided in another aspect of this application includes a remote terminal unit 410, a visible light camera 420, an RGBD camera 430, a wireless ranging module 440, and a monitoring center 450. The monitoring system 400 of this embodiment uses a remote terminal unit (RTU) as its core node to construct a non-contact ground fissure monitoring facility system integrating data acquisition, transmission, and control, enabling fully automated monitoring throughout the entire process.

[0089] Specifically, see Figure 6 Remote terminal units are deployed around the monitored ground fissure areas according to the location of the monitoring points. Each remote terminal unit integrates a main control module and a wireless data transmission module, and is equipped with connection ports for a visible light camera, an RGBD camera, and a wireless ranging module. Based on Internet of Things (IoT) technology, the remote terminal units have the function of collecting and storing geological and environmental data, and transmitting them to a remote monitoring center via a wireless network. They can also receive instructions from the monitoring center to control the equipment on-site, achieving remote automated monitoring and management.

[0090] The Remote Terminal Unit (RTU) adopts a modular design, with the main unit 411 featuring multiple interfaces (supporting 4G / 5G / WIFI access). For example, the N-type interface 412 is used to install a data transmission antenna. Another example is the first connection port 413 (e.g., a 5-pin aviation connector), used to connect a wireless ranging module. Yet another example is the second connection port 414 (e.g., a 7-pin aviation connector), used to connect visible light cameras or RGBD cameras, depending on the monitoring point requirements. Other ports, such as 2-pin or 9-pin aviation connectors, can also be reserved for installing peripherals such as charging devices and seismic detectors. If no accessories are required, waterproof plugs are connected to the interfaces to ensure dust and water resistance.

[0091] The Remote Terminal Unit (RTU) supports manual installation, but it is recommended to use a cast metal pole with clamps to fix the device.

[0092] For example, see Figure 7 and Figure 8 With the bottom of the remote terminal unit facing upwards, place the short side of the L-shaped mounting bracket 415 on the bottom of the device, align it with the bolt mounting holes on the bottom of the device, and install the bolts and nuts (nut size: M8).

[0093] Next, align the column tightly with the outside of the L-shaped mounting bracket. Then, insert the U-shaped clamp 416 through the mounting hole of the L-shaped mounting bracket from the outside, ensuring the closed end of the U-shaped clamp is flush against the column. Secure the other end with the clip 417 and nut. If a camera is selected, align the camera with the area where the image needs to be captured and tighten the screws. If the column's dimensions are different at the top and bottom, insert another U-shaped clamp following the same steps to ensure the equipment is securely installed.

[0094] A visible light camera on a remote terminal unit is used to set monitoring point locations around the ground fissure area to be monitored, and is used to acquire a sequence of two-dimensional visible light images of the ground fissure area to be monitored.

[0095] An RGBD camera on a remote terminal unit, with monitoring points set around the area of ​​the ground fissure to be monitored, is used to acquire a sequence of three-dimensional depth images of the area of ​​the ground fissure to be monitored.

[0096] A wireless ranging module is deployed on a remote terminal unit around the area of ​​ground fissures to be monitored, with monitoring points set at their locations. This module is used to acquire a sequence of ranging data for key points at the edges of continuous ground fissures in the area to be monitored.

[0097] The monitoring center receives two-dimensional visible light image sequences, three-dimensional depth image sequences, and ranging data sequences, and performs the following processing: Two-dimensional planar temporal deformation vectors of key points at the edge of ground fissures are obtained based on two-dimensional visible light image sequences, and three-dimensional solid deformation feature sequences of key points at the edge of ground fissures are obtained based on three-dimensional depth image sequences.

[0098] The weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data.

[0099] A transfer function is established between the fused data matrix and the deformation results of the ground fissure target, constructed using a fusion data matrix consisting of a two-dimensional planar temporal deformation vector, a three-dimensional solid deformation feature sequence, and a ranging data sequence.

[0100] Based on the measurement errors of the visible light camera, RGBD camera, and wireless ranging module used to acquire visual images and wireless ranging data, as well as the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence, the observation data conditional probability is formed. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

[0101] Since most of the relevant content has already been described in monitoring method 100, it will not be described again.

[0102] Furthermore, according to embodiments of this application, this application also provides a storage medium on which a computer program is stored. When the computer program is run by a processor, it is used to execute corresponding steps of the monitoring method 100 of this application. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0103] Furthermore, according to embodiments of this application, this application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the determination method of embodiments of this application.

[0104] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of this application. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of this application. All such changes and modifications are intended to be included within the scope of this application as claimed in the appended claims.

[0105] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0107] Furthermore, those skilled in the art will understand that although some embodiments described herein include certain features but not others included in other embodiments, combinations of features from different embodiments are intended to be within the scope of this application and form different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0108] It should be noted that the above embodiments are illustrative of this application and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0109] The above description is merely a specific embodiment or illustration of the embodiments of this application. The scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. The scope of protection of this application shall be determined by the scope of the claims.

Claims

1. A non-contact ground fissure monitoring method integrating multi-source data, characterized in that, The monitoring method includes: Acquire continuous visual images of the ground fissure area to be monitored, including a two-dimensional visible light image sequence and a three-dimensional depth image sequence; Two-dimensional planar temporal deformation vectors of key points at the edge of ground fissures are obtained based on two-dimensional visible light image sequences, and three-dimensional solid deformation feature sequences of key points at the edge of ground fissures are obtained based on three-dimensional depth image sequences; the key points at the edge of ground fissures are preset targets at the edge of ground fissures. Obtain a continuous sequence of distance measurement data for key points at the edge of the ground fissure in the area to be monitored. The weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared with historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data. A transfer function between the fused data matrix and the deformation results of the ground fissure target is established by constructing a fused data matrix using two-dimensional planar temporal deformation vectors, three-dimensional solid deformation feature sequences, and ranging data sequences. Based on the sensor measurement errors used to acquire visual images and wireless ranging data, and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence, the observation data conditional probability is formed. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

2. The monitoring method according to claim 1, characterized in that, The dynamic determination of the weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared to historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data specifically refers to: When the visual image feature matching rate is greater than a set value, the weights of the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence are both configured to the first range; When the visual image feature matching rate is less than a set value, the wireless ranging data stability is less than a set value, and the wireless ranging data signal strength is greater than a set value, the weights of the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence are both configured to the second range. The specific weight values ​​in the first or second range are determined based on visual indicators whose contributions to the weights are monotonically increasing. These visual indicators include visual image sharpness, the number of feature points per frame of the visual image, and the repetition rate of matching point pairs in the visual image compared to historical matching point pairs.

3. The monitoring method according to claim 2, characterized in that, The monitoring method also includes: When the visual image feature matching rate is greater than a set value, the displacement deviation of the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence in the wireless ranging direction from the ranging data sequence is judged. If the deviation is less than the set value, the fused data vector is constructed based on the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence. If the deviation is greater than the set value, a data anomaly warning is triggered.

4. The monitoring method according to claim 2, characterized in that, When the visual image feature matching rate is less than the set value, the stability of wireless ranging data is less than the set value, and the signal strength of wireless ranging data is greater than the set value, the ranging data sequence is interpolated to the two-dimensional planar temporal deformation vector and the three-dimensional solid deformation feature sequence through the spatial correlation algorithm to ensure the continuity of visual deformation monitoring.

5. The monitoring method according to claim 1, characterized in that, The process of obtaining the ground fissure target deformation result by performing maximum likelihood estimation on the observation data conditional probability formed by the sensor measurement error used to acquire visual images and wireless ranging data, and the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional deformation feature sequence, and the ranging data sequence, based on the sensor measurement error used to acquire visual images and wireless ranging data, specifically refers to: Measurement error using pre-determined sensors for acquiring visual images and wireless ranging data And the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence. Correction of standard deviation of each measurement This forms the conditional probability of the observed data. in, To merge the data matrix, The deformation result of the ground fissure target. It is a three-dimensional deformation feature sequence. It is a two-dimensional planar temporal deformation vector. For ranging data sequences, For transfer functions, The transformation relationship between the deformation results of the ground fissure target and the three-dimensional point cloud coordinate system formed by the three-dimensional deformation feature sequence is given. The relationship between the deformation results of the ground fissure target and the projection of the camera focal plane used to acquire the two-dimensional visible light image sequence is shown. The projection relationship between the deformation result of the ground fissure target and the wireless ranging direction; Maximum likelihood estimation is performed on the conditional probability of the observed data to obtain the deformation results of the ground fissure target. 。 6. The monitoring method according to claim 1, characterized in that, The monitoring method further includes: inputting the target deformation results of the ground fissure into the ARIMA model to obtain the ground fissure deformation prediction results; Based on the target deformation results of the ground fissure, a spatiotemporal distribution map of ground fissure deformation is constructed, with different colors representing different ground fissure deformation intensities.

7. The monitoring method according to claim 1, characterized in that, The acquisition of the two-dimensional planar temporal deformation vector of key points at the edge of the ground fissure based on the two-dimensional visible light image sequence specifically refers to: Preprocessing of two-dimensional visible light image sequences includes image grayscale conversion, logarithmic transformation, and histogram equalization; The YOLO model was used to segment the ground fissure region in the preprocessed two-dimensional visible light image sequence, and key points of the ground fissure edge in the segmented region were extracted. Based on the key points of the ground fissure edge in historical images, the currently extracted key points of the ground fissure edge are matched to obtain the two-dimensional planar temporal deformation vector of the key points of the ground fissure edge. The acquisition of the three-dimensional deformation feature sequence of key points at the edge of the ground fissure based on the three-dimensional depth image sequence specifically refers to: Preprocessing is performed on the depth map in the 3D depth image sequence by bilateral filtering and Gaussian kernel weighted calculation; The preprocessed depth map is pixel-level aligned with the corresponding color image in the 3D depth image sequence. The u and v coordinates and depth values ​​of each pixel are converted into 3D spatial coordinates to generate and register 3D point cloud data of the ground fissure area. The ground fissure region was segmented based on the registered 3D point cloud data. The segmented fissure point cloud was compared point by point, and the spatial 3D displacement vector of the point cloud at adjacent time points was calculated. The 3D deformation feature sequence of key points at the edge of the ground fissure was obtained, including the uplift / settlement of the ground fissure in the vertical Z-axis, the displacement in the horizontal X and Y axes, and the volume change of the ground fissure.

8. A non-contact ground fissure monitoring device that integrates multi-source data, characterized in that, The monitoring device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the monitoring method according to any one of claims 1 to 7.

9. A storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the monitoring method according to any one of claims 1 to 7.

10. A non-contact ground fissure monitoring system that fuses multi-source data for implementing the monitoring method of any one of claims 1 to 7, characterized in that, The monitoring system includes: Remote terminal units are deployed around the ground fissure area to be monitored according to the location of the monitoring points; each remote terminal unit integrates a main control module and a wireless data transmission module, and is equipped with connection ports for a visible light camera, an RGBD camera and a wireless ranging module. A visible light camera is placed on a remote terminal unit around the ground fissure area to be monitored, with monitoring points set at the locations, to acquire a two-dimensional visible light image sequence of the ground fissure area to be monitored. An RGBD camera on a remote terminal unit, with monitoring points set around the area of ​​the ground fissure to be monitored, is used to acquire a sequence of three-dimensional depth images of the area of ​​the ground fissure to be monitored. A wireless ranging module on a remote terminal unit, which is set with monitoring point locations around the ground fissure area to be monitored, is used to acquire a continuous ranging data sequence of key points on the edge of the ground fissure in the ground fissure area to be monitored. The monitoring center receives two-dimensional visible light image sequences, three-dimensional depth image sequences, and ranging data sequences, and performs the following processing: Two-dimensional planar temporal deformation vectors of key points at the edge of ground fissures are obtained based on two-dimensional visible light image sequences, and three-dimensional solid deformation feature sequences of key points at the edge of ground fissures are obtained based on three-dimensional depth image sequences. The weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence are dynamically determined based on the visual image feature matching rate, visual image sharpness, number of feature points in a single frame of the visual image, repetition rate of visual image matching point pairs compared with historical matching point pairs, stability of wireless ranging data, and signal strength of wireless ranging data. A transfer function between the fused data matrix and the deformation results of the ground fissure target is established by constructing a fused data matrix using two-dimensional planar temporal deformation vectors, three-dimensional solid deformation feature sequences, and ranging data sequences. Based on the measurement errors of the visible light camera, RGBD camera, and wireless ranging module used to acquire visual images and wireless ranging data, as well as the corresponding weights of the two-dimensional planar temporal deformation vector, the three-dimensional solid deformation feature sequence, and the ranging data sequence, the observation data conditional probability is formed. The observation data conditional probability is then estimated by maximum likelihood to obtain the deformation result of the ground fissure target.

Images