Tailing dam surface deformation patrol method based on rail-mounted robot

Abstract

Disclosed in the present invention is a tailing dam surface deformation patrol method based on a rail-mounted rail robot, comprising: collecting data of the surface of a tailing pond dam body, and pre

Description

Tailings dam surface deformation inspection method based on rail-mounted robot Technical Field

[0001] The present invention belongs to the field of deformation inspection, and in particular relates to a tailings dam surface deformation inspection method based on a rail-mounted robot. Background Art

[0002] Currently, rail-mounted robots are widely used in workshops and other enclosed environments. They use cameras to capture images and process the captured two-dimensional images to monitor and detect abnormalities. Existing technologies are limited in that they can only capture basic two-dimensional image data and process the image data simply on a two-dimensional plane, failing to detect specific deformation changes in abnormal areas.

[0003] Summary of the Invention

[0004] To solve the above problems, the present invention provides a tailings dam surface deformation inspection method based on a track-mounted robot, comprising:

[0005] Collecting data on the surface of the tailings dam, pre-processing the data, and obtaining three-dimensional monitoring data;

[0006] Performing modeling based on the three-dimensional monitoring data to obtain a three-dimensional monitoring model;

[0007] The three-dimensional monitoring model is processed and analyzed to obtain the deformation of the dam body.

[0008] Preferably, the process of collecting data on the surface of the tailings dam includes collecting data on the tailings dam based on a track slide robot, obtaining image data through the binocular camera of the track slide robot, and obtaining three-dimensional point cloud data through the lidar of the track slide robot.

[0009] Preferably, the track slide robot is obtained by building a data acquisition layer, a host computer monitoring layer, a communication layer, and a control layer; the data acquisition layer is used to collect the image data and the three-dimensional point cloud data; the host computer monitoring layer is used to view the position information of the track slide robot and the operating status of the equipment in real time; the communication layer is used to transmit data with the host computer; the control layer is remotely controlled by the host computer and is used to control the operation of the motor of the equipment.

[0010] Preferably, the process of preprocessing the data includes denoising the image data based on superpixels and intelligent optimization methods, and performing data fusion processing based on the denoised image data and the three-dimensional point cloud data to obtain the three-dimensional monitoring data.

[0011] Preferably, the process of denoising the image data based on superpixels and intelligent optimization methods includes using a mask to scan each pixel in the image through Gaussian filtering, replacing the value of the center pixel of the mask with the weighted average grayscale value of the pixels in the mask neighborhood to obtain denoised image data.

[0012] Preferably, the data fusion processing includes processing the image data and the three-dimensional point cloud data based on the LVI-SAM algorithm to generate a three-dimensional point cloud of the tailings dam body at each moment after two-phase calibration.

[0013] Preferably, the process of modeling based on the three-dimensional monitoring data to obtain a three-dimensional monitoring model includes: obtaining a monitoring image based on the three-dimensional monitoring data, and preprocessing the monitoring image using a Sobel operator; performing cost calculation on the preprocessed image; performing dynamic planning on the image after cost calculation, and obtaining a disparity map through stereo calibration and stereo matching; obtaining a depth map based on the disparity map; drawing a point cloud map based on the depth map; and constructing the three-dimensional monitoring model based on the point cloud map.

[0014] Preferably, the process of processing and analyzing the three-dimensional monitoring model includes importing the three-dimensional point clouds at different times into the three-dimensional monitoring model for point cloud alignment, and calculating the distance between the front and rear point clouds after alignment; visualizing the difference in the distance between the front and rear point clouds through a preset threshold to obtain a visualization result; calculating the area and volume of the three-dimensional monitoring model, and obtaining the deformation of the dam body based on the area and volume and the visualization result.

[0015] Preferably, the process of importing three-dimensional point clouds at different times into the three-dimensional monitoring model for point cloud alignment includes calculating the chamfer distance between the three-dimensional point clouds, and comparing and aligning the generated point cloud with the original point cloud based on the chamfer distance.

[0016] Preferably, the generated point cloud is compared and aligned with the original point cloud based on the chamfer distance by an ICP point cloud registration algorithm;

[0017] The process of the ICP point cloud registration algorithm includes preprocessing the three-dimensional point cloud to obtain the original transformation; matching the original transformation to obtain the nearest point; adjusting the weights of corresponding point pairs by weighting to eliminate unreasonable corresponding point pairs; minimizing the loss by calculating the loss; and obtaining the optimal transformation based on the minimized loss.

[0018] The present invention discloses the following technical effects:

[0019] Compared with many existing methods used on the market to collect large-scale outdoor geological image information using drones, the single operation process is complicated, time-consuming and labor-intensive, and the drone needs to be operated each time. The method of constructing a track robot track in this application has the advantage of being a one-time setup and can run automatically for a long time.

[0020] The present invention provides a tailings dam surface deformation inspection method based on a track-mounted robot. The monitored subject is the entire tailings dam body. The constructed track environment is outdoors. By carrying a binocular camera and a laser radar on the robot to scan the entire dam body, a three-dimensional point cloud is generated, and the deformation and properties of the tailings dam surface, including the specific volume, position and other information of the deformation, are captured, thereby realizing three-dimensional monitoring and reconstruction of the tailings dam surface deformation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] FIG1 is a flow chart of a method according to an embodiment of the present invention;

[0023] FIG2 is a diagram of a device model according to an embodiment of the present invention;

[0024] FIG3 is a schematic diagram of a track structure according to an embodiment of the present invention;

[0025] FIG4 is an example diagram of a device information capture area according to an embodiment of the present invention;

[0026] FIG. 5 is a schematic diagram showing the overlap rate of photos taken at the same horizontal position by different classes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0028] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] As shown in FIG1 , the present invention provides a method for inspecting surface deformation of a tailings dam based on a track-mounted robot, comprising:

[0030] (1) Acquisition of three-dimensional monitoring data

[0031] To address the limited coverage of GNSS and inclinometer monitoring of the tailings dam and the insufficient accuracy of drone inspections, we plan to select three track-and-slide robots equipped with binocular cameras and lidar to conduct online monitoring of the entire tailings dam (as shown in Figure 2). The robots will fuse multi-source data, including visual data captured by the binocular cameras and laser point clouds generated by the lidar. This means collecting image data of the tailings dam surface and 3D point clouds obtained by laser scanning.

[0032] When conducting full-area online monitoring of a tailings dam, data collection and track construction are efficiently combined (as shown in Figure 3). A stepped track for the track-mounted robot is constructed around the monitoring target. The track has a built-in power supply circuit to ensure the robot's normal operation. This stepped arrangement allows the equipment to collect information in a more comprehensive area. While the data collection equipment normally maintains its operating orientation, this can be adjusted to meet specific needs by changing the track's direction (as shown in Figure 4).

[0033] In particular, for large-scale acquisition tasks, the track hierarchy construction must comply with the image acquisition requirements, that is, the overlap rate of photos taken at the same horizontal position in different levels must reach 50% (as shown in Figure 5).

[0034] Regarding the control of the rail-slide robot, a system framework consisting of a host computer monitoring layer, communication layer, control layer, and data acquisition layer is planned, along with the development of a remote interactive inspection control system. The host computer monitoring layer provides real-time monitoring of the robot's specific location and the operating status of related equipment, providing real-time feedback to the host computer via the communication layer. The communication layer acts as a data transmission device between the device and the host computer. The control layer, remotely controlled by the host computer, controls the device's motors. The data acquisition layer collects image data using a camera and 3D point cloud data using a lidar radar. Specifically, users can flexibly control the rail-slide robot's operating speed based on pre-set acquisition areas.

[0035] We independently developed a dual-loop track inspection robot for the tailings dam, driven by a high-speed servo motor, to assist in the initial collection of monitoring data. Specifically, we installed a supplementary light on the track-guided robot to compensate for the loss of vision at night.

[0036] (2) Generation of 3D monitoring model

[0037] In terms of vision, we first perform image denoising on the collected images through methods such as superpixels and intelligent optimization to obtain more accurate image data.

[0038] Gaussian filtering is used for denoising. The specific operation is: use a template (or convolution, mask) to scan each pixel in the image, and use the weighted average grayscale value of the pixels in the neighborhood determined by the template to replace the value of the center pixel of the template.

[0039] Then, the image data is preprocessed using the SGBM algorithm to establish a three-dimensional monitoring model of the tailings pond, and its three-dimensional monitoring data is stored in the database.

[0040] The lidar will integrate the collected radar data with the image data collected by the binocular camera. The specific method is to use the LVI-SAM algorithm to process the image data and radar data to generate a three-dimensional point cloud data of the entire tailings dam body after two-phase calibration.

[0041] The specific construction method of the tailings pond three-dimensional monitoring model is to use the Sobel operator to preprocess the image; perform cost calculation on the preprocessed image; perform dynamic planning on the image after cost calculation, and obtain a disparity map through stereo calibration and stereo matching; obtain a depth map based on the disparity map; draw a point cloud map based on the depth map; and construct the three-dimensional monitoring model based on the point cloud map.

[0042] (3) Processing and analysis of the 3D monitoring model

[0043] Select the 3D monitoring model or 3D point cloud generated at different times before and after and import it into CloudCompare for point cloud alignment. After alignment, calculate the distance between the point clouds before and after. By selecting an appropriate threshold, visualize the difference between the point clouds before and after. Then, by further calculating the area and volume of the 3D model, you can further understand the deformation of the dam body and set the alarm threshold based on this.

[0044] The point cloud alignment process involves calculating the chamfer distance between 3D point clouds, that is, calculating the average shortest point distance between the generated point cloud and the ground truth point cloud. The difference between the generated point cloud and the original point cloud is determined by the size of the chamfer distance.

[0045] The chamfer distance is used to compare the generated point cloud with the original point cloud and align them through point cloud registration.

[0046] Point Cloud Registration refers to inputting two point clouds (source) and (target) and outputting a transformation so that the degree of overlap between (source) and (target) is as high as possible. The present invention only considers rigid transformations, that is, the transformation only includes rotation and translation. Point Cloud Registration can be divided into two steps: coarse registration and fine registration. Coarse registration refers to a relatively rough registration when the transformation between the two point clouds is completely unknown. The main purpose is to provide a better initial value of the transformation for fine registration; the fine registration criterion is to give an initial transformation and further optimize it to obtain a more accurate transformation.

[0047] Currently, the most widely used point cloud registration algorithm is the Iterative Closest Point (ICP) algorithm and its various variants. The present invention performs point cloud alignment based on the ICP algorithm.

[0048] For the case where T is a rigid transformation, the point cloud registration problem can be described as:

[0049] Here s and p t are corresponding points in the source and target point clouds.

[0050] The general algorithm flow of ICP is as follows: preprocess the 3D point cloud to obtain the original transformation; match the original transformation to obtain the nearest point; adjust the weights of corresponding point pairs through weighting and eliminate unreasonable corresponding point pairs; minimize the loss by calculating the loss; obtain the optimal transformation based on the minimized loss; iterate the above steps until convergence.

[0051] The contents described in the examples of this specification are merely an enumeration of the implementation forms of the inventive concept. The scope of protection of the present invention should not be regarded as limited to the specific forms described in the examples. The scope of protection of the present invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A tailings dam surface deformation inspection method based on a rail-mounted robot, characterized in that: include: Collecting data on the surface of the tailings dam, pre-processing the data, and obtaining three-dimensional monitoring data; Performing modeling based on the three-dimensional monitoring data to obtain a three-dimensional monitoring model; The three-dimensional monitoring model is processed and analyzed to obtain the deformation of the dam body.

2. The tailings dam surface deformation inspection method based on the rail-mounted robot according to claim 1 is characterized in that: include: The process of collecting data on the surface of the tailings dam includes collecting data on the tailings dam based on a track slide robot, obtaining image data through the binocular camera of the track slide robot, and obtaining three-dimensional point cloud data through the laser radar of the track slide robot.

3. The tailings dam surface deformation inspection method based on the rail-mounted robot according to claim 2 is characterized in that: include: The rail slide robot is obtained by building a data acquisition layer, a host computer monitoring layer, a communication layer, and a control layer; The data acquisition layer is used to acquire the image data and the three-dimensional point cloud data; The host computer monitoring layer is used to view the position information of the track slide robot and the operating status of the equipment in real time; The communication layer is used to transmit data to the host computer; The control layer is remotely controlled by the host computer and is used to control the operation of the motor of the device.

4. The tailings dam surface deformation inspection method based on the rail-mounted robot according to claim 2 is characterized in that: include: The process of preprocessing the data It includes denoising the image data based on superpixels and intelligent optimization methods, and performing data fusion processing based on the denoised image data and the three-dimensional point cloud data to obtain the three-dimensional monitoring data.

5. The tailings dam surface deformation inspection method based on the rail-mounted robot according to claim 4 is characterized in that: include: The process of denoising the image data based on superpixels and intelligent optimization methods includes: using a mask to scan each pixel in the image through Gaussian filtering, replacing the value of the center pixel of the mask with the weighted average grayscale value of the pixels in the mask neighborhood, and obtaining denoised image data.

6. The tailings dam surface deformation inspection method based on a rail-mounted robot according to claim 4 is characterized in that: include: The data fusion processing includes processing the image data and the three-dimensional point cloud data based on the LVI-SAM algorithm to generate a three-dimensional point cloud of the tailings dam body at each moment after two-phase calibration.

7. The method for inspecting surface deformation of a tailings dam based on a track-mounted robot according to claim 1, characterized in that: include: Modeling is performed based on the three-dimensional monitoring data to obtain a three-dimensional monitoring model, which includes: Obtaining a monitoring image based on the three-dimensional monitoring data, and preprocessing the monitoring image using a Sobel operator; Perform cost calculation on the preprocessed image; Cost calculation The calculated image is dynamically planned, and the disparity map is obtained through stereo calibration and stereo matching; Obtaining a depth map based on the disparity map; and drawing a point cloud map based on the depth map; Based on the point cloud image, the three-dimensional monitoring model is constructed.

8. The tailings dam surface deformation inspection method based on a track-mounted robot according to claim 1 is characterized in that: include: The process of processing and analyzing the three-dimensional monitoring model includes importing the three-dimensional point clouds at different times into the three-dimensional monitoring model to perform point cloud alignment, and calculating the distance between the front and rear point clouds after alignment; Visualizing the distance difference between the front and rear point clouds using a preset threshold to obtain a visualization result; The area and volume of the three-dimensional monitoring model are calculated, and the deformation of the dam body is obtained based on the area and volume and the visualization result.

9. The tailings dam surface deformation inspection method based on a track-mounted robot according to claim 8, characterized in that: include: The process of importing 3D point clouds at different times into the 3D monitoring model for point cloud alignment The method includes calculating the chamfer distance between three-dimensional point clouds, and comparing and aligning the generated point cloud with the original point cloud based on the chamfer distance.

10. The tailings dam surface deformation inspection method based on a track-mounted robot according to claim 9, characterized in that: include: Based on the chamfer distance, the generated point cloud is compared and aligned with the original point cloud through the ICP point cloud registration algorithm; The ICP point cloud registration algorithm includes pre-processing the three-dimensional point cloud to obtain the original transformation; matching the original transformation to obtain the closest point; adjusting the weights of corresponding point pairs by weighting to eliminate unreasonable corresponding point pairs; By calculating the loss, Obtain the minimized loss; based on the minimized loss, obtain the optimal transformation.

Images