Unmanned inspection method and system for open-air tailing pond

Abstract

The invention discloses an unmanned inspection method and system for an open-air tailing pond, is applied to the technical field of tailing pond safety monitoring, and aims to solve the problems that an existing open-air tailing pond is low in inspection efficiency, inaccurate in risk management and control, poor in equipment collaboration and not in place in implementation of a list system. According to the method, 'water management, slope protection, well observation and urgency 'are taken as the core, an unmanned inspection hardware system is constructed in combination with an unmanned aerial vehicle and a track robot, 'key dynamic-important steady-state-basic guarantee' three-level inspection indexes are designed based on a list system, a threshold value is quantized, monitoring data are processed through a multi-mode AI recognition algorithm, and the monitoring accuracy is improved. And formulating an equipment cooperation time sequence to realize data linkage, and finally forming a system logic closed loop of collection, research and judgment, alarm, disposal, recheck and archiving. According to the invention, full-scene, high-precision and automatic inspection of the open-air tailings pond can be realized, and the safety management and control efficiency and the emergency response speed are improved.

Description

Technical Field

[0001] This invention belongs to the field of safety monitoring and unmanned inspection technology, and relates to unmanned inspection technology and system for tailings dams. Background Technology

[0002] As an important supporting facility for non-coal mines, the safe operation of open-pit tailings dams directly affects the safety of the surrounding environment and personnel. The following problems exist in the current tailings dam inspections:

[0003] 1. Limitations of manual inspection: Tailings ponds are large and have complex environments (such as steep slopes and toxic gases), making manual inspection inefficient, risky, and difficult to achieve 24-hour continuous monitoring.

[0004] 2. Poor equipment coordination: Existing monitoring equipment (such as fixed sensors and drones) mostly operate independently, with data not linked and lacking a unified coordination sequence, making it impossible to form a full-coverage monitoring system covering "points, lines, and surfaces";

[0005] 3. Inadequate implementation of the list system: Although there are requirements for a safety production list system, most of the list indicators are qualitative descriptions, lack quantitative thresholds, and are not integrated with the monitoring process of unmanned equipment, resulting in poor effectiveness in "focusing on key points and controlling critical aspects".

[0006] 4. Insufficient AI recognition accuracy: Existing monitoring systems use simple algorithms to identify key risks such as dam cracks and displacements, which are easily affected by lighting and weather, resulting in a high false alarm rate;

[0007] 5. Incomplete system closed loop: monitoring data is disconnected from the handling process, there is a lack of rapid verification and closed-loop tracking mechanisms after abnormal alarms, and emergency response is delayed.

[0008] In summary, there is currently a lack of an accurate and efficient inspection solution. Summary of the Invention

[0009] To address the aforementioned technical challenges, this invention combines the hardware advantages of drones and rail robots, integrates a list-based management approach, designs a multimodal AI recognition algorithm and equipment collaborative timing sequence, and constructs a complete closed-loop unmanned inspection system, thus resolving the aforementioned technical pain points.

[0010] The technical solution adopted in this invention is: a method for unmanned inspection of open-pit tailings dams, comprising:

[0011] S1. Establish a communication network for the tailings dam. The communication network includes base stations and tracked robots and drones that communicate with the base stations. Specifically, several base stations are arranged around the tailings dam so that the communication range of these base stations covers the tailings dam area.

[0012] The deployment locations of the track-based robots include the track on the top of the tailings dam and the tracks on both sides of the drainage ditch, where they conduct low-speed inspections; the take-off and landing points of the drones are set in flat areas at the edge of the tailings dam area.

[0013] S2. Data collection is conducted using a track-mounted robot and a drone. Specifically: the track-mounted robot collects visible light images and a first infrared image using a configured dual-spectrum gimbal camera; the track-mounted robot collects real-time positioning data using a configured BeiDou satellite navigation system; the drone collects two-dimensional images of the dam slope from two global perspectives using a configured binocular camera; and the drone collects a second infrared image using a configured infrared thermal imager.

[0014] S3, the track robot, and the drone will each upload the collected data to the cloud platform via the base station;

[0015] S4. The cloud platform identifies dam surface displacement, dam slope cracks, reservoir water level and dry beach length, and personnel violations based on the uploaded data.

[0016] S5. Issue an early warning based on the identification results of step S4.

[0017] The second technical solution adopted in this invention is: an unmanned inspection system for open-pit tailings dams, comprising: a data acquisition module, a data processing module, and an early warning module;

[0018] The data acquisition module includes: a base station, a tracked robot and a drone that communicate with the base station; the base station is arranged around the tailings dam and its communication range covers the tailings dam area; the tracked robot is deployed on the track on the top of the tailings dam and on both sides of the drainage ditch, and performs low-speed inspection along the track on the dam slope; the take-off and landing point of the drone is set in a flat area at the edge of the tailings dam area.

[0019] The tracked robot acquires visible light images and first infrared images based on a configured dual-spectrum gimbal camera; the tracked robot acquires real-time positioning data based on a configured BeiDou satellite navigation system; the UAV acquires two-dimensional images of the dam slope from two global perspectives based on a configured binocular camera; the UAV acquires second infrared images based on a configured infrared thermal imager.

[0020] The track robot and the drone will each upload the collected data to the data processing module via the base station;

[0021] The data processing module includes a dam surface displacement identification unit, a dam slope crack identification unit, a reservoir water level and dry beach length identification unit, and a personnel violation identification unit.

[0022] The early warning module performs early warning processing based on the output results of the data processing module.

[0023] The dam surface displacement identification unit includes:

[0024] The first acquisition subunit acquires the aligned visible light image, the first infrared image, and the second infrared image;

[0025] The second acquisition subunit acquires the result of mapping the second infrared image output by the first acquisition subunit onto the robot's viewpoint;

[0026] The third acquisition subunit acquires the result obtained by weighted fusion of the visible light image and the first infrared image output by the first acquisition subunit and the result output by the second acquisition subunit;

[0027] The fourth acquisition sub-unit obtains the dam surface displacement result based on the output of the third acquisition sub-unit.

[0028] The dam slope crack identification unit includes:

[0029] The fifth acquisition subunit acquires the result of fusing the first infrared image output by the first acquisition subunit with the second infrared image output by the second acquisition subunit;

[0030] The sixth acquisition subunit acquires the RGB three-channel image of the visible light image output by the first acquisition subunit, which is used as the first channel image, the second channel image, and the third channel image, respectively, and acquires the result output by the fifth acquisition subunit as the fourth channel image.

[0031] The improved U-Net network includes an input layer, a downsampling encoder, a bottleneck layer, an upsampling encoder, and an output layer. The input layer receives a 4-channel image. The downsampling encoder extracts deep features from the input 4-channel image by downsampling through several cascaded convolutional blocks. The bottleneck layer extracts crack features from the deep features of the input. The upsampling encoder restores the image size step by step through several cascaded deconvolutional blocks. The output layer outputs a crack mask map.

[0032] The seventh acquisition subunit trains the improved U-Net network based on the 4-channel image dataset with labeled crack information, and obtains the trained improved U-Net network.

[0033] The eighth acquisition subunit inputs the first, second, third, and fourth channel images obtained by the sixth acquisition subunit into the trained improved U-Net network to obtain crack information.

[0034] The beneficial effects of this invention are as follows: This invention combines images collected by a tracked robot and images collected by a drone to identify dam surface displacement, dam slope cracks, reservoir water level and dry beach length, and personnel violations; it can efficiently and accurately complete inspection tasks; the method and system of this invention are applicable to the daily safety management and emergency response of open-pit tailings ponds, and are particularly in line with the requirements of "controlling water, protecting slopes, monitoring wells, and responding well to emergencies" for safe flood control of tailings ponds. Attached Figure Description

[0035] Figure 1 This is a schematic diagram illustrating the implementation process of the inspection method of the present invention;

[0036] Figure 2 This is a flowchart of the dam surface displacement identification process of the present invention;

[0037] Figure 3 This is a flowchart of the dam slope crack identification process of the present invention;

[0038] Figure 4 This is a diagram of the improved U-Net network structure of the present invention;

[0039] Figure 5 This is a flowchart of the reservoir water level identification process of the present invention;

[0040] Figure 6 This is a flowchart of the dry beach length identification process of the present invention;

[0041] Figure 7 This is a flowchart of the personnel violation identification process of the present invention. Detailed Implementation

[0042] To facilitate understanding of the technical content of this invention by those skilled in the art, the following description, in conjunction with the accompanying drawings, further illustrates the invention.

[0043] This invention provides a method for inspecting open-pit tailings ponds, such as... Figure 1 As shown, a data acquisition and transmission system is constructed based on a tracked robot, a drone, and a base station. The data collected by the tracked robot and the drone is uploaded to a cloud platform through the base station and processed by the cloud platform. The number of base stations is determined based on the area of ​​the open-pit tailings dam and is evenly distributed around the open-pit tailings dam, with its communication range covering the entire open-pit tailings dam. It is used for data communication with the tracked robot and the drone.

[0044] The deployed positions of the tracked robot include the track on the top of the tailings dam and the tracks on both sides of the drainage ditch. It performs low-speed inspections along the dam slope track, typically set to 0.2 m / s in practical applications. It is equipped with a dual-spectrum gimbal camera (2688×1520 visible light, 384×288 thermal imaging), BeiDou satellite navigation (domestic high-precision positioning), a gas sensor (for monitoring toxic and flammable gases), and a lidar obstacle avoidance module. The dual-spectrum gimbal camera simultaneously acquires visible light and infrared images. The gas sensor is used for gas monitoring and early warning of the tailings dam. Simultaneously, the cloud platform stores historical gas concentration data. Data analysis identifies gas leakage patterns (such as abnormal concentrations in specific dam sections or time periods), assisting in the assessment of the geological stability of the tailings dam (gas leaks may be associated with dam cracks and seepage).

[0045] The drone is an industrial-grade multi-rotor drone equipped with a high-definition binocular camera (20-megapixel resolution), an infrared thermal imager (640×512 resolution), and millimeter-wave radar (resistant to fog and rain interference). It has a flight endurance of ≥60 minutes, can withstand winds of ≥6 on the Beaufort scale, and its take-off and landing point is located in a flat area at the edge of the reservoir. The binocular camera and infrared thermal imager acquire global two-dimensional images of the dam slope, as well as infrared images. The millimeter-wave radar is used to improve the drone's obstacle avoidance capabilities, enhancing the reliability and safety of the inspection.

[0046] The track-mounted robots and drones complete data collection work daily according to the inspection sequence shown in Table 1:

[0047] Table 1 Inspection Sequence

[0048]

[0049] like Figure 1 As shown, this invention identifies dam surface displacement, dam slope cracks, reservoir water level and dry beach length, and personnel violations based on the collected data.

[0050] 1. For example Figure 2 As shown, this invention identifies dam surface displacement based on visible light images, infrared images acquired by a tracked robot, and infrared images acquired by a drone. The identification method includes the following steps:

[0051] A1. Record the infrared image acquired by the track robot as the first infrared image, and the infrared image acquired by the drone as the second infrared image;

[0052] The visible light image, the first infrared image, and the second infrared image acquired by the orbital robot are aligned using BeiDou timestamps;

[0053] A2. The second infrared image aligned in step A1 is mapped to the robot's viewpoint using GPS (Global Positioning System) to ensure pixel alignment. Mapping the second infrared image aligned in step A1 to the robot's viewpoint using GPS is a known existing technology and will not be described in detail here.

[0054] A3. The visible light image, the first infrared image, and the second infrared image processed in step A2 after alignment in step A1 are fused together to obtain a fused image. In the specific fusion process, the visible light image is weighted during the day and the first and second infrared images are weighted at night according to the different lighting conditions during the day and night, respectively, to ensure that the fused image retains more detailed features.

[0055] Here, daytime and nighttime are confirmed by the average brightness of visible light images (≥50 = daytime, <50 = nighttime) combined with BeiDou timestamps (6:00-18:00 = daytime). For the weighted values, daytime visible light has the highest resolution, resulting in clearer textures of displacement feature points on the dam surface, thus receiving the highest weight. Secondly, the infrared images from the orbital robot, acquired at close range, allow observation of temperature anomalies within the dam, thus receiving the second highest weight. The UAV, primarily used for global thermal imaging and observing long cracks, has the lowest weight. In practical applications, the daytime visible light weight is 0.7, the orbital robot weight is 0.2, and the UAV infrared weight is 0.1; the nighttime visible light weight is 0.1, the orbital robot weight is 0.6, and the UAV infrared weight is 0.3.

[0056] A4. Use the SIFT (Scale Invariant Feature Transform) algorithm to extract feature points from the fused image and calculate the displacement of feature points between adjacent frames. Pixel scale correction based on BeiDou positioning To obtain the actual displacement If the displacement exceeds 2 mm / d, an alarm will be triggered.

[0057] 2. For example Figure 3 As shown, dam slope crack identification is performed based on visible light images and infrared images acquired by a tracked robot, as well as infrared images acquired by a drone. The identification method includes the following steps:

[0058] B1. Data linkage acquisition and preprocessing: The robot conducts low-speed inspections along the dam slope track, acquiring visible light and infrared images; the drone simultaneously takes aerial photos of the entire dam slope in infrared.

[0059] The three types of images are registered, cropped to a uniform size, and then stitched together to form a 4-channel image (visible light + dual infrared). These three types of images include: visible light images acquired by the orbiting robot, infrared images acquired by the orbiting robot, and global infrared images of the dam slope simultaneously captured by the UAV. The registration and cropping to a uniform size here can be understood as the processing steps A1-A2 above.

[0060] The RGB three channels of the UAV visible light image processed in steps A1-A2 are respectively used as channels 1-3. Then, the UAV infrared and the orbital robot infrared are fused by feature fusion (such as weighted fusion) to form a single channel, and finally a 4-channel image is formed.

[0061] B2. Improve the U-Net network:

[0062] like Figure 4 As shown, the improved U-Net network of this invention includes an input layer, a downsampling encoder, a bottleneck layer, an upsampling encoder, and an output layer;

[0063] The input layer receives 4-channel images as the initial input to the network.

[0064] The downsampling encoder consists of four cascaded downsampling coding units. Each downsampling coding unit includes a convolutional block and a max pooling layer. Each convolutional block includes a 3×3 convolution, batch normalization (BN), and ReLU activation function in sequence. The downsampling encoder extracts deep features of the image by downsampling step by step through the four cascaded convolutional blocks.

[0065] The bottleneck layer includes an attention module (CBAM, Convolutional Block Attention Module) and a multi-scale fusion module. The attention module (CBAM) performs channel + spatial attention calculation on the feature map output by the encoder, automatically focusing on the crack region and weakening the background. The multi-scale fusion module is used to fuse the features output by the attention module, taking into account both the fine crack features collected by the robot and the long crack features collected by the UAV.

[0066] The upsampling decoder includes four cascaded upsampling decoding units. Each upsampling decoding unit includes a deconvolution block and a convolution block. Each deconvolution block includes a 2×2 transposed convolution and a 3×3 convolution. The upsampling decoder restores the image size step by step through the four deconvolution blocks and stitches downsampling features to preserve details.

[0067] Output layer: The number of channels is compressed by 1×1 convolution, and then the output is activated by the Sigmoid function; the input is H×W×64, and the output is a crack mask map of H×W×1.

[0068] B3. Model Training and Hardware Adaptation: An improved U-Net network is trained to adapt for dam slope crack detection, improving accuracy. The trained improved U-Net network is compressed and installed into the edge computing module of the robot / drone to ensure real-time processing during inspections.

[0069] The training dataset used in the improved U-Net network training process is a labeled 4-channel image dataset, which mainly includes image data, labeled data, and metadata. The image data is the visible light image of the tracked robot, the infrared image of the tracked robot, and the infrared image of the UAV; the labeled data is the crack annotation data of the 4-channel images; and the metadata includes the image acquisition time, dam slope location, and robot / UAV parameters.

[0070] The U-Net network is improved to a supervised learning model. During training, the loss is calculated using the "crack mask output by the model" and the "human-annotated real mask", and the network parameters are optimized through backpropagation. In this embodiment, a hybrid loss function consisting of DiceLoss and binary cross-entropy loss is used to solve the problem of low crack pixel ratio. The AdamW optimizer is used to update the learning rate through gradient backpropagation, and the learning rate is gradually decayed. Training is stopped when the validation set metric does not improve for 15 consecutive rounds, or the maximum number of training rounds is reached, or the accuracy on the test set meets the target.

[0071] In this embodiment, the background image without cracks is labeled with a mask of all 0s to train the model's background discrimination ability and avoid false detections.

[0072] B4. Crack Parameter Calculation and Output: Improve the U-Net network for image semantic segmentation to generate crack region masks. Through connected component analysis of the masks, combined with GPS positioning, accurately mark the specific location of the cracks on the dam slope. Calculate the average width w based on the number of mask pixels and the scale calibrated by BeiDou. If w ≥ 5mm, immediately trigger an audible and visual alarm, and simultaneously upload the crack image, location, and width data to the backend.

[0073] 3. For example Figure 5 , Figure 6 As shown, this invention identifies reservoir water levels based on water level gauge images collected by UAVs, and identifies dry beach length based on two-dimensional images of the dam slope from two global perspectives collected by a binocular camera.

[0074] Reservoir water level: Canny edge detection extracts the water level gauge scale lines, combined with the zero point elevation of the water level gauge. Calculate the actual water level The water level gauge image is obtained based on the visible light camera of the UAV. The specific water level gauge reading identification is a known existing technology. For reference, please refer to the patent application with application number CN202411172260.7. This invention will not be described in detail here.

[0075] Dry beach length: Images of the dry beach are acquired using a binocular camera, and the actual distance is calculated using the parallax d. ,and In comparison, f is the camera focal length, and B is the binocular baseline distance. The specific calculation process for the dry beach length is as follows:

[0076] 1) The drone is equipped with a binocular camera to take pictures of the tailings dam slope and dry beach area, and simultaneously acquire two-dimensional images from two perspectives. The two images belong to the global dam slope scene at the same time and on the same target surface.

[0077] 2) Preprocess the dual-view images and distinguish the dry beach areas in the images by feature point matching;

[0078] 3) Based on the matched feature points of the dry beach area, calculate the disparity and combine it with the binocular camera parameters to finally calculate the actual length of the dry beach. Disparity acquisition steps: First, calibrate the binocular camera to obtain intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (relative position and orientation of the left and right cameras), complete image distortion correction, and ensure that the left and right images are in the same epipolar plane; based on the two-dimensional images from the left and right perspectives, extract the same dry beach feature points, match the feature points at the horizontal coordinates Xleft and Xright of the left and right images respectively, then the disparity d = Xleft - Xright.

[0079] 4. For example Figure 7 As shown, this invention identifies personnel violations based on video streams acquired by drones:

[0080] Input: Drone video stream (covering the dam body and surrounding flood discharge facilities);

[0081] Processing: The YOLOv8 algorithm detects the head region of the person and determines whether a safety helmet bounding box exists;

[0082] Output: Location of the violator, with an identification accuracy of ≥95%.

[0083] The video stream here should be understood as the continuous frame output of the drone's visible light camera. .

[0084] Based on the identification results, the response procedures are as shown in Table 2.

[0085] Table 2 Abnormal Response Timing

[0086]

[0087] In this embodiment, the track robot has an IP56 protection rating and can adapt to working temperatures from -20℃ to 50℃; its walking speed is 0-1m / s, its maximum climbing angle is 20°, its turning radius is 1m, and it supports contact charging and manual charging via aviation interface.

[0088] In this embodiment, 3-5 4G / 5G / WiFi base stations are deployed around the tailings dam to ensure real-time data interaction between the unmanned equipment and the platform, with a transmission rate of ≥10Mbps and a latency of ≤50ms.

[0089] Taking a Class II open-pit tailings dam (designed dam height 30m, dam crest elevation 35m, sandy dam body, local rainstorm parameters A=25, B=12, n=0.65) as an example, the implementation process of this invention is illustrated:

[0090] (a) Inspection checklist and threshold setting

[0091] 1. Key dynamic indicator thresholds:

[0092] reservoir water level

[0093] Dry beach length

[0094] Immersion line

[0095] 1 hour rainfall

[0096] Surface displacement rate

[0097] 2. Thresholds for important steady-state indicators:

[0098] The external slope ratio of the dam is designed to be 1:2.5, with a threshold ratio of 1:2.375 to 1:2.625.

[0099] The flood drainage system has an integrity rate of ≥98%;

[0100] 3. Basic guarantee indicator thresholds:

[0101] Emergency response plan revision cycle ≤ 1 year;

[0102] Emergency supplies (such as sandbags and water pumps) have an integrity rate of ≥95%.

[0103] (ii) Hardware deployment

[0104] 1. Tracked robots: Two tracked robots are installed along the dam crest (300m in length) and the east side drainage ditch (200m in length), with a spacing of 150m between them;

[0105] 2. Drones: A take-off and landing point will be set up in the flat area on the west side of the reservoir area, and one industrial-grade drone (with a flight time of 60 minutes, binocular camera + infrared thermal imager) will be configured.

[0106] 3. Base stations: Three 5G base stations will be deployed, located at both ends of the dam crest and near the take-off and landing points, to ensure full signal coverage.

[0107] (III) Implementation of daily inspections

[0108] 1. Daily 8:00: Track robot self-check (battery power 90%, sensors normal), drone flight path planning (dam slope + dry beach + flood drainage facilities).

[0109] 2.8:10: The track robot starts, and the first reservoir water level is collected at 8:30 (33.5m, normal).

[0110] 3.8:30: The drone took off and completed the inspection at 9:30, uploading the dry beach length data (38m, ≥36m, normal) and dam slope images (no cracks).

[0111] 4.9:00: The track robot uploads the immersion line data (10m, ≥9m, normal) and surface displacement data (0.5mm / d, ≤2mm / d, normal).

[0112] 5. The platform generated the first inspection report, which showed no abnormalities.

[0113] (iv) Implementation of Abnormal Response

[0114] At 14:00 on a certain day, the tracked robot detected a reservoir water level of 34.5m (>34m), triggering a level two alarm.

[0115] 1.14:00 The track-mounted robot focuses on the water level monitoring point and uploads data in real time.

[0116] 2.14:01: The platform pushes an alarm message to the operations team leader, instructing the drone to take off;

[0117] 3.14:05: The drone arrived at the water level monitoring point, collected images, and the AI ​​verified the water level at 34.6m, confirming the anomaly;

[0118] 4.14:08: The platform generates a disposal list (responsible person: Operation and maintenance team leader, measures: open No. 2 flood discharge valve, time limit: water level drops below 34m within 1 hour).

[0119] 5.14:10: The operations and maintenance team leader reports receipt and proceeds to the site to open the floodgate;

[0120] 6.15:00: The tracked robot detected a water level of 33.8m (<34m), which was confirmed by the drone.

[0121] 7.15:05: The platform shuts down the alarm, records the handling process, and forms a closed loop.

[0122] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the invention should be included within the scope of the claims of the invention.

Claims

1. An open-pit tailings pond unmanned inspection method, characterized by, include: S1. Establish a communication network for the tailings dam, the communication network including a base station, a tracked robot and a drone that communicate with the base station; Specifically, several base stations are deployed around the tailings dam so that the communication range of these base stations covers the tailings dam area; The deployment locations of the track-based robots include the track on the top of the tailings dam and the tracks on both sides of the drainage ditch, and they will conduct inspections along the tracks; the take-off and landing points of the drones will be set in flat areas at the edge of the tailings dam area. S2. Data collection is conducted using a track-mounted robot and a drone. Specifically: the track-mounted robot collects visible light images and a first infrared image using a configured dual-spectrum gimbal camera; the track-mounted robot collects real-time positioning data using a configured BeiDou satellite navigation system; the drone collects two-dimensional images of the dam slope from two global perspectives using a configured binocular camera; and the drone collects a second infrared image using a configured infrared thermal imager. S3, the track robot, and the drone will each upload the collected data to the cloud platform via the base station; S4. The cloud platform identifies dam surface displacement, dam slope cracks, reservoir water level and dry beach length, and personnel violations based on the uploaded data. S5. Issue an early warning based on the identification results of step S4.

2. The method for unmanned inspection of an open pit tailings pond according to claim 1, characterized in that, The process of identifying dam surface displacement is as follows: A1. Align the visible light image, the first infrared image, and the second infrared image using BeiDou timestamps; A2. Map the second infrared image, which has been aligned in step A1, onto the robot's viewpoint to ensure pixel alignment; A3. The visible light image, the first infrared image, and the second infrared image processed in step A2 are weighted and fused to obtain a fused image. A4、SIFT algorithm is used to extract feature points of the fusion image, and the displacement of feature points of adjacent frames is calculated , the pixel scale is corrected combined with Beidou positioning , and the actual displacement and speed are obtained.

3. The method according to claim 2, characterized in that, The process of identifying cracks in the dam slope is as follows: B1. Data linkage acquisition and preprocessing: The robot conducts low-speed inspections along the dam slope track, acquiring visible light and infrared images; the drone simultaneously takes aerial photos of the entire dam slope infrared image. The three types of images are registered and cropped to a uniform size and then stitched together to form a 4-channel image. The three types of images include: visible light images acquired by the orbital robot, infrared images acquired by the orbital robot, and global infrared images of the dam slope taken by the UAV. The registration and cropping to a uniform size is the processing step A1-A2. The RGB three channels of the UAV visible light image processed in steps A1-A2 are respectively used as channels 1-3. Then, the UAV infrared and the orbital robot infrared are fused into a single channel, and finally a 4-channel image is formed. B2. Improve the U-Net network: The improved U-Net network includes an input layer, a downsampling encoder, a bottleneck layer, an upsampling encoder, and an output layer; The input layer receives 4-channel images as the initial input to the network. The downsampling encoder consists of four cascaded downsampling coding units. Each downsampling coding unit includes a convolutional block and a max pooling layer. Each convolutional block includes a 3×3 convolution, batch normalization, and ReLU activation function in sequence. The downsampling encoder extracts deep features of the image by downsampling step by step through the four cascaded convolutional blocks. The bottleneck layer includes an attention module and a multi-scale fusion module. The attention module performs channel + spatial attention calculation on the feature map output by the encoder to automatically focus on the crack region and weaken the background. The multi-scale fusion module is used to fuse the features output by the attention module, taking into account both the fine crack features collected by the robot and the long crack features collected by the drone. The upsampling decoder includes four cascaded upsampling decoding units. Each upsampling decoding unit includes a deconvolution block and a convolution block. Each deconvolution block includes a 2×2 transposed convolution and a 3×3 convolution. The upsampling decoder restores the image size step by step through the four deconvolution blocks and stitches downsampling features to preserve details. Output layer: The number of channels is compressed through 1×1 convolution, and then the output is activated by the Sigmoid function; the input is H×W×64, and the output is an H×W×1 crack mask map; B3. Train the improved U-Net network to adapt it for dam slope crack detection; compress the trained improved U-Net network and install it into the edge computing module of the robot / drone. The training dataset used in the improved U-Net network training process is a labeled 4-channel image dataset, which mainly includes image data, labeled data, and metadata. The image data is the visible light image of the tracked robot, the infrared image of the tracked robot, and the infrared image of the UAV; the labeled data is the crack annotation data of the 4-channel images; and the metadata includes the image acquisition time, dam slope location, and robot / UAV parameters. During training, the loss is calculated by comparing the "crack mask output by the model" with the "human-annotated real mask", and the network parameters are optimized through backpropagation. B4. Crack Parameter Calculation and Output: Improve the U-Net network for image semantic segmentation to generate crack region masks. Through connected component analysis of the masks and combined with GPS positioning, accurately mark the specific location of the cracks on the dam slope.

4. The method for unmanned inspection of an open pit tailings pond according to claim 3, characterized in that, The bottleneck layer includes an attention module and a multi-scale fusion module. The attention module performs channel + spatial attention calculation on the deep feature map output by the encoder to automatically focus on the crack region and weaken the background. The multi-scale fusion module is used to fuse the features output by the attention module.

5. The method for unmanned inspection of an open pit tailings pond according to claim 4, characterized in that, The process for identifying the length of the dry beach is as follows: The length of the dry beach is calculated based on two-dimensional images of the dam slope from two global perspectives acquired by a binocular camera. Specifically, the actual distance is calculated using the disparity d. f is the camera focal length, and B is the binocular baseline distance.

6. An open pit tailings pond unmanned inspection system, characterized in that, include: Data acquisition module, data processing module, and early warning module; The data acquisition module includes: a base station, a tracked robot and a drone that communicate with the base station; the base station is arranged around the tailings dam and its communication range covers the tailings dam area; the tracked robot is deployed on the top of the tailings dam and on both sides of the drainage ditch, and performs low-speed inspection along the track; the take-off and landing point of the drone is set in a flat area at the edge of the tailings dam area. The tracked robot acquires visible light images and first infrared images based on a configured dual-spectrum gimbal camera; the tracked robot acquires real-time positioning data based on a configured BeiDou satellite navigation system; the UAV acquires two-dimensional images of the dam slope from two global perspectives based on a configured binocular camera; the UAV acquires second infrared images based on a configured infrared thermal imager. The track robot and the drone will each upload the collected data to the data processing module via the base station; The data processing module includes a dam surface displacement identification unit, a dam slope crack identification unit, a reservoir water level and dry beach length identification unit, and a personnel violation identification unit. The early warning module performs early warning processing based on the output results of the data processing module.

7. An open pit tailings pond unmanned inspection system according to claim 6, characterized in that, The dam surface displacement identification unit includes: The first acquisition subunit acquires the aligned visible light image, the first infrared image, and the second infrared image; The second acquisition subunit acquires the result of mapping the second infrared image output by the first acquisition subunit onto the robot's viewpoint; The third acquisition subunit acquires the result obtained by weighted fusion of the visible light image and the first infrared image output by the first acquisition subunit and the result output by the second acquisition subunit; The fourth acquisition sub-unit obtains the dam surface displacement result based on the output of the third acquisition sub-unit.

8. An open pit tailings pond unmanned inspection system according to claim 7, characterized in that, The dam slope crack identification unit includes: The fifth acquisition subunit acquires the result of fusing the first infrared image output by the first acquisition subunit with the second infrared image output by the second acquisition subunit; The sixth acquisition subunit acquires the RGB three-channel image of the visible light image output by the first acquisition subunit, which is used as the first channel image, the second channel image, and the third channel image, respectively, and acquires the result output by the fifth acquisition subunit as the fourth channel image. The improved U-Net network includes an input layer, a downsampling encoder, a bottleneck layer, an upsampling encoder, and an output layer. The input layer receives a 4-channel image. The downsampling encoder extracts deep features from the input 4-channel image by downsampling through several cascaded convolutional blocks. The bottleneck layer extracts crack features from the deep features of the input. The upsampling encoder restores the image size step by step through several cascaded deconvolutional blocks. The output layer outputs a crack mask map. The seventh acquisition subunit trains the improved U-Net network based on the 4-channel image dataset with labeled crack information, and obtains the trained improved U-Net network. The eighth acquisition subunit inputs the first, second, third, and fourth channel images obtained by the sixth acquisition subunit into the trained improved U-Net network to obtain crack information.

Images