Intelligent Early Warning System and Method for Rock Landslides in Open-pit Mines Based on Rockfall Video Monitoring

Abstract

The present invention provides an intelligent early warning system and method for open-pit mine rock landslides based on falling rock video monitoring, which relates to the technical field of open-pit mine rock landslide monitoring. The system and method include a video acquisition module, a wireless transmission module, a cloud server, and an early warning release terminal; the video acquisition module collects video image data of the open-pit mine rock slope in real time and transmits it to the cloud server through the wireless transmission module; a falling rock intelligent detection program is deployed in the cloud server to process the video stream transmitted from the video acquisition module in real time, calculate the falling rock frequency and the area of the moving block, and then judge whether there is a landslide danger in the open-pit mine rock slope. If so, an early warning instruction is sent to the early warning release terminal, and the video stream is saved locally; the early warning release terminal issues an early warning according to the early warning instruction sent by the cloud server. The system and method utilize the target tracking algorithm to analyze the monitored video stream in real time, and can perform landslide early warning more accurately and timely.

Description

Technical Field

[0001] The present invention relates to the technical field of open-pit mine rock landslide monitoring and early warning, and in particular to an intelligent early warning system and method for open-pit mine rock landslides based on falling rock video monitoring. Background Art

[0002] With the continuous development and utilization of resources, deep open-pit mines have become the development trend of open-pit mines in the world. The contradiction between ensuring slope safety and improving economic benefits during the deep mining process of open-pit mines has become increasingly prominent. The stability problem of high-steep rock slopes in open-pit mines is the core issue of safe production in open-pit mines. Conducting relevant research on open-pit mine slope monitoring and early warning has important theoretical and practical significance for ensuring mine safety production and improving economic benefits.

[0003] For open-pit mine rock landslides, the sliding mass in a critical state, under the influence of external disturbances (rainfall, blasting vibration), moves downward and disintegrates into crushed stones while transforming into gravel-slope deposits. The initiation of a rock landslide is caused by the fracture of a single rock bridge, the conversion of gravitational potential energy into kinetic energy, which triggers the fracture of the next-level rock bridge, and the energy is transmitted step by step, ultimately resulting in an overall landslide, forming a "domino effect". Using falling rock video monitoring can effectively provide early warning for open-pit mine rock landslides.

[0004] Currently, the stability monitoring of open-pit mine slopes mostly uses devices such as slope surface radar, microseismic, displacement, and stress for monitoring. Due to high costs, these monitoring methods cannot be widely deployed on the overall mine slope. By making full use of the video monitoring data commonly existing in open-pit mines (the Technical Specification for Safety Monitoring of High-Steep Slopes in Metal and Non-Metal Open-Pit Mines AQ / T 2063-2018 requires that video monitoring should cover the main slopes), thus ① giving full play to the characteristics of open-pit mines, using the video monitoring required by the specification as the data source, with low cost and wide monitoring coverage; ② video monitoring can achieve real-time online monitoring, avoiding the missed reporting phenomenon caused by manual monitoring; ③ applying deep learning methods enables the system to be applicable to a wider range of application scenarios and avoid the influence of interference sources such as image shaking and smoke and dust on the recognition results. Using the method based on falling rock video monitoring can provide early warning for open-pit mine rock landslides with a lead time of several seconds to several hours. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an intelligent early warning system and method for open-pit mine rock landslides based on falling rock video monitoring in view of the above-mentioned deficiencies of the prior art, so as to achieve intelligent early warning of open-pit mine rock landslides.

[0006] To solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0007] On the one hand, the present invention provides an intelligent early warning system for open-pit mine rock landslides based on falling rock video monitoring, including a video acquisition module, a wireless transmission module, a cloud server, and an early warning release terminal; the video acquisition module collects video image data of the open-pit mine rock slope in real time and transmits it to the cloud server through the wireless transmission module; a falling rock intelligent detection program is deployed in the cloud server to process the video stream transmitted from the video acquisition module in real time, determine whether there is a landslide risk on the open-pit mine rock slope, and if so, send an early warning instruction to the early warning release terminal and save the video stream locally; the early warning release terminal gives an early warning according to the early warning instruction sent by the cloud server.

[0008] Preferably, the video acquisition module includes a plurality of high-definition network cameras arranged at different positions on the open-pit mine rock slope.

[0009] Preferably, in the image obtained by the high-definition network camera, the actual physical size represented by each pixel is less than or equal to 5 cm, the resolution is higher than 1280*720, and the frame rate is greater than 15; at the same time, the network camera has a wireless communication function.

[0010] Preferably, the early warning release terminal is set locally in the open-pit mine and includes a speaker and an electronic warning sign.

[0011] On the other hand, the present invention provides an intelligent early warning method for open-pit mine rock landslides based on falling rock video monitoring, including the following steps:

[0012] Step 1: Divide the open-pit mine rock slope into several sections, and install a mine high-definition network camera in each section. The camera directly faces the slope surface to ensure that the section can be completely covered by the camera's field of view;

[0013] Step 2: Deploy a falling rock intelligent detection program in the cloud server to process the video stream transmitted from the network camera in real time and save the video stream locally; the falling rock intelligent detection program uses a moving target tracking algorithm to detect each falling rock frame by frame from the video, and then calculates the falling rock frequency of each frame, and automatically calculates and demarcates the slope movement area according to the position where the falling rock occurs; at the same time, calculate the rolling path of each falling rock, calculate the rolling speed and the size of the falling rock;

[0014] The falling rock intelligent detection program uses the frame difference method to detect falling rocks in the video stream, specifically including the following steps:

[0015] Step S1: Grab frames; read the video file through OpenCV to obtain each frame image in the video;

[0016] Step S2: Monitor; compare the obtained two consecutive frame images, find the differences between the two frame images, determine whether this difference belongs to movement, and mark the moving area;

[0017] (1) Divide each frame of image into blocks by pixels; divide each frame of image into multiple blocks;

[0018] (2) Determine moving pixels; take the difference of all corresponding pixel points in two consecutive frames of images, calculate the difference between the two consecutive frames of images. If the difference of the corresponding pixel points in the two consecutive frames of images is greater than the set value, then determine that the pixel is a moving pixel;

[0019] (3) Judge the running blocks; count the number of moving pixels in each block. If more than half of the pixels in a block are moving pixels, then determine that the block is a moving block, and the moving block is the falling rock;

[0020] (4) Merge neighboring blocks; for a moving block, search its neighboring blocks and judge whether each neighboring block is a moving block. If the neighboring block is determined to be a moving block, then judge whether the speed and direction of the neighboring block and the moving block are less than the set threshold. If less, then merge the neighboring block into the moving block;

[0021] (5) Mark the moving blocks; mark the moving blocks with a rectangle;

[0022] Step S3: Restore each processed frame of image into a video according to the image sequence and frame rate of the original video; calculate the falling rock frequency and the area of the moving block through the restored video, and then calculate the rolling path, rolling speed and size of each falling rock; finally, judge whether the falling rock frequency or the area of the moving block in the monitoring area of the network camera exceeds the set threshold;

[0023] Step S4: Arrange the frame difference algorithm program in the Docker container of the cloud server. After the monitoring video is transmitted to the cloud server, automatically call this program to monitor the falling rocks;

[0024] Step 3: Deploy an alarm program in the cloud server to identify dangerous events and publish early warning information;

[0025] The alarm program deployed in the cloud server judges whether the falling rock frequency or the area of the moving block in the monitoring area of each network camera exceeds the set threshold. If it exceeds, the cloud server controls the early warning publishing terminal to publish early warning information to remind the relevant section personnel and equipment to evacuate according to the plan.

[0026] The beneficial effects of adopting the above technical solutions are as follows: The intelligent early warning system and method for open-pit mine rock landslides based on rockfall video monitoring provided by the present invention: (1) The slope is divided into several sections, and mine-use high-definition network cameras are respectively set, which can conduct zonal monitoring and early warning of open-pit mine rock landslides; (2) The target tracking algorithm is used to analyze the monitored video stream in real time, detect the number of rockfalls frame by frame, calculate the rockfall frequency, and automatically calculate and demarcate the slope movement area according to the position where the rockfalls occur. (3) The detection includes the rockfall frequency and the area of the movement area, increasing the applicable scenarios of the system and method. (4) The program is used to calculate the video stream, and it is processed while being transmitted, avoiding the situation of "the program waiting for data", and ensuring "real-time monitoring and real-time early warning". (5) If the rockfall frequency or the area of the movement area exceeds the set threshold, it is considered that the landslide has entered an irreversible stage, so a landslide early warning can be issued dozens of seconds to several hours in advance. (6) The cloud server is connected to the alarm to ensure that the early warning is issued in the first time and ensure that personnel and equipment can be evacuated in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the structural block diagram of the intelligent early warning system for open-pit mine rock landslides based on rockfall video monitoring provided by the embodiment of the present invention;

[0028] Figure 2 It is the installation side view of the intelligent early warning system for open-pit mine rock landslides based on rockfall video monitoring provided by the embodiment of the present invention;

[0029] Figure 3 It is the flow chart of the intelligent early warning method for open-pit mine rock landslides based on rockfall video monitoring provided by the embodiment of the present invention;

[0030] Figure 4 It is the on-site diagram of rockfalls in the landslide of an open-pit iron mine provided by the embodiment of the present invention, where (a) is 35 s before the landslide, (b) is 35 s before the landslide, (c) is 5 s before the landslide, and (d) is the occurrence of the landslide;

[0031] Figure 5 It is the comparison result diagram of the landslide scale and the rockfall monitoring and early warning time provided by the embodiment of the present invention.

[0032] In the figure, 1, mine-use high-definition network camera; 2, cloud server; 3, early warning release terminal; 4, dangerous slope; 5, rockfall. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The following combines the drawings and embodiments to further describe in detail the specific embodiments of the present invention. The following embodiments are used to illustrate the present invention, but are not used to limit the scope of the present invention.

[0034] In this embodiment, an intelligent early warning system for open-pit mine rock landslides based on rockfall video monitoring is as Figure 1As shown, it includes a video acquisition module, a wireless transmission module, a cloud server, and a warning release terminal. The video acquisition module collects video image data of the open-pit mine rock slope in real time and transmits it to the cloud server through the wireless transmission module. The cloud server deploys a rockfall intelligent detection program to process the video stream transmitted from the video acquisition module in real time, judge whether there is a landslide risk on the open-pit mine rock slope. If there is, it sends a warning instruction to the warning release terminal and saves the video stream locally. The warning release terminal is set locally in the open-pit mine, including a speaker and an electronic warning sign, and can display monitoring information according to the warning instruction sent by the cloud server and issue a landslide warning through the speaker.

[0035] In this embodiment, the video acquisition module includes a plurality of high-definition network cameras arranged at different positions on the open-pit mine rock slope. In the images obtained by the high-definition network cameras, the actual physical size represented by each pixel is less than or equal to 5 cm, the resolution is higher than 1280*720, and the frame rate is greater than 15. At the same time, the network cameras have a wireless communication function and can upload the collected video image data to the cloud server in real time through the wireless transmission module. In this embodiment, in an open-pit mine, the system is installed as Figure 2 shown, where the mine-use high-definition network camera 1 directly faces the slope surface of the dangerous slope 4, can collect video images of the rockfall 5 in real time, and transmit the collected video images to the cloud server 2. The cloud server 2 judges whether there is a risk of landslide and rockfall through the video images. If there is a risk of rockfall, it sends a warning signal to the warning release terminal 3.

[0036] In this embodiment, an intelligent early warning method for open-pit mine rock landslides based on rockfall video monitoring, as Figure 3 shown, includes the following steps:

[0037] Step 1: Divide the open-pit mine rock slope into several sections, and deploy a mine-use high-definition network camera in each section. The camera directly faces the slope surface to ensure that the section can be completely covered by the camera's field of view;

[0038] In this embodiment, it is required that in the images obtained by the mine-use high-definition network cameras, the actual physical size represented by each pixel is less than or equal to 5 cm, the resolution is higher than 1280*720, and the frame rate is greater than 15. At the same time, the network cameras have a wireless communication function and can upload the captured videos to the cloud server;

[0039] Step 2: Deploy the rockfall intelligent detection program in the cloud server to process the video stream transmitted from the network camera in real time and save the video stream locally; the rockfall intelligent detection program uses traditional moving object tracking algorithms such as frame difference method, optical flow method, Gaussian mixture model method, self-organizing background monitoring, or deep learning algorithms for moving object tracking such as Cascade CNN, YOLO, FgSegNet, etc. to detect each rockfall frame by frame from the video, and then calculates the rockfall frequency of each frame, and automatically calculates and demarcates the slope movement area according to the position where the rockfall occurs; at the same time, calculates the rolling path of each rockfall, calculates the rolling speed and the size of the rockfall.

[0040] In this embodiment, the rockfall intelligent detection program uses the frame difference method to detect rockfalls in the video stream, which specifically includes the following steps:

[0041] Step S1: Capture frames; read the video file through OpenCV to obtain each frame image in the video.

[0042] Step S2: Monitor; compare the obtained two consecutive frame images, find the differences between the two frame images, determine whether this difference belongs to movement, and mark the moving area with a square box.

[0043] (1) Divide each frame image into blocks by pixels; an image is a dot matrix composed of pixels, and each pixel contains four channels. The first channel is alpha, and the 2nd to 4th channels represent R, G, and B; for subsequent recognition, each frame image is divided into multiple blocks; in this embodiment, each frame image is divided into 10×10 blocks.

[0044] (2) Determine moving pixels; take the difference of all corresponding pixel points in the two consecutive frame images, calculate the difference between the two consecutive frame images. If the difference between the corresponding pixel points in the two consecutive frame images is greater than the set value, then determine that the pixel is a moving pixel.

[0045] (3) Judge the moving block; count the number of moving pixels in each block. If more than half of the pixels in a block are moving pixels, then determine that the block is a moving block, and the moving block is the rockfall.

[0046] (4) Merge neighboring blocks; for a moving block, search its neighboring blocks, and determine whether each neighboring block is a moving block. If the neighboring block is determined to be a moving block, then determine whether the speed and direction of the neighboring block and the moving block are less than the set threshold. If less, then merge the neighboring block into the moving block.

[0047] (5) Mark the moving block; mark the moving block with a square box.

[0048] Step S3: Restore each processed frame image into a video according to the image sequence and frame rate of the original video; calculate the rockfall frequency or the area of the moving block through the restored video, and then calculate the rolling path, rolling speed and rockfall size of each rockfall; finally, determine whether the rockfall frequency or the area of the moving block in the monitoring area of the network camera exceeds the set threshold;

[0049] Step S4: Arrange the frame difference algorithm program in the Docker container of the cloud server. After the monitoring video is transmitted to the cloud server, this program is automatically called for rockfall monitoring;

[0050] Step 3: Deploy an alarm program in the cloud server to identify dangerous events and publish early warning information;

[0051] The alarm program deployed in the cloud server determines whether the rockfall frequency or the area of the moving block in the monitoring area of each network camera exceeds the set threshold. If it exceeds, the cloud server controls the early warning release terminal to publish early warning information to remind the personnel and equipment in the relevant section to evacuate according to the plan.

[0052] In this embodiment, the site of a rockfall landslide in an open-pit iron mine is as Figure 4 shown. Through the intelligent early warning system and method for open-pit rock mass landslide of the present invention, 31 different open-pit mines are monitored and early warned. The comparison of the landslide scale and the rockfall monitoring and early warning time obtained by the monitoring is as Figure 5 shown. It can be seen from the figure that the system and method of the present invention can timely and effectively monitor and early warn the rockfall in the open-pit mine.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope defined by the claims of the present invention.

Claims

1. An intelligent early warning method for rock landslides in open-pit mines based on falling rock video monitoring, which is realized by an intelligent early warning system for rock landslides in open-pit mines based on falling rock video monitoring. It is characterized in that: The system includes a video acquisition module, a wireless transmission module, a cloud server, and an early warning release terminal; the video acquisition module collects video image data of the rock slope in the open-pit mine in real time and transmits it to the cloud server through the wireless transmission module; a falling rock intelligent detection program is deployed in the cloud server to process the video stream transmitted from the video acquisition module in real time, judge whether there is a landslide danger on the rock slope in the open-pit mine, and if so, send an early warning instruction to the early warning release terminal and save the video stream locally; The early warning release terminal gives an early warning according to the early warning instruction sent by the cloud server; The method includes the following steps: Step 1: Divide the rock slope in the open-pit mine into several sections, and install a high-definition mine network camera in each section. The camera faces the slope directly to ensure that the section can be completely covered by the camera's field of view; Step 2: Deploy a falling rock intelligent detection program in the cloud server to process the video stream transmitted from the network camera in real time and save the video stream locally; the falling rock intelligent detection program uses a moving target tracking algorithm to detect each falling rock frame by frame from the video, and then calculates the falling rock frequency of each frame, and automatically calculates and demarcates the slope movement area according to the position where the falling rock occurs; at the same time, calculate the rolling path, rolling speed and size of each falling rock; Step 3: Deploy an alarm program in the cloud server to identify dangerous events and release early warning information; The alarm program deployed in the cloud server judges whether the falling rock frequency or the area of the movement area in the monitoring area of each network camera exceeds the set threshold. If it exceeds, the cloud server controls the early warning release terminal to release early warning information to remind the personnel and equipment in the relevant section to evacuate according to the plan.

2. The intelligent early warning method for open-pit rock landslides based on falling rock video monitoring according to claim 1, wherein: The falling rock intelligent detection program uses the frame difference method to detect falling rocks in the video stream, which specifically includes the following steps: Step S1: Capture frames; read the video file through OpenCV to obtain each frame image in the video; Step S2: Monitor; Compare the obtained front and rear frame images to find the differences between the two frame images, judge whether this difference belongs to movement, and mark the moving area; Step S3: Restore each processed frame image into a video according to the image sequence and frame rate of the original video; calculate the falling rock frequency or the area of the movement area through the restored video, and then calculate the rolling path, rolling speed and size of each falling rock; Step S4: Arrange the frame difference algorithm program in the Docker container of the cloud server. After the monitoring video is transmitted to the cloud server, this program is automatically called for falling rock monitoring.

3. The intelligent early warning method for open-pit rock landslide based on rockfall video monitoring according to claim 2, characterized in that: The specific method of step S2 is: (1) Divide each frame image into blocks through pixels; divide each frame image into multiple blocks; (2) Determine moving pixels; take the difference of all corresponding pixel points in the front and rear frame images, calculate the difference between the front and rear frame images. If the difference of the corresponding pixel points in the front and rear frame images is greater than the set value, then determine that the pixel is a moving pixel; (3) Determine the running block; count the number of moving pixels in each block. If more than half of the pixels in a block are moving pixels, then determine that the block is a moving block, and the moving block is the falling rock; (4) Merge neighboring blocks; for a moving block, search for its neighboring blocks and determine whether each neighboring block is a moving block. If the neighboring block is determined to be a moving block, then determine whether the speed and direction of the neighboring block and the moving block are less than the set threshold. If so, merge the neighboring block into the moving block; (5) Mark the moving block; mark the moving block with a square box.

Images