An AI vision-based dam slope rockfall intelligent monitoring method and system

Abstract

This invention discloses an intelligent monitoring method and system for rockfall on dam slopes based on AI vision. The method includes: acquiring real-time video image data of the dam slope monitoring area using dynamic visual sensing equipment combined with an infrared and laser fusion supplementary lighting module; inputting the video image data to a deployed AI edge computing node to identify rockfall targets and filter out non-target interference; performing three-dimensional spatial calculations on the identified rockfall targets to obtain monitoring information such as their three-dimensional coordinates, trajectory, size, and velocity parameters; and transmitting the rockfall monitoring information in real-time to a remote computer platform terminal via a wireless communication network for visualization and alarm linkage. This invention, through the deep integration of hardware sensing and AI algorithms, achieves high-precision identification and quantitative analysis of rockfall under complex working conditions, effectively solving the problems of large blind spots and easy omissions in traditional manual inspections, and providing reliable protection for proactive defense and early warning of dam slope disasters.

Description

Technical Field

[0001] This invention relates to the field of water conservancy project safety monitoring technology, specifically to an intelligent monitoring method and system for rockfall on dam slopes based on AI vision. Background Technology

[0002] Dam slopes are critical structures for the safe operation of water conservancy projects. Under long-term weathering, rainwater seepage, reservoir water fluctuations, and geological stress, they are highly susceptible to geological disasters such as rockfalls, landslides, and even partial collapses. Collapsed rock masses may directly impact the dam body, spillway tunnels, water diversion tunnels, and other key hydraulic structures, and could even trigger major catastrophic accidents such as dam failure, seriously threatening the lives and property of people in downstream areas.

[0003] Currently, monitoring of rockfalls on dam slopes mainly relies on manual inspections or traditional single-point contact sensors. Manual inspections typically involve inspectors conducting on-site investigations, taking photographic records, and relying on experience to comprehensively assess the likelihood of rockfall hazards. However, manual inspections have significant drawbacks: First, they are labor-intensive and have a low data collection frequency (usually once a month), making continuous 24 / 7 monitoring impossible, and high-risk periods for rockfalls such as heavy rain, snow, and nighttime are often blind spots. Second, due to geographical and topographical limitations, manual access to the upper and middle parts of steep slopes, cliff depressions, and areas obscured by vegetation is difficult, resulting in numerous monitoring blind spots. Third, the precision of the naked eye and ordinary cameras is insufficient to identify minute rock mass displacements; by the time potential hazards are visible to the naked eye, the rock mass is already on the verge of collapse, leaving an extremely short window for intervention. Fourth, they are highly subjective, lack unified and objective quantitative standards, and the data is scattered and difficult to trace, easily delaying emergency response.

[0004] Traditional single-point sensors (such as crack gauges and inclinometers) can achieve automated data acquisition, but they are contact-based point monitoring with extremely limited coverage, making it difficult to capture random rockfall events on large slope surfaces. Furthermore, these sensors are easily damaged in harsh field environments, resulting in high maintenance costs. In recent years, some projects have introduced ordinary video surveillance, but this only allows for post-event recording and playback, lacking real-time intelligent analysis and quantitative measurement capabilities. In complex slope conditions, interference factors such as drastic changes in lighting, rain and fog, vegetation movement, and birds make existing video surveillance prone to false alarms and missed alarms, and it cannot obtain key dynamic parameters such as the trajectory, speed, and size of falling rocks. In summary, current technologies have not yet solved the problem of all-weather, non-contact, interference-resistant, and high-precision real-time quantitative monitoring and early warning of falling rocks. Summary of the Invention

[0005] To address the technical problems in monitoring rockfall on dam slopes, such as reliance on manual labor, large blind spots, delayed response, poor anti-interference ability, and inability to quantitatively obtain rockfall movement parameters, this invention provides an intelligent monitoring method for rockfall on dam slopes based on AI vision. Through the deep integration of hardware sensing and AI algorithms, it achieves high-precision identification and quantitative analysis of rockfall under complex working conditions, providing a guarantee for proactive defense and early warning of dam slope disasters.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A smart monitoring method for rockfall on dam slopes based on AI vision includes the following steps:

[0008] S10. By deploying dynamic visual sensing equipment on the opposite side of the dam slope, combined with an infrared and laser fusion supplementary lighting module, video image data of the dam slope monitoring area is collected in real time.

[0009] S20. Input the video image data into the locally deployed AI edge computing node, and analyze the video image frame by frame through the built-in deep learning target detection model to identify the falling rock target and filter out non-target interference objects.

[0010] S30. Based on photogrammetry principles and multi-view geometric algorithms, perform three-dimensional spatial calculations on the identified rockfall targets to obtain the three-dimensional coordinates, trajectory, size, and velocity parameters of the rockfall.

[0011] S40. The identification results and three-dimensional spatial parameters of the falling rocks are transmitted to the remote computer platform terminal in real time through the wireless communication network.

[0012] S50. After receiving the data, the computer platform terminal displays it visually and generates linkage alarm information according to the preset early warning rules.

[0013] Specifically, in step S10, the dynamic visual sensing device uses a dynamic deformer, whose average response time does not exceed 1ms and whose response time fluctuation range does not exceed ±10%; the infrared and laser fusion supplementary lighting module automatically adjusts the supplementary lighting strategy according to the target distance and ambient illuminance, and when the monitoring distance reaches 100m, the gray level difference at the image edge is not less than 50.

[0014] Specifically, in step S20, the deep learning target detection model adopts the YOLOv11 model; the process of filtering non-target interference includes: extracting the motion vector and appearance features of the detected target, and combining temporal consistency analysis to remove interference items caused by vegetation shaking, birds flying by, and changes in light and shadow.

[0015] Specifically, the detection accuracy of the YOLOv11 model in complex scenarios such as occlusion, low light, and blur does not decrease by more than 10%; the AI ​​edge computing node adopts an AI edge computing chip with an ARM+NPU architecture, and the inference time for a single frame image does not exceed 50ms.

[0016] Specifically, the process of step S30 includes:

[0017] Based on the intrinsic and extrinsic parameters of binocular or multi-view vision sensors, the two-dimensional pixel coordinates of falling rocks in multi-frame images are extracted using epipolar geometry constraints.

[0018] The coordinate sequence of the falling rocks in three-dimensional physical space was reconstructed by feature point matching and bundle adjustment.

[0019] The instantaneous velocity and direction vector of the falling rock are calculated based on the three-dimensional coordinate difference and time interval between adjacent frames.

[0020] By combining the camera's calibration coefficients with the pixel area of ​​the bounding box of the falling rock in the image, the equivalent three-dimensional size of the falling rock is calculated.

[0021] Specifically, in step S40, the wireless communication network is a 4G communication network, which establishes a transmission link with the IoT card through an industrial-grade router. The end-to-end data response time does not exceed 100ms, and the communication rate is not less than 40Mbps.

[0022] Specifically, the process of generating linkage alarm information according to preset early warning rules in step S50 includes:

[0023] Risk levels are determined based on the size threshold, speed threshold, and three-dimensional spatial area where the rockfall occurs. When the rockfall parameters reach the corresponding risk level threshold, at least one of the following alarm methods is triggered: platform pop-up, audible and visual alarm, and SMS push notification.

[0024] Furthermore, the present invention also provides an AI vision-based intelligent monitoring system for dam slope rockfall, used in the aforementioned monitoring method, comprising:

[0025] The front-end sensing module, including a dynamic deformation meter and an infrared and laser fusion lighting device, is used to collect high-definition video streams of the dam slope around the clock;

[0026] The edge computing node module, which has a built-in AI edge computing chip and YOLOv11 algorithm model, is connected to the front-end perception module and is used to perform real-time identification of falling rock targets, interference filtering, and three-dimensional motion parameter calculation on the video stream.

[0027] A communication transmission module, connected to the edge computing node module, is used to wirelessly transmit monitoring data via a 4G network;

[0028] The computer platform terminal, connected to the communication transmission module, includes a computer terminal and a visual monitoring platform, used to receive data, display the three-dimensional trajectory of falling rocks, and execute early warning linkage.

[0029] Specifically, the front-end sensing module, edge computing node module, and communication transmission module are all integrated into an IP67-rated outdoor protective chassis.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (1) By using a dynamic deformer combined with infrared and laser fusion lighting equipment, the present invention achieves all-weather high-definition imaging and high-precision dynamic capture, thereby completely eliminating the monitoring blind spots in low light environments such as night and rain and fog, and solving the technical problems of poor image quality and target loss caused by the limitation of ambient light in traditional visual monitoring.

[0032] (2) This invention achieves a rockfall recognition accuracy of ≥95% in complex slope background by deploying an AI computing chip equipped with YOLOv11 algorithm on the edge side for rockfall recognition and interference filtering. This effectively overcomes the problem of high false alarm rate caused by vegetation shaking, flying birds and changes in light and shadow, and greatly improves the robustness of the system in real field conditions.

[0033] (3) By introducing photogrammetry and multi-view geometric calculation, this invention realizes the real-time quantitative extraction of key parameters such as the three-dimensional coordinates, motion trajectory, size and speed of falling rocks, thereby transforming traditional qualitative video monitoring into quantitative dynamic parameter analysis, providing accurate data support for risk assessment and early warning of rockfall disasters on dam slopes.

[0034] (4) By constructing an integrated hardware and software architecture of “edge computing + 4G wireless transmission + IP67 high protection”, this invention enables the system to operate stably without human intervention in extreme environments such as high temperature, high humidity, and lightning strikes, thereby eliminating the dependence on manual on-site inspections, reducing the safety risks of personnel operations, and significantly improving the automation and intelligence level of dam slope monitoring. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the method flow of an embodiment of the present invention.

[0036] Figure 2 This is a schematic diagram of the system architecture of an embodiment of the present invention.

[0037] Figure 3 This is a schematic diagram of the simulation test setup in an embodiment of the present invention. Detailed Implementation

[0038] The present invention will be further described below with reference to the accompanying drawings and embodiments. The embodiments of the present invention include, but are not limited to, the following embodiments.

[0039] Example

[0040] like Figures 1 to 3 As shown, this embodiment provides an intelligent monitoring method and system for rockfall on dam slopes based on AI vision. In response to the actual working conditions of the Hohhot pumped storage power station dam slope, which is characterized by steep terrain, frequent rockfalls, and high risks and low efficiency of manual inspection, the system achieves proactive identification and accurate early warning of rockfall hazards through the deep integration of multi-source hardware perception and deep learning algorithms.

[0041] The AI-based vision-based intelligent monitoring method for rockfall monitoring on dam slopes in this embodiment includes the following steps:

[0042] Step S10: By deploying dynamic visual sensing equipment on the opposite side of the dam slope, combined with an infrared and laser fusion supplementary lighting module, video image data of the dam slope monitoring area is collected in real time.

[0043] Specifically, considering the extremely high dynamic characteristics of rockfalls on dam slopes, this embodiment abandons the traditional static deformable element (which, according to testing, has an average response time of 68.2ms with a fluctuation of ±30%, making it unable to capture high-speed transient targets). Instead, it selects the AS-DVS00 dynamic deformable element as the visual sensor. Tests show that the dynamic deformable element has an average response time of only 0.0494ms (significantly better than the 1ms requirement), a response time fluctuation range within ±7.3% (meeting the ±10% requirement), and an identification error of no more than ±0.02mm for standard target parts. This perfectly meets the requirement for image capture without motion blur during high-speed rockfalls.

[0044] Meanwhile, to address the issue of insufficient lighting on the dam slope at night and in inclement weather, this embodiment uses the DS-2FL2000-FUSION infrared and laser fusion lighting module. Infrared lighting provides diffuse illumination with a wide viewing angle, while laser lighting provides directional penetrating illumination over a long distance. Testing was conducted on a 10cm standard test board at a monitoring distance of 100 meters. The average edge grayscale difference of the combined lighting image reached 58.7 (maximum 62.3 / minimum 54.8), meeting the threshold requirement of 50, and the maximum effective detection distance reached 116.7 meters. When the target experienced lateral displacement (range not less than ±2.0m) and speed changes (0.2-5.0m / s), the recognition success rate remained above 99%, with no target loss throughout the process, ensuring a high signal-to-noise ratio and high definition of the input image.

[0045] Step S20: Input the video image data into the locally deployed AI edge computing node, and analyze the video image frame by frame through the built-in deep learning target detection model to identify the falling rock target and filter out non-target interference objects.

[0046] Specifically, in this embodiment, the locally deployed AI edge computing node uses an AI edge computing chip with an ARM+NPU architecture. Testing showed that under continuous full-load operation at a constant temperature of 25℃, the chip achieved a peak computing power of 8.72 TOPS, an average effective computing power utilization rate of 78.6% (meeting the 75% threshold requirement), computing power data fluctuation of ±1.2% (meeting the ±3% requirement), a single-frame image inference time of approximately 20ms (meeting the 50ms requirement), and an end-to-end latency of 30ms, perfectly adapting to the dual constraints of low power consumption and high real-time performance in the field.

[0047] At the algorithm level, the deep learning object detection model deployed in this embodiment adopts the optimized YOLOv11 model. Compared with the traditional Faster R-CNN and SSD algorithms, the YOLOv11 model achieves a baseline mAP@0.5 of 88.2% and a localization IoU of 92.5% in standard clear scenes. For the three complex scenarios of occlusion, low light, and blurriness present in actual dam slopes, the YOLOv11 model demonstrates extremely strong anti-accuracy capabilities: specific tests show an accuracy attenuation rate of 7.82% in occluded scenes, 8.73% in low light scenes, and 6.92% in blurriness scenes, with an average accuracy attenuation rate of 7.67%, lower than the design threshold of 10%.

[0048] In filtering out non-target interference, the model utilizes YOLOv11's multi-scale feature fusion and attention mechanism, combined with a temporal tracking algorithm, to perform secondary discrimination on detected moving targets: if the target's trajectory exhibits irregular up-and-down shaking with a large deformation rate (such as vegetation swaying in the wind), or if the target has obvious biological movement characteristics and drastic grayscale changes (such as birds flying by), it is judged as interference and removed; if the target exhibits parabolic or linear motion characteristics along the slope due to gravity, and its outline is hard and its texture matches the characteristics of rocks, it is confirmed as a falling rock target and locked.

[0049] Step S30: Based on the principles of photogrammetry and multi-view geometric algorithms, perform three-dimensional spatial calculations on the identified falling rock targets to obtain the three-dimensional coordinates, trajectory, size, and velocity parameters of the falling rocks.

[0050] Specifically, by distributing binocular dynamic deformable sensors on the opposite side of the monitoring area (or using a monocular camera combined with a priori 3D real-world model of the slope), the two-dimensional pixel coordinates of the falling rocks in the image sequence are extracted. High-precision feature point matching is performed using collinearity equations and epipolar geometric constraints. The absolute coordinates of the falling rocks in physical 3D space are calculated using bundle adjustment. Taking two adjacent image frames as an example, if the falling rock moves from coordinate point (X1,Y1,Z1) to (X2,Y2,Z2) within time Δt, its instantaneous velocity is... The motion direction vector is the normal vector of (X2-X1, Y2-Y1, Z2-Z1). At the same time, combined with the camera's intrinsic and extrinsic parameter matrices and the pixel area ratio of the bounding box of the falling rocks in the image, the equivalent three-dimensional size (length, width, height or equivalent particle size) of the falling rocks is calculated by inversion.

[0051] Step S40: Transmit the rockfall identification results and three-dimensional spatial parameters to the remote computer platform terminal in real time via a wireless communication network.

[0052] Specifically, considering the extreme difficulty of laying cables on the dam slope in the field, this embodiment adopts 4G wireless communication. A China Mobile industrial-grade IoT card and a Sierra Wireless AirLink XR80 industrial router are used to construct the communication link. Field tests showed that under a 5km long-distance transmission condition, the communication rate remained stable at 40Mbps, with an average end-to-end response time of 85ms (shortest 76ms, longest 98ms), meeting the target threshold of 100ms with a 100% compliance rate. This completely avoids the difficulties of fiber optic cabling and the problems of insufficient LoRa bandwidth and excessive latency (average 186ms), ensuring the real-time transmission of high-definition video streams and 3D coordinate data.

[0053] Step S50: After receiving the data, the computer platform terminal displays it visually and generates linkage alarm information according to the preset early warning rules.

[0054] Specifically, the platform terminal uses a high-performance computer as its carrier, which has more than twice the matrix operation and other solution capabilities compared to mobile terminals, meeting the needs of 3D calculation and big data rendering. Tests show that the average video reception latency is 186.8ms (≤240ms), and the average response latency from target triggering to the platform's pop-up alarm is 72.9ms (≤100ms). The platform interface renders the trajectory of falling rocks on the 3D slope model in real time and automatically extracts parameters such as size and velocity. When the size of the falling rock is >20cm or the velocity is >2m / s, the system automatically triggers an audible and visual alarm and pushes information, achieving early warning.

[0055] To ensure the long-term stable operation of the above method in the field, the front-end sensing devices, edge computing nodes, and communication modules involved in this embodiment are all integrated into a customized protective shell. After passing the standard IP67 immersion test (1m water depth, 30min), the water ingress mass of the equipment is only 0.01g (far below the standard of ≤0.02g). After being powered on, all functional parameters are without deviation, and it can effectively resist the high humidity, rainstorms, and dust erosion of the dam slope.

[0056] To verify the overall technical effectiveness of this system, a real-world test was conducted in a typical slope area of ​​the Hohhot pumped storage power station dam. Twelve real rockfall samples were selected for the test (sizes ranging from 20cm to 58cm, including the left bank, right bank, and upper, middle, and lower sections; movement patterns included uniform rolling, jumping rolling, and rapid rolling). The system automatically executed the monitoring process. The test results showed that all 12 rockfall samples were correctly identified, with no missed or false alarms, achieving a 100% accuracy rate (meeting and exceeding the design target of ≥95%). Simultaneously, the system accurately output the trajectory key nodes of rockfall number 1 (size 45cm): the continuous three-dimensional coordinate changes from (128.5, 45.2, 22.8) to (125.7, 38.4, 18.0). Experiments have shown that the system has completely solved the pain points of traditional manual inspections, such as "not being able to see clearly, not being able to see completely, not being able to see accurately, and not being able to see quickly," and has achieved a fundamental transformation from passive post-event remediation to proactive pre-event early warning of rockfalls on dam slopes.

[0057] Corresponding to the above method, this embodiment also provides an AI vision-based intelligent monitoring system for rockfall on dam slopes, including: a front-end sensing module, an edge computing node module, a communication transmission module, and a computer platform terminal. The front-end sensing module includes a dynamic deformation meter and an infrared and laser fusion illumination device; the edge computing node module integrates an AI edge computing chip and a YOLOv11 model and 3D solution unit; the communication transmission module uses a 4G industrial router; and the computer platform terminal includes a computer terminal and an internal visualization interface and early warning linkage unit. These modules work together to achieve high-precision, all-weather, unmanned intelligent monitoring of rockfall on dam slopes.

[0058] The above embodiments are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any changes made based on the design principles of the present invention, or any non-creative modifications made thereon, shall fall within the scope of protection of the present invention.

Claims

1. A method for intelligent monitoring of rockfall on dam slopes based on AI vision, characterized in that, Includes the following steps: S10. By deploying dynamic visual sensing equipment on the opposite side of the dam slope, combined with an infrared and laser fusion supplementary lighting module, video image data of the dam slope monitoring area is collected in real time. S20. Input the video image data into the locally deployed AI edge computing node, and analyze the video image frame by frame through the built-in deep learning target detection model to identify the falling rock target and filter out non-target interference objects. S30. Based on photogrammetry principles and multi-view geometric algorithms, perform three-dimensional spatial calculations on the identified rockfall targets to obtain the three-dimensional coordinates, trajectory, size, and velocity parameters of the rockfall. S40. The identification results and three-dimensional spatial parameters of the falling rocks are transmitted to the remote computer platform terminal in real time through the wireless communication network. S50. After receiving the data, the computer platform terminal displays it visually and generates linkage alarm information according to the preset early warning rules.

2. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 1, characterized in that, In step S10, the dynamic visual sensing device uses a dynamic deformation meter with an average response time of no more than 1ms and a response time fluctuation range of no more than ±10%. The infrared and laser fusion supplementary lighting module automatically adjusts the supplementary lighting strategy according to the target distance and ambient illuminance. When the monitoring distance reaches 100m, the gray level difference at the image edge is no less than 50.

3. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 1, characterized in that, In step S20, the deep learning target detection model adopts the YOLOv11 model; the process of filtering non-target interference includes: extracting the motion vector and appearance features of the detected target, and combining temporal consistency analysis to remove interference items caused by vegetation shaking, birds flying by, and changes in light and shadow.

4. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 3, characterized in that, The detection accuracy of the YOLOv11 model in complex scenarios such as occlusion, low light, and blur does not decrease by more than 10%; the AI ​​edge computing node adopts an AI edge computing chip with an ARM+NPU architecture, and the inference time for a single frame image does not exceed 50ms.

5. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 1, characterized in that, The specific process of step S30 includes: Based on the intrinsic and extrinsic parameters of binocular or multi-view vision sensors, the two-dimensional pixel coordinates of falling rocks in multi-frame images are extracted using epipolar geometry constraints. The coordinate sequence of the falling rocks in three-dimensional physical space was reconstructed by feature point matching and bundle adjustment. The instantaneous velocity and direction vector of the falling rock are calculated based on the three-dimensional coordinate difference and time interval between adjacent frames. By combining the camera's calibration coefficients with the pixel area of ​​the bounding box of the falling rock in the image, the equivalent three-dimensional size of the falling rock is calculated.

6. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 1, characterized in that, In step S40, the wireless communication network is a 4G communication network. A transmission link is established between an industrial-grade router and an IoT card. The end-to-end data response time is no more than 100ms and the communication rate is no less than 40Mbps.

7. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 1, characterized in that, The process of generating linkage alarm information according to preset early warning rules in step S50 includes: Risk levels are determined based on the size threshold, speed threshold, and three-dimensional spatial area where the rockfall occurs. When the rockfall parameters reach the corresponding risk level threshold, at least one of the following alarm methods is triggered: platform pop-up, audible and visual alarm, and SMS push notification.

8. An AI vision-based intelligent monitoring system for rockfall on dam slopes, used to implement the monitoring method as described in any one of claims 1 to 7, characterized in that, include: The front-end sensing module, including a dynamic deformation meter and an infrared and laser fusion lighting device, is used to collect high-definition video streams of the dam slope around the clock; The edge computing node module, which has a built-in AI edge computing chip and YOLOv11 model, is connected to the front-end perception module and is used to perform real-time identification of falling rock targets, interference filtering, and three-dimensional motion parameter calculation on the video stream. A communication transmission module, connected to the edge computing node module, is used to wirelessly transmit monitoring data via a 4G network; The computer platform terminal, connected to the communication transmission module, includes a computer terminal and a visual monitoring platform, used to receive data, display the three-dimensional trajectory of falling rocks, and execute early warning linkage.

9. The intelligent monitoring method for rockfall on dam slopes based on AI vision according to claim 8, characterized in that, The front-end sensing module, edge computing node module, and communication transmission module are all integrated into an IP67-rated outdoor protective chassis.

Images