Open-pit mine slope disaster intelligent inspection system with ai visual recognition

Abstract

The application provides an open-pit mine slope disaster intelligent inspection system loaded with AI visual identification, relates to the technical field of open-pit mine slope disaster detection, adopts a four-level architecture of "end-edge-cloud cooperation", and comprises a perception layer, an edge layer, a cloud layer and an application layer. The perception layer constructs an "air-space-ground integrated" network, and multi-source data are collected through a UAV cluster, a high-definition visible light camera, a thermal imaging camera, a laser radar, a vibration sensor and a micro displacement sensor. The edge layer industrial-grade edge computing gateway deploys a multi-modal fusion identification model to realize local real-time identification. The cloud layer completes data storage, model optimization and dynamic risk assessment, and the application layer realizes early warning, inspection management and other functions. The system breaks through the limitations of traditional technology, improves the inspection accuracy and efficiency, is suitable for mines of different scales, and provides reliable protection for the safety of open-pit mine slopes.

Description

Technical Field

[0001] This invention relates to the field of open-pit mine slope disaster detection technology, specifically to an intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition. Background Technology

[0002] Slope stability in open-pit mines is a core element in ensuring mine production safety. Slope disasters such as landslides, rockfalls, and crack propagation can range from minor equipment damage and production stoppages to major safety accidents causing mass casualties. Slope stability is influenced by multiple factors, including geological structure, hydrological changes, and mining disturbances, placing extremely high demands on the comprehensiveness, real-time nature, and accuracy of monitoring and early warning systems.

[0003] However, existing open-pit mine slope inspection technologies still have many insurmountable shortcomings. Firstly, the limitations and dangers of traditional inspection methods. Currently, many open-pit mines still rely primarily on manual inspections, with inspectors carrying simple tools for on-site surveys. Limited by steep terrain and unpredictable weather, this is labor-intensive, inefficient, and carries safety risks such as falls from heights and rockfalls. More importantly, manual visual inspections have extremely limited ability to identify early-stage hazards such as tiny cracks, localized rockfalls, and deep deformations, resulting in high rates of missed and false diagnoses. Secondly, the "space-air-ground" solution remains stuck in the traditional paradigm of data acquisition + cloud processing. Under this architecture, the combined network transmission latency and cloud computing latency cause a significant lag in early warning. Thirdly, insufficient multi-source heterogeneous data fusion and real-time intelligent identification capabilities. Existing solutions generally suffer from single monitoring technologies and inadequate multi-source data fusion. While satellite and drone data are integrated at the data acquisition level, relatively independent algorithm modules are still used at the data analysis level, failing to achieve true deep fusion and collaborative reasoning of multimodal data such as images and point clouds at the algorithm level. More importantly, data processing relies on centralized computing power in the cloud, making it impossible to complete intelligent identification and preliminary decision-making in real time at the data acquisition end, resulting in a long time gap between the emergence of hidden dangers and system identification. Fourth, the static risk assessment system cannot adapt to the dynamic changes in slope stability. Existing slope risk assessments mostly use traditional static methods, classifying risk levels based on historical geological data and fixed thresholds, and cannot be dynamically adjusted according to real-time monitoring data. Fifth, edge computing solutions lack deep multimodal AI fusion for open-pit mine slope scenarios. Existing edge computing solutions mainly focus on single-parameter threshold judgments such as displacement and vibration, and have not yet brought advanced AI visual recognition models down to the edge side, let alone achieved multimodal fusion recognition of image and point cloud data. The application of AI intelligent video surveillance in open-pit mine slope monitoring is still in the initial exploratory stage, with particularly prominent deficiencies in data algorithm optimization and system integration management.

[0004] In view of this, existing technologies have significant bottlenecks in terms of multi-source fusion capabilities, real-time response performance, dynamic risk assessment, and intelligence level. There is an urgent need for an intelligent inspection system that can integrate multi-source sensing data from air, space, and ground, achieve real-time multimodal AI recognition at the edge, construct a dynamic risk assessment system, and form a closed-loop process to solve the above-mentioned technical problems. Summary of the Invention

[0005] In view of the technical problems existing in the background art, the present invention provides an intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition. This system relies on a four-level architecture of "edge-cloud collaboration" and an "integrated air-ground-space" sensing network to significantly improve the safety and accuracy of open-pit mine slope inspection. An aerial swarm of drones equipped with high-definition visible light cameras and thermal imaging cameras, combined with ground-based high-definition gimbal cameras, LiDAR, vibration sensors, and underground miniature displacement sensors, achieves comprehensive, blind-spot-free monitoring. Multimodal AI visual recognition technology, combined with an improved multimodal fusion recognition model, achieves a recognition accuracy of over 97%, effectively reducing the rate of missed and false judgments. An industrial-grade edge computing gateway at the edge layer enables local real-time decision-making, with response latency controlled within 500ms to 800ms, overcoming environmental and time constraints. A dynamic slope risk model at the cloud layer provides scientific assessment, while a 3D modeling engine and data traceability module support precise management. The system is designed with two implementation forms to adapt to open-pit mines of different sizes; lightweight configuration can reduce costs by 40%, and reserved interfaces facilitate integration with existing systems. The fully automated inspection process reduces manpower input by more than 80%, and the closed-loop iterative system ensures long-term optimization, taking into account safety, economy and popularization.

[0006] This invention provides an intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition, which includes a four-level architecture consisting of a perception layer, an edge layer, a cloud layer, and an application layer that are sequentially connected and form an edge-cloud collaborative architecture. The perception layer constructs an integrated air-space-ground perception network, including: The aerial monitoring unit includes a cluster of drones equipped with multispectral imaging modules, used to acquire large-scale image data of the slope surface. Ground monitoring units, consisting of visual sensors, lidar, and vibration sensors distributed in key areas of the slope, are used to acquire surface conditions and rock vibration data in key areas of the slope. The underground monitoring unit includes miniature displacement sensors implanted on the potential slip surface of the slope to acquire deep deformation data; The edge layer adopts an industrial-grade edge computing gateway with a built-in lightweight AI inference engine. The engine is configured with a dual-branch neural network. The first branch processes the image data based on a deep convolutional network to identify potential hazards on the slope surface. The second branch processes the point cloud data based on a point cloud learning network to extract the three-dimensional morphological features of the slope and calculate relevant parameters. The two branches are fused to output the recognition result. The cloud layer adopts a distributed cloud platform architecture, which includes at least: a big data engine, a time series database, a spatial database, a dynamic slope risk model, a model training platform, and a 3D modeling engine. The application layer includes at least an intelligent early warning module, an inspection management module, an emergency response module, and a data traceability module. These modules work together to achieve intelligent inspection and response of slope disasters throughout the entire process.

[0007] As a further improvement of the present invention, the multispectral imaging module in the sensing layer includes a broadband visible light sensor and a long-wave infrared thermal imaging sensor. The two adopt a time synchronization mechanism to achieve fusion acquisition and output a pseudo-color image containing temperature field information, which is used to enhance the identification contrast of micro-cracks and local slab collapse hazards at night or in severe weather conditions.

[0008] As a further improvement of the present invention, in the edge layer, the lightweight AI inference engine incorporates a multimodal fusion recognition model, which is constructed based on a deep learning framework to create a dual-branch neural network; the dual-branch neural network employs the following collaborative mechanism: The first branch introduces an improved regression loss function, which suppresses the interference of background noise on rare positive samples through a dynamic focusing mechanism, thereby improving the detection rate of small targets. The second branch introduces a differentiable implementation of the iterative nearest point algorithm to achieve end-to-end optimization of point cloud registration and feature extraction, and directly outputs the displacement vector field and slope gradient field. The feature fusion employs an adaptive attention mechanism, which dynamically adjusts the confidence weights of the two branches based on the signal-to-noise ratio of the current scene.

[0009] As a further improvement of the present invention, the improved regression loss function in the first branch is a weighted fusion of CIoU loss and Focal Loss, with the weight coefficient α satisfying 0.5≤α≤0.7 and the focusing parameter γ satisfying 1.5≤γ≤2.5; the first branch has a comprehensive identification accuracy of ≥98% for the three types of hidden dangers: cracks, spalling, and bulging.

[0010] As a further improvement of the present invention, the second branch uses the PointNet++ algorithm to extract point cloud features, registers the current point cloud with the reference point cloud through the iterative nearest point algorithm, calculates the rigid body transformation matrix to obtain the displacement, and calculates the slope change by fitting the slope plane using the least squares method; when the slope change rate of any monitoring area exceeds the preset threshold θ, the edge layer independently triggers a local early warning; the threshold θ is dynamically set according to the slope lithology grade, with 0.3°-0.5° for soft rock and 0.5°-1.0° for hard rock.

[0011] As a further improvement of the present invention, in the cloud layer, the dynamic slope risk model adopts a coupled architecture of random forest and analytic hierarchy process: The prior weights of multi-dimensional risk factors were determined by the analytic hierarchy process, including deep displacement, severity of surface hazards, slope change rate, rock mass vibration energy, geological structure complexity, meteorological and hydrological conditions, and timeliness of monitoring data. The random forest algorithm is used to learn nonlinear correlation patterns in historical disaster cases, dynamically adjust the prior weights, and generate a real-time risk value R. The risk value R is used to divide the warning range into four levels: R<0.3 is the stable level, 0.3≤R<0.6 is the attention level, 0.6≤R<0.85 is the warning level, and R≥0.85 is the danger level.

[0012] As a further improvement of the present invention, in the cloud layer, the model training platform receives the labeled data uploaded by the edge layer and uses an incremental learning algorithm to periodically optimize the multimodal fusion recognition model to achieve a closed-loop iteration of data-model-accuracy.

[0013] As a further improvement of the present invention, in the application layer, the intelligent early warning module pushes early warning information through at least two push channels based on the severity of the hidden danger and the risk level output by the dynamic slope risk model, and automatically associates it with the corresponding response plan. The specific graded early warning strategy is as follows: Stable and Attention Levels: Notifications are sent via mobile push notifications and monitoring platform pop-ups. Warning level: Additionally, the on-site audible and visual alarm device on the slope is triggered, and the electronic fence of the danger zone is automatically locked; Hazard Level: Initiate an emergency command chain response, automatically dispatch the nearest drone to conduct a close-range verification, and simultaneously push the data to the mine emergency command center; And / or, based on the output of the dynamic slope risk model, the inspection management module uses reinforcement learning algorithms to dynamically optimize the UAV inspection path and sensor sampling frequency, implement key and intensive monitoring in high-risk areas, and implement sparse and routine monitoring in stable areas.

[0014] As a further improvement of the present invention, the system supports dual-mode deployment: Standard deployment configuration: Configured with a complete air-space-ground monitoring network and full-featured edge computing nodes, suitable for large open-pit mines; Lightweight deployment: Reduce the size of drone swarms and replace industrial-grade edge computing gateways with compact AI modules, reducing hardware costs by more than 40%, making it suitable for small and medium-sized open-pit mines.

[0015] As a further improvement of the present invention, the edge layer independently completes the entire process of data acquisition, intelligent recognition, and preliminary decision-making, and selectively uploads the recognition results and raw data to the cloud layer; The 3D modeling engine in the cloud layer receives point cloud data and high-definition images collected by the aerial monitoring unit, and after fusion processing, generates a dynamic 3D model of the slope to intuitively display the slope deformation process. The cloud layer reserves standard API interfaces for connecting to the mine's existing production management system and safety monitoring platform to achieve data sharing and collaborative management.

[0016] As a further improvement of the present invention, the response latency of the industrial-grade edge computing gateway is controlled within 500ms, and the industrial-grade edge computing gateway supports dual-mode communication of 5G and industrial Ethernet to ensure the stability of data transmission.

[0017] As a further improvement of the present invention, the dynamic slope risk model in the cloud layer is constructed based on historical hazard data, real-time monitoring data and mine geological data. The dynamic slope risk model dynamically updates the slope risk level through machine learning algorithms to overcome the limitations of traditional static risk assessment.

[0018] As a further improvement of the present invention, the three-dimensional modeling engine in the cloud layer receives point cloud data and high-definition images collected by the UAV cluster. The three-dimensional modeling engine fuses the point cloud data and the high-definition images to generate a dynamic three-dimensional model of the slope. The dynamic three-dimensional model of the slope is used to intuitively display the slope deformation process.

[0019] Beneficial effects: 1. The system provided by this invention transforms slope inspection from "passive handling" to "proactive prevention," significantly improving the safety of open-pit mine production. Based on a four-level "edge-cloud collaborative" architecture, this system achieves full-area coverage monitoring through an integrated air-ground-space sensing network. A cluster of drones deployed in the air carries high-definition visible light cameras and thermal imaging cameras, while high-definition gimbal cameras, lidar, vibration sensors, and miniature displacement sensors are deployed on the ground, forming a comprehensive, blind-spot-free sensing system. Compared to traditional manual inspections and fixed-point monitoring, this system completely solves the problems of limited coverage and high rate of missed hazard detection. In particular, the application of multimodal AI visual recognition technology, combined with the YOLOv8 algorithm optimized through transfer learning and the PointNet++ algorithm, can accurately identify surface hazards such as cracks, spalling, and slope bulges, while simultaneously capturing deeper features such as slope displacement and gradient changes. The recognition accuracy reaches over 97%, enabling early detection and warning of minor hazards, reducing the probability of slope disasters from the source.

[0020] 2. The system provided by this invention overcomes environmental and time constraints, enhancing the all-weather adaptability and real-time response capability of inspections. The system creatively introduces a fusion acquisition technology combining high-definition visible light cameras and thermal imaging cameras, coupled with industrial-grade protective sensing equipment. It can operate stably under harsh conditions such as nighttime, heavy fog, dust storms, high temperatures, and heavy rain, breaking the dependence of traditional visual monitoring on the environment. More importantly, the industrial-grade edge computing gateway at the edge layer enables localized processing of the entire process from "data acquisition to identification and analysis to preliminary decision-making," with response latency controlled within 500ms to 800ms. Compared to traditional solutions relying on cloud computing power, this significantly shortens the time for hazard identification and early warning response, allowing sufficient time for on-site emergency response and effectively preventing hazards from escalating into major disasters.

[0021] 3. The system provided by this invention significantly reduces the false positive rate and improves the accuracy of inspections through multi-technology integration and innovation. The system adopts an improved multimodal fusion recognition model, constructing a dual-branch network based on a deep learning framework. The first branch network solves the problem of imbalance between positive and negative samples of slope hazard samples by using an improved loss function that combines the CIoU loss function and FocalLoss weighted fusion. The second branch network combines the iterative nearest point algorithm and the least squares method to achieve accurate three-dimensional morphological analysis. The judgment result is output after the dual-branch results are weighted and fused, making the recognition accuracy of slope surface hazards and deep deformations far exceed that of single data source recognition schemes. The recognition accuracy reaches over 98.5% in large open-pit mine scenarios and over 94.7% in small and medium-sized open-pit mine scenarios, effectively avoiding false positives and false negatives and reducing ineffective handling costs.

[0022] 4. The dynamic risk assessment system provided by this invention is more scientific, offering precise decision-making support for safety management. The dynamic slope risk model deployed in the cloud layer of the system, based on historical hazard data, real-time monitoring data, and mine geological data, dynamically updates the risk level through multi-feature weighted calculation. Compared to traditional static risk assessment schemes, it better reflects the changing patterns of slope stability, accurately classifying risks into four levels: low, medium, high, and extremely high risk, providing managers with targeted action guidelines. Simultaneously, the centimeter-level dynamic 3D slope model generated by the 3D modeling engine can intuitively display the slope deformation process. Combined with the full lifecycle information of hazards recorded by the data traceability module, it provides complete data support for hazard tracing and optimization of response plans, improving the precision of safety management.

[0023] 5. The system provided by this invention is adaptable to the needs of open-pit mines of different sizes, balancing high precision and economy, and improving the accessibility of the technology. The system is designed in two implementation forms: for large open-pit mines, a complete "space-air-ground integrated" sensing network and full-function modules are configured to meet the needs of high-precision inspection across the entire area; for small and medium-sized open-pit mines, a lightweight configuration is adopted. This is achieved by simplifying the edge layer industrial-grade edge computing gateway to a Huawei Atlas200IDKA2 edge computing module, removing non-core functions, and optimizing the algorithm model, thereby reducing equipment and deployment costs by 40% while ensuring core monitoring and early warning functions. Simultaneously, the cloud layer reserves standard API interfaces, which can be connected to existing mine production management systems and safety monitoring platforms without reconstructing the existing system, lowering the deployment threshold and facilitating widespread application in open-pit mines of different sizes.

[0024] 6. The system provided by this invention achieves full automation and intelligence in the inspection process, significantly reducing labor costs and intensity. The application-layer inspection management module supports automatic drone path planning and customized ground-based high-definition gimbal camera patrol missions, eliminating the need for on-site manual operation and enabling fully automated inspections. Compared to traditional manual inspections, it reduces manpower input by more than 80%, while avoiding safety risks such as high-altitude operations and slope landslides associated with manual inspections. Furthermore, the intelligent early warning module employs a tiered early warning mechanism, pushing early warning information through multiple channels and automatically linking it to contingency plans. The emergency response module assists in quickly developing solutions, and the data traceability module creates an immutable safety ledger, simplifying the entire management process and improving safety management efficiency.

[0025] 7. The system provided by this invention constructs a closed-loop iterative system of "data-model-accuracy," ensuring long-term stable optimization. The cloud-layer model training platform receives labeled data uploaded from the edge layer and periodically optimizes the multimodal fusion recognition model through incremental learning algorithms. This allows the model's recognition accuracy to gradually improve over time, avoiding performance degradation due to environmental changes or slope morphology alterations. This self-optimization capability ensures the system operates efficiently for extended periods, eliminating the need for frequent replacements of core algorithms or equipment, reducing maintenance costs, and extending system lifespan.

[0026] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0027] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0028] Figure 1 This is a four-level system architecture diagram of the intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition provided in an embodiment of the present invention.

[0029] Figure 2 This is a diagram showing the composition of the perception layer of the intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition, provided in an embodiment of the present invention.

[0030] Figure 3 This is a flowchart of the edge layer processing of the intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition provided in an embodiment of the present invention.

[0031] Figure 4 This is a cloud layer composition diagram of the intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition provided in an embodiment of the present invention.

[0032] Figure 5 This is an application layer module diagram of the intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition provided in an embodiment of the present invention. Detailed Implementation

[0033] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0034] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the invention, are intended to cover non-exclusive inclusion.

[0035] In the description of the embodiments of this invention, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this invention, "multiple" means two or more, unless otherwise explicitly defined.

[0036] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0037] In the description of the embodiments of this invention, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0038] In the description of the embodiments of the present invention, the term "multiple" refers to two or more (including two), similarly, "multiple groups" refers to two or more (including two groups), and "multiple pieces" refers to two or more (including two pieces).

[0039] In the description of the embodiments of the present invention, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of the present invention.

[0040] In the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention according to the specific circumstances.

[0041] To address the significant bottlenecks in existing technologies regarding multi-source fusion capabilities, real-time response performance, dynamic risk assessment, and intelligence levels, this invention provides an intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition.

[0042] Please refer to Figures 1 to 5 As shown, the system constructs a four-level architecture of "edge-cloud collaboration" and an "integrated air-space-ground" sensing network. The system sequentially connects the sensing layer, edge layer, cloud layer, and application layer to form a complete link for edge-cloud collaboration.

[0043] Please refer to Figure 2 As shown, the perception layer integrates aerial drone swarms (visible light and thermal imaging), ground monitoring equipment (high-definition gimbal cameras, lidar, vibration sensors), and underground micro-displacement sensors to achieve comprehensive, three-dimensional data acquisition of open-pit mine slopes from the air to the surface and underground. This design breaks through the limitations of traditional single-point, localized monitoring, providing a high-quality, multi-dimensional data foundation for subsequent intelligent identification.

[0044] Please refer to Figure 3 As shown, the edge layer employs an industrial-grade edge computing gateway, which is equipped with a lightweight AI inference engine. This lightweight AI inference engine incorporates a multimodal fusion recognition model. The multimodal fusion recognition model is built upon a deep learning framework to construct a dual-branch network, enabling accurate local real-time analysis. In the dual-branch network, the first branch processes visible light and thermal imaging images using a YOLOv8 algorithm optimized through transfer learning, identifying surface hazards such as cracks, spalling, and bulges. The second branch uses the PointNet++ algorithm to process lidar point cloud data, extracting three-dimensional morphological parameters such as slope displacement and slope change. The results from both branches are fused to output the recognition conclusion. This solution is the first to combine multimodal data fusion with real-time edge computing, controlling the response latency to within 500ms and achieving a recognition accuracy of over 98.5%, completely solving the pain points of high latency and high misjudgment rate from single data sources in traditional cloud processing.

[0045] Please refer to Figure 4As shown, the cloud layer adopts a distributed cloud platform architecture, which includes a big data engine, a time-series database, a spatial database, a dynamic slope risk model, a model training platform, and a 3D modeling engine. The cloud layer establishes a dynamic slope risk model and a data-model-accuracy closed-loop iterative mechanism. Based on historical hazard data, real-time monitoring data, and mine geological data, machine learning algorithms (such as random forests) are used to dynamically calculate and update the slope risk level in real time, replacing traditional static risk assessment. Simultaneously, the cloud model training platform receives labeled data uploaded from the edge layer and continuously optimizes the multimodal fusion recognition model through incremental learning, forming a data-driven, self-reinforcing closed-loop iteration. This mechanism enables risk assessment to accurately reflect changes in slope stability, and the system's long-term operating accuracy continuously improves, ensuring the continuous reliability of monitoring and early warning. A tiered early warning mechanism and lightweight deployment scheme are designed, balancing high accuracy and universality.

[0046] Please refer to Figure 5 As shown, the application layer includes an intelligent early warning module, an inspection management module, an emergency response module, and a data traceability module. These modules work together to achieve intelligent inspection and response of slope disasters throughout the entire process. The intelligent early warning module in the application layer pushes early warnings in a tiered manner through multiple channels such as SMS, platform pop-ups, and on-site audible and visual alarms, based on the severity of the hazard and the dynamic risk level. It also automatically associates the warnings with corresponding emergency response plans, achieving full automation from discovery to response.

[0047] Furthermore, the system supports lightweight deployment: simplifying edge layer configuration reduces equipment and deployment costs by approximately 40% while ensuring core monitoring functions, making it suitable for small and medium-sized mines. This design meets the high-precision requirements of large mines while lowering the technical application threshold, significantly improving the system's accessibility and cost-effectiveness.

[0048] Example 1 Please refer to Figures 1 to 5 As shown in Embodiment 1 of the present invention, an intelligent inspection system for slope hazards in open-pit mines equipped with AI visual recognition is provided. This system is applied to a large open-pit gold mine where the slope height reaches 120 meters, the number of steps is 8, and the total length of the slope is 2000 meters. The system needs to achieve high-precision hazard monitoring and early warning for the entire area and all time periods.

[0049] The specific implementation details for each level of the system are as follows: The perception layer, serving as the core of data acquisition, adopts an integrated air-ground-space perception network layout. The aerial deployment utilizes a swarm of DJI Matrice 350RTK industrial-grade drones, totaling six aircraft, forming a group inspection mode. Each drone is equipped with a high-definition visible light camera and a thermal imaging camera. The high-definition visible light camera has a resolution of 45 megapixels, capable of clearly capturing minute cracks on the slope surface down to 0.1 millimeters. The thermal imaging camera has a temperature measurement range of -20℃ to 550℃ and a thermal sensitivity of 0.01℃, enabling it to accurately identify potential hazards such as small rockfalls and slope bulges by measuring the temperature difference between the slope rock mass and the surrounding environment, even under adverse weather conditions such as nighttime, heavy fog, and dust storms. To achieve accurate image data fusion, this embodiment uses a time synchronizer to synchronize the acquisition frame rates of both the high-definition visible light camera and the thermal imaging camera to 30 frames per second, ensuring complete temporal matching of the two images from the same monitoring point. This provides a high-quality data foundation for subsequent multimodal fusion and recognition at the edge layer.

[0050] Ground-based monitoring equipment is deployed according to key slope areas: a high-definition pan-tilt camera is placed every 20 meters in key areas such as the stepped slope surface, the edge of the slope top, and the water catchment area at the slope toe. This camera supports 360° panoramic cruise and automatic zoom, and the monitoring angle can be remotely controlled and adjusted via a cloud platform to achieve comprehensive monitoring of key areas. A lidar system with a ranging range of 0.1 meters to 100 meters and a point cloud density of 300,000 points / second is used to collect real-time three-dimensional spatial coordinate data of the slope and capture minute displacement changes on the slope surface. Vibration sensors with a measurement range of 0.01Hz to 1000Hz and a sensitivity of 100mV / g are deployed inside the slope rock mass to monitor rock vibration signals and assist in assessing slope stability. Underground, miniature displacement sensors are buried in layers at depths of 1 / 3 and 1 / 2 of the slope height, with a total of 12 monitoring points, to collect deep deformation data of the slope and avoid misjudgments of potential hazards caused by relying solely on surface monitoring.

[0051] The edge layer is the core of real-time response, with its core component being an industrial-grade edge computing gateway. This gateway utilizes edge computing nodes and possesses powerful edge inference capabilities. The edge computing gateway establishes communication connections with all devices in the perception layer via industrial Ethernet, achieving a data transmission rate of 1000Mbps. It enables fully localized processing of the entire process from "data acquisition to identification and analysis to preliminary decision-making," with response latency controlled within 500ms. The edge layer's built-in lightweight AI inference engine is optimized based on TensorRT and deploys a multimodal fusion recognition model. This model is built on the deep learning framework PyTorch, constructing a dual-branch network. The specific implementation logic is as follows: The first branch network processes image data acquired by high-definition visible light cameras and thermal imaging cameras, employing a YOLOv8 algorithm optimized through transfer learning. To address the imbalance between positive and negative samples in the slope hazard sample (the proportion of hazard areas is much smaller than that of normal areas), this embodiment introduces an improved loss function, using a weighted fusion of the CIoU loss function and FocalLoss. The mathematical expression is as follows: ; in, This is the total loss value of the first branch network, used to guide the iterative optimization of network parameters and ensure that the model learns effective features; The weighting coefficient is 0.6, which has been verified through extensive experiments to achieve the optimal balance between the two loss functions. The CIoU loss function measures the difference in position, size, and aspect ratio between the predicted bounding box and the ground truth bounding box. Its expression is: ; In the formula, Intersection over Union (IoU) measures the degree of overlap between two bounding boxes. To predict the center of the bounding box Center of the true bounding box The square of the Euclidean distance; It is the length of the diagonal of the smallest bounding rectangle containing both bounding boxes; These are the weighting coefficients. The aspect ratio consistency parameter is expressed as follows: ; in, The actual bounding box width and height, To predict the width and height of the bounding box.

[0052] FocalLoss is the loss function used to reduce the weight of simple negative samples. Its expression is: ; in, Predict the probability (positive sample) for the model negative samples ), To focus on parameters and improve the model's ability to identify a small number of potential hazards, the first branch network, trained using this loss function, achieved an accuracy of over 98.5% in identifying cracks, spalling, and slope bulges.

[0053] The second branch network uses the PointNet++ algorithm to process lidar point cloud data. Its core function is to extract the three-dimensional morphological features of the slope and calculate slope displacement and gradient change parameters. To accurately calculate slope displacement, the Iterative Closest Point (ICP) algorithm is introduced for point cloud registration. The current point cloud is registered with the reference point cloud (the point cloud without hidden dangers collected during the initial system deployment), and the rigid body transformation matrix is ​​calculated to obtain the displacement. The core mathematical model is as follows: Let the reference point cloud... Current point cloud ( Three-dimensional coordinates The goal is to find the optimal rotation matrix. (3×3) and translation vector (3×1), to minimize the distance between two point clouds, the objective function is: ; in, The L2 norm (Euclidean distance) is obtained through iterative solution. and Afterwards, through The displacement at each point is calculated to obtain the overall displacement distribution of the slope. Slope change calculation is based on registered point cloud data, and the slope plane is fitted using the least squares method. The plane equation is: ( (For plane parameters), slope The calculation formula is: ; in, The slope is the plane slope, which is converted to a slope angle (in degrees) using the arctangent function. When the slope change exceeds a preset threshold of 0.5°, a preliminary warning is triggered.

[0054] The outputs of the two-branch network are fused using a weighted summation method. Let the confidence level of the first branch identification be... The feature matching degree of the second branch is Finally, the confidence level is identified. The expression is: ;(in, Image recognition is given higher weight because it more intuitively reflects surface hazards.

[0055] when If a potential hazard is identified, the edge gateway generates a preliminary decision and synchronizes it to the cloud. when At that time, secondary recognition is triggered; when At that time, it was determined that there were no hidden dangers.

[0056] The edge computing gateway supports dual-mode communication of 5G and industrial Ethernet, and uses a China Mobile industrial-grade 5G module. When the industrial Ethernet fails, it automatically switches to 5G to ensure stable data transmission.

[0057] The cloud layer is built on Huawei Cloud Stack and deployed in the mine's local data center to ensure data security, adopting a distributed cloud platform architecture. The cloud-based big data engine uses Apache Flink, supporting high-throughput, low-latency data processing and capable of processing various types of data uploaded from the edge layer in real time. The time-series database uses InfluxDB, specifically storing timestamped monitoring data (displacement, vibration, slope, etc.) in the format "timestamp-device ID-parameter name-parameter value," facilitating trend analysis and historical tracing. The spatial database uses PostGIS, storing 3D slope model data, equipment deployment coordinates, and other spatial information, supporting spatial querying and analysis.

[0058] The dynamic slope risk model is constructed using a random forest algorithm. Input features include: slope surface hazard type (crack length, rockfall volume), deep displacement, slope change, vibration amplitude, mine geological lithology (quantified according to hardness levels 1-5), rainfall (data from meteorological departments), and monitoring duration. Risk levels are divided into low risk (Level 1), medium risk (Level 2), high risk (Level 3), and extremely high risk (Level 4). The assessment mathematical model is as follows: ; in, This is a risk value used to determine the risk level; The number of input features; The characteristic weights (determined using the Analytic Hierarchy Process (AHP): deep displacement 0.3, hazard type 0.25, slope change 0.15, vibration amplitude 0.1, geological lithology 0.1, rainfall 0.08, and monitoring duration 0.02) are as follows: The standardized values ​​of the features are calculated using min-max standardization: actual ; in, For the actual value of the characteristic, These represent the minimum and maximum values ​​of the feature.

[0059] Risk level assessment criteria: Low risk Medium risk. High risk, This is considered extremely high risk. The model updates the risk level hourly, and triggers corresponding warnings for medium and higher risks.

[0060] The cloud-based model training platform, built on TensorFlow, receives manually labeled data (hazard images, point cloud data, and labels) uploaded from the edge layer. It employs an incremental learning algorithm to fine-tune the multimodal fusion recognition model every 30 days, updating model parameters to achieve a closed-loop iteration of "data-model-accuracy." The 3D modeling engine uses ContextCapture, receiving point cloud data and high-resolution images collected by drones. Through distortion correction, feature extraction, point cloud-image matching, dense point cloud generation, mesh construction, and texture mapping, it generates a dynamic 3D slope model with centimeter-level accuracy, intuitively demonstrating the slope deformation process.

[0061] The application layer is based on a B / S architecture, using the Vue.js front-end framework and the Spring Boot back-end framework, supporting multi-terminal access. The intelligent early warning module adopts a hierarchical early warning mechanism, determining the warning level (blue, yellow, orange, red) according to the severity of the hazard (minor, moderate, severe, extremely severe) and the risk level: blue / yellow warnings are pushed to safety management personnel via platform pop-ups and SMS; orange warnings additionally trigger on-site audible and visual alarms; red warnings add the function of automatically dialing the emergency command center and linking it to the emergency response plan (setting up a warning zone, stopping work, personnel evacuation, etc.).

[0062] The inspection management module supports automatic drone path planning and customized ground camera patrols: drone path planning adopts an improved A... The algorithm, given constraints such as obstacles (equipment, roads) and key monitoring points, generates the optimal path. Ground cameras can be set with patrol periods, angles, and dwell times via the platform to achieve fully automatic patrolling. The emergency response module integrates VR visualization functionality, using the Pico4ProVR device to visually display hazard details and the surrounding environment through a dynamic 3D model of the slope, assisting in the development of response plans. The data traceability module employs blockchain technology to record the entire lifecycle information of hazards (discovery time, location, type, and handling process), forming an immutable safety ledger.

[0063] In this embodiment, all sensing layer devices adopt industrial-grade design with an IP67 protection rating, adapting to high temperature, heavy rain, and dusty conditions. The cloud layer reserves standard API interfaces, enabling data sharing with the mine's existing SAP ERP production management system and KJ95X safety monitoring platform. Six months of trial operation showed that the system identified 48 potential hazards with no missed or false alarms. The average hazard response time was 310ms, the risk warning accuracy rate was 96.5%, and three slope disaster accidents were successfully avoided.

[0064] Example 2 Embodiment 2 of this invention provides an intelligent inspection system for slope hazards in open-pit mines equipped with AI visual recognition. This system is applied to a small to medium-sized open-pit gold mine with a slope height of 50 meters, 4 steps, and a length of 800 meters. The core requirement is to effectively monitor key hidden dangers while controlling costs. The system is based on a simplified design of the architecture of Embodiment 1, and the specific implementation details are as follows: The perception layer adopts a "simplified air-ground integrated perception network": two UAVs are deployed in the air, equipped with high-definition visible light cameras (20 million pixels resolution) and entry-level thermal imaging cameras (temperature measurement range -10℃-50℃) to meet the basic needs of hazard identification; high-definition gimbal cameras are deployed in key areas on the ground (every 30 meters), the lidar point cloud density is 200,000 points / second, and vibration sensors and micro displacement sensors are deployed according to the principle of "prioritizing key points", with 4 monitoring points each, to reduce equipment costs.

[0065] The edge layer uses the Huawei Atlas200IDKA2 edge computing module instead of the industrial-grade edge computing gateway. This module is small in size and low in cost, meeting the needs of lightweight AI inference. The edge layer still deploys a multimodal fusion recognition model, but the algorithm has been optimized for lightweight design: the first branch network uses the YOLOv8-nano model to reduce the number of network parameters; the second branch network simplifies the sampling layer structure of PointNet++, improving inference speed while maintaining recognition accuracy (≥97%). The mathematical models for the loss function, point cloud registration, and slope calculation are completely consistent with those in Example 1, with only some parameters adjusted (such as...). , To adapt to the lightweight model, the final edge layer response latency is controlled within 800ms, meeting the core needs of small and medium-sized mines.

[0066] The cloud layer is built using a lightweight application server, deploying a simplified distributed architecture: the big data engine uses a simplified version of Apache Flink, the time-series database uses InfluxDBLite, and the spatial database uses a simplified version of PostGIS, with some non-core data processing functions removed. The dynamic slope risk model simplifies the number of input features (…). The core characteristics of deep displacement, hazard type, slope change, geological lithology, and rainfall are retained, with the weights adjusted as follows: deep displacement 0.35, hazard type 0.3, slope change 0.2, geological lithology 0.1, and rainfall 0.05. The risk assessment mathematical model remains unchanged. The symbols have the same meaning as in Example 1, and the risk level determination criteria remain unchanged. The model training platform uses TensorFlow Lite, and the model is optimized every 60 days to reduce maintenance costs.

[0067] The application layer has had its VR visualization functionality removed, retaining only the core intelligent early warning, inspection management, and data traceability modules. The intelligent early warning module only retains three warning methods: platform pop-ups, SMS push notifications, and on-site audible and visual alarms; the drone path planning in the inspection management module adopts basic A... The algorithm simplifies constraint settings; the data traceability module uses a traditional encrypted database instead of blockchain technology, reducing deployment difficulty while ensuring data security. The perception layer equipment still achieves an IP65 protection rating, suitable for the working environment of small and medium-sized mines; the cloud layer reserves basic API interfaces for integration with simplified mine management systems.

[0068] The results of a 6-month trial run show that the system identified 19 potential hazards, with one missed hazard and one false hazard, achieving an identification accuracy of 94.7%. The average response time for potential hazards was 620ms, and the risk warning accuracy rate was 93%. The equipment and deployment costs were reduced by 40% compared to the first implementation, fully meeting the safety inspection needs of small and medium-sized open-pit mines.

[0069] The two embodiments provided by this invention are adapted to the needs of open-pit mines of different sizes. Through the "edge-cloud collaboration" architecture and multimodal AI visual recognition technology, they realize the intelligent upgrade of slope inspection and have good practicality and scalability.

[0070] In summary, this invention provides an intelligent inspection system for open-pit mine slope hazards equipped with AI visual recognition, relating to the field of open-pit mine slope hazard detection technology. It adopts a four-level "edge-cloud collaborative" architecture, including a perception layer, edge layer, cloud layer, and application layer. The perception layer constructs an "integrated air-ground-space" network, collecting multi-source data through drone swarms, high-definition visible light cameras, thermal imaging cameras, LiDAR, vibration sensors, and miniature displacement sensors. The edge layer deploys a multimodal fusion recognition model using an industrial-grade edge computing gateway, achieving real-time local recognition. The cloud layer handles data storage, model optimization, and dynamic risk assessment. The application layer implements functions such as early warning and inspection management. This system overcomes the limitations of traditional technologies, improves inspection accuracy and efficiency, is adaptable to mines of different sizes, and provides reliable protection for open-pit mine slope safety.

[0071] It should be noted that the present invention is not limited to the above-described embodiments. The above embodiments are merely examples, and any embodiments that have the same structure and perform the same effects as the technical concept within the scope of the present invention are included within the scope of the present invention. Furthermore, various modifications that can be conceived by those skilled in the art to the embodiments, and other ways of constructing by combining some of the constituent elements of the embodiments, without departing from the spirit of the present invention, are also included within the scope of the present invention.

Claims

1. An intelligent inspection system for slope hazards in open-pit mines equipped with AI visual recognition, characterized in that, It includes a four-level architecture consisting of a perception layer, an edge layer, a cloud layer, and an application layer, which are connected sequentially and form a collaborative end-edge-cloud architecture. The perception layer constructs an integrated air-space-ground perception network, including: The aerial monitoring unit includes a cluster of drones equipped with multispectral imaging modules, used to acquire large-scale image data of the slope surface. Ground monitoring units, consisting of visual sensors, lidar, and vibration sensors distributed in key areas of the slope, are used to acquire surface conditions and rock vibration data in key areas of the slope. The underground monitoring unit includes miniature displacement sensors implanted on the potential slip surface of the slope to acquire deep deformation data; The edge layer adopts an industrial-grade edge computing gateway with a built-in lightweight AI inference engine. The lightweight AI inference engine is configured with a dual-branch neural network. The first branch processes image data based on a deep convolutional network to identify potential hazards on the slope surface. The second branch processes point cloud data based on a point cloud learning network to extract three-dimensional morphological features of the slope and calculate relevant parameters. The two branches are fused to output the recognition result. The cloud layer adopts a distributed cloud platform architecture, which includes at least: a dynamic slope risk model, a model training platform, and a 3D modeling engine. The application layer includes at least an intelligent early warning module, an inspection management module, and an emergency response module. These modules work together to achieve intelligent inspection and response of slope disasters throughout the entire process. In the edge layer, the lightweight AI inference engine incorporates a multimodal fusion recognition model, which is based on a deep learning framework to construct a dual-branch neural network. The dual-branch neural network employs the following collaborative mechanism: The first branch introduces an improved regression loss function, which suppresses the interference of background noise on rare positive samples through a dynamic focusing mechanism, thereby improving the detection rate of small targets. The second branch introduces a differentiable implementation of the iterative nearest point algorithm to achieve end-to-end optimization of point cloud registration and feature extraction, and directly outputs the displacement vector field and slope gradient field. The feature fusion adopts an adaptive attention mechanism, which dynamically adjusts the confidence weights of the two branches according to the signal-to-noise ratio of the current scene. In the cloud layer, the dynamic slope risk model adopts a coupled architecture of random forest and analytic hierarchy process: The prior weights of multi-dimensional risk factors were determined by the analytic hierarchy process, including deep displacement, severity of surface hazards, slope change rate, rock mass vibration energy, geological structure complexity, meteorological and hydrological conditions, and timeliness of monitoring data. The random forest algorithm is used to learn nonlinear correlation patterns in historical disaster cases, dynamically adjust the prior weights, and generate a real-time risk value R. The risk value R is used to divide the warning range into four levels: R<0.3 is the stable level, 0.3≤R<0.6 is the attention level, 0.6≤R<0.85 is the warning level, and R≥0.85 is the danger level.

2. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 1, characterized in that, In the perception layer, the multispectral imaging module includes a broadband visible light sensor and a long-wave infrared thermal imaging sensor. The two sensors use a time synchronization mechanism to achieve fusion acquisition and output a pseudo-color image containing temperature field information. This image is used to enhance the contrast of identifying micro-cracks and potential localized rockfalls at night or in severe weather conditions.

3. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 1, characterized in that, The improved regression loss function in the first branch is a weighted fusion of CIoU loss and Focal Loss, with the weight coefficient α satisfying 0.5≤α≤0.7 and the focusing parameter γ satisfying 1.5≤γ≤2.5; the first branch has a comprehensive identification accuracy of ≥98% for the three types of hidden dangers: cracks, spalling, and bulging.

4. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 1, characterized in that, The second branch uses the PointNet++ algorithm to extract point cloud features, registers the current point cloud with the reference point cloud through the iterative nearest point algorithm, calculates the rigid body transformation matrix to obtain the displacement, and calculates the slope change by fitting the slope plane using the least squares method; when the slope change rate of any monitoring area exceeds the preset threshold θ, the edge layer independently triggers a local early warning. The preset threshold θ is dynamically set according to the slope lithology grade, with 0.3°-0.5° for soft rock and 0.5°-1.0° for hard rock.

5. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 1, characterized in that, In the cloud layer, the model training platform receives the labeled data uploaded by the edge layer and uses an incremental learning algorithm to periodically optimize the multimodal fusion recognition model, thereby achieving a closed-loop iteration of data-model-accuracy.

6. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 2, characterized in that, In the application layer, the intelligent early warning module pushes early warning information through at least two channels based on the severity of the hidden danger and the risk level output by the dynamic slope risk model, and automatically associates it with the corresponding response plan. The specific hierarchical early warning strategy is as follows: Stable and Attention Levels: Notifications are sent via mobile push notifications and monitoring platform pop-ups. Warning level: Additionally, the on-site audible and visual alarm device on the slope is triggered, and the electronic fence of the danger zone is automatically locked; Hazard Level: Initiate an emergency command chain response, automatically dispatch the nearest drone to conduct a close-range verification, and simultaneously push the data to the mine emergency command center; And / or, based on the output of the dynamic slope risk model, the inspection management module uses reinforcement learning algorithms to dynamically optimize the UAV inspection path and sensor sampling frequency, implement key and intensive monitoring in high-risk areas, and implement sparse and routine monitoring in stable areas.

7. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 1, characterized in that, The system supports dual-mode deployment: Standard deployment configuration: Configured with a complete air-space-ground monitoring network and full-featured edge computing nodes, suitable for large open-pit mines; Lightweight deployment: Reduce the size of drone swarms and replace industrial-grade edge computing gateways with compact AI modules, reducing hardware costs by more than 40%, making it suitable for small and medium-sized open-pit mines.

8. The intelligent inspection system for open-pit mine slope disasters equipped with AI visual recognition as described in claim 1, characterized in that, The edge layer independently completes the entire process of data acquisition, intelligent recognition, and preliminary decision-making, and selectively uploads the recognition results and raw data to the cloud layer. The 3D modeling engine in the cloud layer receives point cloud data and high-definition images collected by the aerial monitoring unit, and after fusion processing, generates a dynamic 3D model of the slope to intuitively display the slope deformation process. The cloud layer reserves standard API interfaces for connecting to the mine's existing production management system and safety monitoring platform to achieve data sharing and collaborative management.

Images