A dangerous intelligent perception method for open slope pumice and related equipment

Abstract

The present application relates to the technical field of mine safety production and geological disaster monitoring, in particular to a kind of open pit slope floatstone dangerous intelligent sensing method and related equipment, method is, obtain target slope image, and reconstruct depth map.Prepreg floatstone region segmentation is carried out to slope image using pre-trained PSPNet image segmentation model, and floatstone segmentation map is generated.Pixel-level fusion is carried out to segmentation map and depth map, and three-dimensional point cloud data of floatstone is extracted.Convex hull algorithm is used to carry out three-dimensional grid reconstruction to point cloud, and the geometric characteristic parameter of floatstone is calculated, based on fuzzy comprehensive evaluation theory, in combination with above-mentioned geometric characteristic parameter, the danger level of floatstone is comprehensively evaluated, and scientific basis is provided for mine safety management.The method realizes the full-process automation from image acquisition to danger level evaluation, improves the evaluation efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of mine safety production and geological disaster monitoring technology, specifically to an intelligent sensing method and related equipment for the danger of loose rocks on open-pit slopes. Background Technology

[0002] Loose rocks on open-pit mine slopes pose a significant safety hazard. Their complex shapes and uneven distribution make traditional manual detection methods inefficient, risky, and ill-suited to complex environments. While image segmentation methods have made automatic identification of loose rocks possible with the development of computer vision technology, these rocks in open-pit mines often exhibit characteristics such as small size, blurred boundaries, and strong background interference. Existing general segmentation models, such as the original PSPNet (Pyramid Scene Parsing Network) and U-Net, are prone to missed detections, false detections, and inaccurate boundary segmentation in such complex environments. Furthermore, assessing the hazard of loose rocks requires comprehensive consideration of their geometric features and environmental parameters; traditional single-factor analysis methods struggle to accurately quantify their overall risk level. Therefore, developing a high-precision loose rock segmentation model for mining scenarios and constructing a collaborative multi-dimensional risk assessment system is of significant practical importance for improving mine safety management. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and related equipment for intelligent perception of the danger of loose rocks on open slopes, which addresses the shortcomings of the prior art and solves the technical problem that traditional single-factor analysis methods are difficult to accurately quantify the comprehensive risk level.

[0004] The objective of this invention is achieved through the following technical solutions: In a first aspect, the present invention provides a method for intelligently sensing the danger of loose rocks on an open slope, comprising: Acquire images of the target slope within the open-pit mine area; The target slope image is segmented into a pumice region based on a pre-trained image segmentation model to obtain a pumice segmentation map; the image segmentation model adopts the PSPNet model. Obtain the depth map of the target slope, perform pixel-level fusion of the pumice segmentation map and the depth map, and extract the three-dimensional point cloud data of the pumice based on the fused image; Based on the three-dimensional point cloud data, a three-dimensional mesh is reconstructed using the convex hull algorithm to extract the geometric feature parameters of the pumice. Based on the geometric feature parameters and fuzzy comprehensive evaluation theory, the danger level of the pumice is obtained. The geometric feature parameters include at least one of volume, mass, shape factor, initial height, average slope, and average slope change rate.

[0005] As a further improvement of the present invention, acquiring images of target slopes within open-pit mines includes: The original slope images of the open-pit mine area were obtained by combining drone aerial photography and ground-based fixed-point data acquisition. The original slope image is preprocessed, which includes cropping the original slope image to a uniform size and performing normalization. Data augmentation is performed on the preprocessed original slope image. The data augmentation operations include at least one of horizontal flipping, vertical flipping, random rotation, brightness adjustment, adding Gaussian noise, and color dithering. The original slope after data augmentation was manually annotated to distinguish the pumice area from the background area.

[0006] As a further improvement of the present invention, the PSPNet model is an improved PSPNet model, and the steps for improving the PSPNet model include: ResNet-50 was used as the backbone network of the PSPNet model; An optimization module is introduced after the feature extraction stage of the backbone network. The optimization module includes at least one of a dynamic convolution module, a depthwise separable convolution module, and an SE channel attention module. The dynamic convolution module is used to perform global average pooling on the feature map obtained in the feature extraction stage to obtain the channel description vector. The weights of the channel description vectors are calculated through different attention branches. The original convolution kernels are weighted and aggregated based on the weights to obtain the dynamic convolution kernel weights. The channel description vectors are then convolved based on the dynamic convolution kernel weights. The depthwise separable convolutional module replaces some of the standard convolutional layers in the PSPNet model; The SE channel attention module is set after the output channel of the backbone network. It is used to learn the importance weights of the output channel through global average pooling and two fully connected layers, and multiply them with the original feature map channel by channel to achieve feature recalibration.

[0007] As a further improvement of the present invention, the attention branch includes at least one of channel attention, filter attention, spatial attention, and kernel selection attention.

[0008] As a further improvement of the present invention, the pumice segmentation map and the slope depth map are fused at the pixel level, and the three-dimensional point cloud data of the pumice is extracted based on the fused image, including: By acquiring multi-view image data of the target slope using drones, and matching it with motion reconstruction structure and multi-view stereo technology, a depth map of the target slope is generated. The pumice segmentation map output by the improved PSPNet model is fused with the depth map by pixel-wise multiplication, retaining the depth value corresponding to the pumice region and filtering out the background depth information. Based on the camera intrinsic parameter matrix, the pixel coordinates and depth values ​​of the pumice area are transformed into three-dimensional coordinates in the camera coordinate system; By using the camera extrinsic parameter matrix, the 3D coordinates in the camera coordinate system are transformed to the world coordinate system to obtain the 3D point cloud data of the pumice.

[0009] As a further improvement of the present invention, based on the three-dimensional point cloud data, a three-dimensional mesh is reconstructed using a convex hull algorithm to extract the geometric feature parameters of the pumice, including: An improved convex hull algorithm is used to reconstruct 3D meshes from 3D point cloud data; the improved convex hull algorithm includes dynamic region optimization and depth weight correction. Based on the reconstructed three-dimensional mesh model, the volume of the pumice is calculated, and the mass of the pumice is obtained according to the volume and the preset rock density. Based on the three-dimensional grid of the pumice, the shape factor, initial height, average slope and average slope change rate of the area where the pumice is located are calculated.

[0010] As a further improvement of the present invention, based on the geometric feature parameters and fuzzy comprehensive evaluation theory, the hazard level of the pumice is obtained, including: Establish a pumice hazard assessment index system that includes the aforementioned geometric feature parameters, and determine several hazard assessment levels within the pumice hazard assessment index system; For each evaluation indicator, a membership function corresponding to different evaluation levels is constructed to form a fuzzy relation matrix; The weight vector of each evaluation index is determined using the analytic hierarchy process (AHP). Based on the fuzzy relation matrix and the weight vectors of each evaluation index, the fuzzy comprehensive evaluation vector is calculated according to the fuzzy comprehensive evaluation formula. Based on the fuzzy comprehensive evaluation vector, the comprehensive risk index is calculated using the weighted average method, and the final risk level is determined according to the preset risk level threshold.

[0011] Secondly, the present invention provides a hazard intelligent sensing system for loose rocks on open slopes, comprising: The data acquisition unit is used to acquire images of the target slope within the open-pit mine area; The image segmentation unit is used to segment the pumice region of the target slope image based on a pre-trained image segmentation model to obtain a pumice segmentation map; the image segmentation model adopts the PSPNet model. The point cloud extraction unit is used to perform pixel-level fusion of the pumice segmentation map and the target slope depth map, and extract the three-dimensional point cloud data of the pumice based on the fused image; The feature extraction unit is used to extract the geometric feature parameters of the pumice by performing three-dimensional mesh reconstruction based on the three-dimensional point cloud data using the convex hull algorithm. The risk assessment unit is used to obtain the hazard level of the pumice based on the geometric feature parameters and fuzzy comprehensive evaluation theory; the geometric feature parameters include at least one of volume, mass, shape factor, initial height, average slope and average slope change rate.

[0012] Thirdly, the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the above-described intelligent hazard sensing method for loose rocks on open slopes.

[0013] Fourthly, the present invention provides a computer device, comprising: a processor and a computer-readable storage medium; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the above-described intelligent hazard sensing method for loose rocks on open slopes.

[0014] The beneficial effects of this invention are as follows: This invention provides an intelligent perception method for the danger of floating rocks on open-pit slopes. By employing a pre-trained PSPNet model to segment the floating rock region in the target slope image of an open-pit mine, it achieves the acquisition of a high-precision floating rock segmentation map, significantly improving segmentation efficiency and boundary recognition accuracy compared to traditional image segmentation methods. After pixel-level fusion of the floating rock segmentation map and the depth map, 3D point cloud data is extracted, effectively overcoming the problem of incomplete point cloud data caused by the lack of depth information from a single viewpoint, ensuring the spatial coordinate consistency of the 3D point cloud data. Based on the convex hull algorithm, 3D mesh reconstruction is performed, and geometric feature parameters such as volume, mass, and shape factor are extracted, avoiding errors caused by manual measurement. Combined with fuzzy comprehensive evaluation theory, the geometric feature parameters are used to assess the hazard level, constructing a multi-parameter coupled quantitative evaluation system. This invention achieves automation, precision, and systematization of the floating rock hazard level assessment in open-pit mines, significantly reducing the cost of manual intervention while improving operational safety.

[0015] This invention addresses the challenges of small, blurred, and complex backgrounds in open-pit mines with floating rocks. It specifically improves the PSPNet model by introducing dynamic convolution, depthwise separable convolution, and SE attention mechanisms. This significantly enhances the model's segmentation accuracy and boundary clarity for small floating rocks in complex scenes, increasing mIoU to 81.94% and providing high-quality segmentation results for subsequent feature extraction.

[0016] This invention innovatively fuses two-dimensional image segmentation results with three-dimensional depth maps at the pixel level and combines them with an improved convex hull algorithm for three-dimensional reconstruction, achieving high-precision, non-contact intelligent extraction of geometric features such as volume, mass, and shape factor of pumice stones. The volume calculation accuracy reaches 89.6%, and the initial height calculation accuracy reaches 97.4%.

[0017] This invention constructs a multi-index risk assessment system that integrates the geometric characteristics of pumice with the environmental parameters of the slope it is located on, and uses the fuzzy comprehensive evaluation method to quantify the uncertain factors, thereby realizing the scientific classification of the danger level of pumice and providing an intuitive and quantitative decision-making basis for mine safety management and risk prevention and control.

[0018] This invention combines computer vision, 3D reconstruction, and fuzzy decision theory to form a complete technical solution from data acquisition, intelligent recognition, feature extraction to risk assessment. It realizes automated and intelligent perception of the danger of loose rocks on open slopes and effectively overcomes the drawbacks of traditional manual inspection. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the intelligent sensing method for the danger of loose rocks on open slopes according to the present invention. Figure 2 This is a schematic diagram of the dynamic convolution module structure in an embodiment of the present invention; Figure 3 This is a schematic diagram of a depth-separable convolution module in an embodiment of the present invention; Figure 4 This is a schematic diagram of the SE attention mechanism module in an embodiment of the present invention; Figure 5 This is a schematic diagram of the improved PSPNet model structure in an embodiment of the present invention; Figure 6 This is a comparison chart of segmentation results from different models in this embodiment of the invention; Figure 7 This is a schematic diagram of the fusion of depth map and segmentation map in an embodiment of the present invention; Figure 8 This is a reconstruction effect diagram of the pumice three-dimensional mesh in an embodiment of the present invention; Figure 9 This is a visual distribution map of the pumice hazard levels in an embodiment of the present invention; Figure 10This is an internal structural diagram of the computer device in an embodiment of the present invention. Detailed Implementation

[0021] To make the objectives and technical solutions of this invention clearer and easier to understand, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention.

[0022] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings and specific embodiments. The described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] Example 1 Traditional image segmentation models lack accuracy in identifying pumice areas in complex slope environments, making it difficult to distinguish pumice from background texture. Furthermore, two-dimensional images cannot capture the true three-dimensional size and spatial location of pumice, leading to significant discrepancies between hazard assessment results and actual site conditions, thus failing to provide reliable data support for slope safety management.

[0024] Therefore, as Figure 1 As shown in the figure, this embodiment provides a method for intelligently sensing the danger of loose rocks on an open slope. The specific implementation method is as follows.

[0025] S1: Obtain the target slope image within the open-pit mine area.

[0026] Specifically, a combination of drone aerial photography and ground-based fixed-point data acquisition is used to collect images of the target slope within the open-pit mine area. This effectively covers all areas of the target slope, eliminating blind spots that can easily occur with manual inspections and single-source acquisition methods, and ensuring the integrity of the collected data. The acquired raw slope images undergo preprocessing, which includes cropping the raw slope images to a uniform size and normalizing the cropped images. Normalization eliminates interference caused by differences in image pixel values, improving the processing efficiency and accuracy of subsequent image segmentation models.

[0027] Furthermore, to enhance the generalization ability of the image segmentation model and suppress the influence of environmental factors such as changes in lighting and shooting angles within the open-pit mine on the segmentation results, data augmentation processing is performed on the preprocessed original slope images. Specific data augmentation operations include at least one of the following: horizontal flipping, vertical flipping, random rotation, brightness adjustment, addition of Gaussian noise, and color dithering. These data augmentation operations effectively expand the diversity of the model's training samples, preventing overfitting. The original slope images after data augmentation are then manually annotated to clearly distinguish between the pumice area and the background area, forming a fully annotated dataset for training, validation, and testing of the image segmentation model, ensuring that the model can accurately identify and segment the pumice area.

[0028] In some embodiments of this application, during the data acquisition stage, data is collected using a multi-source data acquisition method combining UAV aerial photography and ground-based fixed-point acquisition to obtain image data of pumice on open-pit mine slopes under different terrains, lighting conditions, and shooting angles. During preprocessing, the original images are standardized and cropped into 512×512 pixel sub-images. A fixed-step sliding window is used to ensure that the complete shape of the pumice is well preserved during cropping. During data augmentation, four aspects are addressed: spatial morphology, lighting changes, noise impact, and color saturation. Methods include random horizontal and vertical flipping, random rotation at different angles (0°, 90°, 180°, 270°), brightness adjustment, Gaussian noise addition, and color dithering. This expands the original dataset to 6315 images, significantly improving the problems of sample scarcity and scene uniformity.

[0029] This embodiment acquires several images of open-pit mine slopes to form a dataset. After performing the preprocessing steps described above, the dataset also includes a data annotation step. During the dataset annotation stage, the LabelMe annotation tool is used for manual annotation. Polygonal annotations are selected to represent pumice structure features, drawing polygon boundaries for each target and assigning a corresponding category label to each target in the image. The polygon mask information for all targets is generated into an annotation file, saved in JSON format, containing information such as contour location and target category name. After the dataset annotation is completed, experts conduct multiple evaluations and adjustments to ensure the completeness and accuracy of the annotated features. Finally, the constructed pumice dataset is randomly divided into a training set, a validation set, and a test set, with a ratio of 8:1:1.

[0030] S2: Based on the pre-trained image segmentation model, the target slope image is segmented into the pumice region to obtain the pumice segmentation map.

[0031] The PSPNet model is used to perform pixel-level segmentation of the pumice region in the target slope image. To further improve the segmentation accuracy and reduce the computational complexity of the model, the improved PSPNet model is preferred as the image segmentation model in this application.

[0032] Specifically, the ResNet-50 network is used as the backbone network of the PSPNet model. Leveraging the deep feature extraction capabilities of the ResNet-50 network, the pumice feature information in the target slope image is effectively extracted, providing support for accurate segmentation of the pumice region in the subsequent process. After the feature extraction stage of the backbone network, an optimization module is introduced. This optimization module may include at least one of a dynamic convolution module, a depthwise separable convolution module, and an SE channel attention module. This optimization module optimizes the features extracted by the backbone network, improving the effectiveness of the features.

[0033] Among them, such as Figure 2 As shown, the working process of the Dynamic Convolutional Module (DCM) is as follows: global average pooling is performed on the feature map obtained in the feature extraction stage of the backbone network to obtain the channel description vector of the feature map; the weights of the channel description vector are calculated through different attention branches respectively; the original convolution kernels are weighted and aggregated based on the calculated weights to obtain the dynamic convolution kernel weights; the channel description vectors are convolved based on the dynamic convolution kernel weights, which can effectively improve the model's ability to focus on the key features of pumice and improve the accuracy of pumice segmentation in complex backgrounds.

[0034] In some embodiments of this application, the attention branches may include at least one of channel attention, filter attention, spatial attention, and kernel selection attention. Appropriate combinations of attention branches can be selected based on the actual environment of the open slope and the specific characteristics of the pumice, further improving the model's segmentation performance and ensuring that the model can accurately identify pumice of different sizes and degrees of occlusion, outputting a pumice segmentation map with clear boundaries and accurate categories. The expressions for each attention branch are as follows:

[0035]

[0036]

[0037]

[0038] In the formula, and The weights are for two groups of 1×1 convolutions; GroupNorm groups and normalizes the channels; ReLU is the activation function. For temperature parameters; It is the Sigmoid function, used for weight normalization; These are the intermediate features after dimensionality reduction; the Softmax function is used for weight normalization. , , These are the weight parameters for filter weights, spatial weights, and kernel selection weights, respectively. , , , These are the weights for channel attention, filter attention, spatial attention, and kernel selection attention, respectively.

[0039] The weights generated by four attention mechanisms are used to weight and aggregate the original convolutional kernels to obtain the final dynamic convolutional kernel weights:

[0040] In the formula, The weights of the i-th convolutional kernel; It is the number of convolution kernels. This indicates point-by-point multiplication.

[0041] The final output features of the dynamic convolution module are:

[0042] In the formula, Conv2d is a two-dimensional convolution operation; This represents the feature map input to the dynamic convolution module.

[0043] like Figure 3 As shown, the Depthwise Separable Convolutional (DSConv) module replaces some of the standard convolutional layers in the PSPNet model. While maintaining the model's feature extraction capabilities, it effectively reduces the number of parameters and computational cost, improving computational efficiency and enabling it to meet the needs of real-time monitoring in open-pit mines. Specifically, in the latter part of the model, depthwise separable convolutions replace some standard convolutions. This operation first performs independent spatial convolution on each input channel, then uses 1×1 convolutions to combine the outputs of each channel, significantly reducing model parameters and computational overhead. A schematic diagram of the module can be seen in [link to module diagram]. Figure 3 The corresponding parameter calculation formula is:

[0044]

[0045]

[0046]

[0047] In the formula, , , , These represent the number of parameters for standard convolution, channel-wise convolution, point-wise convolution, and depthwise separable convolution, respectively. Indicates the kernel size; Indicates the number of input channels; Indicates the number of output channels.

[0048] The reduction in the number of parameters of depthwise separable convolution compared to standard convolution can be expressed as: .

[0049] like Figure 4 As shown, the SE channel attention module is set after the output channel of the backbone network. Its working process is as follows: the feature map output by the backbone network is processed by global average pooling operation, and the importance weights of the output channels are learned by combining the two fully connected layers. The learned importance weights are multiplied with the original feature map channel by channel to realize the recalibration of features, thereby strengthening the feature information useful for pumice segmentation and suppressing the interference of redundant features.

[0050] The improved PSPNet network structure is as follows: Figure 5 As shown, the PSPNet model consists of a ResNet50 module, an optimization module, and a pyramid pooling module. The PSPNet model takes a pumice image as input, and after feature extraction, optimization processing, and contextual information fusion, it outputs a segmentation result of the same size as the input. To demonstrate the improved PSPNet model's performance, five randomly selected images from the test set were compared to the segmentation results of the original PSPNet and the improved PSPNet. Figure 6 As shown.

[0051] Furthermore, to quantitatively evaluate the segmentation performance of the models, comparative experiments were conducted on the performance of commonly used segmentation models U-Net, Segformer, DeepLabV3, Mask2Former, ConvNeXt, the original PSPNet, and the improved PSPNet on the pumice segmentation task. After training all models for 500 epochs on the pumice dataset, the best-performing model was selected and evaluated on the test set. The segmentation performance of different models was systematically compared using metrics such as mean Intersection over Union (mIoU), mean Dice coefficient (mDice), mean pixel precision (mPA), mean pixel accuracy (mPrecision), and mean recall (mRecall). The results are shown in Table 1 below.

[0052] Table 1. Performance evaluation comparison of different segmentation models on the pumice dataset / %

[0053] As shown in Table 1, the improved PSPNet achieves an mIoU of 81.94%, a 5.49 percentage point improvement over the original PSPNet's 76.45%, indicating enhanced segmentation accuracy. The improved PSPNet's mDice is 90.90%, superior to other models, demonstrating its ability to effectively capture pumice of varying sizes and occlusion levels, reducing false negatives and fully adapting to the complex pumice segmentation requirements of open slope environments. The improved PSPNet's mPA is 89.98%, significantly better than other models, indicating higher overall accuracy and lower false positive rate in distinguishing pumice pixels from background pixels. The improved PSPNet's mPrecision is 91.88%, slightly higher than ConvNeXt, indicating that the improved PSPNet effectively reduces misclassification of slope background as pumice, improving the reliability of segmentation results. The improved PSPNet achieves an mRecall of 89.98%, outperforming all other models. This indicates that the improved PSPNet can capture loose rocks in open slopes to the greatest extent possible, especially small, partially obscured loose rocks. Based on the comparative analysis of the five core indicators, the improved PSPNet model proposed in this application outperforms existing commonly used segmentation models in terms of segmentation accuracy, completeness, precision, false positive resistance, and false negative resistance in the open slope loose rock segmentation task. It effectively solves the technical problems of insufficient segmentation accuracy and high false positive / false negative rates of traditional models in complex slope environments.

[0054] The improved PSPNet, through the introduction of backbone network optimization and feature optimization modules, can accurately extract pumice features, effectively distinguish pumice from slope background texture, adapt to complex environments such as changes in lighting and shooting angles in open-pit mines, and output pumice segmentation maps with clear boundaries and accurate categories. This provides high-precision basic data support for subsequent pumice depth map fusion, 3D point cloud data extraction, and hazard level assessment.

[0055] S3: Obtain the depth map of the target slope, perform pixel-level fusion of the pumice segmentation map and the depth map, and extract the 3D point cloud data of the pumice based on the fused image.

[0056] Multi-view image data of the target slope was collected by drones. Structure of Motion (SfM) and Multi-View Stereo (MVS) techniques were used to match and calculate the multi-view image data, generating a depth map covering the entire target slope. This depth map accurately reflects the true spatial distance information of each point on the slope surface. The pumice segmentation map output by the improved PSPNet model was then fused with the aforementioned depth map through pixel-by-pixel multiplication, as shown below. Figure 7As shown, this fusion method can retain the depth value corresponding to the pumice area, effectively filter out invalid depth information from the background area, achieve accurate purification of the pumice target area, and avoid interference from background depth information in the extraction of pumice 3D data. In some embodiments of this application, COLMAP software is used for structure of motion restoration (SfM) and multi-view stereo matching (MVS).

[0057] Based on a pre-defined camera intrinsic parameter matrix, the pixel coordinates and corresponding depth values ​​of the pumice region are transformed into 3D coordinates in the camera coordinate system, completing the conversion from 2D pixel coordinates to 3D coordinates. Then, using a pre-defined camera extrinsic parameter matrix, the 3D coordinates in the camera coordinate system are transformed into the world coordinate system, obtaining the 3D point cloud data of the pumice. This 3D point cloud data can completely and accurately reflect the spatial location and 3D morphology of the pumice, providing a reliable data source for subsequent 3D mesh reconstruction and geometric feature parameter extraction of the pumice.

[0058] In some embodiments of this application, statistical filtering and voxel downsampling optimization of the 3D point cloud data are also included. Specifically, the basic idea of ​​the above statistical filtering method is to use the neighborhood information of the points to calculate the distance distribution between each point and its neighboring points. First, the average distance of the k nearest neighbors of each point in the point cloud is calculated, and then a distance threshold is set to filter out outliers with large distances. The formula for the statistical filtering method is as follows:

[0059] In the formula, Represented as the point to be measured Its neighboring points The Euclidean distance. By setting a reasonable k value and distance threshold, isolated points and noise points in the point cloud can be effectively removed.

[0060] The above downsampling method mainly discretizes the point cloud into a series of voxel grids, with each grid represented by a point within it. The downsampling formula is as follows:

[0061] In the formula, These are representative points of the voxel grid; The number of points within the grid; The points are represented within the grid. The resolution of the point cloud can be controlled by adjusting the size of the voxel grid. In this process, a voxel grid size of 5 mm was chosen to ensure a balance between point cloud detail and computational efficiency.

[0062] S4: Based on 3D point cloud data, 3D mesh reconstruction is performed using the convex hull algorithm to extract the geometric feature parameters of the pumice.

[0063] An improved convex hull algorithm was used to reconstruct a 3D mesh from the 3D point cloud data of the pumice. This improved algorithm includes two optimization steps: dynamic region optimization and depth weight correction. The resulting 3D mesh reconstruction of the pumice is shown in the image below. Figure 8 As shown. Among them, dynamic region optimization can address the problem of uneven distribution of 3D point cloud data, and depth weight correction can solve the reconstruction error problem caused by local missing points in the point cloud. Through the above two optimization steps, the accuracy of 3D mesh reconstruction can be effectively improved, ensuring that the reconstructed 3D mesh model can truly restore the actual shape of the pumice.

[0064] Based on the reconstructed 3D mesh model of the pumice, the volume of the pumice is calculated using a 3D modeling algorithm. The mass of the pumice is calculated by multiplying its volume by the volume and the pre-set rock density. Simultaneously, based on the 3D mesh model, the shape factor, initial height, average slope of the area where the pumice is located, and the rate of change of average slope are calculated. These geometric parameters together constitute a quantitative index system for pumice hazard assessment, comprehensively characterizing the physical properties, spatial orientation, and slope characteristics of the surrounding environment of the pumice, providing a comprehensive and reliable quantitative basis for the subsequent accurate assessment of the pumice hazard level.

[0065] Specifically,

[0066]

[0067]

[0068] In the formula, For the quality of pumice; The average density of the measurement area; For pumice shape factor; The surface area of ​​the pumice after three-dimensional fitting; To determine the initial height of the pumice.

[0069] Slope is an important indicator describing topographic changes, typically used to characterize the degree of inclination of the earth's surface. The gradient method is used to calculate slope, and the average value is taken as the overall average slope. The specific formula is as follows:

[0070]

[0071] In the formula, Indicates slope; The height is the pixel value in the depth map. and respectively along shaft and The gradient of the axis; Indicates the average slope; These represent the number of rows and columns of the depth map, respectively.

[0072] The slope change rate measures the degree of spatial variation in slope, and its mathematical expression is as follows:

[0073]

[0074] In the formula, This is the slope matrix; and Slope along shaft and The gradient of the axis; This represents the rate of change of the average slope; These represent the number of rows and columns of the depth map, respectively.

[0075] S5: Based on the geometric characteristic parameters and the fuzzy comprehensive evaluation theory, the danger level of the pumice is obtained.

[0076] Using the volume, mass, shape factor, initial height, average slope, and average slope change rate of pumice as evaluation indicators, a complete pumice hazard evaluation index system is established. Based on the actual needs of pumice safety management in open-pit mines, several hazard evaluation levels in this pumice hazard evaluation index system are determined, and the classification standards for each hazard evaluation level are clarified.

[0077] For each evaluation indicator, a membership function corresponding to different evaluation levels is constructed. This membership function quantifies the degree of membership of each evaluation indicator to different hazard evaluation levels, thus forming a fuzzy relation matrix and achieving fuzzy quantification of the evaluation indicators. The Analytic Hierarchy Process (AHP) is used to determine the weight vector of each evaluation indicator. Based on the degree of influence of each evaluation indicator on the pumice hazard level, the weights of each evaluation indicator are reasonably allocated to ensure the scientific validity and rationality of the evaluation results.

[0078] Substituting the aforementioned fuzzy relation matrix and the weight vectors of each evaluation index into the preset fuzzy comprehensive evaluation formula, a fuzzy comprehensive evaluation vector is calculated. Based on this fuzzy comprehensive evaluation vector, the comprehensive hazard index of the floating rock is calculated using the weighted average method. The calculated comprehensive hazard index is compared with the preset hazard level threshold, and the final hazard level of the floating rock is determined based on the comparison results. This achieves an objective, accurate, and quantitative determination of the hazard level of the floating rock, providing a clear decision-making basis for the safety management of floating rocks on open slopes.

[0079] In some embodiments of this application, a pumice hazard assessment index system is established based on the fuzzy comprehensive evaluation method, including six key factors such as pumice volume, mass, shape factor, initial height, average slope change rate, and average slope; the hazard level classification of each evaluation index is shown in Table 2 below.

[0080]

[0081] Furthermore, the comprehensive membership degree is calculated based on the fuzzy comprehensive evaluation method, the comprehensive risk index D is obtained by the weighted average method, and the risk level is divided according to the threshold. The weight of each indicator is determined by expert opinion and the analytic hierarchy process (AHP). Finally, the risk level is determined by constructing a fuzzy relation matrix based on intelligent data extraction, as shown in Table 3.

[0082] Table 3 Hazard Level Classification

[0083] To visually represent the hazard distribution of pumice, different colors were used to mark each pumice area on the original mining area image. The hazard levels were indicated by green for low risk, yellow for medium risk, orange for high risk, and red for extremely high risk. The assessment results were then mapped onto the original image to generate a pumice hazard level distribution map, visually displaying the risk areas. The final pumice hazard level distribution map is shown below. Figure 9 As shown.

[0084] Example 2 This embodiment provides a hazardous intelligent sensing system for loose rocks on open slopes, including: The data acquisition unit is used to acquire images of the target slope within the open-pit mine area.

[0085] The image segmentation unit is used to segment the pumice region of the target slope image based on a pre-trained image segmentation model to obtain a pumice segmentation map; the image segmentation model adopts the PSPNet model.

[0086] The point cloud extraction unit is used to perform pixel-level fusion of the pumice segmentation map and the target slope depth map, and extract the three-dimensional point cloud data of the pumice based on the fused image.

[0087] The feature extraction unit is used to extract the geometric feature parameters of pumice by reconstructing a 3D mesh based on 3D point cloud data using the convex hull algorithm.

[0088] The risk assessment unit is used to determine the hazard level of the pumice based on the geometric feature parameters and fuzzy comprehensive evaluation theory. The geometric feature parameters include at least one of volume, mass, shape factor, initial height, average slope, and average slope change rate.

[0089] Specific limitations regarding the intelligent hazard sensing system for loose rocks on open slopes can be found in the above-mentioned limitations on the intelligent hazard sensing method for loose rocks on open slopes; the corresponding technical effects are equivalent and will not be repeated here. Each module in the aforementioned intelligent hazard sensing system for loose rocks on open slopes can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0090] Figure 10 An internal structural diagram of a computer device is shown in one embodiment. This computer device may specifically be a terminal or a server. Figure 10 As shown, the computer device includes a processor, memory, network interface, display, camera, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for intelligently sensing the danger of loose rocks on open slopes. The display screen can be an LCD screen or an e-ink display screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device casing, or an external keyboard, touchpad, or mouse.

[0091] As will be understood by those skilled in the art, computer equipment Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. Specific computing devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0092] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0093] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0094] In summary, the intelligent perception method, system, computer equipment, and storage medium for the danger of floating rocks on open slopes provided in this application improve the accuracy and efficiency of floating rock region segmentation by improving the PSPNet model, solving the technical problems of missed detection and false detection in complex slope environments by traditional models; through pixel-level fusion of floating rock segmentation map and depth map and extraction of 3D point cloud data, accurate acquisition of 3D spatial information of floating rocks is achieved, making up for the deficiency of traditional 2D detection in obtaining 3D parameters of floating rocks; the 3D mesh reconstruction of floating rocks is achieved by improving the convex hull algorithm, ensuring the accuracy of geometric feature parameter extraction; and the hazard assessment system based on fuzzy comprehensive evaluation theory realizes the quantitative evaluation of the hazard level of floating rocks, avoiding the subjective bias of traditional evaluation methods.

[0095] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on its differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0096] The above-described embodiments are merely preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for intelligently sensing the danger of loose rocks on an open slope, characterized in that, include: Acquire images of the target slope within the open-pit mine area; The target slope image is segmented into a pumice region based on a pre-trained image segmentation model to obtain a pumice segmentation map; the image segmentation model adopts the PSPNet model. Obtain the depth map of the target slope, perform pixel-level fusion of the pumice segmentation map and the depth map, and extract the three-dimensional point cloud data of the pumice based on the fused image; Based on the three-dimensional point cloud data, a three-dimensional mesh is reconstructed using the convex hull algorithm to extract the geometric feature parameters of the pumice. Based on the geometric feature parameters and fuzzy comprehensive evaluation theory, the danger level of the pumice is obtained. The geometric feature parameters include at least one of volume, mass, shape factor, initial height, average slope, and average slope change rate.

2. The intelligent hazard sensing method for loose rocks on open slopes according to claim 1, characterized in that, Acquire images of the target slope within the open-pit mine area, including: The original slope images of the open-pit mine area were obtained by combining drone aerial photography and ground-based fixed-point data acquisition. The original slope image is preprocessed, which includes cropping the original slope image to a uniform size and performing normalization. Data augmentation is performed on the preprocessed original slope image. The data augmentation operations include at least one of horizontal flipping, vertical flipping, random rotation, brightness adjustment, adding Gaussian noise, and color dithering. The original slope after data augmentation was manually annotated to distinguish the pumice area from the background area.

3. The intelligent hazard sensing method for loose rocks on open slopes according to claim 1, characterized in that, The PSPNet model is an improved PSPNet model, and the steps for improving the PSPNet model include: ResNet-50 was used as the backbone network of the PSPNet model; An optimization module is introduced after the feature extraction stage of the backbone network. The optimization module includes at least one of a dynamic convolution module, a depthwise separable convolution module, and an SE channel attention module. The dynamic convolution module is used to perform global average pooling on the feature map obtained in the feature extraction stage to obtain the channel description vector. The weights of the channel description vectors are calculated through different attention branches. The original convolution kernels are weighted and aggregated based on the weights to obtain the dynamic convolution kernel weights. The channel description vectors are then convolved based on the dynamic convolution kernel weights. The depthwise separable convolutional module replaces some of the standard convolutional layers in the PSPNet model; The SE channel attention module is set after the output channel of the backbone network. It is used to learn the importance weights of the output channel through global average pooling and two fully connected layers, and multiply them with the original feature map channel by channel to achieve feature recalibration.

4. The intelligent hazard sensing method for loose rocks on open slopes according to claim 3, characterized in that, The attention branches include at least one of channel attention, filter attention, spatial attention, and kernel selection attention.

5. The intelligent hazard sensing method for loose rocks on open slopes according to claim 1, characterized in that, The pumice segmentation map and the slope depth map are fused pixel-level, and the 3D point cloud data of the pumice is extracted based on the fused image, including: By acquiring multi-view image data of the target slope using drones, and matching it with motion reconstruction structure and multi-view stereo technology, a depth map of the target slope is generated. The pumice segmentation map output by the improved PSPNet model is fused with the depth map by pixel-wise multiplication, retaining the depth value corresponding to the pumice region and filtering out the background depth information. Based on the camera intrinsic parameter matrix, the pixel coordinates and depth values ​​of the pumice area are transformed into three-dimensional coordinates in the camera coordinate system; By using the camera extrinsic parameter matrix, the 3D coordinates in the camera coordinate system are transformed to the world coordinate system to obtain the 3D point cloud data of the pumice.

6. The intelligent hazard sensing method for loose rocks on open slopes according to claim 5, characterized in that, Based on the aforementioned 3D point cloud data, a 3D mesh is reconstructed using the convex hull algorithm to extract the geometric feature parameters of the pumice, including: An improved convex hull algorithm is used to reconstruct 3D meshes from 3D point cloud data; the improved convex hull algorithm includes dynamic region optimization and depth weight correction. Based on the reconstructed three-dimensional mesh model, the volume of the pumice is calculated, and the mass of the pumice is obtained according to the volume and the preset rock density. Based on the three-dimensional grid of the pumice, the shape factor, initial height, average slope and average slope change rate of the area where the pumice is located are calculated.

7. The intelligent hazard sensing method for loose rocks on open slopes according to claim 6, characterized in that, Based on the geometric feature parameters and using fuzzy comprehensive evaluation theory, the hazard level of the pumice is obtained, including: Establish a pumice hazard assessment index system that includes the aforementioned geometric feature parameters, and determine several hazard assessment levels within the pumice hazard assessment index system; For each evaluation indicator, a membership function corresponding to different evaluation levels is constructed to form a fuzzy relation matrix; The weight vector of each evaluation index is determined using the analytic hierarchy process (AHP). Based on the fuzzy relation matrix and the weight vectors of each evaluation index, the fuzzy comprehensive evaluation vector is calculated according to the fuzzy comprehensive evaluation formula. Based on the fuzzy comprehensive evaluation vector, the comprehensive risk index is calculated using the weighted average method, and the final risk level is determined according to the preset risk level threshold.

8. A hazardous intelligent sensing system for loose rocks on an open slope, characterized in that, include: The data acquisition unit is used to acquire images of the target slope within the open-pit mine area; The image segmentation unit is used to segment the pumice region of the target slope image based on a pre-trained image segmentation model to obtain a pumice segmentation map; the image segmentation model adopts the PSPNet model. The point cloud extraction unit is used to perform pixel-level fusion of the pumice segmentation map and the target slope depth map, and extract the three-dimensional point cloud data of the pumice based on the fused image; The feature extraction unit is used to extract the geometric feature parameters of the pumice by performing three-dimensional mesh reconstruction based on the three-dimensional point cloud data using the convex hull algorithm. The risk assessment unit is used to obtain the hazard level of the pumice based on the geometric feature parameters and fuzzy comprehensive evaluation theory; the geometric feature parameters include at least one of volume, mass, shape factor, initial height, average slope and average slope change rate.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1 to 7, the method for intelligently sensing the danger of loose rocks on an open slope.

10. A computer device, characterized in that, include: Processor and computer-readable storage media; A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the intelligent hazard sensing method for pumice on open slopes as described in any one of claims 1 to 7.

Images