A mine slope risk prediction method based on multimodal deep learning

Abstract

The invention discloses a mine slope risk prediction method based on multimodal deep learning, and relates to the technical field of mine slope risk prediction. The invention comprises Step 1: inputting multimodal data, Step 2: data preprocessing, Step 3: sample set construction and Step 4: realizing mine slope stability prediction by using a gated recurrent unit (GRU) based on a mutual information mechanism. The invention adopts a multimodal data integrated GRU network to perform sequence modeling on the fused data, extract more complex sequence features, complete slope safety event prediction analysis, realize high-precision monitoring of mine slopes, and improve the accuracy and comprehensiveness of prediction results. The invention applies the mutual information mechanism to the GRU network to maximize the mutual information between samples and features, improve the dependency between features and labels, and ensure not only the prediction ability but also the generalization ability on new data.

Description

Technical Field

[0001] The present invention relates to the technical field of mine slope risk prediction, and in particular to a mine slope risk prediction method based on multimodal deep learning. Background Art

[0002] As an important pillar industry supporting economic construction, mining occupies a pivotal position in resource development and utilization. However, due to the particularity and complexity of mining activities, potential safety risks cannot be ignored. Mine safety accidents will not only cause casualties and property losses, but may also cause serious damage to the ecological environment. Therefore, mine safety monitoring is of great practical significance.

[0003] At present, mine monitoring mainly focuses on single data monitoring such as ground sensors, radar monitoring, geological monitoring, and excavation volume. There are problems such as low monitoring accuracy and difficulty in comprehensively evaluating the stability of the slope. Using single data to predict the stability of mine slopes can only obtain information on one aspect of the slope, such as displacement, inclination or stress, and cannot fully reflect the overall state and changes of the slope. This limitation makes the prediction results may not be accurate and comprehensive, and it is difficult to comprehensively evaluate the stability of the slope. At the same time, in single data monitoring, certain errors may occur due to reasons such as monitoring equipment, data processing methods or human factors. If monitoring relies on single data for a long time, these errors may gradually accumulate, causing the monitoring results to deviate from the actual situation.

[0004] Currently, most of the monitoring is focused on the stability of mine slopes, which cannot prevent accidents before they occur. In addition, the existing technology has not applied the mutual information mechanism to the GRU network to maximize the mutual information between samples and features. The neglect of this level in the existing technology has led to a small mutual information between samples and features, which means that the dependence between features and labels is weak, which not only affects the prediction ability, but also the generalization ability on new data. Summary of the invention

[0005] The purpose of the present invention is to provide a mine slope risk prediction method based on multimodal deep learning, which solves the problems existing in the background technology.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solution: The present invention provides a mine slope risk prediction method based on multimodal deep learning.

[0007] StepS1: Input multimodal data, including elevation data, slope data, aspect data, GNSS displacement data, radar data, rainfall data, historical landslide data, geological rock formation data, and geological disaster risk point data of the study area.

[0008] StepS2: Data preprocessing, preprocessing the input data, including data cleaning, coordinate conversion, format conversion, text data extraction, and normalization.

[0009] StepS3: Sample set construction, combining historical landslide data and expert knowledge visual interpretation, and drawing landslide and hidden danger polygons for the preprocessed radar images.

[0010] StepS4: The gated recurrent unit GRU based on the mutual information mechanism realizes the prediction of mine slope stability. The gated recurrent unit GRU based on the mutual information mechanism maximizes the mutual information between the input sample and its intermediate low-dimensional features, improves the sample learning quality of the model, and provides reliable support for the subsequent use of the model for slope stability prediction.

[0011] The beneficial effects of the present invention are as follows: the present invention, through the core idea of ​​joint characterization of multimodal data, maps the information of multiple modes into a unified multimodal vector space, obtains the feature representation of each mode, and fuses them, uses the multimodal data integrated GRU network to perform sequence modeling on the fused data, extracts more complex sequence features, completes slope safety event prediction and analysis, realizes high-precision monitoring of mine slopes, uses multimodal data to perform mine slope safety prediction, fuses various monitoring data, such as displacement, inclination and stress, to form a comprehensive slope prediction system, and improves the accuracy and comprehensiveness of the prediction results.

[0012] While monitoring landslide disasters, the present invention applies the mutual information mechanism to the GRU network to maximize the mutual information between samples and features, improve the dependency between features and labels, and ensure not only the prediction ability but also the generalization ability on new data. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0014] Figure 1 The present invention is a schematic flow chart of the steps for implementing the method.

[0015] Figure 2 This is a schematic diagram of image slicing.

[0016] Figure 3 This is a schematic diagram of label slicing.

[0017] Figure 4 This is an example map of landslides and potential hazard areas.

[0018] Figure 5 This is a diagram of the network structure of the gated recurrent unit GRU based on the mutual information mechanism.

[0019] Figure 6 This is the GRU hidden layer structure diagram.

[0020] Figure 7 This is an example of the extraction effect of the deformation cluster area identification model. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] Reference Figure 1 As shown, the present invention provides a mine slope risk prediction method based on multimodal deep learning, including: Step S1: input multimodal data, including study area elevation data, slope data, slope aspect data, GNSS displacement data, radar data, rainfall data, historical landslide data, geological rock formation data, and geological disaster risk point data.

[0023] StepS2: Data preprocessing, preprocessing the input data, including data cleaning, coordinate conversion, format conversion, text data extraction, and normalization.

[0024] In a specific embodiment of the present invention, the input data is preprocessed, and the specific process is as follows: data cleaning: the same type of sensor data from different suppliers are unified in units, and abnormal values ​​in the input data are eliminated, such as null values, empty values, 0 values ​​and out-of-limit values.

[0025] Format conversion: The radar is a binary file or json text, which is converted into a csv file for data cleaning, threshold coloring, and then converted into a tiff file. Convert GNSS displacement data and rainfall data from csv or xls files to tiff files. Convert geological disaster risk point data and geological rock formation data from shp files to tiff files.

[0026] Text data extraction: perform text extraction on historical landslide data, extract keywords such as time, longitude and latitude, and size, form a csv file, and then convert the csv file into a tiff file.

[0027] Coordinate transformation: unify the input data into the same spatial reference, such as CGCS2000.

[0028] Normalization: The input data is normalized using the minimum-maximum method to normalize all data to the same dimension, which facilitates subsequent feature extraction and model training.

[0029] StepS3: Sample set construction, combining historical landslide data and expert knowledge visual interpretation, and drawing landslide and hidden danger polygons for the preprocessed radar images.

[0030] In a specific embodiment of the present invention, the landslide and its hidden danger polygons are drawn on the preprocessed radar image, and the specific method is: sample cutting is performed according to the size of 512*512 pixels, and a total of 1830 landslide and hidden danger area samples are obtained. The sample set contains image information, labels and image slices.

[0031] It should be noted that the sample cutting effect is as follows Figure 2 , Figure 3 shown.

[0032] The sample set is divided into training set, validation set and test set at the ratio of 8:1:1, with 1464, 183 and 183 samples of each type respectively, and the landslides and potential danger areas on the radar images are obtained.

[0033] It should be noted that examples of landslides and potential danger areas on radar images are as follows: Figure 4 As shown in the figure, the area where the deformation rate value changes greatly is the landslide hazard area.

[0034] StepS4: The gated recurrent unit GRU based on the mutual information mechanism realizes the prediction of mine slope stability. The gated recurrent unit GRU based on the mutual information mechanism maximizes the mutual information between the input sample and its intermediate low-dimensional features, improves the sample learning quality of the model, and provides reliable support for the subsequent use of the model for slope stability prediction.

[0035] In a specific embodiment of the present invention, the mutual information is specifically: information originates from information entropy, which is a measure of the degree of linear and nonlinear mutual dependence between two random variables. The essence of mutual information is the reduction in the uncertainty of X caused by the determination of Y, that is, the amount of information about X contained in Y.

[0036] Let A be a set of n variables, A i and A j (i≠j) are any two variables in A, and the correlation of set A is:

[0037]

[0038] in:

[0039]

[0040] Where: p(A i ), p(A j ) represents the variable A i and A j The marginal probability of p(A i ,A j ) means A i and A j The probability of the joint distribution.

[0041] In order to reduce data redundancy, different modal data are required to have a smaller correlation, and the variables with smaller mutual information values ​​are retained. Therefore, the optimization formula is as follows, and the mutual information value is normalized to between 0 and 1 to facilitate the comparison of mutual information values ​​between different variables.

[0042]

[0043] In a specific embodiment of the present invention, the variables are retained, and the specific principles are as follows: A1. Traverse all variables and put two variables with mutual information values ​​less than 0.6 into the retained set.

[0044] A2. Traverse the variables with mutual information values ​​greater than or equal to 0.6. If one of the variables has been retained, put the other variable into the elimination set. If one of the variables has been eliminated, put the current variable into the retention set. If both variables have been retained, eliminate one variable according to priority.

[0045] A3. Based on empirical knowledge, the variable priority is: radar data > GNSS displacement data > geological disaster risk point data > rainfall data > historical landslide data > geological rock layer data > slope data > elevation data > slope aspect data.

[0046] Reference Figure 5 As shown, in a specific embodiment of the present invention, the network structure of the gated recurrent unit GRU of the mutual information mechanism includes five layers: input data layer, mutual information estimation layer, GRU hidden layer, mutual information estimation layer, fully connected layer, and output data layer.

[0047] The GRU hidden layer controls the speed of information accumulation by introducing a gating mechanism, selectively retaining and forgetting certain information. Its specific structure is as follows: Figure 6 shown.

[0048] The GRU neuron is reset by the gate R t and update gate Z t The σ function in controls the degree to which the state information of the previous moment is introduced into the current moment. The larger its value is, the more important the state information of the previous moment is. Its internal relationship is:

[0049] R t =σ(W r ·[Ht-1 ,X t ])

[0050] Z t =σ(W z ·[H t-1 ,X t ])

[0051]

[0052] In the formula, R t To reset the gate, Z t To update the gate, is the candidate hidden state, H t is the hidden state passed to the next moment, X t Enter information for the current moment, H t-1 is the hidden state at the previous moment, W r , W z , W h is the weight matrix parameter that needs to be learned.

[0053] However, the GRU neural network uses two gate structures to select the memory and forgetting of historical data, which is suitable for processing problems with time series characteristics. The model can theoretically approximate any nonlinear function, but as the number of layers increases, more and more memory units will be occupied, resulting in training failure.

[0054] Therefore, a mutual information estimation layer is added before the hidden layer to calculate the mutual information values ​​of different influencing factors, extract the influencing factors with smaller mutual information values, reduce data redundancy, and add a mutual information estimation layer after the hidden layer to calculate the mutual information values ​​of the extracted features to extract key features, that is, select features with mutual information values ​​greater than or equal to 0.6, thereby reducing the number of neural network layers.

[0055] The retained influencing factor sequence data and historical mine landslide data are input, high-level feature learning is achieved through the GRU hidden layer, redundant variables are eliminated through the mutual information estimation layer, meaningful features are highlighted, local feature integration is achieved in the fully connected layer, the final mine slope stability prediction is achieved, and the prediction results are output.

[0056] While monitoring landslide disasters, the present invention applies the mutual information mechanism to the GRU network to maximize the mutual information between samples and features, improve the dependency between features and labels, and ensure not only the prediction ability but also the generalization ability on new data.

[0057] In a specific embodiment of the present invention, cross entropy is used as the target loss function, and the SGD (stochastic gradient descent) optimizer is used for iterative optimization. Each training sample is subjected to data enhancement to obtain a sample of 512*512 pixels.

[0058] The batch size is set to 4. The number of iterations is set to 40000. The learning rate change strategy adopts the "poly" method, the power is set to 0.9, and the minimum learning rate (min_lr) is set to 0.0001. The regularization coefficient of the SGD optimizer is set to 0.01, the momentum is set to 0.9, and the weight decay is set to 0.0005.

[0059] During the model training process, whether the loss gradually decreases and stabilizes, and whether the MIOU and F1-Score indicators are improved is used as a reference for whether the model is converging during the training process.

[0060] In the training program, every 5000 iterations, the evaluation index MIOU of the current model is calculated on the validation set and the current model file is saved. The optimal model file is overwritten based on whether the current MIOU index is the highest among the historical indicators.

[0061] The maximum number of iterations for the deformation cluster recognition model training is 40,000. At the 37,000th iteration, the accuracy index is optimal. The extraction effect is as follows: Figure 7 shown.

[0062] The present invention adopts the core idea of ​​joint characterization of multimodal data, maps the information of multiple modes into a unified multimodal vector space, obtains the feature representation of each mode, and fuses them. The multimodal data integration GRU network is used to perform sequence modeling on the fused data, extracts more complex sequence features, completes slope safety event prediction and analysis, and realizes high-precision monitoring of mine slopes. Multimodal data is used to predict the safety of mine slopes, and various monitoring data such as displacement, inclination and stress are integrated to form a comprehensive slope prediction system, thereby improving the accuracy and comprehensiveness of the prediction results.

[0063] The above contents are merely examples and explanations of the concept of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, they shall all fall within the protection scope of the present invention.

Claims

1. A mine slope risk prediction method based on multimodal deep learning, characterized in that: include: StepS1: Input multimodal data, including elevation data, slope data, aspect data, GNSS displacement data, radar data, rainfall data, historical landslide data, geological rock formation data, and geological disaster risk point data of the study area; Step S2: Data preprocessing, preprocessing the input data, including data cleaning, coordinate conversion, format conversion, text data extraction, and normalization; StepS3: Sample set construction, combining historical landslide data and expert knowledge visual interpretation, drawing landslide and hidden danger polygons for preprocessed radar images; StepS4: Gated recurrent unit GRU based on mutual information mechanism realizes the prediction of mine slope stability; The network structure of the gated recurrent unit GRU of the mutual information mechanism includes five layers: input data layer, mutual information estimation layer, GRU hidden layer, mutual information estimation layer, fully connected layer, and output data layer; The GRU hidden layer controls the speed of information accumulation by introducing a gating mechanism, selectively retaining or forgetting certain information; The GRU neuron is gated by resetting and update gate In The function controls the degree to which the state information of the previous moment is introduced into the current moment. The larger the value, the more important the state information of the previous moment is. Its internal relationship is: ； In the formula, To reset the gate, To update the gate, is a candidate hidden state, is the hidden state passed to the next moment, Enter information for the current moment, is the hidden state at the previous moment, , , is the weight matrix parameter that needs to be learned; The GRU neural network selects the memory and forgetting of historical data through two gate structures; A mutual information estimation layer is added before the hidden layer to calculate the mutual information values ​​of different influencing factors and extract the influencing factors with smaller mutual information values. A mutual information estimation layer is added after the hidden layer to calculate the mutual information values ​​of the extracted features and extract the key features. Input the retained influencing factor sequence data and historical mine landslide data, realize high-level feature learning through the GRU hidden layer, eliminate redundant variables through the mutual information estimation layer, highlight meaningful features, and realize local feature integration in the fully connected layer; The mutual information is specifically: Information comes from information entropy, which is a measure of the degree of linear and nonlinear interdependence between two random variables. The essence of mutual information is the reduction in the uncertainty of X caused by the determination of Y, that is, the amount of information about X contained in Y. Let A be a set of n variables, and (i≠j) are any two variables in A, and the correlation of set A is: ； in: ； in: , Representation variables and The marginal probability of express and The probability of a joint distribution; The variables with smaller mutual information values ​​are retained. The optimization formula is as follows: the mutual information value is normalized to between 0 and 1, and the mutual information values ​​between different variables are compared; 。 2. The method for predicting mine slope risk based on multimodal deep learning according to claim 1, characterized in that: The specific process of preprocessing the input data is as follows: Data cleaning: Unify the units of the same type of sensor data from different suppliers and remove outliers in the input data; Format conversion: The radar is a binary file or json text, which is converted into a csv file for data cleaning, threshold coloring, and then converted into a tiff file; GNSS displacement data and rainfall data are converted from csv or xls files to tiff files; geological disaster risk point data and geological rock formation data are converted from shp files to tiff files; Text data extraction: perform text extraction on historical landslide data, extract keywords such as time, longitude and latitude, and size, form a csv file, and then convert the csv file into a tiff file; Coordinate transformation: unify the input data into the same spatial reference; Normalization: The input data is normalized using the minimum-maximum method to normalize all data to the same dimension, which facilitates subsequent feature extraction and model training.

3. The method for predicting mine slope risk based on multimodal deep learning according to claim 1, characterized in that: The specific method of drawing the landslide and its hidden danger polygon on the pre-processed radar image is as follows: The samples were cut into 512*512 pixel size, and a total of 1830 samples of landslides and potential danger areas were obtained. The sample set includes image information, labels and image slices. The sample set is divided into training set, validation set and test set at the ratio of 8:1:1, with 1464, 183 and 183 samples of each type respectively, and the landslides and potential danger areas on the radar images are obtained.

4. The method for predicting mine slope risk based on multimodal deep learning according to claim 1, characterized in that: The variables are retained, and the specific principles are as follows: A1. Traverse all variables and put the two variables with mutual information values ​​less than 0.6 into the reserved set; A2. Traverse the variables whose mutual information value is greater than or equal to 0.

6. If one of the variables has been retained, the other variable is placed in the elimination set; if one of the variables has been eliminated, the current variable is placed in the retention set; if both variables have been retained, one variable is eliminated according to the priority; A3. Based on empirical knowledge, the variable priority is: radar data > GNSS displacement data > geological disaster risk point data > rainfall data > historical landslide data > geological rock layer data > slope data > elevation data > slope aspect data.

5. The method for predicting mine slope risk based on multimodal deep learning according to claim 1, characterized in that: The cross entropy is used as the target loss function, and the SGD optimizer is used for iterative optimization. Each training sample is enhanced to obtain a sample of 512*512 pixels. The batch size is set to 4; the number of iterations is set to 40000; the learning rate change strategy adopts the "poly" method, the power is set to 0.9, and the minimum learning rate is set to 0.0001; the regularization coefficient of the SGD optimizer is set to 0.01, the momentum is set to 0.9, and the weight decay is set to 0.0005; During the model training process, whether the loss gradually decreases and tends to be stable, and whether the MIOU and F1-Score indicators are improved are used as references to whether the model is converging during the training process; In the training program, every 5000 iterations, the evaluation index MIOU of the current model is calculated on the validation set and the current model file is saved. The optimal model file is overwritten based on whether the current MIOU index is the highest among the historical indicators. The maximum number of iterations for deformation cluster recognition model training is 40,000, and the accuracy index is optimal at the 37,000th iteration.

Images