Surrounding rock stability prediction method based on mixed space-time self-attention mechanism

Abstract

The invention discloses a surrounding rock stability prediction method based on a mixed space-time self-attention mechanism, and belongs to the technical field of mine safety engineering, and the method comprises the following steps: 1, data collection and screening; 2, constructing a mixed space-time prediction model; the mixed space-time prediction model comprises an embedding layer, a mixed space-time self-attention mechanism layer and a classification layer; 3, performing supervised learning and iterative training on the mixed space-time prediction model, and outputting a trained model; and 4, acquiring data of surrounding rock of the current coal mine tunnel in real time, inputting the trained model, and predicting a stability category. According to the method, on the basis of a single-head self-attention mechanism, a time attenuation factor and a spatial distance weight are set, a mixed space-time self-attention mechanism is constructed, the mixed space-time self-attention mechanism is introduced into surrounding rock stability prediction, and the prediction precision is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of mine safety engineering, and in particular relates to a surrounding rock stability prediction method based on a hybrid spatiotemporal self-attention mechanism. Background Art

[0002] The mining tunnel is an important passage for underground mining operations such as coal mines and metal mines. The stability of its surrounding rock directly affects the safety and production efficiency of miners. If the surrounding rock stability is insufficient, it may cause tunnel deformation, spalling, roof collapse and even large-scale collapse, resulting in serious safety accidents. If the surrounding rock stability is not predicted, mining companies can only rely on experience or simple geophysical exploration and manual detection methods to judge the state of the surrounding rock. Relying on manual judgment may lead to misjudgment due to lack of experience or complex environment. At the same time, manual survey and analysis take time, which may lead to delayed response and miss the best reinforcement opportunity. Therefore, accurate prediction of the category of surrounding rock stability of the mining tunnel will help to take reinforcement measures in advance, optimize the support plan, reduce the risk of accidents, and improve the safety and economic benefits of the mine.

[0003] In the field of surrounding rock stability prediction of mining tunnels, there are mainly three mainstream technologies: Support Vector Machine (SVM), Naïve Bayes and Decision Tree. SVM is a prediction method based on statistical learning theory. The core idea is to make optimal predictions of data by constructing hyperplanes. In the prediction of surrounding rock stability, this method can find the optimal hyperplane in the high-dimensional feature space to achieve the separation of different types of surrounding rock states, and map the data to the high-dimensional feature space through kernel functions (such as radial basis function RBF, polynomial kernel, etc.), so that the originally inseparable data becomes separable, thereby improving the prediction accuracy. Naïve Bayes is a prediction method based on Bayes' theorem, and its core assumption is that all features are independent of each other. This method calculates the posterior probability that the surrounding rock data belongs to each stability category and selects the category with the highest probability as the prediction result. This method has low computational complexity and is suitable for processing long-term monitoring data (massive data collected by mine surrounding rock sensors). Decision tree is a prediction method based on tree structure. Its core idea is to construct a series of "if-then" rules by recursively dividing the data set. In the prediction of surrounding rock stability, the features that best distinguish the surrounding rock stability categories are automatically selected through indicators such as entropy gain, information gain or Gini coefficient. Mining engineers can intuitively understand the classification logic of the decision tree, which is convenient for adjustment based on engineering experience.

[0004] Regarding the above three mainstream technologies, the defects of existing technologies are as follows: (1) SVM has high computational complexity and is difficult to process large-scale data. Specifically, for the prediction of surrounding rock stability, if the data set is large (such as long-term monitoring data), the training and prediction speed of SVM may be slow. Parameter selection relies on experience. In the problem of surrounding rock stability prediction, the surrounding rock characteristics of different mining areas vary greatly, and the selection of parameters is difficult to be universal. (2) In the analysis of surrounding rock stability, various characteristics (such as stress, joints and fissures, surrounding rock strength, etc.) are often interrelated. For example, high stress areas are usually accompanied by higher crack density, and the naive Bayesian assumption that these variables are independent of each other may lead to a decrease in classification accuracy. In other words, the distribution of surrounding rock parameters is often complex and may not conform to the standard distribution assumption, thus affecting the classification effect. (3) When dealing with the prediction of surrounding rock stability, if the amount of training data is small or there is a lot of noise, the decision tree may form an overly complex tree structure, which makes the model's generalization ability on new data worse. If there is a class imbalance in the surrounding rock stability data (e.g., the “stable” class is in the majority and the “unstable” class is less), the decision tree may tend to learn the features of the major classes, resulting in inaccurate predictions for the minority classes (e.g., extremely unstable surrounding rock). Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a surrounding rock stability prediction method based on a hybrid spatiotemporal self-attention mechanism, and designs a hybrid spatiotemporal prediction model. The hybrid spatiotemporal self-attention mechanism layer based on geology is integrated into the model construction, which acts on the time decay and spatial weight calculation path in the hybrid spatiotemporal self-attention. During model training, a hard physical constraint loss mechanism is designed to learn and adjust the extreme sample prediction to achieve surrounding rock stability prediction. The present invention can effectively explore the deep-level correlation between various parameters and improve the prediction accuracy; and adopts an optimized training process to improve the model convergence speed and reduce the consumption of computing resources. It is suitable for the safety evaluation of surrounding rock and support optimization design of mining tunnels.

[0006] The technical solution of the present invention is as follows: A surrounding rock stability prediction method based on a hybrid spatiotemporal self-attention mechanism includes the following steps: Step 1: Data collection and screening; Step 2: Build a hybrid spatiotemporal prediction model; the hybrid spatiotemporal prediction model includes an embedding layer, a hybrid spatiotemporal self-attention mechanism layer, and a classification layer; Step 3: Perform supervised learning and iterative training on the hybrid spatiotemporal prediction model, and output the trained model; Step 4: Obtain the current coal mine tunnel surrounding rock data in real time, input the trained model, and predict the stability category.

[0007] Furthermore, the specific process of step 1 is as follows: Step 1.1: Collect the surrounding rock stability data of the current coal mine's mining roadway; Step 1.2: Select seven parameters, namely the surrounding rock strength of the roadway roof, the surrounding rock strength of the two sides of the roadway, the surrounding rock strength of the roadway floor, the mining depth, the surrounding rock integrity, the dynamic pressure of mining in this section, and the dynamic pressure of mining in adjacent sections, as the screening indicators; Step 1.3: Screen the surrounding rock stability data of the current coal mine's mining roadway according to the seven parameters, and use the obtained surrounding rock stability samples of the mining roadway after screening as input data, and divide them into training samples and test samples according to a certain proportion.

[0008] Furthermore, in the above-mentioned Step 2, the specific working process of the hybrid spatio-temporal prediction model is as follows: Step 2.1: Input the input data into the embedding layer and perform normalization to form a high-dimensional input vector ; Step 2.2: Perform attention calculation on the high-dimensional input vector based on the hybrid spatio-temporal self-attention mechanism to obtain a weighted feature matrix; Step 2.3: Input the weighted feature matrix into the classification layer to obtain the prediction result.

[0009] Furthermore, the specific process of the above-mentioned Step 2.2 is as follows: Step 2.2.1: Define the query weight matrix , the key weight matrix , and the value weight matrix ; Step 2.2.2: Perform a linear transformation on the high-dimensional input vector obtained in Step 2.1 to generate the query matrix , the key matrix , and the value matrix , and the specific calculation formula is: ; ; ; Step 2.2.3: On the basis of the single-head self-attention mechanism, set up a time decay factor and a spatial distance weight to construct a hybrid spatio-temporal self-attention mechanism, and obtain a weighted feature matrix based on the hybrid spatio-temporal self-attention mechanism; the specific calculation formula is: ; In the formula, is the weighted feature matrix; is the hybrid spatio-temporal self-attention mechanism; , , are respectively the query matrix, the key matrix, and the value matrix of time ; is the Softmax function; is the transpose symbol; is the scaling factor; and are two different time decay factors; and are different acquisition timestamps respectively; is the spatial weight coefficient; is the time and the time is the spatial distance between two measurement points corresponding to the time.

[0010] Furthermore, in the step 2.3, the classification layer adopts a fully connected classification head, performs a fully connected calculation, and obtains a classification prediction value. The specific process is as follows: Step 2.3.1: Establish a classification weight matrix Perform a linear transformation on to obtain the class score , and the formula is: ; ; In the formula, is a 7×3 matrix, each row corresponds to an input feature dimension, and each column corresponds to a certain surrounding rock stability class, and the classes are excellent, good, and medium respectively; is the classification weight corresponding to the 7th row and 3rd column; The specific calculation method of is: ; ; ; In the formula, , , are the scores of the three classes of excellent, good, and medium respectively; is the 7th weighted feature in the weighted feature matrix; Step 2.3.2: Normalize the class score into a probability form, and the formula is: ; ; ; ; In the formula, is the probability set; , , respectively represent the probabilities belonging to the categories of excellent, good, and medium; Step 2.3.3. Take the category corresponding to the maximum probability as the final prediction result of the surrounding rock stability. , and the formula is: ; where, is to take the maximum value.

[0011] Furthermore, in the said Step 3, the training objects include 4: query weight matrix , key weight matrix , value weight matrix , and classification weight matrix ; The training process of the classification weight matrix is as follows: Introduce cross-entropy loss , and the formula is: ; where, is the probability belonging to the category ; is the number of categories; is the true label; Introduce gradient descent to adjust , and the formula is: ; In the formula, is the update operation; is the learning rate; The loss function of model training is: ; In the formula, is the total loss function; is the balance weight coefficient; is the norm operation; is the average value of the three surrounding rock strengths of the roadway roof surrounding rock strength, roadway rib surrounding rock strength, and roadway floor surrounding rock strength; is the elastic modulus; is the strain; is the Poisson's ratio; Update the classification weight matrix of the current time step in the direction of decreasing function value of the total loss function; each adjustment will generate a new cross-entropy loss, and the appearance of a new cross-entropy loss is regarded as a training cycle, and a complete training process requires multiple cycles; the discrimination condition for the end of training is: the decrease amplitude of the dataset loss is less than 0.001 within several consecutive rounds, or the dataset accuracy is continuously higher than 95%, and the fluctuation does not exceed ±0.5%; Query weight matrix , key weight matrix , value weight matrix has the same training process as .

[0012] The beneficial technical effects brought by the present invention: Considering that it is difficult for traditional methods to capture the interaction of parameters, and the present invention first introduces a hybrid spatio-temporal self-attention mechanism into the prediction of surrounding rock stability, breaking through the interaction modeling bottleneck of traditional algorithms; the present invention uses a hybrid spatio-temporal self-attention mechanism to replace the traditional kernel function, and automatically quantifies the interaction effects among 4 parameters through the weights of the hybrid spatio-temporal self-attention mechanism, solving the defect that traditional methods rely on artificial feature engineering. This method does not require artificial design of the kernel function, and the model has stronger adaptability; on the basis of the single-head self-attention mechanism, a time decay factor and a spatial distance weight are set up to construct a hybrid spatio-temporal self-attention mechanism, which combines time decay and spatial correlation and is more in line with the dynamic change characteristics of surrounding rock stability. A "single-layer multi-head attention + fully connected classification head" is designed for small sample data, which can reduce the risk of overfitting. Finally, the interpretability is enhanced because the contribution degree of key parameters (such as mining depth H, dynamic pressure X) to the classification result is clarified through the visualization of attention weights. The hybrid spatio-temporal self-attention mechanism structure designed by the present invention has a small computational amount, can be deployed on a common industrial control computer, is adapted to the existing mine monitoring system (such as Intel i5 processor), and does not require additional hardware upgrade. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of the surrounding rock stability prediction method based on the hybrid spatio-temporal self-attention mechanism of the present invention.

[0014] Figure 2 is a line graph comparing the accuracies of different models in the experiment of the present invention.

[0015] Figure 3 is a comparison graph of the loss decline curves of different models in the experiment of the present invention.

[0016] Figure 4 is a classification error distribution graph of different models at different time steps in the experiment of the present invention.

[0017] Figure 5 is an ROC curve graph of the model of the present invention in the experiment of the present invention.

[0018] Figure 6 is an ROC curve graph of the support vector machine model in the experiment of the present invention.

[0019] Figure 7 is an ROC curve graph of the logistic regression model in the experiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments: During the coal mining process, the mechanical equilibrium state of the surrounding rock mass around the mining roadway under the combined action of natural stress and mining disturbance. Its stability directly determines whether the roadway will undergo failure phenomena such as collapse, floor heave, or lateral deformation. As Figure 1 shown, the present invention proposes a surrounding rock stability prediction method based on a hybrid spatio-temporal self-attention mechanism, which mainly includes the following steps: Step 1: Data collection and screening; The specific process is as follows: Step 1.1: Collect the surrounding rock stability data of the mining roadway in the current coal mine; Step 1.2: Select seven parameters, namely the surrounding rock strength of the roadway roof, the surrounding rock strength of the two sides of the roadway, the surrounding rock strength of the roadway floor, the mining depth, the surrounding rock integrity, the mining dynamic pressure in the current section, and the mining dynamic pressure in the adjacent section, as the screening indicators; The surrounding rock strength of the roadway roof, two sides, and floor only refers to the uniaxial compressive strength of the rock, unit: MPa; The mining dynamic pressure in the current section refers to the ratio of the thickness of the immediate roof to the coal seam mining height, reflecting the roof support capacity; The mining depth refers to the vertical depth of the roadway from the ground surface, which directly affects the ground pressure, unit: meter; The mining dynamic pressure in the adjacent section refers to the width of the coal pillar protecting the roadway (X = 0 when driving along the goaf, X = 100 when driving in solid coal); The surrounding rock integrity is a quantitative index of the structural defects such as rock layer fissures and faults, with a value range of 0 to 1 (1 indicates complete).

[0021] Step 1.3: Screen the surrounding rock stability data of the mining roadway in the current coal mine according to the seven parameters, obtain 100 groups of surrounding rock stability samples of the mining roadway as input data, and divide them into training samples and test samples according to a ratio of 8:1. The first 80 groups are used as the training samples of the model, and the last 20 groups are used as the test samples; Table 1 shows an example of the first 30 groups of samples; Table 1 Example of the first 30 groups of samples .

[0022] Step 2: Construct a hybrid spatio-temporal prediction model; The hybrid spatio-temporal prediction model includes an embedding layer, a hybrid spatio-temporal self-attention mechanism layer, and a classification layer; The specific working process of the hybrid spatio-temporal prediction model is as follows: Step 2.1: In order to capture potential features, input the input data into the embedding layer to perform normalization to form a high-dimensional input vector; The specific process is as follows: The initial input of the data is responsible for the embedding layer. The embedding layer normalizes the input data to the interval [0, 1], and after normalization, a high-dimensional input vector is formed , and this vector is used as the subsequent input vector, It is the 7th parameter after normalization. The "Min - Max Normalization" method is specifically used for normalization, and the formula is as follows: ; In the formula, is the data value after normalization, with a range of [0, 1]; is the original data value; is the minimum value of the current parameter; is the maximum value of the current parameter; Step 2.2: Perform attention calculation on the high - dimensional input vector based on the hybrid spatio - temporal self - attention mechanism to obtain a weighted feature matrix; the specific process is as follows: Step 2.2.1: Define the query weight matrix , the key weight matrix , and the value weight matrix . The query weight matrix is responsible for mapping the input features to the query vector (Query) for calculating the attention score; the key weight matrix is responsible for mapping the input features to the key vector (Key) for measuring the correlation between the query and the key; the value weight matrix is responsible for mapping the input features to the value vector (Value) for weighted summation to form the final feature representation.

[0023] An example of the query weight matrix is: ; In the formula, is the weight corresponding to the 7th row and 7th column in the query weight matrix; The structures of the key weight matrix and the value weight matrix are the same. In actual generation, the "small - value random initialization" uniform distribution method is adopted, and the initial numerical range of all positions of the three weight matrices is set to [-0.01, 0.01]. For example, the query weight matrix can be generated by calling numpy.random.uniform(-0.01, 0.01, size=(7,7)) or a similar function.

[0024] Step 2.2.2: Perform a linear transformation on the high - dimensional input vector obtained in Step 2.1 to generate the query matrix , the key matrix , and the value matrix , and the specific calculation formulas are as follows: ; ; ; The calculated query matrix, key matrix, and value matrix are still three 1×7 row vectors; for example, ; where is the 7th element in the query matrix; in the th query is defined as: ; where is the 7th normalized data value; is the weight corresponding to the th column in the 7th row of the query weight matrix; the query sequence number corresponds to the column sequence number; Step 2.2.3, the stability of surrounding rock is affected by dynamic spatio-temporal factors. For example, in the time dimension: the increase in mining depth leads to the accumulation of ground pressure over time; in the space dimension: the stress transfer effect between adjacent measurement points. Existing technologies (such as SVM, Naive Bayes) only focus on static parameter interactions and cannot capture spatio-temporal evolution laws. The present invention additionally proposes a "Hybrid Spatio-Temporal Attention" specifically designed for geological characteristics. Its technical solution is based on a single-layer self-attention mechanism, and a time decay factor and a spatial distance weight are respectively set up to model the spatio-temporal evolution characteristics of surrounding rock parameters; According to the single-head self-attention mechanism theory, calculate the attention score , and then use the Softmax function to normalize the attention score to obtain the interaction weight between parameters. The formula is: ; ; ; In the formula, is the single-head attention mechanism; is the transpose symbol; is the scaling factor, is 's dimension, which is 7 in this invention; is the Softmax function, and the calculation formula is: ; In the formula, is the th attention score; is the total number of attention scores; is the exponential operation to ensure the result is non-negative; is the sum of all exponential values to ensure normalization (the sum is 1); Softmax is a mathematical function mainly used to convert a set of numerical values (Logits) into a probability distribution; the Softmax function is usually used to process a set of numerical values and convert them into a probability distribution. However, in the case of only a single input value, the normalization denominator of Softmax only contains the exponential operation result of this input value, resulting in the final output always being 1. Therefore, when Softmax acts on a single numerical value, its result is equivalent to the effect of the identity function.

[0025] Given and are both 1×7 matrices, then for transpose it to a 7×1 matrix, that is the result of is a scalar. Divide by the scaling factor to prevent gradient explosion or gradient vanishing. Finally, the Softmax function obviously acts on a single numerical value.

[0026] For the single-head self-attention mechanism, the Softmax function is multiplied by (the final output Attention is the weighted sum of the attention weights and ), where the weight is a scalar, is a 1×7 row matrix, so the final output weighted feature is still a 1×7 row matrix, and the weighted feature matrix is specifically ; is the 7th weighted feature in the weighted feature matrix.

[0027] Based on the single-head self-attention mechanism, a time decay factor and a spatial distance weight are set up to construct a hybrid spatio-temporal self-attention mechanism, and a weighted feature matrix is obtained based on the hybrid spatio-temporal self-attention mechanism; the specific calculation formula is: ; In the formula, is the hybrid spatio-temporal self-attention mechanism; , , are the query matrix, key matrix, and value matrix of time respectively; , are two different time decay factors used to control the time decay intensity; , are different acquisition timestamps respectively; is the spatial weight coefficient; is the time and time the spatial distance between two measurement points corresponding at; The significance of the hybrid spatio-temporal self-attention mechanism lies in that, specifically from the perspective of the present invention, the time decay factor automatically reduces the weight of historical data (e.g., the influence of the H value 10 days ago weakens), and the spatial distance weight enhances the parameter interaction between adjacent measurement points (roof and two sides of the roadway), suppressing the interference of remote measurement points. In summary, it integrates time decay and spatial correlation, better fitting the dynamic change characteristics of surrounding rock stability, thereby solving the problem that traditional self-attention only models parameter interaction and ignores spatio-temporal correlation. That is, it is considered that existing mainstream solutions (such as SVM, Naive Bayes, etc.) cannot handle time series and spatial data, and this design realizes spatio-temporal joint modeling in surrounding rock prediction for the first time.

[0028] Step 2.3: Input the weighted feature matrix into the classification layer to obtain the prediction result; the classification layer adopts a fully connected classification head, performs a fully connected calculation, and obtains the classification prediction value. The specific process is as follows: Step 2.3.1: Establish a classification weight matrix Perform a linear transformation on to obtain the class scores , and the formula is: ; ; In the formula, is a 7×3 matrix, where each row corresponds to an input feature dimension (a total of seven dimensions, respectively referring to the seven major indicators of the surrounding rock strength of the roadway roof, the surrounding rock strength of the two sides of the roadway, the surrounding rock strength of the roadway floor, the mining depth, the integrity of the surrounding rock, the dynamic pressure of mining in this section, and the dynamic pressure of mining in adjacent sections), and each column corresponds to a certain surrounding rock stability category, which are excellent, good, and medium respectively; is the classification weight corresponding to the 7th row and 3rd column; The specific calculation method of is: ; ; ; In the formula, , , are the scores of the three categories of excellent, good, and medium respectively; Step 2.3.2: The class scores themselves are not comparable and need to be normalized into probability form. The formula is: ; ; ; ; In the formula, is a probability set; , , respectively represent the probabilities belonging to the categories of excellent, good, and medium.

[0029] Step 2.3.3: Take the category corresponding to the maximum probability as the final prediction result of the surrounding rock stability. , and the formula is: ; where, is to take the maximum value; For example, if the calculated probabilities of the three situations of excellent, good, and medium categories are 50%, 30%, and 20% respectively, then the final prediction result is "excellent".

[0030] Step 3: Conduct supervised learning and iterative training on the hybrid spatio-temporal prediction model, and output the trained model; The above situation assumes that the output of the model is determined to be "excellent", but when it conflicts with the actual situation, such as the on-site judgment is "good" or "medium", at this time, the model of the present invention needs to be trained to maximize approximation and adaptation to the real result, so as to adjust the parameters in the model to make it tend to predict the correct category in the future.

[0031] There are 4 main training objects: the query weight matrix , the key weight matrix , the value weight matrix , and the classification weight matrix ; The following only shows the training process of the classification weight matrix , and the training processes of the query weight matrix , the key weight matrix , and the value weight matrix are the same as . The training principle is to calculate the error using the true label and the current model prediction result , and the training is carried out using cross-entropy loss and gradient descent, continuously adjusting WFC to make the model more accurate in future predictions.

[0032] Following the previous assumption (not real data values, only for demonstration to facilitate expression), the reason for the current model to predict "excellent" is that the probability at the corresponding position ( ) is the largest, for example ; Let the true (on-site actual) category "good" be represented as , meaning "the second position is true"; Introduce the cross-entropy loss , and the formula is: ; wherein, is the probability belonging to the category ; is the number of categories. Substituting example values: ; the larger the value of the cross-entropy loss, the greater the prediction deviation, and vice versa, indicating that the model prediction is more accurate.

[0033] Gradient descent is introduced to adjust , and the formula is: ; In the formula, is the update operation; is the learning rate, which controls the adjustment step size. As a hyperparameter, the learning rate determines how far to step towards the target direction each time the weights are updated. If this value is too large, the optimal solution will be missed, and if it is too small, the training speed will be too slow. For a medium-scale dataset such as the stability data of the return airway, the initial setting is 0.01; is the derivative of the cross-entropy loss with respect to , and its function is to adjust the direction to make the predicted value more biased towards the true value .

[0034] It should be noted that when designing the loss function of the present invention, referring to the geomechanics theory, a balance equation is customized as a constraint condition to ensure that the prediction results conform to physical laws: ; In the formula, is the total loss function; is the balance weight coefficient; is the norm operation; is the mean value of the surrounding rock strengths of the roadway roof, the two sides of the roadway, and the roadway floor; is the elastic modulus; is the strain; is the Poisson's ratio; In order to avoid physically unreasonable predictions caused by pure data-driven methods, it is decided to hard-embed domain knowledge into the model to improve the reliability of the model under extreme working conditions (such as high stress and strong disturbance), that is, to achieve data-physics dual-driven modeling for the first time. Specifically, data-driven means learning the mapping relationship between input and output based on historical surrounding rock stability data using a deep neural network to achieve classification or prediction tasks. In the present invention, this is reflected in introducing the cross-entropy loss in the formula. Physical-driven is to introduce the balance relationship in geomechanics (strength-stress balance, mining influence) to construct physical constraint terms, forcing the model output to obey the known laws under the engineering background. In the present invention, this is reflected in constructing the geological-physical constraint loss term, corresponding to in the formulaportion.

[0035] In this way, according to the decreasing direction of the function value of the total loss function, update the classification weight matrix at the current time step . Obviously, each adjustment will generate a new cross-entropy loss. The appearance of a new cross-entropy loss is regarded as a training cycle (Iteration), and the complete training process requires multiple cycles. The discrimination condition for the end of training is: the decrease in the dataset loss within several consecutive Epochs (such as 10 rounds) is less than 0.001, that is, the model can no longer learn more; or the dataset accuracy is continuously higher than 95%, and the fluctuation does not exceed ±0.5%, that is, the model performance tends to be stable.

[0036] To ensure that the tunnel surrounding rock stability prediction model involved in the present invention can work correctly, it needs to be trained on a suitable platform to optimize the model parameters. It is recommended to use PyTorch because it supports dynamic graph calculation, is easy to debug, supports GPU parallel computing, and can train more efficiently. For GPU, use NVIDIA Tesla A100, V100 or RTX 3090 for accelerated computing, and use CUDA for efficient matrix operations; for CPU, at least reach Intel i7 / i9 or AMD Ryzen9 processors.

[0037] Step 4: Real-time obtain the data of the current coal mine tunnel surrounding rock, input it into the trained model, and predict the stability category.

[0038] To prove the superiority of the present invention, the following comparative experiments were carried out.

[0039] Experiment 1: Accuracy comparison; During the experiment, the accuracy comparison line chart, as an important visualization tool, shows the accuracy change trend of different models in each training round (Epoch). The purpose of the accuracy comparison line chart is to intuitively compare the improvement of the accuracy of the hybrid spatio-temporal prediction model, support vector machine (SVM) and logistic regression model of the present invention during the training process. Figure 2 As the comparison results of the three models, the horizontal axis represents the number of Epochs of training, starting from 1 and gradually increasing to the maximum number of training rounds. In this experiment, the total number of simulation training rounds is 100, and the horizontal axis shows the training process from the 1st round to the 100th round. Each training round represents the accuracy level reached by the model after training in that round. The scale of the horizontal axis presents the time dimension of model training, which is convenient for observing the training progress and accuracy change of each model. The vertical axis represents the accuracy of the model, usually as a percentage value (0% to 100%). The vertical axis shows the accuracy score of each model after each training round. The accuracy value reflects the prediction accuracy of the model after that training round. A high accuracy means that the model can better predict the stability category of the surrounding rock in the classification task. Such asFigure 2 As shown, the data points on each broken line correspond to the accuracy of the model after a certain number of Epochs. As the number of training rounds increases, the broken line fluctuates upward or downward, indicating an increase or decrease in accuracy. From Figure 2 it can be seen that the model of the present invention has the highest accuracy.

[0040] Experiment 2: Comparison of loss degrees; The comparison chart of the loss decline curve is a key visualization tool in the present invention for evaluating the loss change trend of different models during training. By showing the training losses of different models (the model of the present invention, Support Vector Machine SVM, Naive Bayes) changing over time (i.e., the number of training rounds Epoch), this chart helps to intuitively compare the optimization efficiency of each model during training, and further analyze its performance in the classification task of the surrounding rock stability of the extraction roadway. Figure 3 In it, the vertical axis represents the loss value, a floating value between 0 and a relatively large value. The loss value measures the error size of the model prediction, and the smaller the value, the better the prediction effect of the model. As the number of training rounds of the model of the present invention increases, this broken line shows an obvious downward trend, indicating that the loss of the model of the present invention gradually decreases, indicating that it has strong learning ability during the optimization process. Since the training process of SVM needs to continuously find the optimal hyperplane through support vectors and boundary conditions, the process of loss decline may be relatively gentle, especially in the initial stage of training. SVM may show a slow optimization process in the early stage, with a small decline in loss. The loss decline curve of Naive Bayes is usually relatively gentle, especially when facing complex data, and the fitting ability of Logistic Regression is relatively limited, so its loss decline speed is relatively slow, and there may be large fluctuations especially in the initial stage of training.

[0041] Experiment 3: Confusion matrix; In the model evaluation of machine learning and deep learning, the confusion matrix is a commonly used tool for intuitively showing the relationship between the predicted results and the actual results of a classification model for each category. In order to make the comparison between different categories more intuitive among different models, using a "percentage stacked bar chart" to display the confusion matrix data can effectively provide a profound understanding of the model performance.

[0042] Figure 4 The classification error distributions of three different models (the model of the present invention, Support Vector Machine model SVM, Logistic Regression model) at different time steps are shown through a stacked bar chart, specifically including: True Positive (TP): The model predicts the positive class, and it is actually the positive class.

[0043] False Positive (FP): The model predicts the positive class, and it is actually the negative class.

[0044] True Negative (TN): The model predicts that the class is negative and it is actually negative.

[0045] False Negative (FN): The model predicts a negative class, but it is actually a positive class.

[0046] Each time step corresponds to a stacked column chart, and each part in the column represents the classification error ratio under the time step. The height of each part is determined by the proportion of the number of each category to the total samples, and the sum of these proportions is 100%. This method eliminates the influence of the number of samples at different time steps by percentage, making the classification error of each model more comparable. If the stacked column chart of the model of the present invention shows a larger proportion of TP and a smaller proportion of FP and FN, it means that the model has a stronger classification accuracy. Support vector machine SVM and logistic regression models may show a higher FP or FN ratio in some Epochs, which means that the model may have more classification errors, resulting in inaccurate prediction of positive or negative classes.

[0047] Experiment 4: ROC curve comparison; The ROC (Receiver Operating Characteristic) curve is a tool used to evaluate the performance of a binary classification model. It helps analyze the performance of a model under different conditions by showing the trade-off between the false positive rate (FPR) and the true positive rate (TPR) at different thresholds. The true positive rate (TPR), also known as sensitivity or recall, indicates the proportion of samples correctly predicted by the model as positive to all samples that are actually positive. The false positive rate (FPR), also known as 1-specificity, indicates the proportion of samples incorrectly predicted by the model as positive to all samples that are actually negative.

[0048] The meaning of the curve: The upper left corner indicates that the model can maintain a very low false positive rate while having a high true positive rate, indicating that the model's performance is very good. The lower right corner indicates that the model's performance is poor, because the false positive rate is high and the true positive rate is low, and the performance is similar to random guessing. The diagonal line represents the performance of a random model. Any point on the diagonal line means that the model's prediction is no different from a random prediction.

[0049] like Figure 5 As shown in the figure, the ROC curve of the model of the present invention is usually located near the upper left corner, indicating that the model can obtain a higher true positive rate while maintaining a lower false positive rate. The superiority of the model is reflected in its ability to accurately classify the stability of the surrounding rock, and it can better cope with different input characteristics. Figure 6As shown, the ROC curve of SVM may be slightly closer to the diagonal than that of the model of the present invention, indicating that the prediction performance of this model is slightly inferior in some cases, especially when the classification threshold is relatively extreme. As Figure 7 shown, the curve of the logistic regression model may be between the model of the present invention and the SVM model, indicating that its performance is medium. Although logistic regression is a classic linear model, its performance in complex non-linear data is often inferior to that of deep learning models.

[0050] To prove the feasibility of the present invention, a set of real data cases are given as follows. The data of a certain mine are collected, and the numerical values are processed for convenient demonstration to obtain the results in Table 2; Table 2 Data Processing Results .

[0051] The "min-max normalization" method is used for normalization to obtain a high-dimensional input vector , which is a 1×7 matrix.

[0052] The initially defined query weight matrix , key weight matrix , and value weight matrix are: ; In this embodiment, is set to 1; the attention score is calculated: ; The softmax function is calculated for each row, and it is assumed that: ; Here is a matrix, the number of rows of the matrix is the same as the number of , and the number of columns is equal to numerical values. The calculation is too large, and only the specific numerical values of one row are shown here.

[0053] ; A classification weight matrix is established; The classification score is calculated: ; The classification score is normalized to obtain ; The maximum value 0.55 corresponds to the category "excellent", so the model predicts that the surrounding rock stability is "excellent". If the true label is "good" (the second category) or "medium", the cross-entropy loss is calculated, gradient descent is performed, is adjusted, and then the next round of training is carried out.

[0054] Certainly, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples either. Changes, modifications, additions or substitutions made by those skilled in the art within the scope of the essence of the present invention shall also fall within the protection scope of the present invention.

Claims

1. A surrounding rock stability prediction method based on a hybrid spatio-temporal self-attention mechanism, characterized in that It includes the following steps: Step 1, data collection and screening; Step 2, constructing a hybrid spatio-temporal prediction model; the hybrid spatio-temporal prediction model includes an embedding layer, a hybrid spatio-temporal self-attention mechanism layer, and a classification layer; Step 3, performing supervised learning and iterative training on the hybrid spatio-temporal prediction model, and outputting the trained model; Step 4, obtaining the data of the surrounding rock of the current coal mine tunnel in real time, inputting it into the trained model, and predicting the stability category.

2. The surrounding rock stability prediction method based on the hybrid spatio-temporal self-attention mechanism according to claim 1, wherein, The specific process of the said Step 1 is: Step 1.1, collecting the surrounding rock stability data of the extraction roadway of the current coal mine; Step 1.2, selecting seven parameters including the surrounding rock strength of the roadway roof, the surrounding rock strength of the two sides of the roadway, the surrounding rock strength of the roadway floor, the mining depth, the surrounding rock integrity, the mining dynamic pressure in this section, and the mining dynamic pressure in the adjacent section as the screening indicators; Step 1.3, screening the surrounding rock stability data of the extraction roadway of the current coal mine according to the seven parameters, taking the obtained surrounding rock stability samples of the extraction roadway after screening as the input data, and dividing them into training samples and test samples according to a ratio.

3. The surrounding rock stability prediction method based on the hybrid spatio-temporal self-attention mechanism according to claim 2, wherein, In the said Step 2, the specific working process of the hybrid spatio-temporal prediction model is: Step 2.1: Input the input data into the embedding layer and perform normalization to form a high-dimensional input vector ; Step 2.2, performing attention calculation on the high-dimensional input vector based on the hybrid spatio-temporal self-attention mechanism to obtain a weighted feature matrix; Step 2.3, inputting the weighted feature matrix into the classification layer to obtain a prediction result.

4. The surrounding rock stability prediction method based on the hybrid spatio-temporal self-attention mechanism according to claim 3, characterized in that The specific process of the said Step 2.2 is: Step 2.2.1, define the query weight matrix , key weight matrix , value weight matrix ; Step 2.2.2: Perform a linear transformation on the high-dimensional input vector obtained in Step 2.1 to generate a query matrix , a key matrix , and a value matrix . The specific calculation formula is as follows: ； ； ； Step 2.2.3, based on the single-head self-attention mechanism, setting up a time decay factor and a spatial distance weight to construct a hybrid spatio-temporal self-attention mechanism, and obtaining a weighted feature matrix based on the hybrid spatio-temporal self-attention mechanism; the specific calculation formula is: ； In the formula, is the weighted feature matrix; is the hybrid spatio-temporal self-attention mechanism; , , are the query matrix, key matrix, and value matrix of time respectively; is the Softmax function; is the transpose symbol; is the scaling factor; , are two different time decay factors; , are different acquisition timestamps respectively; is the spatial weight coefficient; is the time and time is the spatial distance between two measurement points corresponding to them.

5. The surrounding rock stability prediction method based on the hybrid spatio-temporal self-attention mechanism according to claim 4, wherein, In the said Step 2.3, the classification layer adopts a fully connected classification head to perform fully connected calculation to obtain a classification prediction value. The specific process is: Step 2.3.1, establish a classification weight matrix For perform a linear transformation to obtain a category score , and the formula is: ； ； In the formula, is a 7×3 matrix, where each row corresponds to an input feature dimension, and each column corresponds to a certain surrounding rock stability category, and the categories are excellent, good, and medium respectively; is the classification weight corresponding to the 7th row and 3rd column; The specific calculation method is as follows: ； ； ； ； In the formula, , , are the scores for the three categories of excellent, good, and medium respectively; is the 7th weighted feature in the weighted feature matrix; Step 2.3.2, normalizing the class scores into a probability form, and the formula is: ； ； ； ； In the formula, is a probability set; , , respectively represent the probabilities belonging to the categories of excellent, good, and medium; Step 2.3.3: Take the category corresponding to the maximum probability as the final prediction result of surrounding rock stability , and the formula is: ； Among them, is to take the maximum value.

6. The surrounding rock stability prediction method based on the hybrid spatio-temporal self-attention mechanism according to claim 5, characterized in that In step 3, there are 4 training objects: query weight matrix , key weight matrix , value weight matrix , classification weight matrix ; Classification weight matrix The training process is as follows: Introduce cross-entropy loss , and the formula is: ； wherein, is the probability belonging to class ; is the number of classes; is the true label; Gradient descent is introduced for adjustment , and the formula is as follows: ； In the formula, is the update operation; is the learning rate; The loss function for model training is: ； Wherein, is the total loss function; is the balance weight coefficient; is the norm operation; is the average value of the surrounding rock strengths of the roadway roof, the two sides of the roadway, and the roadway floor; is the elastic modulus; is the strain; is the Poisson's ratio; According to the decreasing direction of the function value of the total loss function, update the classification weight matrix of the current time step; each adjustment will generate a new cross-entropy loss, and the appearance of a new cross-entropy loss is regarded as a training cycle, and a complete training process requires multiple cycles; the discrimination condition for the end of training is: the decrease amplitude of the dataset loss is less than 0.001 within several consecutive rounds, or the dataset accuracy is continuously higher than 95% and the fluctuation does not exceed ±0.5%; Query weight matrix , key weight matrix , value weight matrix has the same training process as .

Images