A multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method

Abstract

The present invention relates to a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method, which belongs to the field of gas concentration prediction in coal tunnel engineering. The present invention includes the following steps: obtaining a gas concentration data set; filling missing values ​​in the obtained gas concentration data set; performing data set division and normalization processing on the data; slicing the data set according to the selected time window length; introducing a convolutional neural network to process historical gas concentration sequence data with time series, and setting parameters; introducing a gated recurrent unit neural network and a long short-term memory neural network to process historical gas concentration sequence data with time series; and optimizing and solving the model more effectively in a multi-objective manner. The present invention establishes a prediction model for gas concentration sequence data through statistical learning and machine learning methods, so that the prediction accuracy of gas concentration is improved.

Description

Technical Field

[0001] The present invention relates to a gas concentration prediction method based on time series, and in particular to a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method, belonging to the field of gas concentration prediction in coal tunnel engineering. Background Art

[0002] Abnormal gas concentration will endanger the normalized safety production of coal mines, and early prediction is of great significance. At present, there are still problems in the prediction of gas concentration, such as low prediction accuracy. The internal mechanism of gas outburst is extremely complex, and there is a certain inaccuracy and ambiguity in the correlation between the influencing factors and the outburst events. The application of traditional prediction technology based on experience and statistical prediction methods based on mathematical modeling has been greatly limited. At present, some advanced theoretical methods such as computer simulation, fuzzy mathematics theory, gray system theory, neural network, expert system, fractal theory and nonlinear theory, rheology and mutation theory have begun to be applied to the quantitative evaluation and analysis of coal and gas outburst, and have achieved certain research results. Early scholars studied the relationship between gas prediction and mining depth, but for coal seams with complex geological environment, this view was quickly replaced by new methods. Subsequently, many scholars have focused on artificial neural networks and other methods to achieve gas concentration prediction. With the continuous increase of data in the coal mine field, the gas concentration prediction based on statistical learning methods and machine learning cannot artificially analyze its nonlinear relationship, and the shortcomings of being unable to predict gas concentration in real time and continuously are gradually revealed, which greatly promotes the use of methods based on deep neural networks in the prediction of mine gas concentration. Previous studies have improved the prediction accuracy of gas concentration, but have not yet achieved satisfactory results. The reason lies in the selection of optimization algorithms. Some optimization algorithms have the disadvantage of being easy to fall into local optimality, making it difficult to ensure the effectiveness of optimization. Therefore, it is of great significance to explore an effective method to improve the accuracy of gas concentration prediction. In response to this problem, the present invention proposes a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method. This method first divides and normalizes the data set, then introduces one-dimensional CNN (convolutional neural network) cells, GRU (gated recurrent unit neural network) cells and LSTM (long short-term memory neural network) cells to process the historical gas concentration series data with time series, designs the network structure to learn the internal dynamic change law of the gas concentration series, and optimizes and solves the model more effectively through a multi-objective approach to jump out of local extreme points. The obtained prediction method is used to complete the gas concentration prediction. Summary of the invention

[0003] The present invention provides a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method, which is used to solve the problem of low accuracy of traditional coal mine gas concentration prediction.

[0004] The technical solution of the present invention is: a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method, the specific steps of the method are as follows:

[0005] Step 1. Obtain gas concentration data set;

[0006] Step 2: Fill missing values ​​in the acquired gas concentration data set;

[0007] Step 3, divide the data set and normalize the data;

[0008] Step 4: Select a time window length that meets the predetermined requirements based on the gas concentration data set, and slice the data set according to the selected time window length;

[0009] Step 5: Introduce the 1DCNN-GRU-LSTM neural network with multi-objective optimization to process the historical gas concentration sequence data with time series and set parameters to predict the gas concentration at the face of the coal-penetrating gas tunnel;

[0010] Step 6: Fixed parameters and loop through Step 1 to Step 5 to make the next prediction.

[0011] As a further solution of the present invention, Step 2 includes:

[0012] Multiple filling method is used to fill missing values. The filling value is calculated according to the distribution of data tuple values. The specific steps are as follows:

[0013] Generate a set of possible imputation values ​​for each missing value, which reflect the uncertainty of the non-response model; each value is used to fill the missing value in the data set, generating several complete data sets;

[0014] Each imputed data set is statistically analyzed using the same statistical methods used for the full data set; the results from each imputed data set are combined to produce a final statistical inference that takes into account the uncertainty caused by data imputation.

[0015] As a further solution of the present invention, in Step 4, the time window length is selected as 20, and the sequence is divided into multiple input patterns, called samples, where twenty time steps are used as input and one time step is used as output, and the output value y of one time step is predicted based on this, and the resulting data is converted into a time series data matrix to obtain the input data of the 1DCNN-GRU-LSTM model.

[0016] As a further solution of the present invention, the specific steps of Step 5 include:

[0017] Step 5.1. Build a parallel model of 1DCNN-GRU-LSTM neural network and set parameters;

[0018] Step 5.2, use the sliced ​​data set to input into the prediction model to train the model;

[0019] Step 5.3, perform multi-objective optimization on the constructed 1DCNN-GRU-LSTM neural network prediction model;

[0020] Step 5.4: Use the optimized prediction model to predict the gas concentration at the face of the coal-penetrating gas tunnel.

[0021] As a further solution of the present invention, Step 5.1 includes:

[0022] Step5.1.1. First, input the data set sliced ​​in Step4 into a one-dimensional convolution, and extract the features of the input data by establishing multiple filters. The feature extraction formula is as follows:

[0023]

[0024] Among them, M j is the input feature vector; l is the lth layer network; k is the convolution kernel; b is the network bias; is the output of layer l, is the input of the lth layer;

[0025] Step 5.1.2, the output data set of one-dimensional CNN is CNN out Input the GRU neural network. The GRU update gate controls the extent to which the information of the previous moment is saved to the next moment. The reset gate controls whether the current state information is combined with the information of the previous moment. The feature extraction formula is as follows:

[0026] z t =σ(W zx x t +U zh H t-1 +b z ) (2)

[0027] r t =σ(W rx x t +U rh H t-1 +b r ) (3)

[0028]

[0029]

[0030] Among them, x t is the input, H t is the output of the hidden layer, z t and r t are the update gate and the reset gate, is the input x t And the output H at the previous moment t-1 is a summary of, σ is the sigmoid function, W hx , W zx , W rx , U hh , U zh , U rh is the weight coefficient matrix, b h , b z , b r is the bias vector, Yes t and H t-1 The conformity relationship;

[0031] Step5.1.3, GRU output data set GRU out Input LSTM neural network, LSTM input gate controls the information of input unit to be transmitted to state unit, output gate controls part of the information of state unit as output of LSTM, forget gate controls the preservation or deletion of state unit information, and its internal structure is as follows:

[0032] i t =σ(W ix x t +W ih h t-1 +b i ) (6)

[0033] f t =σ(W fx x t +W fh h t-1 +b f ) (7)

[0034]

[0035]

[0036] o t =σ(W ox x t +W oh h t-1 +b o ) (10)

[0037] h t =o t ·tanh(c t ) (11)

[0038] Among them, σ is the activation function sigmoid, x t is the input vector at time t; W fx ,W ix ,W cx ,W ox ,W fh ,W ih ,W ch ,W oh is the weight coefficient matrix, i t ,f t ,o t They are the activation functions of the input gate, forget gate and output gate, c t , is a vector of state units and instant units, h t is the current output of LSTM;

[0039] The results of processing the one-dimensional CNN, GRU and LSTM neural networks in sequence are concatenated using a fully connected layer to obtain a 1DCNN-GRU-LSTM prediction model.

[0040] As a further solution of the present invention, in Step 5.3, a hybrid optimization method based on multi-objective optimization and gradient descent is used to perform multi-objective optimization on the prediction model;

[0041] In this way, multi-objective optimization focuses on global search, and the gradient descent algorithm is used for local search; multi-objective optimization can help the gradient descent algorithm jump out of local extreme points, and the gradient descent algorithm can also speed up the convergence of multi-objective optimization. The combination of the two can more effectively optimize and solve the prediction model; in this way, the optimization problem is first decomposed into multiple single-objective optimization problems, and then the gradient descent algorithm is used to optimize and solve them.

[0042] As a further solution of the present invention, the feature extraction formula when the multi-objective problem is converted into n single-objective optimization problems is as follows:

[0043]

[0044] Where w={w1,w2,...,w C} is the weight of the misclassification rate of each category, reflecting the influence of each category sample on the final classification, fcr is the false negative example, C represents the number of categories, and θ is the parameter of the given classifier; i∈(1, C), j∈(1, n);

[0045] For a given set of w, the loss function J of the 1DCNN-GRU-LSTM model is M (θ) is optimized using the gradient descent algorithm under the back-propagation framework, and the loss function J M The partial derivative of (θ) with respect to θ is:

[0046]

[0047] The characteristic formula for weight w optimization solution is as follows:

[0048] min1DCNN-GRU-LSTM(w)=(fcr1(θ),fcr2(θ),...,fcr C (θ))(w) (14)

[0049]

[0050] For a given w, the parameter w of the 1DCNN-GRU-LSTM model is optimized using the gradient descent algorithm. The gradient descent algorithm can effectively find the local optimal solution, and the evolutionary algorithm can find a better initial solution for the gradient descent algorithm. The global optimal solution is found by combining the multi-objective optimization algorithm and the gradient descent algorithm.

[0051] The beneficial effects of the present invention are:

[0052] The present invention proposes a method of introducing one-dimensional convolutional neural network cells, gated recurrent unit neural network cells and long short-term memory neural network cells and processing the historical gas concentration sequence data of the coal-penetrating gas tunnel face with time series through a multi-objective optimization method, designs a network structure to learn the internal dynamic change law of the gas concentration sequence, takes minimizing error loss as the goal, and can complete the gas concentration prediction with a high-precision prediction method. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention shows, by way of example, a flow chart of a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method. DETAILED DESCRIPTION

[0054] Example 1, a multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method, the specific steps of the method are as follows:

[0055] Step 1. Obtain gas concentration data set;

[0056] Step 2: Fill missing values ​​in the acquired gas concentration data set; Step 2 includes:

[0057] Multiple filling method is used to fill missing values. The filling value is calculated according to the distribution of data tuple values. The specific steps are as follows:

[0058] Generate a set of possible imputation values ​​for each missing value, which reflect the uncertainty of the non-response model; each value is used to fill the missing value in the data set, generating several complete data sets;

[0059] Each imputed data set is statistically analyzed using the same statistical methods used for the full data set; the results from each imputed data set are combined to produce a final statistical inference that takes into account the uncertainty caused by data imputation.

[0060] Step 3, divide the data set and normalize the data;

[0061] Step 4: Select a time window length that meets the predetermined requirements based on the gas concentration data set, and slice the data set according to the selected time window length;

[0062] As a further solution of the present invention, the specific steps of Step 4 include:

[0063] In the Step 4, the time window length is selected as 20, and the sequence is divided into multiple input patterns, called samples, where twenty time steps are used as input and one time step is used as output. The output value y of one time step is predicted based on this, and the result data is converted into a time series data matrix to obtain the input data of the 1DCNN-GRU-LSTM model.

[0064] Step 5: Introduce the 1DCNN-GRU-LSTM neural network with multi-objective optimization to process the historical gas concentration sequence data with time series and set parameters to predict the gas concentration at the face of the coal-penetrating gas tunnel;

[0065] The specific steps of Step 5 include:

[0066] Step 5.1. Build a parallel model of 1DCNN-GRU-LSTM neural network and set parameters;

[0067] Step5.1.1. First, input the data set sliced ​​in Step4 into a one-dimensional convolution, and extract the features of the input data by establishing multiple filters. The feature extraction formula is as follows:

[0068]

[0069] Among them, M j is the input feature vector; l is the lth layer network; k is the convolution kernel; b is the network bias; is the output of layer l, is the input of the lth layer;

[0070] Step 5.1.2, the output data set of one-dimensional CNN is CNN out Input the GRU neural network. The GRU update gate controls the extent to which the information of the previous moment is saved to the next moment. The reset gate controls whether the current state information is combined with the information of the previous moment. The feature extraction formula is as follows:

[0071] z t =σ(W zx x t +U zh H t-1 +b z ) (2)

[0072] r t =σ(W rx x t +U rh H t-1 +b r ) (3)

[0073]

[0074]

[0075] Among them, x t is the input, H t is the output of the hidden layer, z t and r t are the update gate and the reset gate, is the input x t And the output H at the previous moment t-1 is a summary of, σ is the sigmoid function, W hx , W zx , W rx , U hh , U zh , U rh is the weight coefficient matrix, b h , b z , b r is the bias vector, Yest and H t-1 The conformity relationship;

[0076] Step5.1.3, GRU output data set GRU out Input LSTM neural network, LSTM input gate controls the information of input unit to be transmitted to state unit, output gate controls part of the information of state unit as output of LSTM, forget gate controls the preservation or deletion of state unit information, and its internal structure is as follows:

[0077] i t =σ(W ix x t +W ih h t-1 +b i ) (6)

[0078] f t =σ(W fx x t +W fh h t-1 +b f ) (7)

[0079]

[0080]

[0081] o t =σ(W ox x t +W oh h t-1 +b o ) (10)

[0082] h t =o t ·tanh(c t ) (11)

[0083] Among them, σ is the activation function sigmoid, x t is the input vector at time t;

[0084] W fx ,W ix ,W cx ,W ox ,W fh ,W ih ,W ch ,W oh is the weight coefficient matrix, i t ,f t ,o t They are the activation functions of the input gate, forget gate and output gate, c t , is a vector of state units and instant units, h t is the current output of LSTM;

[0085] The results of processing the one-dimensional CNN, GRU and LSTM neural networks in sequence are concatenated using a fully connected layer to obtain a 1DCNN-GRU-LSTM prediction model.

[0086] The multi-objective optimization algorithm is a heuristic search algorithm based on swarm intelligence. However, the heuristic search algorithm based on swarm intelligence will bring a great computational burden. The computational complexity of the deep learning network itself is high. This method keeps the complexity of the neural network prediction algorithm based on Pareto dominance within a reasonable range, and effectively uses gradient information to assist the evolutionary algorithm in finding the optimal solution. Finally, a 1DCNN-GRU-LSTM neural network prediction framework based on multi-objective optimization is proposed.

[0087] Step 5.2, use the sliced ​​data set to input into the prediction model to train the model;

[0088] Step 5.3, perform multi-objective optimization on the constructed 1DCNN-GRU-LSTM neural network prediction model;

[0089] In the Step 5.3, a hybrid optimization method based on multi-objective optimization and gradient descent is used to perform multi-objective optimization on the prediction model;

[0090] In this way, multi-objective optimization focuses on global search, and the gradient descent algorithm is used for local search; multi-objective optimization can help the gradient descent algorithm jump out of local extreme points, and the gradient descent algorithm can also speed up the convergence of multi-objective optimization. The combination of the two can more effectively optimize and solve the prediction model; in this way, the optimization problem is first decomposed into multiple single-objective optimization problems, and then the gradient descent algorithm is used to optimize and solve them. When the multi-objective problem is converted into n single-objective optimization problems, the feature extraction formula is as follows:

[0091]

[0092] Where w={w1,w2,...,w C} is the weight of the misclassification rate of each category, reflecting the influence of each category sample on the final classification, fcr is the false negative example, C represents the number of categories, and θ is the parameter of the given classifier; i∈(1, C), j∈(1, n);

[0093] For a given set of w, the loss function J of the 1DCNN-GRU-LSTM model is M (θ) is optimized using the gradient descent algorithm under the back-propagation framework, and the loss function JM The partial derivative of (θ) with respect to θ is:

[0094]

[0095] The characteristic formula for weight w optimization solution is as follows:

[0096] min1DCNN-GRU-LSTM(w)=(fcr1(θ),fcr2(θ),...,fcr C (θ))(w) (14)

[0097]

[0098] For a given w, the parameter w of the 1DCNN-GRU-LSTM model is optimized using the gradient descent algorithm. The gradient descent algorithm can effectively find the local optimal solution, and the evolutionary algorithm can find a better initial solution for the gradient descent algorithm. The global optimal solution is found by combining the multi-objective optimization algorithm and the gradient descent algorithm.

[0099] Step 5.4: Use the optimized prediction model to predict the gas concentration at the face of the coal-penetrating gas tunnel.

[0100] Step 6: Fixed parameters and loop through Step 1 to Step 5 to make the next prediction.

[0101] The specific embodiments of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above embodiments. Various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method, characterized by: The specific steps of the method are as follows: Step 1. Obtain gas concentration data set; Step 2: Fill missing values ​​in the acquired gas concentration data set; Step 3, divide the data set and normalize the data; Step 4: Select a time window length that meets the predetermined requirements based on the gas concentration data set, and slice the data set according to the selected time window length; Step 5: Introduce the 1DCNN-GRU-LSTM neural network with multi-objective optimization to process the historical gas concentration sequence data with time series and set parameters to predict the gas concentration at the face of the coal-penetrating gas tunnel; Step 6, fix the parameters and loop through Step 1 to Step 5 to make the next prediction; The specific steps of Step 5 include: Step 5.

1. Build a parallel model of 1DCNN-GRU-LSTM neural network and set parameters; Step 5.2, use the sliced ​​data set to input into the prediction model to train the model; Step 5.3, perform multi-objective optimization on the constructed 1DCNN-GRU-LSTM neural network prediction model; Step 5.4, use the optimized prediction model to predict the gas concentration at the face of the coal-penetrating gas tunnel; The Step 5.1 includes: Step5.1.

1. First, input the data set sliced ​​in Step4 into a one-dimensional convolution, and extract the features of the input data by establishing multiple filters. The feature extraction formula is as follows: Among them, M j is the input feature vector; l is the lth layer network; k is the convolution kernel; b is the network bias; is the output of layer l, is the input of the lth layer; Step 5.1.2, the output data set of one-dimensional CNN is CNN out Input the GRU neural network. The GRU update gate controls the extent to which the information of the previous moment is saved to the next moment. The reset gate controls whether the current state information is combined with the information of the previous moment. The feature extraction formula is as follows: z t =σ(W zx x t +U zh H t-1 +b z ) (2) r t =σ(W rx x t +U rh H t-1 +b r ) (3) Among them, x t is the input, H t is the output of the hidden layer, z t and r t are the update gate and the reset gate, is the input x t And the output H at the previous moment t-1 is a summary of, σ is the sigmoid function, W hx , W zx , W rx , U hh , U zh , U rh is the weight coefficient matrix, b h , b z , b r is the bias vector, Yes t and H t-1 The conformity relationship; Step5.1.3, GRU output data set GRU out Input LSTM neural network, LSTM input gate controls the information of input unit to be transmitted to state unit, output gate controls part of the information of state unit as output of LSTM, forget gate controls the preservation or deletion of state unit information, and its internal structure is as follows: i t =σ(W ix x t +W ih h t-1 +b i ) (6) f t =σ(W fx x t +W fh h t-1 +b f ) (7) o t =σ(W ox x t +W oh h t-1 +b o ) (10) h t =o t ·tanh(c t ) (11) where σ is the activation function sigmoid, x t is the input vector at time t; W fx ,W ix ,W cx ,W ox ,W fh ,W ih ,W ch ,W oh is the weight coefficient matrix, i t ,f t ,o t They are the activation functions of the input gate, forget gate and output gate, c t , is a vector of state units and instant units, h t is the current output of LSTM; The results of processing the one-dimensional CNN, GRU and LSTM neural networks in sequence are concatenated using a fully connected layer to obtain a 1DCNN-GRU-LSTM prediction model.

2. The multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method according to claim 1 is characterized in that: The Step 2 includes: Multiple filling method is used to fill missing values. The filling value is calculated according to the distribution of data tuple values. The specific steps are as follows: Generate a set of possible imputation values ​​for each missing value, which reflect the uncertainty of the non-response model; each value is used to fill the missing value in the data set, generating several complete data sets; Each imputed data set is statistically analyzed using the same statistical methods used for the full data set; the results from each imputed data set are combined to produce a final statistical inference that takes into account the uncertainty caused by data imputation.

3. The multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method according to claim 1 is characterized by: In the Step 4, the time window length is selected as 20, and the sequence is divided into multiple input patterns, called samples, where twenty time steps are used as input and one time step is used as output. The output value y of one time step is predicted based on this, and the result data is converted into a time series data matrix to obtain the input data of the 1DCNN-GRU-LSTM model.

4. The multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method according to claim 1 is characterized by: In the Step 5.3, a hybrid optimization method based on multi-objective optimization and gradient descent is used to perform multi-objective optimization on the prediction model; In this way, multi-objective optimization focuses on global search, and the gradient descent algorithm is used for local search; multi-objective optimization can help the gradient descent algorithm jump out of local extreme points, and the gradient descent algorithm can also speed up the convergence of multi-objective optimization. The combination of the two can more effectively optimize and solve the prediction model; in this way, the optimization problem is first decomposed into multiple single-objective optimization problems, and then the gradient descent algorithm is used to optimize and solve them.

5. The multi-objective optimized 1DCNN-GRU-LSTM neural network tunnel face gas concentration prediction method according to claim 1 is characterized by: When a multi-objective problem is transformed into n single-objective optimization problems, the feature extraction formula is as follows: Where w={w1,w2,...,w C } is the weight of the misclassification rate of each category, reflecting the influence of each category sample on the final classification, fcr is the false negative example, C represents the number of categories, and θ is the parameter of the given classifier; i∈(1, C), j∈(1, n); For a given set of w, the loss function J of the 1DCNN-GRU-LSTM model is M (θ) is optimized using the gradient descent algorithm under the back-propagation framework, and the loss function J M The partial derivative of (θ) with respect to θ is: The characteristic formula for weight w optimization solution is as follows: min1DCNN-GRU-LSTM(w)=(fcr1(θ),fcr2(θ),...,fcr C (i) (w) (14) For a given w, the parameter w of the 1DCNN-GRU-LSTM model is optimized using the gradient descent algorithm. The gradient descent algorithm can effectively find the local optimal solution, and the evolutionary algorithm can find a better initial solution for the gradient descent algorithm. The global optimal solution is found by combining the multi-objective optimization algorithm and the gradient descent algorithm.

Images