Underground cavern construction ventilation control method with offline self-adaptive learning capability

Abstract

The invention relates to an underground cavern construction ventilation control method with off-line self-adaptive learning capability, which comprises the following steps of: acquiring environmental data and ventilation system operation parameters in real time, and constructing a data set after the environmental data and the ventilation system operation parameters are combined with historical data; inputting a training set in the data set into a bidirectional long short-term memory network BiLSTM for training, and packaging the trained bidirectional long short-term memory network BiLSTM into a ventilation system environment function; in a ventilation environment constructed by a BiLSTM model, interactive learning with the environment is carried out through an improved soft actor-commentator algorithm SAC, and finally an optimal control strategy is obtained; and deploying the optimal strategy obtained by learning into the ventilation system, and dynamically adjusting operation parameters according to a real-time environment state. According to the underground cavern construction ventilation control method based on the off-line adaptive learning capability, the problems that in traditional underground cavern ventilation control, the environment prediction precision is low, the control strategy convergence is unstable, the anti-interference capability is weak, and the adaptability is poor are solved.

Description

Technical Field

[0001] This invention relates to the field of underground cavern construction safety technology, specifically to an offline adaptive learning-enabled ventilation control method for underground cavern construction. Background Technology

[0002] During the construction of underground caverns, ventilation control faces significant technical challenges and complexities due to their high degree of enclosure, complex structure, and intricate internal airflow organization. Air volume is the core parameter for ventilation control. A reasonable air volume not only ensures the timely removal of harmful gases and dust generated during construction, protecting the safety and health of construction personnel, but also helps maintain a stable temperature within the cavern, preventing adverse effects of high temperatures on the structure and mechanical equipment. However, current underground cavern ventilation system control still relies primarily on fixed thresholds and experience-based control, making it difficult to respond in real-time to dynamic changes in the construction environment, easily leading to energy waste or insufficient ventilation. Therefore, achieving scientific and precise adaptive ventilation control is of great significance for ensuring the safe and efficient construction of underground caverns.

[0003] Current ventilation control methods mainly involve setting static control points and using reinforcement learning for control, but these methods still have shortcomings. When facing complex and time-varying underground cavern ventilation environments, the adaptive control capability of static setpoint control is limited. Existing simulation models do not adequately depict the coupling mechanism between multivariate environmental data, making it difficult to accurately reflect the dynamic changes in gas diffusion and pollutant accumulation in underground caverns. Existing ventilation control optimization methods are unable to adapt to the strong nonlinearity and non-stationary characteristics of underground cavern ventilation systems, resulting in the inability to achieve stable and adaptive intelligent ventilation control under complex working conditions. Training strategies in real construction environments carries significant risks, as random exploration processes may lead to abnormal environmental conditions, equipment wear and tear, and increased energy consumption. Summary of the Invention

[0004] The technical problem of this invention is to achieve adaptive and stable control of the ventilation system during underground cavern construction by using an offline adaptive learning method and a new framework for constructing a soft actor-commentator algorithm (SAC) training environment through a bidirectional long short-term memory (BiLSTM) network to accurately simulate the operation of a real ventilation system.

[0005] The purpose of this invention is to solve the above-mentioned problems and propose an offline adaptive learning-based ventilation control method for underground cavern construction, comprising the following steps: S1. Collect environmental data and ventilation system operating parameters in real time, and combine them with historical data to construct a dataset; S2. Input the training set in the dataset into the Bidirectional Long Short-Term Memory (BiLSTM) network for training, and encapsulate the trained BiLSTM network as a ventilation system environmental function. S3. In the ventilation environment constructed by the BiLSTM model, the improved soft actor-critic algorithm SAC is used to interact and learn from the environment to obtain the optimal control strategy. S4. Based on the optimal control strategy, the operating parameters under real-time environmental conditions are dynamically adjusted to achieve adaptive control of ventilation in underground caverns.

[0006] Furthermore, in step S1, the environmental data consists of wind speed, temperature, CO concentration, and dust concentration collected by monitoring equipment deployed during the construction of the underground cavern.

[0007] Furthermore, in step S1, the operating parameters of the ventilation system are the air volume during the construction of the underground cavern.

[0008] Preferably, step S1 includes preprocessing the dataset and dividing it proportionally: 1) Outlier detection: Outliers are identified using a statistical method based on the interquartile range (IQR). 2) Missing value handling: Missing values ​​are handled using the mean imputation method to ensure the integrity and continuity of the data.

[0009] Furthermore, step S2 includes the following steps: S21. Use wind speed, temperature, CO concentration and dust concentration data to train a BiLSTM model to characterize the temporal coupling effect between these four environmental parameters, and provide a high-precision dynamic environment model for the SAC agent. S22. The trained BiLSTM model is encapsulated as a ventilation system environment function. The inputs are the wind speed, temperature, CO concentration and dust concentration of the current state, and the output is the prediction results of the above four environmental parameters at the next time step. At the same time, a ventilation effect evaluation index is generated based on the output.

[0010] Preferably, step S21 includes setting the training loss function and accuracy evaluation metric for the BiLSTM model: 1) Training loss function: The mean squared error (MSE) is used as the loss function to measure the overall deviation between the model's predicted values ​​and the true values. 2) Accuracy evaluation index: When using the test set to verify the prediction accuracy of the BiLSTM model, the mean absolute error (MAE) and root mean square error (RMSE) are selected as evaluation indexes.

[0011] Preferably, the formulas for calculating the mean square error (MSE), mean absolute error (MAE), and root mean square error (RMSE) are as follows: ; ; ; In the formula, Indicates the predicted value. Represents the observed value. This indicates the number of observations.

[0012] Preferably, in step S3, the improved soft actor-critic algorithm SAC includes the following sub-steps: S31. A low-pass filter mechanism is introduced to handle environmental noise, and smoothing is performed at the state input terminal. The calculation formula is as follows: ; In the formula, This indicates the state before filtering. This indicates the filtered state. Represents the smoothing coefficient. ; S32. Reward shaping is used to mitigate convergence oscillations; the calculation formula is as follows: ; In the formula, This represents the reward value before scaling. This represents the scaled reward value. Represents the hyperbolic tangent function; S33. Introduce historical target fusion and temporal consistency constraints to enhance the temporal consistency of SAC target value updates. The calculation formula is as follows: ; In the formula, Indicates the target of the current calculation value, Indicates the previous moment Value estimation, Represents the historical fusion coefficient. ; S34. Priority Experience Replay (PER) mechanism based on temporal difference error (TD) enables high-value samples to receive higher weights during training; S35. Entropy coefficient adjustment maintains exploration and training stability. Strategies such as the minimum entropy lower bound are used to balance exploration intensity and training stability. The calculation formula is: ; In the formula, The entropy coefficient, This is the threshold value for the minimum entropy coefficient.

[0013] Preferably, step S34 includes the calculation of time-series differential (TD) error and sampling probability: 1) The formula for calculating the timing difference (TD) error is: ; In the formula, For the first TD error of each sample As a discount factor, For the goal network, For state-action value function, These represent the sample state and action, respectively. 2) The formula for calculating the sampling probability is: ; In the formula, For probability, To control sampling priority.

[0014] Compared with the prior art, the beneficial effects of the present invention include: 1) The offline adaptive learning capability underground cavern construction ventilation control method provided by the present invention solves the problems of low environmental prediction accuracy, unstable convergence of control strategy, weak anti-interference ability and poor adaptability in traditional underground cavern ventilation control.

[0015] 2) This invention employs a Bidirectional Long Short-Term Memory (BiLSTM) network to train the environmental functions of the ventilation system, accurately capturing the inherent patterns and interrelationships of various environmental parameters over time. Simultaneously, a dual evaluation system constructed using Mean Square Error (MSE), Mean Absolute Error (MAE), and Root Mean Square Error (RMSE) ensures the controllability of the model training process and the high accuracy of the prediction results. Furthermore, it provides real-time and accurate environmental prediction data for control strategy learning, solving the problem of traditional models' difficulty in quantifying the coupling relationships of time-series parameters. 3) This invention achieves offline adaptive learning and online dynamic control of the ventilation system through the technical architecture of BiLSTM environment prediction and improved SAC strategy learning. It accurately predicts the environmental state at the next moment through pre-trained environmental functions and dynamically adjusts operating parameters such as ventilation volume based on the optimized control strategy, which can quickly respond to the dynamic changes of environmental parameters during the construction of underground caverns. Attached Figure Description

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0017] Figure 1 This is a schematic diagram of the overall process of the offline adaptive learning capability underground cavern construction ventilation control method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall framework of the offline adaptive learning capability underground cavern construction ventilation control method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of outlier detection according to an embodiment of the present invention; Figure 4 This is a schematic diagram of BiLSTM model prediction in an embodiment of the present invention; Figure 5 This is a schematic diagram of SAC agent training according to an embodiment of the present invention; Figure 6 This is a schematic diagram comparing the training of different models in an embodiment of the present invention; Figure 7 This is a schematic diagram comparing the present invention with traditional ventilation control in an embodiment of the present invention; Figure 8 This is a control chart comparing conventional control and SAC control of environmental parameters in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 and Figure 2 As shown, a method for controlling ventilation during underground cavern construction with offline adaptive learning capability includes the following steps: S1. Collect environmental data and ventilation system operating parameters in real time, and combine them with historical data to construct a dataset; The environmental data consisted of wind speed, temperature, CO concentration, and dust concentration collected by monitoring equipment deployed during the construction of the underground cavern. The ventilation system operating parameters were the air volume during the construction of the underground cavern.

[0020] The dataset is preprocessed and divided proportionally: 1) Outlier detection: such as Figure 3 As shown, outliers are identified using a statistical method based on interquartile range (IQR), with data points outside ±1.5 IQR considered as outliers. 2) Missing value handling: Missing values ​​are handled using the mean imputation method to ensure the integrity and continuity of the data.

[0021] S2. Input the training set in the dataset into the Bidirectional Long Short-Term Memory (BiLSTM) network for training, and encapsulate the trained BiLSTM network into a ventilation system environment function.

[0022] S21. Use wind speed, temperature, CO concentration and dust concentration data to train a BiLSTM model to characterize the temporal coupling effect between these four environmental parameters, and provide a high-precision dynamic environment model for the SAC agent.

[0023] Specifically, the BiLSTM model is an extended version of the Long Short-Term Memory (LSTM) network, considering both past and future states to improve prediction performance. The training process is as follows: Input data sequence data sequence The expression is: ; In the formula, Let n represent a data element, and n represent the maximum value of the data element.

[0024] Combining two LSTMs, one processing data from front to back and the other from back to front, the LSTM units control the information flow through a gating mechanism. The expressions for the input gate, forget gate, and output gate of the gating mechanism are as follows: ; ; ; ; ; ; In the formula, This is the input for the current moment; This is the hidden state from the previous time step; The current hidden state; It is the sigmoid function; A vector of candidate values; The output of the input gate; The output of the forget gate; This is the output of the output gate; This represents a candidate value for the cell state at the current moment. This represents the cell state at the previous moment; For bias; This is the weight matrix.

[0025] The prediction outputs of these two LSTM units are concatenated to obtain more comprehensive information; the expression is: ; ; ; In the formula, It is in a positive hidden state; This is a reverse hidden state; This is a connection between the forward and reverse hidden states.

[0026] Setting the training loss function and accuracy evaluation metric for BiLSTM models: 1) Training loss function: The mean squared error (MSE) is used as the loss function to measure the overall deviation between the model's predicted values ​​and the true values. 2) Accuracy evaluation index: When using the test set to verify the prediction accuracy of the BiLSTM model, the mean absolute error (MAE) and root mean square error (RMSE) are selected as evaluation indexes.

[0027] The formulas for calculating the mean square error (MSE), mean absolute error (MAE), and root mean square error (RMSE) are as follows: ; ; ; In the formula, Indicates the predicted value. Represents the observed value. This indicates the number of observations.

[0028] S22. The trained BiLSTM model is encapsulated as a ventilation system environment function. The inputs are the wind speed, temperature, CO concentration and dust concentration of the current state, and the output is the prediction results of the above four environmental parameters at the next time step. At the same time, a ventilation effect evaluation index is generated based on the output.

[0029] The prediction results of the BiLSTM model are as follows Figure 4 As shown.

[0030] S3. In the ventilation environment constructed by the BiLSTM model, the improved soft actor-critic algorithm SAC is used to interact and learn from the environment to obtain the optimal control strategy.

[0031] Specifically, the SAC agent selects control actions based on the current environmental state at each moment. After executing the actions, the ventilation system generates new environmental feedback, thus forming a dynamic interactive process of state, action, and state transition. Through continuous iterative learning, the agent can establish a mapping relationship between state changes and control actions, achieving adaptive and coordinated control of airflow and environmental parameters.

[0032] SAC is based on an actor-critic framework, consisting of one actor network, two critic networks, and two target-critic networks. The actor networks are responsible for decision-making and update themselves through policy functions. The SAC's actor network dynamically adjusts the airflow based on real-time environmental conditions. The SAC's critic networks evaluate the overall impact of these adjustments and update them through the target Q-network.

[0033] The policy function is expressed as: ; In the formula, The environmental conditions are set (wind speed, temperature, CO concentration, and dust concentration). For action sets; For the reward function; This refers to the state-action distribution generated after the agent makes a decision.

[0034] SAC employs a dual-critic network approach to mitigate the Q-value overestimation problem, with the target-critic network guiding the critical network updates. The expression for the value function Q is: ; In the formula, This is the discount factor; Let be the mathematical expectation.

[0035] In step S3, the improved soft actor-critic algorithm SAC includes the following sub-steps: S31. A low-pass filter mechanism is introduced to handle environmental noise, and smoothing is performed at the state input terminal. The calculation formula is as follows: ; In the formula, This indicates the state before filtering. This indicates the filtered state. Represents the smoothing coefficient. ; S32. Reward shaping is used to mitigate convergence oscillations; the calculation formula is as follows: ; In the formula, This represents the reward value before scaling. This represents the scaled reward value. Represents the hyperbolic tangent function; S33. Introduce historical target fusion and temporal consistency constraints to enhance the temporal consistency of SAC target value updates. The calculation formula is as follows: ; In the formula, Indicates the target of the current calculation value, Indicates the previous moment Value estimation, Represents the historical fusion coefficient. ; S34. The Priority Experience Replay (PER) mechanism based on Temporal Difference Error (TD) enables high-value samples to receive higher weights during training.

[0036] Calculation of time-series differential (TD) error and sampling probability: 1) The formula for calculating the timing difference (TD) error is: ; In the formula, For the first TD error of each sample As a discount factor, For the goal network, For state-action value function, These represent the sample state and action, respectively. 2) The formula for calculating the sampling probability is: ; In the formula, For probability, To control sampling priority.

[0037] S35. Entropy coefficient adjustment maintains exploration and training stability. Strategies such as the minimum entropy lower bound are used to balance exploration intensity and training stability. The calculation formula is: ; In the formula, The entropy coefficient, This is the threshold value for the minimum entropy coefficient.

[0038] The training process of the improved SAC is as follows: Figure 5 The figure shows the changes in reward value and average reward value with the number of iterations. The figure reveals that the reward per iteration fluctuates significantly in the early stages of training, while the average reward gradually increases with the number of iterations and eventually stabilizes. This result demonstrates that the proposed method can effectively converge to the optimal policy in complex, nonlinear underground cavern environments.

[0039] The improved SAC has training advantages over the two typical baseline algorithms, DDPG and PPO, such as: Figure 6As shown, the reward value of DDPG decreases in the first 10 rounds, then almost stagnates, and only suddenly rises after 60 rounds, exhibiting a clear delayed convergence. This indicates that DDPG is sensitive to hyperparameters and exploration strategies, and has low learning efficiency in the early stages. The reward of PPO decreases with increasing steps and then oscillates within a low reward range for a long period, without showing a significant upward trend. This indicates that PPO struggles to effectively learn strategies that sustainably increase rewards in high-dimensional continuous action spaces, exhibiting low convergence efficiency and insufficient adaptability. In conclusion, the improved SAC is more suitable for adaptive control tasks in underground cavern ventilation systems.

[0040] S4. Based on the optimal control strategy, the operating parameters under real-time environmental conditions are dynamically adjusted to achieve adaptive control of ventilation in underground caverns.

[0041] Figure 7 The demonstration showcases the SAC agent's control over airflow. As can be seen, under traditional control mode, airflow exhibits significant random fluctuations throughout the process. However, after SAC control, the airflow curve clearly converges and remains within a relatively stable range.

[0042] Figure 8 This study demonstrates the control effectiveness of the SAC agent on four key environmental parameters: temperature, dust, CO concentration, and wind speed. It shows that the agent can effectively control airflow, keeping temperature and dust levels below control thresholds. Furthermore, the results for CO and wind speed indicate that the agent can dynamically adjust airflow distribution to cope with multivariate disturbances, exhibiting good overall stability and ensuring the safety and comfort of the tunnel construction environment.

[0043] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for controlling ventilation during underground cavern construction with offline adaptive learning capability, characterized in that, Includes the following steps: S1. Collect environmental data and ventilation system operating parameters in real time, and combine them with historical data to construct a dataset; S2. Input the training set in the dataset into the Bidirectional Long Short-Term Memory (BiLSTM) network for training, and encapsulate the trained BiLSTM network as a ventilation system environmental function. S3. In the ventilation environment constructed by the BiLSTM model, the improved soft actor-critic algorithm SAC is used to interact and learn from the environment to obtain the optimal control strategy. S4. Based on the optimal control strategy, the operating parameters under real-time environmental conditions are dynamically adjusted to achieve adaptive control of ventilation in underground caverns.

2. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 1, characterized in that, In step S1, the environmental data refers to the wind speed, temperature, CO concentration, and dust concentration collected by monitoring equipment deployed during the construction of the underground cavern.

3. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 1, characterized in that, In step S1, the operating parameter of the ventilation system is the air volume during the construction of the underground cavern.

4. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 1, characterized in that, Step S1 includes preprocessing the dataset and dividing it proportionally; the preprocessing includes: 1) Outlier detection: Outliers are identified using a statistical method based on the interquartile range (IQR). 2) Missing value handling: Missing values ​​are handled using the mean imputation method to ensure the integrity and continuity of the data.

5. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 1, characterized in that, Step S2 includes the following steps: S21. Use wind speed, temperature, CO concentration and dust concentration data to train a BiLSTM model to characterize the temporal coupling effect between these four environmental parameters, and provide a high-precision dynamic environment model for the SAC agent. S22. The trained BiLSTM model is encapsulated as an environment function. The input is the wind speed, temperature, CO concentration and dust concentration of the current state. The output is the prediction results of the above four environmental parameters at the next time step. At the same time, a ventilation effect evaluation index is generated based on the output.

6. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 5, characterized in that, Step S21 includes setting the training loss function and accuracy evaluation metric for the BiLSTM model: 1) Training loss function: The mean squared error (MSE) is used as the loss function to measure the overall deviation between the model's predicted values ​​and the true values. 2) Accuracy evaluation index: When using the test set to verify the prediction accuracy of the BiLSTM model, the mean absolute error (MAE) and root mean square error (RMSE) are selected as evaluation indexes.

7. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 6, characterized in that, The formulas for calculating the mean square error (MSE), mean absolute error (MAE), and root mean square error (RMSE) are as follows: ； ； ； In the formula, Indicates the predicted value. Represents the observed value. Indicates the number of observations.

8. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 1, characterized in that, In step S3, the improved soft actor-critic algorithm SAC includes the following sub-steps: S31. A low-pass filter mechanism is introduced to handle environmental noise, and smoothing is performed at the state input terminal. The calculation formula is as follows: ； In the formula, This indicates the state before filtering. This indicates the filtered state. Represents the smoothing coefficient. ; S32. Reward shaping is used to mitigate convergence oscillations; the calculation formula is as follows: ； In the formula, This represents the reward value before scaling. This represents the scaled reward value. Represents the hyperbolic tangent function; S33. Introduce historical target fusion and temporal consistency constraints to enhance the temporal consistency of SAC target value updates. The calculation formula is as follows: ； In the formula, Indicates the target of the current calculation value, Indicates the previous moment Value estimation, Represents the historical fusion coefficient. ; S34. Priority Experience Replay (PER) mechanism based on temporal difference error (TD) enables high-value samples to receive higher weights during training; S35. Entropy coefficient adjustment maintains exploration and training stability. Strategies such as the minimum entropy lower bound are used to balance exploration intensity and training stability. The calculation formula is: ； In the formula, The entropy coefficient, This is the threshold value for the minimum entropy coefficient.

9. The method for controlling ventilation during underground cavern construction with offline adaptive learning capability according to claim 1, characterized in that, Step S34 includes the calculation of time-series differential (TD) error and sampling probability: 1) The formula for calculating the timing difference (TD) error is: ； In the formula, For the first TD error of each sample As a discount factor, For the goal network, For state-action value function, These represent the sample state and action, respectively. 2) The formula for calculating the sampling probability is: ； In the formula, For probability, To control sampling priority.

Images