Man-machine cooperation mine ventilation intelligent control method

Abstract

The invention relates to a man-machine cooperation mine ventilation intelligent control method which comprises the steps that a state space S is defined as a set of finite states, and each state is composed of carbon dioxide concentration C, gas concentration D and dust concentration E, namely S = (C, D, E); performing normalization processing on each state attribute; constructing an action space; executing an action according to the state transition probability to perform state transition; setting a reward function based on the carbon dioxide concentration, the gas concentration and the dust concentration; setting a reward weight matrix, and providing additional rewards according to expert guidance feedback; setting a discount factor; making a simulation scene which is scaled down in equal proportion, and simulating a real mine environment; initializing a simulation environment and intelligent agent neural network parameters; the fan is automatically controlled to adjust parameters through a random-greedy exploration mechanism, and the state, action and reward data are recorded; and further performing data set construction, model training, man-machine collaborative decision making and model application.

Description

Technical Field

[0001] The present invention relates to the technical field of mine ventilation control, and more specifically, to a human-machine collaborative mine ventilation intelligent control method. Background Art

[0002] Mine ventilation is a key link in ensuring the safety of the underground working environment. It not only requires the introduction of fresh air to dilute and remove harmful gases, but also requires regulating the temperature in the mine and providing sufficient oxygen. Traditional ventilation control systems often cannot respond to changes in the underground environment in real time, which may cause harmful gases such as methane to accumulate and increase safety hazards. In addition, the "one-wind-blowing" ventilation mode is also difficult to achieve on-demand ventilation, resulting in serious energy waste.

[0003] In recent years, with the development of artificial intelligence technology, especially the application of reinforcement learning, mine ventilation control has gradually developed in the direction of intelligence. Some studies have tried to use reinforcement learning to automatically adjust the size and angle of the vents to adapt to changes in the underground environment and improve ventilation efficiency and safety. However, these methods often require a large amount of data during the training process, and the model performance is highly dependent on the data quality. If the training data is insufficient or of poor quality, it may lead to slow model convergence and unsatisfactory results.

[0004] The existing intelligent control system for mine ventilation based on reinforcement learning mainly has the following technical problems:

[0005] High data dependence: The training of reinforcement learning models requires a large amount of high-quality data, but it is often difficult to obtain this data in practical applications.

[0006] Inefficient training: Due to the lack of effective guidance, the model may take a long time to converge to a better strategy during the exploration phase.

[0007] Limited applicability: For some special situations, such as extreme environmental conditions or emergencies, it is difficult to make correct decisions relying solely on machine learning. Summary of the invention

[0008] The purpose of the present invention is to provide a human-machine collaborative mine ventilation intelligent control method to solve the problems in the prior art.

[0009] The human-machine collaborative mine ventilation intelligent control method comprises the following steps:

[0010] The mine ventilation control process is abstracted into a partially observable Markov decision process, which includes state space, action space, state transition probability, reward function and discount factor.

[0011] Construct the state space:

[0012] Define the state space S as a set of finite states, each state consists of carbon dioxide concentration C, gas concentration D, and dust concentration E, that is, S = (C, D, E);

[0013] Normalize each state attribute;

[0014] Constructing the action space:

[0015] The action space includes fan efficiency, wind tube diameter and wind tube deflection angle;

[0016] The action is specifically represented as a triplet, including fan efficiency, wind tube diameter, and wind tube deflection angle;

[0017] Execute actions to transfer states according to the state transition probability;

[0018] Constructing the reward function:

[0019] Set reward functions based on carbon dioxide concentration, gas concentration, and dust concentration to ensure that the agent's behavior can effectively improve environmental quality;

[0020] Set up a reward weight matrix and provide additional rewards based on expert guidance feedback;

[0021] Set discount factors;

[0022] Produce a scaled-down simulation scene to simulate the real mine environment;

[0023] Initialize the simulation environment and agent neural network parameters;

[0024] Automatically control the fan adjustment parameters through a random-greedy exploration mechanism, and record the state, action and reward data;

[0025] Further carry out data set construction, model training, human-machine collaborative decision-making, and model application.

[0026] Furthermore, executing actions to transfer states according to the state transfer probability specifically includes:

[0027] When an action a is executed, the state s changes to s′ according to the state transition probability; where s′∈S, mathematically described as: P(s′|s,a):S×A→S; a fixed state transition mechanism is adopted, that is, in state s t Next, perform action a t After that, the state transitions to s′ t The probability of is 1.

[0028] Furthermore, the reward function calculation formula is as follows:

[0029]

[0030] represents the generation reward of different feature items, ω T Represents the weight transposition of each reward in the environment, which is used to balance the importance of different feature items, r e represents the additional reward for expert guidance feedback, which helps the agent train faster; the reward weights of carbon dioxide concentration, gas concentration, and dust concentration are set to 0.3, 0.4, and 0.3 respectively; using r c 、r d 、r e Respectively represent the reward values ​​of the above three indicators; r c 、r d 、r e The calculation method is as follows:

[0031]

[0032]

[0033]

[0034] where c t d t 、e t It means that after the action is executed, the concentration of carbon dioxide, gas and dust in the environment will be greater than the safety threshold. A large negative reward will be given, and the smaller the index value, the greater the reward value.

[0035] Furthermore, the discount factor γ is set to 0.99.

[0036] Furthermore, the dataset is constructed as follows:

[0037] Integrate common scene and special scene data to build a training data set;

[0038] Discretize the data to reduce the correlation between the data.

[0039] Further, the model training is specifically as follows:

[0040] Using the experience replay mechanism, N groups of tuple data are extracted from the data set each time for network training;

[0041] A recurrent network with memory is used for training.

[0042] Furthermore, human-machine collaborative decision-making is specifically as follows:

[0043] When the agent faces similar action probabilities, it actively requests expert intervention to assist in selecting actions that are more beneficial to long-term benefits.

[0044] Furthermore, the model application is as follows:

[0045] After completing the model training, the model is deployed to the mine ventilation system to achieve intelligent control.

[0046] The beneficial effects of the present invention are as follows: improving training efficiency and decision quality:

[0047] By integrating expert experience, the present invention can significantly improve the learning efficiency and decision-making quality of the intelligent agent, especially when the amount of data is limited or the data quality is not high, and an effective ventilation control strategy can still be quickly trained.

[0048] Enhanced adaptability to special scenarios:

[0049] The present invention enhances the adaptability and robustness of the intelligent agent under special or extreme environmental conditions by incorporating data from special scenarios into the training data set, enabling it to handle these situations more effectively.

[0050] Realize human-machine collaborative decision-making:

[0051] When the intelligent agent faces similar action probabilities, the present invention sets up a human-machine collaborative decision-making mechanism, allowing the intelligent agent to request expert intervention to help the intelligent agent choose actions that are more conducive to long-term benefits in a high-uncertainty or complex environment.

[0052] Improve ventilation efficiency and safety:

[0053] The present invention can realize ventilation on demand in different environments, improve effective ventilation rate, and reduce energy waste through the above technical means. At the same time, it can also better remove harmful gases, reduce underground dust concentration, and ensure the life safety of miners.

[0054] Improving reinforcement learning training:

[0055] Through the intervention of expert experience, the present invention effectively improves the training efficiency and quality of the reinforcement learning algorithm, reduces the dependence on a large amount of high-quality data, and accelerates the speed of model convergence.

[0056] Optimizing ventilation control strategies:

[0057] The intelligent agent proposed in the present invention can adjust the ventilation strategy in real time according to the underground environmental data, thereby improving the response speed and accuracy of the ventilation system, thereby improving the overall performance of the entire mine ventilation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 This is a flowchart of offline training of the algorithm involved in this application;

[0059] Figure 2 This is a flowchart of the online execution of the algorithm involved in this application. DETAILED DESCRIPTION

[0060] In order to make the technical purpose, technical solution and beneficial effects of the present invention clearer, the technical solution of the present invention is further described below in conjunction with the drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0061] This application discloses a human-machine collaborative mine ventilation intelligent control method (reference Figure 1 and Figure 2 ), comprising the steps of:

[0062] Step 1: Environmental modeling, abstracting the wind turbine control process into a partially observable Markov decision process (POMDP), which consists of state space, action space, state transition probability, reward function and discount factor.

[0063] Step 2: Construct a state space. The state space is used to describe the problem scenario and is a set of state collections. The state space attributes of the present invention include: fan frequency, wind speed sensor data, C02 sensor data, dust sensor data, and gas sensor data.

[0064] Step 2: Construct a state space. The state space is used to describe the problem scenario. It is a finite set of states and is represented by S. State attributes include CO2 concentration, gas concentration, and dust concentration. In this invention, the CO2 concentration is recorded as C, the gas concentration is recorded as D, and the dust concentration is recorded as E. Then the state space S can be represented by a 3-tuple, S = (C, D, E). At time t, the current environmental state s t =(c t ,d t ,e t ), where c t ∈C,d t ∈D,e t ∈E.

[0065] Specifically, the present invention sets the carbon dioxide concentration range to [0, 1], the gas concentration range to [0, 2], and the dust concentration range to [0, 10]. Therefore, for a certain state s t , which can be represented by a triple (0.5,1,8). In order to avoid the influence of different dimensional data on neural network training, this application uses the formula:

[0066]

[0067] Normalize the data.

[0068] Step 3: Construct an action space. The action control describes the available action range that the agent can perform. The present invention abstracts the fan as an agent and uses the adjustable frequency range of the fan as the action space that the agent can perform.

[0069] Step 3: Construct an action space. The action control describes the available action range that the agent can perform. The present invention abstracts the fan as an agent and uses the fan efficiency, fan diameter, and fan deflection angle as the action space that the agent can perform. Specifically, the fan efficiency represents the percentage of the current fan frequency to the rated frequency, with a value range of [0,1], the fan diameter range of [0.5,1.5], and the fan deflection angle of [0,30]. Action a t It can be represented by a triple (0.5, 0.8, 23).

[0070] Step 4: Status transition matrix: After executing an action a, the state s changes to s′ according to the state transition probability. Where s′∈S, mathematically described as P(s′|s,a):S×A→S. The state transition probability is the state transition mechanism of the environment itself and has nothing to do with external information changes. The present invention adopts a fixed state transition mechanism, that is, in state s t Next, perform action a t After that, the state transitions to s′ t The probability of is 1.

[0071] Step 5: Construct a reward function. The reward is used to evaluate the effect of the agent's action and use the reward value to guide the agent to choose a more appropriate action. The present invention selects the environmental comprehensive index as the reward function, where the comprehensive index consists of the CO2 index state, the dust index state, and the gas index state.

[0072] Step 5: Construct a reward function. The reward is used to evaluate the effect of the agent's action, and the reward value is used to guide the agent to choose a more appropriate action. The present invention selects the comprehensive environmental index as the reward function, where the comprehensive index consists of the CO2 index state, the dust index state, and the gas index state. The reward function calculation formula is as follows:

[0073]

[0074] represents the generation reward of different feature items, ω TRepresents the weight transposition of each reward in the environment, which is used to balance the importance of different feature items, r e Represents an additional reward of expert guidance feedback, helping the agent train faster.

[0075] Specifically, this application sets the reward weights of carbon dioxide concentration, gas concentration, and dust concentration to 0.3, 0.4, and 0.3 respectively. c 、r d 、r e Respectively represent the reward values ​​of the above three indicators. The calculation methods of each are as follows:

[0076]

[0077]

[0078]

[0079] where c t ,d t 、e t It means that after the action is executed, the concentration of carbon dioxide, gas and dust in the environment will be greater than the safety regulations, and a large negative reward will be given. When the index value is smaller, the reward value will be greater.

[0080] Step 6: Set the discount factor. The discount factor range is (0-1]. The larger the value, the more attention the agent pays to future rewards. This application sets the discount factor to 0.99, which means that the agent pays more attention to long-term rewards.

[0081] Step 7: Construct a simulation environment. Since the real scene involves the life safety of miners, the present invention produces a simulation scene by scaling down the structure of the coal mine in equal proportion.

[0082] Step 8: Collect data from common scenarios, initialize the simulation environment and the agent neural network parameters (Deep-QLearning, DQN), and enable the agent to use the random-greedy exploration mechanism to automatically control the fan adjustment parameters. First, obtain the current environmental state S t 、The agent is based on the state value function Q π (s,a)=∑ s′ P(s′|(s,a)[R(s,a,s′)+γ∑ a′ π(a′|s′)Q π(a′, s′)], and get the probability of action value distribution. If it is a random action, execute the random action, otherwise determine whether the difference between the maximum and second largest values ​​of the action value probability is less than 0.05. If so, professionals intervene and set the action to be performed by the agent, otherwise the agent selects the action with the greedy maximum strategy. Then the environment returns the reward for the action according to the effect of the action after the agent executes it. After the above process is completed, the state, action, and reward data are stored as tuples and saved as training data.

[0083] Step 9: Collect data for special scenarios, initialize the simulation environment parameters and agent neural network parameters in special scenarios, enable professionals to control the fan adjustment parameters in characteristic environmental conditions through the human-computer interface, and package the current environment, agent actions, and rewards obtained as training data.

[0084] Step 10: Dataset construction: merge the common dataset and expert dataset into a training dataset, and discretize the data to reduce the correlation between the data.

[0085] Step 11: Model training, using the experience replay mechanism, each time take out N groups of data in the form of (s, a, r) ​​tuples from the data set, and use the data for DQN network training.

[0086] Step 11: Model training, using the experience replay mechanism, each time take out N groups of data in the form of (s, a, r) ​​tuples from the data set, and use the data for (Deep Recurrent Q-Network, DRQN) network training.

[0087] Since this application models the environment as a partially observable model, a recurrent network with memory is used for training.

[0088] Step 12: Complete model training and use the model to control the fan to achieve intelligent control of underground ventilation.

[0089] The embodiments that require protection in this application include:

[0090] The human-machine collaborative mine ventilation intelligent control method comprises the following steps:

[0091] The mine ventilation control process is abstracted into a partially observable Markov decision process, which includes state space, action space, state transition probability, reward function and discount factor.

[0092] Construct the state space:

[0093] Define the state space S as a set of finite states, each state consists of carbon dioxide concentration C, gas concentration D, and dust concentration E, that is, S = (C, D, E);

[0094] Normalize each state attribute;

[0095] Constructing the action space:

[0096] The action space includes fan efficiency, wind tube diameter and wind tube deflection angle;

[0097] The action is specifically represented as a triplet, including fan efficiency, wind tube diameter, and wind tube deflection angle;

[0098] Execute actions to transfer states according to the state transition probability;

[0099] Constructing the reward function:

[0100] Set reward functions based on carbon dioxide concentration, gas concentration, and dust concentration to ensure that the agent's behavior can effectively improve environmental quality;

[0101] Set up a reward weight matrix and provide additional rewards based on expert guidance feedback;

[0102] Set discount factors;

[0103] Produce a scaled-down simulation scene to simulate the real mine environment;

[0104] Initialize the simulation environment and agent neural network parameters;

[0105] Automatically control the fan adjustment parameters through a random-greedy exploration mechanism, and record the state, action and reward data;

[0106] Further carry out data set construction, model training, human-machine collaborative decision-making, and model application.

[0107] Preferably, executing an action to transfer a state according to the state transfer probability specifically includes:

[0108] When an action a is executed, the state s changes to s′ according to the state transition probability; where s′∈S, mathematically described as: P(s′|s,a):S×A→S; a fixed state transition mechanism is adopted, that is, in state s t Next, perform action a t After that, the state changes to s t ′ The probability of is 1.

[0109] The reward function calculation formula is as follows:

[0110]

[0111] represents the generation reward of different feature items, ω T Represents the weight transposition of each reward in the environment, which is used to balance the importance of different feature items, r erepresents the additional reward for expert guidance feedback, which helps the agent train faster; the reward weights of carbon dioxide concentration, gas concentration, and dust concentration are set to 0.3, 0.4, and 0.3 respectively; using r c 、r d 、r e Respectively represent the reward values ​​of the above three indicators; r c 、r d 、r e The calculation method is as follows:

[0112]

[0113]

[0114]

[0115] where c t d t 、e t It means that after the action is executed, the concentration of carbon dioxide, gas and dust in the environment will be greater than the safety threshold. A large negative reward will be given, and the smaller the index value, the greater the reward value.

[0116] Preferably, the discount factor γ is set to 0.99.

[0117] Preferably, the data set is constructed as follows:

[0118] Integrate common scene and special scene data to build a training data set;

[0119] Discretize the data to reduce the correlation between the data.

[0120] Preferably, the model training is specifically as follows:

[0121] Using the experience replay mechanism, N groups of tuple data are extracted from the data set each time for network training;

[0122] A recurrent network with memory is used for training.

[0123] Preferably, the human-machine collaborative decision-making is specifically:

[0124] When the agent faces similar action probabilities, it actively requests expert intervention to assist in selecting actions that are more beneficial to long-term benefits.

[0125] Preferably, the model application is specifically:

[0126] After completing the model training, the model is deployed to the mine ventilation system to achieve intelligent control.

[0127] This application proposes a human-machine collaborative mine ventilation intelligent control method, which aims to solve the problems existing in traditional mine ventilation control, such as the inability to make timely adjustments according to real-time changes in the tunnel environment, low effective ventilation rate, and energy waste. By combining expert experience with reinforcement learning technology, the present invention can significantly improve the intelligence and practicality of the mine ventilation system. The specific technical effects are as follows:

[0128] Improve training efficiency and decision quality:

[0129] By integrating expert experience, the present invention can significantly improve the learning efficiency and decision-making quality of the intelligent agent, especially when the amount of data is limited or the data quality is not high, and an effective ventilation control strategy can still be quickly trained.

[0130] Enhanced adaptability to special scenarios:

[0131] The present invention enhances the adaptability and robustness of the intelligent agent under special or extreme environmental conditions by incorporating data from special scenarios into the training data set, enabling it to handle these situations more effectively.

[0132] Realize human-machine collaborative decision-making:

[0133] When the intelligent agent faces similar action probabilities, the present invention sets up a human-machine collaborative decision-making mechanism, allowing the intelligent agent to request expert intervention to help the intelligent agent choose actions that are more conducive to long-term benefits in a high-uncertainty or complex environment.

[0134] Improve ventilation efficiency and safety:

[0135] The present invention can realize ventilation on demand in different environments, improve effective ventilation rate, and reduce energy waste through the above technical means. At the same time, it can also better remove harmful gases, reduce underground dust concentration, and ensure the life safety of miners.

[0136] Improving reinforcement learning training:

[0137] Through the intervention of expert experience, the present invention effectively improves the training efficiency and quality of the reinforcement learning algorithm, reduces the dependence on a large amount of high-quality data, and accelerates the speed of model convergence.

[0138] Optimizing ventilation control strategies:

[0139] The intelligent agent proposed in the present invention can adjust the ventilation strategy in real time according to the underground environmental data, thereby improving the response speed and accuracy of the ventilation system, thereby improving the overall performance of the entire mine ventilation system.

[0140] In summary, the present invention significantly improves the intelligence and practicality of the mine ventilation control system by combining reinforcement learning technology with expert experience, and can more effectively cope with the complex and changeable underground environment, thereby achieving safer and more efficient mine ventilation management.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate but not limit the technical solutions of the present invention, and any equivalent substitutions of the present invention and any modifications or partial substitutions that do not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A human-machine collaborative mine ventilation intelligent control method, characterized in that: Includes steps: The mine ventilation control process is abstracted into a partially observable Markov decision process, which includes state space, action space, state transition probability, reward function and discount factor. Construct the state space: Define the state space S as a set of finite states, each state consists of carbon dioxide concentration C, gas concentration D, and dust concentration E, that is, S = (C, D, E); Normalize each state attribute; Constructing the action space: The action space includes fan efficiency, wind tube diameter and wind tube deflection angle; The action is specifically represented as a triplet, including fan efficiency, wind tube diameter, and wind tube deflection angle; Execute actions to transfer states according to the state transition probability; Constructing the reward function: Set reward functions based on carbon dioxide concentration, gas concentration, and dust concentration to ensure that the agent's behavior can effectively improve environmental quality; Set up a reward weight matrix and provide additional rewards based on expert guidance feedback; Set discount factors; Produce a scaled-down simulation scene to simulate the real mine environment; Initialize the simulation environment and agent neural network parameters; Automatically control the fan adjustment parameters through a random-greedy exploration mechanism, and record the state, action and reward data; Further carry out data set construction, model training, human-machine collaborative decision-making, and model application.

2. The human-machine collaborative mine ventilation intelligent control method according to claim 1 is characterized in that: Executing actions according to the state transition probability to transfer states specifically includes: When an action a is executed, the state s changes to s′ according to the state transition probability; where s′∈S, mathematically described as: P(s ′ |s,a):S×A→S; adopt a fixed state transfer mechanism, that is, in state s t Next, perform action a t After that, the state transitions to s′ t The probability of is 1.

3. The human-machine collaborative mine ventilation intelligent control method according to claim 1 is characterized in that: The reward function calculation formula is as follows: represents the generation reward of different feature items, ω T Represents the weight transposition of each reward in the environment, which is used to balance the importance of different feature items, r e represents the additional reward for expert guidance feedback, which helps the agent train faster; the reward weights of carbon dioxide concentration, gas concentration, and dust concentration are set to 0.3, 0.4, and 0.3 respectively; using r c 、r d 、r e Respectively represent the reward values ​​of the above three indicators; r c 、r d 、r e The calculation method is as follows: where c t d t 、e t It means that after the action is executed, the concentration of carbon dioxide, gas and dust in the environment will be greater than the safety threshold. A large negative reward will be given, and the smaller the index value, the greater the reward value.

4. The human-machine collaborative mine ventilation intelligent control method according to claim 1 is characterized in that: The discount factor γ is set to 0.

99.

5. The human-machine collaborative mine ventilation intelligent control method according to claim 1 is characterized in that: The data set construction is as follows: Integrate common scene and special scene data to build a training data set; Discretize the data to reduce the correlation between the data.

6. The method for intelligent control of mine ventilation by human-machine collaboration according to claim 1, characterized in that: The specific model training is as follows: Using the experience replay mechanism, N groups of tuple data are extracted from the data set each time for network training; A recurrent network with memory is used for training.

7. The human-machine collaborative mine ventilation intelligent control method according to claim 1 is characterized in that: Human-machine collaborative decision-making is specifically as follows: when the intelligent agent faces similar action probabilities, it actively requests expert intervention to assist in selecting actions that are more conducive to long-term benefits.

8. The human-machine collaborative mine ventilation intelligent control method according to claim 1 is characterized in that: The model applications are as follows: After completing the model training, the model is deployed to the mine ventilation system to achieve intelligent control.

Images