A training method of a fully-mechanized mining face dust removal system adaptive control model based on reinforcement learning

Abstract

This invention discloses a training method for an adaptive control model of a fully mechanized tunneling face dust removal system based on reinforcement learning. The method includes: constructing a physical model of the fully mechanized tunneling face; generating dust at the tunneling points of the physical model; capturing dust concentration at various locations on the tunneling face using dust sensors to obtain dust distribution data; constructing a reinforcement learning model; training the reinforcement learning model based on the obtained dust distribution data; and using the trained reinforcement learning model for adaptive control of the ventilation system. This invention can generate a large amount of training data, and through training, an adaptive control ventilation and dust removal model can be obtained, effectively improving dust removal efficiency, effectively reducing equipment power consumption and manual intervention, and promoting the intelligent development of coal mines.

Description

Technical Field

[0001] This invention belongs to the field of mine ventilation and dust removal technology, and particularly relates to a training method for an adaptive control model of a fully mechanized tunnel face dust removal system based on reinforcement learning. Background Technology

[0002] Reinforcement learning, a branch of machine learning, refers to the training process in which an agent continuously interacts with its environment, using actions, rewards, and observations to progressively update and optimize the agent. In this process, the agent continuously learns from its interactions with the environment and adjusts its behavioral strategies, thereby achieving a higher level of intelligence. Current mine dust control systems can collect dust concentration data using dust concentration sensors and dynamically adjust fan power and spray water volume to reduce dust concentration. However, at the tunnel face, dust distribution is extremely uneven, and the dust concentrations captured by different sensors vary greatly, making it impossible to determine the control strategy based on a single parameter. Moreover, with the widespread application of "long-pressure, short-extraction" ventilation and dust control systems at tunnel faces, multiple control variables, such as extraction and compression power and air outlet location, are coupled together, making it impossible to improve dust control efficiency simply by increasing the extraction and compression air volume. Therefore, adaptive control of dust control systems at tunnel faces is difficult to achieve. For dynamically changing environments, reinforcement learning models can continuously adjust control strategies based on real-time changes in dust concentration at the tunnel face, ultimately obtaining an ideal model that can reduce dust concentration to a safe level in the shortest possible time under various environmental conditions. However, training reinforcement learning models requires a large amount of dynamic data on environmental and equipment changes, and currently there are no devices or methods to train adaptive control models for dust removal systems at tunnel faces based on reinforcement learning. Summary of the Invention

[0003] To address the aforementioned technical issues, this invention proposes a training method for an adaptive control model of a fully mechanized tunneling face dust removal system based on reinforcement learning. This method can generate a large amount of training data, and through training, an adaptive control ventilation and dust removal model can be obtained, effectively improving dust removal efficiency, reducing equipment power consumption and manual intervention, and promoting the intelligent development of coal mines.

[0004] To achieve the above objectives, this invention provides a training method for an adaptive control model of a fully mechanized tunnel face dust removal system based on reinforcement learning, comprising:

[0005] A physical model of the tunnel face is constructed, dust is generated at the tunneling points of the physical model, and dust sensors are used to capture the dust concentration at various locations on the tunnel face to obtain dust distribution data.

[0006] A reinforcement learning model is constructed and trained based on the obtained dust distribution data. The trained reinforcement learning model is then used for adaptive control of the ventilation system.

[0007] Optionally, building a physical model of the tunnel face includes:

[0008] The physical model of the tunnel face is divided into three surfaces: the tunnel face, the tunnel wall, and the model opening. The tunnel face is sealed to the tunnel wall, and the right side of the model opening is open. The bottom edge length of the tunnel is a preset length. The tunnel face is composed of an upper arch and a lower rectangle. The physical model of the tunnel face includes a motor, a fan, a dust generator, a dust sensor, and a proportionally replaced traditional exhaust duct. The motor, fan, and dust sensor are selected to support networking or network connection via an interface module.

[0009] Optionally, dust generation at the tunneling points of the fully mechanized face physical model includes:

[0010] The outlet of the forced-in ventilation duct is installed facing the tunneling face to introduce fresh air into the tunneling area and cover the dust source. The outlet of the exhaust ventilation duct is located on the right side of the model opening end to discharge dust. The extraction and pressure pipes are installed above the platform support and connected to the extraction and pressure fans respectively. The control terminal of the fan is connected to the main unit to control the ventilation equipment.

[0011] Several holes are drilled in the tunneling face of the physical model of the tunnel face to serve as dust inlets for dust generators. Dust generators are installed at the corresponding dust inlets, and the output ports of the dust generators are connected to pipes, which are then connected to the tunnel face. Dust is transmitted through the holes in the tunnel face, and dust generation is completed in conjunction with the control of ventilation equipment.

[0012] Optionally, installing dust sensors includes: distributing the dust sensors in an array evenly above the support according to the area and characteristics of the excavation face.

[0013] Optionally, the dust concentration at various locations on the tunnel face captured by the dust sensor includes:

[0014]

[0015] Where C represents the overall dust concentration; C i W represents the dust concentration measured by the i-th sensor. i Let be the weight of the i-th sensor.

[0016] Optionally, constructing a reinforcement learning model includes: an input layer, a hidden layer, and an output layer;

[0017] The input layer is used to receive the current status, which includes dust concentration, ambient temperature, humidity, wind speed, and power data of each device.

[0018] The hidden layer is used to set up several fully connected layers;

[0019] The output layer is used to output the expected reward of the action taken in the current state according to each action of the system. The action includes adjusting the exhaust air volume, the compressed air volume, and the air outlet position.

[0020] Optionally, before training the reinforcement learning model based on the obtained dust distribution data, it is necessary to design a strategy to select the optimal action and update the expected reward of taking the action in the current state based on the reinforcement learning model.

[0021] Optionally, the design strategy for selecting the optimal action includes:

[0022] a * =max a Q(s, a);

[0023] Where s is each state; a is the action chosen by the agent that has the maximum Q value; Q is the expected reward of taking the action in the current state; a * The optimal action to be chosen for the current state s.

[0024] Optionally, updating the expected reward for taking an action in the current state based on the reinforcement learning model includes:

[0025] Q(s, a)=(1-λ) t )[R+γmax a Q(s′,a′;

[0026] Where Q(s, a) is the total reward expected by taking action a in state s; s is each state; a is the action chosen by the agent that has the maximum Q value; λ t R is the learning rate; R is the immediate reward; γ is the discount factor; max a Q(s′,a′) is the maximum value of Q among all possible actions a′ in the next state s′; Q is the expected reward of taking an action in the current state.

[0027] Optionally, training the reinforcement learning model based on the obtained dust distribution data includes:

[0028] The reward value for each data sample is calculated according to the set reward function. The empirical data with the reward value is added to the replay buffer for the early training of the model and to generate a preliminary dust control strategy.

[0029] The reinforcement learning model employs an experience replay mechanism to store experience data with reward values ​​in a replay buffer.

[0030] Calculate the Q-value for each action in the current state, and update the Q-value by minimizing the loss function, where Q is the expected reward for taking the action in the current state;

[0031] When the loss function reaches the predetermined convergence condition or the set threshold, training stops, and the trained reinforcement learning model is output for adaptive control of the ventilation system.

[0032] Technical effects of the invention: The invention discloses a training method for an adaptive control model of a fully mechanized tunneling face dust removal system based on reinforcement learning. This method can generate a large amount of training data and obtain an adaptive control ventilation and dust removal model through training, which can effectively improve dust removal efficiency, reduce equipment power consumption and manual intervention, and promote the intelligent development of coal mines. Attached Figure Description

[0033] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a flowchart illustrating a method for training an adaptive control model for a fully mechanized tunnel face dust removal system according to an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the structure for building the physical model of the tunnel face in an embodiment of the present invention;

[0036] Figure 3 This is a schematic diagram of the sensor bracket installation according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram of the network topology according to an embodiment of the present invention. Detailed Implementation

[0038] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0039] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0040] In the field of intelligent dust control, existing technologies include tunneling systems, long-pressure short-extraction intelligent frequency conversion control systems, two-stage air curtain generation systems based on Laval nozzles, wet scrubber systems, and intelligent control systems. The use of a two-stage pneumatic air curtain system to isolate dust and a wet scrubber system for dust suppression effectively prevents dust diffusion and solves the problem of poor dust control performance with single technologies. Simultaneously, intelligent control technology enables dynamic regulation of airflow and dust concentration, improving production safety. For dynamically changing environments, a new control strategy is derived by reconstructing a simulation model and performing calculations and analysis. This control method requires significant computational resources, and the simulation process is time-consuming and lacks real-time performance. In contrast, building a training platform to train a reinforcement learning model provides a data-driven control method rather than a physical model-based approach. Once the model is trained, the system can generate adaptive strategies and actions in real time based on environmental changes, effectively controlling dust concentration at the tunneling surface. Existing technologies also include an image acquisition device for acquiring images of the tunneling face, an environmental monitoring module for real-time acquisition of environmental data from the tunneling face, an operation monitoring module for real-time monitoring of local ventilation fan operation data, and an operation optimization module. This module performs time-series prediction based on environmental and operational data to obtain a predicted air volume. It then constructs a convolutional neural network based on a combination of images, environmental, and operational data from the tunneling face to obtain an auxiliary air volume value. Finally, it calculates a controlled air volume value by combining the predicted and auxiliary air volume values. A controller adjusts the local ventilation fan based on the controlled air volume value calculated by the operation optimization module. This ventilation system exhibits good scalability and flexibility, adapting to the air volume prediction needs of coal mines of different types and scales, and possesses significant application value and market prospects. This disclosure provides an operation optimization module that performs time-series prediction based on environmental and operational data to obtain a predicted air volume value. It then constructs a convolutional neural network based on a combination of images, environmental, and operational data from the tunneling face to obtain an auxiliary air volume value. Finally, it calculates a controlled air volume value by combining the predicted and auxiliary air volume values. The airflow control method disclosed herein uses deep learning to predict the required airflow and then controls the airflow, which differs from the control principle of reinforcement learning used in this invention. Furthermore, the object controlled by this disclosure is a local ventilation fan, and it cannot control multivariable systems such as "long-pressure, short-extraction" dust removal systems.

[0041] like Figure 1 As shown in the figure, this embodiment provides a training method for an adaptive control model of a fully mechanized tunnel face dust removal system based on reinforcement learning, including:

[0042] A physical model of the tunnel face is constructed, dust is generated at the tunneling points of the physical model, and dust sensors are used to capture the dust concentration at various locations on the tunnel face to obtain dust distribution data.

[0043] A reinforcement learning model is constructed and trained based on the obtained dust distribution data. The trained reinforcement learning model is then used for adaptive control of the ventilation system.

[0044] Furthermore, such as Figure 2 The physical model of the tunnel face shown includes:

[0045] The physical model of the tunnel face is divided into three surfaces: the tunnel face, the tunnel wall, and the model opening. The tunnel face is sealed to the tunnel wall, and the right side of the model opening is open. The bottom edge length of the tunnel is a preset length. The tunnel face is composed of an upper arch and a lower rectangle. The physical model of the tunnel face includes a motor, a fan, a dust generator, a dust sensor, and a proportionally scaled-down exhaust fan. The motor, fan, and dust sensor are selected to support networking or network connection via an interface module. A represents the tunnel face, B represents the tunnel wall, and C represents the model opening.

[0046] Furthermore, dust generation at the tunneling points of the physical model of the tunnel face includes:

[0047] The outlet of the forced-in ventilation duct is installed facing the tunneling face to introduce fresh air into the tunneling area and cover the dust source. The outlet of the exhaust ventilation duct is located on the right side of the model opening end to discharge dust. The extraction and pressure pipes are installed above the platform support and connected to the extraction and pressure fans respectively. The control terminal of the fan is connected to the main unit to control the ventilation equipment.

[0048] Several holes are drilled in the tunneling face of the physical model of the tunnel face to serve as dust inlets for dust generators. Dust generators are installed at the corresponding dust inlets, and the output ports of the dust generators are connected to pipes, which are then connected to the tunnel face. Dust is transmitted through the holes in the tunnel face, and dust generation is completed in conjunction with the control of ventilation equipment.

[0049] Furthermore, the installation of dust sensors at the dust inlet includes: distributing the dust sensors evenly in an array above the support structure according to the area and characteristics of the excavation face, such as... Figure 3 As shown.

[0050] Specifically, based on the area and characteristics of the tunnel face, the sensors are evenly distributed in an array above the support to ensure coverage of high-risk areas and near dust sources.

[0051] Use a random function to generate dust generation rate values ​​to ensure that the dust generation rate fluctuates within a set range:

[0052] R(t)=R min +(R max -R min )·rand;

[0053] Where R(t) is the dust generation rate at the current time t; R min and R max These are the minimum and maximum values ​​of the dust generation rate, respectively; rand() generates a random number in the range [0,1].

[0054] The system is configured to update R(t) every minute to simulate a dynamic dust-generating environment. At regular intervals, the dust generator's dust inlet connection points are manually changed, altering the pipe distribution. This method simulates the diversity of dust distribution in the tunnel section, such as different high-concentration and low-concentration zones.

[0055] Start the ventilation and dust generation equipment, and transmit the dust concentration data of the tunnel boring machine collected by the sensors to the host computer for further processing and analysis via wireless or wired network.

[0056] Furthermore, the use of dust sensors to capture dust concentrations at various locations on the tunnel face includes:

[0057]

[0058] Where C represents the overall dust concentration; C i W represents the dust concentration measured by the i-th sensor. i Let be the weight of the i-th sensor.

[0059] Specifically, based on the importance of each sensor location (the area near the dust generator has a higher weight, while the area far from the dust generator has a lower weight), the dust concentration collected by each sensor is weighted and averaged to obtain the dust concentration of the entire tunnel face.

[0060] Furthermore, such as Figure 4 The reinforcement learning model shown includes an input layer, a hidden layer, and an output layer.

[0061] The input layer is used to receive the current status, which includes dust concentration, ambient temperature, humidity, wind speed, and power data of each device.

[0062] The hidden layer is used to set up several fully connected layers;

[0063] The output layer is used to output the expected reward of the action taken in the current state according to each action of the system. The action includes adjusting the exhaust air volume, the compressed air volume, and the air outlet position.

[0064] Furthermore, before training the reinforcement learning model based on the obtained dust distribution data, it is necessary to design a strategy to select the optimal action and update the expected reward of taking the action in the current state according to the reinforcement learning model.

[0065] Furthermore, the design strategy for selecting the optimal action includes:

[0066] a * =max a Q(s,a);

[0067] Where s is each state; a is the action chosen by the agent that has the maximum Q value; Q is the expected reward of taking the action in the current state; a * The optimal action to be chosen for the current state s.

[0068] Furthermore, updating the expected reward for taking an action in the current state according to the reinforcement learning model includes:

[0069] Q(s,a)=(1-λ t )[R+γmax a Q(s′,a′)];

[0070] Where Q(s,a) is the total reward expected by taking action a in state s; s is each state; a is the action chosen by the agent that has the maximum Q value; λ t R is the learning rate; R is the immediate reward; γ is the discount factor; max a Q(s′,a′) is the maximum value of Q among all possible actions a′ in the next state s′; Q is the expected reward of taking an action in the current state.

[0071] A reward function is set based on the rate of dust concentration reduction. The faster the dust concentration decreases before reaching the safety threshold, the higher the reward.

[0072] Set reward value:

[0073]

[0074] Where C is the current dust concentration; ΔC is the rate of decrease in dust concentration per unit time, ΔC = C prev -C current W c This is the weighting factor for dust concentration. When the dust concentration reaches the safety threshold C... safe Afterwards, the reward is fixed at zero to prevent over-adjustment.

[0075] Furthermore, training the reinforcement learning model based on the obtained dust distribution data includes:

[0076] The reward value for each data sample is calculated according to the set reward function. The empirical data with the reward value is added to the replay buffer for the early training of the model and to generate a preliminary dust control strategy.

[0077] The reinforcement learning model employs an experience replay mechanism to store experience data with reward values ​​in a replay buffer.

[0078] Calculate the Q-value for each action in the current state, and update the Q-value by minimizing the loss function, where Q is the expected reward for taking the action in the current state;

[0079] Wherein, the loss function is:

[0080] L(θ)=E[(R+γmax a∈A Q(s t+1 q t+1 ;θ′)-Q(s t a t ;θ)) 2 ];

[0081] Where L(θ) is the loss function; E is the expected value; R is the immediate reward; γ is the discount factor; max a∈A Search for all possible actions a in the action space A; s is the action that the agent selects with the maximum Q value; Q is the expected reward of taking the action in the current state; A is the action space (the set of all possible actions); Q(s) t+1 a t+1 ;θ′) represents the state s of the target Q-network for the next time step. t+1 and action a t+1 Q-value estimation; s t+1 To perform action a t The next state after transition; a t+1 For state s t+1 Below, the action selected by the agent; θ′ represents the parameters of the target network; Q(s) t a t ;θ) 2 For the current Q network pair state s t and action a t Q-value estimation; s t The current state; a t For state a t The action to be selected; θ is the parameter of the Q network.

[0082] When the loss function reaches the predetermined convergence condition or the set threshold, training stops, and the trained reinforcement learning model is output for adaptive control of the ventilation system.

[0083] This invention also discloses a training system for an adaptive control model of a fully mechanized tunneling face dust removal system based on reinforcement learning. The system includes a dust generation module, a data acquisition module, a data transmission module, a reinforcement learning module, and a fully mechanized tunneling face control module. The dust generation module consists of a dust generator, connecting pipes, and a dust generation port. It simulates actual working conditions at the fully mechanized tunneling face to generate dust in the area, including different dust concentrations and various dust distributions. The data acquisition module consists of dust sensors, a host computer, and equipment data acquisition software. It collects data from the area and transmits the data to the processing terminal. The reinforcement learning module, based on a DQN model network architecture, receives the current state and uses a reward function and experience replay mechanism for reinforcement learning to achieve adaptive dust control at the fully mechanized tunneling face. The fully mechanized tunneling face control module includes a control program, a host computer, an exhaust fan, a compressor fan, a ventilation duct, and a motor. The fully mechanized tunneling face control module makes real-time decisions based on the current environmental state of the physical model, including adjusting parameters such as exhaust power, compressor power, and vent position to achieve intelligent dust removal control at the fully mechanized tunneling face.

[0084] This invention employs a reinforcement learning control method that dynamically controls key equipment in the dust removal system through reinforcement learning algorithms. This includes adjusting the power of each fan and the distance between the air outlet and the tunnel face. This achieves adaptive regulation of ventilation and dust removal at the underground tunnel face. It can precisely control dust concentration under dynamic and unsteady-state conditions, effectively reducing dust concentration and improving dust removal efficiency. The reinforcement learning training method used in this invention inputs real-time environmental conditions (dust concentration, temperature, humidity, wind speed, etc.) and equipment operating parameters into a reinforcement learning algorithm based on a DQN structure, and trains the DQN model using an experience playback mechanism. The DQN reinforcement learning model outputs the optimal control action for the current real-time environment. The model dynamically adjusts ventilation equipment parameters, optimizes dust control strategies, and effectively reduces dust concentration in the tunnel face area.

[0085] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an adaptive control model of a fully mechanized tunnel face dust removal system based on reinforcement learning, characterized in that, include: A physical model of the tunnel face is constructed, dust is generated at the tunneling points of the physical model, and dust sensors are used to capture the dust concentration at various locations on the tunnel face to obtain dust distribution data. A reinforcement learning model is constructed, and the reinforcement learning model is trained based on the obtained dust distribution data. The trained reinforcement learning model is used for adaptive control of the ventilation system. The construction of the physical model of the tunnel face includes: The physical model of the tunnel face is divided into three surfaces: the tunnel face, the tunnel wall, and the model opening. The tunnel face is sealed to the tunnel wall, and the right side of the model opening is open. The bottom edge length of the tunnel is a preset length. The tunnel face is composed of an upper arch and a lower rectangle. The physical model of the tunnel face includes a motor, a fan, a dust generator, a dust sensor, and a proportionally replaced traditional exhaust ventilation duct. The motor, fan, and dust sensor are selected to support networking or network connection via an interface module. Dust generated at the tunneling points of the physical model of the fully mechanized tunnel face includes: The outlet of the forced-in ventilation duct is installed facing the tunneling face to introduce fresh air into the tunneling area and cover the dust source. The outlet of the exhaust ventilation duct is located on the right side of the model opening end to discharge dust. The extraction and pressure pipes are installed above the platform support and connected to the extraction and pressure fans respectively. The control terminal of the fan is connected to the main unit to control the ventilation equipment. Several holes are drilled in the tunneling face of the physical model of the tunnel face to serve as dust inlets for dust generators. Dust generators are installed at the corresponding dust inlets, and the output ports of the dust generators are connected to pipes, which are then connected to the tunnel face. Dust is transmitted through the holes in the tunnel face, and dust generation is completed in conjunction with the control of ventilation equipment. Dust sensors were used to capture dust concentrations at various locations on the tunnel face, including: Where C represents the overall dust concentration; C i W represents the dust concentration measured by the i-th sensor. i Let be the weight of the i-th sensor; Training the reinforcement learning model based on the obtained dust distribution data includes: The reward value for each data sample is calculated according to the set reward function. The empirical data with the reward value is added to the replay buffer for the early training of the model and to generate a preliminary dust control strategy. The reinforcement learning model employs an experience replay mechanism to store experience data with reward values ​​in a replay buffer. Calculate the Q-value for each action in the current state, and update the Q-value by minimizing the loss function, where Q is the expected reward for taking the action in the current state; When the loss function reaches the predetermined convergence condition or the set threshold, training stops, and the trained reinforcement learning model is output for adaptive control of the ventilation system.

2. The training method for the adaptive control model of the fully mechanized tunnel face dust removal system based on reinforcement learning as described in claim 1, characterized in that, The installation of dust sensors includes: distributing the dust sensors evenly in an array above the support according to the area and characteristics of the tunnel face.

3. The training method for the adaptive control model of the fully mechanized tunnel face dust removal system based on reinforcement learning as described in claim 1, characterized in that, Building a reinforcement learning model includes: an input layer, a hidden layer, and an output layer; The input layer is used to receive the current status, which includes dust concentration, ambient temperature, humidity, wind speed, and power data of each device. The hidden layer is used to set up several fully connected layers; The output layer is used to output the expected reward of the action taken in the current state according to each action of the system. The action includes adjusting the exhaust air volume, the compressed air volume, and the air outlet position.

4. The training method for the adaptive control model of the fully mechanized tunnel face dust removal system based on reinforcement learning as described in claim 1, characterized in that, Before training the reinforcement learning model based on the obtained dust distribution data, it is necessary to design a strategy to select the optimal action and update the expected reward of taking the action in the current state according to the reinforcement learning model.

5. The training method for the adaptive control model of the fully mechanized tunnel face dust removal system based on reinforcement learning as described in claim 4, characterized in that, The design strategy for selecting the optimal action includes: a * =max a Q(s,a); Where s is each state; a is the action chosen by the agent that has the maximum Q value; Q is the expected reward of taking the action in the current state; a * The optimal action to be chosen for the current state s.

6. The training method for the adaptive control model of the fully mechanized tunnel face dust removal system based on reinforcement learning as described in claim 4, characterized in that, The expected reward for taking an action in the current state is updated based on the reinforcement learning model, including: Q(s,a)=(1-λ t ) [R+γmax a Q(s′,a')]; Where Q(s, a) is the total reward expected by taking action a in state s; s is each state; a is the action chosen by the agent that has the maximum Q value; λ t R is the learning rate; R is the immediate reward; γ is the discount factor; max a Q(s′,a′) is the maximum value of Q among all possible actions a′ in the next state s′; Q is the expected reward of taking an action in the current state.

Images