A decision-making method for escaping from a hydraulic support failure on a working face

Abstract

The present invention discloses a method for making decisions on how to get out of a hydraulic support pull-up failure on a working face, comprising the following steps: S1, defining a hydraulic support state space according to the pitch angle of the top beam state of the hydraulic support and the height of the support support state; S2, defining a hydraulic support action space A according to the action of adjusting the attitude of the hydraulic support; S3, setting a reward function; S4, learning the mapping relationship between the hydraulic support state space and the hydraulic support action space A by using a Q-learning algorithm; S5, outputting a hydraulic support attitude control strategy according to the mapping relationship between the hydraulic support state space and the hydraulic support action space A obtained by training in step S4. The present invention can realize automated processing, improve the support safety of the coal mining working face, and reduce the workload of operators, thereby meeting the requirements of intelligent and unmanned coal mines.

Description

Technical Field

[0001] The invention relates to the field of coal mining, and in particular to a method for making decisions on how to escape from a hydraulic support failure on a working face. Background Art

[0002] With the vigorous development of my country's economy, energy consumption has increased year by year, and the demand for coal has also continued to rise. How to mine coal efficiently and safely is an issue that must be taken seriously. Hydraulic support is one of the core equipment for safe support of underground working faces and top coal recovery in coal mines. It is an important equipment to ensure the normal operation of other coal mining equipment and the safety of workers. In the production process, the processes of top coal venting, transportation, and crushing are closely related to the posture of the hydraulic support. The change in the posture of the hydraulic support must adapt to the changes in the geological pressure of the roof and surrounding rock of the coal mining working face to achieve the purpose of safe support. If the posture of the hydraulic support is abnormal and cannot adapt to the changes in the geological conditions of the surrounding environment, it will cause damage to the support components at the least, and even cause the support to become unstable and tip over, causing safety accidents.

[0003] At present, the working faces of coal mines in my country generally monitor the posture of hydraulic supports, so as to timely grasp the working status of hydraulic supports and improve the production efficiency of the working face. However, the existing hydraulic support monitoring system of the working face only monitors the posture of the hydraulic support. When the hydraulic support is in a fault state such as raising or lowering the head, the posture of the hydraulic support cannot be adjusted, and manual operation is required to get it out of trouble, which significantly increases the workload of the operators and cannot meet the requirements of intelligent and unmanned coal mines; at the same time, it cannot be discovered in time, affecting the support safety of the coal mining working face. Summary of the invention

[0004] The object of the present invention is to solve the above-mentioned problems and provide a method for making decisions on the escape of hydraulic support failure in a working face to improve the safety performance of a coal mining working face.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] A method for making decisions on escaping a hydraulic support failure on a working face, comprising the following steps:

[0007] S1. Define the hydraulic support state space s according to the pitch angle θ of the top beam state of the hydraulic support and the support support state height H. The definition formula is:

[0008] s=(θ,H);

[0009] S2. Define the hydraulic support action space A according to the action of adjusting the posture of the hydraulic support, and its definition formula is:

[0010] A={a1,a2,a3};

[0011] Among them, a1 is the action of raising the column; a2 is the action of lowering the column; a3 is the action of stopping the raising or lowering of the column;

[0012] S3. Set the reward function R, whose calculation formula is:

[0013] R=-[(H-H0) 2 +(θ-θ0) 2 ];

[0014] Among them, H0 is the target support height of the bracket; θ0 is the target pitch angle of the top beam;

[0015] S4, using Q-learning algorithm to learn the mapping relationship between the hydraulic support state space s and the hydraulic support action space A;

[0016] S5. Output the hydraulic support posture control strategy according to the mapping relationship between the hydraulic support state space s and the hydraulic support action space A obtained by training in step S4. The formula is:

[0017]

[0018] Furthermore, in the step S1, the hydraulic support state space s is discretized, and is combined according to the pitch angle of the top beam state and the height discrete level of the support support state to form 25 hydraulic support learning states.

[0019] Furthermore, the step S4 specifically includes the following steps:

[0020] S41, initialize Q(s,a), Given parameters α, γ, ε;

[0021] S42, initializing the hydraulic support learning state s0, selecting the hydraulic support to perform action a under the hydraulic support learning state s0 according to the ε-greedy strategy, the hydraulic support learning state is transferred to s`, and a reward r is obtained at the same time;

[0022] S43, incrementally update the action value function, and the update formula is:

[0023]

[0024] S44, updating the hydraulic support learning state until s=s`, and ending this training;

[0025] S45, initialize another hydraulic support learning state and start the next round of training; when the action value function Q(s,a) is fully converged, the training is completed.

[0026] Compared with the prior art, the present invention has the following advantages and positive effects:

[0027] The present invention performs reinforcement learning on the mapping relationship between the posture and action of the hydraulic support based on the Q-learning algorithm. It accurately determines whether the posture of the hydraulic support is abnormal by fusing information such as the pitch angle of the top beam state of the hydraulic support and the height of the support support state, and then adjusts the posture of the hydraulic support in real time, thereby ensuring that the hydraulic support of the coal mining face is in a normal working state, so that the hydraulic support can operate safely and reliably; it can realize automated processing, improve the support safety of the coal mining face, and at the same time reduce the workload of operators, meeting the requirements of intelligent and unmanned coal mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0029] Figure 1 This is a schematic diagram of an abnormal head-up posture of a hydraulic support;

[0030] Figure 2 This is a schematic diagram of the abnormal head-down posture of the hydraulic support;

[0031] Figure 3 This is the posture model diagram of the hydraulic support. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work, any modifications, equivalent substitutions, improvements, etc., should be included in the protection scope of the present invention.

[0033] The present invention discloses a method for making decisions on how to get out of a hydraulic support rack failure on a working surface, and the content thereof is analyzed as follows:

[0034] 1. Abnormal posture of hydraulic support such as Figure 1 , Figure 2 As shown;

[0035] 2. The process and purpose of hydraulic support posture adjustment;

[0036] When the hydraulic support is in an abnormal posture such as raising or lowering the head, it is necessary to raise and lower the columns in time to adjust the hydraulic support posture. If it is not handled in time, it may cause damage to the support components at the least, or even cause the support to become unstable and fall over, causing a safety accident.

[0037] 3. Hydraulic support posture model and parameters such as Figure 3 As shown, Figure 3 In the figure, θ is the pitch angle of the top beam; H is the support height of the hydraulic support;

[0038] 4. Modeling of Markov decision process for hydraulic support attitude control;

[0039] The problem of adjusting the hydraulic support posture is converted into the problem of learning the mapping relationship between the hydraulic support posture and the support movement. Once the mapping relationship between the two is obtained, the automatic adjustment of the support posture can be achieved.

[0040] (1) State space S

[0041] The state s is defined according to the pitch angle θ of the top beam of the hydraulic support and the support height H of the support.

[0042] s=(θ,H) (1)

[0043] Since the top beam pitch angle θ and the support height of the bracket change continuously, discretization processing is required, as shown in Tables 1 and 2.

[0044] Table 1 Discrete interval division of top beam pitch angle θ

[0045]

[0046] Table 2 Division of support height H interval

[0047]

[0048] According to the discrete levels of the top beam pitch angle θ and the bracket support height H, a total of 25 reinforcement learning states can be formed.

[0049] (2) Action Space A

[0050] The actions of adjusting the posture of the hydraulic support on the underground working surface are mainly the actions of raising and lowering the column. Therefore, the reinforcement learning action space is defined as:

[0051] A={a1,a2,a3} (2)

[0052] In the formula, a1 is the action of raising the column; a2 is the action of lowering the column; a3 is the action of stopping the raising or lowering of the column.

[0053] (3) Reward function R

[0054] It is hoped that the hydraulic support is in a reasonable posture on the working surface. Therefore, when the support posture is abnormal, it needs to be adjusted to a normal state. Therefore, the design reward function R is:

[0055] R=-[(H-H0) 2 +(θ-θ0) 2 ] (3)

[0056] Where H0 is the target support height of the bracket; θ0 is the target pitch angle of the top beam.

[0057] After the Markov decision process is modeled, the mapping relationship between the hydraulic support state and action is learned.

[0058] 5. Solution method;

[0059] The Q-learning algorithm is used to learn the mapping relationship between the hydraulic support state and action. The process is as follows;

[0060] (1) Initialize Q(s,a), Given parameters α, γ, ε.

[0061] (2) Repeat the following steps:

[0062] ① Initialize the hydraulic support state s0, select the support to perform action a in state s0 according to the ε-greedy strategy, and the hydraulic support state is transferred to s`, and the reward r is obtained at the same time;

[0063] ②Incremental update of the action value function:

[0064]

[0065] ③Update status:

[0066] s=s` (5)

[0067] Until s is in the terminal state, end this training and start the next round of training;

[0068] When all action-value functions Q(s,a) in the Q table converge, that is, when all 25 reinforcement learning states are trained, the training is completed.

[0069] (4) Output hydraulic support attitude control strategy:

[0070]

[0071] The present invention performs reinforcement learning on the mapping relationship between the posture and action of the hydraulic support based on the Q-learning algorithm. It accurately determines whether the posture of the hydraulic support is abnormal by fusing information such as the pitch angle of the top beam state of the hydraulic support and the height of the support support state, and then adjusts the posture of the hydraulic support in real time, thereby ensuring that the hydraulic support of the coal mining face is in a normal working state, so that the hydraulic support can operate safely and reliably; it can realize automated processing, improve the support safety of the coal mining face, and at the same time reduce the workload of operators, meeting the requirements of intelligent and unmanned coal mines.

Claims

1. A method for making decisions on how to get out of a hydraulic support failure on a working face, characterized by: The following steps are involved: S1, according to the pitch angle of the top beam of the hydraulic support Height of support status Define the hydraulic support state space , whose definition formula is: ； S2. Define the hydraulic support action space A according to the action of adjusting the posture of the hydraulic support, and its definition formula is: ； in, It is the lifting column action; It is the column lowering action; To stop the action of raising or lowering the column; S3. Set reward function , and its calculation formula is: ； in, is the target support height of the bracket; is the target pitch angle of the top beam; S4. Utilization Algorithm learning hydraulic support state space The mapping relationship between the hydraulic support action space A; S5, the hydraulic support state space obtained by training in step S4 The mapping relationship between the hydraulic support action space A outputs the hydraulic support attitude control strategy, and its formula is: ； The step S4 specifically comprises the following steps: S41, Initialization , , , given parameters ; S42, Initialize the hydraulic support learning state ,according to Strategy in hydraulic support learning state Select the hydraulic support to perform the action , the hydraulic support learning state transfer is , and get rewards ; S43, incrementally update the action value function, and the update formula is: ； S44, update the hydraulic support learning status until , end this training; S45, initialize another hydraulic support learning state and start the next round of training; when the action value function After all convergence, the training is completed.

2. The method for making decisions on escaping a hydraulic support failure on a working face according to claim 1, characterized in that: In step S1, the hydraulic support state space Discretization processing is performed, and 25 hydraulic support learning states are formed according to the combination of the pitch angle of the top beam state and the highly discrete levels of the support state.

Images