Intelligent management and control system for coal mine production safety based on reinforcement learning model

Abstract

This invention relates to the field of intelligent control technology for coal mine production safety, and in particular to an intelligent control system for coal mine production safety based on a reinforcement learning model. Addressing the problems of easy boundary crossing and the inability to adapt static safety thresholds in existing reinforcement learning methods, this system constructs a state vector through a multi-source perception module; outputs a risk index through a safety assessment module; generates nominal actions through a decision-making module; projects the safety barrier function into actual safety actions based on the control barrier function and calculates the implicit function partial derivatives to fuse gradient backpropagation; and dynamically adjusts boundary parameters by combining working conditions and procedures. This invention achieves deep synergy between physical-level safety hard constraints and intelligent decision optimization.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology for coal mine production safety, and in particular to an intelligent control system for coal mine production safety based on a reinforcement learning model. Background Technology

[0002] With the increasing prevalence of deep coal mining, the threats posed by various natural disasters such as gas, roof collapse, and water hazards underground are intensifying, making the demand for intelligent control systems in the coal industry more urgent. Machine learning technologies, represented by reinforcement learning and multi-agent reinforcement learning, are gradually being introduced into the collaborative control of multiple business subsystems in coal mines, such as ventilation, transportation, and mining, thanks to their sequential decision-making and multi-parameter global optimization capabilities. However, reinforcement learning algorithms rely on trial-and-error mechanisms in the interaction between the agent and the environment to maximize long-term cumulative rewards, inevitably leading to actions that exceed safety limits during the exploration phase. In safety-critical scenarios like coal mines, such as gas emission control, variable frequency fan pressure regulation, and collision avoidance for unmanned locomotives, any single action that crosses the safety red line could trigger irreversible accidents such as explosions or equipment damage. Existing safety reinforcement learning methods mostly employ the constrained Markov decision process framework, using expected negative penalties or Lagrange multiplier methods to suppress unsafe behaviors. However, this soft constraint based on expected value assessment can only reduce the probability of accidents in a statistical sense, failing to provide an absolute barrier of zero violations on the physical control chain. Furthermore, when faced with sudden extreme working conditions outside the distribution, the model is prone to generating illegal outputs. Meanwhile, the safe operation of coal mine systems is not only limited by the physical limits of equipment but also by the combined constraints of safety regulations, operating procedures, and dynamic changes in the geological environment. Existing multi-agent cooperative scheduling algorithms typically preset static safety thresholds, operating the safety assessment module and the underlying control algorithm in isolation. This fails to dynamically transform unstructured safety regulations into decision boundary parameters for reinforcement learning. When geological conditions change abruptly or regulatory requirements are updated, the static boundary is difficult to adaptively adjust. These two problems are mutually restrictive: improving the overall efficiency of the system requires reinforcement learning to explore a broader policy space, while ensuring absolute safety necessitates strictly converging the behavioral boundaries of the agents. Existing technologies lack a deeply collaborative fusion architecture between physical-level safety hard constraints and intelligent decision optimization. Summary of the Invention

[0003] The purpose of this invention is to provide an intelligent control system for coal mine production safety based on a reinforcement learning model, in order to solve the technical problems in existing intelligent control systems for coal mines where reinforcement learning algorithms are prone to overstepping boundaries during the exploration phase, leading to physical hazards, and where static safety boundaries are difficult to adapt to dynamic changes in geological conditions and regulatory requirements. This invention aims to establish deep synergy between physical-level safety constraints and intelligent decision-making for global optimization.

[0004] To achieve the above objectives, the present invention provides an intelligent control system for coal mine production safety based on a reinforcement learning model, comprising: The multi-source perception and state construction module is used to collect the original physical state vectors of various business subsystems in the coal mine, and to perform time-series encoding on the local observation sequence to construct the local observation state vector and global state vector of the corresponding intelligent agent. The safety risk dynamic assessment module is used to receive the original physical state vector, calculate and output the intrinsic safety risk index that represents the safety status of the entire mine in real time; The multi-agent collaborative decision-making module configures a policy network and a centralized value network for each business subsystem. The policy network outputs the nominal action of the corresponding agent based on the local observation state vector. The safety barrier and action projection module receives the original physical state vector and the nominal action, constructs a quadratic programming problem based on the control barrier function containing the system's continuous-time physical dynamics equations, and projects the nominal action into an actual execution action that satisfies the forward invariance condition within the control time step, which is then sent to the physical actuator. It also calculates the implicit function partial derivative of the KKT conditions at the optimal solution of the quadratic programming problem to obtain the Jacobian matrix of the actual execution action with respect to the nominal action. During the backpropagation phase of the policy network training, the multi-agent collaborative decision-making module uses the Jacobian matrix to fuse the value network gradient to update the policy network parameters. This module provides an absolute barrier with zero violations on the control physical chain, while avoiding truncating the deep learning backpropagation link, thus accelerating the model's convergence to the safe and feasible region. The large model boundary parameter generation module is triggered when the intrinsic safety risk index changes abnormally. It combines real-time operating data to infer and output structured parameter adjustment instructions, and updates the parameter register of the control barrier function in the safety barrier and action projection module after parsing the instructions, so as to dynamically adjust the safety boundary parameters. This module effectively realizes the seamless connection between unstructured safety supervision text and underlying mathematical control logic.

[0005] As a preferred embodiment of the present invention, the multi-source perception and state construction module uses a gated loop unit to perform time-series encoding on historical observation data, compressing the local observation sequence into a feature vector; the multi-source perception and state construction module adopts a dual-channel data distribution mechanism: the global state vector and the local observation state vector after time-series encoding by the gated loop unit are distributed to the multi-agent collaborative decision-making module; the original physical state vector without time-series encoding is simultaneously distributed to the security risk dynamic assessment module, the security barrier and action projection module, and the large model boundary parameter generation module.

[0006] As a preferred embodiment of the present invention, the dynamic safety risk assessment module adopts an architecture that integrates a fuzzy logic system and a BP neural network. In the fuzzification stage, a Gaussian membership function is configured for the input variables to map the original physical state into a membership vector representing the semantic level of the risk, effectively eliminating system oscillations caused by repeated triggering of critical intervals. In the neural network inference stage, the membership vector is input into the BP neural network, and the output range is within a certain range through forward propagation. The intrinsic safety risk index between them.

[0007] As a preferred embodiment of the present invention, in the safety barrier and action projection module, the physical state evolution of each business subsystem is assumed to satisfy an affine dynamics model. ,in This is the original physical state vector. To control the input, and These are the drift term and the control term, respectively; the Lie derivative constraint condition for constructing the control barrier function is: ;

[0008] in, A scalar function to characterize the physical security boundary, and They are respectively along and Lie derivative of direction, For parameterized extended K-class functions, For reference working point, The dynamic boundary adjustment coefficient is jointly determined by the intrinsic safety risk index and the large model boundary parameter generation module. This is a set of adjustable parameters.

[0009] As a preferred embodiment of the present invention, the objective function and constraints of the quadratic programming problem are respectively expressed as follows: ;

[0010] The constraints include: ; ;

[0011] in, The nominal action output by the policy network. Slack variables are introduced to prevent the solver from having no solution under extremely poor conditions. Penalize weights for slack variables. and These represent the minimum and maximum output limits of the corresponding subsystem's physical actuator, respectively.

[0012] As a preferred embodiment of the present invention, the centralized value network in the multi-agent collaborative decision-making module receives the global state vector and the actual actions of all agents after correction by the security barrier and action projection module as input, evaluates the action value of the current joint strategy, and ensures causal consistency; the immediate reward function of the multi-agent collaborative decision-making module Defined as: ;

[0013] in, To reward positive production efficiency, Based on actual actions The calculated device energy consumption penalty component, Based on the intrinsic security risk index With static risk fundamental constant The security risk penalties constituted together , , These are the weighting coefficients for each reward component.

[0014] As a preferred embodiment of the present invention, during the policy iteration process, the parameters of the policy network are... The formula for updating gradient calculation includes: ;

[0015] in, For centralized value networks in actual execution of actions The action value gradient at the location, The Jacobian matrix obtained by differentiating the implicit function. This outputs the gradients of the policy network with respect to its parameters.

[0016] As a preferred technical solution of the present invention, the large language model in the large model boundary parameter generation module is fine-tuned offline using low-rank adaptive technology, the pre-trained backbone weight matrix is ​​frozen and a trainable low-rank dimensionality reduction matrix and dimensionality increase matrix are injected in the side path; after the low-rank adaptive fine-tuning is completed, the group sequence strategy optimization algorithm is further used to perform alignment training on the large language model to ensure that the instruction output meets the expectations of industrial experts.

[0017] As a preferred embodiment of the present invention, the large model boundary parameter generation module, during the online inference phase, converts the current structured chemical condition data into a natural language description, and combines it with retrieval enhancement generation technology to retrieve relevant procedural provisions and handling cases from the safety knowledge base, assembling them into prompt words that are input into the fine-tuned large language model; after comprehensive inference, the large language model outputs a structured parameter adjustment instruction in the form of key-value pairs, which includes target subsystem identifier, adjustment parameter type, adjustment direction, and adjustment magnitude fields; the system converts the adjustment magnitude into a specific numerical change in the structural parameters of the control barrier function through a parameter mapping rule table and writes it into the parameter register.

[0018] As a preferred embodiment of the present invention, each of the business subsystems includes at least a locomotive anti-collision transportation subsystem and an intelligent ventilation control subsystem; wherein, for the locomotive anti-collision transportation subsystem, the original physical state vector includes the vehicle's speed, acceleration, following distance to the preceding vehicle, the preceding vehicle's speed, and heading angle; for the intelligent ventilation control subsystem, the original physical state vector includes the roadway wind speed, gas concentration, fan power, motor temperature, and pressure difference.

[0019] Compared with the prior art, the present invention has the following beneficial effects: This invention introduces a differentiable controllable barrier function layer based on quadratic programming between the reinforcement learning policy network and the physical actuator. This layer uses Lie derivative constraints to formalize the dynamic safety boundary of each subsystem into a quadratic programming problem. In each control time step, the nominal action output by the policy network is projected in real time to a subset of safe actions that satisfy the forward invariance condition, so that the system state always remains within the safety set boundary, avoiding the direct action of out-of-bounds actions caused by algorithm exploration to the physical actuator. At the same time, by implicitly differentiating the KKT conditions at the optimal solution of the quadratic programming, the Jacobian matrix of the safety-corrected action to the nominal action is obtained, so that the action deviation error generated by the safety correction layer can be backpropagated to the upstream policy network through the chain rule. During the training process, the policy network can perceive the constraint strength of the physical safety boundary through the gradient signal, thereby autonomously converging the policy distribution into the safe feasible region, reducing invalid exploration actions and improving the training convergence speed. This invention also introduces a large language model for coal mine safety, trained with low-rank adaptive fine-tuning and group sequence strategy optimization alignment. When the intrinsic safety risk index changes significantly or a sudden abnormal event occurs, this large language model combines retrieval enhancement generation technology to perform comprehensive reasoning on real-time operating data and safety regulations, outputting structured parameter adjustment instructions. After mapping and parsing, these instructions are directly written into the parameter register of the control barrier function, realizing a dynamic mapping between safety supervision text and underlying mathematical control parameters. This allows the decay characteristics and boundary definition of the control barrier function to be adaptively adjusted according to changes in geological conditions, abnormal equipment status, or regulatory requirements. When system risk increases, the safe operation envelope is tightened, and when conditions are stable, it is appropriately relaxed to improve production efficiency. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the overall system architecture according to an embodiment of the present invention.

[0021] Figure labeling: 101, Multi-source perception and state construction module; 102, Dynamic assessment of security risks module; 103, Security barrier and action projection module; 104, Multi-agent collaborative decision-making module; 105, Large model boundary parameter generation module. Detailed Implementation

[0022] like Figure 1As shown, the intelligent control system for coal mine production safety based on a reinforcement learning model provided by this invention is deployed using the existing 10-gigabit industrial ring network and underground wireless communication network in the coal mine. The main control software logic runs on the edge computing server cluster in the ground control center, and completes data interaction and control command issuance with various underground business subsystems through industrial communication protocols. The overall system is divided into five functional modules: multi-source perception and state construction module 101, dynamic safety risk assessment module 102, safety barrier and action projection module 103, multi-agent collaborative decision-making module 104, and large model boundary parameter generation module 105. The above five modules form a deeply coupled closed-loop structure at the data flow and control flow levels, and the modules communicate with each other at high frequency through an internal signal bus.

[0023] The multi-source sensing and state construction module 101 constitutes the data foundation of the entire control system, undertaking the tasks of sensing data acquisition, cleaning, time-series alignment, and state space vectorization for the entire coal mine business chain. In the underground coal mine production scenario, the sensor types and physical dimensions of various business subsystems differ significantly. The multi-source sensing and state construction module 101 needs to uniformly process heterogeneous time-series data from different physical domains, transforming them into high-dimensional continuous state space vectors required by Markov decision processes. For the intelligent ventilation control subsystem, the multi-source sensing and state construction module 101 acquires real-time data from the laser methane sensor, anemometer, and inverter operating status interface. The constructed local observation state vector includes five dimensions of continuous physical quantities: roadway wind speed, gas concentration, fan power, motor temperature, and pressure difference. For the heavy medium coal washing and separation subsystem, the multi-source sensing and state construction module 101 collects six dimensions of state quantities, including hydrocyclone pressure, magnetic content, raw coal feed rate, raw coal ash content, suspension density, and clean coal ash content deviation, through pressure transmitters, densitometers, and the feeder weighing bridge. For the locomotive anti-collision transportation subsystem, the multi-source sensing and state construction module 101 integrates data from millimeter-wave radar, ultra-wideband positioning, and wheel speed sensors to construct a local observation vector. Each component corresponds to the vehicle's speed, acceleration, following distance to the preceding vehicle, preceding vehicle speed, and heading angle, respectively. For the collaborative mining subsystem at the working face, the multi-source sensing and state construction module 101 reads signals from the support pressure sensor and the coal mining machine encoder, constructing a local observation vector as follows: Each component represents the working resistance of the support, the traction speed of the coal mining machine, the running speed of the conveyor, and the mining height.

[0024] Due to the complex communication environment and limited channel capacity in underground coal mines, agents often only acquire local observation information, leading to the problem of partially observable Markov decision processes. To address this issue, the multi-source sensing and state construction module 101 introduces a gated loop unit to process historical data. The observation data at each time step undergoes temporal encoding. A gating mechanism is used to selectively retain and forget information from different time steps, thereby compressing the local observation sequence into a feature vector with temporal memory capabilities. The update gate of the gating loop unit... With Reset Door The mixing ratio of historical information in the current hidden state with the current input is controlled separately, and its update process can be described as follows: ; ; ;

[0025] in The raw input observations for the current time step. Let this be the hidden state vector from the previous time step. It is the Sigmoid activation function. This represents element-wise multiplication. , , and , , For trainable weight matrix, , , This is the bias term. After being encoded by the gated recurrent unit, the local observations of each agent are expanded into feature vectors containing temporal context information. Let the total number of agents running simultaneously in the system be . The multi-source sensing and state construction module 101 will completely The encoded local observations of each agent are concatenated to form a global state vector. The multi-source sensing and state construction module 101 constructs the encoded global state vector. At the same time, the original physical state vector of each agent at the current moment is preserved. This refers to the raw sensor data acquired without time-series encoding by the gated cyclic unit. The data distribution from the multi-source sensing and state construction module 101 to subsequent modules follows this path: global state vector encoded by the gated cyclic unit. Provided for use by the multi-agent collaborative decision-making module 104, providing its policy network and value network with decision-making basis including temporal memory information; the original physical state vectors of each subsystem Simultaneously, the data is distributed to the dynamic safety risk assessment module 102, the safety barrier and action projection module 103, and the large model boundary parameter generation module 105, providing them with raw data inputs with clear physical dimensions. This dual-channel data distribution mechanism is designed because: the multi-agent collaborative decision-making module 104 needs time-encoded feature vectors to obtain historical state trend information to support long-term strategy optimization; the fuzzification processing of the dynamic safety risk assessment module 102 requires direct manipulation of raw numerical quantities with independent physical semantics (such as gas concentration percentage, tunnel wind speed in meters per second, etc.); and the Lie derivative calculation of the safety barrier and action projection module 103 also requires inputs that are state variables directly corresponding to the physical dynamics model. In the data preprocessing stage, the multi-source sensing and state construction module 101 also performs outlier filtering and time delay alignment operations. Outlier filtering uses a sliding window-based statistical detection method to remove and interpolate instantaneous jumps, packet loss, and noise interference in the sensor data. Delay alignment involves uniformly resampling sensor data at different sampling frequencies to ensure that the timestamps of each channel are strictly aligned on the time scale of the same control cycle, thus eliminating data timing deviations caused by inconsistent transmission links.

[0026] The security risk dynamic assessment module 102 receives the original physical state vectors of each subsystem distributed by the multi-source sensing and state construction module 101. Subsequently, using the four dimensions of people, machines, environment, and management in systems engineering as an evaluation framework, the intrinsic safety risk index, which characterizes the overall safety situation of the mine, is output in real time. Its value range is The closer the value is to 1, the closer the system is to a catastrophic state. Coal mine safety evaluation involves numerous factors with ambiguous boundaries and difficult to quantify precisely. The personnel dimension encompasses soft indicators such as worker fatigue, certification status, and violation records; the equipment dimension includes factors such as deviations in the operating conditions of key equipment, maintenance cycle margins, and spare parts availability; the environmental dimension includes physical parameters such as gas concentration, dust content, temperature, humidity, ground pressure changes, and water inflow; and the management dimension covers the implementation of safety management systems, the completeness of emergency plans, and organizational coordination and response efficiency. Some indicators within these dimensions are inherently ambiguous and difficult to precisely classify using a single threshold. Therefore, the dynamic safety risk assessment module 102 adopts a technical approach that integrates fuzzy logic systems and BP neural networks.

[0027] The dynamic safety risk assessment module 102's processing is divided into three stages: fuzzification, neural network inference, and output. In the fuzzification stage, for each vector derived from the original physical state... Input variables in Five fuzzy sets are defined to correspond to the risk attribution of the variable at different semantic levels. For example, for gas concentration, five levels can be set: "extremely low," "low," "medium," "high," and "extremely high." Each fuzzy set is configured with a Gaussian membership function. ;

[0028] in The center value of this fuzzy set. The parameters for controlling the width of the membership function are as follows. The advantage of using a Gaussian membership function lies in its smooth transition characteristics, effectively eliminating the system oscillation problem caused by repeated triggering and cancellation of traditional single-threshold alarms in the critical interval. After fuzzification, each original input variable is mapped to a set of membership vectors reflecting its risk semantic affiliation.

[0029] The fuzzy membership vectors are integrated and input into a multi-layer feedforward backpropagation (BP) neural network. This BP neural network architecture consists of an input layer, two hidden layers, and a single-node output layer. The number of neurons in the input layer matches the total dimension of the fuzzy membership vectors. Each of the two hidden layers has an appropriate number of neurons, and a sigmoid activation function is used for non-linear mapping between layers. The single-node output of the output layer is the intrinsic security risk index. The offline training of this neural network utilizes a historical coal mine accident database and a set of safety operation feature samples as supervisory signals, adjusting the network weights through an error backpropagation algorithm. Since the output layer has a single-node structure, the error function for a single training sample is defined as: ;

[0030] in This represents the expected value of the annotation corresponding to the training sample. This represents the actual output value of the network. In batch training, the total loss is the mean of the errors of all training samples. Weight updates follow the gradient descent rule: ;

[0031] in For learning rate, To connect the first The first neuron and the second The weights of each neuron. Through this training mechanism, the BP neural network, while retaining the semantic reasoning ability of the fuzzy logic system for fuzzy boundary concepts, endows the entire evaluation system with data-driven adaptive learning characteristics. The intrinsic security risk index is calculated in real time by the security risk dynamic assessment module 102. As a global signal, it is sent to the safety barrier and action projection module 103 and the multi-agent collaborative decision-making module 104. At the same time, the changing trend of this index also provides a key basis for the triggering judgment of the large model boundary parameter generation module 105.

[0032] The safety barrier and action projection module 103 is the core module that distinguishes this invention from existing reinforcement learning applications. The safety barrier and action projection module 103 does not participate in the long-term cumulative reward optimization process; its function is limited to each control time step. Internally, the nominal action output by the multi-agent collaborative decision-making module 104. Perform physical-level hard safety filtering and output the modified final safety action. To drive the physical actuator.

[0033] The algorithm for the safety barrier and motion projection module 103 is built upon a control affine dynamics model. Let the coal mine... The physical state vectors of the subsystems are Its state evolution satisfies the following continuous-time differential equation: ;

[0034] in This is the physical state vector of the subsystem, and its components directly correspond to the raw sensor physical quantities collected by the multi-source sensing and state construction module 101. For the corresponding control input, and These represent the drift term and control term of the subsystem, respectively. Taking the locomotive collision avoidance and transportation subsystem as an example, its physical state vector... The components include raw physical quantities such as the vehicle's speed, acceleration, following distance to the vehicle in front, speed of the vehicle in front, and heading angle, which are control inputs. This corresponds to traction or braking force commands. Taking the intelligent ventilation control subsystem as an example, its physical state vector... The components include raw physical quantities such as tunnel wind speed, gas concentration, and fan power, which are control inputs. This corresponds to the frequency setting command of the wind turbine inverter. It should be particularly noted that when the safety barrier and action projection module 103 performs Lie derivative calculations and safety constraint verification, the input used is the original physical state vector of each subsystem. The high-dimensional feature vector, rather than the one encoded by the gated recurrent unit. or global concatenation vector This is because the mathematical foundation of the control barrier function is the continuous-time physical dynamics equation, and the calculation of the Lie derivative requires that the state variables have a well-defined physical dynamics model. and However, the feature vector encoded by the neural network does not possess this kind of physical differentiability.

[0035] Define a continuously differentiable scalar function, but the eigenvector encoded by the neural network does not possess this physical differentiability.

[0036] Define a continuously differentiable scalar function Used to depict the first The physical security boundary and security set of each subsystem Defined as making A set of valid states. In practical applications, The structure varies depending on the object being controlled. Taking locomotive collision avoidance control as an example, It can be set as the actual distance between the vehicle and the vehicle in front minus the minimum safe braking distance required under the current speed difference; taking ventilation control as an example, It can be set to the value obtained by correcting the difference between the upper limit of the safe gas concentration and the current measured concentration with an appropriate safety margin. This ensures that the system state remains within the safe set during operation. Internally, the forward invariance condition must be satisfied. The safety barrier and motion projection module 103 introduces extended K-class functions. The function is a strictly monotonically increasing function and satisfies Based on this, regarding the first The subsystem is constructed with the following constraints based on Lie derivatives: ;

[0037] in for along Lie derivative of direction, for along Lie derivative of direction, This is a parameterized extended K-class function. The input parameters of this function include... For the current moment, the first The safety function value of each subsystem This is the reference operating point for the subsystem in the current control cycle, used to characterize the equilibrium state of the system under nominal operating conditions. This is the dynamic boundary adjustment coefficient, the value of which is derived from the intrinsic safety risk index output by the dynamic safety risk assessment module 102. Together with the control parameters injected by the large model boundary parameter generation module 105, these parameters are used to adaptively adjust the decay characteristics of the control barrier function based on the real-time safety situation. constitute The adjustable parameter set can be dynamically updated by the large model boundary parameter generation module 105 according to real-time operating conditions. The physical meaning of this constraint is that the control input at any given time... Safety functions must be guaranteed. The time derivative satisfies the lower bound constraint, thus ensuring that the system state does not escape the safe set. The boundary.

[0038] Based on these constraints, the safety barrier and action projection module 103 independently constructs a quadratic programming problem for each agent. Taking the first... Taking an intelligent agent as an example, its quadratic programming problem is expressed as: ;

[0039] The constraints are: ; ;

[0040] The objective function of this quadratic programming problem consists of two parts. The first part... Ensured the output action after safety correction The nominal action given by the reinforcement learning policy network The deviation between them should be as small as possible to maximize the preservation of the reinforcement learning's optimization intent. Part Two In As slack variables, To assign a penalty weight with a very large value, in this embodiment it can be set to The magnitude. The purpose of introducing slack variables is to ensure that the quadratic programming problem still has a feasible solution under extreme conditions, avoiding system crashes caused by the solver becoming unsolvable due to constraint conflicts. In inequality constraints... and Corresponding to the first The minimum and maximum output limits of the physical actuators of each subsystem. When there are coupled safety constraints across subsystems in the system, such as the collision avoidance constraints between adjacent vehicles in the operation of multiple locomotives, which involve the control inputs of two locomotives at the same time, the safety barrier and action projection module 103 can merge the control inputs of multiple related agents into joint decision variables and construct a joint quadratic programming problem to simultaneously satisfy multiple coupled safety constraints.

[0041] The key feature of the safety barrier and action projection module 103 in its architectural design lies in its differentiability. Standard quadratic programming solvers are numerical optimization processes, and their outputs are not differentiable. Directly embedding them into the deep learning computation graph would truncate the backpropagation path. To address this issue, the safety barrier and action projection module 103 utilizes KKT conditions to find implicit partial derivatives of the aforementioned convex quadratic programming problem. Specifically, at the optimal solution of the quadratic programming problem, the KKT conditions provide a set of implicit equations containing the original variables, dual variables, and complementary relaxation conditions. By performing total differential and rearranging this set of equations, the optimal action after safety correction can be obtained. Input nominal action Jacobian matrix Using this Jacobian matrix, the action bias error generated by the safety correction layer can penetrate the safety barrier and action projection module 103 through the chain rule and continue to propagate backward to the upstream policy network. The engineering significance of this mechanism is that the reinforcement learning agent can perceive the constraint strength of the physical safety boundary through gradient signals in the early stage of training, thereby autonomously converging the policy distribution towards the safe and feasible region, significantly reducing ineffective or dangerous exploration actions.

[0042] Within the safety boundary constraints provided by the safety barrier and action projection module 103, the multi-agent collaborative decision-making module 104 performs optimal policy search for multi-equipment collaboration across the entire mine, aiming to maximize production efficiency and minimize system energy consumption. The multi-agent collaborative decision-making module 104 adopts a centralized training and distributed execution architecture. For specific algorithm selection, either the multi-agent deep deterministic policy gradient algorithm or the multi-agent near-end policy optimization algorithm can be used. Taking the multi-agent deep deterministic policy gradient algorithm as an example, an independent policy network is assigned to each control subsystem in the coal mine. The strategy network is based on local observations encoded by the corresponding agent through a gated recurrent unit. Output nominal action Deploy a centralized value network at the ground control center. The value network receives the global state vector. and all actions actually performed by the intelligent agents after being corrected by the safety barrier and action projection module 103. As input, the overall action value under the current joint strategy is evaluated. The value network is based on the actual actions performed. Non-nominal action As input, this is because the actual actions of the system interacting with the physical environment are modified by the security barrier. Value networks must evaluate the state transitions and cumulative benefits resulting from actions actually applied to the environment to ensure consistency between state transition probabilities, reward signals, and causal relationships between actions during Markov decision-making.

[0043] The reward function design of the multi-agent collaborative decision-making module 104 incorporates the intrinsic security risk index output by the security risk dynamic assessment module 102. The composition of the included reward signal is specifically defined as follows: ;

[0044] in This is a positive reward component that is correlated with current production output, sorting accuracy, and other production efficiency indicators. Based on actual actions The calculated device energy consumption penalty component, Based on the intrinsic safety risk index With static risk fundamental constant The security risk penalties constituted together , , The weighting coefficients for each reward component can be determined based on the specific production conditions and safety requirements of the mine. Energy consumption penalties are based on actual actions performed. Non-nominal action The reason for this calculation is that the actual energy consumption of the system is determined by the actual control output of the physical actuator. After the nominal action is corrected by the safety barrier and action projection module 103, the power consumption of the actual equipment operation may differ significantly from the nominal command. Calculating the energy consumption based on the corrected actual action ensures the causal consistency between the reward signal and physical reality, avoiding value assessment bias that could lead to abnormal convergence of the policy network. The design of this reward function allows the reinforcement learning policy to pursue long-term economic benefits while mitigating the impact of the system state approaching the safety boundary. As the value approaches 1, the value of the safety penalty term increases dramatically, thereby guiding the policy network of each agent to actively avoid high-risk operating ranges at the value function level.

[0045] During the strategy iteration process During the policy iteration process, the multi-agent collaborative decision-making module 104 maintains an experience replay buffer, which stores interaction experience tuples. The nominal motion was recorded simultaneously. The actual executed action after correction by the safety barrier and motion projection module 103 During the reverse update process of the policy network, the gradient update of the policy network not only includes the Q-value gradient from the value network, but also incorporates the partial derivative signals returned by the safety barrier and action projection module 103 through implicit function differentiation. Policy network parameters The update gradient can be expressed as: ;

[0046] The gradient expression follows a strict chain rule: value network The input includes the actual action to be performed. Therefore, the gradient of Q-value is in The result is obtained at the given location; then multiplied by the implicit function Jacobian matrix output by the safety barrier and motion projection module 103. The gradient is mapped from the safe action space back to the nominal action space; finally, it is multiplied by the gradient of the policy network output with respect to its parameters. This completes the entire gradient propagation chain from value assessment to policy parameters. This gradient signal enables the policy network to simultaneously perceive the reward orientation from value assessment and the constraint orientation from security barriers during the optimization process, achieving a deep integration of policy optimization and physical security assurance in the gradient propagation dimension. As training iterations continue, the policy distribution of each agent gradually converges to an efficient operating range that satisfies security constraints.

[0047] The large model boundary parameter generation module 105 controls the obstacle function parameters in the safety barrier and motion projection module 103. To address the technical bottleneck of adapting to dynamic environmental changes, a domain-fine-tuned large language model is introduced as a high-level cognitive decision-making center, enabling cross-modal mapping from unstructured safety supervision texts and real-time operational information to underlying mathematical control parameters. The implementation of the large model boundary parameter generation module 105 is divided into two stages: offline fine-tuning and online inference.

[0048] During the offline fine-tuning phase, an open-source large language model with a suitable parameter size was selected as the base model. Since performing full parameter fine-tuning of a large language model in an industrial environment would consume excessive GPU memory, the large model boundary parameter generation module 105 employs a low-rank adaptive technique to efficiently fine-tune the parameters of the base model. The core idea of ​​this technique is to freeze the backbone weight matrix of the pre-trained model. Inject a pair of trainable low-rank matrices, including a dimensionality-reduced matrix, into its side path. With increasing dimension matrix , where rank much smaller During fine-tuning, the model's equivalent weights become: ;

[0049] Due to low-rank matrices and The total number of trainable parameters is only a tiny fraction of the total number of parameters. This method achieves deep injection of domain knowledge with extremely low computational cost while maintaining the general language understanding capability of the base model. The corpus used for fine-tuning covers a variety of text resources, including the full text of coal mine safety regulations, historical accident exploration reports, equipment operation manuals, safety supervision notices, and relevant industry standards. The model is trained to understand proprietary concepts, risk level classification logic, and parameter adjustment rules in the field of coal mine safety through instruction fine-tuning.

[0050] After low-rank adaptive fine-tuning, the large model boundary parameter generation module 105 further utilizes the group sequence policy optimization algorithm to perform alignment training on the fine-tuned model. Group sequence policy optimization is a large model alignment method based on reinforcement learning. By constructing a preference dataset labeled by experts in the coal mine safety field, it selectively strengthens and suppresses inferior performance in the model's inference output when facing complex combinations of working conditions, ensuring that the generated safety instructions strictly follow the inference paths and judgment criteria expected by the experts. The large language model, after two stages of fine-tuning—low-rank adaptive and group sequence policy optimization—possesses both a deep understanding of knowledge in the coal mine safety field and meets industrial application standards in terms of the format standardization and logical accuracy of its inference output.

[0051] The triggering mechanism and operation process of the online inference phase are as follows. When the intrinsic security risk index output by the security risk dynamic assessment module 102... When a significant spike occurs within an adjacent control cycle, or when the multi-source sensing and state construction module 101 detects a sudden abnormal event signal, the online inference process of the large model boundary parameter generation module 105 is activated. The system converts the current structured working condition data into a natural language description through prompt word engineering, and combines it with retrieval enhancement generation technology to retrieve the most relevant procedural provisions and historical handling cases from the safety knowledge base. These are then assembled into input prompt words and fed into the fine-tuned large language model for inference. The introduction of retrieval enhancement generation technology enables the large language model to not only rely on its parameterized stored knowledge during the inference process, but also to reference the latest regulatory requirements and technical specifications from the external knowledge base in real time, ensuring the timeliness and authority of the inference basis.

[0052] The large language model performs comprehensive reasoning based on the input working condition description and retrieved knowledge fragments to determine the hazard level of the current working condition combination and outputs predefined structured parameter adjustment instructions. These structured instructions are organized in key-value pair format and include fields such as target subsystem identifier, adjustment parameter type, adjustment direction, and adjustment magnitude. For example, a typical output might generate the following instruction sequence: Target subsystem is the locomotive collision avoidance transportation subsystem; adjustment parameter is the safety distance margin coefficient; adjustment direction is tightening; adjustment magnitude is 1.5 times. Target subsystem is the intelligent ventilation control subsystem; adjustment parameter is the gas concentration boundary mitigation coefficient; adjustment direction is tightening; adjustment magnitude is reduction by 20%. After parsing the above structured instructions, the mapping parsing script converts the adjustment magnitude into specific numerical changes in the control obstacle function parameters according to a predefined parameter mapping rule table. The mapping rule table defines the types of each adjustment parameter and... The correspondence and conversion formulas between the components of the function are as follows: for example, the multiplier value of the safety distance margin coefficient is directly multiplied by the baseline value of the safety distance threshold in the control barrier function to obtain the updated threshold, and the reduction ratio of the gas concentration boundary mitigation coefficient is reduced proportionally. The corresponding decay rate parameter is mapped and transformed. The resulting value is directly written into the parameter register of the control barrier function in the safety barrier and motion projection module 103, completing the real-time update of the boundary parameters. Through this mechanism, the safety barrier in the safety barrier and motion projection module 103 no longer relies on static preset parameters, but instead acquires dynamic adaptive adjustment capabilities based on cognitive reasoning, realizing seamless connection and bidirectional driving between unstructured safety knowledge and underlying mathematical control logic.

[0053] The following describes the complete workflow of the system using a typical complex working condition scenario in a coal mine. Suppose that during the advance of the longwall face, a geological fault is encountered, causing abnormal fluctuations in the local gas emission. At the same time, multiple unmanned aerial vehicles (UAVs) are performing coal transportation tasks in the transport roadway.

[0054] When the system starts up or enters a new control cycle, the ground-based centralized control server loads the weight parameters of the pre-trained multi-agent policy network and value network into the GPU memory of the computing device, and simultaneously loads the training weights of the BP-fuzzy neural network in the dynamic safety risk assessment module 102. The low-rank neighborhood parameter matrix of the large model boundary parameter generation module 105 is also loaded. and The parameters are loaded and merged into the corresponding weight layer of the base large language model. The safety barrier and action projection module 103 initializes the safety envelope boundary parameters of the control barrier functions of each subsystem according to the current normal operating conditions of the system. At this point, the system enters the standard control and monitoring state.

[0055] exist At any given moment, sensor nodes deployed in various underground areas transmit real-time data back to the edge computing center at high frequency through network slicing channels. After receiving this heterogeneous time-series data, the multi-source sensing and state construction module 101 performs outlier filtering and delay alignment processing, and uses a gated loop unit to perform time-series encoding on the historical observation sequences of each agent, constructing local state observation vectors for each agent such as the ventilation fan, coal mining machine, and transport locomotive. At the same time, the original physical state vectors of each subsystem are preserved. Then, all local observations are concatenated to form a global continuous state vector. Distribute the original physical state vectors of each subsystem to the multi-agent collaborative decision-making module 104. Distribute to the safety risk dynamic assessment module 102, the safety barrier and motion projection module 103, and the large model boundary parameter generation module 105.

[0056] The safety risk dynamic assessment module 102 receives the original physical state vectors of each subsystem. The calculation is then initiated. In this scenario, the gas concentration monitoring data shows an abnormally steep upward slope, and multiple vehicles are densely packed in the locomotive tunnel. The safety risk dynamic assessment module 102 calls a Gaussian membership function to fuzzify these physical quantities, transforming them into a risk membership vector in fuzzy space, which is then input into an offline-trained BP neural network to perform forward propagation. The calculation results show that the intrinsic safety risk index... The previously low steady-state level has risen sharply to a high value. This high-risk signal is immediately broadcast to all modules of the system as an interrupt-level event.

[0057] Intrinsic safety risk index The surge signal triggers the online inference process of the large model boundary parameter generation module 105. The system converts the current abnormal operating condition data into prompt words, and combines retrieval enhancement generation technology to retrieve the regulation clause with the highest matching degree to the current operating condition from the safety knowledge base, and inputs it into the large language model that has been fine-tuned in two stages: low-rank adaptive and group sequence strategy optimization. After comprehensive analysis, the large language model determines that the current operating condition combination has a potential risk of inducing secondary disasters, and outputs a structured parameter tightening instruction. The mapping parsing script converts the instruction into specific numerical variables according to the parameter mapping rule table, and applies the structural parameters of the control obstacle functions of each subsystem in the safety barrier and action projection module 103. The modifications are made immediately, specifically by increasing the safe distance threshold in the locomotive collision avoidance control obstacle function and narrowing the conservative control envelope of the ventilation fan speed regulation.

[0058] During the aforementioned security boundary update process, the policy networks of each agent in the multi-agent collaborative decision-making module 104 are updating based on their respective local observations. Generate a nominal action. Due to inherent exploration noise in reinforcement learning and the fact that the policy has not yet fully converged, the locomotive agent may issue a nominal instruction to continue accelerating to maintain transportation efficiency. The ventilation agent may issue a nominal command that fails to increase the fan speed in a timely manner to match the gas outburst rate. The nominal actions of all agents are aggregated into a nominal action matrix. The signal is sent to the safety barrier and motion projection module 103 for safety verification.

[0059] The safety barrier and motion projection module 103 uses the dynamic parameters updated by the large model boundary parameter generation module 105, combined with the current original physical state vectors of each subsystem. Safety checks are performed on the nominal action matrix. Taking the locomotive anti-collision subsystem as an example, this is achieved by calculating the Lie derivative condition. The system determines that if the nominal strategy is executed directly, the locomotive will exceed the updated minimum safe collision avoidance distance within several subsequent control steps, and the gas accumulation rate of the ventilation subsystem will exceed the safety limit. The quadratic programming solvers for each subsystem then activate, intercepting the dangerous nominal actions under relaxed variable constraints. Calculate the corrective action that is closest to the original exploration intention but strictly satisfies the current safety boundary constraints. In this scenario, the corrective action corresponds to forcibly initiating the locomotive's slow braking while simultaneously significantly increasing the operating frequency of the local ventilation fans. The final safety action... The command is sent to the corresponding physical actuator via the industrial communication bus for execution.

[0060] Physical system in safe operation The command is sent to the corresponding physical actuator via the industrial communication bus for execution.

[0061] Physical system in safe operation Driven by the evolution to the state of the next moment Meanwhile, the centralized value network in the multi-agent collaborative decision-making module 104, based on the new global state vector and actual actions, and instant rewards including dynamic risk penalties Value assessment is performed. During the reverse update phase of the policy network, the differentiable architecture of the safety barrier and action projection module 103 demonstrates its core value: the implicit function Jacobian matrix output by the quadratic programming solver through the KKT conditions allows the gradient signal generated by the safety correction to penetrate the shielding layer and propagate back to the parameter updates of each policy network in the multi-agent collaborative decision-making module 104. Through continuous closed-loop iterative training, the policy networks of each agent gradually learn the ability to converge safely under dynamic physical constraints, ultimately finding a balanced strategy that balances production efficiency and energy consumption optimization within the safety barrier constraints.

[0062] When the system is in normal operation and the intrinsic security risk index When maintained at a low level, the large model boundary parameter generation module 105 does not frequently trigger inference, and the control barrier function parameters in the safety barrier and action projection module 103 remain at their default settings under standard operating conditions. At this time, the policy network of the multi-agent collaborative decision-making module 104 has a relatively relaxed exploration space to continuously optimize production efficiency. When geological conditions change, equipment status becomes abnormal, or external regulatory requirements are updated, the large model boundary parameter generation module 105 is triggered to execute inference and dynamically adjust the control barrier function parameters. The safety envelope of the safety barrier and action projection module 103 tightens accordingly or is appropriately relaxed when conditions permit, thereby achieving a dynamic balance between the strictness of safety constraints and the degree of freedom in production. Through high-frequency closed-loop collaboration among the above modules, the entire system forms a complete signal loop from data acquisition, risk quantification, dynamic boundary adjustment, action generation, safety interception, physical execution to gradient backpropagation optimization. Under the absolute premise of preventing catastrophic safety accidents, it fully releases the optimization potential of reinforcement learning algorithms in the field of multi-device collaborative control.

[0063] Those skilled in the art should understand that the above embodiments are merely illustrative of the technical solutions of the present invention and not intended to limit them. Equivalent substitutions or modifications made to the specific sensor types, neural network layers and nodes, membership function types, quadratic programming solver implementations, large language model base selections, and reinforcement learning algorithm variants involved in the above embodiments, without departing from the technical concept of the present invention, should all fall within the protection scope of the claims of the present invention.

Claims

1. A coal mine production safety intelligent control system based on a reinforcement learning model, characterized in that, include: The multi-source perception and state construction module is used to collect the original physical state vectors of various business subsystems in the coal mine, and to perform time-series encoding on the local observation sequence to construct the local observation state vector and global state vector of the corresponding intelligent agent. The safety risk dynamic assessment module is used to receive the original physical state vector, calculate and output the intrinsic safety risk index that represents the safety status of the entire mine in real time; The multi-agent collaborative decision-making module configures a policy network and a centralized value network for each business subsystem. The policy network outputs the nominal action of the corresponding agent based on the local observation state vector. The safety barrier and action projection module is used to receive the original physical state vector and the nominal action, construct a quadratic programming problem based on the control barrier function containing the system's continuous-time physical dynamics equations, and project the nominal action into an actual execution action that satisfies the forward invariance condition within the control time step, and send it to the physical actuator; and calculate the implicit function partial derivative of the KKT condition at the optimal solution of the quadratic programming problem to obtain the Jacobian matrix of the actual execution action with respect to the nominal action; the multi-agent collaborative decision-making module uses the Jacobian matrix to fuse the value network gradient to update the policy network parameters during the backpropagation stage of the policy network training. The large model boundary parameter generation module is triggered when the intrinsic safety risk index changes abnormally. It combines real-time operating data to infer and output structured parameter adjustment instructions, and updates the parameter register of the control obstacle function in the safety barrier and action projection module after parsing the instructions, so as to dynamically adjust the safety boundary parameters.

2. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 1, characterized in that, The multi-source perception and state construction module uses a gated loop unit to perform temporal encoding on historical observation data, compressing the local observation sequence into a feature vector. The multi-source perception and state construction module adopts a dual-channel data distribution mechanism: the global state vector and the local observation state vector after temporal encoding by the gated loop unit are distributed to the multi-agent collaborative decision-making module; the original physical state vector without temporal encoding is simultaneously distributed to the security risk dynamic assessment module, the security barrier and action projection module, and the large model boundary parameter generation module.

3. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 1, characterized in that, The security risk dynamic assessment module adopts an architecture that integrates fuzzy logic system and BP neural network; In the fuzzification stage, a Gaussian membership function is configured for the input variables to map the original physical state into a membership vector representing the risk semantic level. In the neural network inference stage, the membership vector is input into the BP neural network, and the output range is within the range of forward propagation. The intrinsic safety risk index between them.

4. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 1, characterized in that, In the aforementioned security barrier and motion projection module, the physical state evolution of each business subsystem is assumed to satisfy an affine dynamics model. ,in This is the original physical state vector. To control the input, and These are the drift term and the control term, respectively; the Lie derivative constraint condition for constructing the control barrier function is: ; in, A scalar function to characterize the physical security boundary, and They are respectively along and Lie derivative of direction, For parameterized extended K-class functions, For reference working point, The dynamic boundary adjustment coefficient is jointly determined by the intrinsic safety risk index and the large model boundary parameter generation module. This is a set of adjustable parameters.

5. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 4, characterized in that, The objective function and constraints of the quadratic programming problem are respectively expressed as follows: ; The constraints include: ; in, The nominal action output by the policy network. Slack variables are introduced to avoid unsolvable problems. Penalize weights for slack variables. and These represent the minimum and maximum output limits of the corresponding subsystem's physical actuator, respectively.

6. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 1, characterized in that, The centralized value network in the multi-agent collaborative decision-making module receives the global state vector and the actual actions of all agents after correction by the security barrier and action projection module as input to evaluate the action value of the current joint strategy; the immediate reward function of the multi-agent collaborative decision-making module Defined as: ; in, To reward positive production efficiency, Based on actual actions The calculated device energy consumption penalty component, Based on the intrinsic security risk index With static risk fundamental constant The security risk penalties constituted together , , These are the weighting coefficients for each reward component.

7. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 6, characterized in that, During policy iteration, the parameters of the policy network The formula for updating gradient calculation includes: ； in, For centralized value networks in actual execution of actions The action value gradient at the location, The Jacobian matrix obtained by differentiating the implicit function. This outputs the gradients of the policy network with respect to its parameters.

8. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 1, characterized in that, The large language model in the large model boundary parameter generation module is fine-tuned offline using low-rank adaptive technology. The pre-trained backbone weight matrix is ​​frozen and a trainable low-rank dimensionality reduction matrix and dimensionality increase matrix are injected in the side path. After the low-rank adaptive fine-tuning is completed, the group sequence strategy optimization algorithm is further used to perform alignment training on the large language model.

9. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 8, characterized in that, During the online inference phase, the large model boundary parameter generation module converts the current structured chemical condition data into a natural language description and, in conjunction with retrieval enhancement generation technology, retrieves relevant procedural provisions and handling cases from the safety knowledge base, assembling them into prompt words that are then input into the fine-tuned large language model. After comprehensive inference, the large language model outputs a key-value pair-based structured parameter adjustment instruction, which includes fields for target subsystem identifier, adjustment parameter type, adjustment direction, and adjustment magnitude. The system then uses a parameter mapping rule table to convert the adjustment magnitude into a specific numerical change in the structural parameters of the control barrier function and writes it into the parameter register.

10. The intelligent control system for coal mine production safety based on a reinforcement learning model according to claim 1, characterized in that, Each of the aforementioned business subsystems includes at least a locomotive collision avoidance and transportation subsystem and an intelligent ventilation control subsystem; wherein, for the locomotive collision avoidance and transportation subsystem, the original physical state vector includes the vehicle's speed, acceleration, following distance to the vehicle in front, speed of the vehicle in front, and heading angle; for the intelligent ventilation control subsystem, the original physical state vector includes the roadway wind speed, gas concentration, fan power, motor temperature, and pressure difference.

Images