Coal rock fracture evolution mechanism construction method and system based on reinforcement learning architecture

Abstract

The invention discloses a coal rock fracture evolution mechanism construction method and system based on a reinforcement learning architecture. The exploration method comprises the following steps: obtaining staged time CT detection and full space-time acoustic emission monitoring data in a coal rock sample loading process; performing correspondence on the CT detection data and the whole space-time acoustic emission monitoring data at the staged moments according to the moments to obtain a fracture parameter data set; obtaining a symbol data set through operation symbols and variable parameters, and inputting the symbol data set into a mechanism equation generation network to obtain an initial mechanism equation; carrying out optimization factor extraction on fracture and acoustic emission parameter data of a training set and a verification set in the fracture parameter data set through sparse regression and a multi-objective optimization algorithm, and meanwhile, obtaining an advanced mechanism equation; optimizing the advanced equation through a reinforcement learning architecture to obtain a coal rock fracture evolution mechanism equation; and solving through a mechanism equation to obtain a three-dimensional coal rock sample fracture evolution result, and performing deviation calculation through the fracture evolution result and CT detection data to optimize architecture parameters.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coal rock monitoring, and particularly relates to a coal rock fracture evolution mechanism construction method and system based on a reinforcement learning architecture. BACKGROUND

[0002] In the process of coal mining, mining activities and other weight factors will destroy the original stress balance state of the coal rock mass. This process usually involves rapid and unstable expansion of fractures, which is the root cause of inducing dynamic disasters such as rock burst, coal and gas outburst, etc. Therefore, in order to deeply understand the mechanism of coal rock dynamic disasters and realize early warning of the disasters, it is necessary to carry out in-depth research on the fracture evolution mechanism and other aspects.

[0003] Although existing rock mechanics theories and methods attempt to explain the evolution of the fracture field of the coal rock mass caused by mining activities, this complex process is still considered as a "black box" that is difficult to measure and reveal. In exploring the formation mechanism of the fractures and revealing the development and evolution process thereof, there are still many challenges. At present, certain progress has been made in industrial CT detection and numerical simulation methods, which can be used to reconstruct the internal structure of the coal rock mass, and especially show potential in constructing three-dimensional digital models of non-continuous structures and fracture fields. For example, patent CN119559271A discloses a full-diameter coal rock composition quantitative identification method and device based on CT scanning, patent CN119269282A discloses a method and a simulation device for simulating the development and fragmentation process of fractured rock mass, and patent CN112435332A discloses a fine numerical modeling method for fractured coal bodies based on CT three-dimensional reconstruction. However, these methods still have some limitations. They usually rely on high-specification laboratory conditions and cumbersome model reconstruction methods, which limits their wide application in actual engineering. In addition, these methods cannot realize evolution mechanism analysis, which may cause certain limitations in actual production processes.

[0004] In recent years, some experimental methods combining exploration-monitoring technology with numerical simulation have been applied in the laboratory. However, the current utilization of these exploration-monitoring data still mainly focuses on manually analyzing the time sequence variation law. In addition, completely relying on numerical simulation for crack evolution mechanism analysis requires extremely high computational cost, and even a slight modification in the model may cause a large negative weight in the model calculation process. Therefore, how to utilize the exploration-monitoring data to realize crack evolution mechanism analysis and make up for the limitations of traditional methods still needs further research. With the continuous progress and development of science and technology, the breakthrough progress of reinforcement learning method attracts people's attention in the exploration of crack evolution mechanism equation. Especially by using the reinforcement learning method, it has a significant advantage in exploring the equation parameters, which can combine a large number of mechanism equations to find the optimal solution in the complex combination space. In addition, the multi-objective optimization algorithm can balance multiple conflicting objective functions, and the sparse regression algorithm automatically eliminates redundant or noisy features through regularization method, thereby improving the accuracy of exploring the equation.

[0005] This provides the possibility for realizing the exploration and optimization of crack evolution mechanism equation by using the above method, but there is still a lack of specific feasible ideas and implementation methods. Therefore, it is urgent to carry out the actual measurement of exploration-monitoring signals in the loading process of coal and rock mass, propose a coal and rock crack evolution mechanism construction method and system based on reinforcement learning architecture, realize the exploration of crack evolution mechanism equation based on full-time and space acoustic emission monitoring data, and have a significant promoting effect on revealing the “black box” evolution process of coal and rock dynamic disaster and realizing the monitoring and early warning of coal and rock dynamic disaster. SUMMARY

[0006] The present scheme proposes a coal and rock crack evolution mechanism construction method and system based on reinforcement learning architecture to solve the problems and needs mentioned above. The technical features adopted can achieve the above technical purposes and bring other technical effects.

[0007] One object of the present application is to propose a coal and rock crack evolution mechanism construction method based on reinforcement learning architecture, characterized in that it comprises the following steps:

[0008] S10: arranging CT detection sensors and acoustic emission monitoring sensors on the coal and rock sample, and placing the coal and rock mass in the testing machine for loading test, and acquiring the CT detection data and full-time and space acoustic emission monitoring data of the coal and rock mass at the stage time through the CT detection sensors and acoustic emission monitoring sensors; wherein the CT detection data is the full-space crack parameter data at the stage time, and the acoustic emission monitoring data is the full-time and space acoustic emission parameter data;

[0009] S20: corresponding the CT detection data and the full-time and space acoustic emission monitoring data in the loading process of the coal rock mass to time, obtaining a crack parameter data set, and dividing the crack parameter data set into a training set, a verification set and a test set; and manufacturing an operator symbol and a variable parameter of the mechanism equation into a symbol data set;

[0010] S30: reading the symbol data set by the mechanism equation generation network to obtain an initial mechanism equation;

[0011] S40: obtaining each coefficient in the initial mechanism equation through a sparse regression algorithm based on the acoustic emission monitoring data and the crack parameter data in the crack parameter data set, and then obtaining an advanced mechanism equation and calculating a fitting error;

[0012] S50: obtaining an optimization factor through a fitting error, and an accuracy and complexity of calculating an expression of the advanced mechanism equation by a multi-objective optimization algorithm;

[0013] S60: inputting the optimization factor into a reinforcement learning architecture, and optimizing a generation strategy of the mechanism equation generation network through a reward function and a risk preference policy gradient function, and obtaining a crack evolution mechanism equation;

[0014] S70: obtaining a three-dimensional coal rock sample crack evolution result through the mechanism equation, judging whether the accuracy of the three-dimensional coal rock sample crack evolution result meets a requirement, and when the accuracy of the three-dimensional coal rock sample crack evolution result cannot meet the requirement, performing deviation calculation through the test set CT detection data and adjusting parameters of the reinforcement learning architecture until the accuracy of the three-dimensional coal rock sample crack evolution result meets the requirement.

[0015] In addition, the coal rock crack evolution mechanism construction method and system based on the reinforcement learning architecture can have the following technical features:

[0016] In an example of the present application, in the step S20, manufacturing the operator symbol of the mechanism equation into the symbol data set includes the following steps:

[0017] First, the operator symbol required for mechanism equation exploration is sorted out;

[0018] Then, the operator symbol is manufactured into a binary tree structure symbol data set, and the symbol data set contains the following contents:

[0019]

[0020] In the formula, OS represents the symbol data set; + represents the addition operator; - represents the addition operator; * represents the addition operator; / represents the division operator; ^2 represents the square operator; and ^3 represents the cube operator; represents a first-order partial derivative operator; Represents the second-order partial derivative operator; represents the third-order partial derivative operator; AE represents acoustic emission variable parameter data; F represents fracture variable parameter data.

[0021] In one example of the present invention, in step S30, generating the network reads the symbol dataset from the mechanism equation to obtain the initial mechanism equation includes the following steps:

[0022] S31: Transmit the operator O through the data encoding layer m and variable parameter S n Extracting the enhanced operator feature of Of e and variable parameter features Sf e ;

[0023] S32: The encoded data is concatenated to the feature extraction layer B through the residual connection layer to obtain feature O. b and S b middle;

[0024] S33: The operator's characteristics are determined by the first Normalization function. e and variable parameter features Sf e Regularization yields feature O n and S n ;

[0025] S34: Regularized Feature O n and S n Feature O is obtained by performing feature extraction through the residual connection formed by the second linear feature extraction layer A and the second activation function layer. a+s and S a+s Feature O spliced ​​into the output of the selective state-space architecture sssm and S sssm O SSSM and S SSSM ;

[0026] S35: Feature O output by the first linear feature extraction layer A a and S a Higher-dimensional features O are obtained through feature extraction via convolutional layers and a first activation function layer. conv+s and S conv+s ;

[0027] S36: Gated computation of high-dimensional features is performed using a selective state-space architecture, and then concatenated with the features extracted by the convolutional layer and the first activation function layer to obtain feature O. SSSM and S SSSM ;

[0028] S37: The linear feature extraction layer B extracts features from the spliced features, and splices the extracted features with the features of the encoded data by the residual connection layer to obtain the feature Of b+se and Sf b+se ;

[0029] S38: The spliced features are regularized by the second normalization function to obtain the feature Of n and Sf n ;

[0030] S39: The extracted features are calculated by the full connection layer to obtain the initial mechanism equation.

[0031] In an example of the present application, in the step S40, the expression of the coefficient of each term in the initial mechanism equation is as follows:

[0032]

[0033] In the formula, represents the optimal coefficient in the initial mechanism equation; represents the coefficient vector; Θ(F, AE) represents the initial equation library, which converts the observed F(AE, t) and its derivative information into numerical features, so as to realize the coefficient vector The matrix multiplication can obtain the prediction result of F t ; F t represents the numerical approximation of the time derivative in the mechanism equation; the objective function represents the fitting error between the prediction and the true value; λ represents the regularization parameter, which is used to suppress the coefficient vector represents the regularization term, which ensures the simplicity of the initial mechanism equation; represents the square of the norm, which is used to measure the size of the coefficient vector .

[0034] In an example of the present application, in the step S50, the expression of the accuracy and complexity of the advanced mechanism equation is as follows:

[0035]

[0036] f2 = min (α·k)

[0037] In the formula, f1 represents the accuracy function of the evaluation mechanism equation; f2 represents the complexity function of the evaluation mechanism equation; N represents the total number of fitting error observations; α represents the penalty factor; represents the accuracy optimization factor; k represents the complexity optimization factor.

[0038] In one example of the present application, in the step S60, the reward function expression is as follows:

[0039]

[0040] wherein R represents the reward function; β represents the hyperparameter, controlling the influence of the mechanism equation complexity on the optimization factor; represents the fitting error of all observation values.

[0041] In one example of the present application, in the step S60, the risk preference policy gradient function expression is as follows:

[0042] J risk (θ; ∈) = E τ～p(τ|θ) [R(τ) | R(τ) ≥ R ∈ (θ)]

[0043]

[0044] wherein J risk (θ; ∈) represents the risk preference policy gradient function; θ represents the network parameter of the mechanism equation generation network; ∈ represents the risk preference parameter, used for screening the proportion of high-reward advanced mechanism equations; N represents the number of the current batch of advanced mechanism equations; R(τ (i) ) represents the reward value of the advanced mechanism equation τ (i) , calculated by the reward function; represents the threshold value of the reward value (1-∈) in the current batch of advanced mechanism equations; 1 {·} represents the exponential function, taking 1 if the current condition is true, and 0 otherwise; p(τ (i) | θ) represents the probability of the mechanism equation generation network parameter θ generating the advanced mechanism equation τ (i) , T represents the mechanism equation length; log p(τ (i) | θ) represents the logarithmic value of the probability; represents the partial derivative of log p(τ (i) | θ) with respect to the parameter θ.

[0045] In one example of the present application, in the step S60, the expression of the generation strategy of the optimization mechanism equation generation network is as follows:

[0046]

[0047] wherein θ * represents the updated mechanism equation generation network parameter; η represents the learning rate.

[0048] In one example of the present application, in the step S70, the bias calculation formula is as follows:

[0049]

[0050] In the formula, MSE represents a deviation calculation equation; M represents a sample number; i represents a serial index of a crack parameter; F M,pred represents a predicted value of the Mth sample; F M,true represents a true value of the Mth sample.

[0051] Another object of the present application is to provide a coal rock crack evolution mechanism construction system based on a reinforcement learning architecture, characterized in that it comprises:

[0052] An information collection module is configured to arrange CT detection sensors and acoustic emission monitoring sensors on a coal rock sample, and place a coal rock mass in a testing machine for loading test, and collect CT detection data and full-time and space acoustic emission monitoring data at a stage in a loading process of the coal rock mass through the CT detection sensors and the acoustic emission monitoring sensors; wherein the CT detection data are stage-time full-space crack parameter data, and the acoustic emission monitoring data are full-time and space acoustic emission parameter data;

[0053] A data division module is configured to correspond the CT detection data and the full-time and space acoustic emission monitoring data at the stage in the loading process of the coal rock mass according to time, obtain a crack parameter data set, and divide the crack parameter data set into a training set, a verification set and a test set; and make operator symbols and variable parameters of a mechanism equation into a symbol data set;

[0054] An initial mechanism equation module is configured to read the symbol data set by a mechanism equation generation network, and obtain an initial mechanism equation;

[0055] An advanced mechanism equation module is configured to obtain each coefficient in the initial mechanism equation through a sparse regression algorithm by the acoustic emission monitoring data and the crack parameter data in the crack parameter data set, and further obtain an advanced mechanism equation and calculate a fitting error;

[0056] An optimization factor module is configured to obtain an optimization factor through a fitting error, and an accuracy and complexity of a calculation expression of the advanced mechanism equation by a multi-objective optimization algorithm;

[0057] A crack evolution mechanism equation module is configured to input the optimization factor into the reinforcement learning architecture, optimize a generation strategy of a mechanism equation generation network through a reward function and a risk preference policy gradient function by the reinforcement learning architecture, and obtain a crack evolution mechanism equation;

[0058] The architecture parameter optimization module is configured to obtain a three-dimensional coal rock sample crack evolution result through mechanism equation solving, judge whether the accuracy of the three-dimensional coal rock sample crack evolution result meets the requirements, and when the accuracy of the three-dimensional coal rock sample crack evolution result cannot meet the requirements, carry out deviation calculation through test set CT detection data and adjust the parameters of the reinforcement learning architecture until the accuracy of the three-dimensional coal rock sample crack evolution result meets the requirements.

[0059] Compared with the prior art, the present application has the following beneficial effects:

[0060] 1. The present application focuses on realizing the exploration and discovery of the crack evolution mechanism equation in the coal rock sample loading test process through the coal rock crack evolution mechanism construction method and system based on the reinforcement learning architecture, and reveals the crack parameter evolution mechanism in the coal rock sample loading test process. Compared with the traditional numerical simulation and neural network method, the coal rock crack evolution mechanism construction method based on the reinforcement learning architecture realizes the end-to-end autonomous optimization of the equation structure and parameters by constructing a customized reward function with the crack evolution mechanism equation fitting accuracy as the target; combined with strategy optimization and value evaluation, the high-dimensional parameter space is efficiently explored, and the efficiency and accuracy of equation discovery are significantly improved, providing an efficient and reliable new paradigm for coal rock crack evolution mechanism research.

[0061] 2. The mechanism generation network in the present application is obtained by performing spatio-temporal feature extraction on the data in the symbolic data set through the neural network based on the selective state space architecture, and a feature extraction method for the evolution mechanism symbols in the coal rock sample loading process is proposed. Based on the feature extraction method of the selective state space architecture, through the feature learning mechanism of multiple matrix parameters, high-dimensional features in the input data can be effectively extracted while reducing the dependence on computing resources, and the initial mechanism equation of coal rock crack evolution is generated through the symbolic data set.

[0062] 3. The equation coefficients of the coal rock crack evolution mechanism equation in the present application are obtained through sparse regression algorithm. The sparse regression algorithm extracts features from the crack parameter data set, selects the optimal equation coefficient combination of the coal rock crack evolution mechanism equation, and automatically eliminates redundant features on this basis to reduce the model complexity, realizes accurate coefficient screening in high-dimensional parameter space; combined with the multi-objective optimization method, the equation optimization factor is output, and the reinforcement learning architecture is used to realize the optimization of the exploration strategy of the coal rock crack evolution mechanism equation, providing a new perspective for mechanism exploration of coal rock crack evolution mechanism.

[0063] The optimal embodiments for implementing the present application will be described in more detail below with reference to the accompanying drawings, so that the features and advantages of the present application can be easily understood. BRIEF DESCRIPTION OF DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application will be briefly introduced below. Among them, the drawings are only used to show some embodiments of the present application, and not to limit all embodiments of the present application to this.

[0065] Figure 1 The flow chart of the coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to the embodiments of the present application is shown in the figure.

[0066] Figure 2 The structural schematic diagram of the initial mechanism equation according to the embodiments of the present application is shown in the figure.

[0067] Figure 3 The arrangement structural schematic diagram of the coal rock sample in the testing machine according to the embodiments of the present application is shown in the figure.

[0068] List of reference signs:

[0069] Testing machine 100;

[0070] Loading device 110;

[0071] Acoustic emission sensor 120;

[0072] Coupler 130;

[0073] Pre-amplifier 140;

[0074] X-ray detector 150;

[0075] X-ray source 160;

[0076] Coal rock sample 200;

[0077] Transverse direction X;

[0078] Longitudinal direction Y. DETAILED DESCRIPTION

[0079] In order to make the purpose, technical solutions and advantages of the technical solutions of the present application more clear, the technical solutions of the embodiments of the present application will be clearly and completely described below in combination with the drawings of the embodiments of the present application. The same reference signs in the drawings represent the same parts. It should be noted that the described embodiments are part of the embodiments of the present application, not all. Based on the described embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0080] Unless otherwise defined, technical terms or scientific terms used herein shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms "first", "second", and similar terms, as used in the specification and claims of this application, do not imply any order, quantity, or importance, but are used to distinguish one element from another. Also, the terms "a" or "an", as used in the specification and claims of this application, do not limit the number of elements. The terms "including", "comprising", and similar terms, as used in the specification and claims of this application, mean that elements or objects preceding the term encompass elements or objects recited after the term, and equivalents thereof, while not excluding other elements or objects. The terms "connected" or "coupled", as used in the specification and claims of this application, do not necessarily mean physically or mechanically connected, but can include electrical connection, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like, are used only to indicate relative positions, and when the absolute positions of the described objects are changed, the relative positions may also be changed accordingly.

[0081] According to the coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture, the method comprises the following steps: Figure 1 as shown in the figure, comprising the following steps:

[0082] S10: CT detection sensors and acoustic emission monitoring sensors are arranged on the coal rock sample, and the coal rock body is placed in the testing machine for loading test. CT detection data and full-time and space acoustic emission monitoring data in the loading process of the coal rock body are collected by the CT detection sensors and the acoustic emission monitoring sensors. The CT detection data is the full-space fracture parameter data at the stage time, and the acoustic emission monitoring data is the full-time and space acoustic emission parameter data. The acoustic emission monitoring data is the full-time and space acoustic emission parameter data, and the CT detection data is the full-space fracture parameter data at the stage time. The sample is, for example, Figure 3As shown, the testing machine 100 comprises a loading device 110, an acoustic emission sensor 120, a coupler 130, a preamplifier 140, an X-ray detector 150, an X-ray source 160 and a box sample 170; wherein the loading device 110, the acoustic emission sensor 120, the coupler 130, the preamplifier 140, the X-ray detector 150 and the X-ray source 160 are all installed in the box sample 170, a coal rock sample 200 is arranged in the loading device 110, and the acoustic emission sensors 120 are arranged on both sides of the coal rock sample 200 in the transverse direction X, wherein the acoustic emission sensors 120 are electrically connected with the coupler 130 and are electrically connected with the preamplifier 140, and the X-ray detector 150 and the X-ray source 160 are arranged on both sides of the coal rock sample 200 in the longitudinal direction Y, wherein the transverse direction X and the longitudinal direction Y are perpendicular to each other. The working process is as follows: the coal rock sample 200 is loaded by the loading device 110, in the process, the X-ray is emitted by the X-ray source 160 and received by the X-ray detector 150, so as to obtain the CT detection data of the coal rock sample 200 in the breaking process, at the same time, the full-time-space acoustic emission monitoring data is obtained by the acoustic emission sensor 120.

[0083] S20: corresponding the stage time CT detection data and the full-time-space acoustic emission monitoring data in the loading process of the coal rock body according to time, obtaining a crack parameter data set, and dividing the crack parameter data set into a training set, a verification set and a test set; and manufacturing the operator symbol and the variable parameter of the mechanism equation into a symbol data set;

[0084] S30: reading the symbol data set by the mechanism equation generation network to obtain an initial mechanism equation;

[0085] S40: obtaining each coefficient in the initial mechanism equation by the sparse regression algorithm of the acoustic emission monitoring data and the crack parameter data in the crack parameter data set, and then obtaining an advanced mechanism equation and calculating a fitting error;

[0086] S50: obtaining an optimization factor by the multi-objective optimization algorithm through the fitting error, the accuracy and the complexity of calculating the expression of the advanced mechanism equation;

[0087] S60: inputting the optimization factor into the reinforcement learning architecture, and optimizing the generation strategy of the mechanism equation generation network by the reinforcement learning architecture through a reward function and a risk preference policy gradient function, and obtaining a crack evolution mechanism equation;

[0088] S70: obtain the three-dimensional coal and rock sample crack evolution result by solving the mechanism equation, judge whether the accuracy of the three-dimensional coal and rock sample crack evolution result meets the requirement, when the accuracy of the three-dimensional coal and rock sample crack evolution result meets the requirement, end the operation; when the accuracy of the three-dimensional coal and rock sample crack evolution result cannot meet the requirement, carry out deviation calculation through the CT detection data of the test set and adjust the parameters of the reinforcement learning architecture until the accuracy of the three-dimensional coal and rock sample crack evolution result meets the requirement.

[0089] The exploration method focuses on the exploration and discovery of the crack evolution mechanism equation in the coal rock sample loading test process through the coal rock crack evolution mechanism construction method and system based on the reinforcement learning architecture, and reveals the crack parameter evolution mechanism in the coal rock sample loading test process. Compared with the traditional numerical simulation and neural network method, the coal rock crack evolution mechanism construction method based on the reinforcement learning architecture realizes the end-to-end autonomous optimization of the equation structure and parameters by constructing a customized reward function with the fitting accuracy of the crack evolution mechanism equation as the target; combined with strategy optimization and value evaluation, it efficiently explores the high-dimensional parameter space and significantly improves the efficiency and accuracy of equation discovery, providing an efficient and reliable new paradigm for coal rock crack evolution mechanism research.

[0090] The mechanism generation network in the exploration method is obtained by spatiotemporal feature extraction of data in the symbolic data set based on the selective state space architecture neural network, and a feature extraction method for the evolution mechanism symbols in the coal rock sample loading process is proposed. Based on the feature extraction method of the selective state space architecture, the feature learning mechanism of multiple matrix parameters can effectively extract high-dimensional features in the input data while reducing the dependence on computing resources, and realize the generation of the initial mechanism equation of the coal rock crack evolution through the symbolic data set.

[0091] The equation coefficients of the coal rock crack evolution mechanism equation in the exploration method are obtained by a sparse regression algorithm. The sparse regression algorithm extracts features from the crack parameter data set, selects the optimal equation coefficient combination of the coal rock crack evolution mechanism equation, and automatically eliminates redundant features to reduce the model complexity, realizes the accurate coefficient screening in the high-dimensional parameter space; combined with a multi-objective optimization method, an equation optimization factor is output, and the reinforcement learning architecture is used to realize the optimization of the exploration strategy of the coal rock crack evolution mechanism equation, providing a new perspective for the mechanism exploration of the coal rock crack evolution mechanism.

[0092] In one example of the present application, in the step S20, manufacturing the operation symbols of the mechanism equation into a symbolic data set includes the following steps:

[0093] Firstly, the operation symbols required for mechanism equation exploration are arranged;

[0094] Then, the operator symbols are made into a binary tree structure symbol dataset, and the symbol dataset contains the following contents:

[0095]

[0096] In the formula, OS represents the symbol dataset; + represents the addition operator; - represents the addition operator; * represents the addition operator; / represents the division operator; ^2 represents the square operator; and ^3 represents the cube operator. represents the first-order partial derivative operator; represents the second-order partial derivative operator; represents the third-order partial derivative operator; AE represents the acoustic emission variable parameter data; and F represents the fracture variable parameter data.

[0097] In an example of the present application, in the step S30, the symbol dataset is read from the mechanism equation generation network to obtain the initial mechanism equation, including the following steps: Figure 2

[0098] S31: The operator symbol O m and the variable parameter S n are extracted to obtain enhanced operator symbol features Of e and variable parameter features Sf e ;

[0099] S32: The encoded data are spliced into the features O b and S b obtained by the feature extraction layer B through the residual connection layer;

[0100] S33: The operator symbol features Of e and the variable parameter features Sf e are regularized by the first Normalization function to obtain the features O n and S n ;

[0101] S34: The regularized features O n and S n are subjected to feature extraction by the residual connection composed of the second linear feature extraction layer A and the second activation function layer to obtain the features O a+s and S a+s , which are spliced into the features O sssm and S sssm output by the selective state space architecture to obtain O SSSM and S SSSM ;

[0102] S35: The features O a and S a ​Feature extraction through convolutional layer and first activation function layer to obtain higher dimensional features O conv+s and S conv+s ;

[0103] S36: Gating calculation on high-dimensional features through a selective state space architecture, and splicing the extracted features through the convolutional layer and the first activation function layer to obtain features O SSSM and S SSSM ;

[0104] S37: Feature extraction on the spliced features through a linear feature extraction layer B, and splicing the extracted features with the features of the encoded data through a residual connection layer to obtain features Of b+se and Sf b+se ;

[0105] S38: Regularization of the spliced features through a second Normalization function to obtain features Of n and Sf n ;

[0106] S39: Calculation of the extracted features through a fully connected layer to obtain an initial mechanism equation.

[0107] In one example of the present application, in the step S36, the selective state space architecture is implemented as follows:

[0108] First, a state space architecture is built through a continuous state space equation, wherein the expression of the continuous state space equation is as follows:

[0109] h t =Ah t-1 +Bx t

[0110] y t =Ch t

[0111] wherein h t-1 represents the system state; h t represents the updated system state; matrix A represents the state transition matrix; B represents the input gating matrix; C represents the output mapping matrix; x t represents the data of the current state input architecture;

[0112] Second, the state space architecture is adapted to discrete data through a discrete state space equation, wherein the equation expression of the state transition matrix A is as follows:

[0113]

[0114] In the formula, n is a row index related to an orthogonal basis function in a polynomial space, and k is a column index related to the orthogonal basis function in the polynomial space.

[0115] In one example of the present application, the state transition matrix A is randomly initialized, and a low-rank matrix after matrix decomposition is used, and the low-rank matrix equation is as follows:

[0116]

[0117] In the formula, P and Q are both vectors with a length of N; V represents an eigenvector matrix; V * represents a transposed matrix; Lambda represents an eigenmatrix; and * represents a projection operation.

[0118] In one example of the present application, the expression of the discrete state space equation is as follows:

[0119]

[0120] In the formula, h k and y k respectively represent a memory control unit and an output control unit for selecting a current state in a state space architecture; h k-1 represents a memory control unit for selecting a previous state in the state space architecture; h0 represents an initialized state of the memory control unit; x k represents data of a current state input architecture; and x0 represents data of an initial state input architecture. A, B, and C are all matrices after discrete adaptation.

[0121] In one example of the present application, the matrix is obtained by the following derivation formula:

[0122]

[0123] In the formula, Delta represents a discrete nonlinear function for discretizing a continuous state; and I represents a unit matrix.

[0124] In one example of the present application, in the step S40, the expression of each coefficient in the initial mechanism equation is as follows:

[0125]

[0126] In the formula, h represents an optimal coefficient in the initial mechanism equation; represents a coefficient vector; and Theta (F, AE) represents an initial equation library, which converts F(AE, t) and derivative information thereof obtained through observation into numerical characteristics, so as to realize the coefficient vector F can be obtained by matrix multiplication.t the prediction result; F t represents the derivative of the mechanism equation with respect to time numerical approximation of the mechanism equation; objective function represents the fitting error between the prediction and the true value; λ represents the regularization parameter, used to suppress the coefficient vector represents the regularization term, ensuring the simplicity of the initial mechanism equation; represents the square of the norm, used to measure the size of the coefficient vector .

[0127] In an example of the present application, in the step S50, the expression of the accuracy and complexity of the advanced mechanism equation expression is as follows:

[0128]

[0129] f2 = min (a k)

[0130] In the formula, f1 represents the evaluation mechanism equation accuracy function; f2 represents the evaluation mechanism equation complexity function; N represents the total number of fitting error observation values; a represents the penalty factor; represents the accuracy optimization factor; k represents the complexity optimization factor.

[0131] In an example of the present application, in the step S60, the reward function expression is as follows:

[0132]

[0133] Wherein, R represents the reward function; β represents the hyperparameter, controlling the influence of the mechanism equation complexity on the optimization factor; represents the fitting error of all observation values.

[0134] In an example of the present application, in the step S60, the risk preference strategy gradient function expression is as follows:

[0135] J risk (θ; ∈) = E τ～p(τ|θ) [R (τ) | R (τ) ≥ R ∈ (θ)]

[0136]

[0137] Wherein, J risk (θ; ∈) represents the risk preference strategy gradient function; θ represents the network parameter of the mechanism equation generation network; ∈ represents the risk preference parameter, used to screen the proportion of the advanced mechanism equation with high reward; N represents the number of the current batch of advanced mechanism equations; R(τ (i) ) represents the advanced mechanism equation τ (i)reward value of the current batch, calculated by the reward function; threshold value representing the reward value (1-∈) of the current batch in the mechanism equation; {·} representing the exponential function, taking 1 if the current condition is true, otherwise taking 0; p(τ (i) |θ) represents the probability that the mechanism equation generation network parameter θ generates the mechanism equation τ (i) T represents the length of the mechanism equation; log p(τ (i) |θ) represents the logarithmic value of the probability; representing the partial derivative of log p(τ (i) |θ) with respect to the parameter θ.

[0138] In one example of the present application, in the step S60, the expression of the generation strategy of the mechanism equation generation network is as follows:

[0139]

[0140] where θ * represents the updated mechanism equation generation network parameter; η represents the learning rate.

[0141] In one example of the present application, in the step S70, the deviation calculation formula is as follows:

[0142]

[0143] In the formula, MSE represents the deviation calculation equation; M represents the number of samples; i represents the serial index of the fracture parameter; F M,pred represents the predicted value of the Mth sample; F M,true represents the true value of the Mth sample.

[0144] According to the coal rock fracture evolution mechanism construction system based on the reinforcement learning architecture according to the second aspect of the present application, the system comprises:

[0145] The information acquisition module is configured to arrange CT detection sensors and acoustic emission monitoring sensors on the coal rock sample, and place the coal rock mass in the testing machine for loading test, and acquire CT detection data and acoustic emission monitoring data at the stage time and in the whole space during the loading process of the coal rock mass through the CT detection sensors and the acoustic emission monitoring sensors; wherein the CT detection data are stage time whole space fracture parameter data, and the acoustic emission monitoring data are whole space acoustic emission parameter data.

[0146] ​a data division module configured to correspond CT detection data and full-time and space acoustic emission monitoring data at a stage in a coal rock mass loading process according to time, to obtain a fracture parameter data set, and to divide the fracture parameter data set into a training set, a verification set, and a test set; and to make an operator symbol and a variable parameter of a mechanism equation into a symbol data set;

[0147] an initial mechanism equation module configured to read the symbol data set by a mechanism equation generation network to obtain an initial mechanism equation;

[0148] an advanced mechanism equation module configured to obtain each coefficient in the initial mechanism equation from acoustic emission monitoring data and fracture parameter data in the fracture parameter data set through a sparse regression algorithm, to further obtain an advanced mechanism equation and to calculate a fitting error;

[0149] an optimization factor module configured to obtain an optimization factor through a fitting error, and an accuracy and complexity of a calculation of an expression of the advanced mechanism equation by a multi-objective optimization algorithm;

[0150] a fracture evolution mechanism equation module configured to input the optimization factor into a reinforcement learning architecture, to optimize a generation strategy of a mechanism equation generation network by a reward function and a risk preference policy gradient function, and to obtain a fracture evolution mechanism equation;

[0151] an architecture parameter optimization module configured to obtain a three-dimensional coal rock sample fracture evolution result by a mechanism equation solution, to determine whether an accuracy of the three-dimensional coal rock sample fracture evolution result meets a requirement, to end an operation when the accuracy of the three-dimensional coal rock sample fracture evolution result meets the requirement, and to calculate a deviation by test set CT detection data and to adjust parameters of the reinforcement learning architecture until the accuracy of the three-dimensional coal rock sample fracture evolution result meets the requirement when the accuracy of the three-dimensional coal rock sample fracture evolution result does not meet the requirement.

[0152] The exploration system focuses on realizing exploration and discovery of a fracture evolution mechanism equation in a coal rock sample loading test process by a coal rock fracture evolution mechanism construction method and system based on a reinforcement learning architecture, and revealing a fracture parameter evolution mechanism in the coal rock sample loading test process. Compared with a traditional numerical simulation and neural network method, the coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture realizes end-to-end autonomous optimization of an equation structure and parameters by constructing a customized reward function with a fracture evolution mechanism equation fitting accuracy as a target; combines strategy optimization and value evaluation, efficiently explores a high-dimensional parameter space, and significantly improves the efficiency and accuracy of equation discovery, providing an efficient and reliable new paradigm for coal rock fracture evolution mechanism research.

[0153] The mechanism generation network in the exploration system is obtained by spatiotemporal feature extraction of data in the symbolic data set based on a neural network based on a selective state space architecture, and a feature extraction method for evolution mechanism symbols in the loading process of coal rock samples is proposed.

[0154] The equation coefficients of the coal rock fracture evolution mechanism equation in the exploration system are obtained by a sparse regression algorithm. The sparse regression algorithm extracts features from the fracture parameter data set, then selects the optimal equation coefficient combination of the coal rock fracture evolution mechanism equation, and automatically eliminates redundant features to reduce the model complexity, thereby realizing accurate coefficient screening in a high-dimensional parameter space; in combination with a multi-objective optimization method, an equation optimization factor is output, and through a reinforcement learning architecture, the exploration strategy of the coal rock fracture evolution mechanism equation is optimized, thereby providing a new perspective for mechanism exploration of the coal rock fracture evolution mechanism.

[0155] It should be noted that the coal rock fracture evolution mechanism construction system based on the reinforcement learning architecture of the present application can also perform any processing in the coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture as previously described, and specific details will not be described here.

[0156] The exemplary embodiments of the coal rock fracture evolution mechanism construction method and system based on the reinforcement learning architecture proposed by the present application are described in detail above with reference to preferred embodiments, however, those skilled in the art can understand that various modifications and improvements can be made to the above specific embodiments without departing from the concept of the present application, and various technical features and structures proposed by the present application can be combined without exceeding the protection scope of the present application, and the protection scope of the present application is determined by the appended claims.

Claims

1. A method for constructing a coal rock fracture evolution mechanism based on a reinforcement learning architecture, characterized in that, The method comprises the following steps: S10: arranging CT detection sensors and acoustic emission monitoring sensors on the coal rock sample, and placing the coal rock body in the testing machine for loading test, and collecting CT detection data and full-time and space acoustic emission monitoring data of the coal rock body at the stage time during the loading process through the CT detection sensors and the acoustic emission monitoring sensors; wherein the CT detection data are full-space fracture parameter data at the stage time, and the acoustic emission monitoring data are full-time and space acoustic emission parameter data; S20: corresponding the CT detection data and the full-time and space acoustic emission monitoring data at the stage time during the loading process of the coal rock body according to the time, obtaining a fracture parameter data set, and dividing the fracture parameter data set into a training set, a verification set and a test set; and manufacturing the operator symbols and variable parameters of the mechanism equation into a symbol data set; S30: reading the symbol data set by the mechanism equation generation network to obtain an initial mechanism equation; S40: obtaining each coefficient in the initial mechanism equation through sparse regression algorithm based on the acoustic emission monitoring data and the fracture parameter data in the training set and the verification set in the fracture parameter data set, and then obtaining an advanced mechanism equation and calculating a fitting error; S50: obtaining an optimization factor through the fitting error, the accuracy and the complexity of calculating the expression of the advanced mechanism equation by a multi-objective optimization algorithm; S60: inputting the optimization factor into a reinforcement learning architecture, and optimizing the generation strategy of the mechanism equation generation network through a reward function and a risk preference policy gradient function in the reinforcement learning architecture, and obtaining a fracture evolution mechanism equation; S70: obtaining a three-dimensional coal rock sample fracture evolution result through the mechanism equation, judging whether the precision of the three-dimensional coal rock sample fracture evolution result meets the requirements, and when the precision of the three-dimensional coal rock sample fracture evolution result cannot meet the requirements, calculating the deviation through the test set CT detection data and adjusting the parameters of the reinforcement learning architecture until the precision of the three-dimensional coal rock sample fracture evolution result meets the requirements.

2. The coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, wherein in the step S20, manufacturing the operator symbols of the mechanism equation into the symbol data set comprises the following steps: First, sorting the operator symbols needed for mechanism equation exploration; Then, manufacturing the operator symbols into a binary tree structure symbol data set, and the symbol data set contains the following contents: In the formula, OS represents the symbol data set; + represents the addition operator; - represents the addition operator; 3. The coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, wherein in the step S30, reading the symbol data set by the mechanism equation generation network to obtain the initial mechanism equation comprises the following steps: * represents an addition operator; / represents a division operator; 2 represents a square operator; 3 represents a cube operator; represents a first order partial derivative operator; represents a second order partial derivative operator; represents a third order partial derivative operator; AE represents acoustic emission variable parameter data; F represents fracture variable parameter data. S39: calculating the initial mechanism equation through the extracted features in the full connection layer.

4. The coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, wherein in the step S40, the expression of each coefficient in the initial mechanism equation is as follows: S31: extracting an enhanced operator feature Of from the operator symbol O m and the variable parameter S n e e ;​​ S32: the encoded data is spliced to the feature O obtained by the feature extraction layer B through a residual connection layer b and S b in the middle; S33: normalizing the operator feature Of by a first Normalization function to obtain a feature O e and the variable parameter feature Sf e to obtain a feature O n and S n ; S34: Regularized feature O n and S n Feature extraction by residual connection composed of a second linear feature extraction layer A and a second activation function layer to obtain feature O a+s and S a+s Feature O spliced to the output of the selective state space architecture sssm and S sssm O in SSSM and S SSSM ; S35: the feature O output by the first linear feature extraction layer A a and S a The feature O is obtained by feature extraction through the convolution layer and the first activation function layer conv+s and S conv+s ; S36: The high-dimensional features are calculated by a gating calculation of a selective state space architecture, and the extracted features are spliced with the features of the convolutional layer and the first activation function layer to obtain features O SSSM and S SSSM ; S37: The linear feature extraction layer B extracts features from the spliced features, and splices the extracted features with the features of the encoded data spliced by the residual connection layer to obtain the feature Of b+se and Sf b+se ; S38: normalizing the spliced features by a second Normalization function to obtain features Of n and Sf n ; 5. The coal rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, wherein ​ ​ where represents the optimal coefficients in the initial mechanism equation; represents the coefficient vector; Θ(F, AE) represents the initial equation bank, which converts the observed F(AE, t) and its derivative information into numerical characteristics, thereby realizing the coefficient vector by matrix multiplication, to obtain the prediction result of F t t represents the numerical approximation of the time derivative in the mechanism equation; the objective function represents the fitting error between the prediction and the true value; λ represents the regularization parameter, which is used to suppress the coefficient vector represents the regularization term, which ensures the simplicity of the initial mechanism equation; represents the norm square, which is used to measure the size of the coefficient vector .​ ​ In the step S50, the expression of the accuracy and complexity of the advanced mechanism equation expression is as follows: f2 = min (a k) wherein f1 represents an evaluation mechanism equation accuracy function; f2 represents an evaluation mechanism equation complexity function; N represents the total number of fitted error observations; a represents a penalty factor; represents an accuracy optimization factor; k represents a complexity optimization factor.

6. The coal and rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, characterized in that, In the step S60, the reward function expression is as follows: In the formula, R represents the reward function; β represents a hyperparameter that controls the influence of the mechanism equation complexity on the optimization factor; represents the fitting error for all observations.

7. The coal and rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, characterized in that, In the step S60, the risk preference strategy gradient function expression is as follows: J risk (θ; ∈) = E τ～p(τ|θ) [R(τ) | R(τ) ≥ R ∈ (θ)] In the formula, J risk (θ; ∈) represents the risk preference policy gradient function; θ represents the network parameter of the mechanism equation generation network; ∈ represents the risk preference parameter, which is used to screen the proportion of the high-reward advanced mechanism equation; N represents the number of current batch mechanism equations; R(τ (i) ) represents the reward value of the mechanism equation τ (i) , which is calculated by the reward function; represents the threshold value of the reward value (1-∈) in the current batch mechanism equation; 1 {·} represents the exponential function, which takes 1 when the current condition is true, and 0 otherwise; p(τ (i) |θ) represents the probability that the mechanism equation generation network parameter θ generates the mechanism equation τ (i) ; T represents the length of the mechanism equation; logp(τ (i) |θ) represents the logarithmic value of the probability; represents the partial derivative of logp(τ (i) |θ) with respect to the parameter θ.

8. The coal and rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, characterized in that, In the step S60, the expression of the generation strategy of the optimized mechanism equation generation network is as follows: where θ * denotes the updated mechanism equation generating network parameters; η denotes the learning rate.

9. The coal and rock fracture evolution mechanism construction method based on the reinforcement learning architecture according to claim 1, characterized in that, In the step S70, the bias calculation formula is as follows: where MSE represents the error calculation equation; M represents the number of samples; i represents the serial index of the fracture parameter; F M,pred represents the predicted value of the Mth sample; F M,true represents the true value of the Mth sample. 10.A system for constructing a coal rock fracture evolution mechanism based on a reinforcement learning architecture, characterized in that, including: The information acquisition module is configured to arrange CT detection sensors and acoustic emission monitoring sensors on the coal and rock sample, and place the coal and rock mass in the testing machine for loading test, and acquire the CT detection data and full-time and space acoustic emission monitoring data of the coal and rock mass at the stage time through the CT detection sensors and acoustic emission monitoring sensors; wherein the CT detection data is the full-space fracture parameter data at the stage time, and the acoustic emission monitoring data is the full-time and space acoustic emission parameter data; The data division module is configured to correspond the CT detection data and full-time and space acoustic emission monitoring data of the coal and rock mass at the stage time according to time, obtain a fracture parameter data set, and divide the fracture parameter data set into a training set, a verification set and a test set; and the operator symbol and variable parameter of the mechanism equation are made into a symbol data set; The initial mechanism equation module is configured to read the symbol data set by the mechanism equation generation network to obtain an initial mechanism equation; The advanced mechanism equation module is configured to obtain the coefficients of each term in the initial mechanism equation through sparse regression algorithm of the acoustic emission monitoring data and the fracture parameter data in the fracture parameter data set, and then obtain the advanced mechanism equation and calculate the fitting error; The optimization factor module is configured to obtain the optimization factor through the fitting error and the accuracy and complexity of the calculation of the advanced mechanism equation expression by the multi-objective optimization algorithm; The fracture evolution mechanism equation module is configured to input the optimization factor into the reinforcement learning architecture, and the reinforcement learning architecture optimizes the generation strategy of the mechanism equation generation network through the reward function and the risk preference strategy gradient function, and obtains the fracture evolution mechanism equation. The architecture parameter optimization module is configured to obtain the three-dimensional coal rock sample crack evolution result by mechanism equation solving, judge whether the accuracy of the three-dimensional coal rock sample crack evolution result meets the requirement, and when the accuracy of the three-dimensional coal rock sample crack evolution result cannot meet the requirement, carry out deviation calculation through the test set CT detection data and adjust the parameters of the reinforcement learning architecture until the accuracy of the three-dimensional coal rock sample crack evolution result meets the requirement.

Images