Self-adaptive bleeding pumping drainage control system based on reinforcement learning
By using a reinforcement learning-based adaptive bleed drainage control system, combined with an LSTM-QL-Transformer model and an improved Zebra algorithm, and dynamically adjusting parameters such as vacuum negative pressure, the dynamic adaptability problem of bleed control in concrete structures is solved, achieving efficient bleed removal and improved structural safety.
Patent Information
- Application Number
- CN202511712522.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-20
- Publication Date
- 2026-01-23
AI Technical Summary
Existing technologies are unable to dynamically adapt to the hydration process in concrete structures, leading to improper control of bleeding and the formation of capillary pores or plastic shrinkage cracks. Furthermore, existing automated systems neglect the strong coupling effect between temperature gradient and steel stress, resulting in delayed control decisions.
An adaptive bleed drainage control system based on reinforcement learning is adopted, which combines the LSTM-QL-Transformer prediction model and the improved Zebra algorithm to dynamically adjust the vacuum negative pressure, cooling water flow rate and micro-expansion slurry injection time, so as to achieve multi-objective optimization control of bleed rate, temperature and steel stress.
It significantly improves the timeliness and accuracy of water seepage control, reduces the risk of cracks, and enhances structural safety and control efficiency.
Smart Images
Figure CN121386416A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of reinforcement learning and intelligent control, and particularly relates to an adaptive bleeding pumping control system based on reinforcement learning. BACKGROUND
[0002] In the construction of concrete structures, the fine control of the bleeding phenomenon is directly related to the engineering quality and structural durability. In the early stage of hardening of freshly poured concrete, free bleeding will occur inside due to water migration and cement hydration. If not removed in time, a network of capillary pores will be formed; and excessive pumping is prone to cause plastic shrinkage cracks, especially in high temperature and strong wind environment, the risk of cracks increases by more than 40%. The traditional control method has three bottlenecks:
[0003] Firstly, it relies on manual experience to develop fixed pumping strategies, such as vacuum pump start-stop mechanism based on time threshold, which is difficult to adapt to the dynamic changes of hydration reaction process.
[0004] Secondly, the existing automatic system mostly uses single variable control logic, ignoring the strong coupling effect of temperature gradient and steel stress - when the core temperature is reduced by reducing the cooling water flow, if the vacuum negative pressure is not adjusted synchronously, it will cause the retention of surface bleeding.
[0005] Thirdly, the prediction model has insufficient accuracy, resulting in lagging control decisions. Industry research shows that the surface defect rate of large bridge piers due to improper bleeding control is more than 18%, and the repair cost accounts for 3-5% of the total project cost. Although advanced methods such as model predictive control (MPC) have been introduced, the strong nonlinearity of the multi-physical field of concrete makes accurate modeling an impossible task.
[0006] Therefore, it is urgent to build an adaptive control system with dynamic prediction ability and multi-objective decision intelligence to maximize the efficiency of bleeding removal while ensuring structural safety. SUMMARY
[0007] The present application provides an adaptive bleeding pumping control system based on reinforcement learning, which improves the real-time prediction ability of data, and dynamically adjusts the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time based on the predicted bleeding rate, temperature and steel stress, thereby improving the safety performance under the premise of ensuring the rate.
[0008] Technical scheme: The present application discloses an adaptive bleeding pumping control system based on reinforcement learning, comprising a data acquisition module, a dynamic decision module and a control module arranged in a central processing unit.
[0009] The data acquisition module is used for real-time acquisition of concrete internal bleeding rate, temperature and steel stress data.
[0010] The dynamic decision module is provided with an LSTM-QL-Transformer prediction model, which is used to predict the bleeding rate, temperature and steel stress of the concrete in a future period of time based on the historical data of the data acquisition module; the dynamic decision module also constructs a target function based on the output result of the prediction model, with the bleeding removal rate, temperature crack prevention and steel stress safety as the targets, solves the target function, and determines the optimal vacuum negative pressure, cooling water flow and micro-expansion slurry injection time;
[0011] The control module dynamically adjusts the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time based on the optimal control strategy obtained by the dynamic decision module.
[0012] Further, the establishment process of the LSTM-QL-Transformer prediction model includes the following steps:
[0013] (1.1) Construct an LSTM model, train the LSTM model using a historical data set, and learn the time-dependent pattern and short-term memory characteristics in the sequence;
[0014] (1.2) Construct a Transformer model, train the Transformer model using the same historical data set, and learn the long-term dependence relationship and global pattern in the sequence;
[0015] (1.3) Use the Q-Learning algorithm to weight and sum the prediction results of LSTM and Transformer to obtain the prediction result of each sub-sequence.
[0016] Further, the Q-Learning algorithm solves the combined model weight, and the steps are as follows:
[0017] (1.3.1) Establish a state set , represents the th state, defined as the possible prediction model; establish an action set , represents the th action, represents the expected return of each action under each state, i.e. each state is the LSTM model and the Transformer model, and each action corresponds to the weight and , establish a Q table representing the state and action, and the iteration formula of the Q table is:
[0018] ;
[0019] In the formula, is the updated value; is the action performed in the stateQ value of ; is the next state of ; the immediate reward of performing action ; ; is any one action value; is the learning factor; is the discount factor;
[0020] (1.3.2) Establishing the loss function and the reward and punishment mechanism , the formula is as follows:
[0021] ;
[0022] ;
[0023] In the formula, is the true value; is the predicted value; N is the sample number; is the loss function value obtained by the selected action.
[0024] (1.3.3) Q-Learning algorithm is used to solve the combined model weight, and the weight distribution is as follows:
[0025] ;
[0026] In the formula, is the prediction result of the LSTM model; is the prediction result weight of the LSTM model; is the prediction result weight of the Transformer model; is the prediction result weight of the Transformer model, .
[0027] Further, the objective function is as follows:
[0028] ;
[0029] In the formula, is the weight coefficient; is the target of water bleeding removal rate; is the temperature crack prevention target; is the steel stress safety target; is the control cost penalty term;
[0030] The target of water bleeding removal rate is as follows:
[0031] ;
[0032] wherein, is the predicted the water bleeding rate at time t; is the preset optimal water bleeding rate; is the prediction horizon;
[0033] The temperature crack prevention target is as follows:
[0034] ;
[0035] wherein, is the predicted the temperature at time t; is the safety temperature; is the temperature gradient penalty coefficient; is the temperature gradient suppression;
[0036] The steel stress safety target is as follows:
[0037] ;
[0038] wherein, is the predicted the steel stress at time t; is the steel stress allowable value; is the steel stress change rate; , is the weight coefficient;
[0039] The control cost penalty term is as follows:
[0040] ;
[0041] wherein, is the control variable vector; is the vacuum negative pressure intensity; is the cooling water flow rate; is the micro-expansion slurry injection time;
[0042] The constraint conditions are as follows:
[0043] ;
[0044] wherein, is the the vacuum negative pressure intensity at time t, is the minimum vacuum negative pressure, is the maximum vacuum negative pressure; is the the cooling water flow rate at time t, is the minimum cooling water, is the maximum cooling water; is the the micro-expansion slurry injection time, the minimum micro-expansion slurry injection time, the maximum micro-expansion slurry injection time; the steel bar stress safety standard.
[0045] Further, aiming at the bleeding removal rate, temperature crack prevention and steel bar stress safety, the improved zebra algorithm is used for solving, and the vacuum negative pressure, the cooling water flow and the micro-expansion slurry injection time are dynamically adjusted, including the following steps:
[0046] (2.1) Establish a zebra population and initialize to form an initial population, and generate a group of randomly distributed zebra position matrices as follows:
[0047] ;
[0048] In the formula, is the zebra population; is the th zebra; is the th position of the th zebra; is the total number of zebras in the zebra population; is the dimension of the problem to be solved, and each row of the matrix represents a control scheme, that is, a group of optimal solutions: vacuum negative pressure, cooling water flow and micro-expansion slurry injection time (value);
[0049] (2.2) The randomly generated zebra position is brought into the objective function for evaluation, and each zebra represents a group of optimal solutions, that is, the optimal vacuum negative pressure, cooling water flow and micro-expansion slurry injection time;
[0050] (2.3) Update the population members. In the zebra algorithm, the pioneer zebra leads other members to its search space in the population, that is, finds the most suitable dynamically adjusted vacuum negative pressure, cooling water flow and micro-expansion slurry injection time, and the position formula of the zebra in the foraging stage is as follows:
[0051] ;
[0052] ;
[0053] In the formula, is the new position of the th zebra in the foraging stage, is the th value thereof; is the objective function value; is the pioneer zebra, is the th value thereof; is a random number uniformly distributed between ; wherein is a random number uniformly distributed between ;
[0054] (2.4) updating the position of the zebra population member in the search space, the position of the zebra in the search space is as follows:
[0055] ;
[0056] ;
[0057] wherein, is the new position of the only zebra in the search space, is the jth dimension value thereof; is the target function value; is the current iteration number; is the maximum iteration number of the algorithm; is a constant equal to 0.01; is a random number uniformly distributed between ; is the state of the attacked zebra, is the jth dimension value thereof; (2.5) determining whether the termination condition is met, if the termination condition is not met, returning to step (2.2); if the constraint condition is met, turning to step (2.6);
[0058] (2.6) outputting the lowest target function value and the optimal vacuum negative pressure, cooling water flow and micro-expansion slurry injection time thereof.
[0059] (2.6) outputting the lowest target function value and the optimal vacuum negative pressure, cooling water flow and micro-expansion slurry injection time thereof.
[0060] Further, step (2.3) is improved, the search scale of the zebra algorithm is expanded by introducing the Cauchy mutation strategy, and the improved formula is as follows:
[0061] ;
[0062] wherein, is the new position after improvement; is a standard Cauchy distribution function; denotes multiplication.
[0063] Beneficial effects:
[0064] (1) The application combines the short-term time sequence modeling advantage of LSTM, the global feature extraction capability of Transformer and the dynamic weighting mechanism of Q-Learning, constructs a multi-modal prediction framework, and significantly improves the control timeliness.
[0065] (2) The present application takes the water bleeding removal rate, temperature crack prevention and steel stress safety as the objective function, uses the improved zebra algorithm optimization solution to determine the optimal control strategy, dynamically adjusts the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time, so as to realize the efficient operation of the system.
[0066] (3) The zebra algorithm is improved, the Cauchy mutation strategy is introduced to expand the search scale of the zebra algorithm, avoid falling into local optimum, and further improve the algorithm jumping out of local optimum ability. BRIEF DESCRIPTION OF DRAWINGS
[0067] Figure 1 It is the system structure diagram of the present application;
[0068] Figure 2 It is the system schematic diagram of the present application;
[0069] Figure 3 It is the weight allocation process diagram of Q-Learning;
[0070] Figure 4 It is the improved zebra algorithm flow chart;
[0071] Figure 5 It is the internal bleeding speed of concrete, internal temperature of concrete and steel stress measured by the data acquisition module;
[0072] Figure 6 It is the internal bleeding speed of concrete, internal temperature of concrete and steel stress data in a period of time in the future obtained by the LSTM-QL-Transformer prediction model based on historical data;
[0073] Figure 7 It is the improved zebra algorithm objective function convergence iteration diagram;
[0074] Figure 8 It is the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time optimization process diagram. DETAILED DESCRIPTION
[0075] The technical solutions of the present application will be further described below in combination with the drawings.
[0076] As Figures 1-2As shown, the application discloses an adaptive bleeding pumping control system based on reinforcement learning, which comprises a data acquisition module, a dynamic decision module, a central processing unit and a control module. The data acquisition module is used for real-time acquisition of concrete internal bleeding rate, temperature and steel stress data. The LSTM-QL-Transformer prediction model in the dynamic decision module predicts the concrete internal bleeding rate, temperature and steel stress in a future period of time based on the historical data of the data acquisition module; the objective function is based on the output result of the prediction model, and the bleeding removal rate, temperature crack prevention and steel stress safety are taken as the target, and the algorithm is used to solve the objective function. The control module is based on the optimal control strategy obtained by the dynamic decision module, and dynamically adjusts the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time.
[0077] As Figure 3 shown, the establishment process of the LSTM-QL-Transformer prediction model includes the following steps:
[0078] (1.1) Construct the LSTM model, train the LSTM model using the historical data set, and learn the time dependence pattern and short-term memory characteristics in the sequence.
[0079] The LSTM model is represented as:
[0080] The LSTM model is mainly composed of a forgetting gate, an input gate, a memory gate and an output gate. The operation formula is as follows:
[0081]
[0082]
[0083]
[0084]
[0085]
[0086]
[0087] In the formula, , , , is the information updating process of the forgetting gate, the input gate, the memory gate and the output gate, , , , is the corresponding weight matrix thereof, , , , is the corresponding bias constant; is the temporary state; sigmoid function; hyperbolic tangent function; current state of the internal bleeding rate of concrete, temperature and steel stress data; the last node of the internal bleeding rate of concrete, temperature and steel stress data; the output of the internal bleeding rate of concrete, temperature and steel stress data.
[0088] (1.2) Construct the Transformer model, train the Transformer model using the same historical data set, learn the long-term dependencies and global patterns in the sequence.
[0089] The Transformer model is established as follows:
[0090] The main body of the Transformer is composed of an encoder and a decoder. The encoder and the decoder mainly contain three parts: input, multi-head attention and fully connected feedforward neural network.
[0091] (1.2.1) Considering the time order in the input sequence, position encoding is added after word encoding. The input expression is as follows:
[0092]
[0093]
[0094] In the formula, is the position encoding matrix; is the fixed position of the time order in the input sequence; is the number of input sequences; is the model dimension.
[0095] (1.2.2) The multi-head attention part is composed of the output results of multiple self-attention. Its expression is as follows:
[0096]
[0097]
[0098] In the formula, is the attention function; is the vector converted from each element of the input sequence; is the projection matrix of the th head; The range of ; is the number of heads; is the weight matrix.
[0099] (1.2.3) The fully connected feedforward neural network is used to spatially transform the results of the multi-head attention output, including two linear transformations and a ReLU activation output. Its expression is as follows:
[0100]
[0101] wherein, is the weight; is the bias.
[0102] (1.3) The prediction results of LSTM and Transformer are weighted and summed using the Q-Learning algorithm to obtain the prediction results of each sub-sequence.
[0103] The steps of the Q-Learning algorithm for solving the combined model weight are as follows:
[0104] (1.3.1) Establish the state set , represents the th state, defined as the possible prediction model; establish the action set , represents the th action, represents the expected return of each action under each state, i.e. for each state, each action corresponding to the LSTM model and the Transformer model, i.e. the weight and , establish the Q table of the row representing the state and the column representing the action, and the iteration formula of the Q table is:
[0105] ;
[0106] wherein, is the updated value; is the Q value of performing action in state ; is the immediate return of performing action in state ; is the next state of ; is any one action value; is the learning factor; is the discount factor.
[0107] (1.3.2) Establish the loss function and the reward and punishment mechanism , the formula is as follows:
[0108] ;
[0109] ;
[0110] wherein, is the true value; is the predicted value; N is the number of samples; is the loss function value of the th selected action.
[0111] (1.3.3) The Q-Learning algorithm is used to solve the combined model weight, and the weight distribution is as follows:
[0112] ;
[0113] wherein, is the LSTM model prediction result; is the LSTM model prediction result weight; is the Transformer model prediction result weight; is the Transformer model prediction result weight, .
[0114] As shown in Figure 4 , taking the water secretion removal rate, temperature crack prevention, and steel stress safety as the target, the improved zebra algorithm is used to solve: dynamically adjusting the vacuum negative pressure, cooling water flow, and micro-expansion slurry injection time, including the following steps:
[0115] (2.1) Establish a zebra population and initialize to form an initial population. Generate a set of randomly distributed zebra position matrix as follows:
[0116]
[0117] wherein, is the zebra population; is the th zebra; is the th zebra's th dimensional position; is the total number of zebras in the zebra population; is the dimension of the problem to be solved.
[0118] (2.2) The randomly generated zebra position is brought into the objective function for evaluation. Each zebra represents a set of optimal solutions, i.e., the optimal dynamic adjustment of vacuum negative pressure, cooling water flow, and micro-expansion slurry injection time. The objective function formula is as follows:
[0119]
[0120] wherein, is the weight coefficient; is the water secretion removal rate target, unit is ; is the temperature crack prevention target, unit is ; is the steel stress safety target, unit is ; is the control cost penalty term.
[0121] The bleeding removal rate target is as follows:
[0122]
[0123] In the formula, is the predicted bleeding rate at time t, unit is ; is the preset optimal bleeding rate, unit is ; is the prediction time domain.
[0124] The temperature crack prevention target is as follows:
[0125]
[0126] In the formula, is the predicted temperature at time t, unit is ; is the safety temperature, unit is ; is the temperature gradient penalty coefficient; is the temperature gradient suppression.
[0127] The steel stress safety target is as follows:
[0128]
[0129] In the formula, is the predicted steel stress at time t, unit is ; is the steel stress allowable value, unit is ; is the steel stress change rate; , is the weight coefficient.
[0130] The control cost penalty term is as follows:
[0131]
[0132] In the formula, is the control variable vector; is the vacuum negative pressure intensity; is the cooling water flow; The micro-expansion slurry injection time.
[0133] The constraints are as follows:
[0134]
[0135] wherein, is The vacuum negative pressure intensity at the moment, is the minimum value, is the maximum value; is The cooling water flow at the moment, is the minimum value, is the maximum value; is The micro-expansion slurry injection time at the moment, is the minimum value, is the maximum value; is the steel bar stress safety standard.
[0136] (2.3) Update the population members. In the Zebra Algorithm, the pioneer zebra leads other members to its search space in the population, i.e. to find the most suitable dynamic adjustment of the vacuum negative pressure, the cooling water flow and the micro-expansion slurry injection time. Therefore, the formula for updating the position of the zebra in the foraging stage is as follows:
[0137]
[0138]
[0139] wherein, is the new position of the zebra in the foraging stage, is the dimensional value thereof; is the objective function value; is the pioneer zebra, is the dimensional value thereof; is a random number uniformly distributed between ; wherein is a random number uniformly distributed between .
[0140] (2.4) Update the position of the zebra population members in the search space. The formula for the position of the zebra in the search space is as follows:
[0141]
[0142]
[0143] wherein, For the Only zebra in search space new position, For its first The value of dimension; For the target function value; For the current number of iterations; For the maximum number of iterations of the algorithm; Is a constant equal to 0.01; Is a random number uniformly distributed between Between; For the state of the attacked zebra, For its jth dimension value.
[0144] (2.5) Determine whether the termination condition is met. If the termination condition is not met, return to step (2.2); if the constraint condition is met, go to step (2.6).
[0145] (2.6) Output the lowest target function value, i.e. the most suitable dynamic adjustment of vacuum negative pressure, cooling water flow and micro-expansion slurry injection time.
[0146] In this embodiment, step (2.3) is improved by introducing Cauchy mutation strategy to expand the search scale of zebra algorithm, thereby improving the ability of the algorithm to jump out of local optimum. The improved formula is as follows:
[0147]
[0148] Wherein, The new position after improvement; The standard Cauchy distribution function; Indicates multiplication.
[0149] The above-mentioned water bleeding and pumping control system is verified in the concrete pouring test. In the pouring and curing process of mass concrete members, the data acquisition module and the control module are arranged according to the arrangement shown in Figure 2 The vacuum negative pressure device, the cooling water circulation system and the micro-expansion slurry injection device are connected with the concrete test piece and the formwork, the steel reinforcement, respectively, to form a test platform consistent with the actual engineering conditions. The continuous monitoring data of the concrete internal bleeding rate, the concrete internal temperature and the steel stress obtained by the data acquisition module are used as the basic data source. The steel stress sensor, the temperature sensor and the bleeding rate sensor are arranged at different depths and key stress positions in the concrete, and the experimental data of 100 minutes are continuously collected as the historical data set. On this basis, the LSTM-QL-Transformer prediction model is trained and the multi-objective optimization control strategy is constructed, which is used to evaluate the control effect of the system in the actual application scene.
[0150] Figures 5-8The experimental result graph. Figure 5 The internal bleeding speed of concrete, the internal temperature of concrete and the steel stress measured by the data acquisition module, in this embodiment, are collected by the steel stress sensor, the temperature sensor and the bleeding speed sensor, and 100 minutes of experimental data are collected.
[0151] Figure 6 The internal bleeding speed of concrete, the internal temperature of concrete and the steel stress data in a future period of time obtained by the LSTM-QL-Transformer prediction model based on historical data. It can be seen that the predicted bleeding speed, temperature and steel stress change trend with time are consistent with the historical measured curves shown in Figure 5 The model can accurately depict the change process of the bleeding speed and the evolution law of the temperature and the steel stress, and the predicted curves of each variable are smooth and continuous without obvious abnormal fluctuations, indicating that the prediction model has good fitting and multi-step prediction ability for the multi-physical field coupling process, and provides a reliable input data basis for subsequent target function construction and optimization control.
[0152] Figure 7 The improved zebra algorithm target function convergence iteration graph. It can be seen that with the increase of the number of iterations, the target function value rapidly decreases in the initial stage and gradually tends to be stable, and it can converge to a lower target function value within a limited number of iterations, indicating that the improved zebra algorithm with the introduction of the Cauchy mutation strategy has a faster convergence speed and better global optimization ability.
[0153] Figure 8 The optimization process graph of vacuum negative pressure, cooling water flow and micro-expansion slurry injection time. It can be seen that the three control variables gradually converge to a stable interval from the initial random distribution in the optimization process, and the change process of the vacuum negative pressure, the cooling water flow and the micro-expansion slurry injection time all show a trend from dispersion to concentration, and finally remain near the basic stable optimal combination in the late iteration, without sharp oscillation or frequent jump, indicating that the obtained control strategy has good implementability and stability in engineering application.
[0154] The above-described embodiments only express several embodiments of the present application, and the description is more specific and detailed, but it should not be understood as limiting the scope of the present application. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.
Claims
1. A reinforcement learning based adaptive water discharge pumping control system, characterized in that, The system comprises a data acquisition module, a dynamic decision module arranged in a central processing unit, and a control module. The data acquisition module is used for real-time acquisition of concrete internal bleeding rate, temperature and steel stress data. The dynamic decision module is provided with an LSTM-QL-Transformer prediction model, which is used for prediction of the concrete internal bleeding rate, temperature and steel stress in a future period of time based on historical data of the data acquisition module. The dynamic decision module further constructs a target function based on the output result of the prediction model, with the bleeding removal rate, temperature crack prevention and steel stress safety as the targets, solves the target function, and determines the optimal vacuum negative pressure, cooling water flow and micro-expansion slurry injection time. The control module dynamically adjusts the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time based on the optimal control strategy obtained by the dynamic decision module.
2. The adaptive water excretion pumping control system based on reinforcement learning according to claim 1, wherein, The establishment process of the LSTM-QL-Transformer prediction model comprises the following steps: (1.1) Construct an LSTM model, train the LSTM model using a historical data set, and learn the time dependence pattern and short-term memory characteristics in the sequence. (1.2) Construct a Transformer model, train the Transformer model using the same historical data set, and learn the long-term dependence relationship and global pattern in the sequence. (1.3) Use the Q-Learning algorithm to weight and sum the prediction results of LSTM and Transformer to obtain the prediction result of each sub-sequence.
3. The adaptive water excretion pumping control system based on reinforcement learning according to claim 2, wherein, The Q-Learning algorithm solves the combined model weight, and the steps are as follows: (1.3.1) Set up state set , represents the first state, defined as the possible prediction model; Set up action set , represents the first action, represents the expected return of taking each action under each state, that is, for each state, the LSTM model and the Transformer model, each action corresponds to the weight and , and the row represents the state and the list represents the action Q table, and the iteration formula of the Q table is: ； wherein, is the updated value; is the state at action Q-value; is the immediate reward for action in state ; is the next state of ; is any one action value; is the learning factor; is the discount factor; (1.3.2) Establishing loss function and reward mechanism , as follows: ； ； In the formula, is a true value; is a predicted value; N is the number of samples; is the loss function value obtained by the action selected for the time. (1.3.3) The Q-Learning algorithm solves the combined model weight, and the weight distribution is as follows: ； In the formula, is the LSTM model prediction result; is the LSTM model prediction result weight; is the Transformer model prediction result weight; is the Transformer model prediction result weight, .
4. The adaptive water discharge pumping control system based on reinforcement learning according to claim 1, wherein, The target function is as follows: ； In the formula, is a weight coefficient; is a bleeding removal rate target; is a temperature crack prevention target; is a steel bar stress safety target; is a control cost penalty term; The bleeding removal rate target is as follows: ； In the formula, is a predicted time instants; is a preset optimal bleeding rate; is a prediction horizon; The temperature crack prevention target is as follows: ； wherein is a predicted is a temperature at time instant is a safety temperature is a temperature gradient penalty coefficient is a temperature gradient suppression The steel stress safety target is as follows: ； In the formula, is the predicted is the time instant of the reinforcement stress; is the reinforcement stress allowable value; is the reinforcement stress rate of change; , is the weight coefficient; The control cost penalty term is as follows: ； wherein is a control variable vector; is a vacuum negative pressure intensity; is a cooling water flow rate; is a micro-expansion slurry injection time; The constraint condition is as follows: ； In the formula, is the vacuum negative pressure intensity at the moment, is the minimum value of the vacuum negative pressure, is the maximum value of the vacuum negative pressure; is the cooling water flow at the moment, is the minimum value of the cooling water, is the maximum value of the cooling water; is the micro-expansion slurry injection time at the moment, is the minimum value of the micro-expansion slurry injection time, is the maximum value of the micro-expansion slurry injection time; is the steel bar stress safety standard.
5. The adaptive water discharge pumping control system based on reinforcement learning according to claim 4, wherein, With the bleeding removal rate, temperature crack prevention and steel stress safety as the targets, the improved zebra algorithm is used to solve the dynamic adjustment of the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time, comprising the following steps: (2.1) Establish a zebra population and initialize to form an initial population, and generate a random distribution of zebra position matrix as follows: ； wherein is the zebra population; is the is the zebra; is the is the is the z position; is the total number of zebras in the zebra population; is the dimension of the problem to be solved, each row of the matrix represents a control scheme, i.e. a set of optimal solutions: vacuum underpressure, cooling water flow and micro-expanded slurry injection time (values); (2.2) The randomly generated zebra position is brought into the target function for evaluation, and each zebra represents a set of optimal solutions, i.e. the optimal vacuum negative pressure, cooling water flow and micro-expansion slurry injection time; (2.3) Update the population members. In the zebra algorithm, the pioneer zebra leads other members to its search space in the population, i.e. finds the most suitable dynamic adjustment of the vacuum negative pressure, cooling water flow and micro-expansion slurry injection time, and the position formula of the zebra in the foraging stage is as follows: ； ； wherein is the first is the new position of the only zebra at the foraging stage, is the first dimension value; is the objective function value; is the pioneer zebra, is the first dimension value; is a random number uniformly distributed between 0 and 1; is a random number uniformly distributed between 0 and 1; wherein is a random number uniformly distributed between 0 and 1; is a random number uniformly distributed between 0 and 1; (2.4) Update the position of the zebra population members in the search space. The position formula of the zebra in the search space is as follows: ； ； wherein, is the first Only zebras in the search space of the new location, is the first dimension value; is the target function value; is the current iteration number; is the maximum iteration number of the algorithm; is a constant equal to 0.01; is a random number uniformly distributed between ; is the state of the attacked zebra, is the jthdimension value thereof; (2.5) Determine whether the termination condition is met. If the termination condition is not met, return to step (2.2); if the constraint condition is met, go to step (2.6); (2.6) Output the minimum objective function value and its optimal vacuum negative pressure, cooling water flow rate and micro-expansion slurry injection time.
6. The adaptive water excretion pumping control system based on reinforcement learning according to claim 5, wherein, The step (2.3) is improved by introducing Cauchy mutation strategy to expand the search scale of the Zebra algorithm, and the improved formula is as follows: ； wherein is the improved new position; is a standard Cauchy distribution function; denotes multiplication.