A method and equipment for controlling the forming process of fly ash autoclaved bricks

Abstract

This application provides a control method and system for the forming process of fly ash autoclaved bricks, relating to the technical field of fly ash autoclaved bricks. The method includes: firstly, collecting historical parameters of each process step in the forming process of different batches, using them as the basis for constructing the state space, constructing the state space, and integrating the adjustment amounts of the corresponding historical parameters into the action space; estimating the state transition probability based on a statistical analysis algorithm, and designing a reward function by combining the brick blank qualification rate and energy consumption target, thus constructing an MDP model; secondly, collecting the parameter values ​​of the current process step in real time, using them as the initial state input to a reinforcement learning algorithm, solving the MDP model through iterative optimization, outputting the optimal parameters that maximize the long-term cumulative reward, and calculating the difference between the optimal parameters and the real-time parameters, using this difference as the adjustment input of a PID controller to dynamically adjust the output of actuators such as valve opening and pressure setpoint, thereby achieving closed-loop control of process parameters.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of fly ash autoclaved bricks, and in particular to a control method and equipment for a fly ash autoclaved brick forming process. BACKGROUND

[0002] As a new type of wall material, fly ash autoclaved bricks play an important role in the building industrialization process due to the resource utilization of industrial waste (fly ash) and low-carbon environmental protection characteristics. The fly ash autoclaved brick forming process involves strong coupling of multiple parameters, including raw material moisture content, mixing time, forming pressure, autoclaving temperature, etc., and the process has nonlinear dynamic characteristics. A PID controller adjusts the equipment parameters through proportional (P), integral (I) and derivative (D) gain to realize closed-loop control of key parameters such as forming pressure and mixing speed, but the dynamic wear and environmental disturbances (such as mold gap expansion and autoclaving kettle sealing deterioration) of the fly ash autoclaved brick equipment will cause time-varying drift of the process parameter-quality relationship, and the PID controller gain needs to be dynamically adjusted to maintain control performance.

[0003] For example, in terms of forming pressure control, the PID controller dynamically adjusts the pressure with the brick density deviation as input, the proportional term quickly responds to the density deviation, the integral term eliminates the steady-state error, and the derivative term suppresses the overshoot. However, the fly ash autoclaved brick equipment has dynamic wear and environmental disturbances such as mold gap expansion and autoclaving kettle sealing deterioration, which will cause time-varying drift of the process parameter-quality relationship, and thus the PID controller gain needs to be dynamically adjusted to maintain control performance. However, the response delay of the derivative term of the PID controller to changes in the mold gap or autoclaving kettle sealing will cause pressure regulation lag and prolong the time for the brick density to reach a stable value. SUMMARY

[0004] To realize prediction of the trend of changes in parameters such as mold gap and adjust the PID gain in advance, the application provides a control method and equipment for a fly ash autoclaved brick forming process.

[0005] In a first aspect, the application provides a control method for a fly ash autoclaved brick forming process, which adopts the following technical solution:

[0006] A control method for a fly ash autoclaved brick forming process, comprising the following steps:

[0007] Modeling: Obtain historical parameters of each process step in different fly ash autoclaved brick forming processes, use the historical parameters of each process step in the same fly ash autoclaved brick forming process as the state, integrate all states into a state space, use the adjustment amount of each historical parameter as the action, integrate all actions into an action space, set the state transition probability based on all historical parameters using a statistical analysis algorithm, set a reward function according to the qualified rate of the brick and energy consumption, and establish an MDP model;

[0008] Solving: obtaining real-time parameters of the current process step, solving the MDP model based on the real-time parameters using a reinforcement learning algorithm, obtaining a first parameter that maximizes the value of the reward function, and calculating the difference between the first parameter and the real-time parameter;

[0009] Control: adjusting the real-time parameter according to the difference.

[0010] The present application realizes intelligent control of various parameters in the forming process of fly ash autoclaved brick through MDP modeling and reinforcement learning. The present application first takes the historical parameters (such as pressure and temperature) of each process step as the state and the parameter adjustment amount as the action, implicitly captures the process dynamic law by combining statistical analysis algorithm (such as Markov chain), and guides optimization through multi-objective reward function (considering yield rate and energy consumption); then, according to the state space characteristics (continuous / discrete), the reinforcement learning algorithm (such as DDPG / DQN) is selected, the real-time parameter is input into the MDP model to output the predicted optimal control strategy (i.e. the first parameter); finally, the difference between the first parameter and the real-time parameter is calculated to drive the PID controller to adjust. The present application can predict the difference of each parameter based on historical data and real-time feedback, and control the PID controller to adjust the parameters in advance according to the difference, so as to continuously improve the control strategy and reduce the drift problem of PID controller gain caused by nonlinearity.

[0011] Optionally, the method further comprises:

[0012] Calculating the Euclidean distance of the real-time parameter and the first parameter, denoted as first data; obtaining a second parameter that minimizes the value of the reward function, calculating the Euclidean distance of the real-time parameter and the second parameter, denoted as second data;

[0013] Determining whether the first data is greater than the second data, if yes, executing the control step; if no, outputting an alarm signal.

[0014] The present application significantly enhances the safety and robustness of the fly ash autoclaved brick forming control system by introducing the Euclidean distance comparison and alarm decision mechanism between the real-time parameter and the optimal / worst parameter: by calculating the distance difference between the current parameter and the reinforcement learning optimal solution (first parameter) and the reward function worst solution (second parameter), the present application can indirectly determine whether the working condition needs to be alarmed, and trigger the alarm when deviating from the optimal solution and significantly deteriorating to reduce the occurrence of false control.

[0015] Optionally, before executing the control step, the method further comprises:

[0016] Calculating the decay rate: calculating the decay rate of the reward function and setting the normal value range of the decay rate, determining whether the decay rate at the current time is within the normal value range, if yes, executing the control step, if no, outputting a warning signal;

[0017] the decay rate at the current time The calculation model is as follows:

[0018] ;

[0019] wherein, is the numerical value of the reward function at the current time; is the average value of the reward function in a preset time period before the current time.

[0020] The application calculates the decay rate, which represents the relative performance change of the current control strategy relative to the historical average level. If the decay rate is within the preset normal range, it means that the control strategy is effective, and the step of executing control and adjusting real-time parameters is continued. If the decay rate exceeds the normal value range, a warning signal is triggered, prompting that the MDP model may be invalid or the process environment has mutated. By quantifying the performance change of the MDP model, the robustness of the MDP model is significantly improved.

[0021] Optionally, after the step of executing control, the method further comprises:

[0022] counting the number of times of issuing a warning signal in a time period T after adjusting the real-time parameter, denoted as a third data; counting the number of times of issuing a warning signal in n time periods T before adjusting the real-time parameter, denoted as a fourth data, and determining whether the third data is normal based on the fourth data using a rule. If yes, no processing is performed; if no, a warning signal is output.

[0023] By comparing the frequency change of the warning signal before and after adjustment, the effectiveness of the adjustment action is quantitatively evaluated, and the dependence on a single adjustment result is reduced. The application adds a secondary warning mechanism based on the original parameter adjustment, forming a closed loop of "adjustment-evaluation-re-warning", which is especially suitable for high-risk processes and improves the safety of the production process.

[0024] Optionally, the third data and the fourth data are the number of warnings in the same process step in the fly ash autoclaved brick forming process.

[0025] By adopting the above technical solution, the application can compare the number of warnings before and after adjustment in the same process step, and then evaluate the local effect of the adjustment action in real time, thereby avoiding the risk spread caused by global statistical delay and improving the fault positioning accuracy and risk intervention timeliness.

[0026] Optionally, the method further comprises:

[0027] The MDP model is decoupled into multiple MDP sub-models, each corresponding to a different process step in the fly ash autoclaved brick forming process, a Monte Carlo tree search algorithm is used to select the MDP sub-models that need to be activated, the MDP sub-models that need to be activated are integrated into a new MDP model according to the order of occurrence in the fly ash autoclaved brick forming process, and the solving steps are executed.

[0028] The application disassembles the complex control tasks (such as raw material mixing, pressing forming, steam curing, etc.) of the fly ash autoclaved brick forming process into multiple independent MDP sub-models, each corresponding to a single process step, containing state space (such as humidity, pressure, temperature), action space (such as parameter adjustment) and reward function (such as quality pass rate, energy consumption). The MDP sub-models of the application only focus on local process constraints (such as the dynamic of the hydraulic system in the pressing step), which can reduce the global state space explosion (such as the joint state dimension of the mixing + pressing + drying step being too high). Subsequently, the application dynamically selects the sub-models that need to be activated (such as preferentially activating the humidity control sub-model rather than the steam curing sub-model) according to the current process state (such as the humidity deviation in the mixing step, the pressure fluctuation in the pressing step) through the Monte Carlo tree search algorithm, preferentially activates the sub-models that have the greatest impact on the current process state (such as preferentially activating the mixing step control sub-model when the humidity is abnormal), and reduces invalid calculations. Subsequently, the application reorganizes the activated sub-models into a new MDP model according to the physical timing of the fly ash autoclaved brick forming process (such as mixing → pressing → drying), so that the control logic meets the requirements of the process flow. The reorganized MDP model is executed to solve the reinforcement learning (such as policy gradient method, Q-learning), and the optimal control strategy is generated.

[0029] Optionally, the method further comprises: formulating a plurality of adjustment strategies according to the difference, optimizing the adjustment strategies according to a genetic algorithm, constructing a fitness function of the genetic algorithm according to the fly ash autoclaved brick forming efficiency, the brick billet pass rate and the energy consumption, iteratively screening the adjustment strategy that maximizes the fitness function value, and recording it as the optimal condition strategy; the adjustment strategy includes: adjustment amplitude, adjustment sequence and adjustment period corresponding to the PID controller, and the adjustment amplitude is equal to the size of the difference;

[0030] In the control step, the corresponding PID controller is controlled to adjust the real-time parameters based on the optimal adjustment strategy.

[0031] The application can generate a multi-dimensional adjustment strategy based on the deviation of real-time monitoring parameters (such as humidity, pressure, temperature) from target values (such as humidity difference ΔH = target humidity - actual humidity), covering the adjustment range, adjustment sequence and adjustment cycle of the PID controller. Subsequently, by combining different dimensions of adjustment parameters (such as humidity adjustment range ± 5%, pressure adjustment sequence front, adjustment cycle shortened to 5 seconds), a strategy pool is formed. The application can iteratively optimize the strategy pool by selecting, crossing and mutating, retaining high fitness strategies and eliminating low efficiency strategies. After multiple generations of genetic iteration, the strategy with the maximum fitness function value is selected as the optimal condition strategy (such as humidity adjustment range ± 3%, pressure adjustment sequence rear, adjustment cycle 8 seconds), and the adjustment range, sequence and cycle parameters of the optimal strategy are written into the PID controller to real-time adjust the actuator (such as variable frequency motor, steam valve), forming a closed loop control of "difference → strategy → PID → execution".

[0032] Optionally, the step of controlling further comprises:

[0033] In the process of adjusting the real-time parameter, the energy consumption of each PID controller is detected in real time, and if the energy consumption of any one PID controller exceeds the preset energy consumption threshold, an alarm signal is sent, otherwise, the step is re-executed.

[0034] By adopting the above technical solution, the application can early warn potential faults when the energy consumption of the hydraulic system or motor increases due to aging, and reduce unplanned downtime.

[0035] Optionally, the method further comprises: based on historical parameters, using a statistical analysis algorithm to obtain an adjustment step of a process step corresponding to the real-time parameter;

[0036] In the step of controlling, the corresponding PID controller is controlled to adjust the real-time parameter according to the adjustment step based on the optimal adjustment strategy.

[0037] By analyzing historical parameters, the application can obtain an adjustment step adapted to different process steps, reducing overshoot or oscillation caused by fixed step length. In the scenario of raw material fluctuation or equipment noise, the adjustment step can dynamically adapt to the parameter change rate, reducing the overshoot of the PID control. By limiting the adjustment step, the energy waste caused by high-frequency large adjustment is reduced, and the adjustment range and step length in the optimal adjustment strategy are complementary, balancing the adjustment speed and stability.

[0038] In a second aspect, the application provides a control device for the forming process of fly ash autoclaved brick, which adopts the following technical solution:

[0039] A control device for the forming process of fly ash autoclaved brick, the control device is used to execute the control method, and the control device further comprises:

[0040] The modeling module is configured to obtain historical parameters of each process step in the forming process of different fly ash autoclaved bricks, take the historical parameters of each process step in the forming process of the same fly ash autoclaved brick as a state, integrate all the states into a state space, take the adjustment amount of each historical parameter as an action, integrate all the actions into an action space, set a state transition probability based on all the historical parameters by using a statistical analysis algorithm, set a reward function according to the qualified rate of the brick and the energy consumption, and establish an MDP model.

[0041] The solving module is configured to obtain real-time parameters of a current process step, solve the MDP model by using a reinforcement learning algorithm based on the real-time parameters, obtain a first parameter that maximizes the value of the reward function, and calculate a difference between the first parameter and the real-time parameters.

[0042] The control module is configured to control a PID controller to adjust the real-time parameters according to the difference.

[0043] In summary, the present application has at least one of the following beneficial technical effects:

[0044] 1. The present application realizes intelligent control of various parameters in the forming process of fly ash autoclaved bricks by MDP modeling and reinforcement learning. The present application firstly takes historical parameters (such as pressure and temperature) of each process step as a state and an adjustment amount of the parameters as an action, implicitly captures the process dynamic law by combining a statistical analysis algorithm (such as Markov chain), and guides optimization through a multi-objective reward function (considering the qualified rate and the energy consumption); then, according to the characteristics of the state space (continuous / discrete), a reinforcement learning algorithm (such as DDPG / DQN) is selected, real-time parameters are input into the MDP model to output a predicted optimal control strategy (i.e., a first parameter); finally, by calculating the difference between the first parameter and the real-time parameters, a PID controller is driven to adjust, the present application can predict the difference of each parameter based on historical data and real-time feedback, and according to the difference, the PID controller is controlled in advance to adjust the parameters, so as to continuously improve the control strategy and reduce the drift problem of the gain of the PID controller caused by nonlinearity.

[0045] 2. The present application significantly enhances the safety and robustness of the fly ash autoclaved brick forming control system by introducing a Euclidean distance comparison between the real-time parameters and the optimal / worst parameters and an alarm decision mechanism: by calculating the distance difference between the current parameters and the optimal solution (a first parameter) of the reinforcement learning and the worst solution (a second parameter) of the reward function, the present application can indirectly determine whether the working condition needs to be alarmed, and when the optimal solution is deviated and significantly deteriorated, the alarm is triggered to reduce the occurrence of false control. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 FIG. 1 is a flowchart of S1 modeling to S3 control in Embodiment 1 of the present application. DETAILED DESCRIPTION

[0047] The application is further described in detail below with reference to the accompanying drawings of the specification.

[0048] Embodiment 1: The embodiment discloses a control method for a fly ash autoclaved brick forming process, referring to Figure 1 , the control method comprises: S1 modeling, S2 solving and S3 controlling, first, historical parameters of each process step in different batches of forming processes are collected, which are taken as states to construct a state space, and adjustment amounts of the corresponding historical parameters are integrated into an action space, a state transition probability is estimated based on a statistical analysis algorithm, a reward function is designed in combination with a brick billet qualification rate and an energy consumption target, a Markov decision process (MDP) model is constructed, parameter values of a current process step are collected in real time, which are taken as an initial state and input into a reinforcement learning algorithm, the MDP model is solved through iterative optimization, an optimal parameter that maximizes a long-term cumulative reward is output, and a difference between the optimal parameter and a real-time parameter is calculated, the difference is taken as an adjustment input of a PID controller, and a valve opening degree, a pressure set value and other actuator outputs are dynamically adjusted to realize closed-loop control of process parameters, and the embodiment comprises the following steps:

[0049] S1 modeling, historical parameters of each process step in a fly ash autoclaved brick forming process are collected, including but not limited to:

[0050] a mixing stage: components of fly ash, a fly ash and binder ratio, a stirring speed, and a mixing time;

[0051] a pressing stage: a pressing pressure, a pressing speed, and a pressure maintaining time;

[0052] a curing stage: a steam temperature, a steam pressure, a curing time length, and a humidity gradient.

[0053] The historical parameters are from PLC records, sensor logs, quality detection reports and the like.

[0054] Parameters of each step in the same batch of forming processes are combined as a state vector, all possible state combinations are mapped to a state space, adjustment amounts of each historical parameter are taken as actions, and all actions are integrated into an action space.

[0055] Based on the historical parameter sequence, the embodiment can estimate the state transition probability in the following methods:

[0056] a frequency statistical method: the frequency of transition to a next state after a state executes an action in historical data is counted, and the result after normalization processing is recorded as a probability.

[0057] a hidden Markov model (HMM): if a state cannot be completely observed (for example, some parameters are missing), a potential state transition is modeled through the HMM.

[0058] Time series analysis: ARIMA modeling on continuous parameter sequences (such as pressure) to predict their dynamic change patterns.

[0059] Establish a MDP model based on the reward function set according to the qualified rate and energy consumption of the brick.

[0060] S2 solution, get the real-time parameters corresponding to the historical parameter categories in S1 modeling, and solve the MDP model based on real-time parameters using reinforcement learning algorithm.

[0061] Markov Decision Process (MDP) model is a mathematical model used to describe problems with uncertainty and sequential decision-making. It consists of state, action, state transition probability and reward function. In the context of process steps, the state is the state space composed of current real-time parameters, the action is the adjustment operation on process parameters, the state transition probability describes the possibility of moving to other states after taking a certain action in the current state, and the reward function is used to evaluate the reward obtained after taking a certain action in a certain state.

[0062] Reinforcement learning is a machine learning method that aims to let the agent find the optimal behavior policy to maximize long-term rewards by constantly trying and learning in the environment. Common reinforcement learning algorithms include Q-learning, Deep Q Network (DQN), Policy Gradient, etc. Based on real-time parameters, the agent will choose an action (adjustment scheme for process parameters) according to the current state (state composed of real-time parameters), then observe the feedback of the environment (process system), i.e. the change of state and the reward obtained. Through continuous iteration of this process, the agent gradually learns the optimal strategy, so that in the MDP model it can maximize the value of the reward function.

[0063] In the learning process of reinforcement learning algorithm, the agent will constantly try different actions (adjustment of process parameters) and record the rewards brought by each action. As the learning progresses, the agent will gradually find a set of optimal action sequences, and the process parameter values corresponding to these actions are the parameter values that maximize the value of the reward function, i.e. the first parameter.

[0064] After obtaining the first parameter that maximizes the reward function, it is compared with the real-time parameter of the current process step. The difference between each parameter is calculated, for example, if the temperature value in the first parameter is T1 and the temperature value in the real-time parameter is T2, the temperature difference is T1-T2. By calculating the difference, the gap between the current process parameter and the optimal parameter can be understood, thereby providing a basis for subsequent process adjustment. If the difference is large, it means that the current process has a large optimization space, and the adjustment direction and amplitude of the process parameter can be determined according to the positive and negative and size of the difference to gradually approach the optimal process state.

[0065] S3 control, control the PID controller to adjust the real-time parameter according to the difference obtained in S2 solving.

[0066] The following will be described in conjunction with a specific case.

[0067] S1 modeling

[0068] Historical parameter collection: In the whole process of fly ash autoclaved brick forming, various historical parameters are collected from different process steps.

[0069] In the mixing stage, the composition of fly ash (such as the content of silicon dioxide, aluminum oxide, etc.), the ratio of fly ash to binder (for example, 10:1 ratio), the stirring speed (such as revolutions per minute), and the mixing time (in minutes) are collected. In the pressing stage, the pressing pressure (units may be megapascals), the pressing speed (displacement amount per unit time), and the holding time (minutes) are focused on.

[0070] In the curing stage, the steam temperature (degrees Celsius), the steam pressure (megapascals), the curing time (hours), and the humidity gradient (humidity change per meter) are recorded. These parameters are widely sourced, PLC (Programmable Logic Controller) records parameters during equipment operation, sensor logs detail real-time data measured by sensors, and quality inspection reports contain parameter information related to product quality.

[0071] The parameters of each step in the same batch forming process are combined as a state vector. For example, a state vector may be [fly ash composition: silicon dioxide 60%, aluminum oxide 20%...; fly ash to binder ratio 8:1; stirring speed 100 revolutions / minute; mixing time 5 minutes; pressing pressure 10 megapascals; pressing speed 0.5 millimeters / second; holding time 3 minutes; steam temperature 120 degrees Celsius; steam pressure 0.8 megapascals; curing time 10 hours; humidity gradient 0.5% / meter]. All such possible state combinations constitute a state space. The adjustment amount of each historical parameter (such as an increase of 0.1 in the ratio of fly ash to binder, an increase of 1 megapascal in the pressing pressure, etc.) is regarded as an action, and all these actions are integrated to form an action space.

[0072] State transition probability estimation methods include:

[0073] Frequency statistics: By counting the frequency of state transitions after performing a certain action in historical data, and then normalizing the frequency to obtain the state transition probability. For example, in 100 historical records, state A performs action a and transitions to state B 20 times, then the probability of state A performing action a to transition to state B is 20 ÷ 100 = 0.2.

[0074] Hidden Markov Model (HMM): When there are missing parameters that cause the state to be incompletely observed, use HMM to model the potential state transition. For example, in some records, the humidity gradient parameter of the curing stage is missing, but the state transition can be inferred from other observable parameters (such as steam temperature, pressure, etc.) through HMM.

[0075] Time series analysis: For continuous parameter sequences such as pressure, use ARIMA (Autoregressive Integrated Moving Average Model) modeling to predict the dynamic change rule. For example, by modeling the historical pressing pressure data with ARIMA, the trend of pressure change in the future pressing process can be predicted.

[0076] According to the qualified rate of brick and energy consumption, set up a reward function. If the qualified rate of brick is high and the energy consumption is low, give a higher reward value; otherwise, give a lower reward value. For example, when the qualified rate of brick reaches 90% or more and the energy consumption is lower than a certain standard, the reward value is 10; if the qualified rate is less than 80% or the energy consumption is too high, the reward value is -5. Combined with state, action, state transition probability and reward function, a Markov Decision Process (MDP) model is established.

[0077] S2 solution

[0078] Get real-time parameters corresponding to the historical parameter categories in S1 modeling, which reflect the actual state of the current fly ash autoclaved brick forming process. For example, the real-time fly ash composition in the mixing stage, the real-time fly ash and binder ratio, etc.

[0079] Solve the MDP model based on real-time parameters using common reinforcement learning algorithms such as Q-learning, DQN, or policy gradient algorithm, etc. The agent selects an action (such as adjusting the pressing pressure) from the action space based on the current state composed of real-time parameters, then observes the feedback of the process system, i.e. the change of state (such as the change of brick body density after pressing) and the reward obtained (calculate the reward value according to the pass rate and energy consumption of the brick body in the new state). Through the continuous repetition of this process of selecting actions, observing feedback, and learning, the agent gradually finds a set of optimal action sequences, and the process parameter values corresponding to these actions are the first parameters that maximize the reward function value. For example, after multiple iterations of learning, it is found that when the pressing pressure is 12 MPa and the pressing speed is 0.4 mm / s, the reward function value is maximum.

[0080] Compare the obtained first parameters that maximize the reward function with the current real-time parameters. Calculate the difference between each parameter, such as the pressing pressure in the first parameter is 12 MPa and the pressing speed is 0.4 mm / s, and the pressing pressure in the real-time parameter is 10 MPa and the pressing speed is 0.4 mm / s, then the difference of the pressing pressure is 2 MPa.

[0081] S3 control, according to the difference obtained in S2 solving, control the PID (proportional-integral-derivative) controller to adjust the real-time parameters.

[0082] The PID controller adjusts the process parameters by outputting appropriate control signals through proportional, integral, and differential operations according to the size and trend of the difference. For example, if the pressing pressure difference is positive, it means that the real-time pressing pressure is lower than the optimal pressing pressure, and the PID controller will output a control signal to increase the pressing pressure, gradually making the real-time parameters close to the optimal parameters, and achieving optimal control of the fly ash autoclaved brick forming process.

[0083] In other embodiments, the method further comprises:

[0084] Calculate the Euclidean distance between the real-time parameters and the first parameters, denoted as the first data; by reverse exploration (such as modifying the reward function to negative incentive) or sampling strategy, find the second parameters that minimize the value of the reward function, calculate the Euclidean distance between the real-time parameters and the second parameters, denoted as the second data.

[0085] Determine whether the first data is greater than the second data, if yes, execute S3 control; if not, output an alarm signal.

[0086] In other embodiments, although the real-time parameters are close to the ideal parameters, the decay rate of the reward function is large, at this time it is determined that the process mismatch phenomenon occurs, therefore before executing S3 control, the method further comprises:

[0087] S4 calculates the decay rate, calculates the decay rate of the reward function and sets the normal value range of the decay rate, judges whether the decay rate at the current time is within the normal value range, if yes, executes S3 control, if not, outputs a warning signal.

[0088] the decay rate at the current time The calculation model is as follows:

[0089]

[0090] Among them, is the value of the reward function at the current time; is the average value of the reward function in the preset time length before the current time.

[0091] The positive decay rate means that the current reward is lower than the historical average, indicating that the process performance is declining (such as lower yield or higher energy consumption); the negative decay rate means that the current reward is higher than the historical average, indicating that the process performance is improved; the zero decay rate means that the current reward is flat with the historical average.

[0092] After executing S3 control, the method further comprises:

[0093] Counting the number of warning signals sent within the time T (the time T is a time interval set according to the characteristics of the fly ash autoclaved brick production process and actual needs. For example, considering that the impact of process parameter adjustment on the production process may take some time to appear, T can be set to 30 minutes, 1 hour, etc. The length of T should be able to reasonably reflect the changes in process state after parameter adjustment) after the adjustment of real-time parameters, and recording the number as the third data; maintaining a historical warning queue with a length of n x T, and adding the number of warning signals in the time T to the end of the queue when the new time T ends, while removing the data at the beginning of the queue (i.e. the number of warnings in the earliest time T). In this way, the queue always maintains the latest warning signal number information in the last n time T. For example, initially, the queue has the number of warnings in the past 3 hours as 2, 1, and 3 respectively. When a new hour passes, the number of warnings in this hour is 4, so the queue is updated to 1, 3, and 4.

[0094] Counting the number of warning signals sent within n time T before each adjustment of real-time parameters respectively, and recording the number as the fourth data, the third data and the fourth data are the number of warnings in the same process step in the fly ash autoclaved brick forming process.

[0095] Based on the fourth data, the third data is judged to be normal or not using the rule, if yes, no processing is done; if not, a warning signal is output.

[0096] ​Embodiment 2: The difference between this embodiment and embodiment 1 is that the method further comprises:

[0097] The MDP model is decoupled into multiple MDP sub-models, each corresponding to a different process step in the process of forming fly ash autoclaved brick. According to the process flow of fly ash autoclaved brick forming, the process steps corresponding to each MDP sub-model are clearly divided. For example, all relevant parameters of the mixing stage (such as the composition of fly ash, the ratio of fly ash to binder, stirring speed, mixing time, etc.) are taken as state variables of an MDP sub-model, and the operations that can be performed in this stage (such as adjusting the ratio, changing the stirring speed, etc.) are taken as action variables to construct the MDP sub-model of the mixing stage. Similarly, the MDP sub-models of the pressing stage and the curing stage are constructed respectively.

[0098] Then the Monte Carlo tree search algorithm is used to select the MDP sub-models that need to be activated, including:

[0099] Selection stage: based on the UCT (Upper Confidence Bound for Trees) algorithm, high-value nodes (such as the steam pressure fluctuation rate of the curing stage) are preferentially explored;

[0100] Expansion stage: when the sub-model state deviates from the expected value (such as a brick blank density deviation > 3%), the sub-model is dynamically expanded;

[0101] Simulation stage: the cumulative reward of different sub-model combinations is evaluated through fast Monte Carlo simulation;

[0102] Backtracking stage: update the visit count and value estimate of each sub-model to provide a basis for decision-making at the next time.

[0103] The activated sub-models are connected in the order MDP1→MDP2→MDP3→MDP4 to form a global MDP model, and S2 is solved.

[0104] According to the difference, a plurality of adjustment strategies are formulated, the adjustment strategies are optimized according to a genetic algorithm, a fitness function of the genetic algorithm is constructed according to the forming efficiency of fly ash autoclaved brick, the qualified rate of brick blanks, and energy consumption, the adjustment strategy that maximizes the fitness function value is iteratively selected and is recorded as the optimal condition strategy; the adjustment strategy includes an adjustment amplitude, an adjustment sequence, and an adjustment period corresponding to the PID controller, and the size of the adjustment amplitude is equal to the size of the difference.

[0105] Based on historical parameters, a statistical analysis algorithm is used to obtain the adjustment step of the process step corresponding to the real-time parameter.

[0106] In the S3 control, the optimal adjustment strategy is used to control the corresponding PID controller to adjust the real-time parameter according to the adjustment step.

[0107] In other embodiments, the S3 control further comprises: detecting the energy consumption of each PID controller in real time during the adjustment of the real-time parameters, and if the energy consumption of any one PID controller exceeds a preset energy consumption threshold, an alarm signal is sent, otherwise, the step is re-executed.

[0108] The embodiment decouples the fly ash autoclaved brick forming process into four independent sub-models of raw material pretreatment, forming and pressing, steam curing, and finished product detection, dynamically activates and integrates them into a global MDP model according to the process time sequence, generates an initial adjustment strategy based on the target-actual difference, optimizes the strategy parameters (such as PID adjustment amplitude, sequence, and period) using a genetic algorithm with forming efficiency, brick billet qualification rate, and unit energy consumption as the fitness function, determines the real-time adjustment step based on historical data statistical analysis, drives the PID controller to realize parameter closed-loop adjustment through the optimal strategy, and introduces a real-time energy consumption monitoring mechanism to trigger an alarm and rollback control when the energy consumption of any PID exceeds the limit, and finally forms an adaptive intelligent control system covering the whole process.

[0109] Embodiment 3: The embodiment discloses a control device for a fly ash autoclaved brick forming process, the control device comprising:

[0110] A modeling module is configured to obtain historical parameters of each process step in different fly ash autoclaved brick forming processes, use the historical parameters of each process step in the same fly ash autoclaved brick forming process as states, integrate all the states into a state space, use the adjustment amount of each historical parameter as an action, integrate all the actions into an action space, set a state transition probability based on all the historical parameters using a statistical analysis algorithm, set a reward function according to the qualification rate of the brick billet and the energy consumption, and establish an MDP model.

[0111] A solving module is configured to obtain real-time parameters of the current process step, solve the MDP model based on the real-time parameters using a reinforcement learning algorithm, obtain a first parameter that maximizes the value of the reward function, and calculate the difference between the first parameter and the real-time parameters.

[0112] A control module is configured to control a PID controller to adjust the real-time parameters based on the difference.

[0113] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, therefore: any equivalent changes made according to the structure, shape, principle of the present application should be covered within the protection scope of the present application.

Claims

1. A control method of a process for forming a fly ash autoclaved brick, characterized by, The method comprises: Modeling: obtaining historical parameters of each process step in the forming process of fly ash autoclaved brick, taking the historical parameters of each process step in the forming process of fly ash autoclaved brick as a state, integrating all states into a state space, taking the adjustment amount of each historical parameter as an action, integrating all actions into an action space, setting a state transition probability based on all historical parameters using a statistical analysis algorithm, setting a reward function according to the qualified rate of the brick and the energy consumption, and establishing an MDP model; Solving: obtaining real-time parameters of the current process step, solving the MDP model based on the real-time parameters using a reinforcement learning algorithm, obtaining a first parameter that maximizes the value of the reward function, and calculating the difference between the first parameter and the real-time parameter; Controlling: controlling the PID controller to adjust the real-time parameter according to the difference.

2. The control method of the process for forming fly ash autoclaved brick according to claim 1, characterized by, The method further comprises: Calculating the Euclidean distance between the real-time parameter and the first parameter, denoted as first data; obtaining a second parameter that minimizes the value of the reward function, calculating the Euclidean distance between the real-time parameter and the second parameter, denoted as second data; Determining whether the first data is greater than the second data, if yes, executing the controlling step, if not, outputting an alarm signal.

3. The control method of the process for forming fly ash autoclaved brick according to claim 2, characterized by, Before executing the controlling step, the method further comprises: Calculating the decay rate: calculating the decay rate of the reward function and setting the normal value range of the decay rate, determining whether the decay rate at the current time is within the normal value range, if yes, executing the controlling step, if not, outputting a warning signal; the decay rate at the current time instant The calculation model is as follows: ； wherein, is a value of the reward function at the current time; is an average value of the reward function in a preset time length before the current time.

4. The control method of the process for forming fly ash autoclaved brick according to claim 3, characterized by, After executing the controlling step, the method further comprises: The number of times of sending the early warning signal within the time length T after adjusting the real-time parameter is recorded as third data; the number of times of sending the early warning signal within n time lengths T before adjusting the real-time parameter is recorded as fourth data, and whether the third data is normal is judged based on the fourth data using the rule If yes, no processing is performed; if no, the early warning signal is output.

5. The control method of the process for forming fly ash autoclaved brick according to claim 4, characterized by, The third data and the fourth data are the number of warnings in the same process step in the forming process of fly ash autoclaved brick.

6. The control method of the process for forming fly ash autoclaved brick according to claim 1, wherein, The method further comprises: Decoupling the MDP model into multiple MDP sub-models, each MDP sub-model corresponding to a different process step in the forming process of fly ash autoclaved brick, selecting the MDP sub-models that need to be activated using a Monte Carlo tree search algorithm, integrating the MDP sub-models that need to be activated into a new MDP model according to the order of appearance in the forming process of fly ash autoclaved brick, and executing the solving step.

7. The control method of the process for forming fly ash autoclaved brick according to claim 6, characterized by, The method further comprises: formulating multiple adjustment strategies according to the difference, optimizing the adjustment strategies according to a genetic algorithm, constructing a fitness function of the genetic algorithm based on the forming efficiency of fly ash autoclaved brick, the qualified rate of the brick and the energy consumption, iteratively selecting an adjustment strategy that maximizes the fitness function value, denoted as the optimal condition strategy; the adjustment strategy includes: adjustment amplitude, adjustment sequence and adjustment period corresponding to the PID controller, and the adjustment amplitude is equal to the size of the difference; In the controlling step, the corresponding PID controller is controlled to adjust the real-time parameter based on the optimal adjustment strategy.

8. The control method of the process for forming fly ash autoclaved brick according to claim 7, characterized by, The controlling step further comprises: During the adjustment of the real-time parameter, the energy consumption of each PID controller is detected in real time, if the energy consumption of any PID controller exceeds a preset energy consumption threshold, an alarm signal is sent, otherwise, the step is executed again.

9. The control method of the process for forming fly ash autoclaved brick according to claim 8, characterized by, The method further comprises: based on the historical parameters, using a statistical analysis algorithm to obtain the adjustment step of the process step corresponding to the real-time parameter; In the controlling step, the corresponding PID controller is controlled to adjust the real-time parameter according to an adjustment step based on the optimal adjustment strategy.

10. A control device for a process of forming a fly ash autoclaved brick, characterized by, The control device is used to execute the control method in any one of claims 1-9, and the control device further comprises: The modeling module is used to acquire historical parameters of each process step in a different fly ash autoclaved brick forming process, take the historical parameters of each process step in the same fly ash autoclaved brick forming process as a state, integrate all the states into a state space, take the adjustment amount of each historical parameter as an action, integrate all the actions into an action space, set a state transition probability based on all the historical parameters by using a statistical analysis algorithm, set a reward function according to the qualified rate of the brick blank and the energy consumption, and establish an MDP model; The solving module is used to acquire real-time parameters of a current process step, solve the MDP model by using a reinforcement learning algorithm based on the real-time parameters, obtain a first parameter that makes the value of the reward function maximum, and calculate a difference between the first parameter and the real-time parameters. The control module is used to control the PID controller to adjust the real-time parameters according to the difference.

Images