Method for the optimal control of a cement clinker burning system

Abstract

The application discloses a kind of optimization control methods of cement firing system, record the process parameters in the process of raw material preheating and pre-decomposition, raw material sintering into clinker, clinker quenching and recovery in cement firing system, according to process parameters, establish kiln state model and clinker parameter model, as firing system object simulation model, collect historical data batch pretreatment, use gradient descent method, train two models, using reinforcement learning agent, and control object constantly interact, obtain state information, learn the dynamic characteristics and decision strategy of object, make decision strategy maximization cumulative return value, control object changes towards cumulative return maximization direction, online update object simulation model and agent, to adapt to kiln working condition change.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automation, and particularly relates to an industrial control algorithm technology. BACKGROUND

[0002] In a cement burning system, raw meal passes through a pre-decomposition furnace and then enters a rotary kiln. During the sintering process, a series of physical and chemical reactions occur. The process has characteristics such as large inertia, strong coupling, nonlinearity, and uncertainty, and the sintering condition of the material has a significant impact on the quality of the cement clinker. Changes in the composition of the raw meal, or material collapse and kiln skin dropping, have a disturbing effect, making the burning condition in the kiln complex and variable, and increasing the difficulty of operating the kiln.

[0003] The current popular control technology mainly uses linear MPC, adopts model prediction technology, solves the problems of large inertia and multivariable coupling, and maintains the stability of the production condition. The target values of the process parameters required by the technology are manually given by the operator, which cannot make the burning condition in the best benefit state, and cannot adapt to the variability and nonlinearity of the working condition. SUMMARY

[0004] In order to solve the problems existing in the prior art, the application provides an optimization control method for a cement burning system, improves the running stability of the rotary kiln burning system, enhances the adaptability of the kiln condition changes, ensures the quality, increases the yield, and reduces the consumption. In order to achieve the above purpose, the following technical scheme is adopted.

[0005] In the cement burning system, the raw meal is preheated by a preheater, pre-decomposed in a pre-decomposition furnace, then sintered into clinker in a rotary kiln, and finally quenched and recovered in a grate cooler.

[0006] The process parameters in this process include:

[0007] The control parameters u = {raw meal amount, kiln head coal, kiln tail coal, kiln rotation speed}

[0008] The state parameters s = {decomposition furnace outlet temperature, kiln NOx content, kiln current, kiln flame temperature, secondary air temperature}

[0009] The clinker parameters y = {clinker yield, clinker quality, clinker heat consumption}

[0010] According to the parameters u, s, and y, a kiln state model and a clinker parameter model are established as a simulation model of the burning system object, historical data are batch preprocessed, the gradient descent method is used to train the two models, an intelligent agent of reinforcement learning is used to interact with the control object, state information is obtained, the dynamic characteristics and decision-making strategies of the object are learned, the decision-making strategies are maximized cumulative reward values, the control object changes in the direction of maximizing the cumulative reward, and the simulation model of the object and the intelligent agent are updated online to adapt to the changes of the kiln working condition

[0011] Further, the kiln state model and the clinker parameter model are described by using an affine nonlinear system, u(t), s(t), y(t) represent the control parameter, the state parameter, and the clinker parameter at time t respectively, nonlinear functions f1, f2, g1, g2 represent bounded Lipschitz functions, s(t+1) = f1(s(t)) + g1(s(t))u(t) represents the kiln state model, y(t+1) = f2(y(t)) + g2(y(t))s(t) represents the clinker parameter model, a neural network is used to map the nonlinear affine, θ represents the neural network parameter, a neural network nonlinear mapping function representing the state parameter model, a neural network nonlinear mapping function representing the clinker parameter model, and

[0012] Further, during training, the indicator function J1 = E T δ -1 + log |δ| is minimized, where E is the deviation vector between the predicted value of the current batch data according to the model and the target value thereof, and δ is the variance of the deviation vector.

[0013] After the current batch data is trained for multiple rounds, other equal amounts of data are collected, and a random descent optimizer is used for training, such as SGD, Adam, etc., and the training indicators of the kiln state model and the clinker parameter model are the same.

[0014] Further, the agent is composed of a policy network and a planner, the burning state and the clinker trend are simulated and predicted according to the object model, the reward value that can be obtained in the current state information under the current policy is calculated, and the predicted advantage value is estimated.

[0015] The policy network maximizes the advantage and the performance stability as indicators, updates the policy, controls the object to change according to the new policy, collects batch data from historical data when training the policy network parameters, trains the model parameters by using the gradient descent method, and the optimizer uses Adam, SGD, etc., and when planning the current control policy online, the neural network obtained by training may be non-convex, and a population evolutionary algorithm is used as the planner, such as PSO, DE, CEM, etc.

[0016] Further, Δp1, Δp2, Δp3 represent the change amount of the unit output, the mass, and the unit heat consumption of the clinker respectively, c1, c2, c3 represent the corresponding cost, and the reward function r(t) = c1Δp1 + c2Δp2 - c3Δp3 represents the reward of the clinker input and the cost at time t, wherein c1 and c3 are positive values, and if c2 is positive, it means that the clinker mass cost is within the specified range, and if c2 is negative, it means that the clinker mass cost exceeds or is lower than the specified range.

[0017] Let the policy function be π(t), the H-step ahead prediction of Q-value function according to the policy function π(t) at time t be where γ1 is a discount factor less than 1, the advantage value function estimated according to the Q-value function Q(t) be where γ2 is a positive coefficient less than 1, the policy network adopts a neural network, and the nonlinear mapping function from state information to policy value is π θ,λ (u t )=NN θ (u t |y t ,s t ,u t-1 ), where θ is the network parameter.

[0018] Let y H represent the H-step ahead prediction value of the clinker parameter, y sp represent the expected value of the clinker parameter in the production process, Δu represent the change of the control parameter, and D(y, s, u) represent the upper and lower limits and rate constraints of each parameter of the sintering object, where the constraint limit is d, the Lagrange multiplier is λ, and the index weight coefficient is β1β2β2. The function train the network parameter θ and the Lagrange multiplier λ, where represents the KL divergence, the smoothness of the policy change before and after regularization, the first two terms of the function J(θ, λ) represent the economic indicators, the middle two terms represent the stability indicators, and the last term represents the constraint indicators.

[0019] During training, the optimization parameters are θ and λ. During online decision-making, the optimal control policy is planned according to the numerical values of the current parameters θ and λ.

[0020] Further, changes in the process characteristics of each device of the cement sintering system, changes in the quality of raw material ingredients, and changes in fuel calorific value will all lead to changes in sintering conditions. Therefore, during online operation, the control parameters, state parameters, and clinker parameters are stored in the playback buffer in chronological order, and the parameters of the simulation model and the agent of the learning object are periodically or conditionally triggered to update, adapt to changes in working conditions, and make optimal decisions.

[0021] The beneficial effects of the present application: based on affine mapping to establish a nonlinear firing object simulation model, using affine nonlinear system hierarchical implementation; strategy net implementation scheme, the economic indicators and stability indicators and constraint indicators are integrated; network offline training and online learning are combined, and the playback data buffer is set to update the network parameters online to adapt to the working condition changes; using reinforcement learning as an artificial intelligence technology, using the historical data of the cement firing system, offline training and learning the nonlinear object model and agent decision model of the firing system; when planning control online, the firing state and product information are obtained by interacting with the object in real time, based on the current state information, the object model is used for prediction to predict the firing state and clinker parameter information H steps in advance, and the optimal planning decision conforming to the current constraints is made according to the optimization target based on the advantage benefit; at the same time, the current state information and the decision are stored in the playback cache area, and the cached data are sampled at a certain time interval to train and update the firing object model and the agent decision model, learn to adapt to the latest working condition to adapt to the working condition changes, so as to make the optimal decision; using nonlinear intelligent economic optimization control technology, improving the running stability of the rotary kiln firing system, solving the optimization control problems such as nonlinear strong coupling, large disturbance and slow time-varying in the cement firing process, realizing the production running state of safe, stable, long and full, enhancing the adaptability of kiln condition changes, improving the production benefit and enhancing the competitiveness of enterprises. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is an optimization flowchart. DETAILED DESCRIPTION

[0023] The technical solutions of the present application are specifically described below in combination with the drawings.

[0024] The process parameters include:

[0025] The control parameters u = {raw material amount, kiln head coal, kiln tail coal, kiln speed}

[0026] The state parameters s = {decomposition furnace outlet temperature, kiln NOx content, kiln current, kiln flame temperature, secondary air temperature}

[0027] The clinker parameters y = {clinker yield, clinker quality, clinker heat consumption}

[0028] As shown in Figure 1 , according to the parameters u, s, y, the kiln state model and the clinker parameter model are established as the firing system object simulation model, the historical data are collected and pretreated in batches, the gradient descent method is used to train the two models, the reinforcement learning agent is used, and the state information is obtained by continuously interacting with the control object, the dynamic characteristics and decision strategy of the object are learned, the decision strategy is maximized cumulative return value, the control object changes in the direction of maximizing cumulative return, and the object simulation model and the agent are updated online to adapt to the kiln working condition changes

[0029] The kiln state model and the clinker parameter model are described by using affine nonlinear systems, u(t), s(t), y(t) represent the control parameter, the state parameter, and the clinker parameter at time t respectively, nonlinear functions f1, f2, g1, g2 represent bounded Lipschitz functions, s(t+1) = f1(s(t)) + g1(s(t))u(t) represents the kiln state model, y(t+1) = f2(y(t)) + g2(y(t))s(t) represents the clinker parameter model, a neural network is used to map the nonlinear affine, θ represents the neural network parameter, a neural network nonlinear mapping function representing the state parameter model, a neural network nonlinear mapping function representing the clinker parameter model, then

[0030] During training, the indicator function J1 = E T δ -1 + log |δ| is minimized, where E is the deviation vector between the predicted value of the current batch data according to the model and the target value thereof, and δ is the variance of the deviation vector.

[0031] After the current batch data is trained for multiple rounds, other equal amounts of data are collected, and a random descent optimizer is used for training, such as SGD, Adam, etc. The training indicators of the kiln state model and the clinker parameter model are of the same form.

[0032] The agent is composed of a policy network and a planner, which simulates and predicts the firing state and clinker trend according to the object model, calculates the reward value that can be obtained in advance H steps under the current policy and with the current state information, and estimates the predicted advantage value.

[0033] The policy network maximizes the advantage and stabilizes the performance as indicators, plans and updates the policy, controls the object to change according to the new policy, and trains the policy network parameters by collecting batch data from historical data and using the gradient descent method to train the model parameters. The optimizer uses Adam, SGD, etc. When planning the current control policy online, the neural network obtained by training may be non-convex, and a population evolution algorithm is used as the planner, such as PSO, DE, CEM, etc.

[0034] Let Δc1, Δp2, Δp3 represent the change in unit output, mass, and unit heat consumption of the clinker respectively, and c1, c2, c3 represent the corresponding cost. The reward function r(t) = c1Δp1 + c2Δp2 - c3Δp3 represents the reward of inputting clinker and paying cost at time t, where c1 and c3 are positive values. If c2 is positive, it means that the clinker mass cost is within the specified range, and if c2 is negative, it means that the clinker mass cost exceeds or is lower than the specified range.

[0035] Let the strategy function be π(t), the H-step ahead prediction Q-value function according to the strategy function π(t) at time t be where γ1 is a discount coefficient less than 1, the advantage value function estimated according to the Q-value function Q(t) be where γ2 is a positive coefficient less than 1, the strategy network adopts a neural network, and the nonlinear mapping function from state information to strategy value is π θ,λ (u t )=NN θ (u t |y t ,s t ,u t-1 ), where θ is a network parameter.

[0036] Let y H represent the H-step ahead prediction value of the clinker parameter, y sp represent the expected value of the clinker parameter in the production process, Δu represent the change of the control parameter, and D(y, s, u) represent the upper and lower limits and the rate constraints of each parameter of the sintering object, where the constraint limit is d, the Lagrange multiplier is λ, and the index weight coefficient is β1β2β2. The function is The network parameter θ and the Lagrange multiplier λ are trained, where represents the KL divergence, the strategy change smoothness before and after regularization, the first two terms of the function J(θ, λ) represent the economic index, the middle two terms represent the stability index, and the last term represents the constraint index.

[0037] During training, the optimization parameters are θ and λ. During online decision-making, the optimal control strategy is planned according to the numerical values of the current parameters θ and λ.

[0038] Changes in the process characteristics of each device of the cement sintering system, changes in the quality of the raw material composition, and changes in the fuel heat value will all lead to changes in the sintering working condition. Therefore, during online operation, the control parameters, state parameters, and clinker parameters are stored in the playback buffer in chronological order, and the parameters of the simulation model and the agent of the learning object are periodically or conditionally triggered to update, adapt to changes in the working condition, and make optimal decisions.

[0039] The above is an embodiment of the present application, which does not limit the present application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present application are included in the protection scope of the present application.

Claims

1. An optimized control method for a cement calcination system, comprising recording process parameters during the preheating and pre-decomposition of raw meal, the sintering of raw meal into clinker, and the rapid cooling and recycling of clinker in the cement calcination system, including: The system features control parameters u = {raw material quantity, kiln head coal, kiln tail coal, kiln rotation speed}, calcination state parameters s = {decomposer outlet temperature, kiln NOx content, kiln current, kiln flame temperature, secondary air temperature}, and clinker parameters y = {clinker yield, clinker quality, clinker heat consumption}. Its key features include: establishing a kiln state model and a clinker parameter model based on parameters u, s, and y, which serve as simulation models for the calcination system; batch preprocessing of historical data; training the two models using gradient descent; employing a reinforcement learning agent that continuously interacts with the controlled object to acquire state information, learn the dynamic characteristics and decision-making strategies of the object, and maximize the accumulation of decision-making strategies. The return value is determined by controlling the object to change in the direction of maximizing the cumulative return. The object simulation model and intelligent agent are updated online to adapt to changes in kiln operating conditions. The decision strategy to maximize the cumulative return value includes: Let Δp1, Δp2, and Δp3 represent the changes in unit output, quality, and unit heat consumption of clinker, respectively, and let c1, c2, and c3 represent the corresponding costs. Then, the return function r(t) = c1Δp1 + c2Δp2 - c3Δp3 represents the return for inputting clinker and paying costs at time t, where c1 and c3 are positive values. If c2 is positive, it means that the clinker quality cost is within the specified range; if it is negative, it means that the clinker quality cost exceeds or falls below the specified range. The establishment of the kiln state model and clinker parameter model based on parameters u, s, and y includes: using an affine nonlinear system to describe the kiln state model and clinker parameter model, where u(t), s(t), and y(t) represent the control parameters, firing state parameters, and clinker parameters at time t, respectively; and the nonlinear functions f1, f2, g1, and g2 represent bounded Lipschitz functions. The kiln state model is represented by the formula s(t+1) = f1(s(t)) + g1(s(t))u(t), and the clinker parameter model is represented by the formula y(t+1) = f2(y(t)) + g2(y(t))s(t). A neural network is used to map the nonlinear affine system, where θ represents the neural network parameters, ... The neural network nonlinear mapping function representing the kiln state model The neural network nonlinear mapping function representing the clinker parameter model is then...

2. The optimized control method for a cement firing system according to claim 1, characterized in that, The training of the two models includes: minimizing the index function J1 = E during training. T δ -1 E+log|δ|, where E is the deviation vector between the predicted value of the current batch of data and its target value based on the model, and δ is the variance of the deviation vector.

3. The optimized control method for a cement firing system according to claim 2, characterized in that, The training of the two models includes: after the current batch of data has been trained through multiple rounds, other equal amounts of data are collected and trained using a stochastic descent optimizer. The training indices for the kiln state model and the clinker parameter model are in the same form.

4. The optimized control method for a cement firing system according to claim 1, characterized in that, The agent consists of a policy network and a planner. It predicts the calcination state and clinker trend based on the object simulation model, calculates the reward value that can be obtained with the current state information and H steps ahead of the prediction under the current policy, and estimates the prediction advantage value.

5. The optimized control method for a cement firing system according to claim 4, characterized in that, The policy network uses advantage maximization and performance stability as indicators to plan and update policies. When training policy network parameters, it collects batch data from historical data and uses gradient descent to train model parameters based on changes in the controlled object according to the new policy. When planning the current control policy online, it uses a population evolution algorithm as the planner.

6. The optimized control method for a cement firing system according to claim 1, characterized in that, The online updating of the object simulation model and intelligent agent includes: storing control parameters, calcination state parameters, and clinker parameters into the playback buffer in a time sequence, and periodically or conditionally updating the parameters of the learning object simulation model and intelligent agent.

Images