Kiln state optimization method and system based on deep learning

Abstract

The invention provides a kiln state optimization method and system based on deep learning, and relates to the technical field of state optimization, and the method comprises the steps: collecting the operation parameters of a kiln in real time through a sensor, and the operation parameters comprise temperature, pressure, gas concentration and energy consumption data; a deep reinforcement learning model is constructed, the deep reinforcement learning model comprises a state encoder and a strategy gradient network, the deep reinforcement learning model is trained based on the historical operation parameters and the historical adjustment amount, and the state encoder adopts a convolutional neural network and a long-short-term memory network to perform feature extraction on the historical operation parameters; the strategy gradient network is used for learning the mapping relation between the extracted features and the historical adjustment amount; and inputting real-time data into the trained deep reinforcement learning model to obtain a real-time adjustment amount, and adjusting kiln operation parameters according to the real-time adjustment amount. According to the method, the reliability of the kiln state optimization result based on deep learning can be improved.

Description

Technical Field

[0001] This application relates to the technical field of state optimization, and in particular to a kiln state optimization method and system based on deep learning. Background Technology

[0002] In industrial production, kilns, as key thermal equipment, are widely used in numerous industries such as ceramics, building materials, metallurgy, and chemicals. The operating status of a kiln directly affects product quality, production efficiency, and energy consumption. Kiln condition optimization aims to ensure the kiln is always in optimal operating condition through precise control and optimization of the kiln operation process. This, in turn, improves production efficiency, reduces energy consumption, and minimizes pollutant emissions while maintaining product quality. With the increasing demands for energy conservation, emission reduction, and product quality in industrial production, achieving efficient kiln condition optimization has become a crucial issue that urgently needs to be addressed in the industrial sector.

[0003] Traditional optimization algorithms used in kiln state optimization mainly include genetic algorithms (GA), particle swarm optimization (PSO), and simulated annealing (SA). These methods theoretically approximate the globally optimal control parameters by iteratively searching for the optimal solution. However, in actual industrial scenarios, kiln control involves multi-variable coupling (such as temperature, pressure, and gas flow rate), with a high-dimensional parameter space and strong nonlinearity. In high-dimensional spaces, kiln control is prone to getting trapped in local optima or experiencing slow convergence, making it difficult to guarantee the reliability of the optimization results. Summary of the Invention

[0004] In order to improve the reliability of kiln state optimization results based on deep learning, this application provides a kiln state optimization method and system based on deep learning.

[0005] Firstly, this application provides a deep learning-based method for optimizing kiln conditions, employing the following technical solution: A deep learning-based method for optimizing kiln conditions includes the following steps: The kiln's operating parameters are collected in real time by sensors, including temperature, pressure, gas concentration, and energy consumption data. A deep reinforcement learning model is constructed, comprising a state encoder and a policy gradient network. The deep reinforcement learning model is trained based on historical running parameters and historical adjustment values. The state encoder uses a convolutional neural network and a long short-term memory network to extract features from the historical running parameters. The policy gradient network is used to learn the mapping relationship between the extracted features and the historical adjustment values. Real-time data is input into the trained deep reinforcement learning model to obtain real-time adjustment values, and the kiln operating parameters are adjusted based on these real-time adjustment values.

[0006] This application constructs a deep reinforcement learning model comprising a state encoder and a policy gradient network, leveraging the powerful capabilities of deep learning to process kiln operation data. The state encoder employs a convolutional neural network (CNN) and a long short-term memory network (LSTM) to extract features from historical operating parameters. CNN excels at extracting local features from data, effectively capturing the local variation patterns of kiln operating parameters at different times or locations; LSTM, on the other hand, has unique advantages in processing sequential data, remembering historical information and considering long-term dependencies, enabling the discovery of trends and patterns in historical operating parameters over time. The combination of these two methods allows for a more comprehensive and in-depth extraction of features from historical operating parameters, contributing to improved model accuracy and generalization ability. Subsequently, real-time collected data is input into the trained deep reinforcement learning model. Based on the learned mapping relationship between historical data features and adjustment amounts, the model can quickly and accurately provide real-time adjustment values. This application adopts a data-driven intelligent decision-making approach, eliminating the excessive reliance on experience in traditional manual adjustments and minimizing inaccurate or untimely adjustments caused by human factors. It enables more scientific and reasonable adjustment decisions based on the real-time operating status of the kiln.

[0007] Optionally, the state encoder adopts a hybrid network structure of a one-dimensional convolutional neural network and a long short-term memory network cascaded together; the one-dimensional convolutional neural network is used to extract local correlation features of the running parameters, and the long short-term memory network is used to extract the long-term dependence features of the local correlation features on the time series.

[0008] Optionally, the policy gradient network adopts an Actor-Critic framework, including an Actor network and a Critic network. The Actor network is a fully connected neural network containing at least two hidden layers. The hidden layers are followed by a ReLU activation function, and the output layer uses a tanh activation function to limit the range of the adjustment amount and is used to output the historical adjustment amount. The Critic network is a two-stream structure that receives state vectors and action vectors as inputs, respectively. It evaluates the value of state-action pairs through a feature fusion layer and outputs a value function value to guide the training of the Actor network. During training, an experience replay mechanism and target network technology are used to update network parameters by minimizing the time difference error, which is calculated using the mean square error loss function.

[0009] The state encoder employs a hybrid structure of a cascaded one-dimensional convolutional neural network and a long short-term memory network. The one-dimensional convolutional neural network accurately extracts the local correlation features of the operating parameters, while the long short-term memory network captures their long-term dependency features over time, enhancing the time series modeling capability. The policy gradient network adopts the Actor-Critic framework. The fully connected multi-layer structure of the Actor network and the reasonable selection of activation functions facilitate the learning of complex nonlinear relationships and limit the adjustment range. The dual-stream structure of the Critic network comprehensively evaluates the value of state-action pairs to guide the Actor network training. During training, an experience replay mechanism is used to break data correlation and improve utilization. The target network technology stabilizes the training process and improves model performance. The mean squared error loss function facilitates error measurement and optimization, thereby improving the accuracy and effectiveness of kiln operation adjustments.

[0010] Optionally, training the deep reinforcement learning model based on historical operating parameters and historical adjustment amounts further includes: Build an offline experience replay pool using historical data; The deep reinforcement learning model is pre-trained offline and fine-tuned online using a digital twin model.

[0011] Optionally, the offline pre-training of the digital twin model includes: Initialize the parameters of the deep reinforcement learning model; In the environment simulated by the digital twin model, multiple rounds of interaction are carried out. The interaction includes: given initial running parameters, the policy gradient network outputs a simulated adjustment amount, and the digital twin model calculates the simulated running parameters and corresponding simulated rewards for the next time step based on the simulated adjustment amount. Based on the simulated interaction data stored in the offline experience replay pool, the parameters of the deep reinforcement learning model are updated until the policy converges or a preset number of training rounds are reached.

[0012] Optionally, the online fine-tuning includes: After the trained deep reinforcement learning model is deployed in the kiln, the actual collected operating parameters, the real-time adjustment amounts generated, and the corresponding operating effects are continuously stored as new experience data in the experience playback pool. The parameters of the deep reinforcement learning model are incrementally updated periodically or based on policy performance metrics using newly added running parameters from the experience replay pool.

[0013] Optionally, training the deep reinforcement learning model based on historical operating parameters and historical adjustment amounts further includes: Construct a reward function, which is used to calculate the immediate reward for each state-action pair based on the difference between the kiln's operating state and the preset optimal state target; The training objective of the deep reinforcement learning model is to maximize the long-term cumulative value of the immediate reward; the digital twin model provides the simulated reward in simulated interactions.

[0014] Optionally, the offline experience replay pool adopts a priority-based experience replay mechanism; the sampling priority is dynamically calculated based on the time difference error or reward surprise degree, and experience data with high time difference error or reward surprise degree has a higher probability of being sampled.

[0015] This application utilizes historical data to construct an offline experience replay pool, which can fully integrate a large amount of accumulated kiln operation data. The experience replay pool randomly samples data during training, breaking the temporal correlation of historical data. During kiln operation, data is often generated continuously over time, and there is a strong correlation between adjacent data points. This correlation can affect the stability and convergence of model training. This application, through random sampling in the experience replay pool, makes the training data closer to independent and identically distributed data, improving the stability and efficiency of model training.

[0016] This application utilizes offline pre-training in a digital twin model simulation environment, eliminating the need for operation on a real kiln. This minimizes interference and potential risks to the actual kiln operation caused by unreasonable adjustments during the model exploration process, thereby improving safety.

[0017] Optionally, the method further includes: Based on the data in the offline experience replay pool, a preset state transition prediction model is trained; The trained state transition prediction model is used to predict the state transition of the simulated adjustment amount generated by the policy gradient network. The prediction results are obtained, and the cosine similarity between the prediction results and each sample in the experience replay pool is calculated. If the cosine similarity is less than the preset cosine similarity threshold and the reward value of the prediction results is greater than the preset reward threshold, then the synthetic experience unit is generated using the prediction results and added to the experience replay pool.

[0018] This application trains a state transition prediction model based on offline experience replay pool data, and uses this model to predict state transitions for simulated adjustments generated by a policy gradient network, thereby generating synthetic experience units. These synthetic experience units are expanded from existing real data, covering state-action combinations that are not present or occur with low frequency in real data. This effectively enriches the data types and distribution in the experience replay pool, providing more comprehensive training samples for deep reinforcement learning models. The state transition prediction model can simulate changes in kiln operating parameters under different conditions, and the generated synthetic experience units can reflect more complex and varied real-world scenarios.

[0019] In actual kiln operation, collecting a large amount of comprehensive and high-quality real-world data can face numerous difficulties, such as high data acquisition costs, long cycles, and limitations imposed by actual production. This application reduces reliance on real-world data to some extent by generating synthetic experience units, thereby lowering the difficulty and cost of data acquisition.

[0020] Secondly, this application provides a kiln state optimization system based on deep learning, which adopts the following technical solution: A deep learning-based kiln state optimization system includes: a processor and a memory communicatively connected to the processor; The memory is provided with a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in the first aspect.

[0021] In summary, this application includes at least one of the following beneficial technical effects: 1. This application constructs a deep reinforcement learning model comprising a state encoder and a policy gradient network, leveraging the powerful capabilities of deep learning to process kiln operation data. The state encoder employs a convolutional neural network and a long short-term memory network to extract features from historical operating parameters. CNNs excel at extracting local features from data, effectively capturing the local variation patterns of kiln operating parameters at different time points or locations; LSTMs, on the other hand, have unique advantages in processing sequential data, remembering historical information and considering long-term dependencies, enabling the discovery of trends and patterns in historical operating parameters over time. The combination of these two approaches allows for a more comprehensive and in-depth extraction of features from historical operating parameters, contributing to improved model accuracy and generalization ability.

[0022] 2. This application inputs real-time collected data into the trained deep reinforcement learning model. The deep reinforcement learning model can quickly and accurately provide real-time adjustment amounts based on the mapping relationship between the learned historical data features and the adjustment amounts. This application adopts a data-driven intelligent decision-making method, which gets rid of the excessive reliance on experience in traditional manual adjustment and avoids the problems of inaccurate or untimely adjustment caused by human factors as much as possible. It can make more scientific and reasonable adjustment decisions based on the real-time operating status of the kiln. Attached Figure Description

[0023] Figure 1 This is a flowchart of the method in Embodiment 1 of this application; Figure 2 This is a flowchart of the method in Embodiment 2 of this application. Detailed Implementation

[0024] The following combination Figure 1 and Figure 2 This application will be described in further detail.

[0025] Example 1: This example discloses a kiln state optimization method based on deep learning, referring to... Figure 1 The method includes: S11 data acquisition and processing, S12 modeling, and S13 prediction. Real-time data on kiln operating parameters, including temperature, pressure, gas concentration, and energy consumption, is collected via sensors. A deep reinforcement learning model containing a state encoder and a policy gradient network is constructed. This model is trained based on historical operating parameters and historical adjustment values. Real-time data is then input into the trained model to obtain real-time adjustment values, which are used to adjust the kiln control parameters. The execution process of each step in this embodiment is as follows: S11 data acquisition and processing involves real-time acquisition of kiln operating parameters via sensors, including temperature, pressure, gas concentration, and energy consumption data.

[0026] This embodiment enables the real-time acquisition of temperature data for each area by installing temperature sensors such as thermocouples or infrared thermometers at different locations inside the kiln.

[0027] In this embodiment, pressure sensors are installed at the air inlet, air outlet, and key internal locations of the kiln to monitor the kiln pressure.

[0028] In this embodiment, a gas analyzer is used to detect the concentration of various gases inside the kiln in real time.

[0029] This embodiment uses devices such as electricity meters and flow meters installed on the power supply lines and gas supply pipelines of the kiln to collect real-time data on the kiln's electricity and gas consumption.

[0030] S12 modeling is used to construct a deep reinforcement learning model, which includes a state encoder and a policy gradient network. The deep reinforcement learning model is trained based on historical running parameters and historical adjustments.

[0031] The state encoder uses a convolutional neural network and a long short-term memory network to extract features from historical operating parameters. Specifically, the state encoder uses a hybrid network structure of a cascaded one-dimensional convolutional neural network (1D-CNN) and a long short-term memory network (LSTM) to extract features from historical operating parameters.

[0032] The one-dimensional convolutional neural network can automatically detect and extract locally relevant features by sliding the convolutional kernel on the operating parameters. The long short-term memory network, through a gating mechanism, passes the locally relevant feature vectors extracted by the one-dimensional convolutional neural network through the input gate, forget gate, and output gate in sequence, further mining long-term dependency features in the time series from the locally relevant features.

[0033] A one-dimensional convolutional neural network consists of two convolutional layers. The first layer has a 3n kernel, where n is the dimension of the input data, i.e., the dimension of the input operating parameters. In this embodiment, the dimension is [time step length T, parameter dimension D] (e.g., T=24, D=5: temperature, pressure, flow rate, rotation speed, humidity), and the output dimension is 16. The second layer has a 3×16 kernel and an output dimension of 32. The ReLU activation function is used.

[0034] The Long Short-Term Memory (LSTM) network consists of two LSTM layers with a hidden layer dimension of 64, dropout=0.1, and batch_first=True.

[0035] The output layer of the state encoder takes the hidden state of the last time step of the long short-term memory network and outputs a state vector (64-dimensional).

[0036] The policy gradient network is used to learn the mapping relationship between the extracted features and the historical adjustment amount. In this embodiment, the policy gradient network adopts the Actor-Critic framework, which includes an Actor network and a Critic network.

[0037] The Actor network is a fully connected neural network containing at least two hidden layers. The hidden layers use the ReLU activation function, and the output layer uses the tanh activation function to limit the range of the adjustment amount and is used to output the historical adjustment amount.

[0038] In this embodiment, the input layer of the Actor network receives the 64-dimensional state vector output by the state encoder. The Actor network employs two hidden layers, which are fully connected layers. The first fully connected layer maps the 64-dimensional state vector to a 128-dimensional first intermediate vector, and the second fully connected layer maps the 128-dimensional first intermediate vector to a 64-dimensional second intermediate vector. The second fully connected layer uses the ReLU activation function with dropout=0.1. The output layer of the Actor network maps the 64-dimensional second intermediate vector to an action vector using the tanh activation function. In this embodiment, the action vector includes three dimensions: temperature adjustment, pressure adjustment, and flow rate adjustment.

[0039] The Critic network is a two-stream structure that receives state vectors and action vectors as inputs. After receiving the inputs, the Critic network fuses the state features and action features through a feature fusion layer, then evaluates the value of the state-action pair, and outputs a value function value. This value function value reflects the expected value of the cumulative reward that can be obtained in the future after taking a specific action in the current state.

[0040] The Critic network comprises a state branch and an action branch. The state branch inputs a 64-dimensional state vector into a 64-dimensional hidden layer, employing the ReLU activation function to extract the value features of the state itself. The action branch inputs an action vector into a 64-dimensional hidden layer, employing the ReLU activation function to extract the value features of the action. A feature fusion layer fuses the state and action features, concatenating them and then passing the concatenation through a fully connected layer and a ReLU activation function to reduce the dimensionality to a 64-dimensional third intermediate vector. This 64-dimensional third intermediate vector is then passed through a fully connected layer to output the value function value.

[0041] The value function output by the Critic network is used to guide the training of the Actor network. It helps the Actor network understand the merits and demerits of different actions in terms of long-term rewards, thereby guiding the Actor network to learn in the direction of generating better adjustment values.

[0042] In the training process of the deep reinforcement learning model, this embodiment employs an experience replay mechanism and target network technology. Network parameters are updated by minimizing the temporal difference error, which is calculated using the mean squared error loss function and includes the following: This embodiment uses historical data to construct an offline experience replay pool and employs a priority-based experience replay mechanism. In traditional experience replay mechanisms, experience data is randomly sampled, while the priority-based experience replay mechanism in this embodiment dynamically calculates the sampling priority based on time difference error or reward surprise degree. The process is as follows: Step 1: Initialize the experience replay pool, setting the maximum capacity to 1×10. 5 The initial priority weight coefficient α is set to 0.6, the importance sampling coefficient β is set to 0.4, and the minimum value ε is set to 1×10. -6 The sampling batch size is 64, and the reward discount factor γ is 0.99; Step 2: The deep reinforcement learning model interacts with the digital twin kiln environment to generate experience units (S). t A t , R t ,S t+1 Done t ), and store it in the experience replay pool; Among them, S t The sequence of historical operating parameters of the kiln at time t (i.e., the state input of the deep reinforcement learning model) has the dimension of [time series length T × parameter dimension D], including parameters such as kiln temperature, pressure, fuel flow rate, combustion air flow rate, and roller speed. In this embodiment, T=24 (covering the past 24 sampling periods) and D=5. A tThe kiln parameter adjustment amount (i.e. action) output by the model at time t includes fuel flow adjustment amount, roller speed adjustment amount, etc., and the dimension is [action dimension A]. In this embodiment, A=3. R t The instantaneous reward at time t is calculated by the reward function and reflects the degree of matching between the kiln's operating state and the optimal objective at that time (such as the comprehensive score of parameter compliance rate, energy consumption, and product quality). S t+1 This is the sequence of historical operating parameters of the kiln at time t+1, i.e., the adjustment amount A. t The next state after; Done t The end flag is Done. t =True indicates the end; The experience replay pool adopts a first-in-first-out storage strategy. When the number of experiences reaches the maximum capacity, the earliest stored experience unit is overwritten.

[0043] Step 3: When the number of experiences in the experience replay pool reaches a preset threshold (1000 in this example), priority evaluation is initiated: If the model has completed its initial training, the priority score for each experience is calculated using time difference error. The calculation model is as follows:

[0044]

[0045] in, This refers to the time difference error; State-action pairs The value function value; 1- The ) symbol indicates whether the round has ended. If Doneₜ=True (round ended), there will be no future rewards, and this item will be 0; P is the priority score. If the model is in the initialization phase, the priority score is calculated using the reward surprise factor, as follows: Based on historical reward data, the reward surprise is calculated using the moving average method (in this embodiment, the moving window size is 100).

[0046] Where S represents the surprise factor; The historical average reward value at time t is calculated using the moving average method based on the reward data of historical experience in the experience replay pool. The priority score P is calculated based on the reward surprise level, and the formula is: P = S + ε; Step 4: Calculate the sampling probability of each experience based on the priority score, extract 64 experiences using the roulette wheel method, and calculate the importance sampling weight of each experience; Step 5: Input the sampling experience and importance sampling weights into the deep reinforcement learning model, and update the model parameters by minimizing the weighted mean square error loss function; Step 6: Recalculate the time difference error of the sampling experience for this batch and update its priority score in the experience replay pool; Step 7: Repeat steps 2-6 until the model training converges (in this embodiment, the convergence condition is: the reward value fluctuation is less than 5% over 100 consecutive training rounds).

[0047] Temporal difference error reflects the accuracy of the deep reinforcement learning model in estimating the value of state-action pairs. The larger the error, the higher the value of the empirical data for the deep reinforcement learning model. The reward surprise measure measures the difference between the actual reward and the expected reward. The larger the difference, the more new information the empirical data can bring to the deep reinforcement learning model.

[0048] Experience data with high temporal difference error or high reward surprise has a higher sampling probability, and deep reinforcement learning models can learn the most helpful experiences for their own improvement more frequently, thereby improving training efficiency.

[0049] The calculation model for the reward function in this embodiment is as follows:

[0050]

[0051] in, The value of the reward function; As weight; The parameter matching score is awarded as a reward, with a value ranging from 0 to 1, and the calculation formula is as follows:

[0052]

[0053] in, Let be the value of the i-th running parameter; The optimal parameter value for the i-th running parameter; This represents the maximum permissible deviation of the i-th operating parameter, such as temperature ±100℃ or pressure ±1MPa. This is for calculating the maximum value. Let be the weight of the i-th running parameter; D is the number of types of running parameters; The energy consumption reward value ranges from 0 to 1, and the calculation model is as follows:

[0054] Where E is the current energy consumption value; This is the optimal energy consumption baseline value; p is a penalty term, with a value ranging from 0 to 0.5. The calculation model is as follows:

[0055] Wherein, k is the penalty coefficient, and in this embodiment, the penalty coefficient k is 5.

[0056] This embodiment uses a digital twin model to perform offline pre-training and online fine-tuning of the deep reinforcement learning model, including: First, the parameters of the deep reinforcement learning model are initialized, and multiple rounds of interaction are conducted in the environment simulated by the digital twin model. In each round of interaction, given the initial operating parameters, the policy gradient network outputs a simulated adjustment amount, and the digital twin model calculates the simulated operating parameters and corresponding simulated reward for the next time step based on the simulated adjustment amount. Based on the simulated interaction data stored in the offline experience replay pool, the parameters of the deep reinforcement learning model are updated using a method that minimizes the temporal difference error. The parameters are iteratively updated until the policy converges or the preset number of training rounds is reached. Offline pre-training allows the model to learn certain control policies in a safe environment before deployment to the actual kiln, reducing potential risks during actual training.

[0057] The initial values ​​of the parameters of the deep reinforcement learning model are: The historical data length T is 24, and the training epochs are 1×10. 4 The round and batch size is 64, the learning rate is 0.0001, the reward discount factor is 0.99, the priority weight coefficient α is 0.6, and the importance sampling coefficient β is 0.4.

[0058] After deploying the trained deep reinforcement learning model in the kiln, the actual collected operating parameters, the resulting real-time adjustments, and the corresponding operating effects are continuously stored as new experience data in the experience playback pool, specifically: Real-time acquisition of actual operating data at time t, including: historical parameter sequence S t Adjustment amount A of model output t Actual energy consumption E t Call the reward function to calculate the actual instant reward R. t The above parameters are integrated into practical experience units.

[0059] Periodically or based on strategy performance metrics, when more than 20 new practical experiences are added to the experience replay pool, an incremental model update is triggered, specifically as follows: Sample 32 experiences from the experience replay pool according to priority (actual experience accounts for no less than 30%). Using the same loss function and optimizer as offline pre-training, the learning rate is reduced to 5e-5, and only the main network parameters are incrementally updated; If the adjustment amount of the updated model output causes the actual parameter fluctuation to be greater than 5%, then roll back to the previous version of parameters, readjust the learning rate (down to 3e-5), and update again.

[0060] Online fine-tuning enables the model to continuously adapt to changes in the actual kiln operating environment, further improving the model's performance and adaptability.

[0061] A reward function is constructed to calculate the immediate reward for each state-action pair based on the difference between the kiln's operating state and the preset optimal state target. For example, the closer the kiln's temperature, pressure, and other parameters are to the preset optimal values, and the lower the energy consumption and the higher the product quality, the higher the corresponding immediate reward; conversely, if the operating state deviates from the optimal target, the immediate reward will decrease. The training objective of the deep reinforcement learning model is to maximize the long-term cumulative value of the immediate reward. By continuously optimizing the model parameters, the model can obtain higher cumulative rewards during long-term operation, thereby achieving optimized control of the kiln's operating state. During offline pre-training of the digital twin model, the digital twin model provides simulated rewards in simulated interactions, providing guidance for the model's initial learning.

[0062] S13 predicts that real-time data is input into the trained deep reinforcement learning model to obtain real-time adjustment values, and the kiln operating parameters are adjusted according to the real-time adjustment values.

[0063] Example 2: Refer to Figure 2 The difference between this embodiment and Embodiment 1 is that the method further includes: S21 Model Training: Based on the data in the offline experience replay pool, a preset state transition prediction model is trained. The state transition prediction model adopts a fully connected layer and residual network structure, and the specific structure is as follows: The input layer has an input dimension of 64+3; residual block 1 has an input dimension of 67 and an output dimension of 128, using the ReLU activation function; residual block 2 has an input dimension of 128 and an output dimension of 256, using the ReLU activation function; the dimensionality reduction layer has an input dimension of 256 and an output dimension of 64, using the ReLU activation function; the state prediction head has an input dimension of 64 and an output dimension of 64, outputting the predicted kiln state features for the next time step; the reward prediction head has an input dimension of 64 and an output dimension of 1, outputting the predicted instant reward value.

[0064] Initialize the state transition prediction model parameters, set the optimizer to AdamW (weight decay 1e-5), learning rate 5e-4, and loss function... as follows:

[0065]

[0066]

[0067] in, The state prediction loss function; For reward prediction loss function; Encoding features for the predicted state at time t+1; The encoded features of the true state at time t+1; Let the mean square error function be used. To predict the reward value; This is the actual reward value.

[0068] Iterative training is performed with a batch size of 128 and 100 training rounds. An early stopping mechanism is used (the training is terminated if the validation set loss does not decrease for 5 consecutive rounds).

[0069] S22 generates new samples, inputs the simulated adjustment values ​​generated by the policy gradient network into the trained state transition prediction model, and obtains the prediction results. .

[0070] Calculate the cosine similarity between the predicted result and each sample in the experience replay pool. If the cosine similarity is less than a preset cosine similarity threshold (e.g., 0.1) and the reward value of the predicted result is... If the result exceeds a preset reward threshold (e.g., 0.8), a synthetic experience unit is generated using the prediction result and added to the experience replay pool. The synthetic experience unit is... ,in .

[0071] Example 3: This example discloses a kiln state optimization system based on deep learning. The system includes: a processor and a memory communicatively connected to the processor. The memory is provided with a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the processor processes a computer program stored on the computer-readable storage medium, it implements the method.

[0072] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for optimizing kiln conditions based on deep learning, characterized in that, include: The kiln's operating parameters are collected in real time by sensors, including temperature, pressure, gas concentration, and energy consumption data. A deep reinforcement learning model is constructed, comprising a state encoder and a policy gradient network. The deep reinforcement learning model is trained based on historical running parameters and historical adjustment values. The state encoder uses a convolutional neural network and a long short-term memory network to extract features from the historical running parameters. The policy gradient network is used to learn the mapping relationship between the extracted features and the historical adjustment values. Real-time data is input into the trained deep reinforcement learning model to obtain real-time adjustment values, and the kiln operating parameters are adjusted based on these real-time adjustment values.

2. The kiln state optimization method based on deep learning according to claim 1, characterized in that, The state encoder adopts a hybrid network structure of a one-dimensional convolutional neural network and a long short-term memory network cascaded together; the one-dimensional convolutional neural network is used to extract local correlation features of the running parameters, and the long short-term memory network is used to extract the long-term dependence features of the local correlation features on the time series.

3. The kiln state optimization method based on deep learning according to claim 2, characterized in that, The policy gradient network adopts the Actor-Critic framework, including an Actor network and a Critic network. The Actor network is a fully connected neural network containing at least two hidden layers. The hidden layers are followed by a ReLU activation function, and the output layer uses a tanh activation function to limit the range of the adjustment amount and is used to output the historical adjustment amount. The Critic network is a two-stream structure that receives state vectors and action vectors as inputs, respectively. It evaluates the value of state-action pairs through a feature fusion layer and outputs a value function value to guide the training of the Actor network. During training, an experience replay mechanism and target network technology are used to update network parameters by minimizing the time difference error, which is calculated using the mean square error loss function.

4. The kiln state optimization method based on deep learning according to claim 1, characterized in that, The training of the deep reinforcement learning model based on historical operating parameters and historical adjustment amounts also includes: Build an offline experience replay pool using historical data; The deep reinforcement learning model is pre-trained offline and fine-tuned online using a digital twin model.

5. The kiln state optimization method based on deep learning according to claim 4, characterized in that, The offline pre-training using a digital twin model includes: Initialize the parameters of the deep reinforcement learning model; In the environment simulated by the digital twin model, multiple rounds of interaction are carried out. The interaction includes: given initial running parameters, the policy gradient network outputs a simulated adjustment amount, and the digital twin model calculates the simulated running parameters and corresponding simulated rewards for the next time step based on the simulated adjustment amount. Based on the simulated interaction data stored in the offline experience replay pool, the parameters of the deep reinforcement learning model are updated until the policy converges or a preset number of training rounds are reached.

6. The kiln state optimization method based on deep learning according to claim 5, characterized in that, The online fine-tuning includes: After the trained deep reinforcement learning model is deployed in the kiln, the actual collected operating parameters, the real-time adjustment amounts generated, and the corresponding operating effects are continuously stored as new experience data in the experience playback pool. The parameters of the deep reinforcement learning model are incrementally updated periodically or based on policy performance metrics using newly added running parameters from the experience replay pool.

7. The kiln state optimization method based on deep learning according to claim 6, characterized in that, The training of the deep reinforcement learning model based on historical operating parameters and historical adjustment amounts also includes: Construct a reward function, which is used to calculate the immediate reward for each state-action pair based on the difference between the kiln's operating state and the preset optimal state target; The training objective of the deep reinforcement learning model is to maximize the long-term cumulative value of the immediate reward; the digital twin model provides the simulated reward in simulated interactions.

8. The kiln state optimization method based on deep learning according to claim 4, characterized in that, The offline experience replay pool adopts a priority-based experience replay mechanism; the sampling priority is dynamically calculated based on the time difference error or reward surprise degree, and experience data with high time difference error or reward surprise degree has a higher probability of being sampled.

9. The kiln state optimization method based on deep learning according to claim 8, characterized in that, The method further includes: Based on the data in the offline experience replay pool, a preset state transition prediction model is trained; The trained state transition prediction model is used to predict the state transition of the simulated adjustment amount generated by the policy gradient network. The prediction results are obtained, and the cosine similarity between the prediction results and each sample in the experience replay pool is calculated. If the cosine similarity is less than the preset cosine similarity threshold and the reward value of the prediction results is greater than the preset reward threshold, then the synthetic experience unit is generated using the prediction results and added to the experience replay pool.

10. A kiln state optimization system based on deep learning, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory is provided with a computer-readable storage medium, and a computer program is stored on the computer-readable storage medium. When the processor processes a computer program stored on the computer-readable storage medium, it implements the method as described in any one of claims 1-9.

Images