Optimal control method of chemical reaction in tin smelting based on reinforcement learning

Abstract

The present invention relates to a tin smelting chemical reaction optimization control method based on reinforcement learning, which belongs to the interdisciplinary technical field of metallurgical engineering and artificial intelligence. The present invention uses an improved Fourier synthesis method to simulate state data in tin smelting. Subsequently, a tin smelting process model based on reinforcement learning algorithm is constructed, and the key process parameters in the furnace are used as state variables. An Actor-Critic architecture with a dual-network design is adopted to realize dynamic optimization control. The present invention also introduces a physical and chemical model, combines the process parameters and the thermodynamic equilibrium formula of the chemical reaction, and calculates the changing trend of the carbon monoxide generation. The present invention can intelligently optimize the chemical reaction conditions in the tin smelting process, increase the amount of tin generated, reduce carbon monoxide emissions, and effectively shorten the reaction time and reduce energy waste.

Description

Technical Field

[0001] The present invention relates to a tin smelting chemical reaction optimization control method based on reinforcement learning, belonging to the intersecting technical field of metallurgical engineering and artificial intelligence. Background Art

[0002] Tin smelting is a crucial process in the metallurgical industry. Its core is the chemical reaction that reduces tin oxide in tin ore to metallic tin. This process involves complex, multivariable process parameters, such as oxygen flow, coal flow, lance position, furnace pressure, and smelting temperature. The synergistic effect of these parameters directly affects tin production, energy efficiency, and the emission of reaction byproducts (such as carbon monoxide). Therefore, achieving precise control of chemical reactions under dynamically changing process conditions is a key issue in optimizing the tin smelting process.

[0003] Traditional tin smelting control methods primarily rely on empirical rules or simple feedback control systems. These methods often fail to fully account for the nonlinear coupling relationships between multivariable parameters and lack the ability to adapt to complex reaction mechanisms and dynamic operating conditions. Furthermore, data loss and noise in actual production further exacerbate control difficulties, leading to difficulties in steadily increasing tin production, reducing carbon monoxide emissions, and excessively long reaction times. These issues not only impact production efficiency but also create environmental burdens and waste energy.

[0004] In recent years, with the development of artificial intelligence (AI), reinforcement learning (RL) has become an important tool for industrial process optimization due to its outstanding performance in dynamic decision-making problems. Through continuous interaction with the environment, RL can automatically adjust control strategies based on real-time conditions to maximize objective functions (such as tin production, energy consumption, and environmental indicators). However, the application of traditional RL algorithms in complex industrial scenarios still faces numerous challenges, including how to leverage metallurgical physics to guide model learning, how to efficiently explore optimal strategies in high-dimensional parameter spaces, and how to ensure rapid model adaptation to real-time operating conditions. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a tin smelting chemical reaction optimization control method based on reinforcement learning, which can optimize the operating parameters in the smelting process in real time, thereby solving the above-mentioned problems.

[0006] The technical solution of this invention is a reinforcement learning-based method for optimizing and controlling chemical reactions in tin smelting. This method combines reinforcement learning algorithms with metallurgical physical models. By dynamically learning the coupling relationship between process parameters and chemical reactions, it achieves precise control of parameters such as oxygen flow, coal flow, and lance position during the tin smelting process. This method can increase tin production, reduce carbon monoxide emissions, shorten reaction time, and improve energy efficiency. It adapts to the multi-objective optimization requirements of complex industrial environments and provides a new solution for the intelligent and green development of tin smelting processes.

[0007] A reinforcement learning-based optimization control method for tin smelting chemical reactions, specifically comprising the following steps:

[0008] Step 1: Collect parameter data during the smelting process;

[0009] Step 2: Preprocess the collected original parameter data, clean outliers and fill missing values, and use the improved Fourier synthesis method to simulate the preprocessed parameter data to obtain simulated data;

[0010] Step 3: Build a reinforcement learning model, define the state space, action space, and design the reward function;

[0011] Step 4: Design a dual network architecture for the Actor-Critic network in the reinforcement learning model, introducing a main network and an auxiliary network;

[0012] Step 5: Based on the tin smelting process parameters, a physical and chemical model is established, and the amount of carbon monoxide generated is calculated by combining the thermodynamic equilibrium formula and the chemical reaction formula in the furnace;

[0013] Step 6: Use the calculated carbon monoxide production as the reward function input of the optimized reinforcement learning model to guide the agent in the reinforcement learning to select the action to perform;

[0014] Step 7: Use the generated simulation data and the collected parameter data to train the optimized reinforcement learning model and save the final reinforcement learning model parameters.

[0015] The Step 2 is specifically as follows:

[0016] Step 2.1: According to the sensor measurement range and the actual operating rules of the metallurgical process, set the parameter measurement range and the actual operating rule threshold, and eliminate abnormal values that exceed the parameter measurement range threshold and the parameter change rate exceeds the actual operating rule threshold;

[0017] Step 2.2: Add an adaptive bottom feature detection algorithm to the original Fourier synthesis method to identify the steady-state interval of the parameter data;

[0018] Step 2.3: Use the improved Fourier synthesis algorithm to simulate and generate tin smelting parameter data to expand the scale of the reinforcement learning training set.

[0019] The Step 3 is specifically as follows:

[0020] Step 3.1: Design the parameter values that affect the chemical reaction in the tin smelting furnace as the state space, and define the controllable optimization variables in the smelting process as the action space;

[0021] Step 3.2: The reward function is used as the optimization objective for reinforcement learning. It is designed based on the three goals of maximizing tin production, minimizing carbon monoxide emissions, and minimizing reaction time.

[0022] Step 3.3: Use the data and metallurgy dual-driven approach as the state update model for reinforcement learning.

[0023] The Step 4 is specifically as follows:

[0024] Step 4.1: Introduce two neural networks, the main network and the auxiliary network, into the Actor network. The main network is responsible for generating strategies and outputting optimized control actions for the current state. The auxiliary network interacts with the Critic network to provide action evaluation signals to the main network, guiding the main network to generate actions.

[0025] Step 4.2: Introduce two neural networks, the main network and the auxiliary network, into the Critic network. The main network directly provides a value assessment of the current action to guide the Actor network optimization strategy; the auxiliary network provides a certain target value through a soft update mechanism.

[0026] The Step 5 is specifically as follows:

[0027] Step 5.1: Determine key process parameters related to carbon monoxide and tin production based on actual operating conditions of the tin smelting process. The key process parameters include smelting temperature, furnace pressure, oxygen concentration, tin oxide addition amount, and carbon addition amount.

[0028] Step 5.2: Based on the chemical reaction equation of the tin smelting process, a thermodynamic equilibrium model is established to calculate the amount of carbon monoxide and tin produced.

[0029] The Step 6 is specifically as follows:

[0030] Step 6.1: Use the calculated tin production and carbon monoxide production as inputs to the reward function, which is:

[0031]

[0032] Where, is the reward function, 、 、 is a weight parameter used to balance the importance of different optimization objectives, Sn t Indicates the amount of tin generated in the furnace at the current moment, CO t Indicates the amount of carbon monoxide generated in the furnace at the current moment, and Time is the reaction time;

[0033] Step 6.2: Collect real-time status data of the furnace, and the optimal strategy model recommends the optimal action based on the real-time status data of the furnace.

[0034] The specific steps of Step 2.2 are as follows:

[0035]

[0036] Where, is the window variance ratio, is the data variance in the sliding window centered at time t,

[0037] is the global data variance.

[0038] The specific steps of Step 3.3 are as follows:

[0039] Organize industrial data of tin smelting process into triples , where Indicates the current state. Indicates the operation performed at the current moment. Indicates the state at the next moment;

[0040] Design an LSTM network, use the current state and action as input features, and the state at the next moment as the target variable for training;

[0041] By integrating metallurgical knowledge and using the parameter values calculated by chemical reactions to constrain the parameter values predicted by the LSTM network, dynamic optimization of state updates can be achieved.

[0042] The beneficial effects of the present invention are:

[0043] (1) Automation and intelligence improvement: Replacing traditional manual control and optimization methods with automated intelligent control greatly improves control accuracy and decision-making efficiency, reduces errors and dependence on manual operation, and improves the intelligence level of the production process;

[0044] (2) Reduce data acquisition costs: Through high-fidelity spectrum reconstruction and dynamic noise injection, the simulation data significantly expands the diversity of training samples while retaining the actual process physics laws, and enhances the model's adaptability to extreme working conditions and data missing scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is the overall framework diagram of the present invention;

[0046] Figure 2 It is a diagram of the execution process of the reinforcement learning model;

[0047] Figure 3 This is the experimental result diagram of the original Fourier synthesis method;

[0048] Figure 4 This is the experimental result diagram of the improved Fourier synthesis method. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0050] Example 1: Figure 1 As shown in FIG, a tin smelting chemical reaction optimization control method based on reinforcement learning is shown, and the specific steps are as follows:

[0051] Step 1: Collect parameter data during the smelting process.

[0052] Specifically, key smelting process data, including oxygen flow, spray gun position, coal flow, furnace pressure, smelting temperature, etc., are collected from the tin smelting plant database to ensure the comprehensiveness and timeliness of the data, providing accurate basic data for subsequent model training.

[0053] Furthermore, to verify the present invention's effect in simulating tin smelting process data and real-time dynamic optimization of chemical reactions in the furnace during the tin smelting process, this embodiment is based on actual operating data of an Ausmel furnace in a tin smelting plant.

[0054] Step 2: Preprocess the collected original parameter data, clean outliers and fill missing values to ensure data integrity and avoid data quality issues affecting subsequent model training and prediction. Then use the improved Fourier synthesis method to simulate the preprocessed parameter data to obtain simulated data.

[0055] Step 2.1: According to the sensor measurement range and the actual operating rules of the metallurgical process, set the parameter measurement range and the actual operating rule threshold, and eliminate abnormal values that exceed the parameter measurement range threshold and the parameter change rate exceeds the actual operating rule threshold;

[0056] Step 2.2: Based on the original Fourier synthesis method, an adaptive bottom feature detection algorithm is added to identify the steady-state interval of the parameter data. The formula is:

[0057]

[0058] Where, is the window variance ratio, is the data variance in the sliding window centered at time t,

[0059] is the global data variance. In this way, the improved Fourier synthesis method can better simulate the extreme working conditions in the original data.

[0060] Step 2.3: Use the improved Fourier synthesis algorithm to simulate and generate tin smelting parameter data to expand the scale of the reinforcement learning training set.

[0061] Specifically, the improved Fourier synthesis is performed on the pre-treated tin smelting parameters. First, the sliding window Fourier transform is used to decompose the time series data such as temperature and pressure. The main frequency components are filtered through the dynamic threshold formula to retain the original phase information. Then, outlier replacement and missing filling are implemented based on the window variance ratio. The mean hold method is used to generate data for the steady-state interval, and the transition interval is compensated by the phase extrapolation method to obtain simulation data that retains the real metallurgical process laws. Figure 3 and Figure 4 Experimental results show that the improved Fourier synthesis method can simulate and generate more realistic tin smelting parameter data.

[0062] Step 3: Build a reinforcement learning model, define the state space, action space, and design the reward function.

[0063] Step 3.1: Design the parameter values that affect the chemical reaction in the tin smelting furnace (such as oxygen flow rate, furnace pressure, furnace bottom temperature, oxygen concentration, etc.) as the state space, and define the controllable optimization variables in the smelting process (such as oxygen flow rate adjustment, coal flow rate adjustment, spray gun position adjustment, etc.) as the action space;

[0064] Step 3.2: The reward function is used as the optimization objective for reinforcement learning. It is designed based on the three goals of maximizing tin production, minimizing carbon monoxide emissions, and minimizing reaction time.

[0065] Step 3.3: Use the data and metallurgy dual-driven approach as the state update model for reinforcement learning.

[0066] Specifically, the industrial data of the tin smelting process is organized into triples , where Indicates the current state. Indicates the operation performed at the current moment. Indicates the state at the next moment;

[0067] Design an LSTM network, use the current state and action as input features, and the state at the next moment as the target variable for training;

[0068] By integrating metallurgical knowledge and using the parameter values calculated by chemical reactions to constrain the parameter values predicted by the LSTM network, dynamic optimization of state updates can be achieved.

[0069] Furthermore, during the tin smelting process, the main chemical reactions are:

[0070] SnO+C→Sn+CO

[0071] Where SnO represents tin monoxide. Then, by calculating the thermodynamic equilibrium constant K, the direction of the reaction and the ratio of reactants to products can be predicted. The calculation formula of the thermodynamic equilibrium constant K is:

[0072]

[0073] Where, is the standard free energy change (related to the reaction temperature), K represents the thermodynamic equilibrium constant, R = 8.314 J / (mol⋅K) is the gas constant, and T is the furnace temperature. The calculated K describes the relationship between reactants and products when a chemical reaction reaches equilibrium at a specific temperature and pressure. In the following formulas, K represents the thermodynamic equilibrium constant. Since the reactions are performed in the same furnace, the calculated K values are the same.

[0074] By combining data-driven (LSTM model prediction) and metallurgical knowledge (thermodynamic equilibrium and chemical reaction principles), dynamic optimization of state updates is achieved, providing more reliable environmental feedback for reinforcement learning.

[0075] Step 4: Design a dual network architecture for the Actor-Critic network in the reinforcement learning model, introducing a main network and an auxiliary network.

[0076] Step 4.1: Introduce two neural networks, the main network and the auxiliary network, into the Actor network. The main network is responsible for generating strategies and outputting optimized control actions for the current state. The auxiliary network interacts with the Critic network to provide action evaluation signals to the main network, guiding the main network to generate actions.

[0077] Step 4.2: The dual-network design of the Critic introduces two neural networks: a primary network and an auxiliary network. The primary network directly provides an assessment of the value of the current action, guiding the optimization strategy of the Actor network. The auxiliary network provides a stable target value through a soft update mechanism to avoid overestimation or oscillation of the Q value. This dual-network architecture, by introducing the auxiliary network in both the Actor and the Critic, separates action generation from action evaluation, strengthens the network's task-specific nature, and improves the model's learning efficiency and stability.

[0078] Step 5: Based on the tin smelting process parameters, a physical and chemical model is established, and the amount of carbon monoxide generated is calculated by combining the thermodynamic equilibrium formula and the chemical reaction formula in the furnace.

[0079] Step 5.1: Determine key process parameters related to carbon monoxide and tin production based on actual operating conditions of the tin smelting process. The key process parameters include smelting temperature, furnace pressure, oxygen concentration, tin oxide addition amount, and carbon addition amount.

[0080] Step 5.2: Based on the chemical reaction equation of the tin smelting process, a thermodynamic equilibrium model is established to calculate the amount of carbon monoxide and tin produced.

[0081] Specifically, the amount of tin produced is directly proportional to the amount of SnO added. The amount of tin produced when the reaction reaches equilibrium can be calculated using the thermodynamic equilibrium constant K. The thermodynamic equilibrium constant K can be defined as:

[0082]

[0083] The equilibrium concentration of carbon monoxide can be calculated using the following formula:

[0084]

[0085]

[0086] Since the stoichiometric ratio is 1:1, the amount of tin produced is equal to the amount of carbon monoxide produced. The ratio of carbon monoxide to O2 is 2:1, so the amount of tin produced = .

[0087] Step 6: Use the calculated carbon monoxide production as the reward function input of the optimized reinforcement learning model to guide the agent in the reinforcement learning to select the execution action.

[0088] Step 6.1: Use the calculated tin production and carbon monoxide production as inputs to the reward function, which is:

[0089]

[0090] Where, is the reward function, 、 、 is a weight parameter used to balance the importance of different optimization objectives, Sn t Indicates the amount of tin generated in the furnace at the current moment, CO t Indicates the amount of carbon monoxide generated in the furnace at the current moment, and Time is the reaction time;

[0091] Step 6.2: Collect real-time status data of the furnace. The optimal strategy model recommends the optimal action based on the real-time status data of the furnace to ensure the optimality and stability of the operation.

[0092] Specifically, after the reinforcement learning model is built, it is trained by interacting with the Ausmel furnace simulation environment. The reinforcement learning agent continuously explores and utilizes the environment, and continuously adjusts the control strategy based on the feedback of the reward function to optimize the goals such as tin production, carbon monoxide emissions and reaction time. Through multiple rounds of training, the model gradually converges and can automatically adjust the operating parameters under different working conditions to achieve fully automatic and real-time optimization of the tin smelting process. The operation process of reinforcement learning is as follows: Figure 2 shown.

[0093] Step 7: Use the generated simulation data and the collected parameter data to train the optimized reinforcement learning model and save the final reinforcement learning model parameters.

[0094] Specifically, the trained reinforcement learning model was validated in an actual tin smelting plant. By comparing process data before and after optimization, the improvement in tin production, the reduction in carbon monoxide emissions, and the shortened reaction time were evaluated. Comparative tests based on actual production data showed that the tin content in the discharged waste slag after applying the method of the present invention decreased by 0.5 percentage points, the average carbon monoxide emissions in the discharged waste gas dropped to 598 ppm (from 1600 ppm), and the reaction equilibrium time was shortened to 108±15 seconds (compared to 142±18 seconds under conventional control).

[0095] The above describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A tin smelting chemical reaction optimization control method based on reinforcement learning, characterized in that: The method specifically includes: Step 1: Collect parameter data during the smelting process; Step 2: Preprocess the collected original parameter data, clean outliers and fill missing values, and use the improved Fourier synthesis method to simulate the preprocessed parameter data to obtain simulated data; Step 3: Build a reinforcement learning model, define the state space, action space, and design the reward function; Step 4: Design a dual network architecture for the Actor-Critic network in the reinforcement learning model, introducing a main network and an auxiliary network; Step 5: Based on the tin smelting process parameters, a physical and chemical model is established, and the amount of carbon monoxide generated is calculated by combining the thermodynamic equilibrium formula and the chemical reaction formula in the furnace; Step 6: Use the calculated carbon monoxide production as the reward function input of the optimized reinforcement learning model to guide the agent in the reinforcement learning to select the action to perform; Step 7: Use the generated simulation data and the collected parameter data to train the optimized reinforcement learning model and save the final reinforcement learning model parameters; The Step 2 is specifically as follows: Step 2.1: According to the sensor measurement range and the actual operating rules of the metallurgical process, set the parameter measurement range and the actual operating rule threshold, and eliminate abnormal values that exceed the parameter measurement range threshold and the parameter change rate exceeds the actual operating rule threshold; Step 2.2: Add an adaptive bottom feature detection algorithm to the original Fourier synthesis method to identify the steady-state interval of the parameter data; Step 2.3: Use the improved Fourier synthesis algorithm to simulate and generate tin smelting parameter data to expand the scale of the reinforcement learning training set; The Step 3 is specifically as follows: Step 3.1: Design the parameter values that affect the chemical reaction in the tin smelting furnace as the state space, and define the controllable optimization variables in the smelting process as the action space; Step 3.2: The reward function is used as the optimization objective for reinforcement learning. It is designed based on the three goals of maximizing tin production, minimizing carbon monoxide emissions, and minimizing reaction time. Step 3.3: Use the data and metallurgy dual-driven approach as the state update model for reinforcement learning; The Step 4 is specifically as follows: Step 4.1: Introduce two neural networks, the main network and the auxiliary network, into the Actor network. The main network is responsible for generating strategies and outputting optimized control actions for the current state. The auxiliary network interacts with the Critic network to provide action evaluation signals to the main network, guiding the main network to generate actions. Step 4.2: Introduce two neural networks, the main network and the auxiliary network, into the Critic network. The main network directly provides a value assessment of the current action to guide the Actor network optimization strategy; the auxiliary network provides a fixed target value through a soft update mechanism. The Step 5 is specifically as follows: Step 5.1: Determine the key process parameters related to carbon monoxide and tin production based on the actual operating conditions of the tin smelting process; Step 5.2: Based on the chemical reaction equation of the tin smelting process, a thermodynamic equilibrium model is established to calculate the amount of carbon monoxide and tin produced; The Step 6 is specifically as follows: Step 6.1: Use the calculated tin production and carbon monoxide production as inputs to the reward function, which is: ； Where, is the reward function, 、 、 is a weight parameter used to balance the importance of different optimization objectives, Sn t Indicates the amount of tin generated in the furnace at the current moment, CO t Indicates the amount of carbon monoxide generated in the furnace at the current moment, and Time is the reaction time; Step 6.2: Collect real-time status data of the furnace, and the optimal strategy model recommends the optimal action based on the real-time status data of the furnace; The specific steps of Step 2.2 are as follows: ； Where, is the window variance ratio, is the data variance in the sliding window centered at time t, is the global data variance.

2. The tin smelting chemical reaction optimization control method based on reinforcement learning according to claim 1 is characterized in that: The specific steps of Step 3.3 are as follows: Organize industrial data of tin smelting process into triples , where Indicates the current state. Indicates the operation performed at the current moment. Indicates the state at the next moment; Design an LSTM network, use the current state and action as input features, and the state at the next moment as the target variable for training; By integrating metallurgical knowledge and using the parameter values calculated by chemical reactions to constrain the parameter values predicted by the LSTM network, dynamic optimization of state updates can be achieved.

Images