Hearth temperature control method and system

Abstract

The invention belongs to the technical field of temperature control, and provides a hearth temperature control method and system, and the method comprises the steps: obtaining the hearth temperature, the header pipe pressure, the opening degree of a gas valve for adjusting the gas flow, and the opening degree of an air valve for adjusting the air flow, carrying out the preprocessing of the obtained data, and extracting the features of the preprocessed data; according to the current features, obtaining candidate opening adjustment amounts of the gas valve and the air valve by using a pre-trained decision network model constructed based on a reinforcement learning method; according to the combustion condition of the hearth, multi-stage safety constraints are introduced, the candidate opening degree adjustment amount is corrected, the final opening degree adjustment amount is obtained, and the opening degree of the corresponding valve is controlled; a double-valve linkage mechanism of the gas valve and the air valve is adopted, the opening adjustment amount of the gas valve and the opening adjustment amount of the air valve are output at the same time in one reinforcement learning decision, fuel supply and the combustion air proportion are automatically matched, and cooperative control is achieved.

Description

Technical Field

[0001] This invention belongs to the field of temperature control technology, specifically relating to a furnace temperature control method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Sintering is one of the core processes in steel production, involving the high-temperature sintering of iron ore, fuel, and additives into sinter. The quality of sinter directly impacts the blast furnace's production efficiency, energy consumption, and stability. Currently, the global steel industry faces the dual pressures of energy conservation and emission reduction, as well as green manufacturing; therefore, temperature control optimization has become a crucial breakthrough for enhancing competitiveness.

[0004] The following key issues exist in the current production process: frequent manual operation and slow response make it difficult to quickly adjust when the furnace temperature deviates from the normal range (generally 1030°C~1130°C), which affects the consistency of batch quality; at the same time, due to factors such as fluctuations in the calorific value of the gas, changes in the moisture content of the raw materials and environmental disturbances, the furnace temperature fluctuates greatly and the temperature difference between the left and right furnaces is significant, which further exacerbates the problem of uneven combustion.

[0005] Furthermore, the mismatch between the gas and air supply ratio results in low energy utilization and high fuel consumption. Although operators possess mature adjustment experience, they lack a data-driven and algorithmic experience transfer system, making it impossible to efficiently translate this experience into automated control capabilities. At the same time, the existing monitoring system has a single dimension and cannot comprehensively and in real-time judge multiple influencing factors, resulting in insufficient precision in adjustment strategies and limited overall operational efficiency. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a furnace temperature control method and system. This invention comprehensively considers multiple influencing factors and utilizes reinforcement learning to achieve precise temperature control and improve the efficiency of steel sintering.

[0007] According to some embodiments, the present invention adopts the following technical solution: A furnace temperature control method includes the following steps: The furnace temperature, main pipe pressure, gas valve opening for regulating gas flow, and air valve opening for regulating air flow are obtained. The acquired data is preprocessed, and the features of the preprocessed data are extracted. Based on the current characteristics, the candidate opening adjustment amounts of the gas valve and air valve are obtained by using a pre-trained decision network model constructed based on reinforcement learning. Based on the combustion conditions in the furnace, multi-level safety constraints are introduced to correct the candidate opening adjustment amount, so as to obtain the final opening adjustment amount and control the opening of the corresponding valve.

[0008] As an alternative implementation, the training process of the decision network model constructed based on the reinforcement learning method includes: Historical data and / or simulation data of furnace temperature, main pipe pressure, gas valve opening degree and air valve opening degree signals are acquired at a set sampling period. The acquired data is preprocessed, the features of the preprocessed data are extracted, and training samples are constructed according to the format of state action and reward. The gas valve opening adjustment amount and the air valve opening adjustment amount are used as the output of the decision network model. An out-of-distribution action penalty term is introduced for conservative Q-learning, which applies additional penalties to Q-value actions that exceed a threshold but do not appear in the acquired data.

[0009] As a further defined implementation, during the training process of the decision network model, when the temperature is within the target range and the furnace temperature difference is less than the set value and the energy consumption is lower than the set value, a positive reward exceeding the set value is given; when the temperature deviates from the target range or the distance from the safety limit is less than the set threshold, a negative reward exceeding the predetermined value is given.

[0010] As an alternative implementation, the decision network model includes an input layer, an intermediate layer, and an output layer. The input layer is used to receive the extracted features, the intermediate layer adopts a multi-layer feedforward neural network structure to capture the time lag of furnace temperature and the nonlinear influence of gas-air coupling on furnace temperature, and the output layer is used to output the obtained candidate opening adjustment amount.

[0011] As an alternative implementation method, the process of preprocessing the acquired data includes: physical threshold filtering and outlier removal of the acquired data; when a certain data exceeds its corresponding set range, the corresponding record is deleted; if the change between adjacent sampling points exceeds the threshold, the corresponding sampling point is regarded as an anomaly and removed.

[0012] As an alternative implementation, the features include the current temperature of each furnace chamber, the furnace temperature difference, the furnace temperature change, the upper and lower limits of the target furnace temperature range, the current opening degree of the gas valve and air valve, and the current pressure and its change in the main gas pipe.

[0013] As an alternative implementation method, the process of introducing multi-level safety constraints includes introducing action amplitude constraints, absolute opening boundary constraints, furnace temperature-safety logic constraints, and pressure and interlocking constraints.

[0014] As a further defined implementation, the action amplitude constraint is used to limit the opening degree change of the gas valve and air valve within a single control cycle to not exceed a preset value. Absolute opening boundary constraints are used to set minimum and maximum allowable openings for gas valves and air valves according to furnace structure and equipment safety specifications. When the candidate opening adjustment exceeds the above range, it is truncated by a saturation function to limit it to the safe operating range. The furnace temperature-safety logic constraint is used to prohibit any further positive adjustment of the gas supply when the furnace temperature on either side approaches or exceeds the process limit, thus imposing a mandatory constraint; when the temperature is less than the equipment safety limit value, the emergency cooling logic is triggered, which forcibly reduces the gas valve opening and appropriately increases the air valve opening as needed to reduce the furnace temperature and increase the safety margin. Pressure and interlock constraints are used to switch the valve opening adjustment to a preset safety annealing strategy when the main gas pipe pressure exceeds the set safety range or when a gas system safety interlock warning signal is received.

[0015] As an alternative implementation, based on multi-level security constraints, a limited-range ε-greedy security exploration mechanism is introduced. Within each control cycle, a probability (1 / 2) is used to explore the security within a certain range. ε) Directly use the output of the decision network model and constrain the modified greedy action to select an exploratory action in the neighborhood of the safe action with probability ε; The safety action neighborhood is designed based on the furnace temperature characteristics. Centered on the current valve opening, it limits the exploration range of the gas valve adjustment amount and the air valve adjustment amount, and this exploration range is smaller than the adjustment range allowed in the multi-level safety constraints.

[0016] As a further defined implementation, within the period of triggering exploration, exploration actions are randomly sampled from the defined range according to a preset distribution, and these actions are superimposed or replaced with the actions output by the decision network model. Multi-level safety constraints are then introduced again to perform amplitude and boundary verification.

[0017] As a further defined implementation, the limited-range ε-greedy safety exploration mechanism is activated only when the furnace temperature is within the target range, the furnace temperature difference is lower than the predetermined equilibrium threshold, and there is no over-temperature or under-temperature trend within a continuously set number of sampling periods. When the temperature deviates from the target range, the furnace temperature difference increases, or an abnormal pressure occurs, ε is decayed to 0, and only a greedy strategy is used to perform steady-state control.

[0018] As an alternative implementation, the steps of the method are executed iteratively. When the furnace temperature remains stable within the target temperature range over multiple sampling periods and the furnace temperature difference continuously meets the equilibrium threshold requirement, the candidate opening adjustment amount obtained by the decision network model approaches zero or is less than the set value, and a steady-state control mode is entered. In this mode, the control strategy prioritizes suppressing actions with changes greater than the threshold. Only when a temperature trend deviation caused by changes in the main gas pipe pressure, ambient temperature disturbances, or changes in raw material properties is detected, a new significant adjustment action is triggered.

[0019] A furnace temperature control system, comprising: The real-time data acquisition module is used to acquire furnace temperature, main pipe pressure, gas valve opening for regulating gas flow, and air valve opening for regulating air flow. The data cleaning and feature construction module is used to preprocess the acquired data and extract features from the preprocessed data. The reinforcement learning decision module is used to obtain the candidate opening adjustment amount of the gas valve and the air valve based on the current features and a pre-trained decision network model built on the reinforcement learning method. The safety strategy module is used to introduce multi-level safety constraints based on the furnace combustion conditions, correct the candidate opening adjustment amount, and obtain the final opening adjustment amount to control the opening of the corresponding valve.

[0020] As an alternative implementation, the real-time data acquisition module includes a temperature sensor for acquiring the furnace temperature at the furnace outlet. The main gas pressure sensor is installed in the main gas supply pipeline to measure the gas pressure inside the furnace in real time. The gas control valve is located on the main gas supply pipeline and is used to regulate the gas flow rate. An air control valve, located in the combustion air line, is used to regulate airflow.

[0021] As an alternative implementation, an execution module is also included, including a PLC control unit, for receiving the final opening adjustment amount after correction by the safety strategy module, driving the electric actuator of the gas control valve to change the gas supply flow rate, and simultaneously driving the electric actuator of the air control valve to change the combustion air flow rate.

[0022] An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps in the method described above.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention innovatively provides a reinforcement learning-based collaborative control strategy that leverages the coupling characteristics of dual-valve linkage and furnace. Employing a dual-valve linkage mechanism involving both gas and air valves, the opening adjustments of both valves are simultaneously output during a single reinforcement learning decision. This automatically matches the fuel supply with the combustion air ratio, achieving collaborative control tailored to the characteristics of the sinter roasting furnace: coupling of the left and right temperature fields within the furnace and the mutual influence between combustion rate and oxygen supply ratio. Compared to traditional solutions that only adjust a single valve, this invention significantly suppresses furnace temperature fluctuations caused by lag in single-valve adjustment through dual-valve linkage, improving furnace temperature stability to over 95%, with deviations controlled within ±20°C. This ensures stable sinter crystallization quality and reduces production line quality fluctuations caused by batches with temperature deviations.

[0024] This invention proposes a method for explicit feature embedding of furnace temperature difference and dual-furnace thermal balance control. The invention constructs the furnace temperature difference using independent temperature measurement data from the left and right furnaces, and explicitly inputs this difference as a state feature into a conservative Q-learning decision network. Simultaneously, it combines the temperature change rate characteristics of the left and right furnaces, enabling the model to perceive the distribution and dynamic changes of the furnace thermal field. When the furnace temperature difference exceeds a set threshold, the strategy can proactively adjust the gas distribution or air flow to maintain a relative balance between the heat input and combustion intensity of the left and right furnaces. This mechanism effectively reduces the temperature difference between the left and right furnaces, significantly reducing the structural differences in sinter caused by uneven heat distribution, and fundamentally improving the consistency of the semi-finished product.

[0025] This invention introduces a conservative Q-learning (CQL) algorithm during the offline training phase. By imposing additional penalties on extreme actions outside the historical data distribution in the Q-value function loss, the learned policy maintains a conservative estimate in the unseen state-action region. During the online operation phase, it combines action amplitude constraints, absolute opening boundary constraints, and furnace temperature-pressure logic constraints to safely filter the opening adjustment amount of each CQL output. Through the integrated design of conservative model and online constraints, it avoids the risk of combustion instability and equipment overload caused by drastic adjustments in industrial settings, significantly improving the usability and safety of reinforcement learning control in high-temperature industrial furnace scenarios.

[0026] This invention achieves rapid tracking of furnace temperature trends by combining short-cycle sampling with single-step decision-making. The invention allows for flexible setting of the sampling period and the construction of a single-step CQL decision network, using furnace temperature changes and mains pressure changes as explicit inputs. This enables the model to capture fluctuations in gas calorific value and furnace temperature trends on a short timescale, reducing decision delay. Compared to traditional control strategies that rely on long-term window statistics, this invention significantly shortens the furnace temperature recovery time, controlling the recovery time of the target temperature range to within approximately 15 seconds, thus reducing excessive energy consumption and process risks caused by temperature control lag.

[0027] This invention features adaptive suppression of abnormal pressure disturbances and multi-source interlocking protection. Addressing potential short-term pressure fluctuations in the main gas pipeline at industrial sites, this invention introduces embedded pressure-valve linkage logic. When pressure changes naturally push the furnace temperature back towards the target range, unnecessary valve adjustments are suppressed to avoid over-adjustment and energy fluctuations. When a pressure exceedance or safety interlock signal is detected, the output of the decision network is automatically overridden and switched to a safe annealing strategy, gradually reducing the gas flow to prevent abnormal combustion. This adaptive logic based on pressure anomalies not only reduces the frequency of system actions and actuator wear but also improves the overall safety redundancy of the temperature control system.

[0028] This invention first employs a conservative Q-learning algorithm for offline training based on historical furnace operation data to obtain a stable basic control strategy, ensuring reliability and safety under typical operating conditions. During field operation, a limited-amplitude epsilon-greedy exploration mechanism is introduced, activated only when the furnace temperature difference meets the equilibrium requirements within the target temperature range and no over-temperature / under-temperature trend appears for several consecutive cycles. Under strict safety boundary constraints, this mechanism attempts local perturbations with a low probability to finely adjust the gas-air ratio. This ensures the stability and safety of furnace temperature control while enabling the strategy to gradually adapt to the slowly changing characteristics of gas quality, environmental conditions, and equipment status, extending the strategy's effective lifespan and reducing the need for redevelopment and frequent manual intervention.

[0029] This invention proposes a robust temperature control decision-making mechanism driven by full parameter acquisition and multi-feature fusion. This invention not only acquires temperature and pressure signals but also simultaneously obtains the opening status of actuators such as gas valves and air valves, and constructs various derived features including temperature change rate, pressure change, and furnace temperature difference to achieve a multi-dimensional characterization of furnace operating conditions. Based on this, a state input vector-driven CQL decision network is constructed, making the control strategy more robust and anti-interference to fuel calorific value fluctuations, environmental disturbances, and equipment aging, maintaining high temperature control accuracy even under complex operating conditions.

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0031] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0032] Figure 1 This is a schematic diagram of the furnace temperature control system architecture of one embodiment; Figure 2 This is a control logic flowchart of one embodiment. Detailed Implementation

[0033] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0034] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0035] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0036] Where there is no conflict, the embodiments and features described in this application may be combined with each other.

[0037] Example 1 A furnace temperature control method includes the following steps: Step S1: Obtain the furnace temperature, main pipe pressure, gas valve opening for regulating gas flow, and air valve opening for regulating air flow. Step S2: Preprocess the acquired data and extract the features of the preprocessed data; Step S3: Based on the current features, use the pre-trained decision network model built on reinforcement learning to obtain the candidate opening adjustment amount of the gas valve and the air valve. Step S4: Based on the furnace combustion conditions, introduce multi-level safety constraints to correct the candidate opening adjustment amount and obtain the final opening adjustment amount to control the opening of the corresponding valve.

[0038] In this embodiment, the training process of the decision network model constructed based on the reinforcement learning method in step S3 includes: Historical data and / or simulation data of furnace temperature, main pipe pressure, gas valve opening degree and air valve opening degree signals are acquired at a set sampling period. The acquired data is preprocessed, the features of the preprocessed data are extracted, and training samples are constructed according to the format of state action and reward. The gas valve opening adjustment amount and the air valve opening adjustment amount are used as the output of the decision network model. An out-of-distribution action penalty term is introduced for conservative Q-learning, which applies additional penalties to Q-value actions that exceed a threshold but do not appear in the acquired data.

[0039] In this embodiment, during the training process of the decision network model, when the temperature is within the target range and the furnace temperature difference is less than the set value and the energy consumption is lower than the set value, a positive reward exceeding the set value is given; when the temperature deviates from the target range or the distance from the safety limit is less than the set threshold, a negative reward exceeding the predetermined value is given.

[0040] In this embodiment, the decision network model includes an input layer, an intermediate layer, and an output layer. The input layer is used to receive the extracted features, the intermediate layer adopts a multi-layer feedforward neural network structure to capture the time lag of furnace temperature and the nonlinear influence of gas-air coupling on furnace temperature, and the output layer is used to output the obtained candidate opening adjustment amount.

[0041] In this embodiment, the process of preprocessing the acquired data in step S2 includes: physical threshold filtering and outlier removal of the acquired data; when a certain data exceeds its corresponding set range, the corresponding record is deleted; if the change between adjacent sampling points exceeds the threshold, the corresponding sampling point is regarded as an anomaly and removed.

[0042] Features include the current temperature of each furnace chamber, furnace temperature difference, furnace temperature change, upper and lower limits of the target furnace temperature range, current opening degree of gas valve and air valve, and current pressure and change of the main gas pipe.

[0043] The process of introducing multi-level safety constraints in step S4 includes introducing action amplitude constraints, absolute opening boundary constraints, furnace temperature-safety logic constraints, and pressure and interlocking constraints, wherein: The action amplitude constraint is used to limit the opening degree change of the gas valve and air valve within a single control cycle to not exceed a preset value. Absolute opening boundary constraints are used to set minimum and maximum allowable openings for gas valves and air valves according to furnace structure and equipment safety specifications. When the candidate opening adjustment exceeds the above range, it is truncated by a saturation function to limit it to the safe operating range. The furnace temperature-safety logic constraint is used to prohibit any further positive adjustment of the gas supply when the furnace temperature on either side approaches or exceeds the process limit, thus imposing a mandatory constraint; when the temperature is less than the equipment safety limit value, the emergency cooling logic is triggered, which forcibly reduces the gas valve opening and appropriately increases the air valve opening as needed to reduce the furnace temperature and increase the safety margin. Pressure and interlock constraints are used to switch the valve opening adjustment to a preset safety annealing strategy when the main gas pipe pressure exceeds the set safety range or when a gas system safety interlock warning signal is received.

[0044] In some embodiments, based on the introduction of multi-level security constraints, a limited-range ε-greedy security exploration mechanism is introduced. Within each control cycle, the ε-greedy exploration mechanism is explored with a probability (1 / 2) of 1 / 2 ≤ ... ε) Directly use the output of the decision network model and constrain the modified greedy action to select an exploratory action in the neighborhood of the safe action with probability ε; The safety action neighborhood is designed based on the furnace temperature characteristics. Centered on the current valve opening, it limits the exploration range of the gas valve adjustment amount and the air valve adjustment amount, and this exploration range is smaller than the adjustment range allowed in the multi-level safety constraints.

[0045] Within the period of triggering exploration, exploration actions are randomly sampled from the limited range according to a preset distribution, and these actions are superimposed or replaced with the actions output by the decision network model. Multi-level safety constraints are then introduced again to perform amplitude and boundary verification.

[0046] The limited-range ε-greedy safety exploration mechanism is activated only when the furnace temperature is within the target range, the furnace temperature difference is lower than the predetermined equilibrium threshold, and there is no over-temperature or under-temperature trend within a set number of consecutive sampling periods. When the temperature deviates from the target range, the furnace temperature difference increases, or an abnormal pressure occurs, ε is decayed to 0, and only a greedy strategy is used to perform steady-state control.

[0047] In some embodiments, the steps of the method are executed iteratively. When the furnace temperature remains stable within the target temperature range over multiple sampling periods and the furnace temperature difference continuously meets the equilibrium threshold requirement, the candidate opening adjustment amount obtained by the decision network model approaches zero or is less than the set value, and the system enters a steady-state control mode. In this mode, the control strategy prioritizes suppressing actions with changes greater than the threshold. Only when a temperature trend deviation caused by changes in the main gas pipe pressure, ambient temperature disturbances, or changes in raw material properties is detected, a new significant adjustment action is triggered.

[0048] Example 2 A furnace temperature control system, comprising: The real-time data acquisition module is used to acquire furnace temperature, main pipe pressure, gas valve opening for regulating gas flow, and air valve opening for regulating air flow. The data cleaning and feature construction module is used to preprocess the acquired data and extract features from the preprocessed data. The reinforcement learning decision module is used to obtain the candidate opening adjustment amount of the gas valve and the air valve based on the current features and a pre-trained decision network model built on the reinforcement learning method. The safety strategy module is used to introduce multi-level safety constraints based on the furnace combustion conditions, correct the candidate opening adjustment amount, and obtain the final opening adjustment amount to control the opening of the corresponding valve.

[0049] The real-time data acquisition module includes a temperature sensor for acquiring furnace temperature at the furnace outlet. The main gas pressure sensor is installed in the main gas supply pipeline to measure the gas pressure inside the furnace in real time. The gas control valve is located on the main gas supply pipeline and is used to regulate the gas flow rate. An air control valve, located in the combustion air line, is used to regulate airflow.

[0050] This embodiment also includes an execution module, comprising a PLC control unit, used to receive the final opening adjustment amount after correction by the safety strategy module, drive the electric actuator of the gas control valve to change the gas supply flow rate, and simultaneously drive the electric actuator of the air control valve to change the combustion air flow rate. In this embodiment, the PLC control unit adopts the Siemens S7-300 series to realize signal acquisition, logic execution, and valve control, and supports Snap7 communication.

[0051] The following is a detailed description.

[0052] In this embodiment, as Figure 1 As shown, the real-time data acquisition module runs on an industrial PC. It reads signals such as furnace temperature, main pipe pressure, gas valve opening and air valve opening from the PLC's DB block at a sampling period of 1 second via the Snap7 protocol, and writes the data to the local historical database. It also supports batch export as a structured CSV file.

[0053] The data cleaning and feature construction module is used to perform physical threshold filtering and outlier removal on the collected data. For example, if the temperature exceeds the range of 0~1400°C or the pressure exceeds the set safety limit, the corresponding record will be deleted. If the temperature change between adjacent sampling points exceeds ±50°C, the current sampling point is considered an anomaly and is removed. Based on the cleaned data, key parameters for offline reinforcement learning are generated, including the temperature change ΔT_left, ΔT_right, the temperature difference between the left and right furnaces ΔT_lr, and the pressure change in the main pipe, etc. The training samples are output in the format of (state, action, target) to ensure that the model input is highly matched with the industrial scenario.

[0054] The reinforcement learning decision-making module constructs a single-step decision network based on the Conservative Q-Learning (CQL) algorithm. It inputs the current state features into the model and outputs instructions for adjusting the opening of the gas valve and air valve. This output is filtered by a safety constraint algorithm, such as limiting the adjustment range to no more than a preset maximum value, before being sent to the analog output port of the PLC.

[0055] In some embodiments, an online fine-tuning module is also included, which is used to conduct a limited range of safety exploration using an ε-greedy strategy and to perform incremental training after collecting new operating condition data, so as to ensure that the control strategy can adapt to gas quality fluctuations, changes in environmental conditions and equipment aging in real time, without exceeding the process safety boundary.

[0056] In this system, the output signals from the temperature and pressure sensors are uniformly converted into analog signals by the signal conditioning module and directly connected to the analog input of the PLC, with a fixed sampling period of 1 second. The analog output of the PLC is connected to the actuators of the gas valve and air valve, respectively. Control commands are transmitted with millisecond-level signal refresh to ensure fast response. The host computer and the PLC exchange data and commands via industrial Ethernet, using the Snap7 communication protocol to ensure high-speed, non-blocking transmission. The overall hardware layout adopts a structure of symmetrical temperature measurement on both sides of the furnace, centralized control cabinet installation, and point-to-point signal control to reduce signal path length, lower latency, and ensure that the feedback closed-loop control of the dual-valve actuator can verify the consistency between the opening command and the actual position in real time.

[0057] Of course, in other embodiments, the above modules / units can be selected from other models or structures, or use other communication protocols. The above changes can be flexibly selected according to specific scenarios, environments and control requirements, and will not be elaborated here. Moreover, the above selections are easy for those skilled in the art to think of, and should fall within the protection scope of this invention.

[0058] The following is in the order of operation, such as Figure 2 The diagram illustrates the operation process of each module.

[0059] In the initial stage of system operation, managers input and issue the corresponding target temperature range (e.g., 1030°C~1130°C) and the upper limit of allowable furnace temperature difference within the temperature control system platform, based on process requirements and production batch information. The system platform communicates with the host computer PC via industrial Ethernet. The host computer stores the process temperature range and control parameters in the current batch operation strategy configuration file and simultaneously loads it into the offline reinforcement learning (CQL) decision module for real-time calculation of target deviation correction.

[0060] The PLC control unit uses high-temperature thermocouple temperature sensors connected to the left and right outlets of the furnace, and a pressure sensor installed on the main gas pipe, to read the temperature values ​​of the left and right furnaces, the pressure value of the main pipe, and the feedback signals of the current opening positions of the gas valve and air valve at a sampling period of 1 second (in other embodiments, the sampling period can be flexibly set as needed). This sampled data is uploaded to the industrial PC data acquisition module, which automatically inputs the raw data into the data cleaning and feature construction module for processing. This process yields features including calculating the current temperature changes ΔT_left and ΔT_right, obtaining the furnace temperature difference ΔT_lr, and the main pipe pressure change, and integrating them to form the state input vector of the reinforcement learning model.

[0061] After receiving the latest state input vector, the reinforcement learning decision module calls the conservative Q-learning (CQL) single-step decision network trained on the historical safe action space to calculate the candidate opening adjustment amount of the current gas valve and air valve.

[0062] In this embodiment, the reinforcement learning decision module is constructed for the furnace temperature control object. Its input layer receives data, including the current temperature of the left furnace T_left, the current temperature of the right furnace T_right, the furnace temperature difference ΔT_lr, the temperature change of the left and right furnaces ΔT_left, ΔT_right, the upper and lower limits of the furnace temperature target range, the current opening degree of the gas valve and air valve, the current pressure of the gas main and its change, and other status features. The output layer consists of two continuous motion components, representing the gas valve opening adjustment amount Δu_gas and the air valve opening adjustment amount Δu_air, respectively.

[0063] The intermediate layer employs a multi-layer feedforward neural network structure to capture the time lag of furnace temperature and the nonlinear effect of gas-air coupling on furnace temperature.

[0064] During the offline training phase, the system constructs a state-action-reward-next state (i.e., s, a, r, s') sample set based on historical production data and simulation data. The reward function is designed with weights based on factors such as the degree to which the furnace temperature deviates from the target range, the size of the furnace temperature difference, the amount of fuel consumed, and the over-temperature penalty. A higher positive reward is given when the temperature is in the middle of the target range and the furnace temperature difference is small and the energy consumption is low; a larger negative reward is given when the temperature deviates from the target range or approaches the safety limit.

[0065] In this embodiment, the reinforcement learning decision module introduces an out-of-distribution action penalty term of conservative Q-learning (CQL) to impose additional penalties on high Q-value actions that do not appear in historical data. This makes the Q-network conservative when estimating the value of these unseen actions, thereby ensuring that the control strategy is more inclined to the safe action area that has been verified in history, and adapts to the high requirements of safety and stability in the furnace temperature control scenario.

[0066] During the online operation phase, the reinforcement learning decision module feeds the state vector constructed in the previous control cycle into the trained single-step decision network to obtain the candidate opening adjustment (Δu_gas*, Δu_air*) for the current cycle. The candidate adjustment is then processed by the safety policy module for constraint handling before output.

[0067] In this embodiment, the safety strategy module sets multi-level constraint rules for furnace combustion conditions, specifically including: (1) Action range constraint: The opening degree of the gas valve and air valve is limited to no more than the preset value Δu_gas_max=3 and Δu_air_max=3 in a single control cycle, that is, |Δu_gas*|≤Δu_gas_max and |Δu_air*|≤Δu_air_max, so as to avoid the furnace temperature from fluctuating drastically or the combustion from unstable due to instantaneous large adjustment; (2) Absolute opening boundary constraints: According to the furnace structure and equipment safety specifications, the minimum and maximum allowable openings of the two valves are set as u_gas_min≤u_gas≤u_gas_max and u_air_min≤u_air≤u_air_max; When the candidate opening u_candidate=u_current+Δu_gas* or u_current+Δu_air* exceeds the above range, it is truncated by the saturation function to limit it to the safe operating range; (3) Furnace temperature-safety logic constraint: When the furnace temperature on either side is detected to be close to or exceed the process upper limit (e.g., T_left or T_right ≥ T_high_limit), When ΔT_safe), any further positive adjustment to increase the gas supply is prohibited, and Δu_gas ≤ 0 is forcibly constrained; when the temperature approaches the equipment safety limit, the emergency cooling logic is triggered, forcibly reducing the gas valve opening and appropriately increasing the air valve opening as needed to reduce the furnace temperature and increase the safety margin. (4) Pressure and interlock constraints: When the pressure of the main gas pipe exceeds the set safety range, or when the host computer detects a safety interlock warning signal of the gas system, the safety strategy module automatically overrides the output of the reinforcement learning decision module and switches the valve opening adjustment command to the preset safety annealing strategy. For example, the gas valve opening is gradually reduced to the safety lower limit in multiple cycles, and the adjustment range of the air valve is limited to avoid abnormal combustion caused by abnormal pressure disturbance.

[0068] After being constrained by the safety strategy module, the final opening adjustment amount (Δu_gas, Δu_air) that satisfies the safety boundary of furnace operation is obtained. The industrial PC packages and generates a valve opening control command data frame that can be recognized by the PLC, and sends it to the PLC control unit for execution.

[0069] After receiving the valve opening adjustment command from the industrial PC, the PLC control unit drives the electric actuator of the gas control valve to change the gas supply flow rate, and simultaneously drives the electric actuator of the air control valve to change the combustion air flow rate. The coordinated action of the two valves immediately changes the combustion rate and oxygen supply ratio in the furnace. In this embodiment, the initial action response time is controlled within 10 seconds to meet the requirement of rapid correction of furnace temperature deviation.

[0070] The temperature sensors in the left and right furnace chambers detect new temperature values ​​and furnace temperature differences in the next sampling cycle and transmit them back to the PLC and industrial PC. Based on the new temperature, pressure, and valve opening feedback data, the data cleaning and feature construction module recalculates the temperature change ΔT_left, ΔT_right, furnace temperature difference ΔT_lr, and pressure change to determine whether the current furnace temperature has entered the target range and whether the furnace temperature difference meets the set threshold, and forms the state input vector for the new cycle.

[0071] If the detected temperature is still higher than the target upper limit or lower than the target lower limit, or the furnace temperature difference exceeds the allowable upper limit, the reinforcement learning decision module will calculate the opening correction action again based on the updated state vector in the next control cycle, repeating the above decision-constraint-execution process to achieve closed-loop adjustment of the furnace temperature.

[0072] When the system detects that the furnace temperature has entered and remained stable within the target temperature range over multiple sampling cycles, and the furnace temperature difference continuously meets the equilibrium threshold requirement, the optimal action output by the reinforcement learning decision module approaches zero adjustment or only maintains a small fine-tuning, and the system enters steady-state control mode.

[0073] In this mode, the control strategy prioritizes suppressing large-scale changes in behavior. New significant adjustment actions are only triggered when changes in the main gas pipe pressure, ambient temperature disturbances, or deviations in the temperature trend caused by changes in the properties of raw materials are detected. This reduces frequent valve operations and mechanical wear on equipment while ensuring the target temperature is maintained.

[0074] Throughout the operation, the industrial PC stores each sampled status data, model output action data, actual valve opening feedback, adjusted temperature values, and furnace temperature difference into a local historical database. It also exports the data as production operation log files at set intervals for subsequent offline strategy improvement and process analysis. Simultaneously, key temperature control information is displayed in real-time via the system platform's user interface, including current temperature, furnace temperature difference, valve action status, and deviation correction trend curves.

[0075] To continuously optimize the furnace temperature control strategy while ensuring safe production, this system introduces a limited-range ε-greedy safety exploration mechanism based on the conservative Q-learning strategy. Within each control cycle, the system explores the furnace temperature with a probability (1... ε) The greedy action is directly adopted from the output of the CQL single-step decision network and constrained by the safety policy module, and an exploratory action is selected in the neighborhood of the safe action with probability ε.

[0076] Among them, the safety action neighborhood is specially designed according to the furnace temperature characteristics: taking the current valve opening as the center, the exploration range of the gas valve adjustment amount and air valve adjustment amount is limited to |Δu_gas_explore|≤Δu_gas_explore_max, |Δu_air_explore|≤Δu_air_explore_max, and Δu_gas_explore_max and Δu_air_explore_max are strictly less than the maximum adjustment range allowed by the safety strategy module.

[0077] Within the period of triggering exploration, the system randomly samples exploration actions from the limited range according to a preset distribution, superimposes or replaces them with the output actions of the reinforcement learning decision module, and then performs amplitude and boundary verification again through the safety policy module to ensure that the exploration actions do not cause drastic fluctuations in furnace temperature or exceed the safety boundary.

[0078] Meanwhile, the ε-greedy exploration is only activated when certain furnace stability conditions are met: for example, the furnace temperatures on both sides are in the middle of the target range for multiple consecutive sampling periods, the furnace temperature difference is below the predetermined equilibrium threshold, and there is no over-temperature or under-temperature trend. When a temperature deviation from the target range, an increase in the furnace temperature difference, or an abnormal pressure is detected, the system automatically decays ε to 0 and only uses a greedy strategy to perform steady-state control. Through the above-mentioned limited-range and conditionally triggered ε-greedy safety exploration mechanism, the system can conduct online testing of small-range valve opening combinations in furnace temperature control scenarios with high safety requirements, gradually improving the gas... The air ratio and energy efficiency indicators are optimized, while avoiding the risks to furnace safety caused by boundless exploration in traditional reinforcement learning.

[0079] Example 3 A computer-readable storage medium for storing computer instructions, which, when executed by a processor, perform the steps in the method provided in Embodiment 1.

[0080] Example 4 An electronic device includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, wherein the computer instructions, when executed by the processor, perform the steps in the method provided in Embodiment 1.

[0081] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of one or more computer-usable storage media (including, but not limited to, disk storage, etc.) containing computer-usable program code. CD - ROMIt takes the form of a computer program product implemented on (such as optical memory, etc.).

[0082] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art without creative effort within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A hearth temperature control method, characterized by, The method comprises the following steps: obtaining the furnace temperature, the total pipe pressure, the gas valve opening degree for adjusting the gas flow and the air valve opening degree for adjusting the air flow; preprocessing the obtained data and extracting the features of the preprocessed data; obtaining the candidate opening degree adjustment amount of the gas valve and the air valve by using a decision network model constructed by a pre-trained reinforcement learning method according to the current features; correcting the candidate opening degree adjustment amount according to the multi-level safety constraints to obtain the final opening degree adjustment amount for controlling the opening degree of the corresponding valve according to the furnace combustion condition.

2. A method of furnace temperature control as claimed in claim 1, characterized in that, The training process of the decision network model constructed by the reinforcement learning method comprises: obtaining the historical data or / and simulation data of the furnace temperature, the total pipe pressure, the gas valve opening degree and the air valve opening degree at a set sampling period; preprocessing the obtained data, extracting the features of the preprocessed data, constructing training samples in the format of state action and reward, and taking the gas valve opening degree adjustment amount and the air valve opening degree adjustment amount as the output of the decision network model; introducing a conservative Q-learning distribution outside action penalty term to additionally punish the Q-value action exceeding the threshold value which does not appear in the obtained data.

3. A method of furnace temperature control as claimed in claim 2, c h a r a c t e r i s e d in that In the training process of the decision network model, a positive reward exceeding a set value is given when the temperature is in the target interval and the furnace temperature difference is less than a set value and the energy consumption is lower than a set value; a negative reward exceeding a predetermined value is given when the temperature deviates from the target interval or the distance to the safety limit is less than a set threshold value.

4. A method of controlling the temperature of a furnace as claimed in claim 1 or 2, wherein The preprocessing process of the obtained data comprises: physical threshold filtering and abnormal value elimination on the obtained data, the corresponding record is deleted when a certain data exceeds the corresponding set range; if the change amplitude between adjacent sampling points exceeds the threshold value, the corresponding sampling point is regarded as abnormal and is eliminated.

5. A method of furnace temperature control as defined in claim 1, characterized in that the introduction The multi-level safety constraint process comprises: introducing action amplitude constraint, absolute opening degree boundary constraint, furnace temperature-safety logic constraint and pressure and interlocking constraint. The action amplitude constraint is used to limit the opening degree change of the gas valve and the air valve within a single control period to be less than a preset value; The absolute opening degree boundary constraint is used to set the minimum and maximum allowable opening degrees of the gas valve and the air valve according to the furnace structure and the equipment safety specification, and when the candidate opening degree adjustment amount exceeds the above range, the saturation function is used for truncation to limit it in the safe operation interval; The furnace temperature-safety logic constraint is used to prohibit any further positive adjustment of the gas supply when any side furnace temperature approaches or exceeds the process upper limit, and to trigger the emergency cooling logic to forcibly reduce the gas valve opening degree and moderately increase the air valve opening degree as needed to reduce the furnace temperature and increase the safety margin when the temperature is less than a set value from the equipment safety limit value; The pressure and interlocking constraint is used to switch the valve opening degree adjustment amount to a preset safety annealing strategy when the gas total pipe pressure exceeds the set safety interval or the gas system safety interlocking early warning signal.

6. A method of furnace temperature control as defined in claim 1, characterized in that the introduction On the basis of multi-level security constraints, the range-limited ε-greedy security exploration mechanism is introduced. In each control cycle, with probability (1 ε), the decision network model output is directly adopted and the corrected greedy action is constrained, and with probability ε, an exploratory action is selected within the security action neighborhood. The safety action neighborhood is designed according to the hearth temperature characteristics, with the current valve opening degree as the center, to limit the exploration range of the gas valve adjustment amount and the air valve adjustment amount, and the exploration range is smaller than the adjustment range allowed in the multi-level safety constraint. The exploration action is randomly sampled from the limited range according to the preset distribution, and is superimposed or replaced with the action output by the decision network model, and the multi-level safety constraint is introduced again to perform amplitude and boundary checking.

7. A method of furnace temperature control as claimed in claim 6, c h a r a c t e r i s e d in that The ε-greedy safety exploration mechanism of the limited range is only enabled when the furnace temperature is in the target interval for a continuous number of sampling periods, the furnace temperature difference is lower than the given balance threshold, and there is no over-temperature or under-temperature trend. When the temperature deviates from the target interval, the furnace temperature difference increases, or pressure abnormalities are detected, the ε is decayed to 0, and only the greedy strategy is used to perform steady-state control.

8. A method of furnace temperature control as defined in claim 1, characterized by The steps of the method are iteratively performed, and when the furnace temperature is stably maintained in the target temperature interval for a plurality of sampling periods, and the furnace temperature difference continuously meets the balance threshold requirement, the candidate opening degree adjustment amount obtained by the decision network model approaches zero or is less than a set value, the steady-state control mode is entered, in which the control strategy preferentially suppresses actions with a change amplitude greater than a threshold value, and only when the temperature trend deviates due to changes in the total gas pipe pressure, environmental temperature disturbance, or raw material properties, a new significant adjustment action is triggered.

9. A furnace temperature control system characterized by, The method comprises: a real-time data acquisition module for acquiring furnace temperature, total pipe pressure, gas valve opening degree for adjusting gas flow, and air valve opening degree for adjusting air flow; a data cleaning and feature construction module for preprocessing the acquired data and extracting features of the preprocessed data; a reinforcement learning decision module for obtaining candidate opening degree adjustment amounts of the gas valve and the air valve according to the current features using a decision network model constructed based on a reinforcement learning method; a safety strategy module for introducing multi-level safety constraints according to the hearth combustion condition to correct the candidate opening degree adjustment amounts and obtain the final opening degree adjustment amounts to control the opening degrees of the corresponding valves.

10. An electronic device, characterized by comprising: The computer instructions stored on the memory and running on the processor complete the steps in the method of any one of claims 1-8 when the processor runs the computer instructions.

Images