Reinforcement learning-based claus process control method and apparatus

Abstract

To provide a method for maintaining an appropriate level of H2S / SO2 ratio in a tail gas by controlling operating conditions of the Claus process using various information in the Claus process and a reinforcement learning model.SOLUTION: A reinforcement learning-based Claus process control method includes the steps in which: a controller accepts input of a state value of a Claus process; the controller inputs the state value of the Claus process to a reinforcement learning model; and the controller controls the Claus process based on an action value output by the reinforcement learning model.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The techniques described below relate to methods and apparatus for controlling a Claus process using reinforcement learning. [Background technology]

[0002] Coke oven gas (COG), a by-product gas generated during the coke production process, is used to generate electricity within steelworks or is injected as fuel into the process. However, COG contains impurities such as ammonia (NH3), hydrogen sulfide (H2S), and BTX (benzene, toluene, xylene), so it must be purified before use.

[0003] In order to refine COG, steel plants have a chemical conversion process to remove impurities from the COG. The chemical conversion process includes equipment such as an H2S absorption tower, an NH3 absorption tower, and a BTX absorption tower. These absorption towers remove H2S, NH3, BTX, and other substances contained in COG. After passing through the chemical conversion process, impurities have been removed from the COG, and it can be used for power generation or as fuel.

[0004] The desulfurization process in the chemical process consists of a capture process in which H2S is captured in an absorption tower using concentrated ammonia water, a separation process in which H2S is separated from the concentrated ammonia water in which it is dissolved, and a recovery process in which sulfur is produced from the separated H2S gas. Of these, the process of producing sulfur from H2S mainly consists of the Claus process.

[0005] After the Claus process, tail gas is discharged. The tail gas contains H2S / SO2, and the ratio of the two should be kept at an appropriate level. This is because excessive H2S concentration violates emission regulations, and excessive SO2 can reduce the efficiency of wastewater treatment that utilizes microorganisms. It is also necessary to keep the H2S / SO2 ratio in the tail gas stable so that it does not change suddenly over time. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Korean Patent No. 10-2209400 [Patent Document 2] Korean Patent No. 10-2484968 Summary of the Invention [Problem to be solved by the invention]

[0007] In order to keep the H2S / SO2 ratio in the tail gas of the conventional Claus process constant, the operating conditions of the catalytic oven reactor have been adjusted by PID control or adjusted according to the operator's experience. The PID control or adjustment according to the operator's experience cannot precisely keep the H2S / SO2 ratio in the tail gas at the set value, and cannot sensitively respond to changes in the process conditions, so there are problems in that the H2S / SO2 ratio may change suddenly.

[0008] The technology described below aims to provide a method for maintaining an appropriate level of H2S / SO2 ratio in tail gas by controlling the operating conditions of the Claus process using various information within the Claus process and a reinforcement learning model. [Means for solving the problem]

[0009] The techniques described below provide a reinforcement learning based Claus process control method and apparatus to solve the problems mentioned above.

[0010] The reinforcement learning based Claus process control method includes a step of receiving an input of a state value of a Claus process by a controller, a step of the controller inputting the state value of the Claus process to a reinforcement learning model, and a step of the controller controlling the Claus process based on an action value output by the reinforcement learning model.

[0011] The reinforcement learning based Claus process control device includes an input device that receives input of state values ​​of the Claus process, a calculation device that inputs the state values ​​of the Claus process to a reinforcement learning model and generates a control signal for controlling the Claus process based on an action value output by the reinforcement learning model, and a storage device that stores the reinforcement learning model.

[0012] The reinforcement learning model may be a model that outputs an action value for receiving a maximum compensation value based on an input state value of the Claus process.

[0013] The status values ​​may include at least one of information related to acid gases in the Claus process, information related to a catalytic oven reactor, information related to a waste heat boiler, information related to a Claus reactor, information related to a condenser, and information related to a tail gas.

[0014] The compensation value may be increased as the H2S / SO2 ratio in the Claus process tail gas approaches a predetermined value.

[0015] The action value may include information for controlling the Claus process. Effect of the Invention

[0016] The technique described below can be used to automatically control the Claus process. The technique described below can be used to automatically adjust the operating conditions of the catalytic oven reactor in the Claus process. This allows the H2S / SO2 ratio in the tail gas to be maintained at a set value. This can also reduce environmental regulatory issues caused by excessive H2S emissions and prevent excessive SO2 emissions from reducing the efficiency of the wastewater treatment process. It can also prevent sudden changes in the H2S / SO2 ratio in the tail gas. [Brief description of the drawings]

[0017] [Figure 1] 1 illustrates one embodiment of a Claus process 100. [Diagram 2]1 is an embodiment in which the controller 200 controls the Claus process 100 based on reinforcement learning. [Diagram 3] 3 is a flow chart 300 of an embodiment of a reinforcement learning based Claus process control method. [Figure 4] 4 is an embodiment of a reinforcement learning based Claus process control system 400 configuration. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] The technology described below can be modified in various ways and can have various embodiments. A specific embodiment of the technology described below can be described in the drawings of the specification. However, this is for the purpose of explaining the technology described below, and does not limit the technology described below to a specific embodiment. Therefore, it should be understood that all modifications, equivalents, or alternatives included in the spirit and technical scope of the technology described below are included in the technology described below.

[0019] In the terms used below, the singular expression should be understood to include the plural expressions unless the context clearly indicates otherwise, and terms such as "comprise" should be understood to mean the presence of a stated feature, number, step, operation, component, part, or combination thereof, but not to exclude the presence or additional possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0020] Before describing the drawings in detail, it should be made clear that the components in this specification are merely classified according to the main function of each component. That is, two or more components described below may be combined into one component, or one component may be divided into two or more components according to more specific functions. Furthermore, each of the components described below may perform a part or all of the functions of the other components in addition to its own main function, and it goes without saying that some of the main functions of each component may be exclusively performed by the other components.

[0021] Additionally, in performing a method or method of operation, the steps of the method may occur out of the order stated, unless the context clearly dictates a particular order, i.e., the steps may occur in the same order as stated, may occur substantially simultaneously, or may occur in the reverse order.

[0022] Consider first the Claus process in the technique described below.

[0023] FIG. 1 illustrates one embodiment of a Claus process 100.

[0024] Claus process 100 may refer to a process that removes or recovers sulfur.

[0025] The acid gases fed to the Claus process 100 contain H2S. The acid gases are combusted in a catalytic oven reactor to cause the following reactions (1) and (2): (1) 2H2S+3O2 → 2SO2+2H2O (2) 2H2S+O2 → 2S+2H2O

[0026] After the reaction in the catalytic oven reactor 110, a portion of the sulfur S is condensed via a waste heat boiler 120 to recover liquid sulfur.

[0027] The Claus reactor 130 is charged with the SO2 produced in (1) and unreacted H2S.

[0028] In the Claus reactor 130, the following chemical reaction (3) occurs: (3) 2H2S+SO2 → 3S+2H2O

[0029] Sulfur (S) generated by this reaction is condensed in the condenser 140 to become liquid sulfur.

[0030] The reaction (3) is a thermodynamic equilibrium reaction. Therefore, it is not possible to remove all of the H2S by only one Claus reactor 130 process. In order to further recover sulfur from the unreacted H2S, the Claus reactor 130 and condenser 140 processes can be performed again. That is, the material after the sulfur is partially condensed in the condenser 140 may be charged again into the Claus reactor 130.

[0031] The tail gas from the Claus process 100 containing unreacted H2S and SO2 can be reintroduced into the conversion process.

[0032] Each component of the Claus process may include sensors and the like for measuring data.

[0033] The overall process of the reinforcement learning based Claus process control method performed by the controller will now be described.

[0034] FIG. 2 is an embodiment in which a controller 200 controls a Claus process 100 based on reinforcement learning.

[0035] The control device 200 can receive an input of a state value of the Claus process 100. The control device 200 can input the state value of the Claus process 100 to a reinforcement learning model. The control device 200 can control the Claus process 100 based on an action value output by the reinforcement learning model.

[0036] The reinforcement learning model may be a model that outputs an action value for receiving a maximum compensation value based on the input state values ​​of the Claus process 100. The compensation value may be larger as the ratio of H2S / SO2 in the tail gas of the Claus process 100 approaches a predetermined value. The action value may include information for controlling the Claus process.

[0037] The process by which the control device performs the reinforcement learning based Claus process control method will now be described in detail.

[0038] FIG. 3 is a flow chart 300 of one embodiment of a reinforcement learning based Claus process control method.

[0039] The controller may receive inputs of state values ​​of the Claus process (310).

[0040] The state value in the Claus process can include various information that can be grasped in the Claus process. The reinforcement learning model can receive the state value in the Claus process as an input and output the action value.

[0041] The status values ​​within the Claus process can include at least one of acid gas related information, catalytic oven reactor 110 related information, waste heat boiler 120 related information, Claus reactor 130 related information, condenser 140 related information, and tail gas related information.

[0042] In one embodiment, the information related to the acid gas may include at least one of the pressure, flow rate, temperature, and H2S concentration of the acid gas.

[0043] In one embodiment, the information related to the catalytic oven reactor 110 may include at least one of the primary air injection rate, secondary air injection rate, COG flow rate, top temperature, middle temperature, and bottom temperature of the catalytic oven reactor.

[0044] In one embodiment, the information related to the waste heat boiler 120 may include at least one of an outlet temperature of the waste heat boiler 120, a feedwater flow rate, and a steam flow rate.

[0045] In one embodiment, the information related to the Claus reactor 130 may include at least one of an inlet temperature and an outlet temperature of the Claus reactor.

[0046] In one embodiment, the information related to the condenser 140 may include at least one of the pressure of the condenser 140 and the feedwater flow rate.

[0047] In one embodiment, the information related to the tail gas may include the pressure of the tail gas, the H2S concentration in the tail gas, and the SO2 concentration.

[0048] The controller can input the state values ​​of the Claus process into the reinforcement learning model (320).

[0049] The reinforcement learning model may be a model that outputs an action value for receiving a maximum compensation value based on the state value of the input Claus process.

[0050] The reinforcement learning model may be a model that is reinforced by using a data set constructed by combining accumulated process data for a Claus process that has been operated for a certain period of time and compensation information for control actions taken during the data accumulation period.

[0051] The reinforcement learning model may be a model trained by offline reinforcement learning, i.e., without interacting with the Claus process in real time, but utilizing a data set accumulated as a result of a conventional Claus process, and thus without trial and error. Also, the reinforcement learning model may train without adversely affecting the productivity and safety of the operating Claus process.

[0052] After the reinforcement learning model has completed learning, it can interact with the Claus process in real time and be an optimized fine-tuned model. Specifically, it can receive a state value for the current Claus process, calculate an action value to control the Claus process, calculate a compensation value accordingly, and update the parameters of the model based on the calculated compensation value.

[0053] The compensation value may be increased as the H2S / SO2 ratio in the tail gas of the Claus process approaches a predetermined value. Specifically, the compensation value may be set to or designed based on the negative value of the mean square error between the H2S / SO2 ratio after the Claus process control by the action value and the predetermined H2S / SO2 ratio.

[0054] The action value may include information for controlling the Claus process.

[0055] In one embodiment, the action value may include information for controlling at least one of the primary air injection rate, the secondary air injection rate, and the COG flow rate into the catalytic oven reactor.

[0056] The controller can control the Claus process based on the action value information output by the reinforcement learning model (330).

[0057] In one embodiment, the controller can control the catalytic oven reactor 110 based on an action value. For example, if the action value is for the amount of primary air injected into the catalytic oven reactor, the amount of primary air injected into the catalytic oven reactor can be adjusted to match the action value.

[0058] The control device will now be described.

[0059] FIG. 4 shows an example of the configuration of the control device 400.

[0060] The control device 400 may correspond to the control device 200 described in Fig. 2. In other words, the control device 400 may be a device that executes the above-mentioned reinforcement learning-based Claus process control method.

[0061] The control device 400 can be physically realized in various forms. For example, the control device 400 can have the form of a PC, a notebook, a smart device, a server, or a chipset dedicated to data processing.

[0062] The control device 400 may include an input device 410 , a memory device 420 , a processing device 430 , an output device 440 , an interface device 450 , and a communication device 460 .

[0063] The input device 410 may include an interface device (such as a keyboard, mouse, or touch screen) that receives input of certain commands or data. The input device 410 may also include a configuration that receives input of information via a separate storage device (such as a USB, CD, or hard disk). The input device 410 may receive input of data to be input via a separate measurement device or a separate DB. The input device 410 may receive input of data via wired or wireless communication.

[0064] The input device 410 can receive input of information and models required to execute the reinforcement learning based Claus process control method. The input device 410 can receive input of state values ​​of the Claus process. The input device 410 can receive input of a reinforcement learning model.

[0065] The storage device 420 can store information input via the input device 410. The storage device 420 can store information generated during the process of calculation by the arithmetic device 430. That is, the storage device 420 can include a memory. The storage device 420 can store the results of calculations by the arithmetic device 430.

[0066] The storage device 420 can store information and models required to perform the reinforcement learning based Claus process control method. The storage device 420 can store state values ​​of the Claus process. The storage device 420 can store reinforcement learning models.

[0067] The computing device 430 may be a device such as a processor, AP, or a chip with a built-in program that processes data and performs certain operations. The computing device 430 may generate control signals that control the control device.

[0068] The computing device 430 can perform the computations necessary to implement the reinforcement learning based Claus process control method. The computing device 430 can input state values ​​of the Claus process 100 to the reinforcement learning model. The computing device 430 can control the Claus process 100 based on action values ​​output by the reinforcement learning model.

[0069] The output device 440 may be a device that outputs certain information. The output device 440 may output an interface required for data processing, input data, analysis results, etc. The output device 440 may be realized in various physical forms, such as a display or a device that outputs documents. The output device 440 may output the results of calculations performed by the calculation device 430.

[0070] The interface device 450 may be a device that receives certain commands and data input from the outside. The interface device 450 can receive input of state values ​​of the Claus process 100 from a physically connected input device or an external storage device. The interface device 450 can receive input of control signals for controlling the control device 400. The interface device 450 can output the results analyzed by the control device 400.

[0071] The communication device 460 may refer to a configuration that receives and transmits certain information via a wired or wireless network. The communication device 460 may receive control signals necessary to control the control device 400. The communication device 460 may transmit the results of analysis by the control device 400.

[0072] The reinforcement learning based Claus process control method described above may be implemented in a program (or application) that includes an executable algorithm that can be run on a computer.

[0073] The program may be provided by being stored in a transitory or non-transitory computer-readable medium.

[0074] A non-transitory computer-readable medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short moment, such as a register, cache, memory, etc. Specifically, the above-mentioned various applications or programs can be provided by being stored in a non-transitory computer-readable medium, such as a CD, a DVD, a hard disk, a Blu-ray disk, a USB, a memory card, a ROM (read-only memory), a PROM (programmable read only memory), an EPROM (Erasable PROM, EPROM) or an EEPROM (Electrically EPROM), a flash memory, etc.

[0075] The temporary computer readable medium refers to various types of RAM, such as SRAM (Static RAM), DRAM (Dynamic RAM), SDRAM (Synchronous DRAM), DDR SDRAM (Double Data Rate SDRAM), ESDRAM (Enhanced SDRAM), SLDRAM (Synclink DRAM), and DRRAM (Direct Rambus RAM).

[0076] It should be noted that the present embodiment and the drawings attached to this specification merely clearly show a portion of the technical ideas contained in the above-mentioned technology, and that any modified examples and specific embodiments that a person skilled in the art can easily deduce within the scope of the technical ideas contained in the specification and drawings of the above-mentioned technology are all obviously included in the scope of the rights of the above-mentioned technology.

Claims

1. a controller receiving inputs of state values ​​of the Claus process; The controller inputs state values ​​of the Claus process into a reinforcement learning model; The control device controls a Claus process based on an action value output by the reinforcement learning model, The reinforcement learning model is a model that outputs an action value for receiving a maximum compensation value based on a state value of an input Claus process, the state values ​​include at least one of information related to acid gases in the Claus process, information related to a catalytic oven reactor, information related to a waste heat boiler, information related to a Claus reactor, information related to a condenser, and information related to a tail gas; The compensation value is H in the tail gas of the Claus process. ２ S / SO ２ The closer the ratio of is to a given value, the larger it becomes. A method for controlling a Claus process based on reinforcement learning, wherein the action value includes information for controlling the Claus process.

2. The information related to the acid gas includes pressure, flow rate, temperature, and H ２ The reinforcement learning based Claus process control method of claim 1 , further comprising at least one of: S concentration.

3. 2. The reinforcement learning based Claus process control method of claim 1, wherein the information related to the catalytic oven reactor includes at least one of a primary air injection rate, a secondary air injection rate, a COG flow rate, a top temperature, a middle temperature, and a bottom temperature of the catalytic oven reactor.

4. The method of claim 1 , wherein the information related to the waste heat boiler includes at least one of a waste heat boiler outlet temperature, a feed water flow rate, and a steam flow rate.

5. 2. The method of claim 1, wherein the information related to the Claus reactor comprises at least one of an inlet temperature and an outlet temperature of the Claus reactor.

6. The method of claim 1 , wherein the condenser-related information includes at least one of a condenser pressure and a feedwater flow rate.

7. The information related to the tail gas includes the pressure of the tail gas, H in the tail gas, ２ S concentration, SO ２ The reinforcement learning based Claus process control method of claim 1 including a concentration.

8. 2. The reinforcement learning based Claus process control method of claim 1, wherein the action values ​​include information for controlling at least one of a primary air injection rate, a secondary air injection rate, and a COG flow rate into a catalytic oven reactor.

9. an input device for receiving an input of a state value of the Claus process; A calculation device that inputs state values ​​of the Claus process to a reinforcement learning model and generates a control signal for controlling the Claus process based on an action value output from the reinforcement learning model; A storage device that stores the reinforcement learning model, The reinforcement learning model is a model that outputs an action value for receiving a maximum compensation value based on a state value of an input Claus process, the state values ​​include at least one of information related to acid gases in the Claus process, information related to a catalytic oven reactor, information related to a waste heat boiler, information related to a Claus reactor, information related to a condenser, and information related to a tail gas; The compensation value is H in the tail gas of the Claus process. ２ S / SO ２ The closer the ratio of is to a given value, the larger it becomes. The action value includes information for controlling the Claus process.

10. The information related to the acid gas includes pressure, flow rate, temperature, and H ２ The reinforcement learning based Claus process control system of claim 9, further comprising at least one of: S concentration.

11. 10. The reinforcement learning based Claus process control device of claim 9, wherein the information related to the catalytic oven reactor includes at least one of a primary air injection rate, a secondary air injection rate, a COG flow rate, a top temperature, a middle temperature, and a bottom temperature of the catalytic oven reactor.

12. The reinforcement learning based Claus process controller of claim 9 , wherein the information related to the waste heat boiler includes at least one of a waste heat boiler outlet temperature, a feed water flow rate, and a steam flow rate.

13. The reinforcement learning based Claus process controller of claim 9 , wherein the information related to the Claus reactor includes at least one of an inlet temperature and an outlet temperature of the Claus reactor.

14. The reinforcement learning based Claus process control system of claim 9 , wherein the condenser related information includes at least one of a condenser pressure and a feedwater flow rate.

15. The information related to the tail gas includes the pressure of the tail gas, H in the tail gas, ２ S concentration, SO ２ The reinforcement learning based Claus process control system of claim 9 including a concentration.

16. 10. The reinforcement learning based Claus process controller of claim 9, wherein the action values ​​include information for controlling at least one of a primary air injection rate, a secondary air injection rate, and a COG flow rate into a catalytic oven reactor.

17. 2. A computer-readable recording medium having a program recorded thereon for executing the Claus process control method according to claim 1.

Images