Claus process control method and device based on reinforcement learning

Abstract

The Claus process control method based on reinforcement learning comprises the following steps: a control device receives input of a state value of a Claus process; the control device inputs the state value of the Claus process into the reinforcement learning model; and the control device controls the Claus process on the basis of the operation value output by the reinforcement learning model.

Description

Technical Field

[0001] The technology described below discloses a method and apparatus for controlling a Claus process using reinforcement learning. Background Art

[0002] Coke oven gas (COG), a byproduct gas produced during the production of coking coal, is used for power generation or as process fuel injection in steel plants. However, COG may contain impurities such as ammonia (NH3), hydrogen sulfide (H2S), and BTX (benzene, toluene, xylene) and needs to be refined before use.

[0003] In order to refine COG, steel mills have a formation process to remove impurities in COG. The formation process includes equipment such as H2S absorption tower, NH3 absorption tower, BTX absorption tower, etc. The absorption tower removes H2S, NH3, BTX, etc. contained in COG. COG that has gone through the formation process can be used for power generation or as fuel because impurities are removed.

[0004] The desulfurization process in the chemical formation process includes the following processes: a capture process of H2S using an absorption tower using concentrated ammonia water, a separation process of H2S from the concentrated ammonia water in which H2S is dissolved, and a recovery process of generating sulfur from the separated H2S gas. Among them, the process of generating sulfur from H2S is mainly composed of the Claus process.

[0005] After the Claus process, tail gas is discharged. The tail gas contains H2S / SO2, and the ratio of the two needs to be kept at an appropriate level. This is because if the H2S concentration is too high, it will violate the emission regulations, and if SO2 is too much, it will reduce the efficiency of wastewater treatment using microorganisms. In addition, it is important to maintain the H2S / SO2 ratio in the tail gas stably to avoid it changing drastically over time.

[0006] Prior art literature

[0007] Patent Literature

[0008] Korean Patent Gazette 10-2209400

[0009] Korean Patent Gazette 10-2484968 Summary of the invention

[0010] In order to maintain the H2S / SO2 ratio in the tail gas of the existing Claus process, the operating conditions of the catalytic oven reactor are adjusted by PID control or by relying on the operator's experience. The adjustment method based on PID control or operator experience cannot accurately maintain the H2S / SO2 ratio in the tail gas at the set value and cannot respond to changes in process conditions flexibly, which may cause the H2S / SO2 ratio to change sharply.

[0011] The technology described below discloses a method for controlling the operating conditions of the Claus process by utilizing various information in the Claus process and a reinforcement learning model to maintain an appropriate level of H2S / SO2 ratio in the tail gas.

[0012] The technology described below provides a method and apparatus for controlling a Claus process based on reinforcement learning in order to solve the above-mentioned problems.

[0013] The Claus process control method based on reinforcement learning comprises the following steps: a control device receives an input of a state value of the Claus process; the control device inputs the state value of the Claus process into a reinforcement learning model; and the control device controls the Claus process based on an action value output by the reinforcement learning model.

[0014] A Claus process control device based on reinforcement learning includes: an input device for receiving an input of a Claus process state value; a computing device for inputting the state value of the Claus process into a reinforcement learning model, and generating a control signal for controlling the Claus process based on an action value output by the reinforcement learning model; and a storage device for storing the reinforcement learning model.

[0015] The reinforcement learning model may be a model that outputs an action value for obtaining a maximum compensation value based on an input state value of the Claus process.

[0016] The status value may include at least one of information related to acid gas in the Claus process, information related to a catalytic oven reactor, information related to a waste heat boiler, information related to a Claus reactor, information related to a condenser, and information related to tail gas.

[0017] The closer the H 2 S / SO 2 ratio in the tail gas of the Claus process is to the preset value, the larger the compensation value can be.

[0018] The action value may include information for controlling the Claus process.

[0019] The Claus process can be automatically controlled using the technology described below. The operating conditions of the catalytic oven reactor in the Claus process can be automatically adjusted using the technology described below. By doing so, the H2S / SO2 ratio in the tail gas can be maintained at a set value. In addition, environmental regulatory issues caused by excessive H2S emissions can be alleviated, and the problem of reduced efficiency of the wastewater treatment process caused by excessive SO2 emissions can be prevented. In addition, rapid changes in the H2S / SO2 ratio in the tail gas can be prevented. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 One embodiment of a Claus process 100 is shown.

[0021] Figure 2 This is one of the embodiments in which the control device 200 controls the Claus process 100 based on reinforcement learning.

[0022] Figure 3 A sequence diagram 300 of one embodiment of an embodiment of a Claus process control method based on reinforcement learning.

[0023] Figure 4 It is one of the embodiments of the configuration of the Claus process control device 400 based on reinforcement learning. DETAILED DESCRIPTION

[0024] The technology described below can be modified in various ways and can have various embodiments. The drawings in the specification may record specific implementation forms of the technology described below. However, they are only used to illustrate the technology described below, and are not intended to limit the technology described below to a specific implementation form. Therefore, all modifications, equivalents, and even substitutes included in the spirit and technical scope of the technology described below should be interpreted as belonging to the technology described below.

[0025] In the terms used below, unless it is obvious that a different interpretation can be given in the context of the sentence, the singular expression also includes the plural form. Terms such as "including" only specify the existence of the recorded characteristics, quantities, steps, actions, elements, parts or their combinations, and shall not be deemed to exclude in advance the existence of one or more other characteristics, quantities, steps, actions, elements, parts or their combinations or the existence of additional possibilities.

[0026] Before describing the drawings in detail, it should be pointed out that the distinction between the components in this specification is only made according to the main functions of each component. That is, the two or more components described below can be combined into one component, or one component can be divided into two or more components according to the subdivided functions. In addition to performing the main functions that each component is responsible for, the components described below can also further perform part or all of the functions that other components are responsible for. Of course, part of the main functions that each component is responsible for can also be performed by other components.

[0027] Furthermore, when performing a method or method of action, unless a particular order is expressly stated in the context of the sentence, the processes constituting the method can occur in a different order than that expressly stated. That is, the processes can occur in the same order as that expressly stated, can occur substantially simultaneously, or can occur in a reverse order.

[0028] First, look at the Claus process in the technology explained below.

[0029] Figure 1 One embodiment of a Claus process 100 is shown.

[0030] Claus process 100 may refer to a sulfur removal or sulfur recovery process.

[0031] The acid gas fed into the Claus process 100 includes H2S. The acid gas is burned in the catalytic oven reactor to cause the reactions shown in the following equations (1) and (2).

[0032] (1) 2H2S+3O2→2SO2+2H2O

[0033] (2) 2H2S+O2→2S+2H2O

[0034] After the reaction in the catalytic oven reactor 110 , a portion of the sulfur S is condensed through the waste heat boiler 120 to recover liquid sulfur.

[0035] The Claus reactor 130 is charged with SO2 generated by equation (1) and unreacted H2S.

[0036] In the Claus reactor 130 , a chemical reaction as shown in the following formula (3) occurs.

[0037] (3) 2H2S+SO2→3S+2H2O

[0038] The sulfur S generated by the reaction is condensed into liquid sulfur through the condenser 140 .

[0039] The reaction (3) is a thermodynamic equilibrium reaction. Therefore, it is not possible to remove all H2S by only one process of the Claus reactor 130. In order to further recover sulfur from such unreacted H2S, the Claus reactor 130 and the condenser 140 process can be repeated. That is, the material after a part of sulfur condensation in the condenser 140 can also be recharged into the Claus reactor 130.

[0040] The tail gas (Tail Gas) containing unreacted H2S and SO2 in the Claus process 100 can be re-introduced into the chemical process.

[0041] Elements of the Claus process can include sensors for measuring data.

[0042] The following describes the overall process of the control device executing the Claus process control method based on reinforcement learning.

[0043] Figure 2 This is one of the embodiments in which the control device 200 controls the Claus process 100 based on reinforcement learning.

[0044] The control device 200 can receive an input of a state value of the Claus process 100. The control device 200 can input the state value of the Claus process 100 into the reinforcement learning model. The control device 200 can control the Claus process 100 based on the action value output by the reinforcement learning model.

[0045] The reinforcement learning model may be a model that outputs an action value for obtaining a maximum compensation value based on an input state value of the Claus process 100. The closer the H2S / SO2 ratio in the tail gas of the Claus process 100 is to a preset value, the larger the compensation value can be. The action value may include information for controlling the Claus process.

[0046] The following specifically describes the process in which the control device executes the Claus process control method based on reinforcement learning.

[0047] Figure 3 A sequence diagram 300 of one embodiment of an embodiment of a Claus process control method based on reinforcement learning.

[0048] The control device can receive input of a state value of the Claus process (step 310).

[0049] The state value in the Claus process may include various information that can be grasped in the Claus process. The reinforcement learning model can output an action value after receiving the state value in the Claus process as input.

[0050] The state value in the Claus process may include at least one of acid gas related information, catalytic oven reactor 110 related information, waste heat boiler 120 related information, Claus reactor 130 related information, condenser 140 related information and tail gas related information.

[0051] As an example, the acid gas related information may include at least one of the pressure, flow rate, temperature and H2S concentration of the acid gas.

[0052] As an example, the information related to the catalytic oven reactor 110 may include at least one of the first stage air injection amount, the second stage air injection amount, the COG flow rate, the upper temperature, the middle temperature and the lower temperature of the catalytic oven reactor.

[0053] As an example, the information related to the waste heat boiler 120 may include at least one of the outlet temperature, water supply flow rate, and steam flow rate of the waste heat boiler 120 .

[0054] As an example, the information related to the Claus reactor 130 may include at least one of an inlet temperature and an outlet temperature of the Claus reactor.

[0055] As an example, the information related to the condenser 140 may include at least one of the pressure of the condenser 140 and the water supply flow rate.

[0056] As an example, the exhaust gas related information may include the exhaust gas pressure, H2S concentration, and SO2 concentration in the exhaust gas.

[0057] The control device can input the state value of the Claus process into the reinforcement learning model (step 320).

[0058] The reinforcement learning model may be a model that outputs an action value for obtaining a maximum compensation value based on an input state value of the Claus process.

[0059] The reinforcement learning model may be a model that has been reinforced learned using a data set that is a data set created by integrating process data accumulated from a Claus process that has been running for a certain period of time and compensation information on control actions performed during the data accumulation period.

[0060] The reinforcement learning model can be a model that has been learned by offline reinforcement learning. That is, instead of interacting with the Claus process in real time, reinforcement learning can be performed using a data set accumulated through existing Claus process results. Therefore, there is no trial and error. Moreover, the reinforcement learning model can learn without adversely affecting the production efficiency and safety of the operating Claus process.

[0061] The reinforcement learning model can be a fine-tuning model that is optimized by real-time interaction with the Claus process after learning is completed. Specifically, after receiving the current state value of the Claus process, the action value is calculated and the Claus process is controlled, and the compensation value that comes with it is calculated, and the parameters of the model can be updated based on the calculated compensation value.

[0062] The closer the H2S / SO2 ratio in the tail gas of the Claus process is to the preset value, the larger the compensation value can be. Specifically, the compensation value can be set to the negative value of the mean square error (MSE) between the H2S / SO2 ratio after the Claus process is controlled according to the action value and the preset H2S / SO2 ratio, or a value designed based on this.

[0063] The action value may include information used to control the Claus process.

[0064] As an example, the action value may include information for controlling at least one of a first stage air injection amount, a second stage air injection amount, and a COG flow rate into the catalytic oven reactor.

[0065] The control device can control the Claus process based on the action value information output by the reinforcement learning model (step 330).

[0066] As an example, the control device can control the catalytic oven reactor 110 based on the action value. For example, if the action value is about the amount of air injected into the first stage of the catalytic oven reactor, the amount of air injected into the first stage of the catalytic oven reactor can be injected in accordance with the action value.

[0067] Next, the control device will be described.

[0068] Figure 4 This is a configuration example of the control device 400.

[0069] The control device 400 may be equivalent to Figure 2 The control device 200 described above. That is, the control device 400 may be a device for executing the aforementioned Claus process control method based on reinforcement learning.

[0070] The control device 400 can be physically implemented in various forms, for example, the control device 400 can be in the form of a PC, a notebook computer, a smart device, a server, or a data processing dedicated chipset.

[0071] The control device 400 may include an input device 410 , a storage device 420 , a computing device 430 , an output device 440 , an interface device 450 , and a communication device 460 .

[0072] The input device 410 can also include an interface device (keyboard, mouse, touch screen, etc.) for receiving certain instructions or data. The input device 410 can also include a structure for receiving information through an additional storage device (USB, CD, hard disk, etc.). The input device 410 can receive the input data through an additional measuring device or through an additional DB. The input device 410 can receive data through wired or wireless communication.

[0073] The input device 410 can receive the information and model input required when executing the Claus process control method based on reinforcement learning. The input device 410 can receive the input of the Claus process state value. The input device 410 can receive the input of the reinforcement learning model.

[0074] The storage device 420 can store information inputted through the input device 410. The storage device 420 can store information generated during the operation of the operation device 430. That is, the storage device 420 can include a memory. The storage device 420 can store the result of the operation of the operation device 430.

[0075] The storage device 420 can store information and models required for executing the Claus process control method based on reinforcement learning. The storage device 420 can store Claus process state values. The storage device 420 can store reinforcement learning models.

[0076] The computing device 430 may be a device such as a processor, an AP, or a chip with a program embedded therein, which processes data and performs certain operations. The computing device 430 may generate a control signal for controlling the control device.

[0077] The computing device 430 can perform a computation required for executing the Claus process control method based on reinforcement learning. The computing device 430 can input the state value of the Claus process 100 into the reinforcement learning model. The computing device 430 can control the Claus process 100 based on the action value output by the reinforcement learning model.

[0078] The output device 440 may be a device that outputs certain information. The output device 440 may also output interfaces, input data, analysis results, etc. required in the data processing process. The output device 440 may be implemented in a variety of physical forms such as a display or a file output device. The output device 440 may output the calculation results of the calculation device 430.

[0079] The interface device 450 may be a device that receives certain instructions and data from the outside. The interface device 450 may receive the state value of the Claus process 100 from a physically connected input device or an external storage device. The interface device 450 may receive a control signal for controlling the control device 400. The interface device 450 may output the result analyzed by the control device 400.

[0080] The communication device 460 may mean an element that receives certain information and transmits it through a wired or wireless network. The communication device 460 can receive a control signal required for controlling the control device 400. The communication device 460 can transmit the results analyzed by the control device 400.

[0081] The aforementioned Claus process control method based on reinforcement learning can be implemented in a program (or application program), and the program (or application program) includes an executable algorithm that can be executed on a computer.

[0082] The program can be provided after being stored in a transient or non-transitory computer readable medium.

[0083] A non-transitory computer-readable medium is not a medium that temporarily stores data, such as a register, a cache memory, or a memory, but a medium that semi-permanently stores data and can be read by a device. Specifically, the various applications or programs can be stored in a non-transitory computer-readable medium such as a CD, a DVD, a hard disk, a Blu-ray disc, a USB, a memory card, a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory and then provided.

[0084] Transitory computer readable media means various random access memories (RAM) such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct Rambus RAM (DRRAM).

[0085] This embodiment and the figures attached to this specification are only used to clearly show a part of the technical spirit contained in the aforementioned technology. It is natural that the modified examples and specific embodiments that can be easily deduced by persons skilled in the technical field to which the present invention belongs without departing from the scope of the technical spirit contained in the specification and drawings of the aforementioned technology should be interpreted as belonging to the scope of rights of the aforementioned technology.

Claims

1. A Claus process control method based on reinforcement learning, characterized in that: The following steps are involved: The control device receives an input of a state value of the Claus process; The control device inputs the state value of the Claus process into the reinforcement learning model; and, The control device controls the Claus process based on the action value output by the reinforcement learning model; The reinforcement learning model is a model that outputs an action value for obtaining a maximum compensation value based on the input state value of the Claus process. The state value includes at least one of the following: acid gas related information in the Claus process, catalytic oven reactor related information, waste heat boiler related information, Claus reactor related information, condenser related information, and tail gas related information. The closer the H2S / SO2 ratio in the tail gas of the Claus process is to the preset value, the greater the compensation value is. The action value includes information for controlling the Claus process.

2. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The acid gas related information includes at least one of the pressure, flow rate, temperature and H2S concentration of the acid gas.

3. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The catalytic oven reactor related information includes at least one of the first stage air injection amount, the second stage air injection amount, the COG flow rate, the upper temperature, the middle temperature and the lower temperature of the catalytic oven reactor.

4. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The waste heat boiler related information includes at least one of an outlet temperature, a water supply flow rate, and a steam flow rate of the waste heat boiler.

5. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The Claus reactor related information includes at least one of an inlet temperature and an outlet temperature of the Claus reactor.

6. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The condenser related information includes at least one of the pressure of the condenser and the water supply flow rate.

7. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The exhaust gas related information includes the exhaust gas pressure, H2S concentration and SO2 concentration in the exhaust gas.

8. The Claus process control method based on reinforcement learning according to claim 1, characterized in that: The action value includes information for controlling at least one of a first stage air injection amount, a second stage air injection amount, and a COG flow rate charged into the catalytic oven reactor.

9. A Claus process control device based on reinforcement learning, characterized in that: include: An input device for receiving an input of a Claus process state value; a computing device that inputs the state value of the Claus process into a reinforcement learning model, and generates a control signal for controlling the Claus process based on the action value output by the reinforcement learning model; and A storage device for storing the reinforcement learning model; The reinforcement learning model is a model that outputs an action value for obtaining a maximum compensation value based on the input state value of the Claus process. The state value includes at least one of the following: acid gas related information in the Claus process, catalytic oven reactor related information, waste heat boiler related information, Claus reactor related information, condenser related information, and tail gas related information. The closer the H2S / SO2 ratio in the tail gas of the Claus process is to the preset value, the greater the compensation value is. The action value includes information for controlling the Claus process.

10. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The acid gas related information includes at least one of the pressure, flow rate, temperature and H2S concentration of the acid gas.

11. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The catalytic oven reactor related information includes at least one of the first stage air injection amount, the second stage air injection amount, the COG flow rate, the upper temperature, the middle temperature and the lower temperature of the catalytic oven reactor.

12. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The waste heat boiler related information includes at least one of an outlet temperature, a water supply flow rate, and a steam flow rate of the waste heat boiler.

13. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The Claus reactor related information includes at least one of an inlet temperature and an outlet temperature of the Claus reactor.

14. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The condenser related information includes at least one of the pressure of the condenser and the water supply flow rate.

15. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The exhaust gas related information includes the exhaust gas pressure, H2S concentration and SO2 concentration in the exhaust gas.

16. The Claus process control device based on reinforcement learning according to claim 9, characterized in that: The action value includes information for controlling at least one of a first stage air injection amount, a second stage air injection amount, and a COG flow rate charged into the catalytic oven reactor.

17. A computer-readable storage medium, characterized in that: The method records the following program, which is used to execute the Claus process control method described in claim 1.

Images