Hydrogen sensor optimization layout method and system based on reinforcement learning

Abstract

The invention provides a hydrogen sensor optimization layout method and system based on reinforcement learning, and relates to the technical field of sensor layout. Comprising the following steps: simulating a hydrogen concentration field of various hydrogen leakage working conditions in a hydrogen energy application scene; constructing a feasible region of the hydrogen sensor layout scheme; constructing a reinforcement learning agent based on the feasible region of the hydrogen sensor layout scheme; according to the method, hydrogen energy application scenes of various hydrogen leakage working conditions are regarded as environments, maximization of hydrogen leakage working condition coverage rate, minimization of hydrogen leakage detection time and minimization of hydrogen leakage disaster-causing consequences are taken as targets, and an optimal hydrogen sensor layout scheme is obtained through continuous interaction of an intelligent agent and the environments. The optimal hydrogen sensor laying scheme is obtained according to scientific calculation, the purposes of maximizing the hydrogen leakage working condition coverage rate, minimizing the hydrogen leakage detection time and minimizing the hydrogen leakage disaster-causing consequence can be achieved, and the safety and the intelligent level in a typical hydrogen energy scene are improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of sensor optimization, and in particular relates to a hydrogen sensor optimization layout method and system based on reinforcement learning. Background Art

[0002] Hydrogen energy is regarded as an important component of the future energy system and has profound significance for coping with the depletion of fossil fuels and the global warming crisis. Many countries around the world have incorporated hydrogen energy into their energy development policies. Hydrogen energy is gradually becoming one of the most important carriers of global energy transformation and development, helping to achieve the transition to a net zero carbon circular economy. However, the safety issues facing hydrogen energy are becoming increasingly prominent, which seriously hinders its further widespread application.

[0003] Hydrogen often exists in high pressure in each link of production, storage, transportation and application, and hydrogen leakage occurs from time to time. On the one hand, once a hydrogen leak occurs, it often causes unplanned shutdowns, which in turn causes economic losses. The difference between the industry benchmark and the industry average may lie in the loss of availability caused by unplanned shutdowns. On the other hand, due to the unique physical and chemical properties of hydrogen itself, the leaking flammable hydrogen cloud will evolve into a combustion or even explosion accident when encountering an ignition source, resulting in huge economic losses and casualties. Therefore, hydrogen leakage in typical hydrogen energy application scenarios has become a common problem faced by the industry's safe production.

[0004] Advances in sensor technology, including electrochemical, optical, and acoustic sensors, can quickly identify changes in hydrogen concentration in the environment. At present, hydrogen leak monitoring systems built on hydrogen sensor technology have been widely used, effectively reducing the safety risks brought by hydrogen leaks to a certain extent. However, even if the cost of hydrogen sensors decreases, only a limited number of hydrogen sensors can be used to monitor the hydrogen leakage concentration in a specific area. The type, physical location, and working conditions of each hydrogen sensor will have a great impact on the ability of the sensor network to detect changes. In the existing hydrogen leak monitoring system, the hydrogen sensor laying plan can only rely on expert experience and install sensors on fixed key hydrogen-related equipment to ensure hydrogen-related safety. This sensor layout method often cannot guarantee coverage of all leakage conditions, nor can it guarantee the detection of hydrogen leaks in the shortest possible time.

[0005] Currently, few patents have proposed suitable methods for optimizing the layout of hydrogen sensors. By searching existing patents, the Chinese authorized invention patent with announcement number CN114036734B discloses a method and system for optimizing the layout of vehicle hydrogen sensors based on digital twins. It is only applicable to the layout of vehicle-mounted hydrogen sensors, and has not yet considered most hydrogen application scenarios, such as hydrogen refueling stations, hydrogen production plants, etc., and the patent does not provide a clear optimization method. The Chinese authorized invention patent with announcement number CN113221303B discloses a method for optimizing the layout of hydrogen sensors in semi-enclosed spaces, providing a more general method for optimizing the layout of hydrogen sensors, but has not yet considered the impact of the consequences of hydrogen leakage on the layout plan, nor has it clearly set optimization goals such as coverage or hydrogen leakage detection time. The optimization method is implemented through simple logical judgment and cannot provide the optimal hydrogen sensor layout plan.

[0006] It can be seen from the above-mentioned related patents that there are currently few studies on methods or technologies for optimizing the layout of hydrogen sensors and they are not mature enough, and breakthroughs are urgently needed to ensure the safety of the entire process of hydrogen "production-storage-transportation-use". Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a hydrogen sensor optimization layout method and system based on reinforcement learning. The provided optimization layout method does not need to rely on expert experience, but obtains the optimal hydrogen sensor laying plan based on scientific calculations, and can achieve the goals of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time and minimizing the consequences of hydrogen leakage disasters, thereby improving the safety and intelligence level in typical hydrogen energy scenarios.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] A first aspect of the present invention provides a hydrogen sensor layout optimization method based on reinforcement learning.

[0010] A method for optimizing the layout of hydrogen sensors based on reinforcement learning comprises the following steps:

[0011] Simulate the hydrogen concentration field of various hydrogen leakage conditions in hydrogen energy application scenarios;

[0012] Construct the feasible domain of hydrogen sensor layout scheme;

[0013] Based on the feasible domain of hydrogen sensor layout schemes, a reinforcement learning agent is constructed. The hydrogen energy application scenarios of various hydrogen leakage conditions are regarded as the environment, with the goal of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time and minimizing the consequences of hydrogen leakage disasters. Through the continuous interaction between the agent and the environment, the optimal hydrogen sensor layout scheme is obtained.

[0014] A second aspect of the present invention provides a hydrogen sensor optimization layout system based on reinforcement learning.

[0015] A hydrogen sensor optimization layout system based on reinforcement learning, comprising:

[0016] The hydrogen leakage simulation module is configured to: simulate the hydrogen concentration field of various hydrogen leakage conditions under hydrogen energy application scenarios;

[0017] The feasible domain construction module is configured to: construct a feasible domain of the hydrogen sensor layout solution;

[0018] The reinforcement learning module is configured as follows: based on the feasible domain of the hydrogen sensor layout plan, a reinforcement learning agent is constructed; hydrogen energy application scenarios with various hydrogen leakage conditions are regarded as the environment, with the goal of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time and minimizing the consequences of hydrogen leakage disasters, and obtaining the optimal hydrogen sensor layout plan through continuous interaction between the agent and the environment.

[0019] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the method for optimizing layout of hydrogen sensors based on reinforcement learning as described in the first aspect of the present invention.

[0020] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for optimizing the layout of a hydrogen sensor based on reinforcement learning as described in the first aspect of the present invention are implemented.

[0021] One or more of the above technical solutions have the following beneficial effects:

[0022] The present invention provides a method and system for optimizing the layout of hydrogen sensors based on reinforcement learning. First, a three-dimensional model of a typical hydrogen energy application scenario is constructed based on computational fluid dynamics (CFD) technology. For different hydrogen leakage conditions, professional CFD software is used to simulate the hydrogen leakage diffusion path. Then, an optimization algorithm based on reinforcement learning is constructed, with the goal of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time, and minimizing the consequences of hydrogen leakage disasters. The best hydrogen sensor laying scheme is obtained by interacting with a hydrogen leakage simulation environment that contains as many leakage conditions as possible. Finally, according to the optimized sensor laying scheme, hydrogen sensors are purchased and on-site construction is carried out.

[0023] From the perspectives of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time and minimizing the disastrous consequences of hydrogen leakage, the present invention obtains the optimal hydrogen sensor layout plan through reinforcement learning, including the detection technology, upper detection limit, lower detection limit, response time, alarm threshold and installation location of each hydrogen sensor, thereby improving the safety and intelligence level in typical hydrogen energy scenarios.

[0024] The layout optimization method provided by the present invention does not need to rely on expert experience, but obtains the optimal hydrogen sensor laying plan based on scientific calculations, which is more objective and accurate.

[0025] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0027] Figure 1 This is a simulation and optimization layout architecture diagram of the first embodiment.

[0028] Figure 2 This is a flow chart of the method of the first embodiment. DETAILED DESCRIPTION

[0029] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0030] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0031] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0032] Embodiment 1

[0033] In order to construct a scientific, effective and rationally arranged hydrogen leakage monitoring system, this embodiment provides a method for optimizing the layout of hydrogen sensors, such as Figure 1 and Figure 2 This embodiment discloses a method for optimizing the layout of hydrogen sensors based on reinforcement learning, which is applicable to typical hydrogen energy application scenarios and may include the following steps:

[0034] Simulate the hydrogen concentration field of various hydrogen leakage conditions in hydrogen energy application scenarios;

[0035] Construct the feasible domain of hydrogen sensor layout scheme;

[0036] Based on the feasible domain of hydrogen sensor layout schemes, a reinforcement learning agent is constructed. The hydrogen energy application scenarios of various hydrogen leakage conditions are regarded as the environment, with the goal of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time and minimizing the consequences of hydrogen leakage disasters. Through the continuous interaction between the agent and the environment, the optimal hydrogen sensor layout scheme is obtained.

[0037] This embodiment is applicable to the optimized layout of hydrogen sensors in typical hydrogen energy application scenarios, including but not limited to hydrogen production plants, hydrogen refueling stations, hydrogen energy highway service areas, hydrogen energy communities, hydrogen-electric coupling systems, etc. First, computational fluid dynamics technology is used to simulate the hydrogen concentration field of various hydrogen leakage conditions. Secondly, an "intelligent agent" based on reinforcement learning is developed, and the hydrogen application scenarios of multiple leakage conditions are regarded as "environments", and the feasible domain of hydrogen sensor layout schemes is constructed, and the rewards of reinforcement learning are set. Then, through continuous interaction with the "environment", the intelligent agent gradually optimizes the sensor layout scheme it generates to achieve the goals of maximizing the coverage of hydrogen leakage conditions, minimizing the detection time of hydrogen leaks, and minimizing the consequences of hydrogen leaks. Finally, the best hydrogen sensor optimization layout scheme is obtained, and the corresponding hydrogen sensors are purchased and laid according to the scheme to build a hydrogen leak monitoring system.

[0038] Next, the technical solution of the embodiment of the present application will be explained in detail with reference to the accompanying drawings.

[0039] 1. Simulation

[0040] The premise for achieving the optimal layout of hydrogen sensors is to obtain as much representative hydrogen leakage and diffusion data as possible for the current hydrogen energy application scenario. Computer-aided tools such as FLUENT, CFX, OpenFOAM, etc. are used to simulate different leakage conditions under the hydrogen energy scenario. The preset different hydrogen leakage conditions of the target hydrogen energy application scenario are distinguished by setting environmental parameters such as leakage location, direction, rate, wind speed and direction. For each leakage condition, perform the following steps:

[0041] (1) Import the 3D model of the hydrogen energy scenario (see Section 1.1);

[0042] (2) Divide the grid and perform local encryption;

[0043] (3) Set up the hydrogen leakage diffusion model (see Section 1.2);

[0044] (4) Define boundary conditions based on the current leakage operating parameters (see Section 1.3);

[0045] (5) Select the solver and set the solution parameters;

[0046] (6) Perform simulation calculations;

[0047] (7) Obtain hydrogen diffusion concentration field data under the current leakage condition and export the data to Excel; 1.1 Construct a three-dimensional model of the hydrogen energy scenario

[0048] First, detailed field surveys and measurements are required for the target hydrogen energy application scenarios. Various dimensional information in the scenarios should be collected, including the length, width, height of the site, as well as the specific dimensions and location distribution of various equipment and pipelines. For complex hydrogen energy application scenarios, multiple functional areas may be involved, such as hydrogen storage areas, hydrogenation areas, fuel cell power generation areas, etc. These areas should be accurately mapped separately to ensure that the characteristics of each area can be accurately presented in the three-dimensional model.

[0049] Then, professional 3D modeling software, such as 3DMAX, SketchUp, etc., is used to convert the collected dimensional data into a visual 3D model. In the modeling process, the shape of the equipment, the direction of the pipeline, the connection method and other details are restored as realistically as possible. Buildings, obstacles, etc. in the modeling scene that may affect the leakage and diffusion of hydrogen are accurately modeled.

[0050] 1.2 Constructing a hydrogen diffusion model for hydrogen energy scenarios

[0051] Establishing a hydrogen diffusion model for hydrogen energy scenarios is the key to accurately simulating hydrogen leakage and diffusion. The hydrogen diffusion model should take into full account a variety of factors and make the following assumptions: (1) The leaked hydrogen is a compressible fluid and is considered an ideal gas, following the ideal gas state equation without considering the effect of viscous fluid; (2) The hydrogen-air mixture does not undergo chemical reaction during the flow process; (3) The hydrogen leakage rate remains unchanged during the leakage process.

[0052] Based on the above assumptions, the leakage and diffusion problem of hydrogen in typical hydrogen energy application scenarios can be classified as a single-phase multi-component convection diffusion problem without chemical reaction. At the same time, the diffusion behavior of hydrogen after leakage is usually turbulent. Therefore, the hydrogen leakage and diffusion problem needs to follow the following control equations, namely, the mass conservation equation, momentum conservation equation, energy conservation equation, component mass conservation equation, and turbulent transport equation.

[0053] (1) Mass conservation equation

[0054] The mass conservation equation is also called the continuity equation. Hydrogen leakage diffusion can be described using the mass conservation equation for transient three-dimensional compressible fluids:

[0055]

[0056] Where ρ is the density of the mixture (hydrogen and air) in the confined space, in kg / m 3 ; t is time, unit is s; u, v, w refer to the speed of x, y, z axis respectively, unit is m / s.

[0057] (2) Momentum conservation equation

[0058] The mass conservation equation is also called the momentum equation, the motion equation, or the Navier-Stokes equation (NS equation). The momentum conservation equation in the hydrogen leakage and diffusion problem can be expressed as:

[0059]

[0060] Where μ is the dynamic viscosity of the hydrogen-air mixture, also called the viscous dissipation coefficient, a constant with the unit of Pa·s; ρ a is the density of air, a constant with a value of 1.225 and a unit of kg / m3; g is the acceleration due to gravity, a constant with a value of -9.8 and a unit of m / s 2 ; p is absolute pressure, unit is Pa.

[0061] (3) Energy conservation equation

[0062] The energy conservation equation is a basic law that must be satisfied by a flow system that includes heat exchange, namely the first law of thermodynamics. For the leakage and diffusion problem of hydrogen, the energy conservation equation can be expressed by the following formula:

[0063]

[0064] In the formula, c p is the specific heat capacity, a constant, with a value of 14.3, and a unit of J / (kg·K); k is the heat transfer coefficient, a constant, with a value of 35, and a unit of W / (m·K); T is the temperature, and a unit of K. S T is the viscous dissipation term, and its expression is:

[0065]

[0066] (4) Ideal gas state equation

[0067] Combining the mass conservation equation, momentum conservation equation, and energy conservation equation, there are six unknown quantities: ρ, p, u, v, w, and T. It is also necessary to add a state equation for p and ρ so that the equation group can be closed. In this article, hydrogen in a confined space is considered as an ideal gas and follows the ideal gas state equation. Therefore:

[0068] p=ρRT

[0069] Where R is the molar gas constant, which is 8.314 J / (mol·K).

[0070] (5) Component mass conservation equation

[0071] In a typical hydrogen energy scenario, the hydrogen leakage and diffusion problem complies with the component conservation law, that is, it satisfies the component mass conservation equation, also known as the component equation. The mass conservation equation of a component is actually a concentration transport equation, so the component equation is also called the concentration transport equation, or concentration equation.

[0072]

[0073] Where D h is the diffusion coefficient of hydrogen, a constant with a value of 0.6×10 -4 , unit is m 2 / s;,c h is the mass fraction (volume concentration) of hydrogen, in mol / L; ρc h is the mass concentration of hydrogen, in g / L.

[0074] (6) Standard k-ε turbulence model equation

[0075] In this example, a standard k-ε turbulence model is used, which has the following form:

[0076]

[0077] 1.3 Boundary conditions

[0078] According to the preset hydrogen leakage conditions, the corresponding boundary conditions are set, including the inlet, outlet, wall and other boundary conditions. Boundary conditions are used to reflect different leakage conditions. For example, different environmental wind speed, pressure, temperature and other boundary conditions will have a significant impact on the leakage and diffusion of hydrogen. Changes in the size and direction of the wind speed at the inlet will change the diffusion speed and direction of hydrogen in the air; different environmental pressures may affect the leakage rate and diffusion range of hydrogen; changes in temperature will affect the physical properties of hydrogen such as density, and thus affect its diffusion behavior. In addition, different leakage conditions will be generated according to different leakage locations, leakage rates, etc.

[0079] 2. Optimize layout

[0080] This embodiment provides a hydrogen sensor optimization layout method based on reinforcement learning, which is used to design a hydrogen sensor network to detect hydrogen leaks in typical hydrogen energy application scenarios. Based on the concentration data of different leakage conditions in hydrogen energy application scenarios, a feasible domain of feasible hydrogen sensor layout schemes is defined, and a set of optimal sensor laying schemes are optimized through reinforcement learning to minimize the expected target, so as to maximize the monitoring effect of the hydrogen leak monitoring system.

[0081] 2.1 Feasible domain of hydrogen sensor layout scheme

[0082] There are many kinds of hydrogen sensors, and different sensor combination schemes have different costs and detection effects. The premise of optimizing the layout is to define the feasible domain of the hydrogen sensor layout scheme, that is, the detection technology information of each sensor. The feasible domain of the hydrogen sensor layout scheme includes the detection technology of different sensors, sensor parameters, and the alarm threshold set during installation.

[0083] Specifically, hydrogen sensor detection technologies include electrochemistry, acoustics, optics, and atomic cluster beam technology, etc. Hydrogen sensor parameters include upper detection limit, lower detection limit, and response time.

[0084] 2.2 Reinforcement Learning “Agents”

[0085] Reinforcement learning “agents” are used to generate hydrogen sensor layout plans and continuously optimize the hydrogen sensor layout plans through interaction with the “environment” in an iterative process.

[0086] In this embodiment, eight different neural network agent models together constitute the intelligent agent, which are represented as

[0087] and Assume that the hydrogen sensor layout scheme consists of M sensors.

[0088] For the first neural network proxy model The action representing the output consists of M integers. Action sequence A 1 is used to represent the detection technology of M hydrogen sensors, and the detection technology of any sensor comes from the feasible domain of detection technology, that is, φ represents the feasible domain of hydrogen sensor detection technology. Similarly, Indicates that the input state consists of M integers, and the action in the previous iteration is regarded as S in the current iteration 1 In addition, θ 1 Represents the network parameters of the first neural network agent model.

[0089] For the second neural network proxy model The action representing the output consists of M real numbers. Action sequence A 2 is used to represent the upper detection limit of M hydrogen sensors, and the upper detection limit of any sensor comes from the upper detection limit feasible domain, that is, χ represents the upper limit of detection of hydrogen sensor. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 2 In addition, θ 2 Represents the network parameters of the second neural network agent model.

[0090] For the third neural network agent model The action representing the output consists of M real numbers. Action sequence A 3 is used to represent the detection limit of M hydrogen sensors, and the detection limit of any sensor comes from the detection limit feasible domain, that is, δ represents the feasible region of the lower detection limit of the hydrogen sensor. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 3 In addition, θ 3 Represents the network parameters of the third neural network agent model.

[0091] For the fourth neural network agent model The action representing the output consists of M real numbers. Action sequence A 4 is used to represent the response time of M hydrogen sensors, and the response time of any sensor comes from the feasible domain of response time, that is, λ represents the feasible domain of the response time of the hydrogen sensor. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 4 In addition, θ 4 Represents the network parameters of the fourth neural network agent model.

[0092] For the fifth neural network agent model The action representing the output consists of M real numbers. Action sequence A 5 is used to represent the alarm thresholds of M hydrogen sensors, and the alarm threshold of any sensor comes from the feasible domain of the alarm threshold, that is, represents the feasible domain of the alarm threshold of the hydrogen sensor. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 5 In addition, θ 5 Represents the network parameters of the fifth neural network agent model.

[0093] For the sixth neural network agent model The action representing the output consists of M real numbers. Action sequence A 6 is used to represent the X-axis coordinates of the M hydrogen sensors. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 6 In addition, θ 6 Represents the network parameters of the sixth neural network agent model.

[0094] For the seventh neural network agent model The action representing the output consists of M real numbers. Action sequence A 7 is used to represent the Y-axis coordinates of the M hydrogen sensors. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 7 In addition, θ 7 Represents the network parameters of the seventh neural network agent model.

[0095] For the eighth neural network agent model The action representing the output consists of M real numbers. Action sequence A 8 is used to represent the Z-axis coordinates of the M hydrogen sensors. Similarly, Indicates that the input state consists of M real numbers, and the action in the previous iteration is regarded as S in the current iteration 8 In addition, θ 8 Represents the network parameters of the eighth neural network agent model.

[0096] From the above content, we can see that the intelligent agent composed of eight neural network agent models can output 8 sets of action lists in each iteration. Then these 8 action lists together form a layout plan containing M sensors, which includes the detection technology, upper detection limit, lower detection limit, alarm threshold, response time and installation location of each sensor.

[0097] In this embodiment, the agent is trained through reinforcement learning. Specifically, the agent learns by maximizing its expected reward R through interaction with the environment in an iterative process, and finally obtains the best layout plan of hydrogen sensors. In each iteration, the policy gradient of each neural network agent model is calculated according to the reward value fed back by the environment, and the parameters of each neural network agent model are updated through the gradient ascent algorithm, that is, θ 1 ,θ 2 ,θ 3 ,θ 4 ,θ 5 ,θ 6 ,θ 7 and θ8 .

[0098] 2.3 Reinforcement Learning “Environment”

[0099] In this embodiment, the reinforcement learning "environment" provides evaluation and feedback for each hydrogen sensor layout scheme generated by the agent, and extracts the impact of each hydrogen sensor layout scheme when detecting different leakage conditions. This impact includes the coverage, detection time and disaster consequences of all preset hydrogen leakage conditions, and the impact is evaluated by the evaluator. The evaluation results are fed back to the agent in the form of rewards to obtain a better hydrogen sensor layout scheme.

[0100] Embodiment 2

[0101] This embodiment discloses a hydrogen sensor optimization layout system based on reinforcement learning.

[0102] A hydrogen sensor optimization layout system based on reinforcement learning, comprising:

[0103] The hydrogen leakage simulation module is configured to: simulate the hydrogen concentration field of various hydrogen leakage conditions under hydrogen energy application scenarios;

[0104] The feasible domain construction module is configured to: construct a feasible domain of the hydrogen sensor layout solution;

[0105] The reinforcement learning module is configured as follows: based on the feasible domain of the hydrogen sensor layout plan, a reinforcement learning agent is constructed; hydrogen energy application scenarios with various hydrogen leakage conditions are regarded as the environment, with the goal of maximizing the coverage of hydrogen leakage conditions, minimizing the hydrogen leakage detection time and minimizing the consequences of hydrogen leakage disasters, and obtaining the optimal hydrogen sensor layout plan through continuous interaction between the agent and the environment.

[0106] Embodiment 3

[0107] The purpose of this embodiment is to provide a computer-readable storage medium.

[0108] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for optimizing the layout of hydrogen sensors based on reinforcement learning as described in Example 1 of the present disclosure.

[0109] Embodiment 4

[0110] The purpose of this embodiment is to provide an electronic device.

[0111] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the method for optimizing the layout of a hydrogen sensor based on reinforcement learning as described in Example 1 of the present disclosure are implemented.

[0112] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0113] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0114] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A hydrogen sensor optimization layout method based on reinforcement learning, characterized in that: The following steps are involved: Simulate the hydrogen concentration field of various hydrogen leakage conditions in hydrogen energy application scenarios; Construct the feasible domain of hydrogen sensor layout scheme; Based on the feasible domain of hydrogen sensor layout scheme, a reinforcement learning agent is constructed; The hydrogen energy application scenarios of various hydrogen leakage conditions are regarded as the environment. The goals are to maximize the coverage of hydrogen leakage conditions, minimize the hydrogen leakage detection time and minimize the consequences of hydrogen leakage disasters. Through the continuous interaction between the intelligent agent and the environment, the optimal hydrogen sensor layout plan is obtained.

2. The method for optimizing the layout of hydrogen sensors based on reinforcement learning according to claim 1, characterized in that: Simulate the hydrogen concentration field of various hydrogen leakage conditions under hydrogen energy scenarios, including: Construct a three-dimensional model of hydrogen energy application scenarios based on computational fluid dynamics technology; According to different hydrogen leakage conditions, the hydrogen leakage diffusion path is simulated, and the hydrogen concentration field data of different hydrogen leakage conditions are calculated; Among them, different hydrogen leakage conditions are distinguished by environmental parameters, and the environmental parameters include leakage location, leakage direction, leakage rate and wind speed and direction.

3. The method for optimizing the layout of hydrogen sensors based on reinforcement learning as claimed in claim 2, characterized in that: According to different hydrogen leakage conditions, the hydrogen leakage diffusion path is simulated, and the hydrogen concentration field data of different hydrogen leakage conditions are calculated, including: Based on the mass conservation equation, momentum conservation equation, energy conservation equation, ideal gas state equation, component mass conservation equation and turbulent transport equation, a hydrogen diffusion model for hydrogen energy application scenarios is established; Define boundary conditions according to different hydrogen leakage operating parameters; Combine boundary regulation and hydrogen diffusion model to simulate the hydrogen leakage diffusion path; Select the solver and set the solution parameters to calculate the hydrogen concentration field data for different hydrogen leakage conditions.

4. The method for optimizing the layout of hydrogen sensors based on reinforcement learning according to claim 1, characterized in that: The feasible domain of the hydrogen sensor layout scheme includes detection technologies of different hydrogen sensors, hydrogen sensor parameters and alarm thresholds set during installation, wherein the detection technologies of the different hydrogen sensors include electrochemical, acoustic, optical and atomic cluster beam technologies, and the hydrogen sensor parameters include upper detection limit, lower detection limit and response time.

5. The method for optimizing the layout of hydrogen sensors based on reinforcement learning as claimed in claim 1, characterized in that: Based on the feasible domain of hydrogen sensor layout scheme, a reinforcement learning agent is constructed, including: Build eight different neural network agent models that together form an intelligent agent; Each of the neural network proxy models includes an input state and an output action, wherein the input state of this iteration is the output action of the previous iteration, and the output actions are all from the feasible domain of the hydrogen sensor layout solution; The output actions of the eight different neural network agent models are the detection technology, upper detection limit, lower detection limit, response time, alarm threshold, X-axis coordinate, Y-axis coordinate and Z-axis coordinate of the hydrogen sensor.

6. The method for optimizing the layout of hydrogen sensors based on reinforcement learning according to claim 1, characterized in that: Through the continuous interaction between the agent and the environment, the optimal hydrogen sensor layout solution is obtained, including: Setting rewards for reinforcement learning; In each iteration, the policy gradient of each neural network agent model is calculated according to the reward value fed back by the environment, and the parameters of each neural network agent model are updated; Learning is performed by maximizing the reward value, and finally the optimal hydrogen sensor layout plan is obtained, including the detection technology, upper detection limit, lower detection limit, response time, alarm threshold and installation location of each hydrogen sensor.

7. The method for optimizing the layout of hydrogen sensors based on reinforcement learning as claimed in claim 1, characterized in that: It also includes purchasing and laying corresponding hydrogen sensors according to the optimal hydrogen sensor layout plan, and building a hydrogen leakage monitoring system.

8. A hydrogen sensor optimization layout system based on reinforcement learning, characterized in that: include: The hydrogen leakage simulation module is configured to: simulate the hydrogen concentration field of various hydrogen leakage conditions under hydrogen energy application scenarios; The feasible domain construction module is configured to: construct a feasible domain of the hydrogen sensor layout solution; The reinforcement learning module is configured to: construct a reinforcement learning agent based on the feasible domain of the hydrogen sensor layout scheme; The hydrogen energy application scenarios of various hydrogen leakage conditions are regarded as the environment. The goals are to maximize the coverage of hydrogen leakage conditions, minimize the hydrogen leakage detection time and minimize the consequences of hydrogen leakage disasters. Through the continuous interaction between the intelligent agent and the environment, the optimal hydrogen sensor layout plan is obtained.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the hydrogen sensor optimization layout method based on reinforcement learning as described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the hydrogen sensor optimization layout method based on reinforcement learning as described in any one of claims 1 to 7 are implemented.

Images