Intelligent low-temperature liquid evaporator control optimization method

Abstract

The invention discloses an intelligent low-temperature liquid evaporator control optimization method. The method comprises the steps of low-temperature liquid evaporator multi-valve control unit modeling, self-adaptive evaporator loss function design, multi-valve control unit centralized execution evaluation architecture design, priority experience playback training and evaporator control optimization. The invention belongs to the field of intelligent control, and particularly relates to an intelligent low-temperature liquid evaporator control optimization method, which comprises the following steps of: respectively constructing a precision control reward item, an energy consumption penalty item and a safety penalty item, and balancing control precision and system collaboration; based on self-adaptive evaporator loss function design, action logic contradictions are avoided through conflict punishment, and then control optimization stability is improved; the action conflict degree is incorporated into experience priority calculation, and a conflict scene is learned preferentially; importance weight and collaborative penalty design are added to evaluate network loss, so that the action conflict probability is remarkably reduced; and the distribution offset is corrected through importance sampling, so that the control optimization reliability is improved.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of intelligent control, and particularly relates to an intelligent low-temperature liquid evaporator control optimization method. BACKGROUND

[0002] The low-temperature liquid evaporator control optimization method is a technical means for realizing the goals of stable evaporator outlet medium temperature, system pressure compliance and low energy consumption by regulating and controlling the actions of key components such as a feed valve, a heat source valve and an outlet valve and optimizing the valve opening degree and operating parameters. However, the general evaporator control optimization method has the problems of poor control synergy, lack of stage adaptability, slow startup convergence, easy fluctuation in steady state, disconnection between safety and energy consumption goals, and thus poor control optimization effect. The general evaporator control optimization method has the problems of insufficient control action stability, weak anti-interference ability, low experience learning efficiency, difficulty in coping with complex scenes, and thus low control optimization reliability. SUMMARY

[0003] In view of the above problems, the application provides an intelligent low-temperature liquid evaporator control optimization method to overcome the defects of the prior art. The method balances the control accuracy and system synergy by modeling a multi-valve control unit, respectively constructing an accuracy control reward item, an energy consumption penalty item and a safety penalty item, and shortens the warming-up time by quickly converging at startup and reducing the energy consumption and wear by reducing the frequent action of the valve in the steady state. The method avoids action logic contradiction by conflict penalty and improves the control optimization stability. The method takes the action conflict degree into the experience priority calculation by designing an independent execution network + shared evaluation network architecture, preferentially learns the conflict scene, adds the importance weight and the synergy penalty to design the evaluation network loss, quickly forms the synergy control strategy, significantly reduces the action conflict probability, corrects the distribution deviation by importance sampling, avoids the training deviation caused by preferential sampling, and improves the control optimization reliability.

[0004] The technical scheme adopted by the application is as follows: The application provides an intelligent low-temperature liquid evaporator control optimization method, which comprises the following steps:

[0005] Step S1: modeling a low-temperature liquid evaporator multi-valve control unit;

[0006] Step S2: adaptive evaporator loss function design;

[0007] Step S3: Multi-valve control unit centralized implementation of evaluation architecture design;

[0008] Step S4: Prior experience replay training;

[0009] Step S5: Evaporator control optimization.

[0010] Further, in step S1, the low-temperature liquid evaporator multi-valve control unit modeling specifically includes:

[0011] Define a five-tuple, including evaporator operating state S, valve control unit output action, state transition T, control target quantified reward R, and discount factor γ; the evaporator operating state is obtained by collecting the low-temperature liquid feed flow rate , heat source temperature , evaporator shell side pressure , outlet gas temperature , outlet gas pressure , and ambient temperature ; the valve control unit control output is the valve opening, normalized to [-1, 1], which is the feed valve valve control unit opening , heat source valve valve control unit opening , and outlet valve valve control unit opening ; the state transition is based on the evaporator heat balance equation, describing the dynamic change of action→state, expressed as: ; where t is the sampling time; , and are the time constant, feed cooling coefficient, and temperature regulation coefficient, respectively; is the outlet temperature target value;

[0012] Define the reward function , which encourages control compliance and energy consumption reduction, and punishes over-temperature and over-pressure, and constructs a precision control reward item , expressed as: ; construct an energy consumption penalty item , expressed as: ; construct a safety penalty item , expressed as: ; ; where and are the temperature compliance reward coefficients; and are the pressure compliance reward coefficients; is the energy consumption coefficient; is the over-temperature penalty coefficient, is the over-temperature threshold; is the over-pressure penalty coefficient, is an overpressure threshold value; is an outlet pressure target value.

[0013] Further, in step S2, the adaptive evaporator loss function design is to define an adaptive segmented MSE loss , expressed as: ; wherein the segmented weight is dynamically adjusted with the control phase, expressed as: ; is the current Q value for valve control unit i; is the target Q value; A is the joint action of the three valve control units; is the action conflict penalty, expressed as: ; is the conflict penalty coefficient.

[0014] Further, in step S3, the multi-valve control unit centralized execution evaluation architecture design specifically includes the following:

[0015] Step S31: Architecture design; each of the feed valve, heat source valve, and outlet valve is equipped with an independent execution network, and each network only outputs the opening instruction of the corresponding valve; 1 evaluation network is shared, which inputs the global operating state of the evaporator and the joint action of the three valves, evaluates the long-term control value of the current action combination, and feeds back the evaluation result to the three execution networks;

[0016] Step S32: Execution network definition; each execution network is only responsible for the opening calculation of a single valve, and the input is the global state of the evaporator, and the output is the normalized opening of the valve;

[0017] Step S33: Evaluation network definition; the centralized evaluation network evaluates the joint action value;

[0018] Step S34: Target Q value calculation.

[0019] Further, in step S4, the priority experience replay training specifically includes the following:

[0020] Step S41: Experience pool construction; the interaction experience of the evaporator at each step is stored as a tuple;

[0021] Step S42: Evaluate experience value; the priority of the experience combines the Q value error and the conflict degree, expressed as: ; rank(·) is the ranking function; is the conflict weight; is the current Q value for valve control unit i; is the target Q value;

[0022] Step S43: importance sampling; introduce importance weight correction distribution offset, denoted as: ; ; and are the importance weights after and before correction respectively; D is the experience pool capacity; is the weight coefficient;

[0023] Step S44: network parameter update; join importance weight and collaborative punishment to design evaluation network loss , denoted as: ; wherein, is the expectation.

[0024] Further, in step S5, the evaporator control optimization is to deploy the trained multi-valve control unit model to the evaporator DCS system to form a real-time collection-distributed execution-centralized evaluation-online update closed loop.

[0025] The beneficial effects obtained by the present application using the above scheme are as follows:

[0026] (1) For the problems of poor control collaboration, lack of stage adaptability, slow start convergence, easy fluctuation in steady state, and disconnection between safety and energy consumption target in general evaporator control optimization methods, leading to poor control optimization effect, the present scheme builds precision control reward item, energy consumption penalty item and safety penalty item through multi-valve control unit modeling, balances control precision and system collaboration; based on adaptive evaporator loss function design, quickly converges at start to shorten the warming-up time, reduces valve frequent action at steady state to reduce energy consumption and wear; at the same time, avoids action logic contradiction through conflict penalty, thereby improving control optimization stability.

[0027] (2) For the problems of insufficient control action stability, weak anti-interference ability, low experience learning efficiency, difficult to cope with complex scenes, and low control optimization reliability in general evaporator control optimization methods, the present scheme designs independent execution network + shared evaluation network architecture, includes action conflict degree into experience priority calculation to preferentially learn conflict scenarios; and joins importance weight and collaborative punishment to design evaluation network loss, quickly forms collaborative control strategy, significantly reduces action conflict probability; corrects distribution offset through importance sampling to avoid training deviation caused by preferential sampling; thereby improves control optimization reliability. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 A flowchart of an intelligent low-temperature liquid evaporator control optimization method provided by the present application.

[0029] The accompanying drawings are included to provide a further understanding of the application, and are incorporated in and constitute a part of this specification, illustrate embodiments of the application, and together with the description serve to explain the application, and do not limit the application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the application will be clearly and completely described below with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only part of the embodiments of the application, rather than all the embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the application.

[0031] In the description of the application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and do not indicate or imply that the systems or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application.

[0032] Embodiment one, refer to Figure 1 The application provides an intelligent low-temperature liquid evaporator control optimization method, which comprises the following steps:

[0033] Step S1: modeling of the low-temperature liquid evaporator multi-valve control unit; converting the low-temperature liquid evaporator control problem into a three-valve control unit decision process, defining a five-tuple including state, action, transition, reward and discount factor, and decoupling into three valve control units;

[0034] Step S2: adaptive evaporator loss function design; designing a loss for the three-stage control characteristics of the evaporator, including a segmented dynamic weight and a valve control unit collaborative penalty term, and updating the execution network parameters based on the loss gradient;

[0035] Step S3: multi-valve control unit centralized execution evaluation architecture design; including an independent execution network, a shared evaluation network, a target network and parameter definition;

[0036] Step S4: priority experience replay training; including building an experience pool, evaluating experience value, importance sampling and updating network parameters;

[0037] Step S5: evaporator control optimization; realizing evaporator control optimization based on the multi-valve control unit.

[0038] Embodiment two, refer to Figure 1The embodiment is based on the above embodiment, in step S1, the low-temperature liquid evaporator multi-valve control unit modeling is to transform the control problem into a three-valve control unit interactive Markov decision process, each valve control unit corresponds to a control variable, the joint action is evaluated by a centralized evaluator during training, and is executed in a decentralized manner during execution, balancing control accuracy and system collaboration;

[0039] The specific operation is as follows: a five-tuple is defined, including an evaporator operating state S, a valve control unit output action, a state transition T, a control target quantified reward R, and a discount factor γ; the evaporator operating state is acquired by a sensor from low-temperature liquid feed flow , heat source temperature , evaporator shell side pressure , outlet gas temperature , outlet gas pressure , and ambient temperature ; the valve control unit control output action is a valve opening degree, normalized to [-1, 1], which is a feed valve valve control unit opening degree , a heat source valve valve control unit opening degree , and an outlet valve valve control unit opening degree ; the state transition is based on an evaporator heat balance equation, describes the dynamic change of action→state, and is expressed as: ; wherein t is a sampling time; , and are time constants, feed cooling coefficients, and temperature regulation coefficients, respectively; is an outlet temperature target value;

[0040] A reward function is defined, which simultaneously encourages control compliance and energy consumption reduction, and punishes over-temperature and over-pressure, and a precision control reward item is constructed, which is expressed as: ; an energy consumption penalty item is constructed, which is expressed as: ; a safety penalty item is constructed, which is expressed as: ; ; wherein and are temperature compliance reward coefficients; and are pressure compliance reward coefficients; is an energy consumption coefficient, which avoids waste due to full opening of the heat source valve; is an over-temperature penalty coefficient, is an over-temperature threshold; is an over-pressure penalty coefficient, is an over-pressure threshold; is an outlet pressure target value;

[0041] Decoupling the feed-heat source-outlet into three valve control units avoids the trade-off caused by single-variable control, and directly avoids the risk of over-temperature and over-pressure based on safety penalty reward.

[0042] Embodiment three, see Figure 1 This embodiment is based on the above embodiment. In step S2, the adaptive evaporator loss function design is for the three-stage control characteristics of evaporator startup-transition-steady state. The segmented weighted loss is introduced to increase the weight of large error in the startup stage to accelerate convergence, and to reduce the weight in the steady state to suppress small fluctuations. The valve control unit cooperative penalty term is introduced to solve the action conflict of multiple valve control units.

[0043] The specific operation is as follows:

[0044] Adaptive segmented MSE loss The definition is represented as: ; wherein the segmented weight is dynamically adjusted according to the control stage, represented as: ; the weight in the startup stage is doubled to accelerate large error correction; the weight in the steady state is halved to avoid overfitting to small disturbances; is the current Q value of the valve control unit i; is the target Q value; A is the joint action of the three valve control units; is the action conflict penalty, which is used to quantify the synergy of valve control unit action, represented as: ; is the conflict penalty coefficient, which ensures that synergy is prioritized over single-variable optimization; the execution network parameters of the valve control unit are updated based on the loss gradient.

[0045] Through the above operation, for the problems of poor control synergy, lack of stage adaptability, slow startup convergence, steady state fluctuation, and disconnection between safety and energy consumption in general evaporator control optimization methods, the present scheme balances control precision and system synergy by modeling multiple valve control units, respectively constructing precision control reward items, energy consumption penalty items and safety penalty items; based on adaptive evaporator loss function design, fast convergence is achieved in the startup stage to shorten the warming-up time, and in the steady state, the valve action is reduced to reduce energy consumption and wear; at the same time, the conflict penalty is used to avoid action logic contradiction, thereby improving the stability of control optimization.

[0046] Embodiment four, see Figure 1 This embodiment is based on the above embodiment. In step S3, the multi-valve control unit centralized execution evaluation architecture design specifically includes the following contents:

[0047] Step S31: Architecture design; each of the feed valve, heat source valve, and outlet valve is equipped with an independent execution network, each network only outputs the opening degree instruction of the corresponding valve, parallel computing, meets the real-time control demand of the evaporator; share 1 evaluation network, input the global running state of the evaporator and the joint action of the three valves, evaluate the long-term control value of the current action combination, and feed back the evaluation result to the three execution networks to guide the optimization action and avoid conflicts;

[0048] Step S32: Execution network definition; each execution network is only responsible for the opening degree calculation of a single valve, the input is the global state of the evaporator, and the output is the normalized opening degree of the valve, which ensures that the action can directly drive the actuator; the execution network output of the valve control unit i is expressed as: ; the output is normalized to [-1, 1] by the tanh activation function, which directly corresponds to the valve opening degree; is the strategy function of the valve control unit i, which maps the state to the action, is the corresponding network parameter; S(t) is the running state of the evaporator at time t;

[0049] Step S33: Evaluation network definition; the control effect of the evaporator needs to be judged globally, so the evaluation network needs to input the global state + joint action, and output the quantitative action value Q value, the higher the Q value, the better the current action combination can achieve the multi-objective optimization of temperature standard + pressure stability + low energy consumption, thereby guiding the execution network to adjust the valve opening degree; the centralized evaluation network evaluates the joint action value, which is expressed as: ; MLP is a multi-layer perceptron, the input layer is 6+3=9 dimensions, the hidden layer is 256 dimensions, and the output is 1 dimension Q value; is the evaluation network parameter, globally shared, and updated based on the joint error; A(t) is the joint action of the three valves at time t; ω and b are the weight parameter and bias parameter of the evaluation network, respectively;

[0050] Step S34: Target Q value calculation; if the Q value of the current evaluation network is directly used to guide the execution network optimization, the Q value will be unstable due to the fluctuation of the current network parameter, which will further cause the valve opening degree to change frequently and aggravate the temperature and pressure fluctuation of the evaporator; therefore, a target network is introduced to calculate a stable target Q value as a benchmark for execution optimization; the target Q value is expressed as: ; is the evaluation target network; is the target network parameter, which is obtained by updating the smoothing parameter; the target execution network output of the valve control unit i is expressed as: ; 、 and is the strategy function of the target execution network of the three valve control units, , and is the corresponding target execution network parameter.

[0051] Embodiment five, refer to Figure 1 , based on the above embodiment, in step S4, the priority experience replay training is that the high value experience in the evaporator operation has the characteristics of multi-valve control unit, including multi-variable disturbance experience, feed flow + environmental temperature fluctuation at the same time, which needs to learn the anti-disturbance strategy preferentially; action conflict experience, valve control unit 1 open, valve control unit 2 close, which leads to temperature drop, needs to learn conflict avoidance preferentially; therefore, the experience value evaluation of multi-valve control unit is introduced to ensure that the experience with high coordination value is learned preferentially; specific contents include the following:

[0052] Step S41: experience pool construction; the interaction experience of the evaporator at each step is stored as a tuple, represented as: ; is the nth experience tuple;

[0053] Step S42: evaluate experience value; the priority of experience The comprehensive Q value error and conflict degree are represented as: ; rank(·) is the ranking function; is the conflict weight; is the current Q value for valve control unit i; is the target Q value;

[0054] Step S43: importance sampling; introduce importance weight to correct distribution deviation, represented as: ; ; and are the importance weights after and before correction respectively; D is the capacity of the experience pool; is the weight coefficient;

[0055] Step S44: network parameter update; join importance weight and coordination penalty design to evaluate network loss , represented as: ; wherein, is the expectation; update the execution network parameter based on the loss gradient; update the target network smoothing parameter.

[0056] By performing the above operation, for the general evaporator control optimization method, there is a problem of insufficient control action stability, weak anti-interference ability, low experience learning efficiency, difficulty in coping with complex scenes, and low control optimization reliability. The scheme designs an independent execution network + shared evaluation network architecture, incorporates the action conflict degree into the experience priority calculation, and preferentially learns the conflict scene; and adds importance weight and collaborative penalty to design the evaluation network loss, quickly forms a collaborative control strategy, and significantly reduces the action conflict probability; through importance sampling correction distribution deviation, avoid training deviation caused by preferential sampling; and further improve the control optimization reliability.

[0057] Embodiment six, refer to Figure 1 This embodiment is based on the above-mentioned embodiment. In step S5, the evaporator control optimization is to deploy the trained multi-valve control unit model to the evaporator DCS system to form a real-time acquisition-dispersed execution-centralized evaluation-online update closed loop: the sensor collects the state S(t) once every 1s and sends it to each valve control unit execution network; the three valve control unit outputs valve opening degree in parallel, driving the actuator; the centralized evaluation network evaluates the control effect in real time, and stores the new experience into the experience pool; the model is fine-tuned with historical data every morning to avoid performance degradation caused by evaporator aging.

[0058] It should be noted that in this paper, the term "include", "contain" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device.

[0059] Although the embodiments of the present application have been shown and described, it can be understood by those of ordinary skill in the art that various changes, modifications, replacements and variations can be made to these embodiments without departing from the principles and spirits of the present application.

[0060] The above describes the present application and its embodiments, which are not restrictive, and the embodiments shown in the drawings are only one of the embodiments of the present application, and the actual structure is not limited thereto. In general, if a person skilled in the art is inspired by it, without departing from the purpose of the present application, without creative design, similar structure and embodiments of the technical solution can be designed and belong to the protection scope of the present application.

Claims

1. An intelligent low-temperature liquid evaporator control optimization method, characterized by: The method comprises the following steps: Step S1: Modeling the multi-valve control unit of the cryogenic liquid evaporator; transforming the cryogenic liquid evaporator control problem into a three-valve control unit decision process, defining a five-tuple consisting of state, action, transition, reward, and discount factor, and decoupling it into three valve control units; Step S2: Adaptive evaporator loss function design: Design a loss function tailored to the three-stage control characteristics of the evaporator, including segmented dynamic weights and valve control unit collaborative penalty terms, and update the execution network parameters based on the loss gradient; Step S3: Design of a centralized execution evaluation architecture for a multi-valve control unit, including independent execution networks, shared evaluation networks, target networks, and parameter definitions; Step S4: Prioritized experience replay training; including building an experience pool, evaluating experience value, importance sampling, and updating network parameters; Step S5: Evaporator control optimization; evaporator control optimization is achieved based on the multi-valve control unit.

2. The intelligent low-temperature liquid evaporator control optimization method according to claim 1, characterized in that: In step S1, the modeling of the multi-valve control unit of the cryogenic liquid evaporator specifically includes: Define a five-tuple, including the evaporator operating state S, the valve control unit output action, the state transition T, the control target quantified reward R and the discount factor γ; the evaporator operating state is obtained by collecting the cryogenic liquid feed flow through the sensor , heat source temperature , evaporator shell pressure , outlet gas temperature , outlet gas pressure and ambient temperature The output action of the valve control unit is the valve opening, which is normalized to [-1,1], which are the opening of the feed valve control unit. , heat source valve control unit opening and outlet valve control unit opening The state transition is based on the evaporator heat balance equation, which describes the dynamic change from action to state and is expressed as: ; where t is the sampling time; 、 and They are the time constant, feed cooling coefficient and temperature regulation coefficient; is the outlet temperature target value; Define the reward function .

3. The intelligent low-temperature liquid evaporator control optimization method according to claim 2, characterized in that: In step S1, the reward function is defined It is to simultaneously encourage control compliance and energy consumption reduction, punish over-temperature and over-pressure, and build precision control reward items. , expressed as: ; Construct energy consumption penalty items , expressed as: ;Build security penalty items , expressed as: ; ;in, and is the temperature compliance bonus coefficient; and is the stress-reaching reward coefficient; is the energy consumption coefficient; is the overtemperature penalty coefficient, is the over-temperature threshold; is the overpressure penalty coefficient, is the overpressure threshold; is the outlet pressure target value.

4. The intelligent low-temperature liquid evaporator control optimization method according to claim 3, characterized in that: In step S2, the adaptive evaporator loss function design is to define the adaptive piecewise MSE loss , expressed as: ; Among them, the segment weight Dynamically adjusted with the control stage, expressed as: ; is the current Q value of valve control unit i; is the target Q value; A is the joint action of the three valve control units; is the action conflict penalty, expressed as: ; is the conflict penalty coefficient.

5. The intelligent low-temperature liquid evaporator control optimization method according to claim 4, characterized in that: In step S3, the multi-valve control unit centrally executes the evaluation architecture design, which specifically includes the following contents: Step S31: Architecture design: equip the feed valve, heat source valve, and outlet valve with independent execution networks, each of which outputs only the opening command of the corresponding valve. A shared evaluation network is used to input the global operating status of the evaporator and the combined action of the three valves, evaluate the long-term control value of the current action combination, and feed the evaluation results back to the three execution networks. Step S32: Execution network definition; each execution network is only responsible for the opening calculation of a single valve, with the input being the global state of the evaporator and the output being the normalized opening of the valve; Step S33: Evaluation network definition; centralized evaluation network evaluation of joint action value; Step S34: Calculate the target Q value.

6. The intelligent low-temperature liquid evaporator control optimization method according to claim 5, characterized in that: In step S4, the priority experience replay training specifically includes the following contents: Step S41: experience pool construction; Store the interaction experience of each step of the evaporator as a tuple; Step S42: Evaluate the experience value; Priority of experience The comprehensive Q value error and conflict degree are expressed as: ; rank(·) is the ranking function; is the conflict weight; is the current Q value of valve control unit i; is the target Q value; Step S43: Importance sampling; introducing importance weight to correct distribution offset, expressed as: ; ; and are the importance weights before and after correction, respectively; D is the experience pool capacity; is the weight coefficient; Step S44: Update network parameters; add importance weight and collaborative penalty design to evaluate network loss , expressed as: ;in, It's expectation.

7. The intelligent low-temperature liquid evaporator control optimization method according to claim 6, characterized in that: In step S5, the evaporator control optimization is to deploy the maturely trained multi-valve control unit model to the evaporator DCS system, forming a closed loop of real-time collection - decentralized execution - centralized evaluation - online update.

Images