Dynamic balance control method based on deep reinforcement learning, valve body and system

Abstract

The application relates to the field of control valves, in particular to a dynamic balance control method and system based on deep reinforcement learning, a valve core power model of a first valve component located in a main pipeline is constructed to predict a valve core unbalanced force event of the first valve component, trigger factors of an undesired pressure drop are determined, occurrence characteristics of the undesired pressure drop are predicted according to the occurrence characteristics of the trigger factors, the second valve component is controlled to enter a standby state, a flow dynamic model of the first valve component during formation of the undesired pressure drop is constructed, flow imbalance characteristics of the first valve component are obtained, action strategy deep reinforcement learning is performed on the second valve component according to the flow imbalance characteristics, target action strategies of the second valve component are obtained, and flow balance compensation of the main pipeline is realized. The valve core of the valve component is modeled and flow modeling is performed, and the best action strategy is determined in combination with deep reinforcement learning, pipeline flow compensation is realized, and flow stability and pipeline structure safety are ensured.

Description

Technical Field

[0001] This invention relates to the field of control valves, and more particularly to a dynamic balance control method, valve body, and system based on deep reinforcement learning. Background Technology

[0002] In HVAC and water supply engineering projects, valves are required for heat and cold distribution and water flow control. Currently, valve operation is based on the specific requirements of the engineering scenario. However, given the complexity and diversity of these requirements, existing technologies often rely on experience to select the appropriate valve control mode, which can easily lead to valve regulation deviations and affect the stability of the project's operation.

[0003] Furthermore, the complex pipeline structures in engineering scenarios make them susceptible to external environmental factors and internal structural elements, hindering the maintenance of stable flow rates. Fluctuations in flow rate can cause unbearable pressure drops, threatening the pipeline's structural safety. Existing valve control methods cannot effectively and accurately predict impending flow imbalances, nor can they implement proactive control strategies based on the valve's performance and the specific environment. Therefore, predicting potential flow imbalances based on valve performance and environment, and proactively implementing valve control strategies according to actual operation, is crucial for maintaining stable and continuous flow transmission and mitigating pipeline structural risks. Summary of the Invention

[0004] Considering the different performance conditions of each valve and the different conditions of the pipeline sections in actual engineering scenarios, conventional experience methods cannot cope with the potential abnormalities of each valve, nor can they accurately predict the possible flow imbalance of each valve, which reduces the reliability of the valve in regulating fluid transmission. Furthermore, it is not possible to pre-deploy suitable operation control strategies for each valve, which reduces the flow stability during fluid transmission in the pipeline and increases the structural risk of the pipeline.

[0005] To address the above problems, this invention provides a dynamic equilibrium control method based on deep reinforcement learning, the method comprising the following steps:

[0006] S100: Construct a valve core dynamic model of the first valve component located on the main pipeline to predict valve core imbalance force events of the first valve component; determine the triggering factors for the formation of undesirable pressure drop in the main pipeline based on the valve core imbalance force events;

[0007] S200: Obtain the occurrence characteristics of the triggering factors during the operation of the first valve component; predict the occurrence characteristics of undesirable pressure drop based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter the standby state;

[0008] S300: Construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component;

[0009] S400: Based on the flow imbalance characteristics, perform deep reinforcement learning on the action strategy of the second valve component to obtain the target action strategy of the second valve component, thereby achieving flow balance compensation for the main pipeline.

[0010] Preferably, in S100, a valve core dynamic model of the first valve component located in the main pipeline is constructed to predict unbalanced force events of the valve core of the first valve component; based on the unbalanced force events of the valve core, the triggering factors for the formation of an undesirable pressure drop in the main pipeline are determined, specifically:

[0011] Based on the three-dimensional geometric parameters and valve core action parameters of the first valve component located in the main pipeline, as well as the fluid flow parameters of the main pipeline, the modeling constraints and modeling input and output requirements are determined, thereby constructing the valve core dynamic model of the first valve component.

[0012] Based on the valve core force results output by the valve core dynamic model, predict the valve core unbalanced force event of the first valve component;

[0013] The fluid pressure field of the valve core during the period when the valve core is subjected to unbalanced force is obtained, and the fluid pressure change is extracted from the fluid pressure field to determine whether an undesirable pressure drop is formed in the main pipeline.

[0014] When an undesirable pressure drop occurs in the main pipeline, the valve core action during the valve core unbalanced force event is designated as a triggering factor.

[0015] Preferably, in S200, the occurrence characteristics of the triggering factors during the operation of the first valve component are obtained; based on the occurrence characteristics, the occurrence characteristics of undesirable pressure drops are predicted, thereby controlling the second valve component located in the secondary pipeline to enter a standby state, specifically:

[0016] Obtain the predetermined motion trajectory of the first valve component during its operation;

[0017] By comparing the predetermined action trajectory and the triggering factor, the action range within the predetermined action trajectory that matches the triggering factor is determined, and this range is used as the occurrence feature of the triggering factor.

[0018] All action parameters within the action range are input into the valve core dynamic model to predict the time when the unwanted pressure drop occurs;

[0019] Based on the occurrence time and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is controlled to switch to a standby state before a preset time point; wherein, the standby state refers to the state in which the second valve component can respond to control commands in real time.

[0020] Preferably, in S300, a flow dynamic model of the first valve component during the formation of the undesirable pressure drop is constructed to obtain the flow imbalance characteristics of the first valve component, specifically as follows:

[0021] Based on the valve core jitter parameters and fluid input parameters of the first valve component during the formation of the undesirable pressure drop, a flow dynamic model of the first valve component is constructed.

[0022] A time-domain fluctuation analysis is performed on the downstream fluid flow rate of the first valve component output by the flow dynamic model to obtain the flow imbalance characteristics; wherein, the flow imbalance characteristics include the time-domain distribution of the deviation between the downstream fluid flow rate of the first valve component and the desired fluid flow rate.

[0023] Preferably, in S400, based on the flow imbalance characteristics, deep reinforcement learning of the action strategy is performed on the second valve component to obtain the target action strategy of the second valve component, thereby achieving flow balance compensation for the main pipeline, specifically as follows:

[0024] Determine the environmental state space and action space of the second valve component; determine the flow modulation target of the second valve component based on the flow imbalance characteristics;

[0025] Based on the environmental state space, the action space, and the flow modulation target, the second valve component undergoes deep reinforcement learning of its action strategy to obtain a target action strategy; wherein, the target action strategy includes the amplitude and direction of the action of the second valve component during the formation of the undesirable pressure drop;

[0026] According to the target action strategy, a control command is applied to the second valve component to achieve flow balance compensation of the second valve component to the main pipeline.

[0027] On the other hand, the present invention provides a dynamic equilibrium control system based on deep reinforcement learning, the system comprising:

[0028] The prediction module is used to construct a valve core dynamic model of the first valve component located on the main pipeline, so as to predict the unbalanced force event of the valve core of the first valve component.

[0029] The trigger determination module is used to determine the triggering factors for the formation of an undesirable pressure drop in the main pipeline based on the unbalanced force event of the valve core.

[0030] A trigger identification module is used to acquire the occurrence characteristics of the triggering factors during the operation of the first valve component;

[0031] The status control module is used to predict the occurrence characteristics of undesirable pressure drops based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter the standby state.

[0032] A flow imbalance identification module is used to construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component.

[0033] The reinforcement learning module is used to perform deep reinforcement learning of the action strategy of the second valve component based on the flow imbalance characteristics, so as to obtain the target action strategy of the second valve component.

[0034] The compensation module is used to achieve flow balance compensation for the main pipeline according to the target action strategy.

[0035] Preferably, the prediction module is used to construct a valve core dynamic model of the first valve component located on the main pipeline, thereby predicting unbalanced force events on the valve core of the first valve component, specifically:

[0036] Based on the three-dimensional geometric parameters and valve core action parameters of the first valve component located in the main pipeline, as well as the fluid flow parameters of the main pipeline, the modeling constraints and modeling input and output requirements are determined, thereby constructing the valve core dynamic model of the first valve component.

[0037] Based on the valve core force results output by the valve core dynamic model, predict the valve core unbalanced force event of the first valve component;

[0038] The trigger determination module is used to determine the triggering factors for the formation of an undesirable pressure drop in the main pipeline based on the unbalanced force event of the valve core, specifically:

[0039] The fluid pressure field of the valve core during the period when the valve core is subjected to unbalanced force is obtained, and the fluid pressure change is extracted from the fluid pressure field to determine whether an undesirable pressure drop is formed in the main pipeline.

[0040] When an undesirable pressure drop occurs in the main pipeline, the valve core action during the valve core unbalanced force event is designated as a triggering factor.

[0041] Preferably, the trigger identification module is used to acquire the occurrence characteristics of the triggering factors during the operation of the first valve component, specifically:

[0042] Obtain the predetermined motion trajectory of the first valve component during its operation;

[0043] By comparing the predetermined action trajectory and the triggering factor, the action range within the predetermined action trajectory that matches the triggering factor is determined, and this range is used as the occurrence feature of the triggering factor.

[0044] The state control module is used to predict the occurrence characteristics of unwanted pressure drops based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter a standby state, specifically:

[0045] All action parameters within the action range are input into the valve core dynamic model to predict the time when the unwanted pressure drop occurs;

[0046] Based on the occurrence time and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is controlled to switch to a standby state before a preset time point; wherein, the standby state refers to the state in which the second valve component can respond to control commands in real time.

[0047] Preferably, the flow imbalance identification module is used to construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component, specifically:

[0048] Based on the valve core jitter parameters and fluid input parameters of the first valve component during the formation of the undesirable pressure drop, a flow dynamic model of the first valve component is constructed.

[0049] A time-domain fluctuation analysis is performed on the downstream fluid flow rate of the first valve component output by the flow dynamic model to obtain the flow imbalance characteristics; wherein, the flow imbalance characteristics include the time-domain distribution of the deviation between the downstream fluid flow rate of the first valve component and the desired fluid flow rate;

[0050] The reinforcement learning module is used to perform deep reinforcement learning of the action strategy of the second valve component based on the flow imbalance characteristics, so as to obtain the target action strategy of the second valve component, specifically:

[0051] Determine the environmental state space and action space of the second valve component; determine the flow modulation target of the second valve component based on the flow imbalance characteristics;

[0052] Based on the environmental state space, the action space, and the flow modulation target, the second valve component undergoes deep reinforcement learning of its action strategy to obtain a target action strategy; wherein, the target action strategy includes the amplitude and direction of the action of the second valve component during the formation of the undesirable pressure drop;

[0053] The compensation module is used to achieve flow balance compensation for the main pipeline according to the target action strategy, specifically:

[0054] According to the target action strategy, a control command is applied to the second valve component to achieve flow balance compensation of the second valve component to the main pipeline.

[0055] In addition, the present invention also provides a valve body, including a first valve component located in the main pipeline and a second valve component located in the secondary pipeline; the operation of the first valve component and the second valve component is based on the dynamic balance control method based on deep reinforcement learning as described above.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] This invention presents a dynamic balance control method and system based on deep reinforcement learning. It constructs a dynamic model of the valve core of a first valve component located in the main pipeline to predict unbalanced force events and identify triggering factors for undesirable pressure drops. Based on the occurrence characteristics of these triggering factors, it predicts the occurrence characteristics of undesirable pressure drops, thereby controlling a second valve component to enter a standby state. It also constructs a dynamic flow model of the first valve component during the formation of undesirable pressure drops to obtain its flow imbalance characteristics. Based on these flow imbalance characteristics, it performs deep reinforcement learning on the action strategy of the second valve component to obtain its target action strategy, achieving flow balance compensation for the main pipeline. By modeling the valve core, it predicts the occurrence time of abnormal pressure drops, guiding the valve component to prepare for regulation. Furthermore, by modeling the flow of the valve component, it determines flow imbalance and uses deep reinforcement learning to determine the optimal action strategy, achieving pipeline flow compensation and ensuring stable flow and pipeline structural safety. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0059] Figure 1 This is a flowchart of the dynamic equilibrium control method based on deep reinforcement learning provided by the present invention.

[0060] Figure 2 It is the process of constructing the valve core dynamic model.

[0061] Figure 3 It is the distribution of pressure drop changes on the main pipeline.

[0062] Figure 4 It is a prediction process that does not want pressure drop to occur.

[0063] Figure 5 It is the deviation between the downstream fluid flow rate and the desired fluid flow rate.

[0064] Figure 6 This is the process of applying control commands to the second valve component.

[0065] Figure 7 This is a structural diagram of the dynamic equilibrium control system based on deep reinforcement learning provided by the present invention.

[0066] Figure 8 This is a structural diagram of the valve body provided by the present invention. Detailed Implementation

[0067] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for explaining the present invention and not for limiting the present invention. Furthermore, it should be noted that, for ease of description, only the parts related to the present invention are shown in the accompanying drawings, not all structures. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of the present invention.

[0068] The terms "comprising" and "having," and any variations thereof, used in this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0069] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0070] Please see Figure 1 As shown, this invention provides a dynamic equilibrium control method based on deep reinforcement learning, which includes the following steps:

[0071] S100: Construct a valve core dynamic model of the first valve component located on the main pipeline to predict the valve core unbalanced force event of the first valve component; based on the valve core unbalanced force event, determine the triggering factors for the formation of an undesirable pressure drop in the main pipeline.

[0072] Furthermore, in S100, a valve core dynamic model of the first valve component located in the main pipeline is constructed to predict unbalanced force events on the valve core of the first valve component; based on the unbalanced force events on the valve core, the triggering factors for the formation of undesirable pressure drops in the main pipeline are determined, specifically:

[0073] Based on the three-dimensional geometric parameters and action parameters of the valve core of the first valve component located in the main pipeline, as well as the fluid flow parameters of the main pipeline, the modeling constraints and modeling input and output requirements are determined, thereby constructing the valve core dynamic model of the first valve component.

[0074] Based on the valve core force results output by the valve core dynamic model, predict the valve core unbalanced force event of the first valve component;

[0075] Obtain the fluid pressure field of the valve core during the period when the valve core is subjected to unbalanced force, extract the fluid pressure change from the fluid pressure field, and use this to determine whether an undesirable pressure drop has formed in the main pipeline;

[0076] When an undesirable pressure drop occurs in the main pipeline, the valve core action during the period of valve core imbalance force event is calibrated as the triggering factor.

[0077] Please see Figure 8 The valve body of this invention mainly includes a first valve component and a second valve component, respectively disposed in a main pipeline and a secondary pipeline. The main pipeline is used to transport fluids such as water and has an input end and an output end. The secondary pipeline, as an auxiliary pipeline to the main pipeline, is used for fluid transmission compensation in the main pipeline. The secondary pipeline has a three-branch structure, including an input end and two output ends. The second valve component is disposed in the central connecting area of ​​the three-branch structure and can regulate the fluid transmission flow from the output end to the two output ends. The two output ends of the secondary pipeline are connected to the main pipeline and respectively connected upstream and downstream of the first valve component. The first valve component may include, but is not limited to, three sub-valve components, which are arranged sequentially along the length of the main pipeline. The two outermost sub-valve components correspond to the connection intervals between the two output ends of the secondary pipeline and the main pipeline. The two outermost sub-valve components can independently regulate the first fluid transmission flow from the output end of the secondary pipeline to the main pipeline and regulate the second fluid transmission flow of the main pipeline. The middle sub-valve component can regulate the fluid transmission flow of the main pipeline. Understandably, the two outermost sub-valve components of the first valve assembly each have two independently operating valve cores along the main pipeline and the secondary pipeline, respectively, to regulate the first and second fluid transmission flow rates. The second valve assembly has two independently operating valve cores along the two output ends of the secondary pipeline, to regulate the fluid transmission flow from the input end to the two output ends. The secondary pipeline serves as a fluid transmission compensation pipeline for the main pipeline, and by independently and controllably compensating for fluid transmission upstream and downstream of the main pipeline along the first valve assembly, the fluid transmission stability of the main pipeline can be improved.

[0078] During the operation of the first valve component, the fluid pressure on different sides of the valve core varies as fluid flows through it, resulting in an imbalance of forces on the valve core in different directions (e.g., radial and axial). Under this imbalance, the valve core will deviate from its intended displacement. This deviation can be, but is not limited to, the difference between the actual and desired displacement of the valve core; it can also be, but is not limited to, the valve core's opening displacement. When this imbalance occurs, the valve core's displacement will disturb the fluid pressure field in the main pipeline where the first valve component is located, causing a significant pressure drop in the short-distance flow of the fluid within the main pipeline, thus threatening the structural safety of the main pipeline. It is understood that the imbalance of forces on the first valve component under fluid flow is related to the three-dimensional geometry of the valve core, the range of its actuation amplitude and angular direction, and the fluid flow velocity within the main pipeline where the first valve component is located.

[0079] Please see Figure 2 The modeling process involves three levels of factors: the three-dimensional geometry and dimensions of the valve core, the allowable range of valve core movement amplitude and direction angle, and the fluid flow velocity range within the main pipeline. These factors are then standardized and integrated to form modeling constraints. Simultaneously, the modeling input and output requirements are determined. The input requirements may include, but are not limited to, the actual valve core movement amplitude and direction angle, and the actual fluid flow velocity within the main pipeline. The output requirements may include, but are not limited to, the actual unbalanced force spatial distribution of the first valve core. In practice, a machine learning method that embeds physical laws into a neural network can be used for modeling. This machine learning method may include, but is not limited to, Physics-Informer Neural Networks (PHNs), which can add the aforementioned modeling constraints to the neural network during the modeling process. After modeling is complete, the actual valve core movement amplitude and direction angle, and the actual fluid flow velocity within the main pipeline are input into the valve core dynamic model to output the valve core force results. These force results refer to the magnitude and direction of the fluid pressure acting on the global surface of the valve core. Based on the above force results of the valve core, it is determined whether the valve core is in force balance on its own global surface, thereby predicting the duration of the valve core unbalanced force event of the first valve component.

[0080] As discussed earlier, during an unbalanced force event on the valve core, the deviation of the valve core's displacement will disturb the original fluid pressure field within the main pipeline, causing a decrease in fluid pressure along the length of the main pipeline. Please refer to [link to relevant documentation]. Figure 3Considering the inherent mechanical strength of the main pipeline, it can withstand pressure drops within itself. However, when the pressure drop exceeds the pipeline's own pressure tolerance threshold, structural damage such as rupture may occur. Therefore, fluid pressure changes (fluid pressure variations) are extracted from the aforementioned fluid pressure field to determine whether the pressure drop per unit distance along the pipeline's length exceeds a preset threshold. If so, an undesirable pressure drop is considered to have occurred within the main pipeline; otherwise, no undesirable pressure drop is considered to have occurred. When an undesirable pressure drop occurs, the valve core's amplitude and directional angle during the aforementioned unbalanced force event are designated as triggering factors. It is understood that when the first valve component is at the valve core's amplitude and directional angle corresponding to the triggering factors, an undesirable pressure drop is likely to occur in the main pipeline. Using these triggering factors as a basis for predicting whether an undesirable pressure drop will occur in the main pipeline facilitates the early planning of the second valve component's operation.

[0081] S200: Acquire the occurrence characteristics of triggering factors during the operation of the first valve component; based on the occurrence characteristics, predict the occurrence characteristics of undesirable pressure drops, thereby controlling the second valve component located in the secondary pipeline to enter the standby state.

[0082] Furthermore, in S200, the occurrence characteristics of triggering factors during the operation of the first valve component are obtained; based on the occurrence characteristics, the occurrence characteristics of undesirable pressure drops are predicted, thereby controlling the second valve component located in the secondary pipeline to enter a standby state, specifically:

[0083] Obtain the predetermined motion trajectory of the first valve component during its operation;

[0084] By comparing the predetermined action trajectory with the triggering factor, the action range within the predetermined action trajectory that matches the triggering factor is determined, and this range is used as the appearance characteristic of the triggering factor.

[0085] Input all action parameters within the action range into the valve core dynamic model to predict the time when the unwanted pressure drop occurs;

[0086] Based on the occurrence time and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is controlled to switch to the standby state before a preset time point; wherein, the standby state refers to the state in which the second valve component can respond to control commands in real time.

[0087] When a main pipeline performs fluid transmission tasks such as supplying water to external systems, a corresponding fluid transmission flow rate plan is set. This fluid transmission flow rate plan refers to the plan for the flow rate of fluid transmitted from the main pipeline over time. To achieve this plan, the first valve component within the main pipeline will change its valve core actuation amplitude and / or actuation direction angle over time according to the corresponding actuation plan. Consequently, the first valve component will form a predetermined actuation trajectory matching the aforementioned actuation plan during its operation. It is understood that the predetermined actuation trajectory may contain valve core actuation elements that are the same as or similar to the aforementioned triggering factors (e.g., the difference in valve core actuation amplitude and the difference in valve core actuation direction angle are within a preset range). If the predetermined actuation trajectory contains the same or similar valve core actuation elements, when the first valve component operates according to the predetermined actuation trajectory, as the first valve component approaches the actuation trajectory range containing the same or similar valve core actuation elements, the probability of the first valve component experiencing an unbalanced force event increases accordingly. In other words, the first valve component may experience an unbalanced force event at any time during this period, thus creating an undesirable pressure drop within the main pipeline.

[0088] Please see Figure 4 To predict potential unwanted pressure drops during the actual operation of the first valve component, a predetermined action trajectory of the first valve component is obtained and compared with the aforementioned triggering factors. The predetermined action trajectory is used to identify the range of valve core action amplitude and direction angle values ​​that are the same or similar to those corresponding to the triggering factors, thus serving as the occurrence characteristics of the triggering factors during the actual operation of the first valve component. All valve core action amplitude and direction angle data contained within the aforementioned action trajectory range are then fed back into the valve core dynamic model to output the occurrence time of the unwanted pressure drop in the main pipeline. It is understood that the main pipeline will experience fluctuations in its fluid output flow rate (i.e., fluid output flow rate imbalance) at the occurrence time of the unwanted pressure drop. In this case, adjustment by the first valve component alone cannot correct the flow rate fluctuation; the secondary pipeline and the second valve component must cooperate to correct the fluid output flow rate fluctuation. In actual operation, based on the occurrence time of the unwanted pressure drop and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is instructed to switch to a state capable of responding instantly to external control commands before a preset time point, facilitating the subsequent rapid response and correction of the flow rate fluctuation by the second valve component. The preset time point can be, but is not limited to, the time of occurrence or any time point before the time of occurrence; the second valve component can switch to standby state by clearing all the control commands it has loaded.

[0089] S300: Construct a flow dynamic model of the first valve component during the period of undesirable pressure drop formation to obtain the flow imbalance characteristics of the first valve component.

[0090] Furthermore, in S300, a flow dynamic model of the first valve component during the period of undesirable pressure drop formation is constructed to obtain the flow imbalance characteristics of the first valve component, specifically:

[0091] Based on the valve core jitter parameters and fluid input parameters of the first valve component during the formation of the undesirable pressure drop, a flow dynamic model of the first valve component is constructed.

[0092] A time-domain fluctuation analysis was performed on the downstream fluid flow rate of the first valve component output by the flow dynamic model to obtain the flow imbalance characteristics; among which, the flow imbalance characteristics include the time-domain distribution of the deviation between the downstream fluid flow rate of the first valve component and the expected fluid flow rate.

[0093] When an undesired pressure drop occurs in the main pipeline, the valve core of the first valve component will experience undesired vibration under the pressure drop change. This undesired vibration may include, but is not limited to, vibration displacement of the valve core in the radial and / or axial directions. Under the influence of this undesired vibration, the output fluid flow rate downstream of the first valve component in the main pipeline fluctuates. This fluctuation in output fluid flow rate is related to the valve core vibration of the first valve component and the original fluid flow rate input at the main pipeline input end. Therefore, based on the valve core vibration parameters of the first valve component during the formation of the undesired pressure drop (such as valve core vibration displacement, vibration direction, vibration frequency, etc.) and fluid input parameters (such as the original fluid flow rate input at the main pipeline input end), a flow dynamic model of the first valve component can be constructed using machine learning methods such as deep Q-neural networks. This model outputs the downstream fluid flow rate of the first valve component (i.e., the fluid flow rate corresponding to the output end of the main pipeline). By comparing the downstream fluid flow rate with the desired fluid flow rate and performing time-domain fluctuation analysis, the time-domain distribution of the deviation between the downstream fluid flow rate and the desired fluid flow rate is obtained, providing a precise reference for subsequent timely control of the second valve component. Please refer to [link to relevant documentation]. Figure 5 Downstream fluid flow may fluctuate in certain intervals of the time domain, while it is desirable for the fluid flow to remain stable throughout the entire time domain. By determining the time domain distribution of the deviation between the two, the reliability of the main pipeline flow balance compensation can be improved.

[0094] S400: Based on the characteristics of flow imbalance, deep reinforcement learning is performed on the action strategy of the second valve component to obtain the target action strategy of the second valve component, thereby achieving flow balance compensation for the main pipeline.

[0095] Furthermore, in S400, based on the flow imbalance characteristics, deep reinforcement learning is performed on the action strategy of the second valve component to obtain the target action strategy of the second valve component, thereby achieving flow balance compensation for the main pipeline. Specifically:

[0096] Determine the environmental state space and action space of the second valve component; determine the flow modulation target of the second valve component based on the flow imbalance characteristics;

[0097] Based on the environmental state space, action space, and flow modulation target, deep reinforcement learning is performed on the action strategy of the second valve component to obtain the target action strategy; wherein, the target action strategy includes the action amplitude and direction of the second valve component during the period of undesirable pressure drop formation;

[0098] According to the target action strategy, control commands are applied to the second valve component to achieve flow balance compensation between the second valve component and the main pipeline.

[0099] The secondary pipeline containing the second valve component serves as its corresponding environmental state space, and the valve core movement (amplitude and direction angle) of the second valve component itself serves as its corresponding action space. Based on the aforementioned flow imbalance characteristics, the fluid flow compensation value provided to the first valve component by the main pipeline to restore the desired fluid flow under the current flow imbalance state is determined, and this serves as the flow modulation target for the second valve component. Then, using a Markov decision chain, deep reinforcement learning of the action strategy for the second valve component is performed based on the environmental state space, action space, and flow modulation target to obtain the target action strategy. The aforementioned deep reinforcement learning using a Markov decision chain is a conventional technique in this field and will not be described in detail here. It is understood that the aforementioned target action strategy includes the amplitude and direction of the second valve component's movement during periods of undesirable pressure drop, and can adjust the movement of the second valve component from different dimensions.

[0100] Please see Figure 6 According to the target action strategy, control commands are applied to the second valve component to achieve flow balance compensation between the second valve component and the main pipeline. Specifically, two control command sequences can be generated based on the target action strategy, concerning the valve core movement amplitude and direction of independently adjusting the second valve component. These two control command sequences can be applied to the second valve component at the same time or at different times. Upon receiving the control command sequences, the second valve component performs an appropriate valve core movement, thereby adjusting the fluid flow compensation amount at the two output ends of the secondary pipeline to the upstream and downstream positions of the first valve component along the main pipeline, achieving flow balance compensation between the second valve component and the main pipeline, and restoring the output fluid flow of the main pipeline to the desired fluid flow.

[0101] Please see Figure 7 As shown, this invention provides a dynamic equilibrium control system based on deep reinforcement learning, which includes the following modules:

[0102] The prediction module is used to construct a dynamic model of the valve core of the first valve component located on the main pipeline, so as to predict the unbalanced force events of the valve core of the first valve component.

[0103] The trigger determination module is used to determine the triggering factors that cause an undesirable pressure drop in the main pipeline based on the unbalanced force event of the valve core.

[0104] The trigger identification module is used to acquire the occurrence characteristics of triggering factors during the operation of the first valve component;

[0105] The status control module is used to predict the occurrence characteristics of unwanted pressure drops based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter the standby state;

[0106] The flow imbalance identification module is used to construct a flow dynamic model of the first valve component during the period of undesirable pressure drop formation, thereby obtaining the flow imbalance characteristics of the first valve component;

[0107] The reinforcement learning module is used to perform deep reinforcement learning on the action strategy of the second valve component based on the characteristics of flow imbalance, so as to obtain the target action strategy of the second valve component.

[0108] The compensation module is used to achieve flow balance compensation for the main pipeline based on the target action strategy.

[0109] Furthermore, the prediction module is used to construct a dynamic model of the valve core of the first valve component located on the main pipeline, thereby predicting unbalanced force events on the valve core of the first valve component, specifically:

[0110] Based on the three-dimensional geometric parameters and action parameters of the valve core of the first valve component located in the main pipeline, as well as the fluid flow parameters of the main pipeline, the modeling constraints and modeling input and output requirements are determined, thereby constructing the valve core dynamic model of the first valve component.

[0111] Based on the valve core force results output by the valve core dynamic model, predict the valve core unbalanced force event of the first valve component;

[0112] The trigger determination module is used to determine the triggering factors for the formation of an undesirable pressure drop in the main pipeline based on the unbalanced force event of the valve core. Specifically:

[0113] Obtain the fluid pressure field of the valve core during the period when the valve core is subjected to unbalanced force, extract the fluid pressure change from the fluid pressure field, and use this to determine whether an undesirable pressure drop has formed in the main pipeline;

[0114] When an undesirable pressure drop occurs in the main pipeline, the valve core action during the period of valve core imbalance force event is calibrated as the triggering factor.

[0115] Furthermore, the trigger identification module is used to obtain the occurrence characteristics of triggering factors during the operation of the first valve component, specifically:

[0116] Obtain the predetermined motion trajectory of the first valve component during its operation;

[0117] By comparing the predetermined action trajectory with the triggering factor, the action range within the predetermined action trajectory that matches the triggering factor is determined, and this range is used as the appearance characteristic of the triggering factor.

[0118] The status control module is used to predict the occurrence characteristics of unwanted pressure drops based on the observed features, and thereby control the second valve component located in the secondary pipeline to enter a standby state, specifically:

[0119] Input all action parameters within the action range into the valve core dynamic model to predict the time when the unwanted pressure drop occurs;

[0120] Based on the occurrence time and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is controlled to switch to the standby state before a preset time point; wherein, the standby state refers to the state in which the second valve component can respond to control commands in real time.

[0121] Furthermore, the flow imbalance identification module is used to construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component, specifically:

[0122] Based on the valve core jitter parameters and fluid input parameters of the first valve component during the formation of the undesirable pressure drop, a flow dynamic model of the first valve component is constructed.

[0123] A time-domain fluctuation analysis was performed on the downstream fluid flow rate of the first valve component output by the flow dynamic model to obtain the flow imbalance characteristics; among which, the flow imbalance characteristics include the time-domain distribution of the deviation between the downstream fluid flow rate of the first valve component and the expected fluid flow rate;

[0124] The reinforcement learning module is used to perform deep reinforcement learning on the action strategy of the second valve component based on the characteristics of flow imbalance, so as to obtain the target action strategy of the second valve component, specifically:

[0125] Determine the environmental state space and action space of the second valve component; determine the flow modulation target of the second valve component based on the flow imbalance characteristics;

[0126] Based on the environmental state space, action space, and flow modulation target, deep reinforcement learning is performed on the action strategy of the second valve component to obtain the target action strategy; wherein, the target action strategy includes the action amplitude and direction of the second valve component during the period of undesirable pressure drop formation;

[0127] The compensation module is used to achieve flow balance compensation for the main pipeline based on the target action strategy, specifically:

[0128] According to the target action strategy, control commands are applied to the second valve component to achieve flow balance compensation between the second valve component and the main pipeline.

[0129] The operation and effect of the dynamic balance control system based on deep reinforcement learning of the present invention are consistent with the above-mentioned dynamic balance control method based on deep reinforcement learning, and the description of the dynamic balance control system based on deep reinforcement learning will not be repeated here.

[0130] Please see Figure 8 As shown, the present invention also provides a valve body, including a first valve component located in the main pipeline and a second valve component located in the secondary pipeline; the operation of the first valve component and the second valve component is realized based on the above-mentioned dynamic balance control method based on deep reinforcement learning.

[0131] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of a necessary general-purpose hardware platform, or by a combination of hardware and software. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a computer product. The present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0132] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Other embodiments may also be used. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic balancing control method based on deep reinforcement learning, characterized in that, The method includes the following steps: S100: Construct a valve core dynamic model of the first valve component located on the main pipeline to predict valve core imbalance force events of the first valve component; determine the triggering factors for the formation of undesirable pressure drop in the main pipeline based on the valve core imbalance force events; S200: Obtain the occurrence characteristics of the triggering factors during the operation of the first valve component; predict the occurrence characteristics of undesirable pressure drop based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter the standby state; S300: Construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component; S400: Based on the flow imbalance characteristics, deep reinforcement learning of the action strategy is performed on the second valve component to obtain the target action strategy of the second valve component, thereby achieving flow balance compensation for the main pipeline, specifically: Determine the environmental state space and action space of the second valve component; determine the flow modulation target of the second valve component based on the flow imbalance characteristics; Based on the environmental state space, the action space, and the flow modulation target, the second valve component undergoes deep reinforcement learning of its action strategy to obtain a target action strategy; wherein, the target action strategy includes the amplitude and direction of the action of the second valve component during the formation of the undesirable pressure drop; According to the target action strategy, a control command is applied to the second valve component to achieve flow balance compensation of the second valve component to the main pipeline.

2. The method according to claim 1, characterized in that, In S100, a valve core dynamic model of the first valve component located in the main pipeline is constructed to predict unbalanced force events on the valve core of the first valve component; based on the unbalanced force events on the valve core, the triggering factors for the formation of an undesirable pressure drop in the main pipeline are determined, specifically: Based on the three-dimensional geometric parameters and valve core action parameters of the first valve component located in the main pipeline, as well as the fluid flow parameters of the main pipeline, the modeling constraints and modeling input and output requirements are determined, thereby constructing the valve core dynamic model of the first valve component. Based on the valve core force results output by the valve core dynamic model, predict the valve core unbalanced force event of the first valve component; The fluid pressure field of the valve core during the period when the valve core is subjected to unbalanced force is obtained, and the fluid pressure change is extracted from the fluid pressure field to determine whether an undesirable pressure drop is formed in the main pipeline. When an undesirable pressure drop occurs in the main pipeline, the valve core action during the valve core unbalanced force event is designated as a triggering factor.

3. The method according to claim 1, characterized in that, In S200, the occurrence characteristics of the triggering factors during the operation of the first valve component are obtained; based on the occurrence characteristics, the occurrence characteristics of undesirable pressure drops are predicted, thereby controlling the second valve component located in the secondary pipeline to enter a standby state, specifically: Obtain the predetermined motion trajectory of the first valve component during its operation; By comparing the predetermined action trajectory and the triggering factor, the action range within the predetermined action trajectory that matches the triggering factor is determined, and this range is used as the occurrence feature of the triggering factor. All action parameters within the action range are input into the valve core dynamic model to predict the time when the unwanted pressure drop occurs; Based on the occurrence time and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is controlled to switch to a standby state before a preset time point; wherein, the standby state refers to the state in which the second valve component can respond to control commands in real time.

4. The method according to claim 1, characterized in that, In S300, a flow dynamic model of the first valve component during the formation of the undesirable pressure drop is constructed to obtain the flow imbalance characteristics of the first valve component, specifically: Based on the valve core jitter parameters and fluid input parameters of the first valve component during the formation of the undesirable pressure drop, a flow dynamic model of the first valve component is constructed. A time-domain fluctuation analysis is performed on the downstream fluid flow rate of the first valve component output by the flow dynamic model to obtain the flow imbalance characteristics; wherein, the flow imbalance characteristics include the time-domain distribution of the deviation between the downstream fluid flow rate of the first valve component and the desired fluid flow rate.

5. A dynamic equilibrium control system based on deep reinforcement learning, characterized in that, The system includes: The prediction module is used to construct a dynamic model of the valve core of the first valve component located on the main pipeline, so as to predict the unbalanced force event of the valve core of the first valve component. The trigger determination module is used to determine the triggering factors for the formation of an undesirable pressure drop in the main pipeline based on the unbalanced force event of the valve core. A trigger identification module is used to acquire the occurrence characteristics of the triggering factors during the operation of the first valve component; The status control module is used to predict the occurrence characteristics of undesirable pressure drops based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter the standby state. A flow imbalance identification module is used to construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component. The reinforcement learning module is used to perform deep reinforcement learning on the action strategy of the second valve component based on the flow imbalance characteristics, so as to obtain the target action strategy of the second valve component, specifically: Determine the environmental state space and action space of the second valve component; determine the flow modulation target of the second valve component based on the flow imbalance characteristics; Based on the environmental state space, the action space, and the flow modulation target, the second valve component undergoes deep reinforcement learning of its action strategy to obtain a target action strategy; wherein, the target action strategy includes the amplitude and direction of the action of the second valve component during the formation of the undesirable pressure drop; The compensation module is used to perform flow balance compensation on the main pipeline according to the target action strategy, specifically as follows: According to the target action strategy, a control command is applied to the second valve component to achieve flow balance compensation of the second valve component to the main pipeline.

6. The system according to claim 5, characterized in that, The prediction module is used to construct a valve core dynamic model of the first valve component located on the main pipeline, thereby predicting unbalanced force events of the valve core of the first valve component, specifically: Based on the three-dimensional geometric parameters and valve core action parameters of the first valve component located in the main pipeline, as well as the fluid flow parameters of the main pipeline, the modeling constraints and modeling input and output requirements are determined, thereby constructing the valve core dynamic model of the first valve component. Based on the valve core force results output by the valve core dynamic model, predict the valve core unbalanced force event of the first valve component; The trigger determination module is used to determine the triggering factors for the formation of an undesirable pressure drop in the main pipeline based on the unbalanced force event of the valve core, specifically: The fluid pressure field of the valve core during the period when the valve core is subjected to unbalanced force is obtained, and the fluid pressure change is extracted from the fluid pressure field to determine whether an undesirable pressure drop is formed in the main pipeline. When an undesirable pressure drop occurs in the main pipeline, the valve core action during the valve core unbalanced force event is designated as a triggering factor.

7. The system according to claim 5, characterized in that, The trigger identification module is used to obtain the occurrence characteristics of the triggering factors during the operation of the first valve component, specifically: Obtain the predetermined motion trajectory of the first valve component during its operation; By comparing the predetermined action trajectory and the triggering factor, the action range within the predetermined action trajectory that matches the triggering factor is determined, and this range is used as the occurrence feature of the triggering factor. The state control module is used to predict the occurrence characteristics of unwanted pressure drops based on the occurrence characteristics, thereby controlling the second valve component located in the secondary pipeline to enter a standby state, specifically: All action parameters within the action range are input into the valve core dynamic model to predict the time when the unwanted pressure drop occurs; Based on the occurrence time and the real-time operating status of the second valve component located in the secondary pipeline, the second valve component is controlled to switch to a standby state before a preset time point; wherein, the standby state refers to the state in which the second valve component can respond to control commands in real time.

8. The system according to claim 5, characterized in that, The flow imbalance identification module is used to construct a flow dynamic model of the first valve component during the formation of the undesirable pressure drop, thereby obtaining the flow imbalance characteristics of the first valve component, specifically: Based on the valve core jitter parameters and fluid input parameters of the first valve component during the formation of the undesirable pressure drop, a flow dynamic model of the first valve component is constructed. A time-domain fluctuation analysis is performed on the downstream fluid flow rate of the first valve component output by the flow dynamic model to obtain the flow imbalance characteristics; wherein, the flow imbalance characteristics include the time-domain distribution of the deviation between the downstream fluid flow rate of the first valve component and the desired fluid flow rate.

9. A valve body, characterized in that, It includes a first valve component located on the main pipeline and a second valve component located on the secondary pipeline; the operation of the first valve component and the second valve component is implemented based on the dynamic balance control method based on deep reinforcement learning as described in any one of claims 1-4.

Images