An intelligent electric ball valve control system based on artificial intelligence
Patent Information
- Application Number
- CN202510487936.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-18
- Publication Date
- 2025-08-26
- Estimated Expiration
- 2045-04-18
AI Technical Summary
The existing electric ball valve control system has poor adaptability under complex and nonlinear operating conditions, which is difficult to achieve accurate real-time adjustment, and lacks adaptive learning ability, resulting in insufficient control accuracy and slow response.
A structured reinforcement learning algorithm and strategy optimization mechanism are introduced, and intelligent perception, adaptive optimization and high-precision execution of valve control are achieved by building a three-dimensional instruction-DDPG algorithm and strategy stability enhancement mechanism.
It improves the control accuracy and response speed of the system in a variety of environments, enhances the robustness and adaptability of the system, and is suitable for high-demand and highly dynamic intelligent fluid control systems.
Smart Images
Figure CN120292301A8_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of valve control, and in particular to an intelligent electric ball valve control system based on artificial intelligence. Background Art
[0002] With the development of modern agriculture, the agricultural irrigation method is gradually changing from traditional manual control to the direction of intelligence and automation. As an automatic valve driven by a motor, the electric ball valve has been widely used in the agricultural irrigation system. However, there are still many deficiencies in the intelligent level of the existing electric ball valve control system: First of all, the existing electric ball valve control systems usually adopt algorithms based on classical control theories (such as PID control). Although these algorithms perform well in some stable systems, they have poor adaptability to complex and non-linear working conditions. In the face of dynamic environmental changes, it is difficult for these algorithms to achieve accurate real-time adjustment. Especially when the system faces the interaction of multiple input variables, the control effect is easily disturbed, resulting in a decrease in control accuracy. Secondly, traditional systems often make decisions based on fixed sensor data and rely on simple data filtering and anomaly detection mechanisms. This data processing method fails to fully explore the potential laws in the sensor data and lacks the fusion and analysis of multi-source and multi-dimensional data, resulting in the system being unable to efficiently respond to complex working condition changes. For example, when the system faces sudden pressure fluctuations, flow fluctuations or drastic temperature changes, it cannot make adjustments quickly, and may experience control lag or misoperation, affecting the execution accuracy of the valve. Finally, most of the existing systems adopt control based on priori-set rules or fixed models and lack the ability of adaptive learning. It is difficult for the system to automatically learn and adjust its control strategy, and it cannot ensure the best performance under different working conditions. Summary of the Invention
[0003] The present invention aims to overcome the problems of slow response, poor policy adaptability, and insufficient control accuracy in the existing electric ball valve control system under complex environments, and provides an intelligent electric ball valve control system based on artificial intelligence. By introducing a structured reinforcement learning algorithm and a policy optimization mechanism, this system realizes intelligent perception, adaptive optimization, and high-precision execution of the valve control process, significantly improving the robustness and control efficiency of the system, and is particularly suitable for application scenarios such as agricultural irrigation under variable environments. Specifically, first, based on the traditional Deep Deterministic Policy Gradient (DDPG) algorithm, the present invention introduces three types of operators, namely Frequency, Amplitude, and Direction, to structurally improve the original DDPG actor network and constructs a three-dimensional instruction-DDPG algorithm. By decomposing the valve control action into three physical quantity dimensions of frequency, amplitude, and direction, this algorithm realizes the structured modeling and fine control of the action space, breaking through the problem of limited learning ability of traditional DDPG for continuous action spaces and effectively improving the generalization ability and control stability in multivariable complex systems. Second, to solve the problem of unstable training caused by policy drift during the policy gradient update process, the present invention further proposes a policy stability enhancement mechanism. This mechanism combines a skill neuron protection mechanism and an unbalanced experience replay mechanism to ensure the policy stability and convergence of the F-A-D algorithm during long-term training. In summary, by constructing a three-dimensional structured DDPG control framework and introducing a multi-level policy stability optimization mechanism, the present invention has achieved significant improvements in aspects such as valve action control accuracy, system response speed, and policy adaptation ability, is suitable for high-demand and high-dynamic intelligent fluid control systems, and has broad application prospects and promotion value.
[0004] The present invention provides an intelligent electric ball valve control system based on artificial intelligence, which includes a sensor monitoring module, a data preprocessing module, a ball valve control module, a policy optimization module, a valve execution module, and an electric ball valve.
[0005] The sensor monitoring module obtains the original sensor data by collecting the real-time parameter information of flow rate, pressure, temperature, and liquid level position through pressure sensors, flow sensors, temperature sensors, and liquid level sensors.
[0006] The data preprocessing module filters, normalizes, and performs anomaly detection on the original sensor data to generate cleaned sensor data.
[0007] The ball valve control module constructs a DDPG algorithm. By introducing three types of operators, namely frequency, amplitude, and direction, it structurally improves the actor network of the DDPG algorithm and constructs a three-dimensional instruction-DDPG algorithm. Based on the cleaned sensor data, it uses the three-dimensional instruction-DDPG algorithm to generate accurate valve control instructions.
[0008] The policy optimization module jointly constructs a policy stability enhancement mechanism by adopting a skill neuron protection mechanism and an unbalanced experience replay mechanism; through the policy stability enhancement mechanism, it optimizes the policy gradient update in the three-dimensional instruction - DDPG algorithm.
[0009] The valve execution module controls the electric ball valve according to the precise valve control instruction.
[0010] Further, in the ball valve control module, the process of generating the precise valve control instruction specifically includes the following steps:
[0011] Step S1: Based on the cleaning sensor data, set the initial environmental state.
[0012] Step S2: Initialize the actor network and the critic network of the three-dimensional instruction - DDPG algorithm; introduce the frequency operator, amplitude operator, and direction operator to optimize the structural composition of the actor network, and construct the F - A - D actor network; according to the initial environmental state, generate the valve control instruction through the F - A - D actor network; the F - A - D actor network structurally splits the output of the actor network to separately control the frequency, amplitude, and direction of the valve control instruction.
[0013] Step S3: Evaluate the effect of the valve control instruction through the critic network.
[0014] Step S4: Transmit the valve control instruction to the valve execution module, execute the valve control instruction, and generate the updated environmental state.
[0015] Step S5: Define the target state and the reward function, and calculate the immediate reward through the reward function according to the target state and the updated environmental state.
[0016] Step S6: Based on the immediate reward, according to the Bellman equation, update the critic network to guide the update strategy of the F - A - D actor network; based on the updated critic network, update the F - A - D actor network through policy gradient to improve the valve control instruction.
[0017] Step S7: Iteratively loop through Step S3 - Step S6 to train the F - A - D actor network and generate the precise valve control instruction.
[0018] Further, in the policy optimization module, the process of optimizing the policy gradient in the three-dimensional instruction - DDPG algorithm specifically includes the following steps:
[0019] Step B1: Record the neuron activity during the training process of the F - A - D actor network, calculate the gradient contribution of each neuron, and obtain the skill neuron score; construct the old experience pool and the new experience pool.
[0020] Step B2: According to the skill neuron scores, introduce a gradient masking factor, and apply the gradient masking factor during the backpropagation of the policy gradient to inhibit the update of skill neurons and stabilize the propagation path of the policy gradient;
[0021] Step B3: Construct an angle-experience pool loss function, and dynamically select data from the old experience pool or the new experience pool to participate in training through the angle-experience pool loss function, alleviating catastrophic forgetting and stabilizing the direction of the policy gradient.
[0022] Further, Step B3 specifically includes: introducing an unbalanced gated replay mechanism and gradient direction consistency, constructing an angle-experience pool loss function, and dynamically selecting data from the old experience pool or the new experience pool to participate in training through the angle-experience pool loss function, alleviating catastrophic forgetting and stabilizing the direction of the policy gradient; the unbalanced gated replay mechanism, which dynamically adjusts the proportion of old and new experiences in the angle-experience pool loss function by comparing the direction relationship between the current policy gradient and the old task policy gradient; specifically, in each training iteration, first calculate the policy gradients obtained based on the new experience pool and the old experience pool respectively, and judge whether the current update direction is consistent with the past according to the angle between the two:
[0023] If the angle is small (the directions are similar), it indicates that the current policy update will not damage the performance of the old task. At this time, reduce the participation degree of the old experience to adapt to the new task faster;
[0024] If the angle is large (the directions conflict), then increase the participation weight of the old experience to maintain the continuity of the policy gradient direction and alleviate catastrophic forgetting.
[0025] Adopting the above solution, the beneficial effects obtained by the present invention are as follows:
[0026] The present invention realizes the structured generation and high-precision adjustment of the control instructions for the electric ball valve by constructing a three-dimensional instruction-DDPG algorithm based on three types of operators: frequency, amplitude, and direction; compared with the traditional algorithm that only performs single-dimensional modeling on the control quantity, the F-A-D structure proposed by the present invention explicitly disassembles the action output space into three types of adjustable parameters, making the control instructions more physically interpretable and responsive flexibly, and significantly improving the dynamic response performance and control stability of the valve under different working conditions; this structure is particularly suitable for the complex adjustment requirements caused by multi-dimensional disturbances such as flow rate, pressure, and liquid level in the agricultural irrigation scenario, ensuring the precise adaptation of crops to the water supply and improving the overall irrigation efficiency;
[0027] The strategy optimization module introduced in the present invention realizes the dynamic regulation and protection of the policy update path of the actor network by constructing a policy stability enhancement mechanism. On the one hand, the skill neuron protection mechanism can identify the neurons that play a key role in the control policy and limit the drastic fluctuation of their gradients, effectively avoiding policy degradation caused by key path perturbations during training. On the other hand, through the combined design of the unbalanced experience replay mechanism and the angle-experience pool loss function, the policy gradient directions between old experiences and new experiences are balanced, alleviating the catastrophic forgetting problem in reinforcement learning training. The design of this optimization module closely serves the control objective of the electric ball valve of the present invention, enhancing the robustness and continuous optimization ability of the system during multi-task and long-term operation.
[0028] Through the above innovative design, the present invention realizes the precise control of the electric ball valve based on artificial intelligence, improves the adaptability of the system in an environment with dynamic and multi-source information input, solves the problems of slow response and error accumulation of traditional systems under multi-dimensional coupling interference, and enhances the intelligent level of water resource allocation in the agricultural irrigation process. The overall system has stronger generalization ability, stability and control accuracy, can operate reliably for a long time in complex actual irrigation scenarios, and provides strong support for efficient irrigation and water resource conservation in modern agriculture. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] Figure 1 It is a schematic diagram of the modules of an intelligent electric ball valve control system based on artificial intelligence proposed by the present invention;
[0030] Figure 2 It is a comparison chart of the training effects of the F-A-D actor network and the actor network proposed in Embodiments II and III. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0031] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.
[0032] Embodiment 1, according to Figure 1 , the present invention provides an intelligent electric ball valve control system based on artificial intelligence, which includes a sensor monitoring module, a data preprocessing module, a ball valve control module, a strategy optimization module, a valve execution module and an electric ball valve;
[0033] The sensor monitoring module collects real-time parameter information of flow rate, pressure, temperature, and liquid level position through a pressure sensor, a flow sensor, a temperature sensor, and a liquid level sensor to obtain raw sensor data; the sensor monitoring module transmits the raw sensor data to the data preprocessing module;
[0034] The data preprocessing module filters, normalizes, and performs anomaly detection on the raw sensor data to generate cleaned sensor data; the data preprocessing module transmits the cleaned sensor data to the ball valve control module;
[0035] The ball valve control module constructs a DDPG algorithm, and through the introduction of three types of operators: frequency, amplitude, and direction, structurally improves the actor network of the DDPG algorithm to construct a three-dimensional instruction - DDPG algorithm; based on the cleaned sensor data, uses the three-dimensional instruction - DDPG algorithm to generate precise valve control instructions; the ball valve control module is connected to the policy optimization module and the valve execution module;
[0036] The policy optimization module jointly constructs a policy stability enhancement mechanism using a skill neuron protection mechanism and an unbalanced experience replay mechanism; through the policy stability enhancement mechanism, optimizes the policy gradient update in the three-dimensional instruction - DDPG algorithm;
[0037] The valve execution module controls the opening and closing state, opening angle, opening and closing speed, opening and closing direction, multi-stage adjustment position, flexible opening and closing process, and execution timing of the electric ball valve according to the precise valve control instructions.
[0038] Embodiment 2, according to Figure 2 , this embodiment is based on Embodiment 1. In this embodiment, the process of the ball valve control module generating precise valve control instructions specifically includes the following steps:
[0039] Step S1: Based on the cleaned sensor data, set the initial environmental state;
[0040] Step S2: Initialize the actor network and the critic network of the three-dimensional instruction - DDPG algorithm; introduce frequency operator, amplitude operator, and direction operator to optimize the structural composition of the actor network to construct an F - A - D actor network; according to the initial environmental state, generate valve control instructions through the F - A - D actor network; the F - A - D actor network structurally splits the output of the actor network to respectively control the frequency, amplitude, and direction of the valve control instructions. The used formula is as follows:
[0041] ;
[0042] Wherein, represents the time step, represents the initial environmental state, represents the disturbance term, represents at The environmental state at a moment; Indicates at The valve control instruction at a moment, Indicates the amplitude operator, Indicates the feedforward term, Indicates the direction operator, Indicates the frequency operator;
[0043] Step S3: Evaluate the effect of the valve control instruction through the critic network;
[0044] Step S4: Transmit the valve control instruction to the valve execution module, execute the valve control instruction, and generate an updated environmental state;
[0045] Step S5: Define the target state and the reward function, and calculate the immediate reward through the reward function according to the target state and the updated environmental state;
[0046] Step S6: Based on the immediate reward, according to the Bellman equation, update the critic network to guide the update strategy of the F-A-D actor network; based on the updated critic network, update the F-A-D actor network through policy gradient to improve the valve control instruction;
[0047] Step S7: Iteratively loop through steps S3 - S6 to train the F-A-D actor network and generate accurate valve control instructions.
[0048] Example 3, according to Figure 2 , this example is based on Example 1. In this example, the process of the ball valve control module generating accurate valve control instructions specifically includes the following steps:
[0049] Step R1: Based on cleaning the sensor data, set the initial environmental state;
[0050] Step R2: Initialize the actor network and the critic network of the DDPG algorithm; generate a valve control instruction through the actor network according to the initial environmental state;
[0051] Step R3: Evaluate the effect of the valve control instruction through the critic network;
[0052] Step R4: Transmit the valve control instruction to the valve execution module, execute the valve control instruction, and generate an updated environmental state;
[0053] Step R5: Define the target state and the reward function, and calculate the immediate reward through the reward function according to the target state and the updated environmental state;
[0054] Step R6: Based on the immediate reward, update the critic network according to the Bellman equation to guide the update strategy of the actor network; based on the updated critic network, update the actor network through policy gradient to improve the valve control instruction.
[0055] Step R7: Iteratively loop through Steps R3 - R6 to train the actor network and generate accurate valve control instructions.
[0056] Example 4: This example is based on Example 2. In this example, the policy optimization module optimizes the process of policy gradient in the three-dimensional instruction - DDPG algorithm, which specifically includes the following steps:
[0057] Step B1: Record the neuron activity during the training process of the F - A - D actor network, calculate the gradient contribution of each neuron to obtain the skill neuron score; construct an old experience pool and a new experience pool.
[0058] Step B2: According to the skill neuron score, introduce a gradient masking factor, and apply the gradient masking factor during the backpropagation of the policy gradient to inhibit the update of skill neurons and stabilize the propagation path of the policy gradient. The formula used is as follows:
[0059] Definition of the gradient masking factor:
[0060] ;
[0061] where, represents the neuron index, represents the th gradient masking factor applied by the th neuron during backpropagation; represents the th neuron in the F - A - D actor network; represents the stability score of the th neuron,
[0062] Step B3: Construct an angle - experience pool loss function, and dynamically select data from the old experience pool or the new experience pool to participate in training through the angle - experience pool loss function to alleviate catastrophic forgetting and stabilize the direction of the policy gradient.
[0063] Example 5: This example is based on Example 2. In this example, the policy optimization module optimizes the process of policy gradient in the three - dimensional instruction - DDPG algorithm, which specifically includes the following steps:
[0064] Step E1: Record the neuron activity during the training process of the F - A - D actor network, calculate the gradient contribution of each neuron to obtain the skill neuron score; construct an old experience pool and a new experience pool.
[0065] Step E2: According to the skill neuron scores, introduce a gradient masking factor, and apply the gradient masking factor during the backpropagation of the policy gradient to suppress the large updates of skill neurons and stabilize the propagation path of the policy gradient;
[0066] Step E3: Construct an experience pool loss function, and dynamically select data from the old experience pool or the new experience pool to participate in training through the experience pool loss function, alleviate catastrophic forgetting, and stabilize the direction of the policy gradient.
[0067] Example Six: This example is based on Example Four. In this example, step B3 specifically includes: introducing an imbalance gated replay mechanism and gradient direction consistency, constructing an angle-experience pool loss function, and dynamically selecting data from the old experience pool or the new experience pool to participate in training through the angle-experience pool loss function, alleviating catastrophic forgetting, and stabilizing the direction of the policy gradient; the imbalance gated replay mechanism, which dynamically adjusts the proportion of old experience and new experience in the angle-experience pool loss function by comparing the direction relationship between the current policy gradient and the old task policy gradient; specifically, in each training iteration, first calculate the policy gradients obtained based on the new experience pool and the old experience pool respectively, and judge whether the current update direction is consistent with the past according to the angle between them:
[0068] If the angle is small (the directions are similar), it means that the current policy update will not damage the performance of the old task. At this time, reduce the participation degree of the old experience so as to adapt to the new task faster;
[0069] If the angle is large (the directions conflict), then increase the participation weight of the old experience to maintain the continuity of the policy gradient direction, alleviate catastrophic forgetting. The formula used is as follows:
[0070] The formula for the angle-experience pool loss function is as follows:
[0071] ;
[0072] where, represents the angle between the policy gradients, represents the policy gradient of the old experience pool, represents the policy gradient of the new experience pool, represents and the dot product of, represents and the product of the magnitudes of, represents the arccosine function;
[0073] ;
[0074] where, represents the angle-experience pool loss function, Represents the angular gating function, Represents the current environmental state, Represents the action taken in the state Below, Represents the immediate reward, Represents the next state, Represents the old experience pool, Represents the new experience pool, Represents the loss term of the old experience pool, Represents the loss term of the new experience pool.
[0075] The above describes the present invention and its implementation manners. Such description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention, and the actual structure is not limited thereto. In general, if those of ordinary skill in the art are inspired by it and, without departing from the gist of the present invention, design similar structural manners and embodiments to this technical solution without creative efforts, they shall fall within the protection scope of the present invention.
Claims
1. An intelligent electric ball valve control system based on artificial intelligence, the system comprising: Data preprocessing module and electric ball valve, the data preprocessing module generates cleaned sensor data; characterized in that: the system further includes: a ball valve control module, a policy optimization module, and a valve execution module; The ball valve control module constructs a three-dimensional instruction - DDPG algorithm, and based on the cleaned sensor data, uses the three-dimensional instruction - DDPG algorithm to generate precise valve control instructions; The policy optimization module constructs a policy stability enhancement mechanism; through the policy stability enhancement mechanism, optimizes the policy gradient in the three-dimensional instruction - DDPG algorithm; The valve execution module drives and controls the electric ball valve according to the precise valve control instructions.
2. The intelligent electric ball valve control system based on artificial intelligence according to claim 1, wherein: The process of the ball valve control module constructing the three-dimensional instruction - DDPG algorithm specifically includes: constructing the DDPG algorithm; through the introduction of three types of operators, namely frequency, amplitude, and direction, structurally improving the DDPG algorithm, thereby constructing the three-dimensional instruction - DDPG algorithm.
3. The intelligent electric ball valve control system based on artificial intelligence according to claim 1, characterized in that: The policy stability enhancement mechanism is jointly constructed by adopting a skill neuron protection mechanism and an unbalanced experience replay mechanism.
4. An intelligent electric ball valve control system based on artificial intelligence according to claim 2, characterized in that: The process of the ball valve control module generating precise valve control instructions specifically includes the following steps: Step S1: Based on the cleaned sensor data, set the initial state of the environment; Step S2: Initialize the actor network and critic network of the three-dimensional instruction - DDPG algorithm; introduce frequency operator, amplitude operator, and direction operator to optimize the structural composition of the actor network, and construct an F - A - D actor network; according to the initial state of the environment, generate valve control instructions through the F - A - D actor network; Step S3: Evaluate the valve control instructions through the critic network; Step S4: Transmit the valve control instructions to the valve execution module, execute the valve control instructions, and generate an updated environment state; Step S5: Define the target state and the reward function, and according to the target state and the updated environment state, calculate the immediate reward through the reward function; Step S6: Based on the immediate reward, according to the Bellman equation, update the critic network, and then update the F - A - D actor network through the policy gradient to improve the valve control instructions; Step S7: Iteratively loop through Step S3 - Step S6 to train the F - A - D actor network and generate precise valve control instructions.
5. The intelligent electric ball valve control system based on artificial intelligence according to claim 4, characterized in that: The F - A - D actor network structurally splits the output of the actor network and separately controls the frequency, amplitude, and direction of the valve control instructions.
6. The intelligent electric ball valve control system based on artificial intelligence according to claim 4, wherein: The process of the policy optimization module optimizing the policy gradient in the three-dimensional instruction - DDPG algorithm specifically includes the following steps: Step B1: Record the neuron activity during the training process of the F - A - D actor network, calculate the gradient contribution of each neuron, and obtain the skill neuron score; construct an old experience pool and a new experience pool; Step B2: According to the skill neuron score, introduce a gradient masking factor, and apply the gradient masking factor during the backpropagation of the policy gradient to suppress the update of the skill neurons and stabilize the propagation path of the policy gradient; Step B3: Construct an angle - experience pool loss function, and dynamically select the data in the old experience pool or the new experience pool to participate in the training through the angle - experience pool loss function to alleviate catastrophic forgetting and stabilize the direction of the policy gradient.