Multi-pump zero-displacement hydraulic agile control system and control method thereof

Abstract

The invention provides a multi-pump zero-displacement hydraulic agile control system and a control method thereof. The system is composed of a hydraulic system, a mechanical system and an electric control system. According to the control method, a multi-pump zero-displacement switching mechanism and a deep reinforcement learning strategy based on an Actor-Critic architecture are fused, the effective oil supply amount is adjusted by dynamically starting and stopping part of hydraulic pumps, a servo motor rotating speed instruction is output in real time in combination with a neural network, and the strong coupling dynamic relation between sliding block displacement and the angle of a swing rod is coordinated. The system switches part of the pumps to a bypass state when approaching balance so as to reduce energy consumption, and starts all the pumps to guarantee recovery capacity when greatly deviating. According to the technical scheme, on the premise of not depending on an accurate mathematical model, agile, stable and energy-saving control over the inverted pendulum type under-actuated system can be achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hydraulic automatic control, and particularly relates to a multi-pump zero-displacement hydraulic agile control system and a control method thereof. BACKGROUND

[0002] Under the macro background of new energy technology accelerating the penetration into the field of heavy equipment, the hydraulic drive system continues to be the core power unit of engineering machinery, special vehicles and high-end manufacturing equipment due to its high power density and strong load adaptability. However, the inherent nonlinear characteristics of the traditional hydraulic system, including the dynamic response hysteresis and flow mismatch problems caused by internal leakage, oil compressibility and element friction, seriously restrict its control performance in high-precision and high-dynamic application scenarios. Especially for the system using a constant-displacement pump, it is difficult to maintain a strict linear relationship between the output flow and the input speed, resulting in a significant decrease in the position and speed control accuracy of the actuator. To address this challenge, the industry has long relied on two main technical paths: one is to build a nonlinear controller based on an accurate mathematical model, such as adaptive sliding mode control, robust control or high-order PID structure, to offset system uncertainties through an online compensation mechanism; the other is to use a multi-pump parallel architecture to realize flow redundancy and dynamic allocation at the physical level to enhance the system's response to transient load changes. The above methods can effectively improve the system stability and tracking accuracy under certain working conditions, especially in the full-drive system (i.e., the control input degree of freedom is equal to the controlled output degree of freedom), which has formed a relatively mature application paradigm.

[0003] However, as high-end equipment puts forward higher requirements for agility, energy efficiency ratio and autonomous adaptation, a more complex control object, underactuated hydraulic system, gradually becomes the focus of frontier research. Typical as inverted pendulum structure, its control degree of freedom is less than the number of system generalized coordinates, and it belongs to non-holonomic constraint system in essence, which has strong nonlinearity, internal instability and fast dynamic response demand. Such systems exist widely in robot balance walking, spacecraft attitude adjustment and precision industrial platform, etc. The real-time performance, robustness and energy efficiency of the control algorithm are extremely tested. Although the existing hydraulic control strategy performs well in the full-drive framework, it exposes deep structural defects when facing the coupling dynamics of underactuated systems. The reason is that the traditional method generally assumes that the system has sufficient control input to directly drive all state variables, while the underactuated system lacks the ability to directly control the key degree of freedom, and must achieve stable control through indirect excitation and energy shaping. Under this premise, simply relying on model compensation or multiple pump flow superposition not only makes it difficult to accurately coordinate the strong coupling relationship between slider displacement and pendulum angle, but also easily causes system oscillation or energy waste due to excessive intervention of control command. Further, when introducing neural networks and other data-driven methods to reduce model dependence, the training process often ignores the physical constraints specific to hydraulic systems (such as flow conservation, pressure boundary, power threshold), resulting in good convergence of the strategy in the simulation environment, but performance degradation in actual deployment due to the dynamic characteristics of the oil elastic modulus changing with temperature and pressure, and even triggering the safety protection mechanism.

[0004] It is particularly critical that existing technologies often fall into the performance trade-off dilemma of "this or that" in the process of pursuing control accuracy or response speed: greatly increasing pump output flow to improve the speed of pendulum rod standing will inevitably exacerbate the slider overshoot and system energy consumption; on the contrary, if the power is strictly limited to ensure energy efficiency, it is difficult to complete the agile recovery from a large angle to a vertical balance within a limited time. This contradiction is due to the fact that the traditional control architecture fails to optimize the zero-displacement switching capability of the multi-pump system and the energy transmission mechanism of the underactuated system. Specifically, although the multi-pump parallel system has the potential for flow regulation, its electromagnetic valve switching logic is usually designed based on pre-set working conditions, lacking the ability to perceive the dynamic mapping relationship between the current energy state of the system and the target equilibrium point; while the learning controller can explore complex strategies, it rarely combines it with the zero-displacement mode in the hydraulic circuit (i.e. the pump is in unloading or bypass state, not supplying oil to the main circuit), thus missing the key window of using "non-working action" to achieve fine energy regulation. Therefore, in the context of underactuated, strong nonlinear and high energy efficiency hydraulic agile control, how to build a control method that can fully utilize the structural redundancy and zero-displacement switching characteristics of the multi-pump system, and accurately coordinate the indirect control degree of freedom and system energy flow through intelligent learning mechanism, has become the core of breaking through the bottleneck of existing technologies.

[0005] Therefore, how to integrate a multi-pump zero-displacement switching mechanism with a multi-objective optimization strategy based on reinforcement learning to achieve high-precision, low-energy-consumption, and robust agile balance control of underactuated hydraulic systems under a wide range of initial disturbances, while ensuring the safe operation of the system, has become a key challenge and an urgent technical problem for those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to propose a multi-pump zero-displacement hydraulic agile control system and control method, aiming to solve the technical problem of underactuated hydraulic systems being unable to achieve high-precision, low-energy-consumption, and robust balance control under a wide range of initial disturbances.

[0007] To achieve the above objectives, this invention proposes a multi-pump zero-displacement hydraulic agile control system, comprising a hydraulic system, a mechanical system, and an electrical control system; The hydraulic system includes at least two hydraulic pumps connected in parallel. Each hydraulic pump is driven by an independent servo motor. The outlet of each hydraulic pump is connected to the main working circuit via an electromagnetic directional valve. The main working circuit is connected to a double-acting hydraulic cylinder. The piston rod of the hydraulic cylinder is rigidly connected to the slider. The mechanical system includes a horizontal guide rail, a slider, a pendulum rod, and a pendulum. The slider is placed on the horizontal guide rail, one end of the pendulum rod is movably connected to the top of the slider, and the other end is fixed to the pendulum, forming an underdriven inverted pendulum mechanism. The electronic control system includes a motion controller, multiple servo drives, solenoid valve drive modules, and various types of sensors. The sensors include displacement and acceleration sensors for detecting slider displacement and acceleration, angle sensors for detecting rocker arm angle, pressure sensors for detecting pressure in the main working circuit, and level and temperature sensors for monitoring the oil tank status. The motion controller generates speed commands for each servo motor based on real-time acquired status information and controls the energization status of the solenoid directional valve to achieve dynamic start / stop and flow distribution of the hydraulic pump.

[0008] Furthermore, when the electromagnetic directional valve is de-energized, it causes the corresponding hydraulic pump to supply oil to the main working circuit. When energized, it causes the hydraulic pump to output oil and return it to the oil tank through the bypass pipeline, thus entering a zero-displacement idling state.

[0009] Furthermore, the main working circuit is equipped with a one-way oil replenishment valve. One end of the one-way oil replenishment valve is connected to the oil tank, and the other end is connected to the low-pressure side of the main working circuit. It is used to automatically replenish oil when the pressure on the low-pressure side of the system is lower than the set threshold in order to maintain the minimum working pressure of the closed circuit.

[0010] Furthermore, the hydraulic pump has an external drain port, and the leaked oil flows through pipelines and valve blocks into the oil tank.

[0011] Furthermore, the main working circuit is equipped with an oil cooling system; the oil outlet of the oil tank is equipped with a filter.

[0012] This invention also proposes a control method for a multi-pump zero-displacement hydraulic agile control system, comprising the following steps: S1: Construct a high-fidelity digital simulation environment that includes mechanical and hydraulic subsystems; S2: In this simulation environment, a deep reinforcement learning algorithm based on the Actor-Critic architecture is designed to create a multi-objective reward function to guide the policy network to simultaneously optimize two control objectives: vertical balance of the pendulum and centering of the slider. S3: Iterative training is performed during the simulation to update the neural network; S4: After training is complete, deploy the converged Actor network to the motion controller of the actual hardware platform.

[0013] Furthermore, in S1, the high-fidelity digital simulation environment reproduces the hydraulic cylinder friction, internal leakage, oil compressibility, servo motor response delay, and external load disturbance uncertainty factors, and has a continuous state space and action space to support interactive learning between the agent and the environment.

[0014] Furthermore, in S2, the reward function consists of three parts: Sparse positive rewards are triggered only when the system state enters the preset target area—that is, when the absolute value of the lever angle is less than the threshold, the angular velocity approaches zero, the slider position is at the center of the track, and its velocity is lower than the set limit. The dense negative penalty term is deducted in real time at each time step based on the weighted sum of the squares of the pendulum angle deviation, angular velocity deviation, slider position deviation and velocity deviation; A hard constraint penalty is applied, which immediately applies a large negative reward and terminates the current training round when the slider displacement exceeds the physical boundary of the track or the instantaneous power of the system exceeds the safety threshold.

[0015] Furthermore, in S3, the agent iteratively optimizes the policy through the following steps: initializing the parameters of the Actor network and the Critic network; generating a pump speed command as an action output based on the current observation state at each time step; receiving the next state and immediate reward from the environment after executing the action; storing the complete quadruple: current state, action, reward, and next state in the experience replay buffer; periodically sampling batch data from the buffer and updating the value estimation function of the Critic network and the policy mapping function of the Actor network respectively using gradient descent; superimposing exploration noise following the Ornstein-Uhlenbeck process into the action output and decreasing its intensity with each training round to balance exploration and exploitation; randomly sampling the pendulum tilt angle in the initial state of each round within a preset range to enhance the policy's generalization ability to diverse initial conditions; and prematurely ending the current round when the system successfully maintains balance for more than a preset time or an out-of-bounds fault occurs.

[0016] Furthermore, in S4, the motion controller collects raw signals from level sensors, temperature sensors, pressure sensors, displacement sensors, acceleration sensors, and angle sensors in real time. After analog-to-digital conversion and filtering, a system state vector is formed. This state vector is input into a trained neural network model, which outputs target speed commands for each servo motor. The commands are transmitted to the corresponding servo drivers via the EtherCAT bus, driving the servo motors to adjust the rotational speed of the hydraulic pumps. Simultaneously, the corresponding electromagnetic directional valves are dynamically activated or deactivated according to the current control requirements, causing some hydraulic pumps to enter a zero-displacement bypass state. Thus, without increasing the additional flow supply, the system energy input is finely controlled by changing the number of effective oil supply pumps.

[0017] Compared with the prior art, the advantages of the present invention are: 1. The present invention innovatively adopts a multi-pump parallel hydraulic drive method, which provides a large flow output and realizes high-precision and agile control of heavy-duty underactuated systems.

[0018] 2. This invention designs a related control method to compensate for the system's nonlinearity, ultimately achieving high-precision and agile control of a heavy-load underactuated system. The system exhibits strong robustness and high power density, making it suitable as a teaching device for control systems and an experimental platform for pump control system research. It also provides a perfect research object for in-depth study of nonlinear control theory. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the overall structure of the multi-pump zero-displacement hydraulic agile control system in an embodiment of the present invention; Figure 2 This is a schematic diagram of the hydraulic closed-loop circuit in an embodiment of the present invention; Figure 3These are three views of the mechanical system in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be further described below.

[0021] like Figure 1 As shown, a multi-pump zero-displacement hydraulic agile control system includes a hydraulic system 1, a mechanical system 2, and an electrical control system 3.

[0022] Hydraulic system 1 includes hydraulic cylinder 11, hydraulic pump 12, and servo motor 13, providing hydraulic power to the system; mechanical system 2 includes slider 23, pendulum 22, and pendulum 21, which are the mechanical execution parts of the system; electrical control system 3 includes a host computer 31, various sensors such as level sensor 321, temperature sensor 322, acceleration sensor 323, displacement sensor 324, angle sensor 325, and pressure sensor 326, four servo drives 33, motion controller 34, and eight solenoid directional valves 35. The host computer 31 communicates via Ethernet, and the motion controller 34 interacts with the servo drives 33 via EtherCAT. Hydraulic system 1 is the power unit, mechanical system 2 is the execution unit, and electrical control system 3 is the control unit.

[0023] like Figure 1 As shown, the specific working principle of the hardware system of this invention is as follows: The host computer 31 in the electronic control system 3 writes a program for the motion controller 34. The motion controller 34 collects information from the pressure sensor 326, angle sensor 325, liquid level sensor 321, temperature sensor 322, acceleration sensor 323, and displacement sensor 324, determines the current working state of the mechanical and hydraulic systems, and sends control commands to the servo driver 33. Finally, by changing the speed of the servo motor 13, the hydraulic pump 12 is driven to rotate. The oil output by the hydraulic pump 12 then drives the hydraulic cylinder 11 to extend and retract, driving the slider 23 to move, and in turn driving the pendulum 22 and the pendulum 21 to rotate. Among them, the acceleration sensor 323 and the displacement sensor 324 are both analog (4-20mA) signal outputs, used to measure the displacement and acceleration of the slider 23; the angle sensor (24V pulse signal, ABZ three-phase) 325 is used to measure the angle rotated by the pendulum 22.

[0024] The specific working principle of the control method of the present invention is as follows: S1: First, a high-fidelity digital simulation environment needs to be constructed. This environment is based on joint modeling of multibody dynamics and hydraulic fluid dynamics, implemented using the Simscape Multibody and Simscape Fluids toolboxes. The model fully includes the Coulomb and viscous friction characteristics of the hydraulic cylinder, internal leakage, the bulk modulus of the hydraulic fluid, the first-order inertial delay of the servo motor, and external random disturbances. The state space is a continuous 12-dimensional space, and the action space is a continuous 2-dimensional space, corresponding to the target speeds of the four servo motors, in rpm, with values ​​ranging from -3000 to 3000. The simulation step size is set to 1 ms, consistent with the actual control cycle. It includes mechanical parts (slider, lever) and hydraulic parts (pump, cylinder, servo motor, etc.). The simulation environment is used to simulate the nonlinear dynamic behavior of the system, including uncertainties such as friction, leakage, and load changes. This environment has continuous state and action spaces and can interact with the intelligent agent in the learning algorithm.

[0025] S2: Deploy a deep reinforcement learning algorithm based on the Actor-Critic architecture in the simulation environment. The Actor network adopts a three-layer fully connected structure, with 12 nodes in the input layer, 256 and 128 nodes in the hidden layers, and 2 nodes in the output layer. The activation function is ReLU, and the output layer uses the tanh function to map the original output to [-1, 1], and then linearly scales it to the rotational speed range. The Critic network is also a three-layer fully connected network, with the input being a concatenated vector of state and action (14-dimensional), 256 and 128 nodes in the hidden layers, and the output being a scalar state value estimate. The optimizer is Adam, and the learning rate is set to 3×10. -4 The discount factor γ is 0.99, and the target network soft update coefficient τ is 0.005.

[0026] Design a multi-objective reward function to guide the agent to achieve two control objectives: the pendulum is raised and kept balanced, while the slider is centered. The reward function includes: Sparse positive reward: When the system enters the target state region (the pendulum is vertical, the slider is centered and the speed is close to zero), a positive reward is given.

[0027] Dense negative reward: At each time step, a penalty is imposed based on the deviation of the lever angle, angular velocity, slider position, and velocity from the target value.

[0028] Hard constraint penalties: Severe penalties are imposed when the vehicle exceeds the track limits or the power exceeds the threshold to ensure the safe operation of the system.

[0029] S3: Iterative training is performed during simulation to update the neural network, following the standard Deep Deterministic Policy Gradient (DDPG) process. The experience replay buffer has a capacity of 1×10⁻⁶. 6A quadruple is used. At the start of each round, the pendulum tilt angle is uniformly and randomly sampled within the range of [-0.8, 0.8] rad, and the initial velocity and angular velocity of the slider are set to zero. Exploration noise is handled using the Ornstein-Uhlenbeck process, with an initial standard deviation of 0.2, decreasing by 5% every 1000 rounds until it reaches 0.02. The batch size is 128, and the network is updated once per step. Training continues until the average reward stabilizes above 85 for 100 consecutive rounds, and the successful balance maintenance time exceeds 30 seconds.

[0030] S4: Actual System Deployment and Comparison: The converged Actor network was exported as C code, compiled, and deployed to the real-time task of the motion controller. During actual operation, the motion controller cyclically executes the following steps at a frequency of 1kHz: First, it synchronously acquires all raw signals from the sensors; second, it performs low-pass filtering (cutoff frequency 50 Hz) on the analog signals, and performs quadruple frequency decoding and integration on the encoder signals to obtain the angle and angular velocity; third, it inputs the processed 12-dimensional state vector into the neural network model and outputs two normalized speed commands; then, it multiplies the commands by 3000 and limits them before sending them to the corresponding servo driver via the EtherCAT bus; simultaneously, it determines whether to trigger the zero-displacement switching logic based on the current state: based on the angle and position thresholds satisfied within 500 ms of the system, it decides to start 1-4 hydraulic pumps for operation. The core of the zero-displacement switching mechanism lies in the following: when the system is in a near-equilibrium, small-amplitude oscillation phase, one or more hydraulic pumps are actively switched to bypass mode, ceasing to supply oil to the main circuit and retaining only the necessary number of pumps to maintain fine-tuning capability. In this state, the system relies on inertia and residual pressure to complete precise attitude adjustments, significantly reducing unnecessary energy consumption. When a significant deviation of the lever or overshoot of the slider is detected, all hydraulic pumps are immediately reactivated to provide maximum controllable flow, ensuring rapid recovery capability. This mechanism, through deep coupling with reinforcement learning strategies, enables the controller to automatically select the optimal pump combination and speed configuration at different dynamic stages, achieving synergistic optimization of energy efficiency and control performance.

[0031] like Figure 2The diagram shown is a schematic diagram of the hydraulic system 1 of this invention. This hydraulic system employs a complex circuit configuration, including a hydraulic cylinder 11, a hydraulic pump 12, a servo motor 13, an electromagnetic directional valve 35, a check valve 18, a relief valve 17, a fan 19, a ball valve 16, an oil tank 14, and a filter 15, with a relatively regular structural arrangement. Multiple hydraulic pumps connected in parallel can achieve a large flow rate of oil output (up to approximately 50 L / min). The electromagnetic directional valve 35 connects different hydraulic pumps 12 to the hydraulic system. When the electromagnet is not energized, the hydraulic cylinder 11 is directly connected to the hydraulic pump 12 via a pipeline, forming a working circuit; when the electromagnet is energized, the oil output from the hydraulic pump 12 bypasses the corresponding energized pump and enters the hydraulic circuit. The check valve 18 is responsible for outputting replenishing oil and auxiliary working oil from the oil tank 14, which is cooled by the fan 19. The filter 15 ensures oil cleanliness, and the ball valve 16 controls the connection between the oil tank 14 and the hydraulic working circuit, facilitating installation. Temperature sensor 322 measures the temperature of the oil in the tank and outputs the status to motion controller 34; level sensor 321 monitors the tank status and outputs the tank level to motion controller 34. Through the coordinated action of these components, the system achieves the transmission and control of hydraulic energy to drive the actuator to complete corresponding actions. Different circuits cooperate to meet the hydraulic transmission requirements under various operating conditions. Simultaneously, acceleration sensor 323 and displacement sensor 324 are installed near the hydraulic cylinder to indirectly measure the speed and acceleration values ​​of slider 23.

[0032] In the closed loop of hydraulic system 1, the output flow of hydraulic pump 12 is adjusted by changing the speed of servo motor 13, thereby adjusting the flow of hydraulic cylinder 11 and ultimately adjusting the output speed of hydraulic cylinder 11.

[0033] like Figure 3 The diagram shows a three-view drawing of the mechanical system 2 of this invention, including a pendulum 21, a pendulum rod 22, a slider 23, a guide rail 24, a bracket 25, a bearing 26, a valve block 27, and an output shaft 28. The hydraulic fluid in the hydraulic system 1 is output to the hydraulic cylinder 11 via the valve block 27, ultimately directly driving the slider 23 to move within the guide rail 24. The inertia generated by the movement of the slider 23 eventually drives the pendulum rod 22 and the pendulum 21 to rotate via the bearing 26 and the output shaft 28. Utilizing the relevant feedback facilities in the electronic control system 3, the inverted balance control of the underactuated system is achieved, thereby realizing precise closed-loop control.

[0034] In the electronic control system 3, the electrical parameters of the output signals of the pressure sensor 326, displacement sensor 324, acceleration sensor 323, liquid level sensor 321 and temperature sensor 322 are 4-20mA, and the output type of the angle sensor is 24V pulse signal (ABZ three-phase).

[0035] In the electronic control system 3, the servo driver 33 uses EtherCAT communication and is connected to the EtherCAT interface of the motion controller 34 via a network cable. Simultaneously, the motion controller 34 connects to the relevant interfaces of the host computer 31 via an Ethernet interface to achieve network communication. The host computer 31 is responsible for program writing and secondary development of the motion controller 34, forming a control closed loop.

[0036] like Figure 3 As shown, the mechanical system includes a pendulum 22, a pendulum 21, a guide rail 24, a bracket 25, a slider 23, a valve block 27, a bearing 26, and an output shaft 28, forming an inverted pendulum structure.

[0037] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A multi-pump zero-displacement hydraulic agile control system, characterized in that, This includes hydraulic systems, mechanical systems, and electrical control systems; The hydraulic system includes at least two hydraulic pumps connected in parallel. Each hydraulic pump is driven by an independent servo motor. The outlet of each hydraulic pump is connected to the main working circuit via an electromagnetic reversing valve. The main working circuit is connected to a double-acting hydraulic cylinder. The piston rod of the hydraulic cylinder is rigidly connected to the slider. The mechanical system includes a horizontal guide rail, a slider, a pendulum rod, and a pendulum. The slider is placed on the horizontal guide rail, one end of the pendulum rod is movably connected to the top of the slider, and the other end is fixed to the pendulum, forming an underdriven inverted pendulum mechanism. The electronic control system includes a motion controller, multiple servo drives, a solenoid valve drive module, and various types of sensors. The sensors include displacement and acceleration sensors for detecting slider displacement and acceleration, angle sensors for detecting rocker arm angle, pressure sensors for detecting pressure in the main working circuit, and level and temperature sensors for monitoring the oil tank status. The motion controller generates speed commands for each servo motor based on real-time acquired status information and controls the energization status of the solenoid directional valve to achieve dynamic start / stop and flow distribution of the hydraulic pump.

2. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, When the electromagnetic reversing valve is de-energized, it causes the corresponding hydraulic pump to supply oil to the main working circuit. When energized, it causes the hydraulic pump to output oil and return it to the oil tank through the bypass pipeline, thereby entering a zero-displacement idling state.

3. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, The main working circuit is equipped with a one-way oil replenishment valve. One end of the one-way oil replenishment valve is connected to the oil tank, and the other end is connected to the low-pressure side of the main working circuit. It is used to automatically replenish oil when the pressure on the low-pressure side of the system is lower than a set threshold in order to maintain the minimum working pressure of the closed circuit.

4. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, The hydraulic pump has an external drain port, and the leaked oil flows through pipelines and valve blocks into the oil tank.

5. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, The main working circuit is also equipped with an oil cooling system; the oil outlet of the oil tank is equipped with a filter.

6. A control method for a multi-pump zero-displacement hydraulic agile control system, characterized in that, Includes the following steps: S1: Construct a high-fidelity digital simulation environment that includes mechanical and hydraulic subsystems; S2: In this simulation environment, a deep reinforcement learning algorithm based on the Actor-Critic architecture is designed to create a multi-objective reward function to guide the policy network to simultaneously optimize two control objectives: vertical balance of the pendulum and centering of the slider. S3: Iterative training is performed during the simulation to update the neural network; S4: After training is complete, deploy the converged Actor network to the motion controller of the actual hardware platform.

7. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In S1, the high-fidelity digital simulation environment reproduces the hydraulic cylinder friction, internal leakage, oil compressibility, servo motor response delay, and external load disturbance uncertainty factors, and has a continuous state space and action space to support interactive learning between the intelligent agent and the environment.

8. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In S2, the reward function consists of three parts: Sparse positive rewards are triggered only when the system state enters the preset target area—that is, when the absolute value of the lever angle is less than the threshold, the angular velocity approaches zero, the slider position is at the center of the track, and its velocity is lower than the set limit. The dense negative penalty term is deducted in real time at each time step based on the weighted sum of the squares of the pendulum angle deviation, angular velocity deviation, slider position deviation and velocity deviation; A hard constraint penalty is applied, which immediately applies a large negative reward and terminates the current training round when the slider displacement exceeds the physical boundary of the track or the instantaneous power of the system exceeds the safety threshold.

9. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In S3, the agent iteratively optimizes the strategy through the following steps: initializing the parameters of the Actor network and the Critic network; generating a pump speed command as an action output based on the current observation state at each time step; and receiving the next state and immediate reward from the environment after executing the action. The complete quadruple set—current state, action, reward, and next state—is stored in the experience replay buffer. Batch data is periodically sampled from the buffer, and gradient descent is used to update the value estimation function of the Critic network and the policy mapping function of the Actor network, respectively. Exploration noise following an Ornstein-Uhlenbeck process is superimposed on the action output, and its intensity is decreased with each training round to balance exploration and exploitation. The pendulum tilt angle in the initial state of each round is randomly sampled within a preset range to enhance the policy's generalization ability to diverse initial conditions. The current round ends early when the system successfully maintains balance for more than a preset time or an out-of-bounds fault occurs.

10. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In step S4, the motion controller collects raw signals from the level sensor, temperature sensor, pressure sensor, displacement sensor, acceleration sensor, and angle sensor in real time. After analog-to-digital conversion and filtering, a system state vector is formed. This state vector is input into a trained neural network model, which outputs target speed commands for each servo motor. The commands are transmitted to the corresponding servo drivers via the EtherCAT bus to drive the servo motors to adjust the rotational speed of the hydraulic pumps. Simultaneously, the corresponding electromagnetic directional valves are dynamically activated or deactivated according to the current control requirements, allowing some hydraulic pumps to enter a zero-displacement bypass state. This allows for precise control of the system energy input by changing the number of effective oil supply pumps without increasing the additional flow supply.

Images