A multi-pump zero-displacement hydraulic agile control system and its control method

Abstract

This invention proposes a multi-pump zero-displacement hydraulic agile control system and its control method. The system consists of three parts: a hydraulic system, a mechanical system, and an electronic control system. The control method integrates a multi-pump zero-displacement switching mechanism with a deep reinforcement learning strategy based on an Actor-Critic architecture. It dynamically starts and stops some hydraulic pumps to adjust the effective oil supply quantity and combines this with a neural network to output servo motor speed commands in real time, coordinating the strongly coupled dynamic relationship between the slider displacement and the pendulum angle. When the system approaches equilibrium, it switches some pumps to bypass mode to reduce energy consumption; when there is a significant deviation, it activates all pumps to ensure recovery capability. Through the above technical solution, this application can achieve agile, stable, and energy-saving control of inverted pendulum-type underactuated systems without relying on precise mathematical models.

Description

Technical Field

[0001] This invention relates to the field of hydraulic automatic control, and in particular to a multi-pump zero-displacement hydraulic agile control system and its control method. Background Technology

[0002] Against the backdrop of the accelerated penetration of new energy technologies into the heavy equipment sector, hydraulic drive systems, due to their high power density and strong load adaptability, continue to serve as the core power unit for engineering machinery, special vehicles, and high-end manufacturing equipment. However, the inherent nonlinear characteristics of traditional hydraulic systems—including dynamic response hysteresis and flow mismatch problems caused by internal leakage, oil compressibility, and component friction—severely restrict their control performance in high-precision, high-dynamic application scenarios. Especially for systems using constant displacement pumps, it is difficult to maintain a strictly linear relationship between output flow and input speed, leading to a significant decrease in the position and speed control accuracy of the actuators. To address this challenge, the industry has long relied on two mainstream technical approaches: first, constructing nonlinear controllers based on precise mathematical models, such as adaptive sliding mode control, robust control, or high-order PID structures, to offset system uncertainties through online compensation mechanisms; second, adopting a multi-pump parallel architecture to achieve flow redundancy and dynamic allocation at the physical level, thereby enhancing the system's response to transient load changes. The above methods can effectively improve system stability and tracking accuracy under specific working conditions, and have formed a relatively mature application paradigm, especially in fully driven systems (i.e., the control input degrees of freedom equal the controlled output degrees of freedom).

[0003] However, as high-end equipment places higher demands on agility, energy efficiency, and autonomous adaptability, a more complex type of control object—underactuated hydraulic systems—has gradually become a focus of cutting-edge research. Typical examples include inverted pendulum structures, whose control degrees of freedom are fewer than the number of generalized coordinates of the system. Essentially, they are nonholonomic constrained systems, exhibiting strong nonlinearity, inherent instability, and a need for rapid dynamic response. Such systems are widely used in scenarios such as robot balancing and walking, spacecraft attitude adjustment, and precision industrial platforms, posing extreme challenges to the real-time performance, robustness, and energy efficiency of control algorithms. While existing hydraulic control strategies perform well under a fully actuated framework, they reveal deep-seated structural defects when dealing with the coupled dynamics of underactuated systems. The reason for this is that traditional methods generally assume that the system has sufficient control input to directly drive all state variables, while underactuated systems lack the direct control over key degrees of freedom and must achieve stable control through indirect excitation and energy shaping. Under this premise, simply relying on model compensation or multi-pump flow superposition not only makes it difficult to accurately coordinate the strong coupling relationship between the slider displacement and the pendulum angle but also easily leads to system oscillations or energy waste due to excessive intervention of control commands. Furthermore, when data-driven methods such as neural networks are introduced to reduce model dependence, the training process often ignores the physical constraints unique to hydraulic systems (such as flow conservation, pressure boundaries, and power thresholds). This results in the strategy converging well in the simulation environment, but its performance drops sharply in actual deployment because the dynamic characteristics of the oil's elastic modulus changing with temperature and pressure are not considered, and it may even trigger safety protection mechanisms.

[0004] Crucially, existing technologies, in their pursuit of control precision or response speed, often fall into a performance trade-off: significantly increasing pump output flow to improve the speed of pendulum erection inevitably exacerbates slider overshoot and system energy consumption; conversely, strictly limiting power to ensure energy efficiency makes it difficult to quickly recover from a large-angle tilt to vertical balance within a limited time. This contradiction stems from the failure of traditional control architectures to coordinate and optimize the zero-displacement switching capability of multi-pump systems with the energy transfer mechanism of underactuated systems. Specifically, while multi-pump parallel systems possess flow regulation potential, their solenoid valve switching logic is typically based on preset operating conditions, lacking the ability to perceive the dynamic mapping relationship between the system's current energy state and the target equilibrium point; and while learning controllers can explore complex strategies, they rarely deeply integrate them with the zero-displacement operating mode in the hydraulic circuit (i.e., the pump is in an unloaded or bypassed state, not supplying oil to the main circuit), thus missing the key window for achieving fine energy regulation using "reactive motion". Therefore, in hydraulic agile control scenarios with underactuated, highly nonlinear, and high energy efficiency requirements, the key to breaking through existing technological bottlenecks lies in how to construct a control method that can fully utilize the structural redundancy and zero displacement switching characteristics of multi-pump systems, and accurately coordinate the indirect control degrees of freedom and system energy flow through intelligent learning mechanisms.

[0005] Therefore, how to integrate a multi-pump zero-displacement switching mechanism with a multi-objective optimization strategy based on reinforcement learning to achieve high-precision, low-energy-consumption, and robust agile balance control of underactuated hydraulic systems under a wide range of initial disturbances, while ensuring the safe operation of the system, has become a key challenge and an urgent technical problem for those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to propose a multi-pump zero-displacement hydraulic agile control system and control method, aiming to solve the technical problem of underactuated hydraulic systems being unable to achieve high-precision, low-energy-consumption, and robust balance control under a wide range of initial disturbances.

[0007] To achieve the above objectives, this invention proposes a multi-pump zero-displacement hydraulic agile control system, comprising a hydraulic system, a mechanical system, and an electrical control system;

[0008] The hydraulic system includes at least two hydraulic pumps connected in parallel. Each hydraulic pump is driven by an independent servo motor. The outlet of each hydraulic pump is connected to the main working circuit via an electromagnetic directional valve. The main working circuit is connected to a double-acting hydraulic cylinder. The piston rod of the hydraulic cylinder is rigidly connected to the slider.

[0009] The mechanical system includes a horizontal guide rail, a slider, a pendulum rod, and a pendulum. The slider is placed on the horizontal guide rail, one end of the pendulum rod is movably connected to the top of the slider, and the other end is fixed to the pendulum, forming an underdriven inverted pendulum mechanism.

[0010] The electronic control system includes a motion controller, multiple servo drives, solenoid valve drive modules, and various types of sensors. The sensors include displacement and acceleration sensors for detecting slider displacement and acceleration, angle sensors for detecting rocker arm angle, pressure sensors for detecting pressure in the main working circuit, and level and temperature sensors for monitoring the oil tank status. The motion controller generates speed commands for each servo motor based on real-time acquired status information and controls the energization status of the solenoid directional valve to achieve dynamic start / stop and flow distribution of the hydraulic pump.

[0011] Furthermore, when the electromagnetic directional valve is de-energized, it causes the corresponding hydraulic pump to supply oil to the main working circuit. When energized, it causes the hydraulic pump to output oil and return it to the oil tank through the bypass pipeline, thus entering a zero-displacement idling state.

[0012] Furthermore, the main working circuit is equipped with a one-way oil replenishment valve. One end of the one-way oil replenishment valve is connected to the oil tank, and the other end is connected to the low-pressure side of the main working circuit. It is used to automatically replenish oil when the pressure on the low-pressure side of the system is lower than the set threshold in order to maintain the minimum working pressure of the closed circuit.

[0013] Furthermore, the hydraulic pump has an external drain port, and the leaked oil flows through pipelines and valve blocks into the oil tank.

[0014] Furthermore, the main working circuit is equipped with an oil cooling system; the oil outlet of the oil tank is equipped with a filter.

[0015] This invention also proposes a control method for a multi-pump zero-displacement hydraulic agile control system, comprising the following steps:

[0016] S1: Construct a high-fidelity digital simulation environment that includes mechanical and hydraulic subsystems;

[0017] S2: In this simulation environment, a deep reinforcement learning algorithm based on the Actor-Critic architecture is designed to create a multi-objective reward function to guide the policy network to simultaneously optimize two control objectives: vertical balance of the pendulum and centering of the slider.

[0018] S3: Iterative training is performed during the simulation to update the neural network;

[0019] S4: After training is complete, deploy the converged Actor network to the motion controller of the actual hardware platform.

[0020] Furthermore, in S1, the high-fidelity digital simulation environment reproduces the hydraulic cylinder friction, internal leakage, oil compressibility, servo motor response delay, and external load disturbance uncertainty factors, and has a continuous state space and action space to support interactive learning between the agent and the environment.

[0021] Furthermore, in S2, the reward function consists of three parts:

[0022] Sparse positive rewards are triggered only when the system state enters the preset target area—that is, when the absolute value of the lever angle is less than the threshold, the angular velocity approaches zero, the slider position is at the center of the track, and its velocity is lower than the set limit.

[0023] The dense negative penalty term is deducted in real time at each time step based on the weighted sum of the squares of the pendulum angle deviation, angular velocity deviation, slider position deviation and velocity deviation;

[0024] A hard constraint penalty is applied, which immediately applies a large negative reward and terminates the current training round when the slider displacement exceeds the physical boundary of the track or the instantaneous power of the system exceeds the safety threshold.

[0025] Furthermore, in S3, the agent iteratively optimizes the policy through the following steps: initializing the parameters of the Actor network and the Critic network; generating a pump speed command as an action output based on the current observation state at each time step; receiving the next state and immediate reward from the environment after executing the action; storing the complete quadruple: current state, action, reward, and next state in the experience replay buffer; periodically sampling batch data from the buffer and updating the value estimation function of the Critic network and the policy mapping function of the Actor network respectively using gradient descent; superimposing exploration noise following the Ornstein-Uhlenbeck process into the action output and decreasing its intensity with each training round to balance exploration and exploitation; randomly sampling the pendulum tilt angle in the initial state of each round within a preset range to enhance the policy's generalization ability to diverse initial conditions; and prematurely ending the current round when the system successfully maintains balance for more than a preset time or an out-of-bounds fault occurs.

[0026] Furthermore, in S4, the motion controller collects raw signals from level sensors, temperature sensors, pressure sensors, displacement sensors, acceleration sensors, and angle sensors in real time. After analog-to-digital conversion and filtering, a system state vector is formed. This state vector is input into a trained neural network model, which outputs target speed commands for each servo motor. The commands are transmitted to the corresponding servo drivers via the EtherCAT bus, driving the servo motors to adjust the rotational speed of the hydraulic pumps. Simultaneously, the corresponding electromagnetic directional valves are dynamically activated or deactivated according to the current control requirements, causing some hydraulic pumps to enter a zero-displacement bypass state. Thus, without increasing the additional flow supply, the system energy input is finely controlled by changing the number of effective oil supply pumps.

[0027] Compared with the prior art, the advantages of the present invention are:

[0028] 1. The present invention innovatively adopts a multi-pump parallel hydraulic drive method, which provides a large flow output and realizes high-precision and agile control of heavy-duty underactuated systems.

[0029] 2. This invention designs a related control method to compensate for the system's nonlinearity, ultimately achieving high-precision and agile control of a heavy-load underactuated system. The system exhibits strong robustness and high power density, making it suitable as a teaching device for control systems and an experimental platform for pump control system research. It also provides a perfect research object for in-depth studies of nonlinear control theory. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the overall structure of the multi-pump zero-displacement hydraulic agile control system in an embodiment of the present invention;

[0031] Figure 2 This is a schematic diagram of the hydraulic closed-loop circuit in an embodiment of the present invention;

[0032] Figure 3 These are three views of the mechanical system in an embodiment of the present invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be further described below.

[0034] like Figure 1 As shown, a multi-pump zero-displacement hydraulic agile control system includes a hydraulic system 1, a mechanical system 2, and an electrical control system 3.

[0035] Hydraulic system 1 includes hydraulic cylinder 11, hydraulic pump 12, and servo motor 13, providing hydraulic power to the system; mechanical system 2 includes slider 23, pendulum 22, and pendulum 21, which are the mechanical execution parts of the system; electrical control system 3 includes a host computer 31, various sensors such as level sensor 321, temperature sensor 322, acceleration sensor 323, displacement sensor 324, angle sensor 325, and pressure sensor 326, four servo drives 33, motion controller 34, and eight solenoid directional valves 35. The host computer 31 communicates via Ethernet, and the motion controller 34 interacts with the servo drives 33 via EtherCAT. Hydraulic system 1 is the power unit, mechanical system 2 is the execution unit, and electrical control system 3 is the control unit.

[0036] like Figure 1 As shown, the specific working principle of the hardware system of this invention is as follows: The host computer 31 in the electronic control system 3 writes a program for the motion controller 34. The motion controller 34 collects information from the pressure sensor 326, angle sensor 325, liquid level sensor 321, temperature sensor 322, acceleration sensor 323, and displacement sensor 324, determines the current working state of the mechanical and hydraulic systems, and sends control commands to the servo driver 33. Finally, by changing the speed of the servo motor 13, the hydraulic pump 12 is driven to rotate. The oil output by the hydraulic pump 12 then drives the hydraulic cylinder 11 to extend and retract, driving the slider 23 to move, and in turn driving the pendulum 22 and the pendulum 21 to rotate. Among them, the acceleration sensor 323 and the displacement sensor 324 are both analog (4-20mA) signal outputs, used to measure the displacement and acceleration of the slider 23; the angle sensor (24V pulse signal, ABZ three-phase) 325 is used to measure the angle rotated by the pendulum 22.

[0037] The specific working principle of the control method of the present invention is as follows:

[0038] S1: First, a high-fidelity digital simulation environment needs to be constructed. This environment is based on joint modeling of multibody dynamics and hydraulic fluid dynamics, implemented using the Simscape Multibody and Simscape Fluids toolboxes. The model fully includes the Coulomb and viscous friction characteristics of the hydraulic cylinder, internal leakage, the bulk modulus of the hydraulic fluid, the first-order inertial delay of the servo motor, and external random disturbances. The state space is a continuous 12-dimensional space, and the action space is a continuous 2-dimensional space, corresponding to the target speeds of the four servo motors, in rpm, with values ​​ranging from -3000 to 3000. The simulation step size is set to 1 ms, consistent with the actual control cycle. It includes mechanical parts (slider, lever) and hydraulic parts (pump, cylinder, servo motor, etc.). The simulation environment is used to simulate the nonlinear dynamic behavior of the system, including uncertainties such as friction, leakage, and load changes. This environment has continuous state and action spaces and can interact with the intelligent agent in the learning algorithm.

[0039] S2: Deploy a deep reinforcement learning algorithm based on the Actor-Critic architecture in the simulation environment. The Actor network adopts a three-layer fully connected structure, with 12 nodes in the input layer, 256 and 128 nodes in the hidden layers, and 2 nodes in the output layer. The activation function is ReLU, and the output layer uses the tanh function to map the original output to [-1, 1], and then linearly scales it to the rotational speed range. The Critic network is also a three-layer fully connected network, with the input being a concatenated vector of state and action (14-dimensional), 256 and 128 nodes in the hidden layers, and the output being a scalar state value estimate. The optimizer is Adam, and the learning rate is set to 3×10. -4 The discount factor γ is 0.99, and the target network soft update coefficient τ is 0.005.

[0040] Design a multi-objective reward function to guide the agent to achieve two control objectives: the pendulum is raised and kept balanced, while the slider is centered. The reward function includes:

[0041] Sparse positive reward: When the system enters the target state region (the pendulum is vertical, the slider is centered and the speed is close to zero), a positive reward is given.

[0042] Dense negative reward: At each time step, a penalty is imposed based on the deviation of the lever angle, angular velocity, slider position, and velocity from the target value.

[0043] Hard constraint penalties: Severe penalties are imposed when the vehicle exceeds the track limits or the power exceeds the threshold to ensure the safe operation of the system.

[0044] S3: Iterative training is performed during simulation to update the neural network, following the standard Deep Deterministic Policy Gradient (DDPG) process. The experience replay buffer has a capacity of 1×10⁻⁶. 6A quadruple is used. At the start of each round, the pendulum tilt angle is uniformly and randomly sampled within the range of [-0.8, 0.8] rad, and the initial velocity and angular velocity of the slider are set to zero. Exploration noise is handled using the Ornstein-Uhlenbeck process, with an initial standard deviation of 0.2, decreasing by 5% every 1000 rounds until it reaches 0.02. The batch size is 128, and the network is updated once per step. Training continues until the average reward stabilizes above 85 for 100 consecutive rounds, and the successful balance maintenance time exceeds 30 seconds.

[0045] S4: Actual System Deployment and Comparison: The converged Actor network was exported as C code, compiled, and deployed to the real-time task of the motion controller. During actual operation, the motion controller cyclically executes the following steps at a frequency of 1kHz: First, it synchronously acquires all raw signals from the sensors; second, it performs low-pass filtering (cutoff frequency 50 Hz) on the analog signals, and performs quadruple frequency decoding and integration on the encoder signals to obtain the angle and angular velocity; third, it inputs the processed 12-dimensional state vector into the neural network model and outputs two normalized speed commands; then, it multiplies the commands by 3000 and limits them before sending them to the corresponding servo driver via the EtherCAT bus; simultaneously, it determines whether to trigger the zero-displacement switching logic based on the current state: based on the angle and position thresholds satisfied within 500 ms of the system, it decides to start 1-4 hydraulic pumps for operation. The core of the zero-displacement switching mechanism lies in the following: when the system is in a near-equilibrium, small-amplitude oscillation phase, one or more hydraulic pumps are actively switched to bypass mode, ceasing to supply oil to the main circuit and retaining only the necessary number of pumps to maintain fine-tuning capability. In this state, the system relies on inertia and residual pressure to complete precise attitude adjustments, significantly reducing unnecessary energy consumption. When a significant deviation of the lever or overshoot of the slider is detected, all hydraulic pumps are immediately reactivated to provide maximum controllable flow, ensuring rapid recovery capability. This mechanism, through deep coupling with reinforcement learning strategies, enables the controller to automatically select the optimal pump combination and speed configuration at different dynamic stages, achieving synergistic optimization of energy efficiency and control performance.

[0046] like Figure 2The diagram shown is a schematic diagram of the hydraulic system 1 of this invention. This hydraulic system employs a complex circuit configuration, including a hydraulic cylinder 11, a hydraulic pump 12, a servo motor 13, an electromagnetic directional valve 35, a check valve 18, a relief valve 17, a fan 19, a ball valve 16, an oil tank 14, and a filter 15, with a relatively regular structural arrangement. Multiple hydraulic pumps connected in parallel can achieve a large flow rate of oil output (up to approximately 50 L / min). The electromagnetic directional valve 35 connects different hydraulic pumps 12 to the hydraulic system. When the electromagnet is not energized, the hydraulic cylinder 11 is directly connected to the hydraulic pump 12 via a pipeline, forming a working circuit; when the electromagnet is energized, the oil output from the hydraulic pump 12 bypasses the corresponding energized pump and enters the hydraulic circuit. The check valve 18 is responsible for outputting replenishing oil and auxiliary working oil from the oil tank 14, which is cooled by the fan 19. The filter 15 ensures oil cleanliness, and the ball valve 16 controls the connection between the oil tank 14 and the hydraulic working circuit, facilitating installation. Temperature sensor 322 measures the temperature of the oil in the tank and outputs the status to motion controller 34; level sensor 321 monitors the tank status and outputs the tank level to motion controller 34. Through the coordinated action of these components, the system achieves the transmission and control of hydraulic energy to drive the actuator to complete corresponding actions. Different circuits cooperate to meet the hydraulic transmission requirements under various operating conditions. Simultaneously, acceleration sensor 323 and displacement sensor 324 are installed near the hydraulic cylinder to indirectly measure the speed and acceleration values ​​of slider 23.

[0047] In the closed loop of hydraulic system 1, the output flow of hydraulic pump 12 is adjusted by changing the speed of servo motor 13, thereby adjusting the flow of hydraulic cylinder 11 and ultimately adjusting the output speed of hydraulic cylinder 11.

[0048] like Figure 3 The diagram shows a three-view drawing of the mechanical system 2 of this invention, including a pendulum 21, a pendulum rod 22, a slider 23, a guide rail 24, a bracket 25, a bearing 26, a valve block 27, and an output shaft 28. The hydraulic fluid in the hydraulic system 1 is output to the hydraulic cylinder 11 via the valve block 27, ultimately directly driving the slider 23 to move within the guide rail 24. The inertia generated by the movement of the slider 23 eventually drives the pendulum rod 22 and the pendulum 21 to rotate via the bearing 26 and the output shaft 28. Utilizing the relevant feedback facilities in the electronic control system 3, the inverted balance control of the underactuated system is achieved, thereby realizing precise closed-loop control.

[0049] In the electronic control system 3, the electrical parameters of the output signals of the pressure sensor 326, displacement sensor 324, acceleration sensor 323, liquid level sensor 321 and temperature sensor 322 are 4-20mA, and the output type of the angle sensor is 24V pulse signal (ABZ three-phase).

[0050] In the electronic control system 3, the servo driver 33 uses EtherCAT communication and is connected to the EtherCAT interface of the motion controller 34 via a network cable. Simultaneously, the motion controller 34 connects to the relevant interfaces of the host computer 31 via an Ethernet interface to achieve network communication. The host computer 31 is responsible for program writing and secondary development of the motion controller 34, forming a control closed loop.

[0051] like Figure 3 As shown, the mechanical system includes a pendulum 22, a pendulum 21, a guide rail 24, a bracket 25, a slider 23, a valve block 27, a bearing 26, and an output shaft 28, forming an inverted pendulum structure.

[0052] The above are merely preferred embodiments of the present invention and do not constitute any limitation on the present invention. Any equivalent substitutions or modifications made by those skilled in the art to the technical solutions and content disclosed in the present invention without departing from the scope of the present invention shall be deemed to have remained within the protection scope of the present invention.

Claims

1. A multi-pump zero-displacement hydraulic agile control system, characterized in that, This includes hydraulic systems, mechanical systems, and electrical control systems; The hydraulic system includes at least two hydraulic pumps connected in parallel. Each hydraulic pump is driven by an independent servo motor. The outlet of each hydraulic pump is connected to the main working circuit via an electromagnetic reversing valve. The main working circuit is connected to a double-acting hydraulic cylinder. The piston rod of the hydraulic cylinder is rigidly connected to the slider. The mechanical system includes a horizontal guide rail, a slider, a pendulum rod, and a pendulum. The slider is placed on the horizontal guide rail, one end of the pendulum rod is movably connected to the top of the slider, and the other end is fixed to the pendulum, forming an underdriven inverted pendulum mechanism. The electronic control system includes a motion controller, multiple servo drives, a solenoid valve drive module, and various types of sensors. The sensors include displacement and acceleration sensors for detecting slider displacement and acceleration, angle sensors for detecting rocker arm angle, pressure sensors for detecting pressure in the main working circuit, and level and temperature sensors for monitoring the oil tank status. The motion controller generates speed commands for each servo motor based on real-time acquired status information and controls the energization status of the solenoid directional valve to achieve dynamic start / stop and flow distribution of the hydraulic pump.

2. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, When the electromagnetic reversing valve is de-energized, it causes the corresponding hydraulic pump to supply oil to the main working circuit. When energized, it causes the hydraulic pump to output oil and return it to the oil tank through the bypass pipeline, thereby entering a zero-displacement idling state.

3. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, The main working circuit is equipped with a one-way oil replenishment valve. One end of the one-way oil replenishment valve is connected to the oil tank, and the other end is connected to the low-pressure side of the main working circuit. It is used to automatically replenish oil when the pressure on the low-pressure side of the system is lower than a set threshold in order to maintain the minimum working pressure of the closed circuit.

4. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, The hydraulic pump has an external drain port, and the leaked oil flows through pipelines and valve blocks into the oil tank.

5. The multi-pump zero-displacement hydraulic agile control system according to claim 1, characterized in that, The main working circuit is also equipped with an oil cooling system; the oil outlet of the oil tank is equipped with a filter.

6. A control method for a multi-pump zero-displacement hydraulic agile control system, using the multi-pump zero-displacement hydraulic agile control system as described in any one of claims 1-5, characterized in that, Includes the following steps: S1: Construct a high-fidelity digital simulation environment that includes mechanical and hydraulic subsystems; S2: In this simulation environment, a deep reinforcement learning algorithm based on the Actor-Critic architecture is designed to create a multi-objective reward function to guide the policy network to simultaneously optimize two control objectives: vertical balance of the pendulum and centering of the slider. S3: Iterative training is performed during the simulation to update the neural network; S4: After training is complete, deploy the converged Actor network to the motion controller of the actual hardware platform.

7. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In S1, the high-fidelity digital simulation environment reproduces the hydraulic cylinder friction, internal leakage, oil compressibility, servo motor response delay, and external load disturbance uncertainty factors, and has a continuous state space and action space to support interactive learning between the intelligent agent and the environment.

8. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In S2, the reward function consists of three parts:

1. Sparse positive reward, which is triggered only when the system state enters the preset target area - that is, the absolute value of the pendulum angle is less than the threshold, the angular velocity approaches zero, the slider position is located at the center of the track and its speed is lower than the set limit; 2. Dense negative penalty items: at each time step, deductions are made in real time based on the weighted sum of the squares of the pendulum angle deviation, angular velocity deviation, slider position deviation, and velocity deviation.

3. Hard constraint penalty: When the slider displacement exceeds the physical boundary of the track or the instantaneous power of the system exceeds the safety threshold, a large negative reward is immediately applied and the current training round is terminated.

9. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In S3, the agent iteratively optimizes the strategy through the following steps: initializing the parameters of the Actor network and the Critic network; generating a pump speed command as an action output based on the current observation state at each time step; and receiving the next state and immediate reward from the environment after executing the action. The complete quadruple set—current state, action, reward, and next state—is stored in the experience replay buffer. Batch data is periodically sampled from the buffer, and gradient descent is used to update the value estimation function of the Critic network and the policy mapping function of the Actor network, respectively. Exploration noise following an Ornstein-Uhlenbeck process is superimposed on the action output, and its intensity is decreased with each training round to balance exploration and exploitation. The pendulum tilt angle in the initial state of each round is randomly sampled within a preset range to enhance the policy's generalization ability to diverse initial conditions. The current round ends early when the system successfully maintains balance for more than a preset time or an out-of-bounds fault occurs.

10. The control method of the multi-pump zero-displacement hydraulic agile control system according to claim 6, characterized in that, In step S4, the motion controller collects raw signals from the level sensor, temperature sensor, pressure sensor, displacement sensor, acceleration sensor, and angle sensor in real time. After analog-to-digital conversion and filtering, a system state vector is formed. This state vector is input into a trained neural network model, which outputs target speed commands for each servo motor. The commands are transmitted to the corresponding servo drivers via the EtherCAT bus to drive the servo motors to adjust the rotational speed of the hydraulic pumps. Simultaneously, the corresponding electromagnetic directional valves are dynamically activated or deactivated according to the current control requirements, allowing some hydraulic pumps to enter a zero-displacement bypass state. This allows for precise control of the system energy input by changing the number of effective oil supply pumps without increasing the additional flow supply.

Images