Intelligent servo control system for vibration suppression of sluice gate based on deep reinforcement learning

Abstract

The application belongs to the technical field of automatic control of water conservancy projects, and particularly relates to an intelligent servo control system for vibration suppression of a sluice gate based on deep reinforcement learning. The system comprises: a multi-dimensional situation awareness subsystem for real-time acquisition of operating environment, structural response and actuator parameters; a digital twin dynamic modeling subsystem for constructing a virtual simulation model and predicting a vibration trend; a deep reinforcement learning intelligent decision-making subsystem for generating an optimized control strategy using a reinforcement learning algorithm; and a high-frequency precision servo execution subsystem for driving a motor for compensation adjustment to actively suppress vibration. Through deep integration of digital twin and deep reinforcement learning, the application realizes real-time phase hedging of water flow excitation, solves the problems of low control accuracy and large vibration of the gate under complex working conditions, and significantly improves the robustness, safety and self-evolution ability of the system.

Description

Technical Field

[0001] This invention belongs to the field of automatic control technology for water conservancy projects, specifically relating to an intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning. Background Technology

[0002] As a vital infrastructure of the national economy, water conservancy projects directly impact flood control, drainage, agricultural irrigation, and the optimal allocation of water resources through their operational stability and safety. In modern water conservancy projects, sluice gates, as key structures for regulating water levels and controlling flow, have seen their automation level and operational precision become crucial standards for evaluating the effectiveness of smart water conservancy construction. With the continuous development of sensor technology, the Industrial Internet, and automation control theory, intelligent monitoring and precision servo control of water conservancy facilities are gradually becoming core research directions for ensuring the long-term operation of large-scale water conservancy projects.

[0003] Vibration suppression and intelligent servo control of sluice gates are key technologies for ensuring the safety of hydraulic structures and extending the service life of opening and closing equipment. During the opening and closing or partial opening of the gate, the gate system often experiences severe vibrations due to the dynamic excitation of the high-speed downstream water flow, friction in the mechanical transmission system, and complex fluid-structure interaction. The basic goal of the intelligent servo control system is to precisely adjust the gate's motion posture using high-precision actuators and real-time feedback algorithms, eliminating harmful vibrations through dynamic compensation of disturbance torques, thereby achieving smooth switching and precise position locking during gate operation.

[0004] Existing technologies typically employ traditional proportional-integral-derivative (PID) control or fuzzy control methods based on fixed rules. However, these methods have significant limitations when dealing with complex systems like sluice gates, which exhibit strong nonlinearity, high coupling, and time-varying parameters. Limited by their reliance on precise mathematical models, traditional control schemes struggle to capture and adapt to changing hydrological conditions and complex mechanical wear states in real time, resulting in poor vibration suppression and limited response speed under varying operating conditions. Furthermore, existing servo control strategies lack the adaptive and self-evolving capabilities unique to deep learning, failing to extract deep-level control features from massive amounts of historical operating data. This makes them prone to control instability when faced with extreme sudden loads or sensor noise interference. These technical shortcomings lead to fatigue damage in the gate structure, increasing maintenance costs and creating safety hazards. Therefore, developing an intelligent servo control system with strong robustness and self-learning capabilities has become a pressing technical challenge in the field of hydraulic equipment control. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning, so as to solve the technical problems of insufficient control accuracy, poor vibration suppression effect, weak robustness and lack of self-evolution ability in the face of strong nonlinearity, high coupling and time-varying parameter characteristics of sluice gates.

[0006] The technical solution of this invention includes: a multi-dimensional situational awareness subsystem for real-time acquisition of the operating environment parameters, structural response parameters, and actuator state parameters of the sluice gate; a digital twin dynamic modeling subsystem for constructing and updating a virtual simulation model of the sluice gate in real-time based on the parameters acquired by the multi-dimensional situational awareness subsystem, so as to predict the stress state and vibration trend of the gate; a deep reinforcement learning intelligent decision-making subsystem for generating an optimized control strategy for vibration suppression based on the prediction output of the digital twin dynamic modeling subsystem and the real-time feedback of the multi-dimensional situational awareness subsystem using a deep reinforcement learning algorithm; and a high-frequency precision servo execution subsystem for receiving the control strategy instructions from the deep reinforcement learning intelligent decision-making subsystem, driving the servo motor to compensate and adjust the gate position and motion attitude, thereby achieving active suppression of gate vibration.

[0007] Furthermore, the multi-dimensional situational awareness subsystem includes an environmental sensing unit, a structural sensing unit, and a state feedback unit. The environmental sensing unit includes pressure level gauges and radar flow meters installed upstream and downstream of the sluice gate to acquire the upstream and downstream water level difference and instantaneous flow velocity through the gate. The structural sensing unit includes a three-axis accelerometer, a high-sensitivity dynamic strain gauge, and an acoustic emission sensor installed on the gate body and support arm to capture the gate's vibration frequency, amplitude, and microcrack evolution signals within the material. The state feedback unit includes a high-precision absolute encoder integrated into the end of the servo motor and a multi-functional power analyzer to monitor the gate's absolute opening, operating speed, motor current, and output torque in real time. The multi-dimensional situational awareness subsystem synchronously transmits all acquired raw signals to a data preprocessing center via an industrial Ethernet bus for normalization processing and wavelet transform-based denoising analysis.

[0008] In one embodiment of the present invention, the digital twin dynamic modeling subsystem includes a geometric mapping module, a physical parameter identification module, and a fluid-structure interaction simulation module. The geometric mapping module establishes a 1:1 3D solid model based on the original design parameters of the sluice gate. The physical parameter identification module uses real-time data transmitted from the structural sensing unit to identify and correct the equivalent stiffness coefficient, equivalent damping coefficient, and bearing friction coefficient of the model online using the recursive least squares method. The fluid-structure interaction simulation module uses computational fluid dynamics methods, combined with the water flow parameters obtained from the environmental sensing unit, to calculate the distribution of the dynamic excitation force of the discharged water flow on the bottom edge of the gate. The digital twin dynamic modeling subsystem performs advanced simulation with a period of 50 milliseconds, outputting the predicted vibration response value of the gate under the current control command within the next sampling period.

[0009] In one embodiment of the present invention, the deep reinforcement learning intelligent decision-making subsystem adopts a deep reinforcement learning architecture based on the actor-commentator framework, specifically including a state space construction module, an action space definition module, a reward function generation module, and a neural network training module. The state space construction module concatenates the real-time feedback data from the multi-dimensional situational awareness subsystem with the predicted output of the digital twin dynamic modeling subsystem to form a multi-dimensional vector. The action space definition module outputs the angular acceleration compensation and dynamic torque correction values ​​of the servo motor. The reward function generation module constructs a comprehensive evaluation function, which includes a position tracking error penalty term, a vibration intensity penalty term, a control quantity smoothness reward term, and an energy efficiency consumption penalty term. The neural network training module employs a proximal policy optimization algorithm, continuously optimizing the parameters of the actor network and the commentator network through large-scale parallel interactive learning within the digital twin model.

[0010] Furthermore, the learning process of the deep reinforcement learning intelligent decision-making subsystem is as follows: The actor network outputs an action instruction based on the current state vector. The critic network evaluates the value score of the action based on environmental feedback. The system calculates the real-time reward value based on the reward function. The advantage function is calculated using the generalized advantage estimation method. The network weights are updated using the gradient descent algorithm, where the objective function incorporates a pruning mechanism to limit the probability ratio of the new and old policies to between 0.8 and 1.2, ensuring the stability of policy updates.

[0011] In one embodiment of the present invention, the high-frequency precision servo execution subsystem includes a servo driver, a permanent magnet synchronous servo motor, a planetary gear reducer, and a high-precision coupling. The servo driver adopts a three-loop nested control structure of current loop, speed loop, and position loop. The high-frequency precision servo execution subsystem receives control commands from the deep reinforcement learning intelligent decision-making subsystem and injects them as feedforward compensation quantities into the speed loop and current loop. By adjusting the electromagnetic torque of the servo motor, a compensation load with the opposite direction and phase cancellation to the water flow excitation force is generated, thereby canceling the micro-vibration of the gate and ensuring high-precision positioning of the gate's macro-opening.

[0012] In another embodiment of the present invention, the system further includes a safety constraint and self-healing adjustment unit. This unit sets hard limit boundaries and a maximum permissible vibration threshold for the gate's operation. When the instructions output by the deep reinforcement learning intelligent decision-making subsystem may cause the gate to exceed physical limits or generate resonance risk, the safety constraint and self-healing adjustment unit intervenes, employing a sliding mode control algorithm based on Lyapunov stability theory to take over the actuator and forcibly pull the gate's operating state back to a safe region. Simultaneously, the self-healing adjustment unit marks the abnormal control sequences that trigger safety intervention and feeds them back as negative samples to the deep reinforcement learning intelligent decision-making subsystem for offline reinforcement training, achieving self-correction of the control logic.

[0013] Furthermore, the data sampling frequency of the multi-dimensional situational awareness subsystem is set to 1000 Hz. The data preprocessing center uses a Kalman filter algorithm to achieve spatiotemporal alignment and fusion of multi-source sensor data. The water level difference data provided by the environmental perception unit needs to have an accuracy of 0.001 meters. The acceleration measurement range provided by the structural perception unit needs to cover 0.1 Hz to 500 Hz.

[0014] Furthermore, the physical parameter identification module of the digital twin dynamic modeling subsystem performs a global optimization of model parameters every 300 seconds to adapt to long-term drift in gate mass and damping characteristics caused by mechanical wear, siltation, or aquatic organism attachment. The fluid-structure interaction simulation module adopts a simplified large eddy simulation algorithm to meet real-time requirements while ensuring computational accuracy.

[0015] Furthermore, the neural network structure of the deep reinforcement learning intelligent decision-making subsystem includes one input layer, five fully connected hidden layers, and one output layer. Each hidden layer contains 256 neurons and uses a modified linear unit as the activation function. To prevent overfitting, a random deactivation layer with a dropout rate of 0.2 is introduced between the hidden layers. The system stores historical state transition data through an experience replay pool with a storage capacity of 100,000 trajectories and employs a priority experience replay mechanism to improve the learning efficiency of rare samples.

[0016] Furthermore, the communication latency of the high-frequency precision servo execution subsystem is controlled to within 2 milliseconds. The servo driver integrates an active disturbance rejection control algorithm based on a disturbance observer to eliminate nonlinear frictional resistance within the actuator. The rated speed of the permanent magnet synchronous servo motor must adapt to the fast and slow speed switching requirements of the gate opening and closing; its overload multiple is set to 3 times to cope with sudden transient water hammer loads.

[0017] As one embodiment of the present invention, the system also includes a cloud-based remote monitoring and centralized management platform. The cloud platform connects to the field control system via a mobile communication network or satellite link, enabling massive storage and cross-regional scheduling of operational data. The cloud platform utilizes ensemble learning algorithms to perform group learning on control data from multiple sluice gates, generating control model weights with broader adaptability, and periodically distributes these weights to the field deep reinforcement learning intelligent decision-making subsystem via firmware remote upgrade technology.

[0018] Furthermore, during the gate startup phase, the system predicts the static friction torque through a digital twin dynamic modeling subsystem. A deep reinforcement learning intelligent decision-making subsystem pre-calculates a torque feedforward, ensuring the servo motor overcomes static friction without significant startup jerking. During the constant current phase of partial gate opening, the system focuses on monitoring periodic vibrations caused by the Karman vortex street and achieves vibration cancellation by adjusting the servo motor's minute pulsation frequency. During the gate closing and bottoming phase, the system achieves soft-landing control through sensitive monitoring via the current loop, preventing structural damage caused by bottom impact.

[0019] This invention provides an intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning. Its operational logic covers the entire process from physical perception to model building, and then to intelligent decision-making and precise execution. The system uses digital twin technology to compensate for the randomness risk of reinforcement learning in the early stages of physical environment exploration, and uses a high-frequency servo actuator to transform complex mathematical optimization strategies into real-time physical actions.

[0020] Compared with the prior art, the advantages and positive effects of the present invention are as follows:

[0021] 1. This invention, through the deep integration of deep reinforcement learning algorithms and digital twin technology, completely eliminates the reliance of traditional control methods on precise analytical mathematical models. The digital twin dynamic modeling subsystem can track and predict the nonlinear dynamic evolution of sluice gates in real time, providing a highly realistic policy training and evaluation environment for deep reinforcement learning. This enables the system to exhibit strong adaptability and robustness when facing complex changes in hydrological conditions and drift in mechanical parameters, effectively solving the technical bottleneck of accuracy loss in traditional proportional-integral-derivative control under varying conditions.

[0022] 2. The active suppression mechanism introduced in this invention significantly improves the vibration control effect of the sluice gate. Unlike passive vibration damping or simple fuzzy rule adjustment, this system directly injects the optimized compensation torque generated by deep reinforcement learning into the servo control loop through a high-frequency precision servo execution subsystem. This architecture, based on a combination of feedforward compensation and feedback adjustment, can achieve real-time phase offsetting of the dynamic excitation of the downstream water flow, reducing the vibration intensity of the gate in the critical operating range by more than 60%, greatly alleviating fatigue damage to the gate structure, extending the service life of the equipment, and reducing long-term operation and maintenance costs.

[0023] 3. The system architecture design of this invention possesses superior security and self-evolution capabilities. By establishing security constraints and self-healing adjustment units, a safety barrier is provided for the strategy exploration of deep reinforcement learning, ensuring the inherent safety of water conservancy projects under extreme sudden loads or algorithm failures. Simultaneously, the system has the ability to learn from massive amounts of historical operational data and security intervention events, achieving continuous self-evolution and optimization of control strategies. This model of continuously improving local control efficiency through group learning and cloud iteration represents the core direction of smart water conservancy's transformation from digitalization to intelligence, possessing extremely high engineering application value and technological promotion significance.

[0024] 4. This invention achieves comprehensive digital monitoring of the gate's operational status through a multi-dimensional situational awareness subsystem. By fusing and analyzing heterogeneous data such as water level, flow velocity, vibration, strain, and current, the system can not only precisely suppress vibrations but also provide early warnings of potential mechanical failures, material fatigue, and foreign object obstruction. This full-chain, closed-loop intelligent servo control solution significantly improves the automation level and precision management capabilities of large-scale water conservancy projects, providing solid technical support for building a smart water conservancy security system. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall technical solution architecture of the intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning proposed in this invention.

[0026] Figure 2 This is a schematic diagram of the core principle framework of the deep reinforcement learning intelligent decision-making subsystem in this invention;

[0027] Figure 3 This is a logical flow diagram of the digital twin dynamic modeling subsystem in this invention;

[0028] Figure 4 This is a schematic diagram of the multi-level interaction relationships and data flow of the multi-dimensional situational awareness subsystem in this invention;

[0029] Figure 5 This is a logical flow diagram of the safety constraint and self-healing adjustment unit in this invention. Detailed Implementation

[0030] Example 1

[0031] Please refer to the attached document. Figure 1 This embodiment provides an intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning. The overall architecture of this system aims to solve the problem of strong nonlinear hydraulic vibration encountered by sluice gates in large-scale water conservancy projects during opening, closing, and flow. The entire system deeply integrates core technologies such as multi-dimensional situational awareness, digital twin dynamic simulation, deep reinforcement learning intelligent decision-making, and high-frequency precision servo execution. The system captures real-time operational data from the physical world, performs deduction and strategy optimization in a virtual digital space, and then sends the optimized control sequence to the actuators to achieve precise and active suppression of the gate's micro-vibrations.

[0032] Please refer to the attached document. Figure 4 The multi-dimensional situational awareness subsystem, serving as the data foundation of the entire system, is responsible for comprehensively collecting physical parameters affecting the stability of the gate's operation. This subsystem is specifically divided into an environmental sensing unit, a structural sensing unit, and a state feedback unit. The environmental sensing unit deploys high-precision pressure level gauges on both the upstream and downstream water level sides of the sluice gate. These gauges utilize the piezoresistive sensor principle, with a measurement range of 0 to 20 meters and a measurement accuracy better than 0.001 meters. Simultaneously, a radar flow velocity meter is installed on the frame bridge near the gate outlet, utilizing the Doppler effect to monitor the instantaneous flow velocity through the gate in real time without contact. These environmental parameters constitute the macroscopic boundary conditions of the gate's stress.

[0033] The structural sensing units are deployed at key stress-bearing locations on the gate body and support arms. Among them, the three-dimensional accelerometers utilize piezoelectric ceramic technology to simultaneously acquire horizontal, vertical, and lateral vibration acceleration signals, with a frequency response range covering 0.1 Hz to 500 Hz. High-sensitivity dynamic strain gauges are attached to the skin surfaces of the gate's main beam and support arms, using the Wheatstone bridge principle to monitor dynamic stress changes within the structure in real time. Furthermore, acoustic emission sensors are installed in weld areas prone to fatigue cracking to capture ultrasonic signals released during the evolution of microcracks within the material. These microstructural response parameters provide direct evidence for subsequent vibration assessments.

[0034] The status feedback unit is integrated at the end of the high-frequency precision servo execution subsystem. The high-precision absolute encoder employs 23-bit multi-turn absolute measurement technology, enabling it to provide feedback on the absolute opening degree and real-time operating speed of the gate with extremely high spatial resolution. A multi-functional power analyzer is connected in series in the servo motor's power cable circuit, monitoring the motor's three-phase current, voltage, active power, and output torque in real time. The data sampling frequency of the multi-dimensional situational awareness subsystem is uniformly set to 1000 Hz. Analog signals from all sensors are converted into digital signals through a distributed data acquisition module and synchronously transmitted to the data preprocessing center via an industrial Ethernet bus.

[0035] Within the data preprocessing center, the system first normalizes the raw signals, mapping physical quantities of different dimensions and orders of magnitude to the range of 0 to 1. Subsequently, the system applies a wavelet transform-based denoising algorithm. Specifically, the system uses discrete wavelet transform to perform multi-scale decomposition of the noisy signal, filtering out high-frequency random noise by setting a soft threshold, while fully preserving the low-frequency vibration characteristics generated by the gate's excitation. Furthermore, the data preprocessing center utilizes a Kalman filter algorithm to perform spatiotemporal alignment and fusion of sensor data from different physical properties, such as water level gauges, flow meters, and accelerometers. The state transition matrix of the Kalman filter is constructed based on the physical laws of gate motion, continuously correcting the deviation between predicted and observed values ​​to output the optimal state estimation vector.

[0036] Please refer to the attached document. Figure 3 The digital twin dynamic modeling subsystem receives fused data from the multi-dimensional situational awareness subsystem. This subsystem includes a geometric mapping module, a physical parameter identification module, and a fluid-structure interaction simulation module. Based on the original design parameters, material properties, and 3D scan data of the sluice gate, the geometric mapping module establishes a 1:1 scale 3D solid model in virtual space. This model not only includes the external geometric contour of the gate but also simulates in detail the topological connections of the gate's main beam, transverse diaphragms, support arms, and traveling wheels.

[0037] The physical parameter identification module is crucial for maintaining the realism of the digital twin model. Considering the effects of siltation, aquatic organism attachment, and mechanical wear on sluice gates during long-term operation, their mass distribution, stiffness characteristics, and damping coefficients will slowly drift. This system utilizes real-time acceleration and strain data transmitted from the structural sensing unit, employing a recursive least squares method to identify and correct the model's equivalent stiffness coefficient, equivalent damping coefficient, and support friction coefficient online. The physical parameter identification module performs a global optimization of model parameters every 300 seconds, ensuring a high degree of consistency between the virtual model and the dynamic characteristics of the physical gate.

[0038] The fluid-structure interaction (FSI) simulation module utilizes computational fluid dynamics (CFD) methods, combined with instantaneous water level differences and gate-passing velocities obtained from the environmental sensing unit. To balance simulation accuracy and real-time performance, the FSI module employs a simplified large eddy simulation (LES) algorithm. This algorithm can capture the Karman vortex street generated when water flows over the bottom edge of the gate and the resulting unsteady dynamic excitation force distribution. The digital twin dynamic modeling subsystem performs advance simulations with a period of 50 milliseconds. This advance simulation means that at the current sampling moment, the system has already predicted the displacement trajectory, vibration frequency, and amplitude of the gate under the combined action of the current water flow impact force and the proposed control commands within the next sampling period.

[0039] Please refer to the attached document. Figure 2 The deep reinforcement learning intelligent decision-making subsystem serves as the command center of the entire system. This subsystem employs a deep reinforcement learning architecture based on the actor-critic framework, specifically divided into a state space construction module, an action space definition module, a reward function generation module, and a neural network training module. The state space construction module concatenates the real-time state vector fed back by the multi-dimensional situational awareness subsystem with the advanced prediction vector output by the digital twin dynamic modeling subsystem to construct a high-dimensional feature space. This feature space comprehensively describes the gate's current physical attitude, hydraulic load, and future vibration evolution trend.

[0040] The output of the motion space definition module is directly related to the actuator. Its output is defined as the angular acceleration compensation and dynamic torque correction value of the servo motor. This continuous motion space setting allows the system to perform micron-level fine-tuning of gate vibration. The reward function generation module is the core logic guiding the algorithm's learning. It constructs a comprehensive evaluation function to quantify the quality of each set of control actions.

[0041]

[0042] In the above formula, This represents the real-time reward value. This is for position tracking error, used to ensure the accuracy of the gate's macroscopic opening. Vibration intensity, i.e., the instantaneous amplitude obtained by the accelerometer, is used to minimize the structural vibration of the gate. To control the changes in commands, the system is encouraged to output continuous and smooth control signals to avoid mechanical shocks to the actuators. To control the motor's electrical energy consumption, the system is guided to find the most energy-efficient control strategy. Weighting coefficients are used for each component. - Offline calibration is performed according to specific project requirements.

[0043] The neural network training module employs a proximal policy optimization algorithm. Its network structure consists of one input layer, five fully connected hidden layers, and one output layer. Each hidden layer integrates 256 neurons, all using a modified linear unit (MLU) as the activation function. To enhance the model's generalization ability and prevent overfitting, a random deactivation layer with a dropout rate of 0.2 is introduced between the hidden layers. During training, the actor network generates control actions based on the current state vector, while the critic network evaluates the value score of these actions in the current environment. The system utilizes an experience replay pool with a storage capacity of 100,000 trajectories to store historical state transition data and employs a priority experience replay mechanism, assigning higher sampling weights to rare samples with significant vibration suppression effects or abnormal fluctuations, thereby improving learning efficiency.

[0044] The specific learning process of the deep reinforcement learning intelligent decision-making subsystem follows the gradient descent logic. The system uses the generalized advantage estimation method to calculate the advantage function and update the network weights. To ensure the stationarity of policy updates, a pruning mechanism is introduced into the objective function.

[0045]

[0046] In this formula, Let be the policy loss function. For the expectation operator, The ratio of the probabilities of the new and old strategies is given. This is the estimated value of the dominance function. For the cropping operation, For the clipping boundary parameters, This represents the probability ratio between the old and new strategies, and this value is strictly limited to between 0.8 and 1.2. This mechanism ensures that the control strategy will not change drastically due to a single sampling error in complex nonlinear hydraulic environments.

[0047] The high-frequency precision servo execution subsystem receives commands from the decision subsystem. This subsystem's hardware components include a servo driver, a permanent magnet synchronous servo motor, a planetary gearbox, and a high-precision coupling. The servo driver employs a three-loop nested closed-loop control structure (current loop, speed loop, and position loop) and integrates an active disturbance rejection control algorithm based on a disturbance observer. This algorithm can identify and eliminate the effects of nonlinear frictional resistance and backlash within the planetary gearbox online.

[0048] When the deep reinforcement learning intelligent decision-making subsystem issues an optimized torque correction value for vibration suppression, this signal is directly injected into the speed loop and current loop of the servo driver as a feedforward compensation. The permanent magnet synchronous servo motor dynamically adjusts its electromagnetic torque according to this instruction, generating a compensating load with the same frequency but opposite phase to the water flow excitation force. This phase cancellation mechanism allows the gate to achieve dynamic balance through the motor's microsecond-level torque response when subjected to water flow impact, thereby physically canceling the gate's micro-vibrations. The overload multiple of the permanent magnet synchronous servo motor is set to 3 times, which is sufficient to cope with sudden transient water hammer loads under extreme flood discharge conditions.

[0049] Please refer to the attached document. Figure 5 The safety constraint and self-healing adjustment unit is crucial for ensuring the inherent safety of the system. This unit operates independently of the reinforcement learning algorithm and is equipped with hard limit boundaries for gate operation and a maximum permissible vibration threshold. During the initial exploration phase of reinforcement learning or when encountering unprecedented extreme conditions, if the instructions output by the decision subsystem might cause the gate to exceed its travel limits or potentially excite the gate structure's natural frequencies, leading to resonance risks, the safety constraint unit will immediately intervene. At this point, the system switches to a sliding mode control algorithm based on Lyapunov stability theory to take over the actuator. The sliding mode control algorithm possesses extremely strong robustness and can forcibly pull the gate's operating state back to a preset safe region.

[0050] The self-healing adjustment unit digitally labels abnormal control sequences that trigger safety interventions. These failed samples are stored in a negative sample library and periodically fed back to the deep reinforcement learning intelligent decision-making subsystem for offline reinforcement training. Through this negative reinforcement learning, the system can identify which control combinations lead to risks, thereby automatically avoiding similar control blind spots in future online operations and achieving self-correction and evolution of the control logic.

[0051] This system is also equipped with a cloud-based remote monitoring and centralized management platform. This platform connects to the control sites of sluice gates distributed across various locations via 5G mobile communication networks or satellite links. The cloud platform enables massive storage of operational data, historical curve analysis, and cross-regional expert diagnostics. More importantly, the cloud platform utilizes ensemble learning algorithms to perform group learning on control data from sluice gates of different specifications and in different river basins. By extracting common control features, the cloud platform can generate initial weights for a more adaptable control model, which are periodically distributed to each field system via firmware remote upgrade technology, significantly shortening the on-site deployment and model convergence time for newly installed systems.

[0052] During the gate start-up phase, traditional control methods often suffer from start-up jumps due to static friction. This system uses a digital twin dynamic modeling subsystem to predict the adsorption force at the gate's bottom edge and the static friction torque at the support. A deep reinforcement learning intelligent decision-making subsystem provides a torque feedforward before the motor actuates, ensuring the servo motor maintains stable torque at the moment of overcoming static friction, thus achieving a smooth, shock-free start. During the constant current phase of partial gate opening, the system focuses on intervening in the periodic excitation caused by the Karman vortex street. By capturing the phase of the acceleration signal at high frequency, the servo motor outputs a small pulsating frequency to achieve real-time counterbalancing of vibration energy. During the gate closing and bottom-touching phase, the system utilizes the sensitive monitoring of the servo driver's current loop to switch to torque control mode the instant the gate's bottom edge contacts the sill, achieving a soft landing and preventing cumulative damage to the gate structure and sill caused by mechanical impact.

[0053] Example 2

[0054] Based on the intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning described in Example 1, this embodiment further discloses how the system maintains high-precision vibration suppression by dynamically adjusting the sensing weights and decision logic under extremely non-uniform flow field conditions. In some extra-large water conservancy projects, the water level difference on both sides of the gate may reach more than 30 meters, and due to the influence of topography, the downstream water flow often exhibits strong asymmetry and vortex flow characteristics.

[0055] To address these operational conditions, the structure sensing unit within the multi-dimensional situational awareness subsystem incorporates distributed fiber Bragg grating sensors. These sensors, quasi-distributedly arranged on the main gate frame, provide strain field distribution maps with higher spatial resolution. The fiber optic sensor data is converted into wavelength drift using a demodulator, while the sampling frequency remains at 1000 Hz. The data preprocessing center incorporates a flow field reconstruction algorithm, utilizing limited water level and velocity measurement data, combined with a convolutional neural network, to infer the pressure distribution on the gate surface in real time.

[0056] Under these complex operating conditions, the fluid-structure interaction simulation module of the digital twin dynamic modeling subsystem dynamically refines the computational mesh locally. When a high-intensity pressure pulsation is detected at a certain point on the bottom edge of the gate, the simulation module automatically refines the computational nodes in that area to more accurately simulate the excitation effect of small-scale eddies on the structure. The physical parameter identification module introduces a variable-step recursive least squares method to address the phenomenon of instantaneous and drastic changes in the mass of the water added to the gate caused by high-speed water flow scouring.

[0057] The deep reinforcement learning intelligent decision-making subsystem incorporates a flow field non-uniformity operator into the state space construction. This operator quantifies the degree of force imbalance on the left and right sides of the gate. The action space is expanded to include differential compensation commands for the dual-sided synchronous actuators. In this embodiment, the sluice gate is driven by two sets of completely symmetrical permanent magnet synchronous servo motors. The deep reinforcement learning decision-making subsystem can issue different compensation torques to the left and right motors based on the gate's deflection angle and vibration phase difference.

[0058]

[0059]

[0060] in, This provides the total torque output for the left-side servo motor. This provides the total torque output for the servo motor on the right. The basic torque required to maintain the opening This is the vibration suppression correction amount. The first correction is used for overall vibration suppression, and the second correction is used to correct the gate's attitude deviation. This differential control strategy effectively solves the problem of gate jamming or tilting vibration caused by unilateral excitation.

[0061] Based on Example 1, the reward function adds a penalty term for attitude deviation. If the synchronization error between the left and right motors exceeds 0.5 mm, the reward value will decrease significantly. The neural network training module employs transfer learning technology. The cloud platform pushes pre-trained, mature general model weights under standard operating conditions to the field, and the field system only needs to perform incremental learning for the current extremely non-uniform flow field. This method reduces the model convergence time from the usual 24 hours to less than 30 minutes.

[0062] The communication protocol of the high-frequency precision servo actuator subsystem employs time-sensitive networking technology with hard real-time characteristics. This technology ensures that the data synchronization deviation between the left and right drives is controlled within 1 microsecond, thereby guaranteeing the synchronization of the compensation load application. The planetary gearbox uses a zero-backlash precision-grade product, coupled with high-sensitivity current feedback, which can capture and compensate for high-frequency minute jitters up to 120 Hz.

[0063] The safety constraint and self-healing adjustment unit automatically tightens safety limits under extreme operating conditions. When the support stress on one side of the gate is detected to be close to 70% of the material's yield strength, the system will force entry into a pressure-maintaining state, and the self-healing adjustment unit will initiate an emergency sand discharge procedure or adjust the upstream water diversion flow to reduce the structural load on the gate by changing the external environment. The self-healing adjustment unit will also record the flow field characteristics when such extreme events occur, generate a protection knowledge base, and synchronize it to the cloud.

[0064] The cloud-based remote monitoring and centralized management platform played a crucial role in this collaborative operation. By accessing flow data from upstream sluice gates along the same waterway, it predicted the peak flood flow reaching the station 15 minutes in advance. The deep reinforcement learning intelligent decision-making subsystem utilized this predictive information to adjust the neural network's attention mechanism in advance, shifting the control weights from energy conservation to structural safety. During the flood peak, the system sacrificed some energy efficiency to maximize vibration suppression, ensuring the safe and stable operation of the sluice gates under conditions exceeding design specifications.

[0065] This embodiment also incorporates a fault-tolerance mechanism to address sensor aging issues during long-term operation. When an accelerometer in the multi-dimensional situational awareness subsystem experiences signal interruption or data anomalies, the data preprocessing center automatically utilizes virtual sensing technology based on a deep autoencoder to synthesize a high-confidence virtual observation value based on readings from other related sensors and the deduction results of the digital twin model. This virtual value temporarily replaces the output of the faulty sensor, ensuring that the decision-making subsystem does not experience control logic collapse until maintenance personnel complete the hardware replacement.

[0066] In terms of the full lifecycle management of gates, the cloud platform utilizes accumulated vibration and strain data, combined with fatigue fracture mechanics models such as the Paris Law, to calculate the residual fatigue life of key gate welds in real time. When the predicted remaining life falls below the warning threshold, the system automatically introduces damage mitigation constraints into the control strategy, maximizing the extension of the gate's service life by limiting its operation time in the high vibration range. This intelligent closed loop, extending from real-time vibration control to long-term lifecycle management, demonstrates the high degree of integration of this invention in smart water conservancy applications.

[0067] Example 3

[0068] Combined with appendix Figures 1 to 5 This embodiment further illustrates the application details of the intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning in river conditions with high sediment loads. In such environments, sediment accumulation at the bottom of the gate causes nonlinear changes in the gate's opening and closing resistance, and the abrasive effect of the sediment alters the dynamic response characteristics of the servo actuator.

[0069] In this operating condition, the multi-dimensional situational awareness subsystem is equipped with a sediment concentration monitor and an ultrasonic sedimentation detection array deployed at the bottom edge of the gate. The ultrasonic array measures the sedimentation thickness and profile shape at the bottom of the gate by measuring the time difference between transmitting and receiving reflected waves. This data is transmitted in real time to the data preprocessing center. The data preprocessing center uses multi-modal fusion technology to combine sediment concentration with flow velocity and water level difference to calculate the scouring force of the fluid on the gate and the additional resistance coefficient caused by changes in sediment density.

[0070] The physical parameter identification module of the digital twin dynamic modeling subsystem incorporates a mass compensation algorithm. Since sediment may adhere to the gate structure, increasing the gate's equivalent mass, the physical parameter identification module analyzes the output torque variation of the servo motor under the same angular acceleration and, using recursive least squares, extracts the mass increment caused by sediment adhesion in real time. The fluid-structure interaction simulation module adds a solid particle term to the fluid dynamics calculations to simulate the abrasive impact of high-sediment-laden water flow on the gate wall.

[0071] The state space of the deep reinforcement learning intelligent decision-making subsystem has been further expanded to include the current thickness of sediment deposition and the number of historical scouring cycles. A new oscillation-based dredging mode for the servo motor has been added to the action space. When the system detects an abnormal increase in the gate's starting torque and determines it to be due to sediment blockage, the decision-making subsystem outputs a high-frequency, small-amplitude reciprocating torque sequence to drive the gate to oscillate slightly. This oscillation effectively disrupts the consolidation of the sediment, achieving drag-reduced starting.

[0072] The reward function has been modified specifically for this purpose, based on Example 1.

[0073]

[0074] In the formula, The comprehensive reward value taking into account siltation factors, Basic reward items, This is the additional increase in frictional resistance caused by silt and sand. This is the friction resistance penalty weighting coefficient. As a factor for dredging efficiency, A weighted coefficient is assigned to reward dredging effectiveness. The first penalty term relates to the additional frictional resistance caused by silt. The second reward term relates to flushing efficiency, encouraging the system to automatically remove bottom sediment using the kinetic energy of the downstream flow. The neural network training module simulated operating conditions with different silt densities in the offline phase, enabling the actor network to identify silt resistance characteristics.

[0075] The servo driver of the high-frequency precision servo execution subsystem has enabled torque ripple compensation. To address potential minor wear on the planetary gearbox in muddy environments, the system monitors the high-order harmonics of the current loop and injects a reverse compensation current to counteract transmission instability caused by mechanical wear. The permanent magnet synchronous servo motor's sealing rating has been upgraded to IP68, and it is equipped with a dedicated cooling circulation system to handle the additional heat generated under high-thrust dredging conditions.

[0076] The safety constraint and self-healing adjustment unit is equipped with a maximum starting torque protection for high sediment load conditions. If the resistance cannot be effectively reduced after three dredging commands, the system will determine it as a serious mechanical jam. In this case, the self-healing adjustment unit will cut off the automatic control, trigger an alarm signal to notify manual intervention, and automatically lock the gate position to prevent forced start-up from causing motor burnout or structural damage. Simultaneously, the self-healing adjustment unit will record all sensing data before and after the fault and generate a detailed fault diagnosis report.

[0077] The cloud-based remote monitoring and centralized management platform utilizes big data analytics to generate a dynamic dredging schedule for the sluice gate based on seasonal changes in sediment content. The platform also analyzes operational data from similar sluice gates in multi-sediment basins across the country, continuously optimizing the model weights of the decision-making subsystem. Through this collective intelligence, a single sluice gate can gain mature experience in handling complex sediment conditions.

[0078] During implementation, the data sampling frequency of the data preprocessing center was maintained at 1000 Hz, and edge computing nodes were introduced. These edge computing nodes, deployed at the sluice gate, were responsible for handling the computationally intensive wavelet denoising and initial feature extraction, uploading only the processed, simplified feature vectors to the decision subsystem. This distributed computing architecture further reduced the end-to-end control latency to less than 1.5 milliseconds, significantly improving the system's real-time response capability to high-frequency hydraulic excitation.

[0079] Furthermore, the multi-dimensional situational awareness subsystem in this embodiment also incorporates an environmental temperature compensation mechanism. Addressing the significant diurnal temperature variations characteristic of hydraulic engineering sites, the system uses temperature probes installed near each sensor to correct for zero-point drift in structural strain caused by thermal expansion and contraction, as well as density errors in the water level gauge. This extreme elimination of environmental interference ensures that the data source acquired by the deep reinforcement learning algorithm possesses extremely high purity and consistency.

[0080] Through the coordinated operation of the aforementioned subsystems, the entire system maintains optimal vibration suppression performance under complex physical environments and varying operating conditions. The self-evolutionary nature of deep reinforcement learning allows the system to gain a deeper understanding of the dynamic characteristics of this specific sluice gate as its operating time increases, thereby outputting increasingly precise and efficient servo control strategies. This control paradigm, based on a dual-drive approach of data-driven and physical model-driven approaches, represents the highest level of evolution in sluice gate control technology towards intelligence and precision.

[0081] The control system described in this embodiment, through precise adjustment of the high-frequency precision servo execution subsystem, can control the vibration intensity of the gate during flood discharge to below 0.2 mm / s, far exceeding the limits required by national standards. Simultaneously, due to the effective suppression of vibration, the dynamic stress amplitude of the gate's main beam is reduced by more than 45%, directly translating into a significant extension of the gate structure's fatigue life. The system not only achieves vibration suppression but also realizes micro-adjustment of the flow rate with an accuracy of 0.1 cubic meters per second through precise control of the motor torque. This integrated function, combining structural protection and precise flow control, provides strong technical support for the refined management of smart water conservancy.

[0082] In summary, this invention constructs an intelligent servo control closed loop with sensing, thinking, execution, and self-evolution capabilities through the deep coupling of deep reinforcement learning and digital twin technology. This system successfully solves the vibration control problem of sluice gates under highly nonlinear, highly coupled, and time-varying parameter conditions. Whether under conventional flow regulation, sudden flood peak impacts, or harsh conditions of heavy sediment scouring, the system demonstrates excellent control accuracy and robust operational reliability. The implementation of this technical solution not only improves the operational level of a single sluice gate but also provides a replicable and scalable paradigm for the intelligent upgrading of water conservancy projects across the entire basin and even the whole country through the collective learning capabilities of the cloud platform.

[0083] The embodiments described above are merely illustrative examples of the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention. Any equivalent substitutions, improvements, or structural changes made by those skilled in the art without departing from the concept of the present invention should be included within the scope of protection of the present invention. The connection methods, parameter settings, and algorithm selections between the various components and modules of the system can be optimized and adjusted according to actual engineering needs. The core value of the present invention lies in providing a universal intelligent control architecture with self-learning capabilities to address the increasingly complex dynamic control challenges in water conservancy projects.

Claims

1. A smart servo control system for sluice gate vibration suppression based on deep reinforcement learning, characterized in that, include: The multi-dimensional situational awareness subsystem is used to acquire in real time the operating environment parameters, structural response parameters, and actuator status parameters of the sluice gate. The digital twin dynamic modeling subsystem is used to construct and update the virtual simulation model of the sluice gate in real time based on the parameters obtained by the multi-dimensional situational awareness subsystem, so as to realize the prediction of the gate's stress state and vibration trend. A deep reinforcement learning intelligent decision-making subsystem is used to generate an optimized control strategy for vibration suppression based on the prediction output of the digital twin dynamic modeling subsystem and the real-time feedback of the multi-dimensional situational awareness subsystem using deep reinforcement learning algorithms. The high-frequency precision servo execution subsystem is used to receive the control strategy instructions from the deep reinforcement learning intelligent decision-making subsystem, drive the servo motor to compensate and adjust the gate position and motion posture, and realize the active suppression of gate vibration.

2. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 1, characterized in that, The multi-dimensional situational awareness subsystem includes: The environmental sensing unit is used to collect the water level difference between the upstream and downstream sides of the sluice gate and the instantaneous flow velocity through the gate; The structural sensing unit is used to capture the vibration frequency, amplitude, and internal damage evolution signals of the gate body and supporting structure. The status feedback unit is used to monitor the absolute opening, running speed, current and output torque of the servo motor.

3. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 2, characterized in that, The digital twin dynamic modeling subsystem includes: The geometry mapping module is used to create a three-dimensional solid model based on the original design parameters of the sluice gate. The physical parameter identification module is used to correct the equivalent stiffness, equivalent damping, and friction coefficient of the model online based on the real-time data returned by the structural sensing unit. The fluid-structure interaction simulation module is used to calculate the dynamic excitation force distribution of the discharged water flow on the gate in combination with water flow parameters, and output the predicted value of the vibration response in the future sampling period.

4. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 3, characterized in that, The deep reinforcement learning intelligent decision-making subsystem adopts an actor-critic architecture, including: The state space construction module is used to concatenate the real-time feedback data of the multi-dimensional situational awareness subsystem with the prediction output of the digital twin dynamic modeling subsystem to form a high-dimensional state vector. The motion space definition module is used to output the angular acceleration compensation amount and dynamic torque correction value of the servo motor; The reward function generation module is used to construct a comprehensive evaluation function that includes a position tracking error penalty term, a vibration intensity penalty term, a control quantity smoothness reward term, and an energy efficiency consumption penalty term. The neural network training module is used to optimize the parameters of the actor network and the critic network through interactive learning in a digital twin model.

5. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 4, characterized in that, The neural network training module employs a near-end policy optimization algorithm and introduces a pruning mechanism to limit the range of variation in the probability ratio between the old and new policies, thereby ensuring the stability of policy updates.

6. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 1, characterized in that, The high-frequency precision servo execution subsystem includes: The servo driver adopts a three-loop nested control structure consisting of a current loop, a speed loop, and a position loop. Permanent magnet synchronous servo motors are used to dynamically adjust electromagnetic torque according to control commands; Planetary gearboxes and high-precision couplings are used to accurately transmit motor output to the gate drive mechanism; The servo driver injects the control commands from the deep reinforcement learning intelligent decision-making subsystem as feedforward compensation into the velocity loop and current loop to generate a compensation load that is opposite in phase to the water flow excitation force.

7. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 1, characterized in that, It also includes a safety constraint and self-healing adjustment unit, which is used to set the hard limit boundary and maximum allowable vibration threshold of the gate operation, and when the control command is detected to cause over-limit or resonance risk, it switches to the sliding mode control algorithm based on Lyapunov stability theory to take over the actuator and forcibly pull the gate state back to the safe area.

8. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 7, characterized in that, The self-healing adjustment unit is also used to mark the abnormal control sequence that triggers safety intervention and feed it back as a negative sample to the deep reinforcement learning intelligent decision-making subsystem for offline reinforcement training, so as to realize the self-correction of the control logic.

9. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 1, characterized in that, It also includes a cloud-based remote monitoring and centralized management platform, which connects to the field control system through a communication network to achieve massive storage and cross-regional scheduling of operational data. It also uses ensemble learning algorithms to perform group learning on the control data of multiple sluice gates, generate general control model weights, and distribute them to the field system via remote upgrade.

10. The intelligent servo control system for sluice gate vibration suppression based on deep reinforcement learning according to claim 1, characterized in that, During the gate start-up phase, the system predicts the static friction torque through the digital twin dynamic modeling subsystem, and the deep reinforcement learning intelligent decision-making subsystem outputs the torque feedforward in advance to achieve shock-free start-up; during the local opening constant overcurrent phase, the system adjusts the micro-pulse frequency of the servo motor to counteract the periodic vibration caused by the Karman vortex street; during the closing bottoming-out phase, the system switches to torque control mode based on current loop monitoring to achieve a soft landing.

Images