Unit efficiency-life-stability three-dimensional collaborative optimization method, system and equipment based on deep reinforcement learning, and medium

Abstract

The invention discloses a unit efficiency-life-stability three-dimensional collaborative optimization method, system and device based on deep reinforcement learning, and a medium, and belongs to the technical field of collaborative optimization, and the method comprises the steps: collecting operation data and a power grid regulation and control instruction of a hydropower station unit, and constructing a state space and an action space; based on the state space and the action space, constructing an efficiency-life-stability three-dimensional collaborative optimization model; constructing a layered dynamic reward function; the three-dimensional collaborative optimization model is trained; based on a multi-working-condition strategy switching and strategy model updating mechanism, control strategy switching and online updating in different operation modes are completed. According to the method, the efficiency-life-stability three-dimensional dynamic game optimization model is constructed through deep reinforcement learning, multi-target weight autonomous adjustment under the change of working conditions such as water heads and loads is realized, the comprehensive optimization efficiency is improved, the fatigue damage of the runner is reduced, and the vibration intensity peak value is reduced; and the problems of equipment life loss and stability deterioration caused by single-target optimization are effectively solved.

Description

Technical Field

[0001] This invention relates to the field of collaborative optimization technology, specifically to a three-dimensional collaborative optimization method, system, equipment, and medium for unit efficiency-lifetime-stability based on deep reinforcement learning. Background Technology

[0002] Traditional hydropower station unit control strategies typically focus on a single objective (such as maximizing power generation efficiency), neglecting equipment lifespan degradation and operational stability. Existing multi-objective optimization techniques often employ fixed-weight superposition, making it difficult to dynamically adapt to complex operating conditions of hydropower station units (such as head fluctuations, sudden load changes, and abnormal vibrations), and lacking the ability to accurately model the nonlinear coupling relationship between efficiency, lifespan, and stability. Furthermore, traditional control methods (such as PID control and static scheduling rules) are insufficiently adaptable to the highly nonlinear dynamic characteristics of the multi-field coupling between hydraulic, mechanical, and electrical systems in hydropower stations, leading to optimization lag, accelerated lifespan degradation, or excessive vibration. Summary of the Invention

[0003] In view of the above-mentioned problems, the present invention is proposed.

[0004] Therefore, the technical problem solved by this invention is: how to address the insufficient adaptability of traditional control methods (such as PID regulation and static scheduling rules) to the strong nonlinear dynamic characteristics of the multi-field coupling of hydraulic, mechanical and electrical systems in hydropower stations, which leads to problems such as optimization lag, increased lifespan loss or excessive vibration.

[0005] To address the aforementioned technical problems, this invention provides the following technical solution: a three-dimensional collaborative optimization method for unit efficiency, lifespan, and stability based on deep reinforcement learning, comprising: collecting operating data of hydropower station units and grid control commands to construct a state space and an action space; constructing a three-dimensional collaborative optimization model for efficiency, lifespan, and stability based on the state space and action space, and transforming the three-dimensional collaborative optimization model into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems; constructing a hierarchical dynamic reward function, wherein the main reward function dynamically adjusts the weight coefficients of efficiency, lifespan, and stability according to the real-time operating status, and a constraint penalty function applies nonlinear negative feedback to the out-of-limit operating parameters to constrain the optimization direction and learning path; training the three-dimensional collaborative optimization model using a competitive double-Q network architecture, wherein the strategy generation network outputs joint control commands, and the value evaluation network introduces the decay factor of the equipment remaining lifespan prediction model, dynamically calculating the multi-objective Q value of the Pareto front to generate control strategies; and completing the switching and online updating of control strategies under different operating modes based on a multi-condition strategy switching and strategy model update mechanism.

[0006] As a preferred embodiment of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning described in this invention, the method involves: collecting operating data of hydropower station units and power grid control commands, and constructing a state space and an action space. This includes: preprocessing the unit operating data and control commands to extract feature parameters and controllable execution quantities representing the operating conditions; mapping the feature parameters to the state space and the controllable execution quantities to the action space; and providing an input structure to the three-dimensional collaborative optimization model after the state space and action space are constructed.

[0007] As a preferred embodiment of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning described in this invention, the step of transforming the three-dimensional collaborative optimization model into a Markov decision process includes: defining a state transition function to characterize the dynamic behavior of the hydropower station unit under continuous control; establishing a multi-objective optimization path based on the state transition law, integrating the three control objectives of efficiency, lifetime, and stability into a unified decision framework; and constructing an end-to-end reinforcement learning training process.

[0008] As a preferred embodiment of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning described in this invention, the method of training the three-dimensional collaborative optimization model using a competitive double-Q network architecture includes: constructing a strategy generation network for generating control strategies and a value evaluation network for evaluating target values ​​using a competitive double-Q network architecture; outputting joint control commands through the strategy generation network and dynamically evaluating the Pareto performance of the current strategy under multi-objective conditions through the value evaluation network; and introducing a decay factor of the equipment lifetime prediction model during the training process to guide the strategy to converge in a direction that balances control performance and equipment health.

[0009] As a preferred embodiment of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning described in this invention, the step of establishing a multi-objective optimization path based on state transition laws and integrating the three control objectives of efficiency, lifetime, and stability into a unified decision-making architecture includes: constructing a multi-field coupled simulation model based on a digital twin platform, integrating hydraulic, mechanical, and electrical systems to generate a simulation state sequence; setting a disturbance injection module in the digital twin platform to generate diverse operating condition trajectory data by inputting simulated head changes, load disturbances, and vibration excitations; constructing a state transition function using the operating condition trajectory sequence, which is used in the deep reinforcement learning model to define the evolution path between state and action pairs; monitoring the difference between the simulated state output and the real-time sensor data of the physical unit, and when the difference exceeds a preset tolerance threshold, performing a model parameter update operation, which only applies to the key weight coefficients corresponding to the state transition function.

[0010] This preferred solution introduces a disturbance injection module into the digital twin platform to construct a simulated operating condition trajectory covering multiple types of disturbance conditions. This enables the state transition function to cover a wider range of operating states during the training phase, improving the policy reasoning capability of the deep reinforcement learning model under unsteady and boundary conditions. At the same time, it combines real machine sensor data to correct the state transition function online, avoiding error accumulation in the control strategy due to modeling bias and ensuring that the strategy is consistent with the actual operating environment.

[0011] As a preferred embodiment of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning described in this invention, the following steps are included: constructing a hierarchical dynamic reward function, where the main reward function dynamically adjusts the weight coefficients of efficiency, lifetime, and stability according to the real-time operating status; and a constraint penalty function applies nonlinear negative feedback to the out-of-limit operating parameters to constrain the optimization direction and learning path. This includes identifying the current grid load status, head range, and equipment status indicators based on real-time collected hydropower station unit operating data; allocating weight coefficients corresponding to efficiency, lifetime, and stability in the main reward function based on the identification results, with these weight coefficients serving as reference factors for calculating the target Q value during the control strategy learning process; and implementing the weight allocation mechanism using conditional judgment rules and interpolation, assigning higher coefficients to efficiency indicators when the grid load is high, and lower coefficients to water... When the head is in a low position or the stability index fluctuates abnormally, the coefficient of the stability index is increased; when the equipment operating time or spectral characteristics continue to deviate, the coefficient of the life index is increased. A constraint penalty function is set to handle out-of-limit inputs from unit vibration signals, stress sensor data, and pressure fluctuation monitoring data. The penalty function serves as a negative input source in the Q-value update process of the control strategy. For vibration spectrum indicators, an exponential nonlinear penalty function is constructed, and the parameter mapping method adopts a threshold-driven accelerated decay model. As the signal strength increases, the penalty value increases nonlinearly. For spindle stress indicators, a linear penalty function is constructed, using a monotonically increasing linear proportional mapping method, and the coefficient is calculated according to the stress value amplitude. The main reward function and the constraint penalty function are integrated through a unified reward synthesizer as the optimization objective function for training the deep reinforcement learning model.

[0012] This preferred scheme, by constructing a hierarchical structure of main reward function and constraint penalty function, can guide the model to prioritize the working condition adaptation relationship between efficiency, lifespan and stability during policy training, and quickly suppress vibration and stress rise trends when abnormal operation occurs. The nonlinear penalty function can achieve high-sensitivity control response to disturbance-type indicators, while the linear penalty function can match mechanical load with the model learning path, making the reward mechanism distinguishable and directional in different operating scenarios, thereby improving the constraint capability and parameter interpretability of the policy training process.

[0013] As a preferred embodiment of the three-dimensional collaborative optimization method for unit efficiency-life-stability based on deep reinforcement learning described in this invention, the method for switching and updating control strategies under different operating modes based on a multi-condition strategy switching and strategy model update mechanism includes: jointly analyzing historical operating data and real-time sensor data through multi-condition identification; the historical operating data includes head change trends and vibration spectrum feature sequences; in the multi-condition identification, the current operating condition category is output using a time-series classification structure based on a long short-term memory network, including high-head power generation, low-head frequency modulation, and abnormal vibration suppression modes; switching to the corresponding control strategy sub-network according to the identification results, the control strategy sub-network has been initialized for the target operating conditions during the pre-training stage; during the strategy deployment process, the current strategy network is fine-tuned through a strategy model update mechanism, which includes an importance sampling and elastic weight solidification algorithm, selecting highly correlated historical samples and maintaining the stability of key parameters.

[0014] This preferred solution uses a working condition identification structure based on a long short-term memory network to classify operating modes by combining historical operating trends and real-time signals, enabling identification and response to be completed in the early stages of working condition switching. By pre-training a multi-strategy sub-network and combining incremental transfer learning and elastic weight solidification methods to update the strategy model, it supports the control strategy to maintain structural stability when facing frequent working condition switching and boundary disturbances, effectively supporting multi-strategy collaborative control tasks in complex scenarios.

[0015] This invention provides a three-dimensional collaborative optimization system for unit efficiency, lifespan, and stability based on deep reinforcement learning.

[0016] To address the aforementioned technical problems, this invention provides the following technical solution: a three-dimensional collaborative optimization system for unit efficiency, lifespan, and stability based on deep reinforcement learning, comprising: a digital twin simulation platform, a three-dimensional dynamic game optimization module, a multi-condition strategy switching module, and an edge computing control unit; the digital twin simulation platform is used to collect operating data of hydropower station units and grid control commands to construct a state space and an action space; the three-dimensional dynamic game optimization module is used to construct a three-dimensional collaborative optimization model for efficiency, lifespan, and stability based on the state space and action space, and to transform the three-dimensional collaborative optimization model into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems; it is also used to construct a hierarchical dynamic reward function, the main reward function being based on real-time operating conditions. The system dynamically adjusts the weight coefficients of efficiency, lifespan, and stability, and applies nonlinear negative feedback to the out-of-limit operating parameters using a constraint penalty function to constrain the optimization direction and learning path. It is used to train the three-dimensional collaborative optimization model using a competitive dual-Q network architecture. The strategy generation network outputs joint control commands, and the value assessment network introduces the decay factor of the equipment's remaining lifespan prediction model. It dynamically calculates the multi-objective Q-values ​​of the Pareto front to generate control strategies. The multi-condition strategy switching module is used to complete the switching and online updating of control strategies under different operating modes based on the multi-condition strategy switching and strategy model update mechanism. The edge computing control unit is deployed at the power plant site, receives the control commands output by the three-dimensional dynamic game optimization module, and executes millisecond-level action responses.

[0017] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning.

[0018] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning.

[0019] The beneficial effects of this invention are as follows: This invention constructs a three-dimensional dynamic game optimization model of efficiency, lifespan, and stability through deep reinforcement learning, breaking through the static limitations of traditional fixed-weight superposition methods. It employs a competitive double-Q network architecture and a hierarchical dynamic reward function to achieve autonomous adjustment of multi-objective weights under changing operating conditions such as head and load, thereby improving overall optimization efficiency, reducing turbine fatigue damage, and lowering peak vibration intensity, effectively solving the problems of equipment lifespan loss and stability deterioration caused by single-objective optimization.

[0020] By integrating multiphysics coupling simulation and real-time data closed-loop calibration on a digital twin platform, a dynamic correlation model of hydraulics, mechanics, and electrical systems is established to reduce simulation errors. By incorporating techniques such as rainflow counting and vibration transmission path analysis, the nonlinear conflict between efficiency improvement and lifespan loss is precisely quantified, thereby enhancing the accuracy of control strategy optimization under the constraint of runner stress safety threshold.

[0021] It leverages edge computing units to achieve millisecond-level instruction execution, supporting rapid tracking of power grid frequency regulation commands. Incremental transfer learning is used to update model parameters online, reducing errors in scenarios with sudden head changes and significantly improving robustness under complex operating conditions.

[0022] A long short-term memory network for operating condition identification and a dedicated strategy network group are employed to achieve intelligent switching between modes such as high-head power generation and low-head frequency modulation. An S-shaped function limiting algorithm is used to reduce the overall performance loss rate during mode switching, lower the instantaneous peak vibration intensity, and avoid unplanned downtime caused by delays in manual intervention.

[0023] By integrating the remaining life prediction model with dynamic health index assessment, the cycle of turbine runner overhaul can be extended, the number of unplanned shutdowns per year can be reduced, the overall operation and maintenance costs can be lowered, and the economic efficiency of the unit throughout its entire life cycle can be significantly improved. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 The overall flowchart of a three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning is provided in one embodiment of the present invention.

[0026] Figure 2 The flowchart illustrates a three-dimensional collaborative optimization method for unit efficiency, lifespan, and stability based on deep reinforcement learning, as provided in one embodiment of the present invention.

[0027] Figure 3 This is a diagram illustrating the internal structure of a competitive dual-Q network architecture for a three-dimensional collaborative optimization method for unit efficiency, lifespan, and stability based on deep reinforcement learning, as provided in an embodiment of the present invention.

[0028] Figure 4 A data closed-loop flowchart of a digital twin platform and physical system for a three-dimensional collaborative optimization method of unit efficiency-lifetime-stability based on deep reinforcement learning, provided as an embodiment of the present invention.

[0029] Figure 5 The state machine logic and transition control machine diagram for multi-condition mode switching of a three-dimensional collaborative optimization method for unit efficiency-life-stability based on deep reinforcement learning provided in one embodiment of the present invention.

[0030] Figure 6 The diagram shows the multi-source data fusion architecture and feedback calibration mechanism of the equipment remaining life prediction model based on the three-dimensional collaborative optimization method of unit efficiency-life-stability based on deep reinforcement learning, as provided in an embodiment of the present invention.

[0031] Figure 7 This invention provides the overall system architecture and module interaction relationships of a three-dimensional collaborative optimization system for unit efficiency, lifespan, and stability based on deep reinforcement learning, as an embodiment of the present invention. Detailed Implementation

[0032] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0033] Example 1, referring to Figure 1 and Figure 2 This is one embodiment of the present invention, which provides a three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning, including:

[0034] S1. Collect the operating data of the hydropower station units and the power grid control commands to construct the state space and action space.

[0035] S2. Based on the state space and action space, a three-dimensional collaborative optimization model of efficiency-lifetime-stability is constructed, and the three-dimensional collaborative optimization model is transformed into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems.

[0036] S3. Construct a hierarchical dynamic reward function. The main reward function dynamically adjusts the weight coefficients of efficiency, lifetime, and stability according to the real-time operating status. The constraint penalty function applies nonlinear negative feedback to the out-of-limit operating parameters to constrain the optimization direction and learning path.

[0037] S4. A competitive dual-Q network architecture is used to train the three-dimensional collaborative optimization model. The strategy generation network outputs joint control commands, and the value assessment network introduces the decay factor of the equipment remaining life prediction model. The multi-objective Q value of the Pareto front is dynamically calculated to generate the control strategy.

[0038] S5. Based on the multi-condition strategy switching and strategy model update mechanism, the control strategy switching and online update under different operating modes are completed.

[0039] It should be noted that, as Figure 1 and Figure 2 As shown, combining the five steps S1 to S5, a deep reinforcement learning model is constructed with efficiency, lifespan, and stability as the core optimization objectives. An optimization decision-making framework is established using real-time operating data of hydropower station units. Control strategies are generated through end-to-end training, and a hierarchical reward structure guides the optimization direction. Dynamic switching and structural updates of strategies are achieved under multiple operating conditions. This scheme is used to construct an operation control method for complex coupled systems.

[0040] Example 2, refer to Figures 3-6 As one embodiment of the present invention, based on the previous embodiment, a three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning is provided, including:

[0041] Furthermore, in step S1, the operating data of the hydropower station units and the grid control commands are collected to construct the state space and action space, including the following steps A1-A3:

[0042] A1. By preprocessing the unit's operating data and control commands, characteristic parameters and controllable execution quantities representing the operating status are extracted.

[0043] A2. Map the feature parameters to the state space and the controllable execution quantity to the action space.

[0044] A3. After the state space and action space are constructed, provide the input structure to the three-dimensional collaborative optimization model.

[0045] Specifically, in step A2, the unit operating parameters, equipment health indicators and grid dispatch instructions are mapped into reinforcement learning state vectors, where the equipment health indicators include the runner stress change rate, bearing temperature gradient and vibration spectrum main frequency offset.

[0046] The state space includes unit operating parameters: head, flow rate, output power; equipment status indicators: runner stress spectrum, vibration spectrum main frequency, bearing temperature; and grid commands: frequency regulation requirements and active power commands.

[0047] The action space includes the guide vane opening adjustment, excitation voltage adjustment, and blade angle control.

[0048] Furthermore, in step S2, a three-dimensional collaborative optimization model of efficiency, lifetime, and stability is constructed based on the state space and action space. This three-dimensional collaborative optimization model is then transformed into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems, including the following steps B1-B3:

[0049] B1. Define a state transition function to characterize the dynamic behavior of hydropower station units under continuous control.

[0050] B2. Establish a multi-objective optimization path based on the state transition law, and integrate the three control objectives of efficiency, lifetime and stability into a unified decision-making framework.

[0051] B3. Construct an end-to-end reinforcement learning training process.

[0052] Specifically, in step B1, a three-dimensional dynamic optimization model of "efficiency-life-stability" for hydropower station units is constructed based on a deep reinforcement learning framework. The efficiency indicators include real-time power generation efficiency and water energy utilization rate (based on the head-flow curve). The life indicators include the cumulative fatigue damage of the runner (based on the rainflow counting method), bearing wear rate, and main shaft stress spectrum. The stability indicators include vibration intensity, pressure pulsation amplitude, and rotational speed fluctuation rate.

[0053] The multi-objective optimization problem is transformed into a Markov Decision Process (MDP), encompassing the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems. An end-to-end optimization decision-making chain is established by defining a state space that includes unit operating parameters (such as head, flow rate, and power), equipment status indicators (runner stress, vibration spectrum), and grid commands, as well as an action space (guide vane opening adjustment, excitation voltage adjustment, and blade angle control). The objective function integrates quantitative indicators of efficiency improvement, lifespan extension, and stability maintenance, and a competitive dual-Q network architecture is used to achieve multi-objective dynamic game theory, overcoming the static limitations of traditional weighted optimization methods.

[0054] Efficiency indicators: Real-time power generation efficiency is calculated using data from power sensors and flow meters, while hydropower utilization is based on dynamic fitting of the head-flow curve.

[0055] Life indicators: The cumulative fatigue damage of the impeller was statistically analyzed using the rainflow counting method to collect stress spectrum data; the bearing wear rate was analyzed using vibration acceleration sensor signals; and the main shaft stress spectrum was collected in real time by strain gauges.

[0056] Stability indicators: Vibration intensity is measured by a vibration sensor (unit: mm / s), pressure pulsation amplitude is monitored by a pressure transmitter, and speed fluctuation rate is fed back by a speed sensor.

[0057] In this embodiment of the application, the establishment of a multi-objective optimization path based on state transition laws in step B2 can be achieved by modeling the operation behavior of the hydropower station unit as a Markov decision process, constructing a state transition function based on the state space and action space to describe the dynamic response process of the unit under continuous control; on this basis, an optimization path integrating three types of objectives—efficiency, lifespan, and stability—is defined, and a multi-objective game structure is embedded in the deep reinforcement learning model to construct a control strategy that can be learned and updated in parallel.

[0058] In one alternative implementation, the multi-objective optimization path based on the state transition law can also rely on the multi-field coupled simulation environment built by the digital twin platform to obtain the working condition trajectory data under conditions such as head disturbance, load change, and vibration anomaly. The state transition function is generated using this data sequence and used as a training sample to input into the deep reinforcement learning model, driving the multi-objective strategy to learn the control path under multi-dimensional objective trade-offs.

[0059] In another alternative implementation, the multi-objective optimization path based on the state transition law can also be established by continuously collecting the actual operating status of the physical unit and comparing it with the output of the simulation model, dynamically calibrating the key weight coefficients in the state transition function, updating only the parameters corresponding to the deviation generation region, constructing a state transition structure with time-series characteristics and online correction capabilities, and supporting the deep reinforcement learning model to accurately model the multi-objective control path.

[0060] This invention constructs a state transition mechanism that is linked to simulation feedback, and embeds efficiency, lifetime and stability objectives into the policy learning process in a unified manner, so as to realize multi-objective modeling and dynamic optimization path extraction under coupled operating conditions, and provide a structured training basis for control strategies.

[0061] Specifically, in step B2, a multi-objective optimization path is established based on the state transition law, integrating the three control objectives of efficiency, lifetime, and stability into a unified decision-making framework, including the following steps B21-B24:

[0062] B21. Construct a multi-field coupled simulation model based on a digital twin platform, integrating hydraulic, mechanical, and electrical systems to generate simulation state sequences.

[0063] B22. Set up a disturbance injection module in the digital twin platform to generate diverse working condition trajectory data by inputting simulated head changes, load disturbances and vibration excitations.

[0064] B23. Construct a state transition function using the sequence of working conditions trajectory. In deep reinforcement learning models, the state transition function is used to define the evolution path between state and action pairs.

[0065] B24. Monitor the difference between the simulation state output and the real-time sensor data of the physical unit. When the difference exceeds the preset tolerance threshold, perform the model parameter update operation. The update operation only applies to the key weight coefficients corresponding to the state transition function.

[0066] The multi-objective optimization problem is transformed into an MDP, encompassing the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems. By defining the state space and action space, an end-to-end optimization decision-making chain is established.

[0067] Specifically, in step B3, the end-to-end optimization decision link is constructed, the objective function integrates quantitative indicators of efficiency improvement, lifetime extension and stability maintenance, and a multi-objective dynamic game is realized through a competitive double-Q network architecture, breaking through the static limitations of traditional weighted optimization methods.

[0068] Objective function fusion formula:

[0069] R=ω1·R 效率 +ω2·R 寿命 +ω3·R 稳定性

[0070] Where R is the overall reward value; R 效率 Efficiency is an efficiency-related reward, positively correlated with unit operating efficiency; R 寿命 Rewards related to equipment lifespan are negatively correlated with the wear and tear of critical components (such as bearings and blades); R 稳定性 For operational stability-related rewards, the weighting coefficients ω1, ω2, and ω3, along with vibration and pressure pulsation, are respectively the efficiency weight, lifespan weight, and stability weight, and are dynamically adjusted according to real-time operating conditions.

[0071] In this embodiment, the hierarchical dynamic reward function in step S3 can be a main reward function that dynamically adjusts the weight coefficients of three types of objectives—efficiency, lifespan, and stability—based on real-time changes in water head and grid load demand. It can also be combined with a constraint penalty function that applies nonlinear feedback to parameters exceeding limits, such as vibration intensity and principal shaft stress, and then combined with a unified reward synthesizer to form the final training objective, which is used to guide the deep reinforcement learning strategy generation process.

[0072] In one alternative implementation, the hierarchical dynamic reward function can also increase the proportion of efficiency indicators in the main reward function based on the changing trend of load levels in the operating state; at the same time, a nonlinear exponential penalty function is constructed for the vibration signal to strengthen the policy constraint when the vibration frequency changes beyond the set range, thereby improving the ability to control boundary disturbances during training.

[0073] In another alternative implementation, the hierarchical dynamic reward function can also dynamically increase the weights of stability and lifespan targets based on the stress change trend and spectral frequency shift in equipment condition monitoring. The penalty function for the principal axis stress adopts a linear proportional calculation method, and a gradient-type negative feedback is set according to its amplitude range. The target value is combined to construct an optimized objective function embedded in the deep reinforcement learning network.

[0074] This invention, by setting a hierarchical dynamic reward structure that includes a main reward function and a constraint penalty function, can achieve real-time weight allocation and abnormal signal constraint response among the three-dimensional objectives of efficiency, lifetime, and stability, ensuring that the control strategy maintains the adaptability of the optimization direction and the consistency of the safety boundary during the training process.

[0075] Furthermore, in step S3, a hierarchical dynamic reward function is constructed. The main reward function dynamically adjusts the weight coefficients of efficiency, lifetime, and stability according to the real-time operating status. The constraint penalty function applies nonlinear negative feedback to the out-of-limit operating parameters to constrain the optimization direction and learning path, including the following steps C1-C7:

[0076] C1. Based on the real-time collected hydropower station unit operation data, identify the current power grid load status, head range, and equipment status indicators.

[0077] C2. Based on the recognition results, assign weight coefficients corresponding to efficiency, lifetime, and stability to the main reward function. These weight coefficients serve as reference factors for calculating the target Q value during the control policy learning process.

[0078] C3. The weight allocation mechanism is implemented using conditional judgment rules and interpolation. When the grid load is high, the efficiency index is assigned a higher coefficient. When the water head is low or the stability index fluctuates abnormally, the coefficient of the stability index is increased. When the equipment operating time or spectrum characteristics continue to shift, the coefficient of the life index is increased.

[0079] C4. Set a constraint penalty function to handle over-limit inputs from unit vibration signals, stress sensors, and pressure fluctuation monitoring data. The penalty function serves as a negative input source during the Q-value update process of the control strategy.

[0080] C5. For vibration spectrum indicators, an exponential nonlinear penalty function is constructed. The parameter mapping method adopts a threshold-driven accelerated attenuation model. As the signal strength increases, the penalty value increases in a nonlinear manner.

[0081] C6. Construct a linear penalty function for principal stress indices, using a monotonically increasing linear proportional mapping method, and calculate the coefficients according to the stress value amplitude.

[0082] C7. The main reward function and the constraint penalty function are integrated through a unified reward synthesizer and used as the optimization objective function for training the deep reinforcement learning model.

[0083] Specifically, a hierarchical dynamic reward function is designed to decouple the main optimization objective from the constraints. The main reward function dynamically adjusts the weight ratios of efficiency, lifetime, and stability based on real-time head and grid load demand. The constraint penalty term applies exponential negative feedback to parameters exceeding limits (such as vibration intensity exceeding 4.0 mm / s and principal shaft stress exceeding the threshold), forcing the strategy network to optimize within the safe boundary.

[0084] The main reward function dynamically adjusts the efficiency, lifetime, and stability weight coefficients based on real-time head height and grid load demand. The constraint penalty function applies exponential negative feedback to vibration intensity, principal shaft stress, and pressure pulsation exceeding limits. Based on real-time changes in grid load and head parameters, the reward weight allocation mechanism is dynamically adjusted: when grid load demand increases, the efficiency weight ω1 is increased to 0.6, and the lifetime weight ω2 is reduced to 0.2, prioritizing power generation efficiency; when the head is detected to be below the design value (e.g., the unit enters the inefficient zone), the stability weight ω3 is strengthened to 0.5 to suppress operational risks. Simultaneously, multi-dimensional constraint penalty terms are designed: if the vibration intensity ν exceeds the limit by more than 4.0 mm / s, an exponential penalty P is applied. 振动 =-10·e (ν-0.4) Its nonlinear characteristics force the agent to quickly move away from dangerous working conditions; if the principal axis stress exceeds the material life threshold (σ>150MPa), a linear penalty P is applied. 应力 = -20(σ-150), directly quantifying the risk of mechanical damage. Two types of penalty functions are embedded into the global objective function through a reward synthesizer, achieving a dynamic balance between safety constraints and economic objectives.

[0085] Furthermore, in step S4, a competitive dual-Q network architecture is used to train the three-dimensional collaborative optimization model. The strategy generation network outputs joint control commands, and the value assessment network introduces the decay factor from the equipment remaining life prediction model. The multi-objective Q value of the Pareto front is dynamically calculated to generate the control strategy, including the following steps D1-D3:

[0086] D1. Using a competitive dual-Q network architecture, construct a policy generation network for generating control policies and a value evaluation network for evaluating target values.

[0087] D2. Output joint control instructions through the policy generation network, and dynamically evaluate the Pareto performance of the current policy under multi-objective conditions through the value evaluation network.

[0088] D3. Introduce the decay factor of the equipment life prediction model during the training process to guide the strategy to converge in a direction that balances control performance and equipment health.

[0089] Specifically, in step D1, a competitive double-Q network architecture is used, where the policy generation network (Actor) generates action policies, and the value evaluation network (Critic) evaluates Pareto front solutions, optimizing multi-objective conflict relationships through dynamic game theory. For example... Figure 3 As shown.

[0090] In this embodiment of the application, the joint control command output through the strategy generation network in step D2 can be constructed in a competitive dual-Q network architecture. The strategy generation network receives the operating state information provided by the state space and generates control commands based on the learned strategy function. The control commands include the joint control of the guide vane opening, blade angle and excitation voltage, which are used to act on the execution end of the hydropower station unit.

[0091] In one optional implementation, the strategy generation network can output joint control commands. Alternatively, the strategy generation network can adopt a deep neural network structure, output corresponding action vectors based on the current state vector, and perform multi-dimensional mapping on the guide vanes, blades, and excitation control quantities. The output results are parsed by the edge control unit and synchronously transmitted to the control actuator to achieve end-to-end strategy deployment.

[0092] In another alternative implementation, the policy generation network can be deployed in an edge computing device to output joint control commands. The inference speed is optimized by using FP16 compression. The network outputs a sequence of control commands at fixed time steps based on the input operating state characteristics, forming a periodic closed-loop control flow, and completes local policy inference without relying on a central server.

[0093] This invention achieves unified generation and real-time response of power generation control commands by outputting joint control commands for multiple control dimensions in the policy generation network. It is applicable to complex control scenarios covering hydraulic, mechanical, and electrical coupling characteristics, and provides refined control output capabilities for deep reinforcement learning models.

[0094] Specifically, in step D3, a competitive dual-depth Q-network architecture is constructed to optimize complex game relationships:

[0095] The Actor network receives state space data (head, load command, vibration spectrum, etc.) and outputs high-precision control commands (such as guide vane opening adjustment resolution of 0.1%).

[0096] The Critic network evaluates the optimal Q-value under multi-objective conflict based on the Pareto front solution, while the target network adopts a delayed update strategy (synchronizing parameters once every 1000 steps) to avoid the problem of Q-value overestimation during strategy iteration.

[0097] During the training phase, the Actor-Critic parameters are dynamically corrected through timing differential error, enabling control commands such as guide vanes and excitation to respond quickly to grid demands in efficiency-first mode and actively limit mechanical load fluctuations in stability-first mode, ultimately achieving stable convergence of the multi-objective strategy and improving anti-interference capabilities.

[0098] The Actor outputs combined control commands for guide vane opening, blade angle, and excitation voltage; the Critic dynamically calculates multi-objective Q values ​​based on the Pareto front solution, where the Q value update rule incorporates the decay factor output by the equipment remaining life prediction model.

[0099] like Figure 4 As shown, a high-fidelity simulation environment is built based on the hydropower station's digital twin platform to simulate multimodal scenarios such as sudden head changes, load jumps, and abnormal vibrations. An initial DRL model is pre-trained until convergence. After deployment to the physical units, a hyper-converged cloud-edge collaborative architecture enables real-time data closed-loop feedback. Edge computing nodes (such as NVIDIA Jetson AGXXavier) are used to perform millisecond-level policy inference, and incremental transfer learning technology is employed to fine-tune the model online. When the deviation between actual operating data and simulation predictions exceeds a threshold, only some weight parameters of the Critic network are updated to maintain policy stability while improving dynamic tracking accuracy. This ensures that the model maintains a simulation error of ≤3% even under complex conditions such as changes in water flow sediment content and transient changes in power grid frequency regulation commands.

[0100] First, a high-fidelity simulation environment is constructed to cover potential operating conditions of the unit by simulating extremely complex scenarios (such as sudden head changes of ±10m, load jumps of ±30% power, and abnormal vibration spectrum frequency shifts of ≥10Hz). Based on this environment, the initial deep reinforcement learning (DRL) model is pre-trained until the model converges to a stable state (cumulative reward fluctuation less than 1%). At the same time, the dynamic error between the simulation system and the actual unit is strictly constrained to not exceed 3%, ensuring that the virtual model can accurately map the dynamic characteristics of the physical unit, laying a reliable foundation for subsequent migration to the real environment.

[0101] After pre-training, the model is deployed to physical units via a hyperconverged cloud-edge architecture, achieving millisecond-level real-time inference through edge computing nodes. Simultaneously, a closed-loop data feedback loop is formed between the cloud and the edge to continuously optimize the control strategy. When the deviation between actual operating data and simulation predictions exceeds a preset threshold (e.g., vibration prediction error > 10%), an incremental transfer learning mechanism is triggered—dynamically updating only a portion of the weight layers at the end of the Critic network. This allows for rapid adaptation to actual operating conditions while preserving the policy stability of the pre-trained model, avoiding control oscillation risks caused by global parameter updates, thus balancing response speed.

[0102] In this embodiment, the multi-condition strategy switching and strategy model update mechanism in step S5 can analyze historical head change trends and vibration spectrum characteristics through a condition identification module based on Long Short-Term Memory (LSTM) networks, identify the operating condition categories of hydropower station units, including high-head power generation, low-head frequency regulation, and abnormal vibration suppression modes, and switch to the corresponding pre-trained control strategy sub-network according to the identification results. During the strategy execution process, an incremental transfer learning mechanism is used to fine-tune the current strategy online to maintain the model's adaptability to actual operating data.

[0103] In one alternative implementation, the multi-condition policy switching and policy model update mechanism can also utilize embedded edge computing devices to deploy condition recognition and policy reasoning modules. Based on multi-source feature input within a sliding time window, the mechanism outputs condition classification results, thereby completing the automatic switching of the control policy network. During the model update process, importance sampling technology is introduced to select historical samples with high correlation to the current condition to participate in policy fine-tuning, thereby improving the convergence efficiency of the online learning process.

[0104] In another optional implementation, the multi-condition strategy switching and strategy model update mechanism can also configure strategy sub-networks dedicated to specific conditions in the multi-strategy control structure. Each sub-network is pre-trained in the digital twin platform and uploaded to the edge control unit, and dynamically invoked according to changes in the operating state. During the strategy update process, the elastic weight solidification algorithm is combined to impose constraints on the key parameters in the deep reinforcement learning model to prevent catastrophic forgetting of existing control knowledge.

[0105] This invention integrates a multi-condition identification structure and a strategy update mechanism to achieve automatic switching and online adjustment of control strategies in scenarios with frequent switching of operating modes and drastic changes in operating conditions. This enhances the adaptability and stability of the model under various operating conditions and meets the intelligent control requirements of hydropower station units.

[0106] Furthermore, in step S5, based on the multi-condition strategy switching and strategy model update mechanism, the control strategy switching and online update under different operating modes are completed, including the following steps E1-E4:

[0107] E1. Through multi-condition identification, historical operating data and real-time sensor data are jointly analyzed. Historical operating data includes head change trends and vibration spectrum characteristic sequences.

[0108] E2. In multi-condition identification, the current operating condition category is output using a time-series classification structure based on a long short-term memory network. The categories include high-head power generation, low-head frequency modulation, and abnormal vibration suppression mode.

[0109] E3. Switch to the corresponding control strategy sub-network based on the recognition result. The control strategy sub-network has been initialized for the target working condition during the pre-training stage.

[0110] E4. During the policy deployment process, the current policy network is fine-tuned through the policy model update mechanism. The policy model update mechanism includes importance sampling and elastic weight solidification algorithms, selects highly correlated historical samples, and maintains the stability of key parameters.

[0111] To address the high-frequency switching requirements of hydropower station generating units, multiple dedicated strategy networks were designed and integrated with an LSTM operating condition identification module to achieve autonomous switching between modes such as high-head power generation, low-head frequency regulation, and abnormal vibration suppression. By analyzing historical operating data and real-time sensor signals, the LSTM dynamically identifies current operating condition characteristics (such as head drop rate and vibration spectrum frequency shift), triggering the corresponding strategy network to take over control. During mode transitions, a motion space smooth transition algorithm is employed to limit the guide vane opening adjustment rate and blade angle abrupt change, reducing the overall performance loss rate from ≥10% in traditional methods to ≤3%, while ensuring a frequency regulation response time reduction of over 40% and a 35% decrease in peak vibration intensity.

[0112] Specifically, such as Figure 5 and Figure 6 As shown, the LSTM operating condition identification and safety early warning system is based on a real-time operating condition perception module constructed using an LSTM neural network. It takes 60 seconds of historical operating data (including head, guide vane opening, excitation voltage, etc.) and current sensor signals (vibration spectrum, flow meter readings) as input. Through time-series feature extraction, it outputs operating condition classification probabilities, covering three modes: high-head power generation, low-head frequency modulation, and abnormal vibration suppression. When the detected head drop rate exceeds 0.5 m / min and the vibration frequency deviation reaches 5 Hz, the abnormal vibration suppression mechanism is automatically triggered, intervening in the control logic in advance to prevent the unit from entering a dangerous operating condition. This module achieves a shift from passive response to active defense through dynamic prediction and threshold interlocking, providing a decision-making basis for subsequent strategy switching.

[0113] Multi-strategy dynamic switching and smooth control are implemented, deploying three dedicated strategy networks for different operating conditions: in high-head mode, guide vane opening is prioritized to improve power generation efficiency; in low-head frequency regulation, excitation voltage is rapidly adjusted to ensure grid frequency stability; and during abnormal vibration suppression, mechanical shock is reduced through smooth blade angle adjustment. To ensure seamless integration between different strategies, a smooth transition algorithm for the action space is designed—the guide vane opening adjustment rate is limited to ±0.5% / s, and the blade angle abrupt change amplitude is constrained to ±1° / s. Dynamic interpolation algorithms eliminate control command jumps, reducing the overall performance loss rate caused by mode switching from over 10% in traditional methods to less than 3%, significantly improving control coherence.

[0114] A comprehensive performance verification and optimization closed-loop system, deployed and validated in a real hydropower station, demonstrates that this solution reduces the frequency regulation response time from 2.5 seconds in traditional PID control to 1.5 seconds, and lowers the peak vibration intensity from 6.2 mm / s to the safe threshold of 4.0 mm / s, effectively suppressing mechanical fatigue damage. By integrating the life prediction model with real-time operating data, the estimated equipment lifespan under all operating conditions is improved by 20%, while water energy utilization is increased by 8%. All verification indicators are continuously fed back through a cloud-based digital twin platform, forming an optimization closed loop of simulation-real-machine-feedback, providing data support for subsequent iterations of strategy network parameters, and enabling the self-evolution of the control system.

[0115] Example 3 is an embodiment of the present invention, which provides a three-dimensional collaborative optimization method for unit efficiency, lifespan and stability based on deep reinforcement learning. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through experiments.

[0116] Scenario background: A large hydropower station in China is equipped with a 500MW mixed-flow turbine generator unit and faces the following typical problems, as shown in Table 1.

[0117] Efficiency versus lifespan conflict: In high-water-powered mode, in order to maximize efficiency, the guide vane opening is often adjusted to the critical value, which leads to frequent fatigue cracks in the runner blades (an average of 3 shutdowns for maintenance per year).

[0118] Excessive vibration during frequency regulation: When responding to the grid frequency regulation command, the unit frequently switches loads (50%-100% of rated power), causing pressure pulsation amplitude to exceed 0.3MPa and instantaneous vibration intensity to reach 4.8mm / s (exceeding ISO 10816-5 Class A standard);

[0119] Low head operation results in low efficiency: When the head drops to 70% of the design value during the flood season, the power generation efficiency drops to 88%, and the main shaft oscillation increases to 0.25mm (design threshold 0.15mm).

[0120] Table 1 Problems with Traditional Methods

[0121]

[0122] Step 1: Construct digital twins of hydropower station units and couple them with multiphysics.

[0123] Taking a 500MW mixed-flow hydropower station unit as an example, a three-dimensional simulation model of the hydraulic flow channel was first established based on ANSYS CFX to simulate the dynamic characteristics of water flow under different heads (50-150m). Simultaneously, a mechanical transmission system and generator electromagnetic model were built using MATLAB / Simulink to achieve multi-field coupling of hydraulic, mechanical, and electrical systems. The model incorporates a turbine runner fatigue damage calculation module, uses rainflow counting to track the blade stress spectrum in real time, and integrates a vibration transmission path analysis module to correlate the nonlinear relationship between guide vane opening and pressure pulsation amplitude. Among these:

[0124] Wheel fatigue damage calculation: Damage accumulation formula based on rainflow counting method:

[0125]

[0126] Where D represents the cumulative damage value, which is the sum of fatigue damage to the material under variable amplitude load. Fatigue failure is predicted when D≥1; n i The actual number of cycles corresponding to the i-th stress amplitude is extracted from the load spectrum using the rainflow counting method. For example, for a certain stress amplitude σ i =200MPa corresponding to n i =1.5×10 4 N times; i The fatigue life corresponding to the i-th stress amplitude is determined by the SN curve (stress-life curve), and a typical formula is: σ f denoted as , b is the material fatigue strength coefficient, and k is the total number of stress amplitude levels. In engineering practice, the rainflow counting results are grouped and statistically analyzed according to stress amplitude levels 8-16.

[0127] Vibration transmission path analysis: The nonlinear relationship between the guide vane opening Y and the pressure pulsation amplitude ΔP is modeled as follows:

[0128] ΔP = 0.12Y 2 -1.8Y + 15.6 (Y∈[60%, 95%])

[0129] By accessing real-time data (including head, flow rate, power, and guide vane opening) from the power plant's SCADA system and acceleration signals from eight measuring points of the vibration monitoring system via the OPC UA protocol, a digital twin environment that is synchronously updated with the physical unit is constructed. The simulation error verification table is shown in Table 2, which compares the actual SCADA data with the simulation error data.

[0130] Table 2 Simulation Error Verification Table

[0131]

[0132]

[0133] Step 2: Training the deep reinforcement learning model and implementing the dynamic game mechanism.

[0134] A deep reinforcement learning (DRL) framework is deployed in a digital twin environment, defining a state space containing 12 parameters (head, flow rate, power, vibration spectrum amplitude, impeller stress, etc.) and a 3-dimensional action space (guide vane opening adjustment step size ±1%, excitation voltage adjustment amount ±5%, blade angle fine-tuning ±0.5°). A hierarchical dynamic reward function is designed to quantify the objectives of efficiency improvement (weight 0.5), lifespan extension (weight 0.3), and vibration suppression (weight 0.2) into a comprehensive optimization index, and a constraint penalty term is introduced (such as triggering a 10-fold negative feedback when the vibration intensity exceeds the limit).

[0135] The reward function is:

[0136]

[0137] Where Δη is the rate of change of efficiency, ΔD is the increment of fatigue damage of the impeller, and V RMS Let λ represent the vibration intensity, and λ = 10 be the penalty coefficient.

[0138] A competitive dual-Q network architecture is adopted, with Actors generating action strategies and Critics evaluating Q-values ​​and generating Pareto front solutions. During the pre-training phase, 1 million steps of operation are simulated in a twin environment, covering extreme scenarios such as sudden changes in water head and load jumps. The final model achieves an efficiency improvement of 1.3 percentage points and a reduction of 18% in peak runner stress under high water head power generation mode.

[0139] Step 3: Edge-side deployment and virtual-physical linkage adaptive iteration.

[0140] The trained DRL model was deployed to the power plant's edge computing node (NVIDIA Jetson AGX Xavier), receiving unit sensor signals via a real-time data bus and outputting optimization commands (guide vane opening, excitation voltage, etc.) every 0.5 seconds. When the actual operating data deviated from the twin model's predicted values ​​by more than 5% (e.g., abnormal runner stress caused by sudden changes in water flow sediment content), an incremental transfer learning mechanism was triggered: the Actor network parameters were frozen, and only the Critic network weights were fine-tuned online, updating the reward function constraints using real-time SCADA data. This process enabled the model to maintain a guide vane opening control error of ≤2.3% even when the head dropped sharply by 20m during the flood season, and the dynamic response delay was compressed to within 50ms, significantly better than the 1.2-second lag of traditional offline models. See Tables 3 and 4 for details.

[0141] Table 3 Incremental Transfer Learning Parameters

[0142] Parameter type Initial pre-training values Online fine-tuning value Adjustment range Critic learning rate 1e-4 3e-5 -70％ Experience replay weight Uniform sampling Priority sampling +50％ Batch size 64 32 -50％

[0143] Table 4 Comparison of Dynamic Response Performance

[0144]

[0145] Step 4: Autonomous switching between multiple operating conditions and engineering verification.

[0146] To address the high-frequency switching requirements of the generating unit, a dedicated strategy network was designed and an LSTM operating condition identification module was integrated. When the head suddenly drops from 120m (power generation mode) to 80m (frequency regulation mode), the system automatically switches to a low-head optimization strategy, limiting the guide vane opening adjustment rate to ≤0.5% / s, and simultaneously adjusting the blade angle to compensate for water flow turbulence. The formula for the coordinated adjustment of guide vane opening and blade angle is as follows:

[0147]

[0148] Where Δθ is the blade angle compensation amount, and ΔY is the guide vane opening change amount. The rate of change of water head.

[0149] As shown in Table 5, actual operation data indicates that the peak vibration intensity during frequency regulation decreased from 4.8 mm / s to 3.1 mm / s, and the efficiency fluctuation range narrowed from ±3% to ±1.5%. In abnormal vibration suppression scenarios (such as a sudden increase in 1st harmonic vibration), the system prioritizes the use of the vibration suppression network, reducing the vibration intensity from 4.5 mm / s to 2.9 mm / s within 30 seconds, thus avoiding unplanned shutdowns. After 12 months of continuous verification, the average annual number of unplanned shutdowns decreased from 3 to 0, the turbine runner overhaul cycle was extended by 15%, and the overall operation and maintenance cost was reduced by 30.8%.

[0150] Table 5 Comparison Table

[0151]

[0152] Example 4, refer to Figure 7 This is one embodiment of the present invention, which provides a three-dimensional collaborative optimization system for unit efficiency-lifetime-stability based on deep reinforcement learning, including a digital twin simulation platform, a three-dimensional dynamic game optimization module, a multi-condition strategy switching module, and an edge computing control unit.

[0153] The digital twin simulation platform is used to collect operating data of hydropower station units and power grid control commands to construct state space and action space.

[0154] The three-dimensional dynamic game optimization module is used to construct a three-dimensional collaborative optimization model of efficiency, lifetime, and stability based on state space and action space, and transform the three-dimensional collaborative optimization model into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems. It is used to construct a hierarchical dynamic reward function, where the main reward function dynamically adjusts the weight coefficients of efficiency, lifetime, and stability according to the real-time operating status, and the constraint penalty function applies nonlinear negative feedback to the out-of-limit operating parameters to constrain the optimization direction and learning path. It is used to train the three-dimensional collaborative optimization model using a competitive double-Q network architecture, where the strategy generation network outputs joint control commands, and the value assessment network introduces the decay factor of the equipment's remaining lifetime prediction model, dynamically calculating the multi-objective Q-values ​​of the Pareto front to generate control strategies.

[0155] The multi-condition strategy switching module is used to switch and update control strategies online under different operating modes based on the multi-condition strategy switching and strategy model update mechanism.

[0156] The edge computing control unit is deployed at the power plant site to receive control commands output by the three-dimensional dynamic game optimization module and execute millisecond-level action responses.

[0157] The three-dimensional dynamic game optimization module constructs a multi-objective collaborative optimization model of efficiency, lifetime, and stability based on deep reinforcement learning. It performs end-to-end dynamic decision-making on the hydraulic-mechanical-electrical coupling parameters of the hydropower unit through a competitive double-Q network, generating a Pareto optimal control strategy.

[0158] The digital twin simulation platform integrates high-precision hydraulic machinery simulation models with real-time unit operation data to construct a dynamic training environment that links the virtual and real worlds, supporting online incremental transfer learning of the model.

[0159] The multi-condition strategy switching module, through the long short-term memory network condition identification unit and the dedicated strategy network group, realizes the autonomous switching between high-head power generation, low-head frequency modulation and abnormal suppression modes.

[0160] The edge computing control unit is deployed at the power plant site to receive control commands from the optimization module and execute millisecond-level action responses.

[0161] This embodiment also provides an electronic device applicable to the three-dimensional collaborative optimization method of unit efficiency-lifetime-stability based on deep reinforcement learning, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the three-dimensional collaborative optimization method of unit efficiency-lifetime-stability based on deep reinforcement learning proposed in the above embodiment.

[0162] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as proposed in the above embodiments.

[0163] The storage medium proposed in this embodiment and the method for implementing the three-dimensional collaborative optimization of unit efficiency-lifetime-stability based on deep reinforcement learning proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0164] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0165] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A three-dimensional collaborative optimization method for unit efficiency, lifespan, and stability based on deep reinforcement learning, characterized by: include, Collect operating data of hydropower station units and power grid control commands to construct state space and action space; Based on state space and action space, a three-dimensional collaborative optimization model of efficiency-lifetime-stability is constructed, and the three-dimensional collaborative optimization model is transformed into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics and electrical systems. A hierarchical dynamic reward function is constructed. The main reward function dynamically adjusts the weight coefficients of efficiency, lifetime, and stability according to the real-time running status. The constraint penalty function applies nonlinear negative feedback to the out-of-limit running parameters to constrain the optimization direction and learning path. A competitive dual-Q network architecture is used to train the three-dimensional collaborative optimization model. The strategy generation network outputs joint control commands, and the value assessment network introduces the decay factor of the equipment remaining life prediction model. The multi-objective Q value of the Pareto front is dynamically calculated to generate the control strategy. Based on the multi-condition strategy switching and strategy model update mechanism, the control strategy switching and online update under different operating modes are completed.

2. The three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in claim 1, characterized in that: Collect operational data from hydropower station units and grid control commands to construct state and action spaces, including... By preprocessing the unit's operating data and control commands, characteristic parameters and controllable execution quantities representing the operating status are extracted; Map the feature parameters to the state space, and map the controllable execution quantity to the action space; After the state space and action space are constructed, the input structure is provided to the three-dimensional collaborative optimization model.

3. The three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in claim 2, characterized in that: The three-dimensional collaborative optimization model is transformed into a Markov decision process. include, Define a state transition function to characterize the dynamic behavior of hydropower station units under continuous control. A multi-objective optimization path is established based on the state transition law, and the three control objectives of efficiency, lifetime and stability are integrated into a unified decision-making framework. Construct an end-to-end reinforcement learning training process.

4. The three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in claim 3, characterized in that: The training of the three-dimensional collaborative optimization model using a competitive dual-Q network architecture includes, A competitive dual-Q network architecture is adopted to construct a policy generation network for generating control policies and a value evaluation network for evaluating the value of targets; The policy generation network outputs joint control instructions, and the value evaluation network dynamically evaluates the Pareto performance of the current policy under multi-objective conditions. During the training process, a decay factor is introduced into the equipment life prediction model to guide the strategy to converge in a direction that balances control performance and equipment health.

5. The three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in claim 4, characterized in that: The proposed method establishes a multi-objective optimization path based on state transition laws, integrating efficiency, lifetime, and stability as three control objectives into a unified decision-making framework. include, A multi-field coupled simulation model is constructed based on a digital twin platform, integrating hydraulic, mechanical, and electrical systems to generate a simulation state sequence; A disturbance injection module is set up in the digital twin platform to generate diverse working condition trajectory data by inputting simulated head changes, load disturbances and vibration excitations. A state transition function is constructed using the sequence of working conditions trajectory. In deep reinforcement learning models, the state transition function is used to define the evolution path between state and action pairs. The system monitors the difference between the simulation state output and the real-time sensor data of the physical unit. When the difference exceeds the preset tolerance threshold, it performs a model parameter update operation. The update operation only applies to the key weight coefficients corresponding to the state transition function.

6. The three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in claim 5, characterized in that: The construction of a hierarchical dynamic reward function involves a main reward function that dynamically adjusts the weight coefficients of efficiency, lifetime, and stability based on real-time operating status. A constraint penalty function applies nonlinear negative feedback to out-of-limit operating parameters to constrain the optimization direction and learning path. Based on real-time collected hydropower station unit operation data, identify the current power grid load status, head range, and equipment status indicators; Based on the recognition results, weight coefficients corresponding to efficiency, lifetime, and stability are assigned to the main reward function. These weight coefficients serve as reference factors for calculating the target Q value during the control policy learning process. The weight allocation mechanism is implemented using conditional judgment rules and interpolation. When the grid load is high, the efficiency index is assigned a higher coefficient. When the water head is low or the stability index fluctuates abnormally, the coefficient of the stability index is increased. When the equipment operating time or spectrum characteristics continue to shift, the coefficient of the life index is increased. Set a constraint penalty function to handle over-limit inputs from unit vibration signals, stress sensors and pressure fluctuation monitoring data. The penalty function serves as a negative input source in the Q-value update process of the control strategy. For vibration spectrum indicators, an exponential nonlinear penalty function is constructed. The parameter mapping method adopts a threshold-driven accelerated attenuation model. As the signal strength increases, the penalty value increases nonlinearly. A linear penalty function is constructed for principal stress indices, and a monotonically increasing linear proportional mapping method is adopted to calculate the coefficients according to the stress value amplitude. The main reward function and the constraint penalty function are integrated through a unified reward synthesizer and used as the optimization objective function for training the deep reinforcement learning model.

7. The three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in claim 6, characterized in that: The multi-condition strategy switching and strategy model update mechanism enables the switching and online updating of control strategies under different operating modes. include, By identifying multiple operating conditions, historical operating data and real-time sensor data are jointly analyzed. The historical operating data includes the head change trend and vibration spectrum characteristic sequence. In multi-condition identification, the current operating condition category is output using a temporal classification structure based on a long short-term memory network. The categories include high-head power generation, low-head frequency modulation, and abnormal vibration suppression mode. Based on the recognition result, the system switches to the corresponding control strategy sub-network. The control strategy sub-network has been initialized for the target working condition during the pre-training stage. During the strategy deployment process, the current strategy network is fine-tuned through a strategy model update mechanism, which includes importance sampling and elastic weight solidification algorithms, selects highly correlated historical samples, and maintains the stability of key parameters.

8. A three-dimensional collaborative optimization system for unit efficiency-lifetime-stability based on deep reinforcement learning, employing the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in any one of claims 1 to 7, characterized in that, include: Digital twin simulation platform, 3D dynamic game optimization module, multi-condition strategy switching module and edge computing control unit; The digital twin simulation platform is used to collect operating data of hydropower station units and power grid control commands to construct state space and action space; The three-dimensional dynamic game optimization module is used to construct a three-dimensional collaborative optimization model of efficiency, lifetime, and stability based on the state space and action space, and to transform the three-dimensional collaborative optimization model into a Markov decision process, covering the multi-field coupling characteristics of hydraulics, mechanics, and electrical systems; it is used to construct a hierarchical dynamic reward function, in which the main reward function dynamically adjusts the weight coefficients of efficiency, lifetime, and stability according to the real-time operating status, and the constraint penalty function applies nonlinear negative feedback to the out-of-limit operating parameters to constrain the optimization direction and learning path; This is used to train a three-dimensional collaborative optimization model using a competitive dual-Q network architecture. The strategy generation network outputs joint control instructions, and the value assessment network introduces the decay factor of the equipment remaining life prediction model. The multi-objective Q value of the Pareto front is dynamically calculated to generate control strategies. The multi-condition strategy switching module is used to complete the switching and online updating of control strategies under different operating modes based on the multi-condition strategy switching and strategy model update mechanism. The edge computing control unit is deployed at the power plant site, receives control commands output by the three-dimensional dynamic game optimization module, and executes millisecond-level action responses.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-dimensional collaborative optimization method for unit efficiency-lifetime-stability based on deep reinforcement learning as described in any one of claims 1 to 7.

Images