Control method and system for hydropower station

Abstract

The invention relates to a control method and system for a hydropower station, and the method comprises the following steps: collecting the operation data of the hydropower station, and carrying out the denoising of the operation data; the operation data comprises water level, flow, power, voltage and current; inputting the de-noised operation data into an encoder to construct a state space, and compressing the operation data through the state space to obtain a state vector; determining a control action of the hydropower station by using a depth deterministic strategy gradient algorithm according to the state vector, and obtaining an initial control signal according to the control action; the method comprises the steps of determining a control action of a hydropower station by adopting a depth deterministic strategy gradient algorithm, obtaining an initial control signal according to the control action, calculating a reward value of the initial control signal, updating parameters of a strategy network of the depth deterministic strategy gradient algorithm according to the reward value, and outputting a final control signal to control the hydropower station. According to the method, the control strategy can be automatically learned and optimized, so that the control precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to a control method and system for a hydropower station, belonging to the technical field of water conservancy and hydropower control engineering. Background Art

[0002] In the technical field of water conservancy and hydropower control engineering, the operation control of hydropower stations has always been a key and complex issue. Traditional control methods for hydropower stations mainly rely on manual operation and some simple automated control systems. These methods usually monitor and control the operation data of hydropower stations based on fixed rules and preset parameters. For example, the power output of generators is adjusted according to simple thresholds of water level and flow rate, or equipment is protected from damage by monitoring voltage and current. These traditional control methods can ensure the basic operation of hydropower stations to a certain extent, but they often seem inadequate when faced with complex operating environments and changing working conditions.

[0003] The patent document with the patent number "CN117908456B" discloses a monitoring method and system for a hoist of a hydropower station based on deep learning. The problems of this method are as follows: The deep reinforcement learning (DRL) model, especially the policy gradient algorithm (DPG), is adopted. During the training process, non-convex optimization problems need to be processed, and techniques such as the Adam optimizer, regularization terms, learning rate decay, and gradient clipping are used to optimize the model. In addition, a priority experience replay mechanism needs to be introduced, which increases the complexity and difficulty of model training. A complex simulation environment needs to be constructed and tested and optimized in the simulation environment. This not only increases the development cost and time of the system but also may introduce additional errors. Summary of the Invention

[0004] In order to solve the problems existing in the above-mentioned prior art, the present invention proposes a control method and system for a hydropower station.

[0005] The technical solution of the present invention is as follows:

[0006] On the one hand, the present invention provides a control method for a hydropower station, including the following steps:

[0007] Collect the operation data of the hydropower station and perform noise reduction processing on the operation data; the operation data includes water level, flow rate, power, voltage, and current;

[0008] Input the operation data after noise reduction processing into an encoder to construct a state space, and compress the operation data through the state space to obtain a state vector;

[0009] Determine the control action of the hydropower station according to the state vector using the deep deterministic policy gradient algorithm, and obtain an initial control signal according to the control action;

[0010] Calculate the reward value of the initial control signal, update the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, and output the final control signal to control the hydropower station.

[0011] As a preferred embodiment, the denoising processing method is:

[0012] P ct (t) = P raw (t) + α·ΔP(t) + β·Smooth(P raw (t));

[0013] Among them, P ct (t) represents the operation data at time t after denoising processing, P raw (t) represents the operation data at time t before denoising processing, α represents a preset noise correction coefficient, ΔP(t) represents the change amount of the operation data, β represents a preset smoothing coefficient, and Smooth(P raw (t)) represents the smoothed P raw (t), and t represents time;

[0014] The calculation method of the change amount ΔP(t) is:

[0015] ΔP(t) = P raw (t) - P raw (t - Δt);

[0016] Among them, Δt represents a preset time variable.

[0017] As a preferred embodiment, the method for obtaining the state vector is:

[0018] S t = f eder (P ct (t); θ e );

[0019] The parameters θ e of the encoder network are obtained by the following method:

[0020]

[0021] Among them, S t represents the state vector at time t, f eder () represents the encoder network, θ d represents the parameters of the decoder network, and f doer () represents the decoder network.

[0022] As a preferred embodiment, the method for obtaining the control action is:

[0023] A t=π(S t θ π )+N(0,σ);

[0024]

[0025] Among them, A t represents the vector of control actions at time t, π represents the preset strategy network, θ π represents the parameters of the policy network, N(0,σ) represents Gaussian noise, θ trget represents the parameters of the target network, represents the preset soft update coefficient, θ represents the parameters of the current network, and σ represents the preset standard deviation.

[0026] As a preferred embodiment, the method for obtaining the control signal is:

[0027]

[0028] Among them, U t represents the control signal at time t, μ i (), μ j () represents the fuzzy membership function, us i Represents the output of the i-th fuzzy rule, and n represents the maximum number of preset fuzzy rules.

[0029] As a preferred embodiment, the reward value is calculated as follows:

[0030]

[0031] Among them, R t represents the reward value at time t, Y t Indicates the actual output, Y ref Indicates the preset reference output, penty(U t ) represents the penalty term of the control signal, α, β, γ represent the preset weight coefficients, Indicates the actual output Y t The first-order derivative with respect to time t, Δt represents the preset time variable, U(τ) 2 represents the square of the control signal at time τ.

[0032] As a preferred implementation, the parameter updating method of the policy network is:

[0033] δ t =R t +γ·V(S t+1 θ v )-V(S t θ v );

[0034]

[0035] Among them, δ t represents the time difference error at time t, V() represents the state value function, and θ v represents the value network parameters, and η v represents the learning rate of the value network. represents the gradient of V(S t ; θ v ) with respect to the value network parameters θ v , and η π represents the learning rate of the policy network. represents the gradient of π(A t |S t ; θ π ) with respect to the policy network parameters , and π() represents the state policy function.

[0036] On the other hand, the present invention also provides a control system for a hydropower station, including:

[0037] Data acquisition module: Collect the operation data of the hydropower station and perform noise reduction processing on the operation data; the operation data includes water level, flow rate, power, voltage, and current.

[0038] State vector module: Input the operation data after noise reduction processing into an encoder to construct a state space, and compress the operation data through the state space to obtain a state vector.

[0039] Control action module: Determine the control action of the hydropower station using the deep deterministic policy gradient algorithm according to the state vector, and obtain an initial control signal according to the control action.

[0040] Control signal module: Calculate the reward value of the initial control signal, update the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, and output a final control signal to control the hydropower station.

[0041] The present invention has the following beneficial effects:

[0042] The present invention collects the operation data of a hydropower station (including water level, flow rate, power, voltage, and current) and performs noise removal processing, ensuring the accuracy of the input data and providing a reliable basis for subsequent control processes. An encoder is used to construct a state space, and the operation data is compressed through the state space to obtain a state vector, effectively reducing the dimension of the data and improving the processing speed. The deep deterministic policy gradient algorithm is adopted to determine the control actions of the hydropower station, and an initial control signal is obtained according to the control actions. This method can automatically learn and optimize the control strategy, thereby improving the accuracy and efficiency of control. By calculating the reward value of the initial control signal and updating the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, a closed-loop optimization process is formed. This reward mechanism can motivate the algorithm to continuously search for better control strategies, thereby improving the overall performance of the hydropower station. When obtaining the control signal, the method of fuzzy control is adopted, and the control signal is output through fuzzy membership functions and fuzzy rules. This method can handle uncertainty and ambiguity, enhancing the robustness of the control system. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of the implementation of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0045] It should be understood that the step numbers used in the text are only for convenient description and do not limit the execution order of the steps.

[0046] It should be understood that the terms used in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to include the plural forms.

[0047] The terms "comprising" and "including" indicate the presence of the described features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components, and / or their combinations.

[0048] The term " / and / " refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0049] Example 1:

[0050] See Figure 1 , the present invention provides a control method for a hydropower station, comprising the following steps:

[0051] Collect the operation data of the hydropower station and perform noise reduction processing on the operation data; the operation data includes water level, flow rate, power, voltage, and current;

[0052] Input the operation data after noise reduction processing into an encoder to construct a state space, and compress the operation data through the state space to obtain a state vector;

[0053] Determine the control action of the hydropower station according to the state vector using the deep deterministic policy gradient algorithm, and obtain an initial control signal according to the control action;

[0054] Calculate the reward value of the initial control signal, update the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, and output the final control signal to control the hydropower station.

[0055] As a preferred embodiment, the noise reduction processing method is:

[0056] P ct (t) = P raw (t) + α·ΔP(t) + β·Smooth(P raw (t));

[0057] Among them, P ct (t) represents the operation data at time t after noise reduction processing, P raw (t) represents the operation data at time t before noise reduction processing, α represents a preset noise correction coefficient, ΔP(t) represents the change amount of the operation data, β represents a preset smoothing coefficient, Smooth(P raw (t)) represents the smoothed P raw (t), and t represents time;

[0058] The calculation method of the change amount ΔP(t) is:

[0059] ΔP(t) = P raw (t) - P raw (t - Δt);

[0060] Among them, Δt represents a preset time variable.

[0061] As a preferred embodiment, the method for obtaining the state vector is:

[0062] S t = f eder (P ct(t); θ e );

[0063] The parameters θ of the encoder network e are obtained by the following method:

[0064]

[0065] where S t represents the state vector at time t, and f eder () represents the encoder network, and θ d represents the parameters of the decoder network, and f doer () represents the decoder network.

[0066] As a preferred embodiment, the method for obtaining the control action is:

[0067] A t = π(S t ; θ π ) + N(0, σ);

[0068]

[0069] where A t represents the vector of the control action at time t (preset by the staff according to the on-site situation, such as the control actions of opening and closing the gates of a hydropower station), π represents the preset policy network, and θ π represents the parameters of the policy network, N(0, σ) represents Gaussian noise, which is a random number with a mean of 0 and a standard deviation of σ, and θ trget represents the parameters of the target network, represents the preset soft update coefficient, θ represents the parameters of the current network, and σ represents the preset standard deviation.

[0070] As a preferred embodiment, the method for obtaining the control signal is:

[0071]

[0072] where U t represents the control signal at time t, μ i (), μ j () represent the fuzzy membership functions, μ i (A t ) is the membership degree of the vector A of the control action at time t t to the i-th fuzzy rule, and μ j (A t ) is the membership degree of the vector A of the control action at time t t to the j-th fuzzy rule, and us i represents the output of the i-th fuzzy rule, and n represents the maximum number of preset fuzzy rules.

[0073] As a preferred embodiment, the calculation method of the reward value is as follows:

[0074]

[0075] Among them, R t represents the reward value at time t, Y t represents the actual output, Y ref represents the preset reference output, penty(U t ) represents the penalty term of the control signal, and α, β, γ represent preset weight coefficients. represents the first derivative of the actual output Y t with respect to time t (the rate of change of the actual output Y t with time), Δt represents the preset time variable, and U(τ) 2 represents the square of the control signal at time τ.

[0076] As a preferred embodiment, the method for updating the parameters of the policy network is as follows:

[0077] δ t = R t + γ·V(S t+1 ; θ v ) - V(S t ; θ v );

[0078]

[0079] Among them, δ t represents the temporal difference error at time t, V() represents the state value function, θ v represents the value network parameters, η v represents the learning rate of the value network, represents the gradient of V(S t ; θ v ) with respect to the value network parameters θ v , η π represents the learning rate of the policy network, represents the gradient of π(A t |S t ; θ π ) with respect to the policy network parameters , and π() represents the state policy function.

[0080] On the other hand, the present invention also provides a control system for a hydropower station, including:

[0081] Data acquisition module: Collect the operation data of the hydropower station and perform noise removal processing on the operation data; the operation data includes water level, flow rate, power, voltage, and current.

[0082] State vector module: Input the operation data after noise removal processing into the encoder to construct a state space, and compress the operation data through the state space to obtain a state vector.

[0083] Control action module: Determine the control action of the hydropower station according to the state vector using the deep deterministic policy gradient algorithm, and obtain an initial control signal according to the control action.

[0084] Control signal module: Calculate the reward value of the initial control signal, update the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, and output the final control signal to control the hydropower station.

[0085] This system is used to implement the method in Embodiment 1, which will not be elaborated here.

[0086] In the embodiments of the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent the situation where A exists alone, A and B exist simultaneously, and B exists alone. Where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0087] Those of ordinary skill in the art can realize that the units and algorithm steps described in the embodiments disclosed herein can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0088] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, which will not be elaborated here.

[0089] In several embodiments provided by the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or a part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs.

[0090] The above are only the embodiments of the present invention, and thus do not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A control method for a hydropower station, characterized in that, It includes the following steps: Collect the operation data of the hydropower station and perform noise reduction processing on the operation data; the operation data includes water level, flow rate, power, voltage, and current; Input the operation data after noise reduction processing into the encoder to construct a state space, and compress the operation data through the state space to obtain a state vector; Determine the control action of the hydropower station according to the state vector using the deep deterministic policy gradient algorithm, and obtain an initial control signal according to the control action; Calculate the reward value of the initial control signal, update the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, and output the final control signal to control the hydropower station.

2. The control method for a hydropower station according to claim 1, characterized in that The noise reduction processing method is: P ct P(t) = raw P(t) + α·ΔP(t) + β·Smooth(P raw (t)); Among them, P ct (t) represents the operation data at time t after noise removal, P raw (t) represents the operation data at time t before noise removal, α represents a preset noise correction coefficient, ΔP(t) represents the change in operation data, β represents a preset smoothing coefficient, Smooth(P raw (t)) represents the smoothed P raw (t), and t represents time; The calculation method of the change amount ΔP(t) is: ΔP(t) = P raw (t) - P raw (t - Δt); Where, Δt represents a preset time variable.

3. The control method for a hydropower station according to claim 2, characterized in that, The method for obtaining the state vector is: S t = f eder (P ct (t); θ e ); The parameters θ of the encoder network e are obtained by the following method: Among them, S t represents the state vector at time t, f eder () represents the encoder network, θ d represents the parameters of the decoder network, f doer () represents the decoder network.

4. The control method for a hydropower station according to claim 3, characterized in that The method for obtaining the control action is: A t = π(S t ; θ π ) + N(0, σ); Among them, A t represents the vector of control actions at time t, π represents the preset policy network, and θ π represents the parameters of the policy network, N(0,σ) represents Gaussian noise, and θ trget represents the parameters of the target network, represents the preset soft update coefficient, θ represents the parameters of the current network, and σ represents the preset standard deviation.

5. The control method for a hydropower station according to claim 4, characterized in that, The method for obtaining the control signal is: Among them, U t represents the control signal at time t, μ i (), μ j ( ) represents the fuzzy membership function, us i represents the output of the i-th fuzzy rule, and n represents the maximum number of preset fuzzy rules.

6. The control method for a hydropower station according to claim 5, characterized in that, The calculation method of the reward value is: Among them, R t represents the reward value at time t, Y t represents the actual output, and Y ref represents the preset reference output. penty(U t ) represents the penalty term of the control signal, and α, β, γ represent the preset weight coefficients. represents the first derivative of the actual output Y t with respect to time t, and Δt represents the preset time variable. U(τ) 2 represents the square of the control signal at time τ.

7. The control method for a hydropower station according to claim 6, wherein, The method for updating the parameters of the policy network is: δ t = R t + γ·V(S t+1 ; θ v ) - V(S t ; θ v ); Among them, δ t represents the time difference error at time t, V() represents the state value function, θ v represents the value network parameters, η v represents the learning rate of the value network, represents V(S t ; θ v ) for the gradient of the value network parameters θ v , η π represents the learning rate of the policy network, represents π(A t |S t ; θ π ) for the gradient of the policy network parameters , and π() represents the state policy function.

8. A control system for a hydropower station, characterized in that, It includes: Data acquisition module: Collect the operation data of the hydropower station and perform noise reduction processing on the operation data; the operation data includes water level, flow rate, power, voltage, and current; State vector module: Input the operation data after noise reduction processing into the encoder to construct a state space, and compress the operation data through the state space to obtain a state vector; Control action module: Determine the control action of the hydropower station according to the state vector using the deep deterministic policy gradient algorithm, and obtain an initial control signal according to the control action; Control signal module: Calculate the reward value of the initial control signal, update the parameters of the policy network of the deep deterministic policy gradient algorithm according to the reward value, and output the final control signal to control the hydropower station.

9. The control system for a hydropower station according to claim 8, wherein, For the data acquisition module, the noise reduction processing method is: P ct P(t) = raw P(t) + α·ΔP(t) + β·Smooth(P raw (t)); Among them, P ct (t) represents the operation data at time t after noise removal processing, and P raw (t) represents the operation data at time t before noise removal processing. α represents a preset noise correction coefficient, ΔP(t) represents the change amount of the operation data, β represents a preset smoothing coefficient, and Smooth(P raw (t)) represents the smoothed P raw (t), where t represents time; The calculation method of the change amount ΔP(t) is: ΔP(t) = P raw (t) - P raw (t - Δt); Where, Δt represents a preset time variable.

10. The control system for a hydropower station according to claim 9, wherein For the state vector module, the method for obtaining the state vector is: S t = f eder (P ct (t); θ e ); The parameters θ of the encoder network e are obtained by the following method: Among them, S t represents the state vector at time t, f eder () represents the encoder network, θ d represents the parameters of the decoder network, f doer () represents the decoder network.

Images