A hydropower station gate control method based on artificial intelligence

Abstract

The present application relates to the field of hydropower station control technology, and in particular to a hydropower station gate control method based on artificial intelligence, including: inputting the real-time state of the gate into a reinforcement learning model to obtain an opening adjustment amount, and controlling the hydropower station gate based on the opening adjustment amount at the current moment, the real-time state includes the gate opening and a variety of environmental data, and the environmental data includes rainfall, river flow, and upstream and downstream water levels. The technical solution of the present application can adaptively adjust the hydropower station gate opening to achieve precise control of the hydropower station gate opening.

Description

Technical Field

[0001] The present application relates to the field of hydropower station control technology, and in particular to a hydropower station gate control method based on artificial intelligence. Background Art

[0002] The operation of a hydropower station needs to take into account multiple benefits such as flood control, power generation, shipping, and ecology, making daily work more complicated. The gates of a hydropower station are directly related to the safe operation and power generation efficiency of the hydropower station. Therefore, in order to control the flood discharge or interception operation of the dam and ensure the safe and reliable operation of the hydropower station, it is necessary to adjust the gate opening of the hydropower station in a timely manner.

[0003] At present, the patent application document with the publication number CN117908456A discloses a method and system for monitoring the gate hoist of a hydropower station based on deep learning. The method includes: collecting monitoring data of the gate hoist of a hydropower station and integrating the data in multimodal data; constructing a simulated hydroelectric motor gate hoist operating environment and establishing a deep reinforcement learning algorithm DRL model; using a multimodal data set to train the DRL model, test running and evaluating it in a simulated environment, and optimizing the model according to the evaluation results; integrating the model into the monitoring operation system to complete the automatic monitoring of the gate hoist of a hydropower station. Among them, in the deep reinforcement learning algorithm DRL model, the state space S is used to represent the event state of the simulated hydroelectric motor gate hoist, which includes the parameters collected by various sensors; the action space A containing all executable operations of the hydroelectric motor gate hoist is used to represent the set of executable action behaviors in the model, and the reward function R(s, a) is designed to calculate the reward value according to the state s and the action a; the reward function reflects the expectation of the gate hoist operation.

[0004] The above method can determine the operations performed by the hydropower station's gate opening machine based on the parameters collected by various sensors. However, the external environmental data of the hydropower station will change in different periods and seasons. The above method cannot flexibly and adaptively adjust the hydropower station's gate opening according to different periods and seasons, and cannot achieve precise control of the hydropower station's gate opening. Summary of the invention

[0005] In order to solve the technical problem of being unable to achieve precise control of the opening of a hydropower station gate, the present application provides a hydropower station gate control method based on artificial intelligence, which can adaptively adjust the opening of a hydropower station gate to achieve precise control of the opening of a hydropower station gate.

[0006] The present application provides a hydropower station gate control method based on artificial intelligence, the control method comprising: inputting the real-time state of the gate into a reinforcement learning model to obtain an opening adjustment amount, and controlling the hydropower station gate based on the opening adjustment amount at the current moment, the real-time state includes the gate opening and a variety of environmental data, the environmental data including rainfall, river flow and upstream and downstream water levels; the updating method of the reinforcement learning model comprises: collecting a variety of real-time rewards of the opening adjustment amount, the real-time rewards include power generation rewards and flow rewards, the power generation rewards are power generation power, and the flow rewards are negatively correlated with the absolute value of the difference between the river flow and the expected flow; calculating the environmental change amount of each environmental data at the next moment, inputting all environmental change amounts into a prediction model, and obtaining the reduction amount of each real-time reward; calculating the weight of each real-time reward, the weight is positively correlated with the reduction amount of the real-time reward, and is negatively correlated with the value of the real-time reward at the current moment; weighted summing the multiple real-time rewards according to the weights to obtain a reward function value, back-propagating the reinforcement learning model according to the reward function value, and completing the update process at the next moment.

[0007] Collect the real-time status of the gate, which includes the gate opening and various environmental data; input the real-time status into the reinforcement learning model to output the opening adjustment amount, which is any allowable adjustment amount that can ensure that the hydropower station gate is not damaged; complete the control of the hydropower station gate at the current moment based on the opening adjustment amount, and realize real-time control of the hydropower station gate.

[0008] Preferably, the reinforcement learning model is a DQN model, the input of the reinforcement learning model is the real-time state at any time, the output is the predicted reward value of each allowable adjustment amount, and the opening adjustment amount is the allowable adjustment amount corresponding to the maximum predicted reward value.

[0009] A model structure of an end-to-end reinforcement learning model is given, which can output the predicted reward value of each allowable adjustment amount, ensure that the opening adjustment amount is any one of all allowable adjustment amounts, and avoid gate wear.

[0010] Preferably, calculating the environmental change amount of each environmental data at the next moment includes: subtracting the value of the environmental data at the current moment from the value of each environmental data at the next moment to obtain the environmental change amount of the environmental data.

[0011] Preferably, the method for constructing the prediction model comprises: constructing a prediction model for each real-time reward, The prediction model for real-time rewards is: , is the amount of environmental data, For the Environmental data for the The undetermined coefficient of the real-time reward, For the The amount of environmental change of the environmental data, For the The environmental change of each environmental data in the historical update process and the reduction of each real-time reward are used as training samples, and the least squares method is used on the training samples to determine the undetermined coefficients in the prediction model to complete the construction of the prediction model.

[0012] Since gate control has to take into account multiple benefits such as flood control and power generation, changes in the environmental data of the hydropower station will affect the gate control of the hydropower station. The prediction model can predict the reduction in each real-time reward at the next moment. Different real-time rewards reflect the gate control effect from different angles. The reduction in real-time rewards can reflect the degree of decline in the gate control effect at the corresponding angle, providing direction for the update of the reinforcement learning model at the next moment.

[0013] Preferably, the prediction model is a BP neural network, and the method for constructing the prediction model includes: in the historical update process, taking the environmental change amount of each environmental data as the input sample, and taking the reduction amount of each real-time reward as the sample label of the input sample; inputting the input sample into the prediction model to obtain the output result, and taking the mean square error between the output result and the sample label as the loss function to train the prediction model, until the value of the loss function is less than the preset loss, and the construction of the prediction model is completed.

[0014] Preferably, The weight of the real-time reward for:

[0015] , For the The reduction in real-time rewards, For the current moment The value of the real-time reward, express function, For the The initial weight of the real-time reward; is the normalization coefficient, which is the sum of the initial weights of all real-time rewards.

[0016] The greater the reduction in the real-time reward, the greater the decrease in the gate control effect at the angle corresponding to the real-time reward at the next moment. In order to ensure that the overall control effect of the gate is optimal, the weight of the real-time reward needs to be increased so that the reinforcement learning model pays more attention to the value of the real-time reward; the smaller the value of the real-time reward during the update process at the current moment, the worse the gate control effect corresponding to the real-time reward, and the weight of the real-time reward needs to be increased to ensure that the opening adjustment output by the reinforcement learning model at the next moment meets the control requirements.

[0017] Preferably, back-propagating the reinforcement learning model according to the reward function value includes: taking the predicted reward value corresponding to the opening adjustment amount as the output reward value; inputting the real-time state of the gate at the next moment into the reinforcement learning model, and taking the maximum predicted reward value as the future reward value; constructing the reinforcement learning loss according to the reward function value, and back-propagating the reinforcement learning model to update the reinforcement learning model; the reinforcement learning loss for: , is the output reward value, is the reward function value, is the future reward value, is the loss factor.

[0018] Preferably, after completing the update process at the next moment, the control method also includes: inputting the real-time state of the gate at the next moment into the updated reinforcement learning model, and at the next moment, controlling the hydropower station gate based on the opening adjustment amount output by the updated reinforcement learning model.

[0019] At the next moment, the reinforcement learning model is updated, and the real-time state of the gate at the next moment is input into the updated reinforcement learning model to ensure that the reinforcement learning model at the next moment can output accurate opening adjustment value and realize precise control of the gate opening of the hydropower station.

[0020] The technical solution of this application has the following beneficial technical effects:

[0021] The above-mentioned artificial intelligence-based hydropower station gate control method provided in the embodiment of the present application collects the real-time status of the gate, and the real-time status includes the gate opening and various environmental data; the real-time status is input into the reinforcement learning model to output the opening adjustment amount, and the opening adjustment amount is any allowable adjustment amount, which can ensure that the hydropower station gate is not damaged; the control of the hydropower station gate is completed at the current moment according to the opening adjustment amount, so as to realize real-time control of the hydropower station gate.

[0022] Furthermore, the reward function value of the reinforcement learning model includes multiple real-time rewards such as power generation reward and flow reward. The larger the reward function value, the better the gate control effect. The opening adjustment amount output by the reinforcement learning model is the allowable adjustment amount corresponding to the maximum value of the reward function value, which ensures the control effect of the hydropower station gate. Taking into account the influence of external environmental data on the gate control effect, the reduction amount of each real-time reward is predicted according to the environmental change amount of each environmental data at the next moment. When the reduction amount is larger, it means that more attention should be paid to the value of the corresponding real-time reward, and the weight of the corresponding real-time reward in the reward function value is increased. At the next moment, the reward function value is used to update the reinforcement learning model to ensure that the reinforcement learning model can output accurate opening adjustment amount at the next moment to achieve precise control of the opening of the hydropower station gate. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0024] Figure 1 is a flow chart of a hydropower station gate control method based on artificial intelligence according to an embodiment of the present application;

[0025] Figure 2 It is a flowchart of a method for updating a reinforcement learning model according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0027] It should be understood that when the application uses the terms "first", "second", etc., they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0028] This application provides a hydropower station gate control method based on artificial intelligence. Figure 1 FIG. 1 is a flow chart of a hydropower station gate control method based on artificial intelligence according to an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

[0029] S11, collecting the real-time status of the gate at the current moment, the real-time status includes the gate opening and a variety of environmental data, the environmental data includes rainfall, river flow and upstream and downstream water levels.

[0030] In one embodiment, at any time, the real-time status of the hydropower station gate is collected, and the real-time status includes the gate opening and the real-time value of each environmental data; specifically, the environmental data includes rainfall, river flow and upstream and downstream water levels.

[0031] It is understandable that when the river where the hydropower station is located is in the rainy season, its rainfall, river flow and upstream and downstream water levels will change. When the river where the hydropower station is located is in the dry season, its rainfall, river flow and upstream and downstream water levels are quite different from those in the rainy season. Therefore, when the hydropower station is in different periods or seasons, the value of its environmental data will change, and the environmental data can reflect the period or season in which the hydropower station is located.

[0032] In other optional embodiments, the environmental data also includes river flow rate.

[0033] S12, inputting the real-time state of the gate into the reinforcement learning model to obtain the opening adjustment amount, and controlling the gate of the hydropower station based on the opening adjustment amount at the current moment.

[0034] In one embodiment, the real-time state of the gate is input into the reinforcement learning model to obtain the opening adjustment amount, and the gate opening of the hydropower station is adjusted according to the opening adjustment amount to achieve the control of the hydropower station gate. The opening adjustment amount is any allowable adjustment amount.

[0035] Exemplarily, since the time interval between two adjacent moments is limited, in order to prevent the gate from being damaged due to excessive adjustment between adjacent moments and reducing the service life of the gate, all allowable adjustment amounts are set in advance; as an example, the allowable adjustment amounts include +1, +2, +3, 0, -1, -2, and -3, a total of 7 types, and the opening adjustment amount is any one of the 7 allowable adjustment amounts, +1 means that the gate opening is expanded by 1 degree, and -1 means that the gate opening is reduced by 1 degree.

[0036] In one embodiment, the reinforcement learning model is a DQN model, the input of the reinforcement learning model is the real-time state at any time, the output is the predicted reward value of each allowable adjustment amount, and the opening adjustment amount is the allowable adjustment amount corresponding to the maximum predicted reward value.

[0037] Specifically, the reinforcement learning model includes an input layer, multiple hidden layers and an output layer; the number of neurons in the input layer is equal to the dimension of the real-time state, and is used to receive the real-time state; the multiple hidden layers extract features of the real-time state to obtain feature vectors, and each hidden layer performs dimensionality transformation on the output result of the previous layer to achieve the effect of feature extraction; the number of neurons in the output layer is the number of allowable adjustment quantities, which is used to map the feature vectors into predicted reward values ​​for each allowable adjustment quantity, and the allowable adjustment quantity corresponding to the maximum predicted reward value is used as the opening adjustment quantity.

[0038] In this way, during the control process of the hydropower station gate, the real-time state at the current moment is input into the reinforcement learning model to obtain the opening adjustment amount, and the hydropower station gate is controlled according to the opening adjustment amount at the current moment to realize adaptive control of the hydropower station gate.

[0039] In one embodiment, in the above-mentioned artificial intelligence-based hydropower station gate control method, the reinforcement learning model determines the control effect. The reinforcement learning model is updated after each execution of the opening adjustment amount. Considering the impact of external environmental data on the gate control effect, before each update, it is necessary to determine the weights of multiple real-time rewards in the reward function value according to the change in environmental data, so that the updated reinforcement learning model can output accurate opening adjustment amount at the next moment. Please refer to Figure 2 , is a flow chart of a method for updating a reinforcement learning model according to an embodiment of the present application. According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted, which is described in detail as follows.

[0040] S21, collecting multiple real-time rewards of the opening adjustment amount, the real-time rewards include power generation rewards and flow rewards, the power generation reward is the power generation power, and the flow reward is negatively correlated with the absolute value of the difference between the river flow and the expected flow.

[0041] Among them, after executing the opening adjustment amount at the current moment, the power generation power of the hydropower station at the next moment is collected. The larger the power generation power is, the better the control effect of the opening adjustment amount is. Therefore, the power generation power is used as a power generation reward. The larger the power generation power is, the larger the power generation reward is. The river flow at the next moment is collected, and the absolute value of the difference between the river flow and the expected flow is calculated. The opposite or reciprocal of the absolute value of the difference is used as a flow reward. The larger the flow reward is, the closer the river flow at the next moment is to the expected flow, and the better the control effect of the opening adjustment amount is.

[0042] The expected flow is related to the river where the hydropower station is located and can be pre-set by technical personnel in this field; the average flow of the river in a preset time period can also be used as the expected flow, and the preset time period is a time period of the past quarter including the current moment.

[0043] In other embodiments, the real-time reward also includes a motor reward. Specifically, the real-time power of the motor at the next moment is collected, and the difference between the rated power and the real-time power is used as the motor reward. The larger the motor reward, the smaller the real-time power is than the rated power, and the better the control effect of the opening adjustment amount.

[0044] In this way, after the opening adjustment is executed, all real-time rewards are collected, and each real-time reward reflects the control effect of the opening adjustment from different aspects. The larger the real-time reward, the better the control effect of the opening adjustment.

[0045] S22, calculate the environmental change amount of each environmental data at the next moment, input all environmental change amounts into the prediction model, and obtain the reduction amount of each real-time reward.

[0046] Calculating the environmental change amount of each environmental data at the next moment includes: subtracting the value of the environmental data at the current moment from the value of each environmental data at the next moment to obtain the environmental change amount of the environmental data.

[0047] In this way, the environmental change amount of each environmental data is obtained, which can reflect the environmental change at the next moment relative to the current moment. Subsequently, the weight of each real-time reward in the reinforcement learning model update process is adjusted based on the environmental change, so that the updated reinforcement learning model can adapt to the environment at the next moment.

[0048] In one embodiment, the input of the prediction model is the environmental change amount of each environmental data at the next moment, and the output is the reduction amount of each real-time reward at the next moment; wherein the predicted value of the real-time reward at the next moment is equal to the reduction amount plus the value of the real-time reward at the current moment.

[0049] Specifically, one real-time reward corresponds to one prediction model, and the method for constructing the prediction model includes: constructing a prediction model for each real-time reward, The prediction model for real-time rewards is:

[0050] , is the amount of environmental data, For the Environmental data and The undetermined coefficient of the real-time reward, For the The amount of environmental change of the environmental data, For the The environmental change of each environmental data in the historical update process and the reduction of each real-time reward are used as training samples, and the least squares method is used on the training samples to determine the undetermined coefficients in the prediction model to complete the construction of the prediction model.

[0051] For example, the reinforcement learning model will perform an update process at each moment; in any historical update process, the values ​​of each environmental data at the corresponding moment of the historical update process are collected and recorded as , and the values ​​of various real-time rewards, denoted as ; Further, collect the values ​​of each environmental data at the previous moment of the corresponding moment of the historical update process , and the values ​​of various real-time rewards, denoted as ; Then the environmental change of each environmental data in the historical update process is , the reduction amount of each real-time reward is , get a training sample; multiple training samples can be obtained by the same method.

[0052] In another embodiment, the prediction model is a BP neural network, and the method for constructing the prediction model includes: in the historical update process, the environmental change amount of each environmental data is used as an input sample, and the reduction amount of each real-time reward is used as the sample label of the input sample; the input sample is input into the prediction model to obtain an output result, and the mean square error between the output result and the sample label is used as a loss function to train the prediction model, until the value of the loss function is less than the preset loss, the construction of the prediction model is completed. Wherein, the preset loss value is 0.01.

[0053] In this way, after executing the opening adjustment at the current moment, the environmental change amount of each environmental data at the next moment is collected, and the environmental change amount of each environmental data is input into the prediction model to predict the reduction amount of each real-time reward, accurately quantifying the impact of the environmental data at the next moment on each real-time reward.

[0054] S23, calculating the weight of each real-time reward, wherein the weight is positively correlated with the reduction amount of the real-time reward and negatively correlated with the value of the real-time reward at the current moment.

[0055] Among them, since different implementation rewards comprehensively consider the control effect of the gate from different angles, the greater the reduction in any real-time reward, the greater the impact of the environmental data at the next moment on the real-time reward, and the greater the decrease in the gate control effect at the corresponding angle of the real-time reward at the next moment. Therefore, in order to ensure that the overall control effect of the gate is optimal, it is necessary to increase the weight of the real-time reward so that the reinforcement learning model pays more attention to the value of the real-time reward, ensuring that the opening adjustment output by the reinforcement learning model at the next moment can achieve a better control effect; that is, the weight of the real-time reward is positively correlated with the reduction in the real-time reward.

[0056] Furthermore, the reinforcement learning model will be updated once at each moment. The smaller the value of the real-time reward during the update process at the current moment, the worse the gate control effect corresponding to the real-time reward. The weight of the real-time reward needs to be increased to ensure that the opening adjustment amount output by the reinforcement learning model at the next moment meets the control requirements, that is, the weight of the real-time reward is negatively correlated with the value of the real-time reward at the current moment.

[0057] Specifically, no. The weight of the real-time reward for: , For the The reduction in real-time rewards, For the current moment The value of the real-time reward, express function, For the The initial weight of the real-time reward; is the normalization coefficient, which is equal to the sum of the initial weights of all real-time rewards. The value range of is from negative infinity to positive infinity. The function is a normalization function used for Normalized to 0-1.

[0058] Understandably, the weights The calculation formula Replace with Other forms to calculate the The weight of the real-time reward is not limited in this application.

[0059] S24, weighted summation of multiple real-time rewards is performed according to the weights to obtain a reward function value, and the reinforcement learning model is back-propagated according to the reward function value to complete the update process at the next moment.

[0060] After obtaining the weight of each real-time reward, multiple real-time rewards are weighted and summed to obtain a reward function value, which satisfies the relationship: , For the The weight of the real-time reward, For the current moment The value of the real-time reward, It is the reward function value after controlling the gate of the hydropower station based on the opening adjustment amount.

[0061] In one embodiment, back-propagating the reinforcement learning model according to the reward function value includes: taking the predicted reward value corresponding to the opening adjustment amount as the output reward value; inputting the real-time state of the gate at the next moment into the reinforcement learning model, and taking the maximum predicted reward value as the future reward value; constructing the reinforcement learning loss according to the reward function value, and back-propagating the reinforcement learning model to update the reinforcement learning model; the reinforcement learning loss for: , is the output reward value, is the reward function value, is the future reward value, is the loss factor.

[0062] The real-time status of the gate at the next moment includes the gate opening at the next moment and various environmental data. The value of is 0.5.

[0063] In this way, after obtaining the opening adjustment amount according to the reinforcement learning model at the current moment and controlling the gate of the hydropower station according to the opening adjustment amount, multiple real-time rewards of the opening adjustment amount and various environmental data at the next moment can be collected at the next moment; the weights of multiple real-time rewards are determined according to the environmental change amount of each environmental data, and then the reward function value is obtained; the reward function value is used to update the reinforcement learning model at the next moment, and the updated reinforcement learning model is used to control the gate of the hydropower station at the next moment.

[0064] In one embodiment, after completing the update process at the next moment, the control method also includes: inputting the real-time state of the gate at the next moment into the updated reinforcement learning model, and at the next moment, controlling the hydropower station gate based on the opening adjustment amount output by the updated reinforcement learning model.

[0065] The above-mentioned artificial intelligence-based hydropower station gate control method provided in the embodiment of the present application collects the real-time status of the gate, and the real-time status includes the gate opening and various environmental data; the real-time status is input into the reinforcement learning model to output the opening adjustment amount, and the opening adjustment amount is any allowable adjustment amount, which can ensure that the hydropower station gate is not damaged; the control of the hydropower station gate is completed at the current moment according to the opening adjustment amount, so as to realize real-time control of the hydropower station gate.

[0066] Furthermore, the reward function value of the reinforcement learning model includes multiple real-time rewards such as power generation reward and flow reward. The larger the reward function value, the better the gate control effect. The opening adjustment amount output by the reinforcement learning model is the allowable adjustment amount corresponding to the maximum value of the reward function value, which ensures the control effect of the hydropower station gate. Taking into account the influence of external environmental data on the gate control effect, the reduction amount of each real-time reward is predicted according to the environmental change amount of each environmental data at the next moment. When the reduction amount is larger, it means that more attention should be paid to the value of the corresponding real-time reward, and the weight of the corresponding real-time reward in the reward function value is increased. At the next moment, the reward function value is used to update the reinforcement learning model to ensure that the reinforcement learning model can output accurate opening adjustment amount at the next moment to achieve precise control of the opening of the hydropower station gate.

[0067] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application.

Claims

1. A hydropower station gate control method based on artificial intelligence, characterized in that: The control method comprises: The real-time state of the gate is input into the reinforcement learning model to obtain the opening adjustment amount, and the hydropower station gate is controlled based on the opening adjustment amount at the current moment. The real-time state includes the gate opening and a variety of environmental data, including rainfall, river flow, and upstream and downstream water levels; The update methods of reinforcement learning models include: Collect multiple real-time rewards of the opening adjustment amount, the real-time rewards include power generation rewards and flow rewards, the power generation reward is the power generation, and the flow reward is negatively correlated with the absolute value of the difference between the river flow and the expected flow; calculate the environmental change amount of each environmental data at the next moment, input all environmental change amounts into the prediction model, and obtain the reduction amount of each real-time reward; calculate the weight of each real-time reward, the weight is positively correlated with the reduction amount of the real-time reward, and negatively correlated with the value of the real-time reward at the current moment; weighted sum of multiple real-time rewards according to the weights to obtain the reward function value, and back-propagate the reinforcement learning model according to the reward function value to complete the update process at the next moment; No. The weight of the real-time reward for: , For the The reduction in real-time rewards, For the current moment The value of the real-time reward, express function, For the The initial weight of the real-time reward; is the normalization coefficient, which is the sum of the initial weights of all real-time rewards; The method for constructing the prediction model includes: Build a prediction model for each real-time reward, The prediction model for real-time rewards is: , is the amount of environmental data, is the jth environmental data for the The undetermined coefficient of the real-time reward, is the environmental change of the jth environmental data, For the The reduction in real-time rewards; The environmental changes of each environmental data in the historical update process and the reduction of each real-time reward are used as training samples. The least squares method is used on the training samples to determine the undetermined coefficients in the prediction model to complete the construction of the prediction model.

2. A hydropower station gate control method based on artificial intelligence as claimed in claim 1, characterized in that: The reinforcement learning model is a DQN model, the input of the reinforcement learning model is the real-time state at any time, the output is the predicted reward value of each allowable adjustment amount, and the opening adjustment amount is the allowable adjustment amount corresponding to the maximum predicted reward value.

3. The artificial intelligence-based hydropower station gate control method according to claim 1, characterized in that: Calculating the environmental change of each environmental data at the next moment includes: The value of each environmental data at the next moment is subtracted from the value of the environmental data at the current moment to obtain the environmental change amount of the environmental data.

4. The artificial intelligence-based hydropower station gate control method according to claim 1, characterized in that: Backpropagation of the reinforcement learning model based on the reward function value includes: The predicted reward value corresponding to the opening adjustment amount is used as the output reward value; the real-time state of the gate at the next moment is input into the reinforcement learning model, and the maximum predicted reward value is used as the future reward value; Construct reinforcement learning loss based on the reward function value, back-propagate the reinforcement learning model, and update the reinforcement learning model; reinforcement learning loss for: , is the output reward value, is the reward function value, is the future reward value, is the loss factor.

5. A hydropower station gate control method based on artificial intelligence as claimed in any one of claims 1 to 4, characterized in that: After completing the update process at the next moment, the control method further includes: The real-time state of the gate at the next moment is input into the updated reinforcement learning model, and at the next moment, the gate of the hydropower station is controlled based on the opening adjustment amount output by the updated reinforcement learning model.

Images