Wind power short-term prediction method and system based on deep reinforcement learning

Abstract

The invention provides a wind power short-term prediction method and system based on deep reinforcement learning, and relates to the field of deep learning, and the method comprises the steps: obtaining an initial data set representing historical wind power and related environment data; detecting the initial data set by adopting a Hampel recognizer, and cleaning the initial data set to obtain a target data set; a TD3 algorithm is introduced, a neural network framework of the introduced TD3 algorithm is determined, and the neural network framework comprises an Actor network group and a Critic network group; and performing model training based on the target data set and the neural network framework to obtain a target model so as to predict the wind power and obtain prediction information representing the wind power at the future moment. According to the method, the TD3 algorithm is introduced, a neural network framework is trained based on the TD3 algorithm, the problem of over-estimation of a Q value by a traditional DDPG algorithm can be solved, the accuracy of wind power prediction is further improved, and the problem of large prediction deviation during wind power prediction can be effectively solved.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and specifically to a method and system for short-term wind power prediction based on deep reinforcement learning. Background Art

[0002] As a renewable clean energy, wind power accounts for a large proportion of all power generation methods. However, the instability of wind power has greatly hindered its development. Therefore, accurate prediction of wind power is particularly important. Accurate wind power prediction helps adjust wind turbine operation strategies, reduce energy waste and economic losses, and supports electricity market participation and renewable energy policy formulation, promoting the sustainable development of clean energy.

[0003] In related technologies, machine learning algorithms can be used to predict the power output of wind farms. However, statistical methods have limited predictive capabilities when faced with complex terrain and sudden weather changes. High-resolution numerical weather forecast models can be used to predict the power output of wind farms. Physical models can more accurately consider the impact of terrain and meteorological conditions on wind speed, thereby improving the accuracy of predictions. However, physical modeling methods require a large amount of computing resources and high-precision geographic data, and have strict requirements on the accuracy of model parameters and initial conditions, which limits their widespread application in practical applications.

[0004] Deep reinforcement learning has shown great advantages in various prediction tasks and has achieved good performance in complex environments. The Q-learning method can be used to predict wind power. The DDPG algorithm can also be used to achieve short-term prediction of wind power. However, in the process of using DDPG to predict short-term wind power, the DDPG algorithm has the problem of overestimating the Q value, which will lead to inaccurate prediction results. In addition, when facing extreme situations, such as short-term strong gusts or errors in some sensor data, the prediction results of this technology have large deviations. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present application provides a method and system for short-term wind power prediction based on deep reinforcement learning, which solves the problem of large prediction deviation when predicting wind power based on the DDPG algorithm.

[0006] To achieve the above objectives, this application is implemented through the following technical solutions:

[0007] In the first aspect, an embodiment of the present application provides a short-term wind power prediction method based on deep reinforcement learning, which includes: obtaining an initial data set representing historical wind power and related environmental data; using a Hampel identifier to detect the initial data set, cleaning the initial data set and obtaining a target data set; introducing a TD3 algorithm, and determining a neural network framework of the introduced TD3 algorithm, the neural network framework including an Actor network group and a Critic network group; performing model training based on the target data set and the neural network framework, obtaining a target model to predict wind power, and obtaining prediction information representing wind power at future moments.

[0008] According to the first aspect of the embodiment of the present application, the aforementioned model training is performed based on the target data set and the neural network framework to obtain the target model to predict wind power and obtain prediction information characterizing wind power at future moments, including: determining the data characterizing temperature, wind speed, wind direction and humidity in the target data set as an environmental state vector, wherein the environmental state vector is the basis for predicting wind power; defining the process of obtaining wind power data based on weather data as an action set; taking the absolute value of the deviation between the actual wind power data and the wind power data obtained based on the weather data as a reward set; and using the TD3 algorithm to perform model training to obtain the target model to predict wind power and obtain prediction information.

[0009] According to a first aspect of an embodiment of the present application, an Actor network group includes an Actor network and an Actor target network, and a Critic network group includes two Critic networks and two Critic target networks; wherein the Actor network is used to provide a strategy for the current state, and the Actor target network is used to provide a strategy for the next state; the Critic network is used to output the Q function value of the state-action at the current moment, and the Critic target network is used to approximately estimate the Q function value of the state-action at the next moment, and the Q function value is used to characterize the expectation of future accumulated rewards.

[0010] According to a first aspect of an embodiment of the present application, two critic networks and two critic target networks constitute a first critic group and a second critic group; the first critic group includes a critic network and a critic target network, and the second critic group includes another critic network and another critic target network. During the model training process, the parameters corresponding to the first critic group and the second critic group are updated independently.

[0011] According to the first aspect of the embodiment of the present application, the aforementioned model training is performed using the TD3 algorithm to obtain a target model to predict wind power and obtain prediction information, including: according to a preset target strategy, selecting and outputting a target action from a set of actions through an Actor network group; evaluating the potential value of the target action through a first Critic group and a second Critic group respectively, and obtaining two Q function values ​​that characterize the potential value; determining the minimum of the two Q function values ​​as the target Q value to solve the problem of over-estimation of the Q value; evaluating the target action based on the target Q value, and obtaining a target model through interaction and repeated training of the Actor network group and the Critic network group; predicting wind power through the target model to obtain prediction information that characterizes wind power at future moments.

[0012] According to a first aspect of an embodiment of the present application, the target strategy is provided with a delayed update mechanism, which is used to delay the update of the Actor network group; the delayed update mechanism includes: when the number of target Q values ​​reaches a preset target, updating the Actor network group to provide sufficient time to learn accurate Q value estimation.

[0013] According to the first aspect of the embodiment of the present application, the aforementioned Hampel identifier is used to detect the initial data set, the initial data set is cleaned and the target data set is obtained, including: calculating the first median of the data in the initial data set through the Hampel identifier; calculating the distance from each data point in the initial data set to the first median to obtain multiple distances; determining the second median of the multiple distances as the median absolute deviation MAE, and correcting the median absolute deviation MAE through preset adjustment parameters to obtain a target threshold value σ; determining that the data points corresponding to the target distances greater than 3σ in the multiple distances are outliers, and eliminating them from the initial data set to obtain the target data set.

[0014] In the second aspect, the embodiment of the present application provides a short-term wind power prediction system based on deep reinforcement learning, which includes a data acquisition module, a detection and cleaning module, an algorithm introduction module and a training and prediction module. Specifically, the data acquisition module is used to obtain an initial data set representing historical wind power and related environmental data; the detection and cleaning module is used to use a Hampel identifier to detect the initial data set, clean the initial data set and obtain a target data set; the algorithm introduction module is used to introduce the TD3 algorithm and determine the neural network framework of the introduced TD3 algorithm, and the neural network framework includes an Actor network group and a Critic network group; the training and prediction module is used to perform model training based on the target data set and the neural network framework, obtain a target model to predict wind power, and obtain prediction information representing wind power at future moments.

[0015] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the method for short-term wind power prediction based on deep reinforcement learning in the first aspect described above is implemented.

[0016] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a program or instruction. When the program or instruction is executed by a processor, the short-term wind power prediction method based on deep reinforcement learning in the aforementioned first aspect is implemented.

[0017] This application provides a method and system for short-term wind power prediction based on deep reinforcement learning. Compared with the prior art, it has the following beneficial effects:

[0018] After obtaining the initial data set representing historical wind power and related environmental data, the present application uses the Hampel identifier for data cleaning, which can eliminate abnormal values ​​to obtain the target data set, and can avoid the problem of excessive data deviation caused by extreme situations, such as short-term strong gusts or partial sensor data errors, which is beneficial to improving prediction accuracy; further, the present application introduces the TD3 algorithm, constructs the Actor network group and the Critic network group, and by training the neural network framework based on the TD3 algorithm, it can solve the problem of over-estimation of the Q value in the traditional DDPG algorithm, further increase the accuracy of wind power prediction, and can effectively solve the problem of large prediction deviation in wind power prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a flow chart of a method for short-term wind power prediction based on deep reinforcement learning provided in an embodiment of the present application;

[0021] Figure 2 This is an exemplary flowchart of a model training using the TD3 algorithm provided in an embodiment of the present application;

[0022] Figure 3 It is a structural schematic diagram of a wind power short-term prediction system based on deep reinforcement learning provided in an embodiment of the present application;

[0023] Figure 4It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0026] The embodiments of the present application provide a method and system for short-term wind power prediction based on deep reinforcement learning, thereby solving the problem of large prediction deviation when predicting wind power based on the DDPG algorithm.

[0027] The technical solution in the embodiment of the present application is to solve the above technical problems, and the overall idea is as follows:

[0028] As environmental issues receive more and more attention, the use of clean energy is also gradually increasing. Wind power, as a renewable clean energy, accounts for a large proportion of all power generation methods. However, the instability of wind power has greatly hindered its development. Therefore, it is particularly important to accurately predict wind power. Accurate wind power forecasting helps adjust wind turbine operation strategies, reduce energy waste and economic losses, and support electricity market participation and renewable energy policy formulation, promoting the sustainable development of clean energy.

[0029] In the related art, machine learning algorithms can be used to predict the power output of wind farms, indicating that these methods can provide good prediction accuracy under standard meteorological conditions. However, statistical methods have limited prediction capabilities when faced with complex terrain and sudden weather changes. Furthermore, high-resolution numerical weather forecast models can be used to predict the power output of wind farms. The physical model can more accurately consider the impact of terrain and meteorological conditions on wind speed, thereby improving the accuracy of the prediction; however, the physical modeling method requires a large amount of computing resources and high-precision geographic data, and has strict requirements on the accuracy of model parameters and initial conditions, which limits its widespread application in practical applications.

[0030] Deep reinforcement learning methods have shown great advantages in various prediction tasks and have achieved good performance in complex environments; the Q-learning method can be used to predict wind power; and the DDPG algorithm can also be used to achieve short-term prediction of wind power. The DDPG (Deep Deterministic Policy Gradient) algorithm is a reinforcement learning algorithm that combines the policy gradient method and deep learning technology, and is designed to solve problems with continuous action space. DDPG is derived from the Actor-Critic framework. In particular, it is a combination of Q learning (especially deep Q network DQN) and deterministic policy gradient (DPG). However, in the process of using DDPG to predict short-term wind power, the DDPG algorithm has the problem of overestimating the Q value, which will lead to inaccurate prediction results; and this technology has a large deviation in prediction results when facing extreme situations, such as short-term strong gusts or errors in some sensor data.

[0031] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0032] The following first introduces a short-term wind power prediction method based on deep reinforcement learning provided in an embodiment of the present application.

[0033] The present application embodiment provides a flow chart of a method for short-term wind power prediction based on deep reinforcement learning, such as Figure 1 As shown, the wind power short-term prediction method based on deep reinforcement learning may include the following steps S110-S140.

[0034] S110, obtaining an initial data set representing historical wind power and related environmental data;

[0035] S120, using a Hampel identifier to detect an initial data set, cleaning the initial data set and obtaining a target data set;

[0036] S130, introducing the TD3 algorithm, and determining a neural network framework of the introduced TD3 algorithm, the neural network framework including an Actor network group and a Critic network group;

[0037] S140, performing model training based on the target data set and the neural network framework, obtaining a target model to predict wind power, and obtaining prediction information characterizing wind power at future moments.

[0038] The above is a specific implementation method of a short-term wind power prediction method based on deep reinforcement learning provided in an embodiment of the present application. It can be understood that the TD3 algorithm is a deterministic deep reinforcement learning algorithm based on the Actor-critic framework; after obtaining the initial data set representing historical wind power and related environmental data, the present application uses the Hampel identifier for data cleaning, which can eliminate abnormal values ​​to obtain the target data set, and can avoid the problem of excessive data deviation caused by extreme situations such as short-term strong gusts or partial sensor data errors, which is conducive to improving prediction accuracy.

[0039] Furthermore, this application introduces the TD3 algorithm, constructs the Actor network group and the Critic network group, and by training the neural network framework based on the TD3 algorithm, it can solve the over-estimation problem of the Q value in the traditional DDPG algorithm, further increase the accuracy of wind power prediction, and effectively solve the problem of large prediction deviation in wind power prediction.

[0040] In some embodiments, the aforementioned use of a Hampel identifier to detect an initial data set, cleans the initial data set and obtains a target data set, that is, the aforementioned S120 may specifically include the following steps:

[0041] S210, calculating the first median of the data in the initial data set by using a Hampel identifier;

[0042] S220, calculating the distance from each data point in the initial data set to the first median to obtain multiple distances;

[0043] S230, determining a second median among the multiple distances as a median absolute deviation MAE, and correcting the median absolute deviation MAE by a preset adjustment parameter to obtain a target threshold σ;

[0044] S240 , determining data points corresponding to target distances greater than 3σ among the multiple distances as outliers, and removing them from the initial data set to obtain a target data set.

[0045] In the embodiment of the present application, it is understandable that before performing wind power prediction based on the initial data set, it is necessary to detect the outliers in the initial data set and remove the outliers in the initial data set to improve the prediction accuracy. The present application uses the Hampel identifier for data processing. The Hampel identifier is a tool for data outlier detection. The Hampel identifier is based on statistical methods to identify those data points in the data set that are significantly different from other data points, that is, outliers.

[0046] It should be noted that the Hampel identifier has high robustness and flexibility, and has good tolerance to noise and non-normal distribution in the data. For example, in the presence of a small number of extreme values, it is not as easily disturbed as the mean and standard deviation method, because the mean can be easily pulled up or down by extreme values, while the median is relatively stable. In addition, the Hampel identifier can adjust the multiples of the target threshold σ according to the characteristics of the data and actual needs.

[0047] In some embodiments, the aforementioned model training is performed based on the target data set and the neural network framework to obtain a target model to predict wind power, and obtain prediction information characterizing wind power at a future moment, that is, the aforementioned S140 may specifically include the following steps:

[0048] S310, determining data representing temperature, wind speed, wind direction and humidity in the target data set as an environmental state vector, wherein the environmental state vector is a basis for wind power prediction;

[0049] S320, defining a process of obtaining wind power data according to weather data as an action set;

[0050] S330, taking the absolute value of the deviation between the actual wind power data and the wind power data obtained according to the weather data as a feedback set;

[0051] S340, using the TD3 algorithm to perform model training, obtain a target model to predict wind power, and obtain prediction information.

[0052] In the embodiments of the present application, it can be understood that the full name of the TD3 algorithm in English is Twin Delayed Deep Deterministic Policy Gradient algorithm, and its full name in Chinese is Twin Delayed Deep Deterministic Policy Gradient Algorithm. The workflow of the TD3 algorithm mainly includes: sending the current state and action into the network, and the network predicts the expected reward of the next state; calculating the gradient between the two evaluators; updating the parameters between the two networks; repeating the above steps until the network reaches a stable state.

[0053] In some embodiments, please refer to Figure 2The Actor network group includes an Actor network and an Actor target network, and the Critic network group includes two Critic networks and two Critic target networks; the Actor network is used to provide a strategy for the current state, and the Actor target network is used to provide a strategy for the next state; the Critic network is used to output the Q function value of the state-action at the current moment, and the Critic target network is used to approximate the Q function value of the state-action at the next moment, and the Q function value is used to characterize the expectation of future accumulated rewards. Figure 2 The Target Critic network is the aforementioned Critic target network.

[0054] In the embodiment of the present application, the TD3 algorithm is configured with two Critic networks and one Actor network, and a target network corresponding to the two Critic networks and one Actor network. The two Critic networks can form a contrast, and then select the minimum Q function value to avoid overestimation, thereby reducing the estimated Q value variance and making the result more stable.

[0055] It is understandable that the Critic network is mainly used to evaluate the quality of the actions taken by the Actor network. The Critic network learns the state-value function, which estimates the expected value of the future cumulative rewards that may be obtained by following the current strategy in a specific state; when the Actor network takes an action, the environment will feedback a new state and immediate reward; the Critic network will evaluate the long-term value of this action based on this new state and previous experience.

[0056] It should be noted that the Critic network of the TD3 algorithm can calculate the time difference error, that is, the absolute value of the deviation between the aforementioned real wind power data and the wind power data obtained based on weather data; in other words, the Critic network can estimate the difference between future rewards and actual rewards. This difference can be used to guide the update of the Actor network and help the Actor network adjust its strategy so that subsequent selected actions are more likely to obtain higher rewards.

[0057] It should also be noted that in the TD3 algorithm, the Actor network and the Critic network work closely together. The Actor network outputs actions based on the strategy, and the Critic network evaluates the actions. Through the interaction and repeated training of the two, the entire system can learn the optimal strategy faster and more stably, thereby making more reasonable decisions in complex environments.

[0058] In one example, two critic networks and two critic target networks constitute a first critic group and a second critic group; the first critic group includes a critic network and a critic target network, and the second critic group includes another critic network and another critic target network. During the model training process, the parameters corresponding to the first critic group and the second critic group are updated independently.

[0059] In some embodiments, the above-mentioned TD3 algorithm is used to perform model training to obtain a target model to predict wind power and obtain prediction information, that is, the above-mentioned S340 may specifically include the following steps:

[0060] S410, according to the preset target strategy, select and output the target action from the action set through the Actor network group;

[0061] S420, respectively evaluating the potential value of the target action by using the first critic group and the second critic group to obtain two Q function values ​​representing the potential value;

[0062] S430, determining the minimum of the two Q function values ​​as the target Q value to solve the problem of over-estimation of the Q value;

[0063] S440, evaluating the target action based on the target Q value, and obtaining the target model through interaction and repeated training of the Actor network group and the Critic network group;

[0064] S450: forecast wind power through the target model to obtain forecast information representing wind power at a future time.

[0065] In the embodiment of the present application, it can be understood that in the wind power prediction task, the first critic group and the second critic group can evaluate the potential value after a certain action from different angles, and estimate the value more conservatively by taking the minimum value to avoid overly optimistic estimates.

[0066] In one example, the target strategy is set with a delayed update mechanism, which is used to delay the update of the Actor network group; the delayed update mechanism includes: when the number of target Q values ​​reaches a preset target, updating the Actor network group to provide sufficient time to learn accurate Q value estimation.

[0067] In another example, noise is added to the target policy to achieve target policy smoothing; in this way, a wider exploration range can be used to increase the robustness of the target policy. Adding noise can try more different action modes and avoid over-reliance on a certain action mode, thereby better adapting to complex and changing environments.

[0068] In some embodiments, the present application provides a wind power short-term prediction system 500 based on deep reinforcement learning, such as Figure 3 As shown, the wind power short-term prediction system 500 may include the following modules:

[0069] A data acquisition module 510 is used to acquire an initial data set representing historical wind power and related environmental data;

[0070] A detection and cleaning module 520 is used to detect the initial data set using a Hampel identifier, clean the initial data set and obtain a target data set;

[0071] The algorithm introduction module 530 is used to introduce the TD3 algorithm and determine the neural network framework of the introduced TD3 algorithm, where the neural network framework includes an Actor network group and a Critic network group;

[0072] The training prediction module 540 is used to perform model training based on the target data set and the neural network framework, obtain a target model to predict wind power, and obtain prediction information representing wind power at future moments.

[0073] According to an embodiment of the present application, any multiple modules among the data acquisition module 510, the detection and cleaning module 520, the algorithm introduction module 530 and the training and prediction module 540 can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module.

[0074] In some embodiments, the detection and cleaning module 520 can be specifically used for:

[0075] Calculate the first median of the data in the initial data set through the Hampel identifier;

[0076] Calculate the distance from each data point in the initial data set to the first median to obtain multiple distances;

[0077] Determine the second median of the multiple distances as the median absolute deviation MAE, and correct the median absolute deviation MAE through a preset adjustment parameter to obtain a target threshold σ;

[0078] The data points corresponding to the target distances greater than 3σ among multiple distances are determined as outliers and are removed from the initial data set to obtain the target data set.

[0079] In some embodiments, the training prediction module 540 may specifically include:

[0080] A vector determination unit 541 is used to determine the data representing temperature, wind speed, wind direction and humidity in the target data set as an environmental state vector, wherein the environmental state vector is the basis for wind power prediction;

[0081] An action definition unit 542, used to define a process of obtaining wind power data according to weather data as an action set;

[0082] A report definition unit 543, used to take the absolute value of the deviation between the actual wind power data and the wind power data obtained according to the weather data as a report set;

[0083] The model training unit 544 is used to perform model training using the TD3 algorithm to obtain a target model to predict wind power and obtain prediction information.

[0084] In some embodiments, the model training unit 544 may be specifically used to:

[0085] According to the preset target strategy, the target action is selected and output from the action set through the Actor network group;

[0086] The first critic group and the second critic group evaluate the potential value of the target action respectively, and obtain two Q function values ​​representing the potential value;

[0087] Determine the minimum of the two Q function values ​​as the target Q value to solve the problem of overestimation of the Q value;

[0088] The target action is evaluated based on the target Q value, and the target model is obtained through the interaction and repeated training of the Actor network group and the Critic network group;

[0089] The wind power is predicted through the target model to obtain the prediction information representing the wind power at the future moment.

[0090] Figure 3 Each module in the system shown has the function of implementing each step in the aforementioned wind power short-term prediction method based on deep reinforcement learning, and can achieve its corresponding technical effect. For the sake of concise description, it will not be repeated here.

[0091] In some embodiments, the present application provides an electronic device, the structure diagram of the electronic device is as follows Figure 4 shown.

[0092] The electronic device may include a processor 610 and a memory 620 storing computer program instructions.

[0093] Specifically, the processor 610 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0094] The memory 620 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 620 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 620 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 620 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 620 is a non-volatile solid-state memory.

[0095] The memory 620 may include a read-only memory (ROM), a random access memory (RAM), a disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Therefore, generally, the memory 620 includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer executable instructions, and when the software is executed (e.g., by one or more processors), it can perform the operations described in any one of the methods for short-term wind power prediction based on deep reinforcement learning in the above-mentioned embodiments.

[0096] The processor 610 implements any one of the wind power short-term prediction methods based on deep reinforcement learning in the above embodiments by reading and executing computer program instructions stored in the memory 620 .

[0097] In one example, the electronic device may further include a communication interface 630 and a bus 600. Figure 4 As shown, the processor 610, the memory 620, and the communication interface 630 are connected via a bus 600 and communicate with each other.

[0098] The communication interface 630 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0099] Bus 600 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 600 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0100] In addition, in combination with the wind power short-term prediction method based on deep reinforcement learning in the above embodiments, the present application embodiment can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any one of the wind power short-term prediction methods based on deep reinforcement learning in the above embodiments is implemented.

[0101] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0102] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0103] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0104] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0105] In summary, compared with the prior art, this application has the following beneficial effects:

[0106] 1. After obtaining the initial data set representing historical wind power and related environmental data, this application uses the Hampel identifier for data cleaning, which can eliminate abnormal values ​​to obtain the target data set, and can avoid the problem of excessive data deviation caused by extreme situations such as short-term strong gusts or partial sensor data errors, which is conducive to improving the accuracy of wind power prediction.

[0107] 2. This application introduces the TD3 algorithm and constructs the Actor network group and the Critic network group. By training the neural network framework based on the TD3 algorithm, in the wind power prediction task, the first Critic group and the second Critic group can evaluate the potential value after a certain action from different angles, and estimate the value more conservatively by taking the minimum value, thereby solving the over-estimation problem of the traditional DDPG algorithm on the Q value and increasing the accuracy of wind power prediction.

[0108] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A short-term wind power forecasting method based on deep reinforcement learning, characterized in that: include: Acquire an initial dataset characterizing historical wind power and related environmental data; Using a Hampel identifier to detect the initial data set, cleaning the initial data set and obtaining a target data set; Introducing the TD3 algorithm, and determining a neural network framework of the introduced TD3 algorithm, wherein the neural network framework includes an Actor network group and a Critic network group; Model training is performed based on the target data set and the neural network framework to obtain a target model to predict wind power and obtain prediction information characterizing wind power at future moments.

2. The method for short-term wind power prediction based on deep reinforcement learning according to claim 1, characterized in that: The model training is performed based on the target data set and the neural network framework to obtain a target model to predict wind power, and obtain prediction information representing wind power at a future moment, including: Determine that the data representing temperature, wind speed, wind direction and humidity in the target data set is an environmental state vector, wherein the environmental state vector is a basis for predicting wind power; The process of obtaining wind power data based on weather data is defined as an action set; The absolute value of the deviation between the actual wind power data and the wind power data obtained according to the weather data is taken as the reward set; The TD3 algorithm is used to perform model training to obtain a target model to predict wind power and obtain prediction information.

3. The method for short-term wind power prediction based on deep reinforcement learning according to claim 2, characterized in that: The Actor network group includes an Actor network and an Actor target network, and the Critic network group includes two Critic networks and two Critic target networks; Among them, the Actor network is used to provide a strategy for the current state, and the Actor target network is used to provide a strategy for the next state; the Critic network is used to output the Q function value of the state-action at the current moment, and the Critic target network is used to approximate the Q function value of the state-action at the next moment, and the Q function value is used to characterize the expectation of future accumulated rewards.

4. The method for short-term wind power prediction based on deep reinforcement learning according to claim 3, characterized in that: The two critic networks and the two critic target networks constitute a first critic group and a second critic group; The first critic group includes one critic network and one critic target network, and the second critic group includes another critic network and another critic target network. During the model training process, the parameters corresponding to the first critic group and the second critic group are updated independently.

5. The method for short-term wind power prediction based on deep reinforcement learning according to claim 4, characterized in that: The method of using the TD3 algorithm to perform model training and obtain a target model to predict wind power and obtain prediction information includes: According to a preset target strategy, a target action is selected and output from the action set through the Actor network group; The first critic group and the second critic group respectively evaluate the potential value of the target action to obtain two Q function values ​​representing the potential value; Determine the minimum of the two Q function values ​​as the target Q value to solve the problem of overestimation of the Q value; The target action is evaluated based on the target Q value, and a target model is obtained through interaction and repeated training of the Actor network group and the Critic network group; The wind power is predicted by the target model to obtain prediction information representing the wind power at a future moment.

6. The method for short-term wind power prediction based on deep reinforcement learning according to claim 5, characterized in that: The target strategy is provided with a delayed update mechanism, and the delayed update mechanism is used to delay the update of the Actor network group; The delayed update mechanism includes: when the number of target Q values ​​reaches a preset target, updating the Actor network group to provide sufficient time to learn accurate Q value estimation.

7. The method for short-term wind power prediction based on deep reinforcement learning according to claim 1, characterized in that: The method of using a Hampel identifier to detect the initial data set, cleaning the initial data set and obtaining a target data set includes: Calculate the first median of the data in the initial data set by using a Hampel identifier; Calculating the distance from each data point in the initial data set to the first median to obtain multiple distances; Determine a second median of the plurality of distances as a median absolute deviation MAE, and correct the median absolute deviation MAE by a preset adjustment parameter to obtain a target threshold σ; Data points corresponding to target distances greater than 3σ among the multiple distances are determined to be outliers, and are removed from the initial data set to obtain a target data set.

8. A wind power short-term prediction system based on deep reinforcement learning, characterized in that: include: A data acquisition module for acquiring an initial data set representing historical wind power and related environmental data; A detection and cleaning module, used to detect the initial data set using a Hampel identifier, clean the initial data set and obtain a target data set; An algorithm introduction module, used for introducing the TD3 algorithm and determining a neural network framework of the introduced TD3 algorithm, wherein the neural network framework includes an Actor network group and a Critic network group; The training prediction module is used to perform model training based on the target data set and the neural network framework to obtain a target model to predict wind power and obtain prediction information characterizing wind power at future moments.

9. An electronic device, characterized in that: include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein when the program is executed by the processor, the method for short-term wind power prediction based on deep reinforcement learning as described in any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the method for short-term wind power prediction based on deep reinforcement learning as described in any one of claims 1 to 7 is implemented.

Images