Model optimization method and system for new energy station data processing

Abstract

The invention discloses a model optimization method and system for new energy station data processing, and the method comprises the steps: building a deep Q network reinforcement learning model according to an existing data sample, selecting an action based on a current state to calculate a reward function, and designing the reward function to measure the performance of an algorithm, awards are determined according to factors such as the accuracy of data recovery and the degree of help to a station operation decision, and in the training process, a higher extracted probability is given to important experience by calculating a TD error of each experience, playing back by using preferential experience and sampling according to the importance of the experience, so that a model learns key information more quickly, and the training efficiency is improved. Meanwhile, in the training process, a small batch of experiences are randomly extracted from a buffer area for learning, the correlation of data can be broken, finally, optimization training of the model is completed through iterative updating, and the model obtained through the method can solve the noise problem of data missing and the like in the new energy station data collection process.

Description

Technical Field

[0001] The present invention belongs to the technical field of new energy station data processing, and relates to a model optimization method and system for new energy station data processing. Background Art

[0002] With the development of the global economy, energy demand has increased year by year. Due to the depletion of non-renewable energy and environmental pollution, new energy has been highly valued and widely used by various countries for its safety, renewability and environmental protection, and has shown good development prospects. Among them, new energy power generation is mainly photovoltaic power generation and wind power generation. With the increasing proportion of wind power and photovoltaic power in the power grid, ultra-high voltage direct current transmission has been gradually put into operation, the operation and structure of the power grid have become more complex, and the difficulty of system power balance and frequency regulation has continued to increase. In order to increase the proportion of new energy consumption, the power structure has undergone major changes, and the fast frequency response resources available to the power grid have gradually decreased. For large power grids at the power transmission end, the structural dilemma of the power grid frequency control characteristics has become increasingly obvious, and the risk of power grid frequency safety problems has further increased. Therefore, it is very important to study the monitoring data of the grid-connected operation of new energy power generation.

[0003] However, due to the abandonment of wind and solar power, maintenance, extreme weather conditions, external electromagnetic interference or equipment failure during the operation of the power plant, there are a lot of incomplete and abnormal data in the original data. These bad data seriously affect the parameter estimation of the prediction model, resulting in low prediction accuracy and large prediction deviation. Therefore, it is necessary to carry out effective data processing.

[0004] In the paper "Research Review and Prospects of Data Cleaning in New Energy Power Forecasting", the author elaborated, analyzed and summarized the categories and processing methods of new energy abnormal data from two aspects: outlier removal and missing value reconstruction. However, although the traditional data interpolation method is simple and fast, if the data fluctuates greatly and the interpolation is inaccurate, some characteristic information of the data will be lost, and deviations may be introduced, and the time series information between adjacent data is not considered. Summary of the invention

[0005] The purpose of the present invention is to solve the problem in the prior art that when processing abnormal data of new energy, due to large data fluctuations and inaccurate interpolation, some characteristic information of the data will be lost, and deviations may be introduced, affecting the accuracy of missing data prediction, and to provide a model optimization method and system for new energy station data processing.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A model optimization method for new energy station data processing includes the following steps:

[0008] S1: Clarify the missing data of new energy stations;

[0009] S2: Define the state space, action space and reward function based on the missing data;

[0010] S3: Introduce the Q network and the target network, and set up the experience replay buffer;

[0011] S4: Select an action in the action space based on the defined state space, execute the action, obtain the reward function corresponding to the current state, and store the current state, the selected action, the reward function and the next state as an experience in the experience playback buffer;

[0012] S5: Calculate the TD error of each experience in the experience playback buffer based on the Q network and the target network, and select batches of experiences as training samples based on the TD error;

[0013] S6: Calculate the target Q value of each experience according to the selected training sample, calculate the loss function according to the target Q value, and update the target network parameters according to the loss function;

[0014] S7: Determine whether the updated target network still meets the conditions for stopping updating. If not, repeat steps S4-S6. If yes, the model optimization is completed.

[0015] A further improvement of the present invention is:

[0016] The step S2 comprises the following steps:

[0017] Assuming that the actual data is the missing frequency in the new energy station, the state space is expressed by the following formula:

[0018]

[0019] Among them, F represents the fundamental wave of voltage and current, H represents its harmonics, Respectively represent the amplitude, phase angle and frequency change rate of the interharmonic components;

[0020] The action space is expressed as follows:

[0021] a=(a1,a2,a3)

[0022] Among them, a1, a2, and a3 represent the filling values ​​of the regression model, time series model, and multiple imputation method respectively;

[0023] The reward function is expressed as follows:

[0024] The reward function R is:

[0025] R = 1 / (k + MSE)

[0026] Among them, k is a constant that is not 0, and MSE represents the error between the restored data and the true data.

[0027] In step S4, an action is randomly selected in the action space based on a greedy strategy.

[0028] In step S5, the TD error is calculated by the following formula:

[0029] δ=r+γmax a′ Q(s′,a′;θ target )-Q(s,a;θ main )

[0030] Where r is the current reward; γ is the discount factor; Q(s′,a′; θ target ) is the Q value predicted by the main network.

[0031] In step S6, calculating the target Q value of each experience includes:

[0032] y=r+γmax a′ Q(s′,a′;θ target )

[0033] Where r is the current reward and γ is the discount factor.

[0034] In step S6, calculating the loss function according to the target Q value includes:

[0035] L(θ main )=(yQ(s,a;θ main )) 2

[0036] Where y represents the target Q value for each experience.

[0037] In step S6, updating the target network parameters according to the loss function includes:

[0038] Use soft strategy to update the parameters of the target network:

[0039] θ target ←τθ main +(1-τ)θ target

[0040] Among them, θ target are the parameters of the target network, θ main is a parameter of the main network, and τ is an update ratio parameter less than 1.

[0041] A model optimization system for new energy station data processing, comprising:

[0042] Data acquisition module, used to clarify the missing data of new energy stations;

[0043] Model definition module 1, used to define the state space, action space and reward function based on data missing conditions;

[0044] Model definition module 2 is used to introduce the Q network and target network and set the experience playback buffer;

[0045] The experience acquisition module is used to select an action in the action space based on the defined state space, execute the action, obtain the reward function corresponding to the current state, and store the current state, the selected action, the reward function and the next state as an experience in the experience playback buffer;

[0046] The sample acquisition module is used to calculate the TD error of each experience in the experience playback buffer based on the Q network and the target network, and select batches of experience as training samples based on the TD error;

[0047] A network update module is used to calculate the target Q value of each experience according to the selected training samples, calculate the loss function according to the target Q value, and update the target network parameters according to the loss function;

[0048] The model optimization acquisition module is used to determine whether the updated target network still meets the conditions for stopping the update. If not, the experience acquisition module is repeatedly executed to the network update module. If so, the model optimization is completed.

[0049] A terminal device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any method described in the present invention when executing the computer program.

[0050] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any method described in the present invention.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] The present invention discloses a model optimization method for processing data of new energy stations. The present invention establishes a deep Q network reinforcement learning model according to existing data samples, calculates a reward function by selecting actions based on the current state, designs a reward function to measure the performance of the algorithm, and determines the reward according to factors such as the accuracy of data recovery and the degree of help to station operation decision-making. During the training process, by calculating the TD error of each experience, using priority experience playback, sampling is performed according to the importance of the experience, and a higher probability of being extracted is given to important experiences, so that the model can learn key information faster. At the same time, during the training process, small batches of experience are randomly extracted from the buffer for learning, which can break the correlation of the data. Finally, the optimization training of the model is completed through iterative updating. The model obtained by the present invention can solve the noise problem such as data missing in the process of collecting new energy station data, and ensure that when processing new energy abnormal data, the accuracy of missing data prediction is higher, and the processed data can provide data support for subsequent new energy station related research. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0054] Figure 1 is a flow chart of the present invention;

[0055] Figure 2 It is a schematic diagram of the interaction process between the reinforcement learning model and the environment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0059] In the description of the embodiments of the present invention, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. indicate an orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed when in use, it is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0060] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0061] In the description of the embodiments of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0062] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0063] See also Figure 1 to Figure 2 The embodiment of the present invention discloses a model optimization method for new energy station data processing, comprising the following steps:

[0064] Step 1: Identify missing data at new energy stations;

[0065] Specifically, analyze the types of missing data in new energy stations, such as the amplitude, phase angle, frequency and frequency change rate of the fundamental, harmonic and interharmonic components of voltage and current, and determine which data are crucial to the operation and decision-making of the stations.

[0066] In this embodiment, it is assumed that the frequency of the new energy station data is missing.

[0067] Step 2: Define the state space, action space and reward function based on the missing data;

[0068] Define the environment for reinforcement learning:

[0069] Assuming that the frequency of new energy station data is missing, then:

[0070] Furthermore, the state space includes the current existing new energy station data (data that are not missing), time information, location identifiers of missing data, etc., expressed by vector s:

[0071]

[0072] Among them, F represents the fundamental wave of voltage and current, H represents its harmonics, They respectively represent the amplitude, phase angle and frequency change rate of the interharmonic components.

[0073] Furthermore, the action space can be a prediction model or a data interpolation method, such as regression model prediction, time series model (such as autoregressive integrated moving average model) prediction, multiple interpolation method interpolation, etc., which can be expressed by the following formula:

[0074] a=(a1,a2,a3)

[0075] Among them, a1, a2, and a3 represent the filling values ​​of the regression model, time series model, and multiple imputation method, respectively.

[0076] Furthermore, setting the reward function includes:

[0077] In this embodiment, a reward function is designed to measure the performance of the algorithm, and the reward is determined based on factors such as the accuracy of data recovery and the degree of help in station operation decision-making. The present invention determines the reward by calculating the error between the recovered data and the real data. The smaller the error, the higher the reward.

[0078] Among them, the error between the restored data and the real data is calculated by the mean square error method:

[0079]

[0080] Where n is the number of data points, y i is the real data value, is the data value after recovery;

[0081] The reward function R is designed as follows:

[0082] R = 1 / (k + MSE)

[0083] Here, k is a small constant that is not equal to 0 to avoid division by 0.

[0084] Step 3: Introduce the Q network and target network, and set the experience replay buffer;

[0085] Based on Gaussian distribution, the parameters of the current network and target network of DQN are randomly initialized, and hyperparameters such as learning rate lr and discount factor γ are set.

[0086] In this embodiment, the Q network is a DQN model. The DQN model is constructed using a convolutional neural network to extract features of data near missing points. The Batch Normalzation (BN) layer improves the model training efficiency. The pooling layer reduces network parameters. If necessary, the dropout operation can be used when inputting data to reduce overfitting. The input layer accepts the state representation s, and finally the fully connected layer is used as the output layer to estimate the Q value of each action.

[0087] Furthermore, the target network:

[0088] The main function of the target network in DQN is to provide a relatively stable target value for calculating the loss function and updating the main network. The structure of the target network is the same as the main network (i.e. the network currently used for decision-making). The target network has the same input layer, hidden layer, and output layer as the Q network. By regularly updating the parameters of the target network, the training process is made more stable, the fluctuations in training are reduced, and the convergence speed and performance of the algorithm are improved.

[0089] Further, experience replay buffer:

[0090] Set up the experience buffer to store the experience generated by the interaction between the model and the environment:

[0091] replay=(s t ,a t ,r t ,s t+1 )

[0092] Among them, s t ,s t ,r t They are the state of the model at time t, the action selected, the reward obtained, and a t+1 is the state of the model at time t+1.

[0093] During training, small batches of experience are randomly extracted from the buffer for learning to break the correlation of the data.

[0094] Step 4: Based on the defined state space, select an action in the action space and execute the action to obtain the reward function corresponding to the current state. The current state, the selected action, the reward function and the next state are stored as an experience in the experience playback buffer.

[0095] In this embodiment, the action is selected through a greedy strategy. The agent randomly selects an action with a probability of ε, so that the agent does not rely on the current Q value estimate, but tries different actions; the action with the largest Q value output by the current Q network is selected with a probability of 1-ε, that is, the action with the largest reward is selected.

[0096] The dynamic adjustment of ε includes: at the beginning of training, ε is usually set to a higher value, such as 0.5, which allows the agent to conduct a lot of exploration to collect different experiences; then it is reduced in an exponential decay manner: ε = ε start α t , where α is a decay coefficient less than 1, and t is the current training round number.

[0097] Greedy strategies can avoid local optimal solutions, that is, by exploring new actions, the agent has the opportunity to discover better strategies and avoid falling into local optimal solutions. If the agent only exploits, it may always choose the action that is currently considered optimal and miss other potential better strategies. Improve learning efficiency, that is, help the agent quickly collect experience and understand different situations in the environment in the early stage of training. As the number of training rounds increases, the agent gradually turns to utilizing the knowledge it has learned to improve the efficiency of obtaining rewards.

[0098] Step 5: Calculate the TD error of each experience in the experience playback buffer based on the Q network and the target network, and select batches of experience as training samples based on the TD error;

[0099] Furthermore, in this embodiment, priority experience playback is used to sample according to the importance of experience, giving important experience a higher probability of being extracted, so that the model can learn key information faster. In the problem of missing data at new energy stations, important experiences are those state transitions that cause large errors or have a significant impact on data recovery.

[0100] The priority is calculated using the TD error (Temporal Difference error). The TD error is the difference between the current estimated Q value and the target Q value. Each experience has its own TD error. Experience with a large TD error may be more helpful for model learning and should be selected first. At the same time, the selection of experience needs to be random to ensure the diversity of experience in the training sample.

[0101] The TD error is calculated by the following formula:

[0102] δ=r+γmax a′ Q(s′,a′;θ target )-Q(s,a;θ main )

[0103] Furthermore, the absolute value of the TD error is used as the empirical priority.

[0104] In this embodiment, the priority of experience can be set to p = |δ| + ε, where ε is a small positive number, to ensure that all experiences have a certain probability of being selected. The priority of the experience is normalized so that the sum of the priorities of all experiences is 1, which is then used as the probability of being selected. Taking N experiences as an example, the priority of the i-th experience is p i , then the probability of being selected is

[0105] A random sampling method with priorities is used to ensure that high-priority experiences are more likely to be selected.

[0106] Step 6: Calculate the target Q value of each experience based on the selected training samples, calculate the loss function based on the target Q value, and update the target network parameters based on the loss function;

[0107] Specifically, a small batch of experience is randomly extracted from the playback buffer, the target Q value is calculated and the parameters of the current network are updated so that the current Q network estimates the Q value of the action in the state close to the target Q value.

[0108] Specifically, for a state s and action a, the Q value predicted by the main network is: Q(s,a;θ main ), the target value predicted by the target network is:

[0109] y=r+γmax a′ Q(s′,a′;θ target )

[0110] Where r is the current reward and γ is the discount factor;

[0111] Furthermore, the loss function is calculated as:

[0112] L(θ main )=(yQ(s,a;θ main )) 2

[0113] Furthermore, the Adam optimization algorithm is used to minimize the loss function to update the parameters of the main network.

[0114] Step 7: Determine whether the updated target network still meets the conditions for stopping updating. If not, repeat steps 4 to 6. If yes, the model optimization is completed.

[0115] Update the target network using the soft update strategy:

[0116] θ target ←τθ main +(1-τ)θ target

[0117] Among them, θ target are the parameters of the target network, θ main is a parameter of the main network, and τ is an update ratio parameter less than 1. A smaller τ value will make the target network update slower and more stable; a larger τ value will make the target network update faster, but may cause instability. The appropriate soft update ratio parameter is determined through experiments.

[0118] This embodiment makes the training process more stable by regularly updating the parameters of the target network, reduces fluctuations during training, and improves the convergence speed and performance of the algorithm.

[0119] The network update conditions include:

[0120] The stop update condition of the DQN model can be set to a fixed number of steps or rounds, that is, when the model reaches a fixed number of steps or rounds, it stops updating, which can ensure that the training ends within a certain period of time and avoid indefinite training. Usually, the fixed number of steps or rounds can be verified through repeated experiments to achieve the optimal performance of the model.

[0121] The step interval for updating the target network in the DQN model needs to comprehensively consider factors such as model training stability and convergence speed. It can be determined through gradual trials. Usually, the step interval can be tried from updating the target network once every 5,000 steps, and then adjusted according to the training effect. The step interval can also be determined according to the number of training rounds, that is, the target network is updated once after each round of training.

[0122] Furthermore, the trained DQN model is evaluated using the test set data. Indicators such as the accuracy of data recovery and the degree of improvement in station decision-making tasks such as frequency prediction are calculated. The error between the recovered data and the real data is compared to evaluate the performance of the model under different missing modes. The present invention applies the trained DQN model to the actual data missing problem of new energy stations. When data is missing, the model selects the optimal data interpolation method for data recovery based on the current state. The performance of the model is continuously monitored and adjusted and optimized according to actual conditions.

[0123] The present invention achieves effective data amplification by performing feature learning on the collected monitoring data, which can solve noise problems such as data missing in the collection process to a certain extent. On the basis of filling the missing data, the front and back dependencies of the time series data will not be lost, and the noise problems such as data missing in the process of collecting new energy station data can be solved. The processed data can provide data support for subsequent new energy station-related research.

[0124] The present invention establishes a deep Q network reinforcement learning model (Deep Q Network, DQN) based on existing data samples. DQN can handle problems with time series characteristics. The model uses the data near the missing point as a feature, predicts the data of the missing point, and mines similar changes in massive historical data to perform more accurate data filling. Its filling data not only focuses on its own time series evolution, but also pays attention to adjacent time series, especially the influence between time series indicators with influential relationships. For example, for the missing wind speed data, DQN can learn the changing trend and pattern of historical wind speed data to predict the missing wind speed value.

[0125] This embodiment also discloses a model optimization system for new energy station data processing, including:

[0126] Data acquisition module, used to clarify the missing data of new energy stations;

[0127] Model definition module 1, used to define the state space, action space and reward function based on data missing conditions;

[0128] Model definition module 2 is used to introduce the Q network and target network and set the experience playback buffer;

[0129] The experience acquisition module is used to select an action in the action space based on the defined state space, execute the action, obtain the reward function corresponding to the current state, and store the current state, the selected action, the reward function and the next state as an experience in the experience playback buffer;

[0130] The sample acquisition module is used to calculate the TD error of each experience in the experience playback buffer based on the Q network and the target network, and select batches of experience as training samples based on the TD error;

[0131] A network update module is used to calculate the target Q value of each experience according to the selected training samples, calculate the loss function according to the target Q value, and update the target network parameters according to the loss function;

[0132] The model optimization acquisition module is used to determine whether the updated target network still meets the conditions for stopping the update. If not, the experience acquisition module is repeatedly executed to the network update module. If so, the model optimization is completed.

[0133] A schematic diagram of a terminal device provided in an embodiment of the present invention. The terminal device of this embodiment includes: a processor, a memory, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor executes the computer program, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0134] The computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to accomplish the present invention.

[0135] The terminal device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0136] The processor can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0137] The memory may be used to store the computer programs and / or modules, and the processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory.

[0138] If the module / unit integrated in the terminal device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0139] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A model optimization method for new energy station data processing, characterized in that: The following steps are involved: S1: Clarify the missing data of new energy stations; S2: Define the state space, action space and reward function based on the missing data; S3: Introduce the Q network and the target network, and set up the experience replay buffer; S4: Select an action in the action space based on the defined state space, execute the action, obtain the reward function corresponding to the current state, and store the current state, the selected action, the reward function and the next state as an experience in the experience playback buffer; S5: Calculate the TD error of each experience in the experience playback buffer based on the Q network and the target network, and select batches of experiences as training samples based on the TD error; S6: Calculate the target Q value of each experience according to the selected training sample, calculate the loss function according to the target Q value, and update the target network parameters according to the loss function; S7: Determine whether the updated target network still meets the conditions for stopping the update. If not, repeat steps S4-S6. If yes, the model optimization is completed.

2. A model optimization method for new energy station data processing according to claim 1, characterized in that: The step S2 comprises the following steps: Assuming that the actual data is the missing frequency in the new energy station, the state space is expressed by the following formula: Among them, F represents the fundamental wave of voltage and current, H represents its harmonics, A, ρ represents the amplitude, phase angle and frequency change rate of the interharmonic components respectively; The action space is expressed as follows: a=(a1,a2,a3) Among them, a1, a2, and a3 represent the filling values ​​of the regression model, time series model, and multiple imputation method respectively; The reward function is expressed as follows: The reward function R is: R = 1 / (k + MSE) Among them, k is a constant that is not 0, and MSE represents the error between the restored data and the true data.

3. A model optimization method for new energy station data processing according to claim 1, characterized in that: In step S4, an action is randomly selected in the action space based on a greedy strategy.

4. A model optimization method for new energy station data processing according to claim 1, characterized in that: In step S5, the TD error is calculated by the following formula: δ=r+γmax a′ Q(s′,a′;θ target )-Q(s,a;θ main ) Where r is the current reward; γ is the discount factor; Q(s′,a′; θ target ) is the Q value predicted by the main network.

5. A model optimization method for new energy station data processing according to claim 1, characterized in that: In step S6, calculating the target Q value of each experience includes: y=r+γmax a′ Q(s′,a′;θ target ) Where r is the current reward and γ is the discount factor.

6. A model optimization method for new energy station data processing according to claim 5, characterized in that: In step S6, calculating the loss function according to the target Q value includes: L(θ main )=(yQ(s,a;θ main )) 2 Where y represents the target Q value for each experience.

7. A model optimization method for new energy station data processing according to claim 1, characterized in that: In step S6, updating the target network parameters according to the loss function includes: Use soft strategy to update the parameters of the target network: i target ←tth main +(1-τ)θ target Among them, θ target are the parameters of the target network, θ main is a parameter of the main network, and τ is an update ratio parameter less than 1.

8. A model optimization system for new energy station data processing, characterized in that: include Data acquisition module, used to clarify the missing data of new energy stations; Model definition module 1, used to define the state space, action space and reward function based on data missing conditions; Model definition module 2 is used to introduce the Q network and target network and set the experience playback buffer; The experience acquisition module is used to select an action in the action space based on the defined state space, execute the action, obtain the reward function corresponding to the current state, and store the current state, the selected action, the reward function and the next state as an experience in the experience playback buffer; The sample acquisition module is used to calculate the TD error of each experience in the experience playback buffer based on the Q network and the target network, and select batches of experience as training samples based on the TD error; A network update module is used to calculate the target Q value of each experience according to the selected training samples, calculate the loss function according to the target Q value, and update the target network parameters according to the loss function; The model optimization acquisition module is used to determine whether the updated target network still meets the conditions for stopping the update. If not, the experience acquisition module is repeatedly executed to the network update module. If so, the model optimization is completed.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images