New energy data preprocessing strategy self-optimization method, device, equipment and medium

Abstract

The application discloses a new energy data preprocessing strategy self-optimization method, device, equipment and medium, wherein the method comprises the following steps: modeling a data cleaning process as a reinforcement learning task, initializing a preprocessing strategy model containing a state, an action and a reward function; generating a processing decision by the strategy model and performing cleaning based on the data batch state in the coarse cleaning data set, to obtain a fine cleaning data set; training a downstream power prediction model by using the fine cleaning data set and evaluating the performance; and generating a reward signal according to the evaluation result and iteratively updating the strategy model. Through the reinforcement learning closed loop, the data preprocessing autonomous optimization oriented to the prediction performance is realized, the artificial dependence is reduced, and the prediction precision and adaptability are improved.

Description

Technical Field

[0001] This invention relates to the field of new energy power generation technology, and in particular to a method, apparatus, equipment and medium for self-optimization of new energy data preprocessing strategy. Background Technology

[0002] Photovoltaic power generation forecasting is a crucial technical support for the operation and management of photovoltaic power plants and power system dispatching. In practical applications, raw photovoltaic operating data often contains noise, missing values, and outliers due to equipment failures, communication anomalies, and environmental interference, directly affecting the accuracy and reliability of the forecasting model. Therefore, before constructing a forecasting model, it is essential to preprocess the raw data to improve data quality and thus enhance forecast accuracy. Current photovoltaic data preprocessing methods mainly include those based on fixed rules and statistical analysis.

[0003] However, while these methods can improve data quality to some extent, they still have obvious drawbacks, such as poor adaptability, and the strategies are mostly statically set and rely on manual maintenance. Summary of the Invention

[0004] This invention provides a method, apparatus, equipment, and medium for self-optimizing new energy data preprocessing strategies, solving the problem that photovoltaic power generation data preprocessing in the prior art relies on human experience.

[0005] In a first aspect, embodiments of the present invention provide a self-optimization method for photovoltaic data preprocessing strategies. The method includes: acquiring a historical operating dataset of new energy power generation equipment and performing basic preprocessing on it to obtain a coarsely cleaned dataset; modeling the data cleaning process as a reinforcement learning task and initializing a data preprocessing strategy model; the reinforcement learning task includes a state, an action, and a reward function, wherein the state is a feature representation of the current batch of data to be processed extracted from the coarsely cleaned dataset, the action is used to represent the processing decision for data samples within the current batch of data to be processed, and the reward function is used to generate a reward signal; based on the state, generating a corresponding action through the current data preprocessing strategy model, processing the data samples according to the generated action to complete data cleaning and obtain a finely cleaned dataset; using the finely cleaned dataset to train a downstream new energy power generation prediction model, and evaluating the performance of the new energy power generation prediction model on a preset evaluation dataset to obtain a performance evaluation result; generating a reward signal based on the performance evaluation result and the reward function, and updating the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal.

[0006] Furthermore, the reward signal includes a real-time reward and an environmental reward, the reward function includes a first reward function and a second reward function, the basic preprocessing includes labeling samples in the coarsely washed dataset with preset processing labels, and the generation of the reward signal based on the performance evaluation results and the reward function includes: generating the real-time reward using the first reward function based on the degree of matching between the processing decision and the preset processing labels of the corresponding samples in the coarsely washed dataset; and generating the environmental reward using the second reward function based on the performance evaluation results of the new energy power generation prediction model on the preset evaluation dataset.

[0007] Furthermore, updating the current data preprocessing strategy model of the new energy power generation equipment based on the generated reward signal includes: calculating the value of the discounted return based on the real-time reward and the environmental reward; determining the predicted value of the state value function; determining the value of the dominance function based on the difference between the value of the discounted return and the predicted value of the state value function; and updating the strategy model parameters of the current data preprocessing strategy model of the new energy power generation equipment based on the value of the dominance function.

[0008] Furthermore, the state includes at least one of the original data values, statistical features, and contextual information of the current batch of data to be processed.

[0009] Furthermore, the action is to assign a continuous confidence score to the data sample, the continuous confidence score being used to represent the probability that the data sample is retained.

[0010] Furthermore, the processing decision for data samples within the current batch of data to be processed includes: comparing the continuous confidence score corresponding to the data sample with a preset threshold, retaining the data samples whose continuous confidence score is higher than the preset threshold, and performing elimination or weight weakening operations on the data samples whose continuous confidence score is lower than or equal to the preset threshold.

[0011] Furthermore, after updating the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal, the method further includes: returning to the state, generating corresponding action steps based on the current data preprocessing strategy model and continuing to execute until a preset termination condition is met, and outputting the data preprocessing strategy model when the termination condition is met.

[0012] Secondly, embodiments of the present invention provide a self-optimizing device for new energy data preprocessing strategies. The device includes: an acquisition unit, used to acquire historical operating datasets of new energy power generation equipment and perform basic preprocessing on them to obtain a coarsely cleaned dataset; a modeling unit, used to model the data cleaning process as a reinforcement learning task and initialize a data preprocessing strategy model; the reinforcement learning task includes a state, an action, and a reward function, wherein the state is a feature representation of the current batch of data to be processed extracted from the coarsely cleaned dataset, the action is used to represent the processing decision for data samples within the current batch of data to be processed, and the reward function is used to generate a reward signal; a generation unit, used to generate corresponding actions through the current data preprocessing strategy model, and process the data samples according to the generated actions to complete data cleaning and obtain a finely cleaned dataset; a training and evaluation unit, used to train a downstream new energy power generation prediction model using the finely cleaned dataset, and evaluate the performance of the new energy power generation prediction model on a preset evaluation dataset to obtain a performance evaluation result; and an update unit, used to generate a reward signal based on the performance evaluation result and the reward function, and update the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal.

[0013] Thirdly, embodiments of the present invention also provide a computer device, the computer device including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method of the first aspect described above.

[0014] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the method described in the first aspect.

[0015] Compared with existing technologies, the method of this application realizes the automatic optimization of the current photovoltaic data preprocessing strategy of new energy power generation equipment by constructing a reinforcement learning closed loop with prediction performance as the reward signal. This directly links the cleaning process with the performance improvement of the downstream new energy power generation prediction model, effectively reducing the dependence on human experience and significantly improving prediction accuracy and system adaptability. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A flowchart illustrating the steps of the method provided in this embodiment of the invention; Figure 2 A flowchart of the sub-steps of the method provided in the embodiments of the present invention; Figure 3 A flowchart illustrating the steps of the method provided in this embodiment of the invention; Figure 4 A schematic block diagram of the apparatus provided in the embodiments of the present invention; Figure 5 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0022] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0023] See Figure 1 This invention provides a self-optimization method for photovoltaic data preprocessing strategies. Figure 1The diagram shows the steps of the self-optimization method for photovoltaic data preprocessing strategy. This method is mainly applied to data processing in new energy power generation scenarios (such as photovoltaic power generation, wind power generation, and hydropower generation). The core of the method lies in constructing a closed-loop data preprocessing strategy learning framework with terminal prediction performance as the direct optimization objective. This improves the consistency between data cleaning and downstream prediction tasks, enabling the autonomous iterative evolution of the preprocessing strategy, thereby obtaining more accurate and stable new energy power generation prediction results. Figure 1 As shown, the method may include the following steps: S110-S150.

[0024] S110. Obtain the historical operation dataset of new energy power generation equipment and perform basic preprocessing to obtain a coarsely washed dataset.

[0025] In practice, the first step is to collect historical operation datasets of new energy power generation equipment (such as photovoltaic inverters, wind turbine generators, and hydropower units). ,in Input features (such as irradiance, temperature, wind speed, historical power, etc.) This corresponds to the actual values ​​(such as the actual power generation of the inverter). Then, basic data cleaning operations are performed on this original dataset, such as removing outliers that exceed physically reasonable ranges and performing linear interpolation on short-term missing data points. These basic preprocessing rules are relatively fixed and simple, and their main purpose is to remove obvious "noise" from the data. The resulting dataset is called the coarsely washed dataset. This coarsely washed dataset will serve as the initial reference benchmark for subsequent reinforcement learning training. At the same time, each sample in the coarsely washed dataset can be labeled with a pre-defined processing label (e.g., retain or remove) to provide a supervisory signal for subsequent real-time reward calculation.

[0026] S120. Model the data cleaning process as a reinforcement learning task and initialize the data preprocessing strategy model; the reinforcement learning task includes a state, an action, and a reward function, wherein the state is a feature representation of the current batch of data to be processed extracted from the coarsely cleaned dataset, the action is used to represent the processing decision for the data samples in the current batch of data to be processed, and the reward function is used to generate a reward signal.

[0027] In practice, the traditional data preprocessing process, which relies on empirical rules, is transformed into a sequential decision-making problem that can be learned through interaction between the agent and the environment. To this end, the data cleaning process is formalized as a reinforcement learning task, defining three core elements: state, action, and reward function. Simultaneously, a data preprocessing policy model (typically implemented using a policy neural network) is initialized, with initial parameters that can be randomly set or preset based on prior knowledge. Here, the state *St* is the feature representation of the current batch of data to be processed, extracted from the coarsely washed dataset at time *t*. This feature representation incorporates multi-dimensional information to ensure comprehensive decision-making. The action *at* represents the processing decision made by the agent for each data sample within the current batch, such as retaining a sample, removing a sample, or weakening the sample's weight. The reward function *R* is used to generate a quantified reward signal to evaluate the quality of the agent's actions, thereby guiding the optimization direction of the policy model.

[0028] In a preferred embodiment, the state includes at least one of the original data values, statistical features, and contextual information of the current batch of data to be processed.

[0029] Specifically, the state St can be constructed as a feature matrix, where the rows correspond to the individual data samples in the current batch, and the columns contain various types of information. In essence, the original data values ​​are the input features of each sample. The data includes the raw numerical values; statistical features such as the mean, variance, skewness, and kurtosis of the batch within a specific time window, or the missing rate and proportion of zero values ​​for each feature dimension; and contextual information such as the timestamp corresponding to the data point, equipment identifier, and operating mode of the new energy power plant. By integrating this multi-source information, the state can characterize the local distribution characteristics and global background environment of the data, providing rich and comprehensive decision-making basis for the data preprocessing strategy model.

[0030] In a preferred embodiment, the action is to assign a continuous confidence score to a data sample, the continuous confidence score representing the probability that the data sample is retained.

[0031] Specifically, the action is the model for each data sample. Assign a continuous confidence score This score directly and quantitatively represents the probability that the strategy model considers the sample reliable and should be retained for training downstream new energy power generation prediction models. Through this continuous confidence score design, the strategy model can perform fine-grained quality assessments of each sample, rather than simply making a binary retain / reject decision, thus providing a richer expressive space for subsequent data cleaning decisions.

[0032] Furthermore, the processing decision for data samples within the current batch of data to be processed includes: comparing the continuous confidence score corresponding to the data sample with a preset threshold, retaining the data samples whose continuous confidence score is higher than the preset threshold, and performing elimination or weight weakening operations on the data samples whose continuous confidence score is lower than or equal to the preset threshold.

[0033] In practice, a preset threshold θ (e.g., θ=0.5) is set for continuous confidence scores. For data samples with a value greater than θ, a retention operation is performed, and they are included in the refined dataset. For continuous confidence scores... For data samples ≤ θ, a removal operation (directly discarding) or a weight reduction operation is performed (e.g., assigning them a very small sample weight in subsequent downstream model training to minimize their impact on model training). This threshold-based decision-making mechanism transforms the continuous confidence scores output by the policy model into explicit data cleaning actions, effectively filtering low-quality samples while retaining high-quality samples for training downstream prediction models.

[0034] In a preferred embodiment, the reward signal includes a real-time reward and an environmental reward, the reward function includes a first reward function and a second reward function, the basic preprocessing includes labeling samples in the coarsely washed dataset with preset processing labels, and the generation of the reward signal based on the performance evaluation results and the reward function includes: Based on the degree of matching between the processing decision and the preset processing label of the corresponding sample in the coarse washing dataset, the real-time reward is generated through the first reward function; based on the performance evaluation results of the new energy power generation prediction model on the preset evaluation dataset, the environmental reward is generated through the second reward function.

[0035] Specifically, the reward signal includes real-time rewards. and environmental rewards The two key components, correspondingly, define the reward function as a first reward function and a second reward function. The real-time reward aims to provide immediate and explicit supervisory signals, and its calculation relies on the coarse washing of the dataset during the basic preprocessing stage in step S110 described above. Each sample in the dataset has a pre-defined processing label (e.g., 1 indicates it is retained in the basic preprocessing, 0 indicates it is discarded). Real-time rewards. The reward is calculated using the first reward function based on the degree of matching between the processing decision of the current batch of samples (i.e., the retention / removal result derived from the confidence score and threshold) and the aforementioned preset labels, specifically, the real-time reward is calculated. It can be calculated as: , Where B represents the total amount of data, and i represents the ID of the currently selected dataset. It is a logical judgment function, that is Output 1 if the condition is met, otherwise output 0, thus allowing the data cleaning model to prioritize mimicking human cleaning behavior in the short term. Real-time reward. Specifically, this can be manifested as follows: a small positive reward is given for a match, and zero or negative rewards are given for a mismatch.

[0036] Environmental awards This is the core reward driving the strategy towards optimizing the ultimate business goal. It is calculated through a second reward function based on the performance evaluation results of the downstream new energy power generation prediction model on a pre-set evaluation dataset. Environmental Reward By directly linking the quality evaluation of data preprocessing with the improvement of the accuracy of downstream prediction models, the problem of misaligned optimization objectives is fundamentally solved.

[0037] S130. Based on the state, generate corresponding actions through the current data preprocessing strategy model, and process the data samples according to the generated actions to complete data cleaning and obtain a finely cleaned dataset.

[0038] In practice, the state St of the current batch of data to be processed, extracted from the coarse-washed dataset, is input into the current data preprocessing strategy model (such as a policy neural network). The model outputs the corresponding action, that is, the processing decision for each data sample (e.g., retention or removal). Subsequently, specific retention, removal, or weight reduction operations are performed on the data samples according to the generated action. All sample data that are retained (or retained but given a smaller weight) after this decision-making process together constitute the fine-washed dataset, which has a higher expected quality and is more conducive to the learning of downstream new energy power generation prediction models.

[0039] S140. Use the refined dataset to train the downstream new energy power generation prediction model, and evaluate the performance of the new energy power generation prediction model on a preset evaluation dataset to obtain performance evaluation results.

[0040] In practice, the "refined dataset" generated in step S130 is used as training data to train or fine-tune the selected downstream renewable energy power generation prediction model. The downstream renewable energy power generation prediction model is a machine learning model that performs terminal prediction tasks. It receives preprocessed data as input and outputs business-related photovoltaic power prediction results (such as photovoltaic power generation). After the downstream renewable energy power generation prediction model is trained, its performance is evaluated on a pre-reserved, non-strategy optimization loop-exclusive evaluation dataset to obtain quantitative performance evaluation results (e.g., coefficient of determination R). 2 Mean square error (MSE), etc.

[0041] In a preferred embodiment, the downstream new energy power generation prediction model includes at least one of a neural network model, a gradient boosting tree model, and an ensemble learning model.

[0042] Specifically, the downstream new energy power generation prediction model can be a variety of machine learning models, including but not limited to neural network models (such as fully connected networks and recurrent neural networks), gradient boosting tree models (such as XGBoost and LightGBM) or ensemble learning models.

[0043] In a preferred embodiment, the performance evaluation results of the downstream new energy power generation prediction model are calculated based on at least one of the following indicators: coefficient of determination, mean square error, and mean absolute error.

[0044] Specifically, the performance metric used for evaluation can be the coefficient of determination R. 2 At least one of the following indicators: mean squared error (MSE), mean absolute error (MAE). In practical applications, the coefficient of determination R is usually used. 2 Alternatively, a negative mean squared error (-MSE) can be used as an indicator, so that a larger indicator value represents better performance, which is helpful for subsequent reward calculation.

[0045] S150. Generate a reward signal based on the performance evaluation results and the reward function, and update the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal.

[0046] In specific implementation, the performance evaluation results of the new energy power generation prediction model calculated in step S140 above are input into the reward function to calculate the reward signal, which may include real-time rewards. Environmental awards Two parts, including real-time rewards Environmental rewards are calculated based on the degree of matching between the processing decision and the preset processing labels in the coarse-washed dataset. The calculation is then based on the performance evaluation results. Following this reward signal, the parameters of the current data preprocessing strategy model for new energy power generation equipment are optimized and updated (e.g., using policy gradient reinforcement learning algorithms such as PPO and A2C), enabling the strategy model to generate better data cleaning decisions in the next iteration, thereby maximizing the performance of the downstream prediction model.

[0047] In a preferred embodiment, such as Figure 2 As shown, updating the current data preprocessing strategy model of the new energy power generation equipment based on the generated reward signal includes steps S151-S153.

[0048] S151. Calculate the value of the discount reward based on the real-time reward and the environmental reward; S152. Determine the predicted value of the state value function, and determine the value of the dominance function based on the difference between the value of the discounted return and the predicted value of the state value function. S153. Based on the value of the advantage function, update the strategy model parameters of the current data preprocessing strategy model of the new energy power generation equipment.

[0049] In practice, when updating the data preprocessing strategy model, it is first necessary to comprehensively consider real-time rewards. Environmental awards To calculate the value of the discount return The value of cumulative discount return This takes into account all future rewards following the current step. Specifically, it considers the value of the cumulative discount reward. The calculation is as follows: , in, β is a discount factor, usually set to 0.9, used to weigh the importance of short-term versus long-term rewards. T is the length of an episode. β is a scaling factor used to balance the magnitude of real-time rewards and environmental rewards. k is the number of iterations used to pass on the impact of real-time rewards from future steps on the current step.

[0050] Simultaneously, a separate value function network (i.e., state value function) is used to estimate the predicted value in the current state. This predicted value represents the expected cumulative return achievable in the current state. Then, based on the value of the discounted return... The predicted value of the state value function The difference between them determines the value of the dominance function. ,Right now: , This advantage function The superiority or inferiority of a specific action relative to the average level was quantified. Finally, a policy gradient-based reinforcement learning algorithm (such as PPO, A2C, etc.) was employed to maximize the expected reward, utilizing the advantage function. The strategy model parameters of the current data preprocessing strategy model for new energy power generation equipment are updated through gradient ascent. This update mechanism helps stabilize training and seek the long-term optimal strategy.

[0051] In one embodiment, such as Figure 3 As shown, after updating the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal, the method further includes step S160.

[0052] S160. Return to the state and continue to execute the corresponding action steps generated by the current data preprocessing strategy model until the preset termination condition is met, and output the data preprocessing strategy model when the termination condition is met.

[0053] In practice, after step S150 completes the parameter update of the data preprocessing strategy model, it returns to step S130 to continue execution. That is, the updated data preprocessing strategy model is used to process the next batch of coarsely washed data, and the subsequent steps of generating finely washed datasets, downstream model training and evaluation, reward signal calculation, and strategy model updates are repeated, forming a continuous self-optimizing closed loop. This iterative process is repeated until a preset termination condition is met (e.g., reaching the performance threshold of the downstream new energy power generation prediction model, reaching the maximum number of training rounds, or the performance improvement after multiple consecutive strategy updates falling below a set threshold). Finally, the data preprocessing strategy model that meets the termination condition is output. This model can be deployed to automate and intelligently preprocess data from new new energy businesses (such as photovoltaic power generation, hydropower, and wind power generation), thereby continuously improving the performance of the downstream power generation prediction model.

[0054] In summary, the method of this application shifts the goal of data preprocessing from pursuing the statistical purity of the data itself to directly maximizing the performance of the downstream new energy power generation prediction model. The system can automatically explore and learn the most suitable complex photovoltaic data preprocessing strategy for a specific prediction model and data characteristics without human intervention, realizing full-process automation and intelligence, and ultimately significantly improving the accuracy and stability of new energy power generation prediction.

[0055] See Figure 4This application also provides a new energy data preprocessing strategy self-optimization device, corresponding to the new energy data preprocessing strategy self-optimization method described above. The new energy data preprocessing strategy self-optimization device 200 includes a unit for executing the new energy data preprocessing strategy self-optimization method described above, and the device can be configured in a computer device.

[0056] Specifically, the new energy data preprocessing strategy self-optimization device 200 includes: an acquisition unit 201, a modeling unit 202, a generation unit 203, a training and evaluation unit 204, and an update unit 205.

[0057] The acquisition unit 201 is used to acquire the historical operation dataset of new energy power generation equipment and perform basic preprocessing on it to obtain a coarsely washed dataset. The modeling unit 202 is used to model the data cleaning process as a reinforcement learning task and initialize a data preprocessing strategy model. The reinforcement learning task includes a state, an action, and a reward function. The state is a feature representation of the current batch of data to be processed extracted from the coarsely cleaned dataset. The action is used to represent the processing decision for the data samples in the current batch of data to be processed. The reward function is used to generate a reward signal. The generation unit 203 is used to generate corresponding actions through the current data preprocessing strategy model, and process the data sample according to the generated actions to complete data cleaning and obtain a finely cleaned dataset. The training and evaluation unit 204 uses the refined dataset to train the downstream new energy power generation prediction model, and evaluates the performance of the new energy power generation prediction model on a preset evaluation dataset to obtain performance evaluation results. The update unit 205 is used to generate a reward signal based on the performance evaluation results and the reward function, and update the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal.

[0058] The aforementioned new energy data preprocessing strategy self-optimization device 200 can be implemented as a computer program, which can perform tasks such as... Figure 5 It runs on the computer device shown.

[0059] Please see Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 may be a terminal.

[0060] like Figure 5 As shown, the computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0061] The non-volatile storage medium 503 may store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions that, when executed, cause the processor 502 to perform the self-optimization method for the new energy data preprocessing strategy.

[0062] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0063] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the above-mentioned new energy data preprocessing strategy self-optimization method.

[0064] This network interface 505 is used for network communication with other devices. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0065] The processor 502 is used to run a computer program 5032 stored in a memory to implement the steps of the above method.

[0066] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0067] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program includes program instructions and can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0068] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When executed by a processor, the program instructions cause the processor to perform the steps of the above-described method.

[0069] The storage medium can be any computer-readable storage medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0070] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0071] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0072] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0073] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0074] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0075] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0076] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A self-optimization method for new energy data preprocessing strategy, characterized in that, The method includes: Obtain the historical operation dataset of new energy power generation equipment and perform basic preprocessing to obtain a coarsely washed dataset. The data cleaning process is modeled as a reinforcement learning task, and a data preprocessing strategy model is initialized. The reinforcement learning task includes a state, an action, and a reward function. The state is a feature representation of the current batch of data to be processed extracted from the coarsely cleaned dataset. The action is used to represent the processing decision for the data samples in the current batch of data to be processed. The reward function is used to generate a reward signal. Based on the state, corresponding actions are generated through the current data preprocessing strategy model, and the data samples are processed according to the generated actions to complete data cleaning and obtain a finely cleaned dataset. The refined dataset is used to train the downstream new energy power generation prediction model, and the performance of the new energy power generation prediction model is evaluated on a preset evaluation dataset to obtain performance evaluation results. A reward signal is generated based on the performance evaluation results and the reward function, and the current data preprocessing strategy model of the new energy power generation equipment is updated according to the generated reward signal.

2. The method according to claim 1, characterized in that, The reward signal includes a real-time reward and an environmental reward; the reward function includes a first reward function and a second reward function; the basic preprocessing includes labeling samples in the coarsely washed dataset with preset processing labels; and the generation of the reward signal based on the performance evaluation results and the reward function includes: The real-time reward is generated by the first reward function based on the degree of matching between the processing decision and the preset processing label of the corresponding sample in the coarse washing dataset. Based on the performance evaluation results of the new energy power generation prediction model on the preset evaluation dataset, the environmental reward is generated through the second reward function.

3. The method according to claim 2, characterized in that, The step of updating the current data preprocessing strategy model of the new energy power generation equipment based on the generated reward signal includes: Calculate the value of the discount reward based on the real-time reward and the environmental reward; Determine the predicted value of the state value function, and determine the value of the dominance function based on the difference between the value of the discounted return and the predicted value of the state value function. Based on the value of the advantage function, the strategy model parameters of the current data preprocessing strategy model of the new energy power generation equipment are updated.

4. The method according to claim 1, characterized in that, The status includes at least one of the original data values, statistical features, and context information of the current batch of data to be processed.

5. The method according to claim 1, characterized in that, The action is to assign a continuous confidence score to a data sample, which represents the probability that the data sample is retained.

6. The method according to claim 5, characterized in that, The processing decision for data samples within the current batch of data to be processed includes: The continuous confidence score corresponding to the data sample is compared with a preset threshold. Data samples with continuous confidence scores higher than the preset threshold are retained, and data samples with continuous confidence scores lower than or equal to the preset threshold are removed or have their weights weakened.

7. The method according to any one of claims 1-6, characterized in that, After updating the current data preprocessing strategy model of the new energy power generation equipment based on the generated reward signal, the method further includes: Returning to the state, the corresponding action steps are generated based on the current data preprocessing strategy model and continue to be executed until the preset termination condition is met, and the data preprocessing strategy model when the termination condition is met is output.

8. A self-optimizing device for new energy data preprocessing strategy, characterized in that, The device includes: The acquisition unit is used to acquire the historical operation dataset of new energy power generation equipment and perform basic preprocessing on it to obtain a coarsely washed dataset. A modeling unit is used to model the data cleaning process as a reinforcement learning task and initialize a data preprocessing strategy model. The reinforcement learning task includes a state, an action, and a reward function. The state is a feature representation of the current batch of data to be processed extracted from the coarsely cleaned dataset. The action is used to represent the processing decision for the data samples in the current batch of data to be processed. The reward function is used to generate a reward signal. The generation unit is used to generate corresponding actions through the current data preprocessing strategy model, and process the data samples according to the generated actions to complete data cleaning and obtain a finely cleaned dataset. The training and evaluation unit uses the refined dataset to train the downstream new energy power generation prediction model and evaluates the performance of the new energy power generation prediction model on a preset evaluation dataset to obtain performance evaluation results. The update unit is used to generate a reward signal based on the performance evaluation results and the reward function, and update the current data preprocessing strategy model of the new energy power generation equipment according to the generated reward signal.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images