A storage medium, a photovoltaic power generation data preprocessing method and system

Abstract

This invention belongs to the field of data processing technology, specifically relating to a storage medium, a method and system for preprocessing photovoltaic (PV) power generation data. The method acquires normal historical data sample values ​​of each power generation device under different operating conditions in a PV power plant based on historical PV power generation data; it detects the defect type of the current PV power generation data; if the defect type is a sudden value, it determines a data transformation model by combining historical data sample values ​​of the same power generation device under the same operating conditions; and adjusts the data scale of the sudden value based on the data transformation model to reduce the impact of the sudden value. Ultimately, while preserving the original characteristics of the data, it significantly improves the data quality.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a storage medium, a photovoltaic power generation data preprocessing method and system. Background Technology

[0002] Photovoltaic power generation systems are gaining increasing attention as a clean energy source. For example, Chinese patent document CN104934999B discloses a photovoltaic power generation control method, device, and system for improving the load response speed of photovoltaic power generation systems. With the rapid popularization of photovoltaic power generation technology, equipment fault diagnosis and data preprocessing techniques have become core components for ensuring system stability and power generation efficiency. Photovoltaic power generation data often suffers from anomalies due to communication quality issues, equipment failures, or human factors (such as power curtailment), such as missing data, duplicate data, or data mutations caused by noise interference. Therefore, preprocessing is generally used to clean noise or fill in missing values ​​to correct abnormal data and improve the accuracy of subsequent modeling and fault prediction. Existing methods generally use noise reduction algorithms to remove noise and reduce data mutations; however, noise reduction algorithms such as wavelet transform and Kalman filtering have limited adaptability, resulting in still low data quality after preprocessing. Summary of the Invention

[0003] The purpose of this invention is to provide a storage medium, a photovoltaic power generation data preprocessing method and system to solve the problem that the data quality obtained when processing photovoltaic power generation data abruptly through noise reduction is still low.

[0004] To address the aforementioned technical problems, this invention provides a photovoltaic power generation data preprocessing method, comprising: obtaining normal historical data sample values ​​of each power generation device under different operating conditions in a photovoltaic power station based on historical photovoltaic power generation data; detecting the defect type of the current photovoltaic power generation data; if the defect type is a sudden value, determining a data transformation model by combining historical data sample values ​​of the same power generation device under the same operating conditions; and adjusting the data scale of the sudden value based on the data transformation model.

[0005] Furthermore, methods for adjusting the data scale of mutation values ​​based on data transformation models include: If there is a sudden change in the operating condition, and there are similar operating conditions in the historical data, then the median of the photovoltaic power generation data within a certain period before and after the timestamp corresponding to the sudden change data is taken as the adjustment value of the sudden change value. If the operating conditions are stable and the mutation value is within the historical operating condition fluctuation range, the adjustment value of the mutation value is calculated based on the mutation value, the mutation value weight, the historical benchmark weight, the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment, and the average fluctuation step size. If the operating conditions are stable and the sudden change value exceeds the historical fluctuation range, then the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment is calculated. If the sudden change value exceeds the maximum value of the historical fluctuation range, the sudden change value is calibrated to be between the median and the maximum value of the historical fluctuation range. If the sudden change value is lower than the minimum value of the historical fluctuation range, the sudden change value is calibrated to be between the minimum value of the historical fluctuation range and the median. If a completely matching historical operating condition exists, the mutation value is replaced with the historical data sample value of the same operating condition and the same power generation equipment.

[0006] Furthermore, if the defect type is missing values, then the missing values ​​are calculated and filled by combining historical data sample values ​​of the same operating conditions and power generation equipment, based on polynomial fitting or multivariate collaborative calculation.

[0007] Furthermore, if the photovoltaic power generation data is from string devices, then when missing values ​​exist, the missing values ​​are calculated based on polynomial fitting by combining historical data sample values ​​of the same operating conditions and power generation equipment; if the photovoltaic power generation data is from inverters, then when missing values ​​exist, the missing values ​​are calculated based on multivariate collaborative calculation by combining historical data sample values ​​of the same operating conditions and power generation equipment.

[0008] Furthermore, if the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

[0009] Furthermore, if the defect type is one with invariant values, then the data is retained and the invariant values ​​are marked.

[0010] Furthermore, the defect type is determined based on the relationship between the current photovoltaic power generation data and the historical data sample values ​​of the corresponding power generation equipment under the corresponding operating conditions.

[0011] Furthermore, based on the relationship between current photovoltaic power generation data and historical data sample values ​​of corresponding power generation equipment under corresponding operating conditions, abrupt changes are detected using clustering algorithms or multi-parameter joint outlier detection methods.

[0012] The beneficial effects of the above technical solution are as follows: This invention is an improved invention. First, based on historical photovoltaic power generation data, it obtains historical data sample values ​​of each power generation device under different operating conditions in a normally recorded photovoltaic power generation station. After detecting abrupt changes in the photovoltaic power generation data, it uses a data transformation model determined by the historical data sample values ​​to adjust the data scale of the abrupt changes, thereby reducing their impact. Ultimately, while preserving the original characteristics of the data, it significantly improves the data quality.

[0013] To address the aforementioned technical problems, this invention also provides a photovoltaic power generation data preprocessing system, including a processor. The processor is used to implement the steps of a photovoltaic power generation data preprocessing method, which includes: acquiring normal historical data sample values ​​of each power generation device under different operating conditions in a photovoltaic power station based on historical photovoltaic power generation data; detecting the defect type of the current photovoltaic power generation data; if the defect type is a sudden value, determining a data transformation model by combining historical data sample values ​​of the same power generation device under the same operating conditions; and adjusting the data scale of the sudden value based on the data transformation model.

[0014] Furthermore, methods for adjusting the data scale of mutation values ​​based on data transformation models include: If there is a sudden change in the operating condition, and there are similar operating conditions in the historical data, then the median of the photovoltaic power generation data within a certain period before and after the timestamp corresponding to the sudden change data is taken as the adjustment value of the sudden change value. If the operating conditions are stable and the mutation value is within the historical operating condition fluctuation range, the adjustment value of the mutation value is calculated based on the mutation value, the mutation value weight, the historical benchmark weight, the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment, and the average fluctuation step size. If the operating conditions are stable and the sudden change value exceeds the historical fluctuation range, then the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment is calculated. If the sudden change value exceeds the maximum value of the historical fluctuation range, the sudden change value is calibrated to be between the median and the maximum value of the historical fluctuation range. If the sudden change value is lower than the minimum value of the historical fluctuation range, the sudden change value is calibrated to be between the minimum value of the historical fluctuation range and the median. If a completely matching historical operating condition exists, the mutation value is replaced with the historical data sample value of the same operating condition and the same power generation equipment.

[0015] Furthermore, if the defect type is missing values, then the missing values ​​are calculated and filled by combining historical data sample values ​​of the same operating conditions and power generation equipment, based on polynomial fitting or multivariate collaborative calculation.

[0016] Furthermore, if the photovoltaic power generation data is from string devices, then when missing values ​​exist, the missing values ​​are calculated based on polynomial fitting by combining historical data sample values ​​of the same operating conditions and power generation equipment; if the photovoltaic power generation data is from inverters, then when missing values ​​exist, the missing values ​​are calculated based on multivariate collaborative calculation by combining historical data sample values ​​of the same operating conditions and power generation equipment.

[0017] Furthermore, if the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

[0018] Furthermore, if the defect type is one with invariant values, then the data is retained and the invariant values ​​are marked.

[0019] Furthermore, the defect type is determined based on the relationship between the current photovoltaic power generation data and the historical data sample values ​​of the corresponding power generation equipment under the corresponding operating conditions.

[0020] Furthermore, based on the relationship between current photovoltaic power generation data and historical data sample values ​​of corresponding power generation equipment under corresponding operating conditions, abrupt changes are detected using clustering algorithms or multi-parameter joint outlier detection methods.

[0021] To address the aforementioned technical problems, the present invention also provides a storage medium storing computer-executable instructions. These instructions, when executed, implement steps in a photovoltaic power generation data preprocessing method. The method includes: acquiring normal historical data sample values ​​of each power generation device under different operating conditions in a photovoltaic power station based on historical photovoltaic power generation data; detecting the defect type of the current photovoltaic power generation data; if the defect type is a sudden change value, determining a data transformation model by combining historical data sample values ​​of the same power generation device under the same operating conditions; and adjusting the data scale of the sudden change value based on the data transformation model.

[0022] Furthermore, methods for adjusting the data scale of mutation values ​​based on data transformation models include: If there is a sudden change in the operating condition, and there are similar operating conditions in the historical data, then the median of the photovoltaic power generation data within a certain period before and after the timestamp corresponding to the sudden change data is taken as the adjustment value of the sudden change value. If the operating conditions are stable and the mutation value is within the historical operating condition fluctuation range, the adjustment value of the mutation value is calculated based on the mutation value, the mutation value weight, the historical benchmark weight, the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment, and the average fluctuation step size. If the operating conditions are stable and the sudden change value exceeds the historical fluctuation range, then the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment is calculated. If the sudden change value exceeds the maximum value of the historical fluctuation range, the sudden change value is calibrated to be between the median and the maximum value of the historical fluctuation range. If the sudden change value is lower than the minimum value of the historical fluctuation range, the sudden change value is calibrated to be between the minimum value of the historical fluctuation range and the median. If a completely matching historical operating condition exists, the mutation value is replaced with the historical data sample value of the same operating condition and the same power generation equipment.

[0023] Furthermore, if the defect type is missing values, then the missing values ​​are calculated and filled by combining historical data sample values ​​of the same operating conditions and power generation equipment, based on polynomial fitting or multivariate collaborative calculation.

[0024] Furthermore, if the photovoltaic power generation data is from string devices, then when missing values ​​exist, the missing values ​​are calculated based on polynomial fitting by combining historical data sample values ​​of the same operating conditions and power generation equipment; if the photovoltaic power generation data is from inverters, then when missing values ​​exist, the missing values ​​are calculated based on polynomial fitting by combining historical data sample values ​​of the same operating conditions and power generation equipment, and the missing values ​​are calculated using multivariate collaborative calculation.

[0025] Furthermore, if the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

[0026] Furthermore, if the defect type is one with invariant values, then the data is retained and the invariant values ​​are marked.

[0027] Furthermore, the defect type is determined based on the relationship between the current photovoltaic power generation data and the historical data sample values ​​of the corresponding power generation equipment under the corresponding operating conditions.

[0028] Furthermore, based on the relationship between current photovoltaic power generation data and historical data sample values ​​of corresponding power generation equipment under corresponding operating conditions, abrupt changes are detected using clustering algorithms or multi-parameter joint outlier detection methods.

[0029] Furthermore, the photovoltaic power generation data of the string device includes string current and string voltage; the photovoltaic power generation data of the inverter includes inverter power output, inverter DC power, inverter DC voltage, and inverter DC current.

[0030] The beneficial effects of the above technical solution are as follows: This invention is an improved invention. First, based on historical photovoltaic power generation data, it obtains historical data sample values ​​of each power generation device under different operating conditions in a normally recorded photovoltaic power generation station. After detecting abrupt changes in the photovoltaic power generation data, it uses a data transformation model determined by the historical data sample values ​​to adjust the data scale of the abrupt changes, thereby reducing their impact. Ultimately, while preserving the original characteristics of the data, it significantly improves the data quality. Attached Figure Description

[0031] Figure 1 This is a flowchart of the preprocessing process for string photovoltaic power generation data in an embodiment of the present invention; Figure 2 This is a flowchart of the inverter photovoltaic power generation data preprocessing process according to an embodiment of the method of the present invention. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of the present invention clearer, the specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0033] This invention first determines the defect type of the current abnormal data, selects an appropriate preprocessing method for data processing based on the defect type, and when there are abrupt changes, combines historical data sample values ​​of the same operating conditions and power generation equipment, adjusts the data scale of the abrupt changes based on the data transformation model, thereby improving data quality.

[0034] Method Implementation This invention provides a photovoltaic power generation data preprocessing method, primarily targeting the processing of abnormal data from two types of power generation equipment in photovoltaic power plants: strings and inverters. The method includes: 1. Obtain historical data sample values ​​of each power generation device under different operating conditions in the photovoltaic power station based on historical photovoltaic power generation data.

[0035] The power generation equipment includes string devices (composed of solar panels connected in series) and inverters. The photovoltaic power generation data of the string devices includes string current and string voltage, while the photovoltaic power generation data of the inverter includes inverter power output, inverter DC power, inverter DC voltage, and inverter DC current.

[0036] 2. Obtain current photovoltaic power generation data and detect the defect types in the current photovoltaic power generation data.

[0037] Obtain the current photovoltaic power generation data of the photovoltaic power generation equipment in the photovoltaic power station, extract the string data and inverter data, and determine the defect type of the string data and inverter data.

[0038] While existing technologies primarily focus on data cleaning, feature engineering, and multi-source data fusion, challenges remain in photovoltaic (PV) data analysis, including meteorological interference, equipment noise, data gaps, and difficulties in integrating heterogeneous multi-source data. Therefore, as a preferred implementation, this invention determines the defect type of the current PV power generation data based on the relationship between current PV power generation data and historical data samples from corresponding power generation equipment under similar operating conditions. By analyzing and comparing this data with normal data from PV power plants under similar historical operating conditions, the defect type of the current PV power generation data is determined, and then a suitable preprocessing method is selected. By introducing historical operating condition data and using it as a reference benchmark, interference from meteorological conditions and equipment noise can be significantly reduced. Historical data under similar operating conditions provides a reliable reference for handling abnormal data such as missing, duplicate, and abrupt changes. Furthermore, multivariate fitting or collaborative methods are employed to integrate heterogeneous multi-source data, thereby addressing the challenges of existing technologies.

[0039] Defect types include: missing values, invariant values, duplicate values, and mutated values.

[0040] If the current photovoltaic power generation data contains missing values, and the historical sample values ​​of the corresponding power generation equipment under the corresponding operating conditions all contain data within the corresponding time period, and the power generation equipment shows no faults, it is determined that there are missing values.

[0041] If the current photovoltaic power generation data are completely consistent within a continuous time period and the amount of consistent data is greater than the set number, calculate the data fluctuation coefficient (standard deviation / mean) of the historical data sample value of the corresponding power generation equipment under the corresponding operating condition. If the fluctuation coefficient is greater than the set threshold, it is determined that there is an invariant value. The set number and set threshold can be customized according to the needs, such as setting the number to 10 and the set threshold to 0.05.

[0042] If a timestamp appears repeatedly in the current photovoltaic power generation data, but does not appear in the historical data sample value of the corresponding power generation equipment under the corresponding operating condition, then it is determined that there is a duplicate value.

[0043] If outliers are found in the current photovoltaic data within a certain time frame, and there are no outliers in the historical data samples of the corresponding power generation equipment under the same operating conditions within the same time frame, then a sudden change in value is identified. The presence of outliers can be determined using the k-means clustering algorithm or the DBSCAN clustering algorithm.

[0044] 3. Based on the relationship between the corresponding operating conditions and the historical data sample values ​​of the power generation equipment, select the appropriate preprocessing method for data processing.

[0045] If the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

[0046] If the defect type is missing values, then by combining historical data samples from the same operating conditions and power generation equipment, the missing values ​​are calculated and filled based on multinomial fitting or multivariate collaborative calculation. Traditional interpolation methods such as mean interpolation or linear interpolation can lead to error accumulation; this invention can improve the accuracy of data filling.

[0047] Different methods are used to calculate missing values ​​for string devices and inverters, depending on their different characteristics. If the photovoltaic (PV) power generation data is from string devices, then when missing values ​​exist, they are calculated using polynomial fitting by combining historical data samples from the same equipment under the same operating conditions. The data used for polynomial fitting consists of PV power generation data from consecutive valid string devices before and after the missing value's timestamp and corresponding operating condition data. Specifically, a polynomial is first constructed based on the PV power generation data and operating condition data. The coefficients of the polynomial are then solved using historical data samples from the same equipment under the same operating conditions and corresponding operating condition data. Finally, the PV power generation data from consecutive valid string devices before and after the missing value are substituted into the polynomial to calculate the corresponding missing value. For example, a second-order polynomial is constructed by combining multiple variables such as voltage, current, irradiance, and module temperature. The coefficients of the polynomial are then calculated using historical data samples from the same equipment under the same operating conditions and corresponding operating condition data to simulate the current missing value.

[0048] If the photovoltaic (PV) power generation data refers to the PV power generation data of the inverter, then when missing values ​​exist, the missing values ​​are calculated based on multivariate collaborative calculation by combining historical data sample values ​​of the same operating conditions and power generation equipment. The data used for multivariate collaborative calculation includes PV power generation data and operating condition data of the continuously effective inverters before and after the missing value's timestamp information. The operating condition data includes irradiance and module temperature, etc. First, the multivariate collaborative relationship is solved based on historical data sample values ​​of the same operating conditions and power generation equipment and the corresponding operating condition data. Then, the missing values ​​are determined using the PV power generation data and corresponding operating condition data of the continuously effective inverters before and after the missing value.

[0049] If the defect type is "there are invariant values", then retain the data and mark the invariant values.

[0050] If the defect type is characterized by abrupt changes, a data transformation model is determined by combining historical data samples from the same power generation equipment under the same operating conditions. Based on this model, the data scale of the abrupt changes is adjusted to obtain the adjusted values. Clustering algorithms (such as the DBSCAN clustering algorithm) or multi-parameter joint outlier detection methods can be used to detect the presence of abrupt changes. Specifically: Operating conditions (such as irradiance / temperature) can also change abruptly. There are similar operating conditions in the historical data. The median smoothing method is used to take the median of photovoltaic power generation data within a certain period before and after the timestamp corresponding to the abrupt data as the adjustment value. If the operating conditions are stable and the mutation value is within the historical operating condition fluctuation range, then the adjustment value of the mutation value is calculated based on the mutation value, the mutation value weight, the historical benchmark weight, the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment, and the average fluctuation step size.

[0051] Specifically, weights are set using historical fluctuation parameters, including weights for mutation values. w 1 and historical benchmark weightsw 2, w 1+ w 2=1; Adjustment value= w 1×mutation value+ w 2 × (median of historical data sample values ​​for the same operating conditions and power generation equipment ± average fluctuation step size) to preserve the true trend; the historical operating condition fluctuation range refers to the normal and valid range of variation of historical data sample values ​​for the same operating conditions and power generation equipment, obtained by statistically analyzing the interval composed of the maximum and minimum values ​​of historical data sample values ​​for the same power generation equipment under the same operating conditions. The more consistent the degree of mutation is with the historical operating condition fluctuation, that is, the closer it is to the historical normal fluctuation (e.g., small mutation step size or small deviation from the historical median), the higher the weight of the mutation value (prioritizing the preservation of the true trend); the more consistent the degree of mutation is with the upper limit of historical fluctuation (but still within the range), that is, the closer it is to the upper limit of historical fluctuation, the higher the weight of the historical benchmark (avoiding excessive deviation from the operating condition trend).

[0052] If the operating condition is stable but the sudden value exceeds the historical fluctuation range, median filtering combined with historical data is used for calibration. The calibration method is as follows: statistically analyze the median of historical data samples of the same operating condition and the same power generation equipment. If the sudden value exceeds the maximum value of the historical fluctuation range, deviation standardization is used to calibrate the sudden value to between the median and the maximum value of the historical fluctuation range, that is, to [median, maximum value]. If the sudden value is lower than the minimum value of the historical fluctuation range, deviation standardization is used to calibrate the sudden value to between the minimum value of the historical fluctuation range and the median, that is, to [minimum value, median].

[0053] There are historical operating conditions that match perfectly, with most data such as irradiance, temperature, and time period being consistent. The historical data sample values ​​of the same operating conditions and the same power generation equipment are used for replacement.

[0054] The following sections will explain the preprocessing of photovoltaic power generation data for string devices and inverters, respectively.

[0055] like Figure 1 As shown, the following steps are used to preprocess abnormal current and voltage data from string generators: (1) Based on historical photovoltaic power generation data, compile historical data sample values ​​of string equipment under different operating conditions in photovoltaic power plants; (2) Detect whether there are duplicate values ​​in the abnormal data. If so, only retain the data with the latest timestamp. (3) Detect whether there are missing values ​​in the abnormal data. If so, combine the sample values ​​of current and voltage in the historical data of the same working condition, construct a polynomial fitting equation and calculate the missing values ​​to fill them in. (4) Detect whether there are invariant values ​​in the abnormal data. If so, retain the data and mark it as "invariant value". (5) Use clustering algorithms to detect whether there are abrupt values ​​in abnormal data. If so, combine the sample values ​​of current and voltage in the historical data of the same working conditions, set up a data transformation model, adjust the data scale of the abrupt values, and reduce the impact of the abrupt values.

[0056] like Figure 2 As shown, the following steps are taken to address abnormal data in the inverter's power generation, DC power, DC voltage, and DC current: S1: Based on historical data, compile historical data sample values ​​of inverter equipment under different operating conditions in photovoltaic power plants; S2: Detect whether there are duplicate values ​​in the abnormal data. If so, only keep the data with the latest timestamp. S3: Detect whether there are missing values ​​in the abnormal data. If so, combine the sample values ​​of power generation, DC power, DC voltage and DC current in the historical data of the same operating conditions, and use multivariate collaborative calculation to fill the missing values. S4: Detect whether there are invariant values ​​in the abnormal data. If so, retain the data and mark it as "invariant value". S5: Use a multi-parameter joint outlier detection method to detect whether there are abrupt changes in the abnormal data. If so, combine the sample values ​​of power generation, DC power, DC voltage and DC current in the historical data of the same operating conditions, set up a data transformation model, adjust the data scale of the abrupt changes, and reduce the impact of the abrupt changes.

[0057] 4. After cleaning the data according to the above preprocessing method, merge the cleaned data.

[0058] System Implementation This invention discloses a photovoltaic power generation data preprocessing system, comprising a processor and a memory, wherein the memory stores a computer program for execution on the processor. When the processor executes the computer program, it implements a photovoltaic power generation data preprocessing method, comprising: 1. The processor obtains historical data sample values ​​of each power generation device under different operating conditions in the photovoltaic power station based on historical photovoltaic power generation data.

[0059] The power generation equipment includes string devices (composed of solar panels connected in series) and inverters. The photovoltaic power generation data of the string devices includes string current and string voltage, while the photovoltaic power generation data of the inverter includes inverter power output, inverter DC power, inverter DC voltage, and inverter DC current.

[0060] 2. The processor acquires the current photovoltaic power generation data and detects the defect types in the current photovoltaic power generation data.

[0061] Obtain the current photovoltaic power generation data of the photovoltaic power generation equipment in the photovoltaic power station, extract the string data and inverter data, and determine the defect type of the string data and inverter data.

[0062] In one preferred embodiment, the present invention determines the defect type of the current photovoltaic power generation data based on the relationship between the current photovoltaic power generation data and the historical data sample values ​​of the corresponding power generation equipment under the corresponding operating conditions. By analyzing and comparing the current photovoltaic power generation data with normal data from photovoltaic power plants under similar historical operating conditions, the defect type of the current photovoltaic power generation data is determined, and then a suitable preprocessing method is selected for processing.

[0063] Defect types include: missing values, invariant values, duplicate values, and abrupt changes. If the current photovoltaic power generation data contains null values, but the historical sample values ​​for the corresponding power generation equipment under the corresponding operating conditions all contain data within the corresponding time period, and the power generation equipment shows no faults, then it is determined that there are missing values.

[0064] If the current photovoltaic power generation data are completely consistent within a continuous time period and the amount of consistent data is greater than the set number, calculate the data fluctuation coefficient (standard deviation / mean) of the historical data sample value of the corresponding power generation equipment under the corresponding operating condition. If the fluctuation coefficient is greater than the set threshold, it is determined that there is an invariant value. The set number and set threshold can be customized according to the needs, such as setting the number to 10 and the set threshold to 0.05.

[0065] If a timestamp appears repeatedly in the current photovoltaic power generation data, but does not appear in the historical data sample value of the corresponding power generation equipment under the corresponding operating condition, then it is determined that there is a duplicate value.

[0066] If outliers are found in the current photovoltaic data within a certain time frame, and there are no outliers in the historical data samples of the corresponding power generation equipment under the same operating conditions within the same time frame, then a sudden change in value is identified. The presence of outliers can be determined using the k-means clustering algorithm or the DBSCAN clustering algorithm.

[0067] 3. The processor combines the relationship between the corresponding operating conditions and the historical data sample values ​​of the power generation equipment, and selects the corresponding preprocessing method for data processing.

[0068] If the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

[0069] If the defect type is missing values, then the missing values ​​are calculated and filled by combining historical data sample values ​​of the same operating conditions and power generation equipment, based on polynomial fitting or multivariate collaborative calculation.

[0070] Different methods are used to calculate missing values ​​for string devices and inverters, depending on their different characteristics. If the photovoltaic (PV) power generation data is from string devices, then when missing values ​​exist, they are calculated using polynomial fitting by combining historical data samples from the same equipment under the same operating conditions. The data used for polynomial fitting consists of PV power generation data from consecutive valid string devices before and after the missing value's timestamp and corresponding operating condition data. Specifically, a polynomial is first constructed based on the PV power generation data and operating condition data. The coefficients of the polynomial are then solved using historical data samples from the same equipment under the same operating conditions and corresponding operating condition data. Finally, the PV power generation data from consecutive valid string devices before and after the missing value and corresponding operating condition data are substituted into the polynomial to calculate the corresponding missing value. For example, a second-order polynomial is constructed by combining multiple variables such as voltage, current, irradiance, and module temperature. The coefficients of the polynomial are then calculated using historical data samples from the same equipment under the same operating conditions and corresponding operating condition data to simulate the current missing value.

[0071] If the photovoltaic (PV) power generation data refers to the PV power generation data of the inverter, then when missing values ​​exist, the missing values ​​are calculated based on multivariate collaborative calculation by combining historical data sample values ​​of the same operating conditions and power generation equipment. The data used for multivariate collaborative calculation includes PV power generation data and operating condition data of the continuously effective inverters before and after the missing value's timestamp information. The operating condition data includes irradiance and module temperature, etc. First, the multivariate collaborative relationship is solved based on historical data sample values ​​of the same operating conditions and power generation equipment and the corresponding operating condition data. Then, the missing values ​​are determined using the PV power generation data and corresponding operating condition data of the continuously effective inverters before and after the missing value.

[0072] If the defect type is "there are invariant values", then retain the data and mark the invariant values.

[0073] If the defect type involves abrupt changes, a data transformation model is determined by combining historical data samples from the same power generation equipment under the same operating conditions. The data scale for these abrupt changes is then adjusted based on this model. Clustering algorithms (such as DBSCAN) or multi-parameter joint outlier detection methods can be used to detect the presence of abrupt changes. The method for determining the data transformation model is as follows: Operating conditions (such as irradiance / temperature) can also change abruptly. There are similar operating conditions in the historical data. The median smoothing method is used to take the median of photovoltaic power generation data within a certain period before and after the timestamp corresponding to the abrupt data as the adjustment value. If the operating conditions are stable and the mutation value is within the historical operating condition fluctuation range, then the adjustment value of the mutation value is calculated based on the mutation value, the mutation value weight, the historical benchmark weight, the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment, and the average fluctuation step size.

[0074] Specifically, weights are set using historical fluctuation parameters, including weights for mutation values. w 1 and historical benchmark weightsw 2, w 1+ w 2=1; Adjustment value= w 1×mutation value+ w 2 × (median of historical data sample values ​​for the same operating conditions and power generation equipment ± average fluctuation step size) to preserve the true trend; the historical operating condition fluctuation range refers to the normal and valid range of variation of historical data sample values ​​for the same operating conditions and power generation equipment, obtained by statistically analyzing the interval composed of the maximum and minimum values ​​of historical data sample values ​​for the same power generation equipment under the same operating conditions. The more consistent the degree of mutation is with the historical operating condition fluctuation, that is, the closer it is to the historical normal fluctuation (e.g., small mutation step size or small deviation from the historical median), the higher the weight of the mutation value (prioritizing the preservation of the true trend); the more consistent the degree of mutation is with the upper limit of historical fluctuation (but still within the range), that is, the closer it is to the upper limit of historical fluctuation, the higher the weight of the historical benchmark (avoiding excessive deviation from the operating condition trend).

[0075] If the operating condition is stable but the sudden value exceeds the historical fluctuation range, median filtering combined with historical data is used for calibration. The calibration method is as follows: statistically analyze the median of historical data samples of the same operating condition and the same power generation equipment. If the sudden value exceeds the maximum value of the historical fluctuation range, deviation standardization is used to calibrate the sudden value to between the median and the maximum value of the historical fluctuation range, that is, to [median, maximum value]. If the sudden value is lower than the minimum value of the historical fluctuation range, deviation standardization is used to calibrate the sudden value to between the minimum value of the historical fluctuation range and the median, that is, to [minimum value, median].

[0076] There are historical operating conditions that match perfectly, with most data such as irradiance, temperature, and time period being consistent. The historical data sample values ​​of the same operating conditions and the same power generation equipment are used for replacement.

[0077] Specifically, the following steps are used to preprocess abnormal current and voltage data from string generator sets: (1) Based on historical photovoltaic power generation data, compile historical data sample values ​​of string equipment under different operating conditions in photovoltaic power plants; (2) Detect whether there are duplicate values ​​in the abnormal data. If so, only retain the data with the latest timestamp. (3) Detect whether there are missing values ​​in the abnormal data. If so, combine the sample values ​​of current and voltage in the historical data of the same working condition, construct a polynomial fitting equation and calculate the missing values ​​to fill them in. (4) Detect whether there are invariant values ​​in the abnormal data. If so, retain the data and mark it as "invariant value". (5) Use clustering algorithms to detect whether there are abrupt values ​​in abnormal data. If so, combine the sample values ​​of current and voltage in the historical data of the same working conditions, set up a data transformation model, adjust the data scale of the abrupt values, and reduce the impact of the abrupt values.

[0078] The following steps are taken to address abnormal data in the inverter's power generation, DC power, DC voltage, and DC current: S1: Based on historical data, compile historical data sample values ​​of inverter equipment under different operating conditions in photovoltaic power plants; S2: Detect whether there are duplicate values ​​in the abnormal data. If so, only keep the data with the latest timestamp. S3: Detect whether there are missing values ​​in the abnormal data. If so, combine the sample values ​​of power generation, DC power, DC voltage and DC current in the historical data of the same operating conditions, and use multivariate collaborative calculation to fill the missing values. S4: Detect whether there are invariant values ​​in the abnormal data. If so, retain the data and mark it as "invariant value". S5: Use a multi-parameter joint outlier detection method to detect whether there are abrupt changes in the abnormal data. If so, combine the sample values ​​of power generation, DC power, DC voltage and DC current in the historical data of the same operating conditions, set up a data transformation model, adjust the data scale of the abrupt changes, and reduce the impact of the abrupt changes.

[0079] 4. After cleaning the data according to the above preprocessing method, merge the cleaned data.

[0080] Storage Media Implementation The present invention provides a storage medium storing computer-executable instructions, which, when executed, implement steps in a photovoltaic power generation data preprocessing method. These instructions include: Based on historical photovoltaic power generation data, obtain normal historical data sample values ​​of each power generation equipment under different operating conditions in the photovoltaic power station; Acquire current photovoltaic power generation data and detect the types of defects in the current photovoltaic power generation data; Among them, the defect type of the current photovoltaic power generation data is determined based on the relationship between the current photovoltaic power generation data and the historical data sample values ​​of the corresponding power generation equipment under the corresponding operating conditions. The defect types include: missing values, invariant values, duplicate values, and abrupt changes.

[0081] Based on the relationship between the corresponding operating conditions and the historical data sample values ​​of the power generation equipment, the appropriate preprocessing method is selected for data processing; If the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

[0082] If the defect type is missing values, then the missing values ​​are calculated and filled by combining historical data sample values ​​of the same operating conditions and power generation equipment, based on polynomial fitting or multivariate collaborative calculation.

[0083] Different methods are used to calculate missing values ​​for string devices and inverters, depending on their different characteristics. If the photovoltaic power generation data is from string devices, missing values ​​are calculated using polynomial fitting by combining historical data samples from the same equipment under the same operating conditions. If the photovoltaic power generation data is from inverters, missing values ​​are calculated using multivariate collaborative calculation by combining historical data samples from the same equipment under the same operating conditions.

[0084] If the defect type is "there are invariant values", then retain the data and mark the invariant values.

[0085] If the defect type is characterized by a sudden change in value, then the data scale of the sudden change in value is adjusted based on the data transformation model, taking into account historical data sample values ​​of the same operating conditions and power generation equipment.

[0086] After cleaning the data using the preprocessing methods described above, the cleaned data is merged.

[0087] This invention compiles sample values ​​of normal power generation equipment under different operating conditions based on historical data. It then determines the defect type of the current abnormal data based on the relationship between abnormal data and the sample values ​​under corresponding operating conditions. Finally, it selects an appropriate preprocessing method for data processing based on the type of abnormal data and the defect type. All data used are basic power generation equipment data, requiring no additional data acquisition devices and having no preconditions. The calculation process is simple, solving the problems of complex calculations and low efficiency when processing abnormal data from a large number of connected photovoltaic power generation devices. It exhibits high computational efficiency when cleaning large amounts of photovoltaic power generation data. The preprocessing method has broad application capabilities and can adapt well to various types of photovoltaic power plants.

[0088] This system employs a three-step core strategy—dynamic sample library construction, multivariate collaborative modeling, and adaptive anomaly detection—to achieve refined processing of anomaly data from photovoltaic (PV) power plant equipment. For the different characteristics of string devices and inverters, multinomial fitting and multivariate collaborative techniques are used to fill missing values, respectively. Combined with DBSCAN clustering and robust scaling to adjust abrupt changes, the system significantly improves data quality while preserving the original data characteristics. This process is not only applicable to PV scenarios, but its modular design (such as the missing value filling module and anomaly detection module) can be transferred to other new energy fields such as wind power and energy storage.

Claims

1. A method for preprocessing photovoltaic power generation data, characterized in that, include: Based on historical photovoltaic power generation data, obtain normal historical data sample values ​​of each power generation equipment under different operating conditions in the photovoltaic power station; The defect type of the current photovoltaic power generation data is detected. If the defect type is a sudden value, the data transformation model is determined by combining the historical data sample values ​​of the same operating conditions and power generation equipment. The data scale of the sudden value is adjusted based on the data transformation model.

2. The photovoltaic power generation data preprocessing method according to claim 1, characterized in that, Methods for adjusting the data scale of mutation values ​​based on data transformation models include: If there is a sudden change in the operating condition, and there are similar operating conditions in the historical data, then the median of the photovoltaic power generation data within a certain period before and after the timestamp corresponding to the sudden change data is taken as the adjustment value of the sudden change value. If the operating conditions are stable and the mutation value is within the historical operating condition fluctuation range, the adjustment value of the mutation value is calculated based on the mutation value, the mutation value weight, the historical benchmark weight, the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment, and the average fluctuation step size. If the operating conditions are stable and the sudden change value exceeds the historical fluctuation range, then the median of the historical data sample values ​​of the same operating conditions and the same power generation equipment is calculated. If the sudden change value exceeds the maximum value of the historical fluctuation range, the sudden change value is calibrated to be between the median and the maximum value of the historical fluctuation range. If the sudden change value is lower than the minimum value of the historical fluctuation range, the sudden change value is calibrated to be between the minimum value of the historical fluctuation range and the median. If a completely matching historical operating condition exists, the mutation value is replaced with the historical data sample value of the same operating condition and the same power generation equipment.

3. The photovoltaic power generation data preprocessing method according to claim 1, characterized in that, If the defect type is missing values, then the missing values ​​are calculated and filled by combining historical data sample values ​​of the same operating conditions and power generation equipment, based on polynomial fitting or multivariate collaborative calculation.

4. The photovoltaic power generation data preprocessing method according to claim 3, characterized in that, If the photovoltaic power generation data is from string devices, then when missing values ​​exist, the missing values ​​are calculated based on multinomial fitting by combining historical data sample values ​​of the same power generation equipment under the same operating conditions. If the photovoltaic power generation data is from inverters, then when missing values ​​exist, the missing values ​​are calculated based on multivariate collaborative calculation by combining historical data sample values ​​of the same power generation equipment under the same operating conditions.

5. The photovoltaic power generation data preprocessing method according to claim 1, characterized in that, If the defect type is duplicate values, only the photovoltaic power generation data with the latest timestamp among the duplicate values ​​will be retained.

6. The photovoltaic power generation data preprocessing method according to claim 1, characterized in that, If the defect type is one with invariant values, then the data is retained and the invariant values ​​are marked.

7. The photovoltaic power generation data preprocessing method according to any one of claims 1-6, characterized in that, The defect type is determined based on the relationship between current photovoltaic power generation data and historical data sample values ​​of the corresponding power generation equipment under corresponding operating conditions.

8. The photovoltaic power generation data preprocessing method according to claim 1, characterized in that, Based on the relationship between current photovoltaic power generation data and historical data sample values ​​of corresponding power generation equipment under corresponding operating conditions, abrupt changes are detected using clustering algorithms or multi-parameter joint outlier detection methods.

9. A photovoltaic power generation data preprocessing system, comprising a processor, characterized in that, The processor is used to implement the steps in the photovoltaic power generation data preprocessing method as described in any one of claims 1-8.

10. A storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are used, when executed, to implement the steps in the photovoltaic power generation data preprocessing method as described in any one of claims 1-8.

Images