Multi-model fusion photovoltaic power generation power abnormity detection method and device

Abstract

The invention discloses a multi-model fusion photovoltaic power generation power abnormity detection method and device. The method comprises the following steps: dividing obtained photovoltaic data and meteorological data into a training set, a verification set and a test set according to dates; according to the feature data of the training set and the verification set, training optimization is carried out on the LightGBM model, the K neighbor regression model and the random forest model, and the optimized LightGBM model, the optimized K neighbor regression model and the optimized random forest model are obtained; respectively inputting the test set into the optimized LightGBM model, the optimized K-nearest neighbor regression model and the optimized random forest model to obtain a LightGBM model prediction result, a K-nearest neighbor regression model prediction result and a random forest model prediction result; inputting the test set into the trained ridge regression Ridge model to obtain a model fusion prediction value; and performing anomaly diagnosis according to the model fusion prediction value and the test set to obtain an anomaly diagnosis result. The method and the device have the technical effects of improving the diagnosis accuracy and generalization ability.

Description

Technical Field

[0001] The present invention relates to the field of photovoltaic technology applications, and in particular to a multi-model fusion photovoltaic power anomaly detection method and device. Background Art

[0002] In recent years, with the acceleration of urbanization, energy shortages and pollution problems have intensified. In order to reduce dependence on traditional energy, new energy power generation facilities have begun to be widely used. Photovoltaic has surpassed the installed capacity of hydropower and has become the second largest power source in the country. This has put forward higher and higher requirements for the analysis and mining of photovoltaic power generation data. The data anomaly of photovoltaic power generation directly affects the statistics of grid-connected electricity, and indirectly affects the calculation of electricity charges and the formulation of demand response policies. Therefore, it is of practical significance to study the abnormal diagnosis of photovoltaic power generation.

[0003] There are many traditional methods for detecting abnormal photovoltaic power generation, such as using infrared thermal imagers to detect the temperature distribution of photovoltaic modules, and identifying whether the modules are faulty or abnormal through abnormal temperature distribution, which requires additional hardware costs. For example, manual inspection of mechanical anomalies and obstructions can be used to detect whether photovoltaic mechanical parts are damaged or loose, affected by snow accumulation or obstructions, which affect light exposure and lead to a decrease in power generation, which requires additional labor costs. Another example is the historical data comparison method, which verifies whether the power generation in the same time period has abnormal fluctuations or obvious declines. This method can only detect relatively obvious and long-term anomalies.

[0004] Traditional methods rely on additional hardware or manpower to detect abnormal photovoltaic power generation, have low judgment accuracy, slow response speed, and fail to fully consider the influence of multiple factors, resulting in poor generalization ability.

[0005] There is currently no effective solution to the problem that abnormality diagnosis is difficult and inaccurate due to the small amount of data and large fluctuations in existing technologies. Summary of the invention

[0006] In order to solve the above technical problems, the embodiments of the present invention hope to provide a method and device for detecting photovoltaic power anomalies by fusion of multiple models, so as to at least solve the problem in the prior art that abnormality diagnosis is difficult and inaccurate due to small data volume and large fluctuations.

[0007] The technical solution of the present invention is achieved in this way:

[0008] The embodiment of the present invention provides a multi-model fusion photovoltaic power anomaly detection method, comprising: dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date; performing data cleaning on the training set and the validation set, and constructing feature data for the training set and the validation set after data cleaning; training and optimizing the LightGBM model, the K nearest neighbor regression model and the random forest model according to the feature data of the training set and the validation set, respectively, to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; inputting the test set into the optimized LightGBM model, the K nearest neighbor regression model and the random forest model, respectively, to obtain the LightGBM model prediction result, the K nearest neighbor regression model prediction result and the random forest model prediction result; inputting the LightGBM model prediction result, the K nearest neighbor regression model prediction result and the random forest model prediction result into the ridge regression model for model training, to obtain the trained ridge regression model; inputting the test set into the trained ridge regression model to obtain the model fusion prediction value; performing abnormal diagnosis according to the model fusion prediction value and the test set, to obtain the abnormal diagnosis result.

[0009] Optionally, before dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to date, the method also includes: acquiring power generation data, photovoltaic archive data and meteorological data within a preset time period on the diagnosis day and before the diagnosis day; obtaining photovoltaic data based on the power generation data and photovoltaic archive data; and obtaining a database based on the photovoltaic data and meteorological data.

[0010] Optionally, dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to date includes: dividing the photovoltaic data and meteorological data according to a preset ratio according to date to obtain a training set, a validation set and a test set.

[0011] Optionally, data cleaning of the training set and the validation set includes: cleaning historical data in the training set and the validation set to obtain a training set and a validation set with abnormal historical data removed; based on the earliest non-zero power time and the latest non-zero power time combined with the earliest sunrise time and sunset time of each day, removing data during the photovoltaic non-power generation period at night to obtain a training set and a validation set after data cleaning.

[0012] Further, optionally, constructing feature data for the training set and validation set after data cleaning includes: performing feature aggregation, feature crossover and feature transformation on the training set and validation set after data cleaning to obtain feature data; removing feature data whose missing value ratio is greater than a first threshold or whose maximum attribute ratio is greater than a second threshold to obtain removed feature data; and screening feature data whose absolute value of the correlation coefficient with photovoltaic power generation power is greater than a third threshold based on the Pearson correlation coefficient between the removed feature data and photovoltaic power generation power to obtain screened feature data.

[0013] Optionally, the LightGBM model, K nearest neighbor regression model and random forest model are trained and optimized respectively according to the characteristic data of the training set and the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model, including: training the LightGBM model, K nearest neighbor regression model and random forest model respectively according to the characteristic data of the training set, and performing hyperparameter tuning and feature selection on the trained LightGBM model, K nearest neighbor regression model and random forest model through the characteristic data of the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model.

[0014] Furthermore, optionally, hyperparameter tuning and feature selection are performed on the trained LightGBM model, K nearest neighbor regression model and random forest model through the feature data of the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model, including: evaluating the trained LightGBM model, K nearest neighbor regression model and random forest model respectively through the feature data of the validation set to obtain the evaluation results of the LightGBM model, K nearest neighbor regression model and random forest model; selecting hyperparameters from the evaluation results of the LightGBM model, K nearest neighbor regression model and random forest model through grid search GridSearch; screening hyperparameters according to the root mean square error (RMSE), an accuracy evaluation indicator on the validation set, to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model.

[0015] Optionally, anomaly diagnosis is performed based on the model fusion prediction value and the test set, and the anomaly diagnosis result obtained includes: calculating the absolute difference and the difference ratio based on the model fusion prediction value and the test set respectively; if the absolute difference is greater than the fourth threshold, and the difference ratio is greater than the fifth threshold, then the anomaly diagnosis result is determined to be an anomaly at the user time point, and the abnormal date and user are output.

[0016] The embodiment of the present invention provides a multi-model fusion photovoltaic power anomaly detection device, including: a division module, used to divide the acquired photovoltaic data and meteorological data into a training set, a verification set and a test set according to the date; a cleaning module, used to perform data cleaning on the training set and the verification set, and construct feature data for the training set and the verification set after data cleaning; a first training module, used to train and optimize the LightGBM model, the K nearest neighbor regression model and the random forest model according to the feature data of the training set and the verification set, and obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; a first prediction module, used to input the test set into the optimized The LightGBM model, the K nearest neighbor regression model and the random forest model are used to obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model; the second training module is used to input the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model into the Ridge Regression Ridge model for model training to obtain the trained Ridge Regression Ridge model; the second prediction module is used to input the test set into the trained Ridge Regression Ridge model to obtain the model fusion prediction value; the anomaly detection module is used to perform anomaly diagnosis based on the model fusion prediction value and the test set to obtain the anomaly diagnosis result.

[0017] Optionally, the device also includes: an acquisition module, used to acquire power generation data, photovoltaic archive data and meteorological data within a preset time period before the diagnosis day and the diagnosis day before dividing the acquired photovoltaic data and meteorological data into a training set, a verification set and a test set according to the date; a data acquisition module, used to obtain photovoltaic data based on the power generation data and the photovoltaic archive data; and a construction module, used to obtain a database based on the photovoltaic data and the meteorological data.

[0018] The embodiment of the present invention provides a method and device for detecting abnormal photovoltaic power generation by multi-model fusion, which divides the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date; performs data cleaning on the training set and the validation set, and constructs feature data for the training set and the validation set after data cleaning; trains and optimizes the LightGBM model, the K nearest neighbor regression model and the random forest model according to the feature data of the training set and the validation set, respectively, to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; inputs the test set into the optimized LightGBM model, the K nearest neighbor regression model and the random forest model, respectively, to obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model; inputs the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model into the ridge regression model for model training, to obtain the trained ridge regression model; inputs the test set into the trained ridge regression model to obtain the model fusion prediction value; performs abnormal diagnosis according to the model fusion prediction value and the test set, to obtain the abnormal diagnosis result, thereby improving the technical effect of the accuracy of diagnosis and generalization ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 A schematic flow chart of a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0021] Figure 2 A regression relationship diagram between feature data and prediction targets in a multi-model fusion photovoltaic power anomaly detection method provided by an embodiment of the present invention;

[0022] Figure 3 A histogram and a kernel density estimation diagram of feature data in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0023] Figure 4 A heat map of the correlation between feature data and prediction targets in a multi-model fusion photovoltaic power anomaly detection method provided in an embodiment of the present invention;

[0024] Figure 5 A schematic diagram of feature importance ranking in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0025] Figure 6A schematic diagram of Boosting of the LightGBM algorithm in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided in an embodiment of the present invention;

[0026] Figure 7 A schematic diagram of a K-nearest neighbor algorithm in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0027] Figure 8 A schematic diagram of bagging of a random forest algorithm in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0028] Fig. 9 A schematic diagram of comparison between a user's prediction and actual results in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0029] Fig.10 A ridge regression model path diagram in a method for detecting abnormal photovoltaic power generation using multi-model fusion provided by an embodiment of the present invention;

[0030] Fig.11 A schematic diagram of a multi-model fusion photovoltaic power anomaly detection device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second" and the like in the specification, claims and drawings of the present invention are used to distinguish different objects rather than to limit a specific order.

[0033] It should also be noted that the following embodiments of the present invention may be executed separately, or in combination with each other, and the embodiments of the present invention do not impose any specific limitations on this.

[0034] An embodiment of the present invention provides a sound-controlled home appliance system. Figure 1 A schematic diagram of a process flow of a photovoltaic power generation abnormality detection method using multi-model fusion provided by an embodiment of the present invention; Figure 1 As shown, the photovoltaic power generation power anomaly detection method provided by the embodiment of the present application with multi-model fusion includes:

[0035] Step S100, dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date;

[0036] Optionally, before dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to date in step S100, the multi-model fusion photovoltaic power generation anomaly detection method provided in the embodiment of the present application also includes: acquiring power generation data, photovoltaic archive data and meteorological data within a preset time period before the diagnosis date and the diagnosis date; obtaining photovoltaic data based on the power generation data and the photovoltaic archive data; and obtaining a database based on the photovoltaic data and the meteorological data.

[0037] Specifically, the power generation data, photovoltaic archive data and meteorological data of each photovoltaic user at a time point every 15 minutes on the diagnosis day and a period of time before the diagnosis day (i.e., the preset time period before the diagnosis day in the embodiment of the present application) are input to form a basic database.

[0038] Optionally, dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to date in step S100 includes: dividing the photovoltaic data and meteorological data according to a preset ratio according to the date to obtain a training set, a validation set and a test set.

[0039] Specifically, the data (ie, the photovoltaic data and meteorological data in the embodiment of the present application) are divided into a training set, a validation set, and a test set according to date.

[0040] The test set, training set, and validation set are all independent of each other, because some historical data of historical shifts and time windows are used in the feature data of the model (for example, the power generation of the photovoltaic user at the same time 1-N days ago, the mean, median, maximum, minimum, and standard deviation of the power generation), so in order to avoid the crossing of feature data, when dividing the training set and the validation set, the first 60% is divided into the training set, the second 20% is divided into the validation set, and the last 20% is divided into the test set (equivalent to the data of the abnormal diagnosis day in actual application) according to the time sequence. Among them, 60%, 20% and 20% are the preset proportions in the embodiment of this application.

[0041] The training set is mainly used in the model training process. By observing and learning the sample data in the training set, the model can learn the characteristic data and rules, and thus establish a prediction model. First, you need to train on the training set. If the effect on the training set is not good (see the model evaluation indicators, such as the RMSE below), it is underfitting, and you need to add more effective characteristic data, increase the amount of data, or increase the complexity of the model.

[0042] Verify on the validation set. If the effect is much worse than that of the training set, it means that the model is overfitting. You need to adjust the hyperparameters, such as reducing the complexity of the model, reducing the depth of the tree, reducing the number of trees, etc., so as to achieve an effect close to that of the training set.

[0043] Step S102, performing data cleaning on the training set and the validation set, and constructing feature data for the training set and the validation set after the data cleaning;

[0044] Optionally, data cleaning of the training set and the validation set in step S102 includes: cleaning historical data in the training set and the validation set to obtain a training set and a validation set with abnormal historical data removed; based on the earliest non-zero power and the latest non-zero power time combined with the earliest sunrise time and sunset time of each day, clearing the data during the evening photovoltaic non-power generation period to obtain a training set and a validation set after data cleaning.

[0045] Specifically, (1) the historical data used for the training set and validation set are cleaned, while the test set is not cleaned, so as to be used for diagnosing anomalies.

[0046] Eliminate abnormal data where the power generation throughout the day is one or some fixed values, count the number of unique values ​​of the power generation of each device every day, and delete the days with too few unique values ​​(less than K).

[0047] Eliminate the earliest non-zero power and the latest non-zero power time, and exclude the date that is too far away from sunrise and sunset (greater than N minutes). For example, if there is power generation at 4 pm, or there is data at 4 am, such data cannot be used for training and verification.

[0048] (2) For the power generation data, based on the earliest non-zero power and the latest non-zero power time, combined with the earliest sunrise and sunset times of the day, the data during the evening when photovoltaic power generation is not available are removed because the power generation during these periods is 0. In order to avoid data skew and affect the training effect, they are deleted.

[0049] Further, optionally, constructing feature data for the training set and validation set after data cleaning in step S102 includes: performing feature aggregation, feature crossover and feature transformation on the training set and validation set after data cleaning to obtain feature data; removing feature data whose missing value ratio is greater than a first threshold or whose maximum attribute ratio is greater than a second threshold to obtain removed feature data; and screening feature data whose absolute value of the correlation coefficient with photovoltaic power generation power is greater than a third threshold based on the Pearson correlation coefficient between the removed feature data and photovoltaic power generation power to obtain screened feature data.

[0050] Specifically, through feature aggregation, the features of multiple time periods are aggregated into a single feature. For example, the power generation of the photovoltaic user at the same time 1-N days ago, the statistical features such as the mean, median, maximum, minimum, and standard deviation of the power generation are calculated. Through feature intersection, using temperature and hours, the temperature change relative to the previous hour is generated. Through feature transformation, the input requirements of the machine learning algorithm are met. For example, through label encoding, the classification features such as real-time weather conditions and wind direction are transformed. Label encoding maps each category to an integer value, starting from 0 and increasing. The numerical features are normalized, and MinMaxScaler is used for normalization.

[0051] The following six types of feature data are constructed: photovoltaic archive features, historical translation and window statistical features, time-related features, time-related derivative features, meteorological features, and derivative features of meteorological indicators (i.e., feature data in the embodiments of the present application).

[0052] The specific characteristics of each type of data are as follows:

[0053] 1) PV profile characteristics: capacity, number of modules, inverter power.

[0054] 2) Historical shift and window statistical features: The power generation of the PV user at the same time 1-N days ago, the mean, median, maximum, minimum and standard deviation of the power generation are generated.

[0055] 3) Time features

[0056] Year, month, day, season, time point.

[0057] 4) Time-dependent derivative features

[0058] The time point after sunrise, the time point before sunset, the number of minutes from sunrise, the number of minutes from sunset, the date serial number (the number of days counting from the earliest date in the data), the time serial number, and the time point of the day.

[0059] 5) Meteorological characteristics:

[0060] The daily and hourly weather data for the area where the PV user is located, but there is no more fine-grained weather data:

[0061] Daily weather data, including: sunrise time, sunset time, daily maximum temperature, daily minimum temperature, daily average temperature, daily average humidity, and daily precipitation.

[0062] Hourly weather data, including: weather conditions, precipitation intensity, precipitation probability, temperature, perceived temperature, dew point, humidity, air pressure, wind speed, gusts, wind direction, cloud cover, UV index, visibility, and ozone. Although the UV index is related to power generation, it is an integer discrete value of 0-15, the data accuracy is not enough, and it does not reach every time point in time. In addition, the weather changes rapidly and unpredictably, so it is necessary to combine more weather features to increase the robustness of the model.

[0063] 6) Derivative characteristics of meteorological indicators

[0064] The change values ​​of indicators such as temperature and humidity relative to the previous day, and the temperature change relative to the previous hour. Because the impact of sunlight on temperature has a certain lag, the following features are added: temperature and change 1-3 hours after the current time, etc.

[0065] The photovoltaic power anomaly detection method using multi-model fusion provided in the embodiment of the present application converts raw data into feature data that better expresses the essence of the problem, so that these feature data can be applied to the prediction model to improve the prediction accuracy.

[0066] In the embodiment of the present application, unqualified feature data is eliminated, including feature data with too high a proportion of missing data values ​​(i.e., the proportion of missing data values ​​in the embodiment of the present application is greater than the first threshold) or too high a proportion of maximum attributes (i.e., the proportion of maximum attributes in the embodiment of the present application is greater than the second threshold).

[0067] Feature correlation analysis: Analyze the regression relationship between features and prediction targets to find out whether there is a linear relationship, trend or other form of association, so as to identify feature data that is effective for prediction. Figure 2 A regression relationship diagram of feature data and prediction target in a multi-model fusion photovoltaic power anomaly detection method provided by an embodiment of the present invention, such as Figure 2 As shown, the regplot function in seaborn is used for linear regression fitting, and the regression relationship between the feature data and the predicted target y is plotted. The following is an example: the left side is a linear regression relationship between the UV index and the power generation, and the right side is a regression relationship between hours and power generation. From the black oblique line obtained by regression fitting on the left side of the figure, it can be seen that the slope of the straight line of the regression fitting is large, and there is an obvious linear positive correlation between the two, so this feature data is valid and needs to be retained. From the figure on the right, it can be seen that although the slope of the linear regression fitting between hours (hours in day) and power generation is very small, the two have an obvious parabolic relationship, so it is also valid feature data and needs to be retained.

[0068] Combined with the distplot function in seaborn, we can draw the histogram and kernel density estimation diagram of a single feature data to visualize the data distribution of the feature data and help understand the data. If the distribution of the data is very skewed or there are outliers, this may mean that the data needs to be preprocessed or feature transformed. Figure 3 The histogram and kernel density estimation diagram of feature data in a multi-model fusion photovoltaic power anomaly detection method provided by an embodiment of the present invention are as follows: Figure 3 As shown in the figure, for example, the UV index can be seen from the histogram that the UV index is an integer value of discrete distribution. The UV index in the sample data is below 10, and the data granularity is not fine enough, so it is not enough to use only feature data for prediction. More feature data needs to be constructed to improve the accuracy of prediction. The kernel density curve shows that the UV index is concentrated in the range of 0-4, and the distribution volume above 4 decreases sharply, indicating that when the UV index is low, the sample data volume is sufficient, but the discrimination is not enough. The UV index may be 2 or 3 most of the time during the day, but the power generation fluctuates greatly. When the UV index is high and 5, there will be insufficient samples, and there are not many dates for extreme weather, so the training is difficult and requires the help of other feature data. Therefore, other feature data are constructed, such as temperature, precipitation intensity, the number of time points after sunrise, the number of time points before sunset, and month.

[0069] According to the Pearson correlation coefficient between the feature data and the predicted target, the feature data with a larger absolute value of the correlation coefficient with the predicted target (i.e., the photovoltaic power generation in the embodiment of the present application) (i.e., the absolute value of the correlation coefficient in the embodiment of the present application is greater than the third threshold) is selected. The correlation heat map of some feature data and the predicted target, such as Figure 4 As shown, Figure 4 A heat map of the correlation between feature data and predicted targets in a multi-model fusion photovoltaic power anomaly detection method provided by an embodiment of the present invention. The heatmap function in seaborn is used to draw the heat map. From the first column of the figure, it can be seen that the correlation coefficient between the ultraviolet index and the predicted target y is >0.7, which is a strong positive correlation, and the correlation coefficient between humidity and the predicted target y is <-0.5, which is a medium negative correlation. The correlation coefficient between temperature and predicted target y is >0.4, which is a medium positive correlation. All of them are valid feature data that need to be retained.

[0070] Step S104, training and optimizing the LightGBM model, K nearest neighbor regression model and random forest model respectively according to the feature data of the training set and the validation set to obtain optimized LightGBM model, K nearest neighbor regression model and random forest model;

[0071] Optionally, in step S104, the LightGBM model, K nearest neighbor regression model and random forest model are trained and optimized respectively according to the feature data of the training set and the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model, including: training the LightGBM model, K nearest neighbor regression model and random forest model respectively according to the feature data of the training set, and performing hyperparameter tuning and feature selection on the trained LightGBM model, K nearest neighbor regression model and random forest model through the feature data of the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model.

[0072] Furthermore, optionally, hyperparameter tuning and feature selection are performed on the trained LightGBM model, K nearest neighbor regression model and random forest model through the feature data of the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model, including: evaluating the trained LightGBM model, K nearest neighbor regression model and random forest model respectively through the feature data of the validation set to obtain the evaluation results of the LightGBM model, K nearest neighbor regression model and random forest model; selecting hyperparameters from the evaluation results of the LightGBM model, K nearest neighbor regression model and random forest model through grid search GridSearch; screening hyperparameters according to the root mean square error (RMSE), an accuracy evaluation indicator on the validation set, to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model.

[0073] Specifically, a LightGBM model, a K-nearest neighbor regression model, and a random forest model were constructed and trained on the training set respectively. Then, based on the evaluation results on the validation set, hyperparameter tuning and feature selection of the three models were performed.

[0074] Hyperparameter tuning, using grid search GridSearch to select hyperparameters:

[0075] The LightGBM model contains general parameters, auxiliary parameters, and task parameters. The best parameter settings are determined through grid search. The main parameters are: learning rate, maximum depth of the tree, number of leaf nodes, number of iterations, number of early stopping rounds, row / column downsampling ratio, etc.

[0076] The parameters of K-nearest neighbor regression are mainly the number of neighbors K and the measurement method of the distance between data points.

[0077] The parameters of random forest mainly include the number of decision trees, the maximum depth of the tree, the evaluation criteria for the quality of the splitting points, the minimum number of samples required for node splitting, the minimum number of samples required for leaf nodes, etc.

[0078] Feature importance and screening: Based on the accuracy evaluation index rmse root mean square error on the validation set, the optimal number of features to retain is determined to avoid underfitting and overfitting. The importance of the main features of the head is ranked, such as Figure 5 As shown, Figure 5 A schematic diagram of feature importance sorting in a multi-model fusion photovoltaic power anomaly detection method provided by an embodiment of the present invention. The bar chart shows the importance of the features, and the features are sorted from large to small according to their importance, and the first 200, 150, 100, 50... features are retained respectively. The number of features to be retained is determined based on minimizing the RMSE in the validation set.

[0079] Step S106, input the test set into the optimized LightGBM model, K nearest neighbor regression model and random forest model respectively, and obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model;

[0080] Specifically, the LightGBM model, K nearest neighbor regression model and random forest model trained and optimized in step S104 are applied to the test set respectively to obtain the prediction results on the test set to simulate the actual application scenario.

[0081] In the embodiment of the present application, three algorithms, Lightgbm, K nearest neighbor regression model, and random forest are used for prediction as follows:

[0082] LightGBM (Light Gradient Boosting Machine) is a framework for implementing the GBDT algorithm. The Boosting used in GBDT is a type of ensemble learning. Ensemble learning is a meta-algorithm that combines several machine learning techniques into a prediction model to reduce variance, bias, or improve prediction. The main idea is to assemble weak classifiers into a strong classifier.

[0083] LightGBM supports highly efficient parallel training and has the advantages of faster training speed, lower memory consumption, better accuracy, support for distributed processing and rapid processing of massive data.

[0084] LightGBM will build multiple decision trees, one tree at a time. The construction of the next tree is based on the residual of the previous tree as the prediction target, and the final prediction value = the sum of the prediction values ​​of each tree.

[0085] The negative gradient of the loss function is used as the residual approximation of the current decision tree to fit the new decision tree. Using Boosting ensemble learning, the training sets of each weak classifier are not independent. The training set of each weak classifier is the learning result of the previous weak classifier, and sampling is performed according to the error rate (giving different weights to the training samples).

[0086] The LightGBM algorithm mainly includes the following features: histogram algorithm (finding the best split point, histogram difference acceleration), Leaf-wise tree growth strategy (reducing a lot of calculations and avoiding overfitting), GOSS one-side gradient sampling technology (Gradient-based One-Side Sampling), EFB mutually exclusive feature bundling (reducing feature angles), which can solve the sparsity problem of high-dimensional data, support category features, efficient parallelism, and cache hit rate optimization. In general, LightGBM is a highly optimized GBDT algorithm, such as Figure 6 As shown, Figure 6 A schematic diagram of Boosting of the LightGBM algorithm in a multi-model fusion photovoltaic power anomaly detection method provided in an embodiment of the present invention.

[0087] The K nearest neighbor regression model uses the average value of the K training sample points closest to the sample point to be predicted for the regression prediction of the sample point to be predicted. In addition to classification analysis, K nearest neighbors can also perform regression analysis. In this case, KNN is called the K nearest neighbor regression model. The commonly used distance indicator in the KNN algorithm is the Euclidean distance. In the case of n-dimensional features, each data point can be regarded as in an n-dimensional Euclidean space, and each point X can be expressed as (x[1]x[2]…x[n]), where x(i=12…n) is called the i-th coordinate of X. The Euclidean distance between two points A=(a[1]a[2]…a[n]) and B=(b[1]b[2]…b[n]) is d(AB)=sqrt[∑((ab)^2)](i=12…n). The model also needs to normalize the features of the K nearest neighbor regression to their maximum and minimum values, add weights according to the importance of the features, and find the optimal parameter values, such as the optimal value of K, through a grid search algorithm. Figure 7 As shown, Figure 7 A schematic diagram of a K-nearest neighbor algorithm in a multi-model fusion photovoltaic power anomaly detection method provided in an embodiment of the present invention.

[0088] Random forest is an ensemble learning algorithm consisting of multiple decision trees, each of which is trained independently.

[0089] The random forest algorithm improves the generalization ability of the model by introducing two types of randomness: Bootstrap sampling and random feature selection.

[0090] Bootstrap sampling ensures that the training set of each decision tree has a certain degree of randomness. Randomly selecting features makes the segmentation features of each decision tree have a certain degree of difference, which can reduce the risk of overfitting and improve the robustness and accuracy of the model. On the basis of building a bagging ensemble based on the decision tree as the basic learner, random feature selection is further introduced in the training process of the decision tree. The process is roughly divided into four steps: randomly selecting samples (with replacement sampling), randomly selecting feature attributes, building decision trees, and random forest voting or averaging. Therefore, the ability to prevent overfitting is stronger and the variance is reduced. Figure 8 As shown, Figure 8 A bagging schematic diagram of a random forest algorithm in a multi-model fusion photovoltaic power anomaly detection method provided in an embodiment of the present invention.

[0091] Step S108, inputting the prediction results of the LightGBM model, the prediction results of the K nearest neighbor regression model and the prediction results of the random forest model into the Ridge Regression Model for model training to obtain a trained Ridge Regression Model;

[0092] Specifically, the three predicted values ​​of the LightGBM model, the K nearest neighbor regression model, and the random forest model (i.e., the prediction results of the LightGBM model, the K nearest neighbor regression model, and the random forest model in the embodiment of the present application) are used as three features (three columns of independent variables), input into the Ridge Regression Ridge model, fit the dependent variable (the true value of the generated power), and train to obtain a Ridge Regression model through training (equivalent to automatically assigning weights to the three feature columns);

[0093] Stacking fusion is used to combine the results of the LightGBM model, K-nearest neighbor regression model, and random forest model. Although the weighted method is simple to use for fusion, it requires manual determination of weights. Stacking fusion is more intelligent and has better results. Stacking uses a meta-model to integrate the prediction results of the basic models, which can usually achieve better prediction performance than simple weighted average. By fusing three models with large differences, we can learn from each other's strengths and weaknesses, combine the advantages of individual learners, and reduce prediction errors. Examples of results based on the test set, such as Fig. 9 As shown, Fig. 9 A schematic diagram of the comparison between a user's prediction and the actual result in a multi-model fusion photovoltaic power anomaly detection method provided in an embodiment of the present invention. Fig. 9 The upper middle figure shows the actual power generation, and the lower figure shows the predicted power generation.

[0094] Specifically, the Ridge Regression model in the Stacking method is used. Ridge regression is a lightweight and very effective algorithm. It has strong stability. Ridge regression controls the complexity of the model by introducing a regularization term (L2 norm), which helps prevent overfitting and improve the generalization ability of the model. Ridge regression can effectively deal with the problem of collinearity (high correlation between features) and improve the stability and reliability of the model. In addition, the parameter adjustment is simple. By adjusting the regularization parameter (λ), the complexity of the model and the bias-variance balance can be flexibly controlled, making the ridge regression stable on different data sets. The Ridge model has a wide range of applicability and does not rely on the assumption of a linear relationship between models. Therefore, it can perform well when dealing with large model heterogeneity.

[0095] Step S110, input the test set into the trained Ridge Regression model to obtain the model fusion prediction value;

[0096] Specifically, the trained ridge regression model is applied to the test set to obtain the fused prediction value y^ on the test set. This is more accurate than directly averaging the three. It is more applicable than linear regression (least squares regression) because it has lower requirements on input data and does not require independent variables to have no multicollinearity. Fig.10 As shown, Fig.10 A ridge regression model path diagram of a multi-model fusion photovoltaic power anomaly detection method provided in an embodiment of the present invention.

[0097] The root mean square error (RMSE) is used as an indicator to evaluate the degree of fit. The model accuracy is evaluated on the validation set to select the optimal hyperparameters.

[0098] Root Mean Square Error (RMSE) is the square root of the sum of the squares of the differences between the predicted value and the true value. Where m is the number of samples, yi is the photovoltaic power data, yi^ is the predicted power data, and the smaller the RMSE, the better.

[0099] RMSE is a summary statistic of all data on the validation set (all users and all dates) and is used for model tuning, including: tuning of model hyperparameters and determination of the number of features. Hyperparameter adjustment: If the RMSE on the validation set is much larger than that on the training set, it means that the model is overfitting and the hyperparameters need to be adjusted, such as reducing the complexity of the model, such as reducing the depth of the tree, reducing the number of trees, etc., so as to achieve an effect close to the training set. The number of training rounds is also selected based on the degree of RMSE decline. If the RMSE decline begins to flatten or rise after multiple rounds of training, it means that training can be stopped, otherwise it will waste time and lead to overfitting.

[0100] Step S112, performing abnormal diagnosis based on the model fusion prediction value and the test set to obtain an abnormal diagnosis result.

[0101] Optionally, in step S112, anomaly diagnosis is performed based on the model fusion prediction value and the test set, and the anomaly diagnosis result obtained includes: calculating the absolute difference and the difference ratio based on the model fusion prediction value and the test set respectively; if the absolute difference is greater than the fourth threshold, and the difference ratio is greater than the fifth threshold, the anomaly diagnosis result is determined to be an anomaly at the user time point, and the abnormal date and user are output.

[0102] Specifically, the data in the test set will include two columns: the actual value y of photovoltaic power generation and the predicted value y^ of photovoltaic power generation predicted by the model. The absolute difference abs(y^-y) (the absolute value of the difference between the actual value and the predicted value of photovoltaic power generation for each user every day) and the difference ratio abs(y^-y) / y (the absolute value of the difference between the actual value and the predicted value of photovoltaic power generation for each user every day / the actual value of photovoltaic power generation) are calculated respectively. If both are greater than their respective set thresholds (that is, in the embodiment of the present application, if the absolute difference is greater than the fourth threshold and the difference ratio is greater than the fifth threshold), it is judged that the user's time point is abnormal, and the abnormal date and user are finally output.

[0103] The photovoltaic power generation power anomaly detection method with multi-model fusion provided in the embodiment of the present application takes into account the large differences in power generation of users at different time points, the fine prediction granularity, and the influence of multiple factors. It constructs a multi-class feature-based, three machine learning algorithms, namely the LightGBM model, the K nearest neighbor regression model, and the random forest model, and uses the ridge regression Ridge for model fusion. Combining the advantages of the model based on distance metric and the model selected by feature, the accuracy and generalization ability of the algorithm are improved, and it has good practical value. The photovoltaic power generation power anomaly detection method with multi-model fusion provided in the embodiment of the present application can diagnose anomalies with higher accuracy than traditional algorithms, and the algorithm is more robust, which provides a new idea for exploring the problem of photovoltaic power anomaly diagnosis.

[0104] The embodiment of the present invention provides a detection method for photovoltaic power anomaly by multi-model fusion, which divides the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date; performs data cleaning on the training set and the validation set, and constructs feature data for the training set and the validation set after data cleaning; trains and optimizes the LightGBM model, the K nearest neighbor regression model and the random forest model according to the feature data of the training set and the validation set, respectively, to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; inputs the test set into the optimized LightGBM model, the K nearest neighbor regression model and the random forest model, respectively, to obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model; inputs the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model into the ridge regression model for model training, to obtain the trained ridge regression model; inputs the test set into the trained ridge regression model to obtain the model fusion prediction value; performs abnormal diagnosis according to the model fusion prediction value and the test set, to obtain the abnormal diagnosis result, thereby improving the technical effect of the accuracy of diagnosis and generalization ability.

[0105] The embodiment of the present invention provides a photovoltaic power generation power anomaly detection device based on multi-model fusion, such as Fig.11 As shown, Fig.11A schematic diagram of a multi-model fusion photovoltaic power anomaly detection device provided in an embodiment of the present invention. The multi-model fusion photovoltaic power anomaly detection device in the embodiment of the present application includes: a division module 1100, used to divide the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date; a cleaning module 1102, used to perform data cleaning on the training set and the validation set, and construct feature data for the training set and the validation set after data cleaning; a first training module 1104, used to train and optimize the LightGBM model, the K nearest neighbor regression model and the random forest model according to the feature data of the training set and the validation set, respectively, to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; a first prediction module 1106, used to input the test set into the optimized LightGBM model, K nearest neighbor regression model and random forest model respectively, to obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model; the second training module 1108, used to input the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model into the Ridge Regression Model for model training, to obtain the trained Ridge Regression Model; the second prediction module 1110, used to input the test set into the trained Ridge Regression Model, to obtain the model fusion prediction value; the anomaly detection module 1112, used to perform anomaly diagnosis based on the model fusion prediction value and the test set, to obtain the anomaly diagnosis result.

[0106] Optionally, the multi-model fusion photovoltaic power anomaly detection device in the embodiment of the present application also includes: an acquisition module, used to acquire power generation data, photovoltaic archive data and meteorological data within a preset time period before the diagnosis day and the diagnosis day before dividing the acquired photovoltaic data and meteorological data into a training set, a verification set and a test set according to the date; a data acquisition module, used to obtain photovoltaic data based on the power generation data and photovoltaic archive data; a construction module, used to obtain a database based on the photovoltaic data and meteorological data.

[0107] The embodiment of the present invention provides a detection device for photovoltaic power anomaly by multi-model fusion, which divides the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date; performs data cleaning on the training set and the validation set, and constructs feature data for the training set and the validation set after data cleaning; trains and optimizes a LightGBM model, a K nearest neighbor regression model and a random forest model according to the feature data of the training set and the validation set, respectively, to obtain optimized LightGBM model, K nearest neighbor regression model and random forest model; inputs the test set into the optimized LightGBM model, K nearest neighbor regression model and random forest model, respectively, to obtain LightGBM model prediction results, K nearest neighbor regression model prediction results and random forest model prediction results; inputs the LightGBM model prediction results, K nearest neighbor regression model prediction results and random forest model prediction results into a ridge regression model for model training, to obtain a trained ridge regression model; inputs the test set into the trained ridge regression model to obtain a model fusion prediction value; performs abnormal diagnosis according to the model fusion prediction value and the test set, to obtain an abnormal diagnosis result, thereby improving the technical effect of the accuracy of diagnosis and generalization ability.

[0108] The above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention.

Claims

1. A multi-model fusion photovoltaic power generation power anomaly detection method, characterized in that: include: The acquired photovoltaic data and meteorological data are divided into training set, validation set and test set according to the date; Performing data cleaning on the training set and the validation set, and constructing feature data for the training set and the validation set after data cleaning; According to the characteristic data of the training set and the validation set, the LightGBM model, the K nearest neighbor regression model and the random forest model are trained and optimized respectively to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; The test set is respectively input into the optimized LightGBM model, the K nearest neighbor regression model and the random forest model to obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model; Input the prediction results of the LightGBM model, the prediction results of the K nearest neighbor regression model and the prediction results of the random forest model into the Ridge Regression model for model training to obtain the trained Ridge Regression model; Inputting the test set into the trained Ridge Regression model to obtain a model fusion prediction value; Anomaly diagnosis is performed based on the model fusion prediction value and the test set to obtain an abnormality diagnosis result.

2. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 1 is characterized in that: Before dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date, the method further includes: Obtaining power generation data, photovoltaic archive data and meteorological data on the diagnosis day and a preset time period before the diagnosis day; Obtaining the photovoltaic data according to the power generation data and the photovoltaic archive data; A database is obtained according to the photovoltaic data and the meteorological data.

3. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 1 or 2, characterized in that: The step of dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date includes: The photovoltaic data and the meteorological data are divided according to a preset ratio based on the date to obtain the training set, the validation set and the test set.

4. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 1 is characterized in that: The data cleaning of the training set and the validation set includes: Cleaning the historical data in the training set and the verification set to obtain the training set and the verification set without abnormal historical data; According to the earliest non-zero power time and the latest non-zero power time combined with the earliest sunrise time and sunset time of each day, the data of the photovoltaic non-power generation period at night are cleared to obtain the training set and the verification set after data cleaning.

5. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 4 is characterized in that: The constructing feature data of the training set and the validation set after data cleaning includes: Performing feature aggregation, feature crossover and feature transformation on the training set and the validation set after data cleaning to obtain feature data; The feature data whose missing value ratio is greater than a first threshold or whose maximum attribute ratio is greater than a second threshold are removed to obtain the feature data after removal; According to the Pearson correlation coefficient between the removed characteristic data and the photovoltaic power generation power, characteristic data whose absolute value of the correlation coefficient with the photovoltaic power generation power is greater than a third threshold value is screened to obtain the screened characteristic data.

6. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 5 is characterized in that: The LightGBM model, the K nearest neighbor regression model and the random forest model are trained and optimized respectively according to the feature data of the training set and the validation set, and the optimized LightGBM model, the K nearest neighbor regression model and the random forest model are obtained, including: The LightGBM model, K nearest neighbor regression model and random forest model are trained respectively according to the feature data of the training set, and hyperparameter tuning and feature selection are performed on the trained LightGBM model, K nearest neighbor regression model and random forest model through the feature data of the validation set to obtain the optimized LightGBM model, K nearest neighbor regression model and random forest model.

7. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 6 is characterized in that: The hyperparameter tuning and feature selection of the trained LightGBM model, the K nearest neighbor regression model and the random forest model are performed by using the feature data of the validation set to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model, including: The trained LightGBM model, the K nearest neighbor regression model and the random forest model are evaluated respectively by the feature data of the validation set to obtain evaluation results of the LightGBM model, the K nearest neighbor regression model and the random forest model; Select hyperparameters from the evaluation results of the LightGBM model, the K nearest neighbor regression model, and the random forest model through grid search GridSearch; The hyperparameters are screened according to the accuracy evaluation index rmse root mean square error on the validation set to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model.

8. The photovoltaic power generation power anomaly detection method based on multi-model fusion according to claim 1 is characterized in that: The abnormality diagnosis is performed based on the model fusion prediction value and the test set to obtain the abnormality diagnosis result, which includes: Calculating the absolute difference and the difference ratio respectively according to the model fusion prediction value and the test set; If the absolute difference is greater than a fourth threshold value, and the difference ratio is greater than a fifth threshold value, the abnormality diagnosis result is determined as an abnormality at the user time point, and the abnormal date and user are output.

9. A multi-model fusion photovoltaic power anomaly detection device, characterized in that: include: A partitioning module is used to partition the acquired photovoltaic data and meteorological data into a training set, a validation set, and a test set according to the date; A cleaning module, used for performing data cleaning on the training set and the verification set, and constructing feature data for the training set and the verification set after data cleaning; The first training module is used to train and optimize the LightGBM model, the K nearest neighbor regression model and the random forest model according to the feature data of the training set and the validation set, respectively, to obtain the optimized LightGBM model, the K nearest neighbor regression model and the random forest model; The first prediction module is used to input the test set into the optimized LightGBM model, the K nearest neighbor regression model and the random forest model respectively to obtain the prediction results of the LightGBM model, the K nearest neighbor regression model and the random forest model; The second training module is used to input the prediction results of the LightGBM model, the prediction results of the K nearest neighbor regression model and the prediction results of the random forest model into the Ridge Regression Ridge model for model training to obtain the trained Ridge Regression Ridge model; A second prediction module is used to input the test set into the trained Ridge Regression model to obtain a model fusion prediction value; The anomaly detection module is used to perform anomaly diagnosis based on the model fusion prediction value and the test set to obtain anomaly diagnosis results.

10. The photovoltaic power generation abnormality detection device based on multi-model fusion according to claim 9 is characterized in that: The device also includes: An acquisition module, used for acquiring power generation data, photovoltaic archive data and meteorological data within a preset time period before the diagnosis date and the diagnosis date before dividing the acquired photovoltaic data and meteorological data into a training set, a validation set and a test set according to the date; A data acquisition module, used for obtaining the photovoltaic data according to the power generation data and the photovoltaic archive data; A construction module is used to obtain a database based on the photovoltaic data and the meteorological data.

Images