Photovoltaic power prediction method and device based on meta learning and attention mechanism

Abstract

The invention discloses a photovoltaic power prediction method and device based on meta learning and an attention mechanism. The method comprises the steps that an MAML-Transform model based on meta learning and an attention mechanism is constructed and trained in advance, input data of the MAML-Transform model is photovoltaic power generation related data, and output of the MAML-Transform model is a power generation power prediction value; photovoltaic power generation related data collected in real time are obtained, and the photovoltaic power generation related data are used as input data to be input into the MAML-Transform model; and obtaining a power generation power prediction value output by the MAML-Transform model. According to the method, the prediction task can be realized based on a small amount of photovoltaic data, the training data demand of the prediction model is reduced, the accuracy and robustness of photovoltaic power generation power prediction can be remarkably improved through a self-attention mechanism, and the practical application value of the model is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of photovoltaic prediction, and specifically relates to a photovoltaic power prediction method and device based on meta-learning and attention mechanism. Background Art

[0002] As the global demand for renewable energy increases, the importance of photovoltaic power generation has become increasingly prominent. However, since photovoltaic power generation is affected by many factors such as weather and environment, its power generation has significant volatility and uncertainty.

[0003] When using traditional statistical models and deep learning models for prediction, a large amount of training data is required. However, since photovoltaic power generation is affected by many factors such as weather and environment, its power generation has significant volatility and uncertainty. There is a situation where the amount of data for model training is insufficient, which leads to the inability of the trained model to make accurate predictions. How to improve the accuracy of model predictions has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the present invention is to provide a photovoltaic power prediction method and device based on meta-learning and attention mechanism, which can improve the accuracy of photovoltaic power prediction under small sample conditions.

[0005] According to a first aspect of the present invention, a photovoltaic power prediction method based on meta-learning and attention mechanism is provided, comprising:

[0006] Pre-building and training a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation power prediction value;

[0007] Acquire photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data into the MAML-Transformer model as input data;

[0008] Obtain the power generation prediction value output by the MAML-Transformer model.

[0009] According to a second aspect of the present invention, a photovoltaic power prediction device based on meta-learning and attention mechanism is provided, comprising:

[0010] A model training module, used to pre-build and train a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation power prediction value;

[0011] A model application module, used to obtain photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data into the MAML-Transformer model as input data;

[0012] The output module is used to obtain the predicted value of the generated power output by the MAML-Transformer model.

[0013] According to a third aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the photovoltaic power prediction method based on meta-learning and attention mechanism described in any embodiment.

[0017] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the photovoltaic power prediction method based on meta-learning and attention mechanism described in any embodiment of the present invention when executed.

[0018] The technical solution provided by the present invention pre-builds and trains a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation prediction value; real-time collected photovoltaic power generation related data is obtained, and the photovoltaic power generation related data is input into the MAML-Transformer model as input data; and the power generation prediction value output by the MAML-Transformer model is obtained. Compared with traditional statistical models and deep learning models that require a large amount of data for training to obtain accurate prediction models, the technical solution provided by the present invention combines the Transformer model of the attention mechanism with meta-learning to obtain a MAML-Transformer model. The meta-learning framework can realize the training of the Transformer model in a small sample scenario. The Transformer model with the self-attention mechanism can effectively capture the long-range dependencies in the time series data, thereby improving the accuracy of the prediction. In addition, the trained model can be deployed to an actual photovoltaic power prediction system to receive input data in real time and visualize the output prediction results. By combining meta-learning MAML and Transformer models, the present invention can not only realize prediction tasks based on a small amount of photovoltaic data, reducing the training data requirements of the prediction model, but also significantly improve the accuracy and robustness of photovoltaic power prediction through the self-attention mechanism, thereby improving the practical application value of the model.

[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 It is a flow chart of a photovoltaic power prediction method based on meta-learning and attention mechanism provided by an embodiment of the present invention;

[0022] Figure 2 It is a structural diagram of the MAML-Transformer model provided by an embodiment of the present invention;

[0023] Figure 3It is a structural schematic diagram of a photovoltaic power prediction device based on meta-learning and attention mechanism provided by an embodiment of the present invention;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] As the global demand for renewable energy increases, the importance of photovoltaic power generation has become increasingly prominent. However, since photovoltaic power generation is affected by various factors such as weather and environment, its power generation has significant volatility and uncertainty. When using traditional statistical models and deep learning models for prediction, a large amount of training data is required. However, since photovoltaic power generation is affected by various factors such as weather and environment, its power generation has significant volatility and uncertainty, there is a situation where the amount of data for model training is insufficient, which leads to the inability of the trained model to make accurate predictions. How to improve the accuracy of model predictions in small sample scenarios has become an urgent problem to be solved. In the embodiment of the present invention, the small sample scenario refers to a situation where the number of samples is small compared to the number of samples required for training conventional models such as deep learning models.

[0028] Figure 1This is a flow chart of a photovoltaic power prediction method based on meta-learning and attention mechanism provided by an embodiment of the present invention. This embodiment is applicable to the case where the power of photovoltaic equipment is predicted in a small sample scenario. The method can be performed by a photovoltaic power prediction device based on meta-learning and attention mechanism. The photovoltaic power prediction device based on meta-learning and attention mechanism can be implemented in the form of hardware and / or software. The method includes:

[0029] S101. Pre-build and train the MAML-Transformer model based on meta-learning and attention mechanism.

[0030] The input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is the predicted value of power generation.

[0031] Optionally, a pre-built and trained MAML-Transformer model based on meta-learning and attention mechanism can be implemented as follows:

[0032] S11. Obtain a historical characteristic data set of photovoltaic power generation, and perform preprocessing according to the historical characteristic data set.

[0033] The historical feature data set includes relevant features such as irradiance, ambient temperature, ambient humidity, wind speed, etc. The historical feature data set contains data of the above different dimensions.

[0034] Exemplarily, preprocessing is performed based on the historical feature data set, including: filling in default values, normalizing, and / or screening abnormal data based on the historical feature data set.

[0035] The above preprocessing methods may be selected one by one, or may be performed sequentially.

[0036] Specifically, default value filling based on historical feature data sets can be implemented as follows:

[0037] For the missing feature variable data in the historical feature data set, the median of the corresponding features of the photovoltaic station is calculated, and the missing feature variables are filled according to the median.

[0038] The median can be calculated from the historical feature data of each dimension, and the actual feature variables can be filled in according to the median, thereby increasing the number of samples in small sample scenarios.

[0039] Normalization is performed based on the historical feature data set, which can be implemented as follows:

[0040] Eliminate the influence of the variable dimension in the historical feature data set, use the standard score method to set the mean value of the feature variable to 0 and the standard deviation to 1. The calculation formula is:

[0041]

[0042] Among them, X is the feature data value, μ is the feature data mean, and σ is the feature data standard deviation.

[0043] Through the above normalization processing, the historical feature data set can be normalized to eliminate the influence of the variable dimension in the historical feature data set.

[0044] Abnormal data screening based on historical feature data sets can be implemented as follows:

[0045] According to the 3σ principle of data processing, abnormal data distributed outside (μ-3σ,μ+3σ) are screened and cleaned.

[0046] Through the above screening process, abnormal data outside the 3σ numerical range can be deleted.

[0047] The above preprocessing method can increase the number of samples, filter out abnormal data, normalize the data, and improve the quality of sample data.

[0048] S12. Construct a Transformer model based on the attention mechanism, wherein the Transformer model includes an input embedding layer, a multi-head self-attention layer, and a feedforward neural network layer.

[0049] First, the input embedding layer is designed. The input embedding layer embeds the historical feature data of photovoltaic power generation into a high-dimensional space and retains the position information of the input sequence through position encoding. The specific steps are as follows:

[0050] Input embedding: Map the original feature data to a high-dimensional space through an embedding matrix. The original feature data is the dimensional data in the preprocessed historical feature data set. It includes relevant features such as irradiance, ambient temperature, ambient humidity, and wind speed. Assume that the input sequence is X = [x1, x2, ..., x n ], through the embedding matrix W e Get the embedding vector:

[0051] E=XW e

[0052] Position encoding: In order to preserve the sequence information, position encoding is added. The sequence information can be the sequence of input information in time. Position encoding is used to preserve the position information of each data point in the input sequence. The position encoding matrix P is added to the embedding vector to obtain the input vector Z: Z = E + P

[0053] In the formula, the calculation formula of position coding is:

[0054]

[0055] In the formula, pos is the position, i is the dimension index, and dmodel is the dimension of the embedding vector. pos represents the position index of the data point in the input sequence, that is, the position of the data in the entire sequence. It is usually represented by an integer, such as the 1st, 2nd, or 3rd data point.

[0056] Secondly, design a multi-head self-attention layer. The multi-head self-attention layer introduces multiple attention heads, allowing the model to focus on information at different positions in parallel and improve the model's expressiveness. Self-attention calculation calculates the correlation between positions in the input sequence, allowing the model to capture long-range dependencies. The specific steps are as follows:

[0057] Linear transformation: Map the input vector to query, key, and value spaces respectively:

[0058] Q=ZW Q

[0059] K=ZW K

[0060] V=ZW V

[0061] Where WQ, WK, and WV are the weight matrices of query, key, and value, respectively.

[0062] Self-attention calculation: Calculate the attention weights and apply them to the values:

[0063]

[0064] Where Q, K and V are the matrices of query, key and value respectively. is the scaling factor, and the softmax function is used to normalize the weights.

[0065] Among them, the multi-head mechanism refers to extending the self-attention mechanism to multiple heads, calculating multiple self-attentions in parallel, and splicing the results:

[0066] MultiHead(Q,K,V)=[head1,head2,...,head h ]W O

[0067] In the formula, h is the number of heads, is the weight matrix of each head, and WO is the weight matrix of the output.

[0068] Finally, the feedforward neural network is designed. The feedforward neural network layer further processes the output of the self-attention layer through a series of linear transformations and activation functions, and performs normalization after each feedforward neural network layer to stabilize the model training process and improve the generalization ability of the model. The specific steps are as follows:

[0069] Linear transformation and activation function: Apply linear transformation and nonlinear activation function (ReLU):

[0070] FFN(x)=max(0,xW1+b1)W2+b2

[0071] Where W1, W2 are weight matrices, b1, b2 are bias vectors.

[0072] Layer Normalization: Layer normalization is performed after each feed-forward neural network to stabilize the training process:

[0073]

[0074] Where μ and σ are the mean and standard deviation, respectively, and ε is a small constant to prevent division by zero.

[0075] S13. Embed the Transformer model into a meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism.

[0076] Optionally, the Transformer model is embedded in a meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism, which can be implemented in the following ways:

[0077] S31, embedding the Transformer model into the MAML framework to obtain a MAML-Transformer model based on meta-learning and attention mechanism. The goal of the MAML framework is to find a set of model parameters so that the MAML-Transformer model can quickly adapt to new tasks after a small number of update steps.

[0078] The optimization goal of the MAML-Transformer model is:

[0079]

[0080] Among them, θ is the model parameter, α is the learning rate, is the loss function; the initial model parameter θ is trained on multiple related tasks so that it can quickly adapt to new tasks;

[0081] S32, perform inner loop update of Transformer model, including: for each task Ti , perform several steps of gradient descent update based on the current model parameters θ and calculate the task loss

[0082] When the inner loop of the Transformer model is updated, the updated parameters are:

[0083]

[0084] In the formula, θ is the model parameter, α is the learning rate, is the loss function;

[0085] S33. Perform an outer loop update of the MAML-Transformer model, including: updating the initial model parameters θ by aggregating the verification losses of all tasks during meta-level training.

[0086] When the outer loop of the MAML-Transformer model is updated, the update formula is:

[0087]

[0088] Where β is the meta-learning rate.

[0089] The MAML-Transformer model makes it easier for model training to converge, so that less data is needed to achieve the prediction effect. The MAML-Transformer model contains two loops, an inner loop and an outer loop. The inner loop process is trained by inputting n different tasks to obtain n updated weights. The outer loop calculates the total loss of these n updated weights and updates the model through gradient descent, making the model more adaptable to a variety of tasks, that is, to new tasks, that is, it can be quickly fitted with a small amount of new data.

[0090] S14. Perform multi-task sampling based on the preprocessed data to obtain multiple groups of training data.

[0091] Optionally, multi-task sampling is performed based on the preprocessed data to obtain multiple sets of training data, which can be implemented in the following way:

[0092] Multiple tasks are randomly sampled from the preprocessed data, each task corresponding to photovoltaic power generation related data in different time periods or at different locations; the training data obtained for each task is divided into a training set and a validation set.

[0093] S15. Input the multiple groups of training data into the MAML-Transformer model for training.

[0094] S102, obtaining photovoltaic power generation related data collected in real time, and inputting the photovoltaic power generation related data into the MAML-Transformer model as input data.

[0095] S103. Obtain the predicted value of generated power output by the MAML-Transformer model.

[0096] Optionally, the predicted power generation power forecast value can be output to a display.

[0097] The technical solution provided by the present invention pre-builds and trains a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation prediction value; real-time collected photovoltaic power generation related data is obtained, and the photovoltaic power generation related data is input into the MAML-Transformer model as input data; and the power generation prediction value output by the MAML-Transformer model is obtained. Compared with traditional statistical models and deep learning models that require a large amount of data for training to obtain accurate prediction models, the technical solution provided by the present invention combines the Transformer model of the attention mechanism with meta-learning to obtain a MAML-Transformer model. The meta-learning framework can realize the training of the Transformer model in a small sample scenario. The Transformer model with the self-attention mechanism can effectively capture the long-range dependencies in the time series data, thereby improving the accuracy of the prediction. In addition, the trained model can be deployed to an actual photovoltaic power prediction system to receive input data in real time and visualize the output prediction results. By combining meta-learning MAML and Transformer models, the present invention can not only realize prediction tasks based on a small amount of photovoltaic data, reducing the training data requirements of the prediction model, but also significantly improve the accuracy and robustness of photovoltaic power prediction through the self-attention mechanism, thereby improving the practical application value of the model.

[0098] The embodiment of the present invention further provides a photovoltaic power prediction method based on meta-learning and attention mechanism, which is characterized by comprising:

[0099] Obtain a historical characteristic dataset of photovoltaic power generation.

[0100] For the missing characteristic variable data in the historical characteristic data set, the median of the corresponding characteristics of the photovoltaic station is calculated, and the missing characteristic variables are filled according to the median;

[0101] Eliminate the influence of the variable dimension in the historical feature data set, use the standard score method to set the mean value of the feature variable to 0 and the standard deviation to 1. The calculation formula is:

[0102]

[0103] Among them, X is the characteristic data value, μ is the mean of the characteristic data, and σ is the standard deviation of the characteristic data;

[0104] According to the 3σ principle of data processing, abnormal data distributed outside (μ-3σ,μ+3σ) are screened and cleaned.

[0105] Construct a Transformer model based on the attention mechanism, which includes an input embedding layer, a multi-head self-attention layer, and a feedforward neural network layer, including:

[0106] Input embedding layer design. The input embedding layer embeds the historical feature data of photovoltaic power generation into a high-dimensional space and retains the position information of the input sequence through position encoding. The specific steps are as follows:

[0107] Input embedding: Map the original feature data to a high-dimensional space through an embedding matrix. Assume that the input sequence is X = [x1, x2, ..., x n ], through the embedding matrix W e Get the embedding vector:

[0108] E=XW e

[0109] Position encoding: In order to preserve the sequence information, position encoding is added. The position encoding matrix P is added to the embedding vector:

[0110] Z=E+P

[0111] In the formula, the calculation formula of position coding is:

[0112]

[0113] Where pos is the position, i is the dimension index, and dmodel is the dimension of the embedding vector.

[0114] Multi-head self-attention layer design. The multi-head self-attention layer introduces multiple attention heads, allowing the model to focus on information at different positions in parallel and improve the model's expressiveness. Self-attention calculation calculates the correlation between positions in the input sequence, allowing the model to capture long-range dependencies. The specific steps are as follows:

[0115] Linear transformation: Map the input vector to query, key, and value spaces respectively:

[0116] Q=ZW Q

[0117] K=ZWK

[0118] V=ZW V

[0119] Where WQ, WK, and WV are the weight matrices of query, key, and value, respectively.

[0120] Self-attention calculation: Calculate the attention weights and apply them to the values:

[0121]

[0122] Where Q, K and V are the matrices of query, key and value respectively. is the scaling factor, and the softmax function is used to normalize the weights.

[0123] Multi-head mechanism: The self-attention mechanism is extended to multiple heads by computing multiple self-attentions in parallel and concatenating the results:

[0124] MultiHead(Q,K,V)=[head1,head2,...,head h ]W O

[0125] In the formula, h is the number of heads, is the weight matrix of each head, and WO is the weight matrix of the output.

[0126] Feedforward neural network design. The feedforward neural network layer further processes the output of the self-attention layer through a series of linear transformations and activation functions, and performs normalization after each feedforward neural network layer to stabilize the model training process and improve the generalization ability of the model. The specific steps are as follows:

[0127] Linear transformation and activation function: Apply linear transformation and nonlinear activation function (ReLU):

[0128] FFN(x)=max(0,xW1+b1)W2+b2

[0129] Where W1, W2 are weight matrices, b1, b2 are bias vectors.

[0130] Layer Normalization: Layer normalization is performed after each feed-forward neural network to stabilize the training process:

[0131]

[0132] Where μ and σ are the mean and standard deviation, respectively, and ε is a small constant to prevent division by zero.

[0133] Figure 2The structural diagram of the MAML-Transformer model provided for this application is as follows Figure 2 The Transformer model is embedded in the meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism; the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is the power generation power prediction value.

[0134] Specifically, the Transformer model is embedded into the MAML framework so that it can quickly adapt to new prediction tasks with a small number of samples. The goal of the MAML framework is to find a set of model parameters that allows the model to quickly adapt to new tasks after a small number of update steps. Its optimization goal is:

[0135]

[0136] In the formula, θ is the model parameter, α is the learning rate, is the loss function. The initial model parameters θ are trained on multiple related tasks so that they can quickly adapt to new tasks.

[0137] Inner loop update. For each task T i , perform several steps of gradient descent update based on the current model parameters θ and calculate the task loss The updated parameters are:

[0138]

[0139] In the formula, θ is the model parameter, α is the learning rate, is the loss function.

[0140] Outer loop update. During meta-level training, the initial model parameters θ are updated by aggregating the validation losses of all tasks so that they can quickly adapt to different tasks. The update formula is:

[0141]

[0142] Where β is the meta-learning rate.

[0143] Performing multi-task sampling according to the preprocessed data to obtain multiple sets of training data; inputting the multiple sets of training data into the MAML-Transformer model for training, including:

[0144] Randomly sample multiple tasks T from the dataset i ,Each task can be data from different time periods, or photovoltaic power generation data from different locations, to ensure the diversity and extensiveness of the tasks.

[0145] For each task Ti Divide it into a training set and a validation set. The training set is used to update the model parameters, and the validation set is used to evaluate the performance of the model on the task. After the training is completed, save the model.

[0146] The above steps complete the MAML-Transformer model training.

[0147] When power generation prediction is required, real-time collected photovoltaic power generation related data are obtained, and the photovoltaic power generation related data are input into the MAML-Transformer model as input data;

[0148] Obtain the power generation prediction value output by the MAML-Transformer model.

[0149] Figure 3 A schematic diagram of the structure of a photovoltaic power prediction device based on meta-learning and attention mechanism provided in an embodiment of the present invention, wherein the photovoltaic power prediction device based on meta-learning and attention mechanism is used to implement the photovoltaic power prediction method based on meta-learning and attention mechanism, including:

[0150] A model training module 201 is used to pre-build and train a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation power prediction value;

[0151] A model application module 202 is used to obtain photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data into the MAML-Transformer model as input data;

[0152] The output module 203 is used to obtain the predicted value of the generated power output by the MAML-Transformer model.

[0153] Based on the above implementation, optionally, the model training module 201 is used to:

[0154] Obtain a historical characteristic data set of photovoltaic power generation, and perform preprocessing according to the historical characteristic data set;

[0155] Constructing a Transformer model based on an attention mechanism, wherein the Transformer model includes an input embedding layer, a multi-head self-attention layer, and a feedforward neural network layer;

[0156] Embed the Transformer model into the meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism;

[0157] Perform multi-task sampling based on the preprocessed data to obtain multiple sets of training data;

[0158] The multiple sets of training data are input into the MAML-Transformer model for training.

[0159] Based on the above implementation, optionally, the model training module 201 is used to perform preprocessing according to the historical feature data set, including:

[0160] Perform default value filling, normalization and / or abnormal data screening based on historical feature data sets.

[0161] Based on the above implementation, optionally, the model training module 201 is used to fill in default values ​​according to the historical feature data set, including:

[0162] For the missing characteristic variable data in the historical characteristic data set, the median of the corresponding characteristics of the photovoltaic station is calculated, and the missing characteristic variables are filled according to the median;

[0163] Based on the above implementation, optionally, the model training module 201 is used to perform normalization processing according to the historical feature data set, including:

[0164] Eliminate the influence of the variable dimension in the historical feature data set, use the standard score method to set the mean value of the feature variable to 0 and the standard deviation to 1. The calculation formula is:

[0165]

[0166] Among them, X is the characteristic data value, μ is the mean of the characteristic data, and σ is the standard deviation of the characteristic data;

[0167] Based on the above implementation, optionally, the model training module 201 is used to screen abnormal data according to the historical feature data set, including:

[0168] According to the 3σ principle of data processing, abnormal data distributed outside (μ-3σ,μ+3σ) are screened and cleaned.

[0169] Based on the above implementation, optionally, the model training module 201 is used to embed the Transformer model into a meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism, including:

[0170] The Transformer model is embedded into the MAML framework to obtain a MAML-Transformer model based on meta-learning and attention mechanism; the goal of the MAML framework is to find a set of model parameters that enables the MAML-Transformer model to quickly adapt to new tasks after a small number of update steps;

[0171] Perform the inner loop update of the Transformer model, including: for each task T i , perform several steps of gradient descent update based on the current model parameters θ and calculate the task loss L Ti ;

[0172] Perform an outer loop update of the MAML-Transformer model, including: During meta-level training, update the initial model parameters θ by aggregating the validation losses of all tasks.

[0173] Based on the above implementation, optionally, the optimization goal of the MAML-Transformer model is:

[0174]

[0175] Among them, θ is the model parameter, α is the learning rate, is the loss function; the initial model parameter θ is trained on multiple related tasks so that it can quickly adapt to new tasks;

[0176] When the model training module 201 is used for the inner loop update of the Transformer model, the updated parameters are:

[0177]

[0178] In the formula, θ is the model parameter, α is the learning rate, is the loss function;

[0179] When the model training module 201 is used for the outer loop update of the MAML-Transformer model, the update formula is:

[0180]

[0181] Where β is the meta-learning rate.

[0182] Based on the above implementation, optionally, the model training module 201 is used to perform multi-task sampling according to the preprocessed data to obtain multiple sets of training data, including:

[0183] Randomly sample multiple tasks from the preprocessed data, each task corresponding to photovoltaic power generation related data in different time periods or different locations;

[0184] The training data obtained for each task is divided to obtain a training set and a validation set. The technical solution provided by the present invention is a model training module 201, which is used to pre-build and train a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation prediction value; a model application module 202 is used to obtain photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data as input data into the MAML-Transformer model; an output module 203 is used to obtain the power generation prediction value output by the MAML-Transformer model. Compared with traditional statistical models and deep learning models that require a large amount of data for training to obtain accurate prediction models, the technical solution provided by the present invention combines the Transformer model of the attention mechanism with meta-learning to obtain a MAML-Transformer model. The meta-learning framework can realize the training of the Transformer model in a small sample scenario. The Transformer model with a self-attention mechanism can effectively capture the long-range dependencies in the time series data, thereby improving the accuracy of the prediction. In addition, the trained model can be deployed to the actual photovoltaic power prediction system, receiving input data in real time and visualizing the output prediction results. By combining the meta-learning MAML and Transformer models, the present invention can not only realize the prediction task based on a small amount of photovoltaic data, reducing the training data requirements of the prediction model, but also significantly improve the accuracy and robustness of photovoltaic power prediction through the self-attention mechanism, thereby improving the practical application value of the model.

[0185] The photovoltaic power prediction device based on meta-learning and attention mechanism provided in the embodiment of the present invention can execute the photovoltaic power prediction method based on meta-learning and attention mechanism provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0186] Figure 4 1 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0187] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0188] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0189] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the photovoltaic power prediction method based on meta-learning and attention mechanism.

[0190] In some embodiments, the photovoltaic power prediction method based on meta-learning and attention mechanism can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the photovoltaic power prediction method based on meta-learning and attention mechanism described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the photovoltaic power prediction method based on meta-learning and attention mechanism by any other appropriate means (for example, by means of firmware).

[0191] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0192] The computer program for implementing the photovoltaic power prediction method based on meta-learning and attention mechanism of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or completely on a remote machine or server.

[0193] The embodiment of the present invention further provides a computer-readable storage medium, which stores computer instructions. The computer instructions can be used to enable a processor to execute a photovoltaic power prediction method based on meta-learning and attention mechanism, the method comprising:

[0194] Pre-building and training a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation power prediction value;

[0195] Acquire photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data into the MAML-Transformer model as input data;

[0196] Obtain the power generation prediction value output by the MAML-Transformer model.

[0197] Based on the above implementation, a MAML-Transformer model based on meta-learning and attention mechanism is pre-built and trained, including:

[0198] Obtain a historical characteristic data set of photovoltaic power generation, and perform preprocessing according to the historical characteristic data set;

[0199] Constructing a Transformer model based on an attention mechanism, wherein the Transformer model includes an input embedding layer, a multi-head self-attention layer, and a feedforward neural network layer;

[0200] Embed the Transformer model into the meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism;

[0201] Perform multi-task sampling based on the preprocessed data to obtain multiple sets of training data;

[0202] The multiple sets of training data are input into the MAML-Transformer model for training.

[0203] Based on the above implementation, preprocessing is performed according to the historical feature data set, including:

[0204] Perform default value filling, normalization and / or abnormal data screening based on historical feature data sets.

[0205] Based on the above implementation, default value filling is performed according to the historical feature data set, including:

[0206] For the missing characteristic variable data in the historical characteristic data set, the median of the corresponding characteristics of the photovoltaic station is calculated, and the missing characteristic variables are filled according to the median;

[0207] Normalization is performed based on historical feature data sets, including:

[0208] Eliminate the influence of the variable dimension in the historical feature data set, use the standard score method to set the mean value of the feature variable to 0 and the standard deviation to 1. The calculation formula is:

[0209]

[0210] Among them, X is the characteristic data value, μ is the mean of the characteristic data, and σ is the standard deviation of the characteristic data;

[0211] Screening of abnormal data based on historical feature data sets, including:

[0212] According to the 3σ principle of data processing, abnormal data distributed outside (μ-3σ,μ+3σ) are screened and cleaned.

[0213] On the basis of the above implementation, the Transformer model is embedded in the meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism, including:

[0214] The Transformer model is embedded into the MAML framework to obtain a MAML-Transformer model based on meta-learning and attention mechanism; the goal of the MAML framework is to find a set of model parameters that enables the MAML-Transformer model to quickly adapt to new tasks after a small number of update steps;

[0215] Perform the inner loop update of the Transformer model, including: for each task T i , perform several steps of gradient descent update based on the current model parameters θ and calculate the task loss

[0216] Perform an outer loop update of the MAML-Transformer model, including: During meta-level training, update the initial model parameters θ by aggregating the validation losses of all tasks.

[0217] Based on the above implementation, the optimization goal of the MAML-Transformer model is:

[0218]

[0219] Among them, θ is the model parameter, α is the learning rate, is the loss function; the initial model parameter θ is trained on multiple related tasks so that it can quickly adapt to new tasks;

[0220] When the inner loop of the Transformer model is updated, the updated parameters are:

[0221]

[0222] In the formula, θ is the model parameter, α is the learning rate, is the loss function;

[0223] When the outer loop of the MAML-Transformer model is updated, the update formula is:

[0224]

[0225] Where β is the meta-learning rate.

[0226] On the basis of the above implementation, multi-task sampling is performed according to the preprocessed data to obtain multiple sets of training data, including:

[0227] Randomly sample multiple tasks from the preprocessed data, each task corresponding to photovoltaic power generation related data in different time periods or different locations;

[0228] The training data obtained for each task is divided into a training set and a validation set.

[0229] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0230] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0231] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0232] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0233] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0234] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A photovoltaic power prediction method based on meta-learning and attention mechanism, characterized in that: include: Pre-building and training a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation power prediction value; Acquire photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data into the MAML-Transformer model as input data; Obtain the power generation prediction value output by the MAML-Transformer model.

2. The method according to claim 1, characterized in that Pre-built and trained MAML-Transformer model based on meta-learning and attention mechanism, including: Obtain a historical characteristic data set of photovoltaic power generation, and perform preprocessing according to the historical characteristic data set; Constructing a Transformer model based on an attention mechanism, wherein the Transformer model includes an input embedding layer, a multi-head self-attention layer, and a feedforward neural network layer; Embed the Transformer model into the meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism; Perform multi-task sampling based on the preprocessed data to obtain multiple sets of training data; The multiple sets of training data are input into the MAML-Transformer model for training.

3. The method according to claim 2, characterized in that Preprocessing is performed based on historical feature datasets, including: Perform default value filling, normalization and / or abnormal data screening based on historical feature data sets.

4. The method according to claim 3, characterized in that Fill in default values ​​based on historical feature datasets, including: For the missing characteristic variable data in the historical characteristic data set, the median of the corresponding characteristics of the photovoltaic station is calculated, and the missing characteristic variables are filled according to the median; Normalization is performed based on historical feature data sets, including: Eliminate the influence of the variable dimension in the historical feature data set, use the standard score method to set the mean value of the feature variable to 0 and the standard deviation to 1. The calculation formula is: Among them, X is the characteristic data value, μ is the mean of the characteristic data, and σ is the standard deviation of the characteristic data; Screening of abnormal data based on historical feature data sets, including: According to the 3σ principle of data processing, abnormal data distributed outside (μ-3σ,μ+3σ) are screened and cleaned.

5. The method according to claim 2, characterized in that: The Transformer model is embedded in the meta-learning framework to construct a MAML-Transformer model based on meta-learning and attention mechanism, including: The Transformer model is embedded into the MAML framework to obtain a MAML-Transformer model based on meta-learning and attention mechanism; the goal of the MAML framework is to find a set of model parameters that enables the MAML-Transformer model to quickly adapt to new tasks after a small number of update steps; Perform the inner loop update of the Transformer model, including: for each task T i , perform several steps of gradient descent update based on the current model parameters θ and calculate the task loss Perform an outer loop update of the MAML-Transformer model, including: During meta-level training, update the initial model parameters θ by aggregating the validation losses of all tasks.

6. The method according to claim 5, characterized in that The optimization goal of the MAML-Transformer model is: Among them, θ is the model parameter, α is the learning rate, is the loss function; the initial model parameter θ is trained on multiple related tasks so that it can quickly adapt to new tasks; When the inner loop of the Transformer model is updated, the updated parameters are: In the formula, θ is the model parameter, α is the learning rate, is the loss function; When the outer loop of the MAML-Transformer model is updated, the update formula is: Where β is the meta-learning rate.

7. The method according to claim 2, characterized in that Multi-task sampling is performed based on the preprocessed data to obtain multiple sets of training data, including: Randomly sample multiple tasks from the preprocessed data, each task corresponding to photovoltaic power generation related data in different time periods or different locations; The training data obtained for each task is divided into a training set and a validation set.

8. A photovoltaic power prediction device based on meta-learning and attention mechanism, characterized in that: include: A model training module, used to pre-build and train a MAML-Transformer model based on meta-learning and attention mechanism, wherein the input data of the MAML-Transformer model is photovoltaic power generation related data, and the output is a power generation power prediction value; A model application module, used to obtain photovoltaic power generation related data collected in real time, and input the photovoltaic power generation related data into the MAML-Transformer model as input data; The output module is used to obtain the predicted value of the generated power output by the MAML-Transformer model.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the photovoltaic power prediction method based on meta-learning and attention mechanism as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the photovoltaic power prediction method based on meta-learning and attention mechanism described in any one of claims 1 to 7 when executed.

Images