Small sample photovoltaic data generation method based on diffusion model
By constructing diffusion model and reverse denoising model, the problem of small sample time series data enhancement in the field of photovoltaic power generation is solved, and data similar to real data is generated, which improves the performance of the data-driven model.
Patent Information
- Application Number
- CN202411444289.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-16
- Publication Date
- 2025-02-28
AI Technical Summary
In the field of photovoltaic power generation, it is difficult for the prior art to effectively use diffusion models to enhance small sample time series data, resulting in a degradation in the performance of data-driven models.
By reading the photovoltaic dataset, calculate statistical characteristics and standardize the data; set the key parameters of the diffusion process to build a diffusion model; use deep learning models to integrate fully connected layers and position coding to build an inverse denoising model; train the model by minimizing the mean square error loss function; generate simulated data and output time series photovoltaic data with original data trends through the inverse denoising model.
Time series data numerically and structurally similar to the real illumination intensity was successfully generated, improving the performance of the data-driven model, and achieving results that are highly consistent with the real data distribution at a smaller number of training times.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of photovoltaic power generation, and in particular to a method for generating small sample photovoltaic data based on a diffusion model. Background Art
[0002] The quality and quantity of data are key factors affecting the performance of data-driven models. When the data sample is insufficient, the accuracy of the model will decrease. Data acquisition is not only limited by the high cost of collection and the long data collection cycle, but also with the increasing awareness of data security and privacy protection, the introduction of relevant laws and regulations has further aggravated the difficulty of data acquisition. How to perform data enhancement on small samples has important research value and significance.
[0003] In recent years, data enhancement has been widely used in the field of images. There is a significant difference between the distribution of data generated by data enhancement models based on time series data and real data, which limits its application in data protection and improving the performance of data-driven models. Studies have shown that compared with other data enhancement models, diffusion models show their unique advantages in processing small sample data. Diffusion models can generate new data samples while maintaining the original distribution characteristics of data by simulating the random process of data generation. However, diffusion models are currently widely used in the field of image generation, and there are currently few methods for time series data enhancement in the field of photovoltaic power generation based on diffusion models. Therefore, how to effectively apply diffusion models to enhance time series data in the field of photovoltaic power generation is an area worthy of in-depth research and exploration. Summary of the invention
[0004] In view of the above-mentioned problems, the present invention is proposed.
[0005] Therefore, the technical problem solved by the present invention is: applying the diffusion model to time series data enhancement, generating physically credible data samples, and improving the performance of the data-driven model.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for generating small sample photovoltaic data based on a diffusion model, comprising:
[0007] Read photovoltaic data sets, calculate statistical characteristics of the data, and standardize the data;
[0008] Set the key parameters of the diffusion process and build a diffusion model;
[0009] Use deep learning models to integrate fully connected layers with position encoding based on sine and cosine functions to build an inverse denoising model;
[0010] The inverse denoising model is trained by minimizing the mean square error loss function;
[0011] The simulated data were generated using multivariate normal distribution, with the original data mean as the mean parameter and the unit matrix as the covariance structure, to construct a noise data set with independent and identical distribution in each dimension;
[0012] Using the trained denoising model, the noise data in the diffusion process is inversely denoised to output time series photovoltaic data with the original data trend;
[0013] Use the ratio of the sample data and the output data to amplify the output data by the same factor to get the final generated data.
[0014] As a preferred solution of the method for generating small sample photovoltaic data based on a diffusion model according to the present invention, the construction of the diffusion model includes defining a Markov chain, adding random noise in T total diffusion time steps, converting the original photovoltaic data x0 into Gaussian noise, and performing a forward diffusion process, which is expressed as:
[0015]
[0016] Among them, q(x1,x2,…,x T x0) is the joint distribution of the forward diffusion process, x1, x2, …, xT are the time series data points in the diffusion process, x0 is the original data point, T is the total time steps of the diffusion process, q(x t |x t-1 ) is the conditional distribution of the forward diffusion process, x t is the data point at time step t, x t-1 is the data point at time step t-1, β t is the diffusion variance at time step t, β t ∈[0,1), I is the identity matrix.
[0017] As a preferred solution of the method for generating photovoltaic data of small samples based on diffusion model described in the present invention, in which: in the forward diffusion process, at any time step t, x t Sampling is performed, which is expressed as,
[0018]
[0019] Among them, q(x t |x0) is the x in the forward diffusion process t The conditional distribution of α t is the variance ratio at time step t, is the cumulative product of the α values from time step 1 to t, z t represents noise, x t Represents data with noisy data after diffusion.
[0020] As a preferred solution of the method for generating photovoltaic data of small samples based on diffusion model described in the present invention, wherein: the integration of fully connected layer and position coding based on sine and cosine functions includes generating a position coding matrix position, providing position information of each time point in the sequence, expressed as,
[0021]
[0022] Among them, i is the position index, indicating the time point in the sequence, j is the dimension index, and d pos Represents the number of dimensions of the position encoding, that is, the width of the encoding matrix. 10000 is a scaling factor used to control the period of the sine and cosine functions.
[0023] As a preferred solution of the method for generating small sample photovoltaic data based on diffusion model described in the present invention, wherein: the construction of the inverse denoising model is expressed as:
[0024]
[0025] Among them, μ θ (x t ,t) is the mean predicted by the neural network parameter θ, indicating that given x t and the expected value of the inverse distribution at time step t, is the variance ratio of the prediction in the inverse denoising process, learned by the model.
[0026] As a preferred solution of the method for generating photovoltaic data of small samples based on diffusion model described in the present invention, wherein: the construction of noise data sets with independent and identical distribution in each dimension is expressed as:
[0027] x ~ N(μ,I)
[0028] Where x is the generated noisy data vector and μ is the mean vector of the original data.
[0029] As a preferred solution of the method for generating photovoltaic data of small samples based on diffusion model described in the present invention, the method of obtaining the final generated data includes calculating the sum of the original sample data S X , expressed as,
[0030]
[0031] Among them, X i is the i-th data point in the original sample data, and N is the number of samples;
[0032] Calculate the sum S of the output data generated by the inverse denoising process Y , expressed as,
[0033]
[0034] Among them, Y i is the i-th data point in the output data.
[0035] As a preferred solution of the method for generating photovoltaic data of small samples based on the diffusion model described in the present invention, wherein: by comparing the reference value and the target value, the amplification factor k is calculated and generated, which is expressed as:
[0036]
[0037] Using the generated magnification factor k, each data point in the output data is adjusted, expressed as,
[0038] Y i最终 = k·Y i
[0039] Among them, Y i最终 is the adjusted output data point.
[0040] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for generating small sample photovoltaic data based on a diffusion model are implemented.
[0041] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for generating small sample photovoltaic data based on a diffusion model as described above.
[0042] Beneficial effects of the present invention: The method for generating photovoltaic data with a small sample based on a diffusion model provided by the present invention successfully generates time series data similar to the real light intensity in terms of value and structure based on the diffusion model. The similarity between the generated sample and the real data is verified through statistical analysis, demonstrating the innovation and practicality of the present invention in generating small sample time series data in the field of photovoltaic power generation.
[0043] Results that are highly consistent with the true data distribution can be achieved with a smaller number of training times. The lower MMD value also indicates the efficiency of the model in approximating the true data distribution in the feature space. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0045] Figure 1An overall flow chart of a method for generating small sample photovoltaic data based on a diffusion model provided by one embodiment of the present invention.
[0046] Figure 2 A system structure diagram of a diffusion model for a method for generating small sample photovoltaic data based on a diffusion model provided by an embodiment of the present invention.
[0047] Figure 3 A comparison chart of generated values and original values of a diffusion model and a VAE model trained 500 times in a small sample photovoltaic data generation method based on a diffusion model provided by an embodiment of the present invention.
[0048] Figure 4 A comparison chart of generated values and original values of a diffusion model and a VAE model trained 1000 times in a small sample photovoltaic data generation method based on a diffusion model provided by an embodiment of the present invention.
[0049] Figure 5 A local comparison diagram of generated values and original values of a diffusion model and a VAE model trained 500 times in a small sample photovoltaic data generation method based on a diffusion model provided by an embodiment of the present invention.
[0050] Figure 6 A local comparison diagram of generated values and original values of a diffusion model and a VAE model trained 1000 times in a small sample photovoltaic data generation method based on a diffusion model provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0051] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.
[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0053] Example 1, reference Figure 1-Figure 2 , is an embodiment of the present invention, and provides a method for generating small sample photovoltaic data based on a diffusion model, comprising:
[0054] The present invention relates to a method for generating small sample photovoltaic data based on a diffusion model. The basic idea is as follows: first, a Markov chain is defined, and the original photovoltaic data is converted into Gaussian noise by adding random noise. This process is called the forward diffusion process. In the reverse denoising process, the random noise x T Denoising is continuously performed at each moment t until the photovoltaic data x0 is generated. The present invention sets and calculates the diffusion process parameters by preprocessing the photovoltaic data set and calculating the key statistical characteristics, thereby providing a theoretical basis for the inverse denoising process. Secondly, a deep learning denoising model including a fully connected layer and position encoding is constructed, and trained by minimizing the mean square error loss function to optimize the model parameters to improve the accuracy of inverse denoising. Finally, the trained denoising model is used to perform the inverse denoising process on the noise data to generate a variety of data samples. The steps involved are as follows:
[0055] Step 1: Data preprocessing: Read and process the photovoltaic data set, calculate the statistical characteristics of the data, and standardize the data.
[0056] like Figure 2 As shown in Figure 2, step 2, setting diffusion model parameters. Set the key parameters of the diffusion process to construct the forward diffusion process. This process first defines a Markov chain, which converts the original photovoltaic data x0 into Gaussian noise x by adding random noise in T total diffusion time steps. T ~N(0,I), this process is called forward diffusion process, which is composed of distribution q(x t |x t-1 ) is represented by the specific formulas shown in formula (1) and (2).
[0057]
[0058] Among them, q(x1,x2,…,x T x0) is the joint distribution of the forward diffusion process, x1,x2,…,x T is the time series data point in the diffusion process. x0 is the original data point. T is the total time step of the diffusion process. t |x t-1 ) is the conditional distribution of the forward diffusion process, x t is the data point at time step t. t-1 is the data point at time step t-1. t is the diffusion variance at time step t, β t ∈[0,1). represents the data point x at time step t t The mean is and variance is β t Normal distribution of I, where I is the identity matrix.
[0059] In the forward diffusion process, x can be t Sampling is performed, and the specific formulas are shown in formulas (3) and (4).
[0060]
[0061] Among them, q(x t |x0) is the x in the forward diffusion process t The conditional distribution of α t is the variance ratio at time step t, α i is the variance scaling factor at i, is the cumulative product of the α values from time step 1 to t, z t ~N(0,I),z t represents noise, x t Represents data with noisy data after diffusion.
[0062] Step 3. Initialize the model and optimizer. Design and implement a deep learning model that integrates a fully connected layer with a position encoding based on sine and cosine functions to learn the inverse denoising process of data. This process generates a position encoding matrix position, which will provide the model with the position information of each time point in the sequence. The specific formula is shown in formula (5). The encoding is implemented using sine and cosine functions, so that the model can distinguish the features of different time points. When the dimension index j is an even number, the sine function is used to encode the position index i. When the dimension index j is an odd number, the cosine function is used to encode the position index i.
[0063]
[0064] Where i is the position index, which represents a time point in the sequence. j is the dimension index, which specifies the specific dimension of the sine or cosine function used in the position encoding. pos Indicates the number of dimensions of the position encoding, that is, the width of the encoding matrix. 10000 is a scaling factor used to control the period of the sine and cosine functions.
[0065] For the inverse denoising process, its purpose is to T Continuously remove noise to generate new photovoltaic data x0, where x T ~N(0,I). The inverse denoising process can be performed using the inverse distribution p θ (x t-1 |x t ) is expressed. Unlike the forward diffusion process, the marginal likelihood p(x0) = ∫p(x0,x1,...,x T-1 |x T )·p(x T )dx1:T It is difficult to calculate the result. At this time, we assume that p θ (x t-1 |x t ) is the inverse denoising process distribution with learnable parameters θ, which is fitted by a neural network, and the predicted mean μ can be obtained through the neural network θ (x t ,t) and variance in, The inverse distribution is expressed as shown in formula (6):
[0066]
[0067] Among them, μ θ (x t ,t) is the mean value predicted by the neural network parameter θ, which means that given x t and the expected value of the inverse distribution at time step t. It is the variance ratio of the prediction in the inverse denoising process, usually learned by the model.
[0068] Step 4: Train the model. The designed inverse denoising model is systematically trained by minimizing the mean square error loss function. During the training process, the model loops through the steps of selecting time steps to generate noisy samples, calculating the posterior distribution parameters of noisy samples, reverse denoising and calculating losses, performing reverse propagation and updating model parameters. By learning the reverse denoising process, the network parameters are continuously optimized to improve the prediction accuracy of the reverse denoising state.
[0069] Step 5: Generate random noisy data. Use multivariate normal distribution to generate simulated data, use the original data mean as the mean parameter, and the unit matrix as the covariance structure to achieve the construction of noise data sets with independent and identical distribution in each dimension. Use the original data mean μ as the mean parameter of the generated data, and the unit matrix I as the covariance matrix to generate the noisy data x. The specific formula is shown in formula (7).
[0070] x~N(μ,I)(7)
[0071] Where x is the generated noisy data vector. μ is the mean vector of the original data. I is the identity matrix of the same size as the data dimension, representing the covariance structure, that is, the dimensions are independent of each other and the variance of each dimension is 1.
[0072] Step 6: Output data: Using the trained denoising model, perform reverse denoising on the noise data in the diffusion process and output time series photovoltaic data with the original data trend.
[0073] Step 7: Generate data. Use the ratio of the original sample data and the output data to amplify the output data by the same ratio to obtain the final generated data.
[0074] First calculate the sum S of the original sample data X , which will be used as our benchmark value. The specific formula is shown in formula (8).
[0075]
[0076] Among them, X i is the i-th data point in the original sample data, and N is the number of samples.
[0077] Then calculate the sum of the output data generated by the inverse denoising process S Y , which will be used as our target value. The specific formula is shown in formula (9).
[0078]
[0079] Among them, Y i is the i-th data point in the output data.
[0080] By comparing the benchmark value and the target value, the amplification factor k is calculated and generated. This factor represents the magnitude change from the sample data to the output data. The specific formula is shown in formula (10).
[0081]
[0082] The generated magnification factor k is used to adjust each data point in the output data to obtain the final generated data. The processing of each data point in the output data is shown in formula (11).
[0083] Y i最终 = k·Y i (11)
[0084] Among them, Y i最终 is the adjusted output data point.
[0085] Embodiment 2, an embodiment of the present invention, which is different from the previous embodiment in that:
[0086] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0087] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.
[0088] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.
[0089] It should be understood that the various parts of the present invention can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, a plurality of steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented with hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, an application specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0090] Example 4, reference Figure 3-Figure 6 , which is an embodiment of the present invention, provides a method for generating small sample photovoltaic data based on a diffusion model. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.
[0091] The present invention uses the daily average light intensity in 2023 in six counties within 70 km of the Baihetan Hydropower Station, namely, Ningnan County, Qiaojia County, Ludian County, Jinyang County, Butuo County, and Puge County, as an example to verify the effect of the present invention. Figure 2 The overall process of the small sample photovoltaic data generation method based on the diffusion model is shown to generate data to reflect the effect achieved by the present invention.
[0092] The specific technical solutions are as follows:
[0093] Step 1: Data preprocessing. The present invention first uses the Xihe Energy Meteorological Big Data Platform to download the 2023 daily average light intensity of six counties within 70km around the Baihetan Power Station, including Ningnan County, Qiaojia County, Ludian County, Jinyang County, Butuo County, and Puge County, and conducts an in-depth analysis of the distribution characteristics, calculates statistical characteristics such as the mean, and performs a standardization process to convert the data into a format with zero mean and unit variance, which is input into the model to lay the foundation for subsequent model training.
[0094] Step 2: Setting diffusion model parameters. In the process of constructing the forward diffusion model, the present invention β t The start and end parameters were set to 0.0004 and 0.02, respectively, which together controlled the start and end states of the noise level in the forward diffusion process. In addition, the time step parameter was set to 500 and 1000 during the experiment to ensure the full development of the forward diffusion process.
[0095] Step three, initialization of the model and optimizer. The model design proposed in the present invention pays special attention to processing time series data. The deep learning model uses a fully connected layer to extract deep features in the time series data and provide information for the subsequent inverse denoising process. In order to further enhance the model's understanding of the time dependency of time series data, the present invention introduces position encoding based on sine and cosine functions into the model to provide the model with the relative position information of each time step in the sequence. By assigning a position identifier to each time point, the model can capture the long-term dependencies in the time series. In the selection of the optimizer, the AdamW optimizer is adopted. By introducing a weight decay mechanism, it can regularize the model parameters during the training process, thereby improving the generalization ability of the model.
[0096] Step 4: Model training. During the model training phase, the mean square error loss function (MSE) is used as the loss function to quantify the deviation between the model prediction and the actual data. During the training process, the model generates noisy samples by cyclically selecting time steps and calculates the posterior distribution parameters of these samples. Subsequently, the inverse denoising operation is performed and the network parameters are updated based on the prediction error. In this way, the model learns the inverse denoising process of restoring the original data structure from the noisy data, thereby gradually optimizing the network parameters and improving the prediction accuracy of the inverse denoising state.
[0097] Step 5: Generate random noise data. Using the multivariate normal distribution method, the mean of the original data is used as the mean parameter, and the unit matrix is used as the covariance structure to generate noise data sets with independent and identical distribution in each dimension. These data will be used as inputs of the diffusion model to simulate the noise and uncertainty in real data.
[0098] Step 6: Output data. The trained denoising model is used to perform reverse denoising on the noise data generated during the diffusion process. In the reverse denoising process, the model decodes the noise data at each time step, gradually removes the noise components, and restores the original state of the data. The structured photovoltaic data that retains the trend of the original data is output. These data not only reflect the basic characteristics of the original data, but also undergo reverse noise processing, providing a high-quality data set for further analysis and application.
[0099] Step 7: Generate data. First, calculate the sum of the sample data to get the baseline value. Second, calculate the sum of the output data obtained by the inverse denoising process to get the target value. By comparing these two values, we can get the generation amplification factor, which reflects the magnitude change from the sample data to the output data. Finally, using the generation amplification factor, we can enlarge or reduce the output data as a whole to get the final generated data set. Figure 3 , Figure 4 , Figure 5 , Figure 6 It can be seen that although the data generated by the VAE model has the changing trend of the original data, the amplitude is quite different from the real data. The diffusion model learns the sample data structure well, and the generated light intensity data is more realistic, and the local details and overall structure are more natural.
[0100] As shown in Table 1, the quality of the model-generated data and the performance of the model are evaluated by the KS (Kolmogorov-Smirnov) statistic, P value, AD (Anderson-Darling) statistic and MMD (Maximum Mean Discrepancy). The KS statistic is a measure of the similarity between two probability distributions. The smaller the value, the closer the data distribution generated by the model is to the true data distribution. In the table, the KS statistic of the diffusion model is significantly smaller than that of the VAE model at both training times, indicating that the data distribution generated by the diffusion model is closer to the true data distribution. The P value is used to judge statistical significance. The larger the P value, the lower the statistical significance, that is, the smaller the difference between the data distribution generated by the model and the true data distribution. The P value of the diffusion model is very close to 1 or equal to 1, which means that there is not enough evidence to reject the hypothesis that the data distribution generated by the diffusion model is the same as the true data distribution. In contrast, the P value of the VAE model is small, especially when the training time is 500 times, indicating that the data distribution generated by it is significantly different from the true data distribution. The AD statistic is another measure of the similarity between two distributions, which is more sensitive to the difference in the tail of the distribution. The AD statistic of the diffusion model is also lower than that of the VAE model, which further proves the superiority of the diffusion model in simulating the distribution of real data. MMD is a measure of the difference between two probability distributions, especially used to evaluate the performance of generative models. It compares the distribution difference between generated data and real data in feature space. The MMD value of the diffusion model is lower, which indicates that the data generated by the diffusion model is closer to the real data in feature space.
[0101] Table 1. Evaluation index table of generated model
[0102]
[0103] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for generating small sample photovoltaic data based on a diffusion model, characterized in that: include: Read photovoltaic data sets, calculate statistical characteristics of the data, and standardize the data; Set the key parameters of the diffusion process and build a diffusion model; Use deep learning models to integrate fully connected layers with position encoding based on sine and cosine functions to build an inverse denoising model; The inverse denoising model is trained by minimizing the mean square error loss function; The simulated data were generated using multivariate normal distribution, with the original data mean as the mean parameter and the unit matrix as the covariance structure, to construct a noise data set with independent and identical distribution in each dimension; Using the trained denoising model, the noise data in the diffusion process is inversely denoised to output time series photovoltaic data with the original data trend; Use the ratio of the sample data and the output data to amplify the output data by the same factor to get the final generated data.
2. The method for generating small sample photovoltaic data based on a diffusion model according to claim 1, characterized in that: The construction of the diffusion model includes defining a Markov chain, converting the original photovoltaic data x0 into Gaussian noise by adding random noise in T total diffusion time steps, and performing a forward diffusion process, which is expressed as: Among them, q(x1,x2,…,x T x0) is the joint distribution of the forward diffusion process, x1, x2, …, xT are the time series data points in the diffusion process, x0 is the original data point, T is the total time steps of the diffusion process, q(x t |x t-1 ) is the conditional distribution of the forward diffusion process, x t is the data point at time step t, x t-1 is the data point at time step t-1, β t is the diffusion variance at time step t, β t ∈[0,1), I is the identity matrix.
3. The method for generating small sample photovoltaic data based on a diffusion model according to claim 2, characterized in that: In the forward diffusion process, at any time step t, t Sampling is performed, which is expressed as, Among them, q(x t |x0) is the x in the forward diffusion process t The conditional distribution of α t is the variance ratio at time step t, is the cumulative product of the α values from time step 1 to t, z t represents noise, x t Represents data with noisy data after diffusion.
4. The method for generating small sample photovoltaic data based on a diffusion model according to claim 3, characterized in that: The integration of the fully connected layer and the position encoding based on sine and cosine functions includes generating a position encoding matrix position, which provides the position information of each time point in the sequence, expressed as, Among them, i is the position index, indicating the time point in the sequence, j is the dimension index, and d pos Represents the number of dimensions of the position encoding, that is, the width of the encoding matrix. 10000 is a scaling factor used to control the period of the sine and cosine functions.
5. The method for generating small sample photovoltaic data based on a diffusion model according to claim 4, characterized in that: The inverse denoising model is constructed as follows: Among them, μ θ (x t ,t) is the mean predicted by the neural network parameter θ, indicating that given x t and the expected value of the inverse distribution at time step t, is the variance ratio of the prediction in the inverse denoising process, learned by the model.
6. The method for generating small sample photovoltaic data based on a diffusion model according to claim 5, characterized in that: The noise data set constructed in each dimension is expressed as: x~N(μ,I) Where x is the generated noisy data vector and μ is the mean vector of the original data.
7. The method for generating photovoltaic data of a small sample based on a diffusion model according to claim 6, characterized in that: The final generated data includes calculating the sum of the original sample data S X , expressed as, Among them, X i is the i-th data point in the original sample data, and N is the number of samples; Calculate the sum S of the output data generated by the inverse denoising process Y , expressed as, Among them, Y i is the i-th data point in the output data.
8. The method for generating small sample photovoltaic data based on a diffusion model according to claim 7, characterized in that: By comparing the baseline value and the target value, the amplification factor k is calculated and expressed as, Using the generated magnification factor k, each data point in the output data is adjusted, expressed as, AND i最终 =k·Y i Among them, Y i最终 is the adjusted output data point.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for generating small sample photovoltaic data based on a diffusion model according to any one of claims 1 to 8 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for generating small sample photovoltaic data based on a diffusion model according to any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Transaction data generation method and device
CN121301916A
A transaction data generation method and apparatus
CN121301916B