High-energy-consumption data augmentation method and system based on generative large model

By introducing deep convolutional generation adversarial networks with self-attention mechanisms and conditional constraint mechanisms in high-energy consumption units, realistic energy consumption data is generated, and the problems of insufficient data volume and low reliability are solved, and the generalization ability and data of the model are improved Consistency meets the energy efficiency and carbon efficiency analysis needs of high-energy-consuming enterprises.

CN118940039APending Publication Date: 2024-11-12STATE GRID JIANGSU ELECTRIC POWER CO LTD MARKETING SERVICE CENT +1

Patent Information

Application Number
CN202411051395.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

High energy consumption units face the problems of insufficient data volume and low reliability in data acquisition and analysis. The existing data enhancement methods are limited in feature capture and difficult training when processing large dynamic and long-sequence data, and the existing energy consumption data is expanded. The model cannot generate realistic data and cannot be applied to dynamic data changes.

Method used

Using a high-energy data augmentation method based on a generative large model, a deep convolutional generation adversarial network is built by introducing self-attention mechanisms and conditional constraint mechanisms, generating realistic energy consumption data, expanding data sets, increasing data diversity and Richness, reduces the risk of overfitting.

Benefits of technology

It significantly improves the generalization ability of the model, reduces the risk of overfitting, and has high consistency in the distribution characteristics of the generated data with the real data, meets the energy efficiency and carbon efficiency analysis needs of high-energy-consuming enterprises, and provides reliable data support for energy management and optimization.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118940039A_ABST
    Figure CN118940039A_ABST
Patent Text Reader

Abstract

The invention discloses a high-energy-consumption data augmentation method and system based on a generative large model, and the method comprises the steps: obtaining the real energy consumption data of a high-energy-consumption unit, and forming original sample data; performing missing value filling on the original sample data to construct a sample set; introducing a self-attention mechanism, and constructing an energy consumption data augmentation training model based on a deep convolutional generative adversarial network; introducing a conditional constraint mechanism, and constructing a high-energy-consumption data augmentation model based on the generative large model; training, testing and verifying the high-energy-consumption data augmentation model based on the generative large model by using the sample set; and collecting real energy consumption data of a high-energy consumption unit, and performing high-energy consumption data augmentation by using the trained high-energy consumption data augmentation model of the generative large model. Reliable data support is provided for energy management and optimization of a high-energy-consumption unit, and the problems that an existing energy consumption data set of the high-energy-consumption unit is insufficient in data size, low in reliability and the like are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of high-energy consumption data augmentation, and in particular relates to a high-energy consumption data augmentation method and system based on a generative large model. Background Art

[0002] High energy-consuming units, such as electricity, oil, steel, nonferrous metals, chemicals, construction, data centers and other industries, have always played an important role in the national economy. High energy-consuming industries consume a lot of resources in the production process, which not only increases production costs but also damages the environment. In order to promote sustainable economic development and achieve my country's "dual carbon" goals, it is of great significance to study the data of high energy-consuming units.

[0003] When analyzing the energy consumption and carbon emission data of enterprises, there are often two difficulties: First, the amount of data is insufficient. At the beginning of construction, some enterprises did not consider the needs of energy efficiency and carbon efficiency analysis. The collection equipment was imperfect and it was impossible to collect sufficient effective data, resulting in a small amount of data, different granularity and precision of various data. Second, the data reliability is not high. Even if the measurement and collection equipment is installed, the authenticity of the collected data is reduced due to environmental interference or data loss during information transmission. The above difficulties make it lack of effective data support when analyzing the energy efficiency level and carbon emission intensity of high-energy-consuming enterprises. Therefore, it is necessary to increase the energy consumption and carbon emission data through technical means.

[0004] There are two main types of existing data enhancement methods. The first is traditional methods such as data resampling and interpolation, which have the advantages of simple principles and small amount of calculation, but may increase the risk of overfitting. The second is data generation methods based on deep learning, which have achieved remarkable results in image, text, and data enhancement. However, when dealing with large dynamic and long sequence data, it still faces problems such as limited feature capture and difficulty in training.

[0005] CN 202311617853.5A discloses an image classification data augmentation method based on a zero-sample segmentation universal visual large model. The invention utilizes the zero-sample segmentation capability of the universal visual model to perform data augmentation on the image classification data set, but the accuracy is not high when generating different tasks or only generates data of a single feature.

[0006] In view of the problems of small collective data volume of high-energy-consuming units, incomplete coverage, high data acquisition cost, high data privacy requirements, and feature data loss and inability to adapt to dynamic data changes in existing energy consumption data augmentation models, an energy consumption data augmentation model is urgently needed to generate realistic generated data to effectively expand the original data, so as to increase the diversity and richness of the data and provide reliable data support for energy management and optimization of high-energy-consuming units. Summary of the invention

[0007] In order to solve the problems of insufficient data volume and low reliability in the existing energy consumption data augmentation model of high-energy-consuming units, and to address the deficiencies in the prior art, the present invention provides a high-energy consumption data augmentation method based on a generative large model, which expands the data set by generating realistic generated data and increases the diversity and richness of the data, thereby improving the generalization ability of the model and reducing the risk of overfitting.

[0008] The present invention adopts the following technical solution.

[0009] A high-energy consumption data augmentation method based on a generative large model, comprising:

[0010] Step 1: Obtain the real energy consumption data of high energy consumption units to form the original sample data;

[0011] Step 2: Fill missing values ​​in the original sample data and construct a sample set;

[0012] Step 3: Introduce the self-attention mechanism and build an energy consumption data augmentation training model based on deep convolutional generative adversarial network;

[0013] Step 4: According to the energy consumption data augmentation training model based on deep convolutional generative adversarial network constructed in step 3, a conditional constraint mechanism is introduced to build a high energy consumption data augmentation model based on a generative large model;

[0014] Step 5: Use the sample set to train, test and verify the high-energy consumption data augmentation model based on the generative large model to obtain the trained high-energy consumption data augmentation model based on the generative large model.

[0015] The present invention further includes the following preferred embodiments:

[0016] In step 1, the actual energy consumption data of high energy consumption units include energy consumption and carbon dioxide emissions.

[0017] In step 2, filling missing values ​​in the original sample data includes:

[0018] The sliding window technique is used to fill the missing values ​​of the original sample data with adjacent values.

[0019] In step 3, a self-attention mechanism is introduced to construct an energy consumption data augmentation training model based on a deep convolutional generative adversarial network, including: using a deep convolutional network to extract features of high energy consumption data; using a generative adversarial network to augment high energy consumption data based on the features extracted by the deep convolutional network; and introducing a self-attention mechanism to enhance the energy consumption data augmentation model's ability to capture long-distance dependencies in the data.

[0020] The introduction of the self-attention mechanism includes: receiving input tensors with batch size, number of channels, height and width of 64, 1, 48 and 48 respectively; randomly initializing weight parameters q, k and v, and generating matrices of query (Query, Q), key (Key, K) and value (Value, V) through dot product operation; calculating the similarity between each element in the matrix and other elements, and normalizing the similarity to the attention weight of the element; taking weighted sum of each element in the matrix and its attention weight to obtain the output tensor of the self-attention mechanism.

[0021] In step 4, a conditional constraint mechanism is introduced to construct a high-energy consumption data augmentation model based on a generative large model, including: introducing a conditional constraint mechanism, receiving external seasonal condition information, the generator adjusts the data generation process according to the received seasonal conditions, so that the generated data gradually approaches the distribution of the real data, the discriminator compares the consistency of the generated data and the real data in the seasonal condition information, and feeds the comparison result back to the generator to optimize the generator; the energy consumption data augmentation training model based on the deep convolutional generative adversarial network constructed in step 3 is connected in series to form a high-energy consumption data augmentation model based on a generative large model.

[0022] At the same time, the present invention also provides a high-energy consumption data augmentation system based on a generative large model, including: a data acquisition module, a data processing module, a model construction module and a model training module;

[0023] The data acquisition module is used to obtain the real energy consumption data of high energy consumption units to form the original sample data;

[0024] The data processing module uses window sliding technology to fill missing values ​​of the original sample data with adjacent values ​​to construct a sample set;

[0025] The model building module includes a deep convolutional generative adversarial network building module and a generative large model building module;

[0026] The model training module uses the sample set to train, test and verify the high-energy consumption data augmentation model based on the generative large model, and obtains the trained high-energy consumption data augmentation model based on the generative large model;

[0027] The deep convolutional generative adversarial network building module is used to combine the self-attention mechanism to build an energy consumption data augmentation training model based on the deep convolutional generative adversarial network;

[0028] The generative large model construction module is used to introduce a conditional constraint mechanism to build a high-energy consumption data augmentation model based on the generative large model according to the energy consumption data augmentation training model based on the deep convolutional generative adversarial network construction module.

[0029] A terminal comprises a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of any method.

[0030] A computer-readable storage medium stores a computer program, which implements the steps of any one of the methods when executed by a processor.

[0031] The beneficial effect of the present invention is that, compared with the prior art, the present invention effectively optimizes the performance of the generator and discriminator of the adversarial network by adding a conditional constraint mechanism on the basis of the deep convolutional generative adversarial network (DCGAN) and the self-attention mechanism, forming an innovative generative large model (GLM), which enables the model to fully consider external influencing factors when generating data, significantly enhances the model's ability to extract and learn data features, and finally builds a large model with 72 billion parameters. During the training process, due to the large number of parameters of the large model, the calculation and memory requirements are increased, resulting in a complex and time-consuming training process. For this reason, a multi-card distributed training method is used to shorten the training time; the trained model is subjected to enhanced testing of energy consumption and carbon dioxide emissions data on the Steel industry data dataset, and the results show that the generated data has a high consistency with the distribution characteristics of the real data. Compared with traditional data enhancement technology, the mean absolute error (MAE) value of energy consumption and carbon dioxide emissions data generated by the GLM method is greatly reduced, with the maximum values ​​being 38.39% and 41.94% respectively. This meets the energy efficiency and carbon efficiency analysis needs of high-energy-consuming enterprises, provides reliable data support for energy management and optimization of high-energy-consuming units, and solves the problems of insufficient data volume and low reliability of existing energy consumption data sets of high-energy-consuming units. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a step diagram of a high-energy consumption data augmentation method based on a generative large model in an embodiment of the present invention;

[0033] Figure 2 is a diagram of a sliding process of a sliding window in one embodiment of the present invention;

[0034] Figure 3 is a structural diagram of a deep convolutional generative adversarial network in an embodiment of the present invention;

[0035] Figure 4 is a diagram of a generator network structure in an embodiment of the present invention;

[0036] Figure 5 is a diagram of the splicing process of input data and seasonal condition information in an embodiment of the present invention;

[0037] Figure 6 This is a diagram of the structure of a generative large model in an embodiment of the present invention;

[0038] Figure 7 It is a multi-card distributed training framework in an embodiment of the present invention;

[0039] Figure 8 is a comparison chart of power consumption data generated based on a generative large model and original data in one embodiment of the present invention;

[0040] Fig. 9 It is a comparison chart between carbon dioxide data generated based on the generative large model and original data in one embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical scheme and advantages of the present invention clearer, the technical scheme of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. The embodiments described in this application are only embodiments of a part of the present invention, rather than all embodiments. Based on the spirit of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the protection scope of the present invention.

[0042] The embodiment of the present invention provides a high-energy consumption data augmentation method based on a generative large model. Figure 1 The figure shows a step diagram of a high-energy consumption data augmentation method based on a generative large model in an embodiment of the present invention; the high-energy consumption data augmentation method comprises the following steps:

[0043] Step 1: Obtain the real energy consumption data of high energy consumption units to form the original sample data.

[0044] In one embodiment of the present invention, the actual energy consumption data of the high energy consumption unit includes: energy consumption and carbon dioxide emissions.

[0045] Step 2: Fill missing values ​​in the original sample data and construct a sample set.

[0046] In one embodiment of the present invention, a method of filling missing values ​​is adopted for the missing parts of real data, that is, using adjacent values ​​close to the missing values ​​for filling; for each missing value, find its nearest non-missing value and use it as the filling value; the method of filling missing values ​​utilizes the existing information in the data and retains the characteristics and distribution of the original data as much as possible; filling missing values ​​can reduce the information loss of data to a certain extent and ensure the integrity and availability of data.

[0047] When training the energy consumption data augmentation model in the embodiment of the present invention, a sliding window is used to divide the continuous time series data into windows of fixed size to facilitate the training and learning of the model; assuming that there is a sequence of time series data, represented by X = [x1, x2, ..., x n ], where N is the length of the sequence; the sliding window size is W, the sliding step is s, and the i-th window is represented by: X i =[x is ,x i+1 ,...,x is+W-1 ],in Assume W = 4, s = 1, such as Figure 2 FIG. 4 is a diagram showing the sliding process of the sliding window in an embodiment of the present invention.

[0048] Since the performance of the generative network model depends largely on the quality and richness of the training data, the sliding window technology can be used to extract multiple training samples from the original sequence data to expand the size of the data set while retaining the temporal correlation of the sequence data. These subsequences cover different parts of the original data and contain more data changes and patterns, which helps to improve the model's understanding and learning ability of the data, and improve the model's generalization ability and prediction accuracy.

[0049] Step 3: Introduce the self-attention mechanism and build an energy consumption data augmentation training model based on deep convolutional generative adversarial network.

[0050] In one embodiment of the present invention, in step 3, a self-attention mechanism is introduced to construct an energy consumption data augmentation training model based on a deep convolutional generative adversarial network, including:

[0051] Use deep convolutional networks to extract features from high-energy consumption data;

[0052] Adopting generative adversarial networks to augment high-energy consumption data based on features extracted by deep convolutional networks;

[0053] And introduce the self-attention mechanism to enhance the energy consumption data augmentation model's ability to capture long-distance dependencies in the data.

[0054] In this embodiment, the deep convolutional generative adversarial network (DCGAN) combines the convolutional network CNN and the generative adversarial network GAN. By stacking multiple convolutional layers and deconvolutional layers, it can effectively capture and generate complex features of data and use the powerful feature extraction ability of convolution operation to improve learning effect. Figure 3The figure is a diagram of the deep convolutional generative adversarial network structure in an embodiment of the present invention; the core structure of DCGAN includes two main parts: the generator and the discriminator. The generator and the discriminator compete with each other through a dynamic adversarial process to improve their respective performance until a Nash equilibrium is reached. In this process, the goal of the generator is to learn how to generate data from noise z to mimic the distribution P of real data. data (x), while the discriminator aims to accurately distinguish whether the input data comes from the original dataset or the data produced by the generator.

[0055] This process is essentially a zero-sum game, in which the objective functions of the generator and the discriminator are in opposition, that is, the loss of one entity is equal to the gain of the other entity. Therefore, the training process of the generator and the discriminator can be regarded as solving the minimax game problem, represented by the objective function V(D,G), which reflects the dynamic confrontation relationship between the two. Its expression is shown in formula (1):

[0056]

[0057] In the formula, G is the generator network, D is the discriminator network, and P data (x) represents the actual data sample obtained from the real data distribution and the prior distribution P z The noise sampled from (z) (usually a normal distribution or a uniform distribution) is denoted by z, D(x) represents the possibility that the discriminator correctly identifies the actual data as real data, and D(G(z)) is the possibility that the discriminator identifies the generated data from the generator as real data; when P g (x) = P data (x), the optimal solution of formula (1) is obtained, which is proved as follows:

[0058] First, we solve the optimal solution of the discriminator. Starting from the objective function, since V is continuous, we write V in the form of calculus to express the expectation:

[0059]

[0060] Assume that the data generated by G(z) is x, and find the expressions of the noise point z and the differential dz of the noise point respectively:

[0061]

[0062] Substituting z and dz, we get:

[0063]

[0064] Use P g (x) represents the generating distribution of z:

[0065]

[0066] Substituting into the loss function, we can get:

[0067]

[0068] Fix G to find the partial derivative of D and find the maximum value of the loss function with respect to D:

[0069]

[0070] From formula (7), we can get that when P g (x) = P data (x) The discriminator network can no longer distinguish the authenticity of the data generated by the generator. Based on the optimal solution of the discriminator, the solution of the generator can be obtained:

[0071]

[0072] The above expression is processed equivalently. According to the principle of fraction invariance, in the fraction in log, the numerator and denominator are divided by 2 at the same time, and then the denominator is kept unchanged and 1 / 2 of the numerator is moved outside. Then equation (8) can be equivalently transformed into:

[0073]

[0074] In one embodiment of the present invention, the definition of KL divergence is introduced:

[0075]

[0076] When the fitting distribution P g (x) = P data (x), KL = 0, so the minimum value of the generator network is -log4; at this time, while the discriminator network approaches its optimal solution, the generator network also infinitely approaches its minimum value.

[0077] Through the above game process, the generator continuously learns to generate more and more realistic data samples, while the discriminator continuously improves its accuracy in distinguishing true and false data until the system reaches a state of equilibrium. At this time, the generator can generate high-quality data samples, and the discriminator can also distinguish true and false efficiently and accurately.

[0078] In one embodiment of the present invention, when training an energy consumption data augmentation training model based on a deep convolutional generative adversarial network, it also includes: after cyclically training the discriminator for a set number of times, fixing the discriminator, training the generator, randomly collecting M noise samples, and inputting the collected M noise samples into the generator to generate samples; using the AdamW optimizer to update the parameters of the generator so that the generated samples are closer to the real data samples; alternatingly training the generator and the discriminator until the model converges to obtain a trained energy consumption data augmentation model.

[0079] In one embodiment of the present invention, in step 3, introducing the self-attention mechanism includes:

[0080] Accepts an input tensor with batch size, number of channels, height, and width of 64, 1, 48, and 48 respectively;

[0081] Randomly initialize the weight parameters q, k, v, and generate matrices of query (Query, Q), key (Key, K), and value (Value, V) through dot product operations;

[0082] Calculate the similarity between each element in the matrix and other elements, and normalize the similarity to the element's attention mechanism;

[0083] The output tensor of the self-attention mechanism is obtained by weighted summing each element in the matrix with its attention weight.

[0084] A self-attention mechanism is added on the basis of the deep convolutional generative adversarial network to enhance the energy consumption data augmentation model's ability to capture long-distance dependencies in the data and improve the quality and diversity of the generated data; for each element in Z, the self-attention mechanism calculates the similarity a between each element and other elements i,j , and normalize these similarities to the attention weight A'; by weighted summing each element with the corresponding attention weight, the output of the self-attention mechanism can be obtained; the specific steps are as follows:

[0085] Step 1: For each window in the input sequence, it is first converted into a 48×48 matrix through the embedding layer; then, these embedding matrices undergo three different linear transformations to generate the corresponding query (Query, Q), key (Key, K) and value (Value, V) matrices. The calculation formula is shown in formula (11):

[0086]

[0087] Where Z is the embedded representation of the input, W q , W k and W v is the weight matrix of the corresponding query, key, and value.

[0088] Step 2: Calculate the correlation score a between each two input vectors by dot product of the obtained Q and K matrices i,j , the calculation formula is shown in formula (12). The result of the dot product is a score matrix A, which represents the correlation between all positions in the input sequence.

[0089]

[0090] In the formula, a i,j It represents the value of the i-th row and j-th column in the attention weight matrix A.

[0091] Step 3: Use the normalized attention weight matrix A' and the V matrix to calculate the weighted sum, and finally obtain the output matrix B. The core of this step is to update the representation of each position based on the information of other positions in the input sequence, so as to capture the dependencies in the input sequence.

[0092]

[0093] The output matrix B, which further extracts features, is input into DCGAN to improve the quality of generated data. The introduction of the self-attention mechanism helps to integrate contextual information, allowing the model to more focus and accurately capture the salient features of the data during training.

[0094] In one embodiment of the present invention, the implementation of the technical solution of the present invention also includes: initializing the model parameters of the generator and the discriminator, wherein the model parameters of the generator and the discriminator are initialized randomly from a normal distribution with a zero mean and a standard deviation of 0.02, the batch size is 64, the learning rate is 0.0002, and the number of iterations is 2000. The initialization method using the normal distribution can provide a smaller weight value range, thereby ensuring that the output of the network will not be too large or too small in the initial stage, avoiding the problem of gradient disappearance and gradient explosion.

[0095] After initializing the model parameters of the generator and the discriminator, fix the generator and train the discriminator; randomly collect M real data samples and M noise samples, and input the collected real data samples and noise samples into the discriminator for discrimination; use the AdamW optimizer to update the parameters of the discriminator to improve the discrimination ability of the discriminator;

[0096] In one embodiment of the present invention, the AdamW optimizer updates the parameters of the discriminator according to the following steps:

[0097] Input the real samples and generated samples into the discriminator and calculate the loss of the discriminator;

[0098] Use the back-propagation algorithm to calculate the gradient of the loss function with respect to the discriminator parameters;

[0099] The AdamW optimizer is used to update the parameters of the discriminator according to the calculated gradients.

[0100] The AdamW algorithm updates the parameters of the discriminator based on the gradient of each parameter and its historical gradient information in order to converge to the optimal value faster. The updated parameters are shown in formula (14):

[0101]

[0102] Among them, θ t are the parameters of the discriminator, including weight ω and bias b, η is the learning rate, is the exponential moving average of the gradient, is the exponential moving average of the square of the gradient, ò is the noise, and λ is the weight decay coefficient, which is used to control the degree of weight decay.

[0103] Step 4: According to the energy consumption data augmentation training model based on deep convolutional generative adversarial network constructed in step 3, introduce the conditional constraint mechanism to build a high energy consumption data augmentation model based on the generative large model.

[0104] In one embodiment of the present invention, in step 4, a conditional constraint mechanism is introduced to construct a high-energy consumption data augmentation model based on a generative large model, including:

[0105] A conditional constraint mechanism is introduced to receive external seasonal condition information. The generator adjusts the data generation process according to the received seasonal conditions, so that the generated data gradually approaches the distribution of the real data. The discriminator compares the consistency of the generated data and the real data in terms of seasonal condition information, and feeds the comparison result back to the generator to optimize the generator.

[0106] The energy consumption data augmentation training model based on deep convolutional generative adversarial network constructed in step 3 is connected in series to form a high energy consumption data augmentation model based on the generative large model.

[0107] The high-energy consumption data set used in one embodiment of the present invention is the data of a steel company. In the steel industry, seasonal factors have a significant impact on both electricity consumption and carbon dioxide emissions. First, seasonal changes will directly affect the production and consumption needs of steel companies. The off-season of the steel industry is usually in winter, while the peak season is concentrated in spring. The increase in demand will lead to an increase in production, which will lead to an increase in corresponding electricity consumption and carbon dioxide emissions. Secondly, the low temperature in winter will increase the heating demand of the factory, further increasing electricity consumption and carbon dioxide emissions. Therefore, when building the model, in order to generate generated data consistent with the statistical characteristics of the real data, a conditional constraint mechanism is introduced; the conditional constraint mechanism receives the external seasonal information y as part of the input, which is fed into the generator together with the random noise. In this process, the median seasonal condition of the sub-window is taken as the seasonal condition of the sub-window, and is concatenated with random noise into a 48×48 matrix. The last column of the matrix represents the seasonal information. During the training process, the generator learns how to combine these seasonal condition information with random noise, receives input channels with a number, height, and width of 1, 48, and 48, extracts and compresses features through the convolution layer, and then restores the features to the target size (1×48×48) through the deconvolution layer to generate more realistic samples with richer features. The process is as follows: Figure 4 As shown in the figure, the hidden layer of the generator network consists of three convolutional layers and three deconvolutional layers. The number of convolution kernels in each layer is 16, 32, 64, 32, 16, and 1 respectively. The size of the convolution kernel is 4×4. All convolutional layers are implemented with a step size of 1 and the same padding. The discriminator also receives conditional information as part of the input, and processes the input sample and conditional information of each window into a 48×48 matrix to determine whether the sample meets the specified conditions, and outputs a probability value indicating the authenticity of the sample. The generator is iteratively tuned according to the result. The process is as follows Figure 5 At this time, the objective function is as follows:

[0108]

[0109] Where D(x│y) represents the probability that x is true under seasonal condition y; D(G(z│y)) represents the probability that the data generated by the generator is judged as true by the discriminator under condition y.

[0110] By connecting multiple sub-networks in series, the robustness and stability of the overall model can be significantly improved, ensuring that the overall model can maintain excellent performance even when the learning of a sub-network fluctuates. This structure of connecting multiple sub-networks in series is extremely flexible and scalable, allowing the number and architecture of sub-networks to be flexibly adjusted or increased according to actual application and resource requirements to adapt to a variety of data types and generation tasks.

[0111] like Figure 6 The diagram shows the network structure of the generative large model in the embodiment of the present invention. The present invention is a further improvement based on existing research. First, the generative adversarial network is replaced with a deep convolutional generative adversarial network to enhance the model's feature extraction capabilities; second, a self-attention mechanism is added to improve the model's ability to capture the internal correlation of the data; finally, by adding a conditional constraint mechanism, the model is allowed to consider the influence of external seasonal factors when generating data, thereby improving the authenticity of the generated data and making the generated data highly consistent with the actual data; in addition, the design of the serial sub-network increases the robustness of the model while also improving its flexibility and scalability.

[0112] Step 5: Use the sample set to train, test and verify the high-energy consumption data augmentation model based on the generative large model to obtain the trained high-energy consumption data augmentation model based on the generative large model.

[0113] In one embodiment of the present invention, in addition, for the sample set in the embodiment of the present invention, the training set, test set and validation set are divided in a ratio of 8:1:1 to ensure that the data for model training, validation and testing have temporal continuity, so as to more accurately evaluate the performance of the model.

[0114] In one embodiment of the present invention, after step 5, the real energy consumption data of the high energy consumption units are collected, and the high energy consumption data augmentation model of the trained generative large model is used to augment the high energy consumption data.

[0115] The training mechanism of the generative large model in the embodiment of the present invention is based on the adversarial principle, involving the optimization of the generator and the discriminator through alternating training. In each step of the training, the generator first tries to generate data, and then the discriminator evaluates the batch of generated data and the real data to distinguish which are real data and which are generated data. This evaluation result is then used to guide the generator to adjust its parameters, with the aim of improving the quality of its generated data and making the data generated by the generator more difficult to be recognized by the discriminator. At the same time, the discriminator also updates its parameters based on the real data and the data generated by the generator, aiming to improve its ability to distinguish between true and false data.

[0116] In order to improve the model performance and make full use of computing resources, the present invention adopts a multi-card distributed training framework to accelerate the calculation. Figure 7 As shown, the training steps are as follows:

[0117] 1) The parameters of the first node are synchronized to all training cards in the cluster through the Broadcast operation to ensure that the initial parameters of each computing node remain consistent. At the same time, the training script runs in parallel on multiple computing nodes, and each computing node contains the parameters of the overall model.

[0118] 2) Distribute the data samples into shards to each computing node (training card) in the entire cluster, and use data pipelining technology to load the data samples into the high-speed memory space of the training card as input.

[0119] 3) Each training card performs forward propagation on its assigned data samples and calculates the loss difference.

[0120] 4) Back propagate the calculated loss difference to obtain the gradient Gradient i. It should be noted that a gradient is calculated for each layer, and the gradient is represented in the form of an output tensor.

[0121] 5) The partial gradients calculated by all training cards are globally reduced (AllReduce) within and between hosts through collective communication to obtain the global gradient.

[0122] 6) Use this global gradient to update the parameters and iterate steps 2 to 5 to obtain new gradients.

[0123] By utilizing the computing resources of multiple computing nodes to process data in parallel, the model training process can be significantly accelerated. Each computing node can independently perform operations such as forward propagation and back propagation, allowing the training process to proceed at a higher speed.

[0124] In a specific embodiment of the present invention, based on the hardware environment NVIDIA GeForee GTX 2080Ti, IntelCore i9-9900K 3.60GHz CPU, memory of 32GB, programming software Python 3.6.0, machine learning package TensorFlow-CUDA 11.0. A public data set (Steel industry data) from a university is used, and the public data set contains the energy usage of a steel plant; the above data set is the energy consumption of the plant collected according to the load type and weekly status; the sampling interval is 15 seconds, and there are 35040 records in total; including energy consumption, carbon dioxide emissions, lagging reactive power, leading reactive power, lagging power factor, leading power factor, seconds calculated from midnight, working day status, and date.

[0125] Since energy consumption and carbon dioxide emissions are two key performance indicators in steel production, electricity consumption directly reflects the energy consumption required in steel production, while carbon dioxide emissions are one of the main greenhouse gases produced in steel production and have a significant impact on the environment. Secondly, electricity consumption and carbon dioxide emissions are usually relatively easy to obtain, because many steel production companies have corresponding monitoring systems to monitor electricity consumption and carbon dioxide emissions in real time. Based on the above reasons, the embodiment of the present invention selects electricity consumption and carbon dioxide emissions as the objects of data augmentation to train the energy consumption data augmentation model.

[0126] The energy consumption data augmentation model used in the embodiment of the present invention is used to augment the power consumption and carbon dioxide emissions; Figure 8 (a) shows the distribution of the raw data of electricity consumption of the above steel plant. Figure 8 (b) shows the distribution of power consumption data generated by the data augmentation model of the embodiment of the present invention; Figure 8 It can be seen from FIG. 1 that the power consumption data distribution generated by the data augmentation model of the embodiment of the present invention can maintain the characteristic distribution of the original power consumption data to a certain extent; Fig. 9 (a) shows the distribution of the raw source consumption data of carbon dioxide emissions from the above steel plants. Fig. 9 (b) shows the distribution of carbon dioxide emission data generated by the data augmentation model of the embodiment of the present invention; Fig. 9 It can be seen that the data distribution of carbon dioxide emissions generated by the data augmentation model of the embodiment of the present invention can maintain the characteristic distribution of the original carbon dioxide emissions data to a certain extent.

[0127] At the same time, the present invention realizes a high-energy consumption data augmentation system based on a generative large model, including: a data acquisition module, a data processing module, a model construction module and a model training module;

[0128] The data acquisition module is used to obtain the real energy consumption data of high energy consumption units to form the original sample data;

[0129] The data processing module uses window sliding technology to fill missing values ​​of the original sample data with adjacent values ​​to construct a sample set;

[0130] The model building module includes a deep convolutional generative adversarial network building module and a generative large model building module;

[0131] The deep convolutional generative adversarial network building module combines the self-attention mechanism to build an energy consumption data augmentation training model based on the deep convolutional generative adversarial network;

[0132] The generative large model construction module introduces a conditional constraint mechanism to build a high-energy consumption data augmentation model based on the generative large model based on the energy consumption data augmentation training model based on the deep convolutional generative adversarial network construction module;

[0133] The model training module uses the sample set to train, test and verify the high-energy consumption data augmentation model based on the generative large model, and obtains the trained high-energy consumption data augmentation model based on the generative large model.

[0134] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0135] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0136] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0137] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0138] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents, and any modifications or equivalent replacements that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A high-energy consumption data augmentation method based on a generative large model, characterized in that: include: Step 1: Obtain the real energy consumption data of high energy consumption units to form the original sample data; Step 2: Fill missing values ​​in the original sample data and construct a sample set; Step 3: Introduce the self-attention mechanism and build an energy consumption data augmentation training model based on deep convolutional generative adversarial network; Step 4: According to the energy consumption data augmentation training model based on deep convolutional generative adversarial network constructed in step 3, a conditional constraint mechanism is introduced to build a high energy consumption data augmentation model based on a generative large model; Step 5: Use the sample set to train, test and verify the high-energy consumption data augmentation model based on the generative large model to obtain the trained high-energy consumption data augmentation model based on the generative large model.

2. The high-energy consumption data augmentation method based on generative large model according to claim 1 is characterized in that: In step 1, the actual energy consumption data of high energy consumption units include energy consumption and carbon dioxide emissions.

3. The high-energy consumption data augmentation method based on generative large model according to claim 1, characterized in that: In step 2, filling missing values ​​in the original sample data includes: The sliding window technique is used to fill the missing values ​​of the original sample data with adjacent values.

4. The high-energy consumption data augmentation method based on generative large model according to claim 1, characterized in that: In step 3, the self-attention mechanism is introduced to build an energy consumption data augmentation training model based on deep convolutional generative adversarial network, including: Use deep convolutional networks to extract features from high-energy consumption data; Adopting generative adversarial networks to augment high-energy consumption data based on features extracted by deep convolutional networks; And introduce the self-attention mechanism to enhance the energy consumption data augmentation model's ability to capture long-distance dependencies in the data.

5. The high-energy consumption data augmentation method based on generative large model according to claim 4 is characterized in that: The introduction of the self-attention mechanism includes: Accepts an input tensor with batch size, number of channels, height, and width of 64, 1, 48, and 48 respectively; Randomly initialize the weight parameters q, k, v, and generate matrices of query (Query, Q), key (Key, K), and value (Value, V) through dot product operations; Calculate the similarity between each element in the matrix and other elements, and normalize the similarity to the element's attention weight; The output tensor of the self-attention mechanism is obtained by weighted summing each element in the matrix with its attention weight.

6. The high-energy consumption data augmentation method based on generative large model according to claim 1 or 4, characterized in that: In step 4, a conditional constraint mechanism is introduced to construct a high-energy consumption data augmentation model based on a generative large model, including: A conditional constraint mechanism is introduced to receive external seasonal condition information. The generator adjusts the data generation process according to the received seasonal conditions so that the generated data gradually approaches the distribution of the real data. The discriminator compares the consistency of the generated data and the real data in terms of seasonal condition information and feeds the comparison result back to the generator to optimize the generator. The energy consumption data augmentation training model based on deep convolutional generative adversarial network constructed in step 3 is connected in series to form a high energy consumption data augmentation model based on the generative large model.

7. A high-energy consumption data augmentation system based on a generative large model using the method according to any one of claims 1 to 6, the system comprising: The data acquisition module, the data processing module, the model building module and the model training module are characterized by: The data acquisition module is used to obtain the real energy consumption data of high energy consumption units to form the original sample data; The data processing module uses window sliding technology to fill missing values ​​of the original sample data with adjacent values ​​to construct a sample set; The model building module includes a deep convolutional generative adversarial network building module and a generative large model building module; The model training module uses the sample set to train, test and verify the high-energy consumption data augmentation model based on the generative large model, and obtains the trained high-energy consumption data augmentation model based on the generative large model.

8. The high-energy consumption data augmentation system based on generative large model according to claim 7, characterized in that: The deep convolutional generative adversarial network building module is used to combine the self-attention mechanism to build an energy consumption data augmentation training model based on the deep convolutional generative adversarial network; The generative large model construction module is used to introduce a conditional constraint mechanism to build a high-energy consumption data augmentation model based on the generative large model according to the energy consumption data augmentation training model based on the deep convolutional generative adversarial network construction module.

9. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1-6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Citation Information

Patent Citations

  • Image classification data augmentation method and system

    CN117593581A

  • Near-ground infrared target data set establishment method and device based on generative adversarial network

    CN113420639A

  • Power transmission line image augmentation realization method and system based on improved GAN

    CN113673671A

  • Image data augmentation method of power transmission and transformation equipment

    CN114037884A

  • Image realism predictor

    EP3675034A1

Cited By

  • Scenario-based sample editing method and system

    WO2026119147A1