Distributed model training method, device, equipment, medium and product
By generating adversarial networks to generate more comprehensive environmental data for timing prediction, dynamically adjusting the number of distributed model training processes, solving the problems of inaccurate prediction of new energy power generation and high energy consumption, and achieving efficient model training in the case of power fluctuations .
Patent Information
- Application Number
- CN202411111948.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-12-10
- Estimated Expiration
- 2044-08-14
AI Technical Summary
In the prior art, inaccurate prediction of new energy power generation results in model training that cannot adapt to fluctuations in power state, and large-scale model training requires a large amount of energy support, which has environmental protection and controllability problems.
By acquiring multi-dimensional environmental data, generating more comprehensive data using a generative adversarial network, performing timing predictions to obtain accurate predicted power generation, and dynamically adjusting the number of distributed model training processes based on the predicted values, optimizing model training efficiency and energy utilization.
It improves the accuracy of new energy power generation forecasts, can dynamically adjust the number of training processes under the fluctuations in power supply, optimize the training efficiency of distributed models, reduce energy consumption, and make full use of renewable energy.
Smart Images
Figure CN118643921B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a distributed model training method, device, equipment, medium and product. Background Art
[0002] We have entered the era of large models, but the booming development of large models requires huge computing power support, resulting in a large amount of energy consumption. The current model training requires a large amount of stable electricity support. Thermal power has the advantage of stability, but it has disadvantages such as environmental pollution and high prices. The power generation of new energy has the advantages of sufficient, environmental protection, and high cost performance, but it has the influence of the external environment, resulting in greater volatility and low controllability. New energy has not become the main power generation.
[0003] The existing technology uses time series prediction models to predict power generation, but the current time series prediction models are not easy to implement when predicting power generation. The main difficulty lies in the collection of new energy data. There are many factors that affect new energy power generation, and the data of some factors is difficult to collect. However, less data leads to less accurate model prediction results. Therefore, when the power generation forecast value is inaccurate, model training cannot adapt to the fluctuating power state. Summary of the invention
[0004] The present invention provides a distributed model training method, device, equipment, medium and product, which are used to solve the problem in the prior art that when the power generation prediction value is inaccurate, the model training cannot adapt to the fluctuating power state.
[0005] In a first aspect, the present invention provides a distributed model training method, comprising:
[0006] Acquiring first multi-dimensional environmental data;
[0007] Inputting the first multi-dimensional environmental data into a data generation model to obtain second multi-dimensional environmental data output by the data generation model; the data generation model is obtained by training based on a generative adversarial network;
[0008] Perform time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period;
[0009] Based on the predicted value of power generation within the future preset time period, the number of training processes started during the distributed model training process is adjusted to train the distributed model.
[0010] In one embodiment, there are N training nodes in the distributed model training process; adjusting the number of training processes started in the distributed model training process based on the power generation prediction value in the future preset time period to train the distributed model includes:
[0011] Fixed batch size for a single training process;
[0012] Start a training process on a training node, and record the overall power consumption of the training node until the training is stable;
[0013] Add a training node and start a training process on each training node until the training is stable and record the overall power consumption of all training nodes;
[0014] Iteratively perform the steps of adding a training node and starting a training process on each training node until the overall power consumption of all training nodes is recorded when the training is stable, until N training nodes are added, and the overall power consumption of the N training nodes when the training is stable is obtained;
[0015] Associate the overall power consumption recorded each time with the corresponding number of training processes to obtain a mapping table;
[0016] Based on the predicted value of power generation within the future preset time period and the mapping table, the number of training processes started during the distributed model training process is adjusted to train the distributed model.
[0017] In one embodiment, adjusting the number of training processes started in the distributed model training process based on the power generation prediction value in the future preset time period and the mapping table to train the distributed model includes:
[0018] Based on the predicted value of power generation in the future preset time period, determine the power consumption of the model training provided in the future preset time period;
[0019] Matching the target number of training processes associated with the model training power consumption in the mapping table;
[0020] Initialize the distributed training environment according to the target number of training processes;
[0021] The distributed model is trained in the distributed training environment.
[0022] In one embodiment, after adjusting the number of training processes started in the distributed model training process based on the power generation prediction value within the future preset time period to train the distributed model, the method further includes:
[0023] Evaluate whether the distributed model after this round of training meets the accuracy requirements;
[0024] If the distributed model after the current round of training meets the requirements, the distributed model after the current round of training is saved and the distributed model training process ends;
[0025] If the distributed model after this round of training does not meet the standard, the step of performing time series prediction based on the first multidimensional environmental data and the second multidimensional environmental data to obtain the predicted value of power generation within a preset time period in the future is re-executed until the distributed model after this round of training meets the standard, the distributed model after this round of training is saved, and the distributed model training process is ended.
[0026] In one embodiment, performing time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period includes:
[0027] Integrating the first multi-dimensional environment data and the second multi-dimensional environment data to obtain target multi-dimensional environment data;
[0028] Inputting the target multi-dimensional environmental data into a time series prediction model to obtain a power generation prediction value within a future preset time period output by the time series prediction model;
[0029] The time series prediction model is trained in the following way:
[0030] Acquire a first historical environmental data sample and a historical power generation data sample;
[0031] Inputting the first historical environment data sample into the data generation model to obtain a second historical environment data sample output by the data generation model;
[0032] Integrate the first historical environment data sample and the second historical environment data sample into a target historical environment data sample;
[0033] Based on the historical power generation data sample, determining label data of the target historical environment data sample;
[0034] Model training is performed based on the second historical environment data sample and the corresponding label data to obtain the time series prediction model.
[0035] In one embodiment, the data generation model includes a generator and a discriminator; the data generation model is trained in the following manner:
[0036] Obtain multi-dimensional environmental data samples;
[0037] Randomly setting multiple parameters in the multidimensional environmental data sample to zero to obtain an initial environmental data sample;
[0038] Inputting the initial environment data sample into a generator to obtain a target environment data sample output by the generator;
[0039] Based on the multidimensional environment data sample and the target environment data sample, authenticity discrimination is performed by a discriminator to obtain a true prediction probability of the target environment data sample;
[0040] Calculating a loss value based on the true predicted probability of the target environment data sample;
[0041] Based on the loss value, the gradient of the generator parameters and the gradient of the discriminator parameters are calculated by the back propagation algorithm;
[0042] By means of an optimization algorithm, the generator parameters are updated according to the gradient of the generator parameters, and the discriminator parameters are updated according to the gradient of the discriminator parameters;
[0043] The steps of inputting the initial environment data sample into the generator to obtain the target environment data sample output by the generator are iteratively executed until both the generator and the discriminator meet the standards, thereby obtaining a trained data generation model.
[0044] In a second aspect, the present invention further provides a distributed model training device, comprising:
[0045] An acquisition module, used for acquiring first multi-dimensional environmental data;
[0046] A data generation module, used for inputting the first multi-dimensional environment data into a data generation model to obtain second multi-dimensional environment data output by the data generation model; the data generation model is obtained by training based on a generative adversarial network;
[0047] A power generation prediction module, used to perform time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a power generation prediction value within a future preset time period;
[0048] The distributed model training module is used to adjust the number of training processes started during the distributed model training process based on the power generation prediction value within the future preset time period to train the distributed model.
[0049] In a third aspect, the present invention provides a device, comprising an electronic device, wherein the electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described distributed model training methods are implemented.
[0050] In a fourth aspect, the present invention further provides a medium, comprising a non-transitory computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the distributed model training method as described in any one of the above are implemented.
[0051] In a fifth aspect, the present invention also provides a product, comprising a computer program product, wherein the computer program product comprises a computer program, wherein the computer program can be stored on a non-transitory computer-readable storage medium, and wherein when the computer program is executed by the processor, the steps of the distributed model training method as described above are implemented.
[0052] The distributed model training method, device, equipment, medium and product provided by the present invention obtain first multi-dimensional environmental data of multiple dimensions, generate new data with similar statistical characteristics by generating adversarial network learning data distribution, and obtain second multi-dimensional environmental data, which is helpful to expand the data set or enhance the diversity of data to generate more comprehensive environmental data, and perform time series prediction through more comprehensive environmental data to accurately predict the power generation prediction value within a preset time period in the future. Further, based on the accurate power generation prediction value, the number of training processes started during distributed model training can be dynamically adjusted to make full use of renewable energy to realize the training of distributed models under the state of power supply fluctuations and optimize the efficiency of distributed model training. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0054] Figure 1 This is one of the flow charts of the distributed model training method provided by the present invention.
[0055] Figure 2 This is the second flow chart of the distributed model training method provided by the present invention.
[0056] Figure 3 It is a structural schematic diagram of the distributed model training device provided by the present invention.
[0057] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0058] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] The terms "first", "second", etc. in the present invention are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described herein.
[0060] Combine the following Figure 1-Figure 4 Describe the distributed model training method, device, equipment, medium and product provided by the present invention.
[0061] Combination Figure 1 and Figure 2 , Figure 1 This is one of the flow charts of the distributed model training method provided by the present invention. Figure 2 This is the second flow chart of the distributed model training method provided by the present invention.
[0062] like Figure 1 As shown, the method includes the following:
[0063] Step 101: Acquire first multi-dimensional environmental data;
[0064] Step 102: input the first multi-dimensional environment data into a data generation model to obtain second multi-dimensional environment data output by the data generation model;
[0065] Step 103: performing time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period;
[0066] Step 104: Based on the predicted value of power generation within the future preset time period, adjust the number of training processes started during the distributed model training process to train the distributed model.
[0067] It should be noted that the distributed model training method provided in the embodiment of the present invention is implemented based on a distributed model training device. In order to solve the problem that the prior art cannot predict power generation based on a small amount of environmental data and cannot cope with the elastic changes in power supply, the embodiment of the present invention provides a method based on real-time power prediction and elastic distributed model training, which can make full use of renewable energy and realize the training of distributed models under the condition of fluctuating power supply.
[0068] The embodiment of the present invention describes the distributed model training method by taking a distributed model training device as an execution subject as an example.
[0069] Specifically, the distributed model training device collects environmental data of multiple dimensions in the current time period, and integrates the environmental data of multiple dimensions into first multi-dimensional environmental data. The environmental data of multiple dimensions include but are not limited to parameters such as longitude and latitude, time, light intensity, air humidity, air pressure, cloud cover, air quality, wind direction and speed, and historical power generation.
[0070] It should be noted that through the above-mentioned environmental data, it is possible to effectively analyze how much electricity renewable energy can convert into in the future. Renewable energy can be converted into electricity through photovoltaic power generation and wind power generation, etc., which are not restricted here. In practical applications, photovoltaic power generation is usually used to achieve electricity supply because the technology maturity and adaptability of photovoltaic power generation are relatively wide, and the power generation can be predicted more stably.
[0071] Taking solar energy as an example, photovoltaic power generation is easily affected by natural weather, resulting in periodic fluctuations in power generation, so the utilization rate is not high, and a large amount of solar energy cannot be fully utilized. If the power generation can be predicted through a time series prediction model based on the existing environmental data, the utilization rate of solar energy can be greatly improved. However, the current time series prediction model is not easy to implement when predicting photovoltaic power. The main difficulty lies in the collection of photovoltaic data, that is, the multi-dimensional environmental data mentioned above. There are many factors that affect photovoltaic power generation, such as regional climate, weather, light, shadow, dust cover, etc., and the data of these factors are difficult to collect. Less data will lead to inaccurate model prediction results.
[0072] Since the generator in the generative adversarial network can generate new samples by learning data distribution, the trained generator can be used to generate more comprehensive environmental data based on incomplete data. Further, based on the more comprehensive environmental data, the results predicted by the model are more accurate.
[0073] Furthermore, the distributed model training device inputs the first multidimensional environmental data into the data generation model to obtain the second multidimensional environmental data output by the data generation model, wherein the data generation model is obtained by training based on a generative adversarial network.
[0074] Therefore, it can be understood that the distributed model training device inputs the first multidimensional environment data into the data generation model, generates new real environment data through the generator in the data generation model, that is, the second multidimensional environment data, and outputs the second multidimensional environment data. Figure 2 The steps are shown in Step 1.
[0075] Furthermore, the distributed model training device integrates the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain target multi-dimensional environmental data.
[0076] Furthermore, the distributed model training device inputs the target multi-dimensional environmental data into the time series prediction model to obtain the power generation prediction value within the future preset time period output by the time series prediction model, which can be referred to Figure 2 As shown in Step 2 in , the future preset time period is set according to the actual situation, and can be set to 2 hours or 3 hours, because the power generation forecast trend in this short period of time is generally relatively flat.
[0077] Furthermore, the distributed model training device adjusts the number of training processes started during the distributed model training process based on the predicted value of power generation within a preset time period in the future to train the distributed model.
[0078] It should be noted that due to the large size of the model and the large amount of training data, researchers and developers often use distributed training technology to speed up model training, such as distributed data parallelism. Distributed data parallelism is to start several training processes at the same time, each using a copy of the training data, and synchronize data between processes in the forward and reverse processes, thereby ensuring that the model parameters of each process are consistent after each training iteration. It can be seen that the current training of artificial intelligence models requires a stable power supply, because currently training and optimization are based on a fixed training process, a fixed number of accelerator cards, and a fixed batch data size. Therefore, the power consumption during the training process is stable, and the training cluster, that is, multiple training nodes, must also provide stable power support when training the model, otherwise normal training will not be possible.
[0079] Therefore, when the power generation can be predicted more accurately, the fluctuating electricity can be fully utilized to determine the number of training processes initiated by the distributed model training process, thereby simulating the distributed training environment to train the model. Through this dynamic training model method, the consumption of non-renewable energy can be reduced.
[0080] The distributed model training method provided by the present invention obtains first multi-dimensional environmental data of multiple dimensions, generates new data with similar statistical characteristics by generating an adversarial network learning data distribution, and obtains second multi-dimensional environmental data, which helps to expand the data set or enhance the diversity of data to generate more comprehensive environmental data, and perform time series prediction through the more comprehensive environmental data to accurately predict the power generation prediction value within a preset time period in the future. Further, based on the accurate power generation prediction value, the number of training processes started during distributed model training can be dynamically adjusted to make full use of renewable energy to realize the training of distributed models under the state of power supply fluctuations and optimize the efficiency of distributed model training.
[0081] Further, based on step 104, adjusting the number of training processes started in the distributed model training process based on the predicted value of power generation in the future preset time period to train the distributed model includes:
[0082] Fixed batch size for a single training process;
[0083] Start a training process on a training node, and record the overall power consumption of the training node until the training is stable;
[0084] Add a training node and start a training process on each training node until the training is stable and record the overall power consumption of all training nodes;
[0085] Iteratively perform the steps of adding a training node and starting a training process on each training node until the overall power consumption of all training nodes is recorded when the training is stable, until N training nodes are added, and the overall power consumption of the N training nodes when the training is stable is obtained;
[0086] Associate the overall power consumption recorded each time with the corresponding number of training processes to obtain a mapping table;
[0087] Based on the predicted value of power generation within the future preset time period and the mapping table, the number of training processes started during the distributed model training process is adjusted to train the distributed model.
[0088] It should be noted that the embodiments of the present invention are mainly used for large-scale training, and therefore a distributed training method is adopted. Assuming that there are a total of N available training nodes during large-scale training, a training node is capable of starting at least one training process for a distributed model (the embodiments of the present invention are explained by taking the example that a training node can start a training process), and each training node has M acceleration cards, which can be used to start a training process. The more acceleration cards are used, the more power the training node consumes. Therefore, it is necessary to know the total power consumed when starting different numbers of training processes during model training. Generally, a server can support a training node to be able to start a training process. If it cannot be supported, n servers can be set to support a training node to be able to start a training process (the embodiments of the present invention are explained by taking the example that a server can support a training node to be able to start a training process). The following content can be referred to. Figure 2 The steps are shown in Step 3.
[0089] Specifically, the distributed model training device fixes the batch data size of a single training process. Therefore, the acceleration card used to start each training process to execute the model training process is the same, and the power consumed is also the same, which facilitates the calculation of the total power consumption.
[0090] Furthermore, the distributed model training device starts a training process on a training node and records the overall power consumption of a training node until the training is stable, which can be considered as the overall power consumption of a service node.
[0091] Furthermore, the distributed model training device adds a training node and starts a training process on each training node until the training is stable and the overall power consumption of all training nodes is recorded.
[0092] Furthermore, the distributed model training device iteratively executes the steps of adding a training node and starting a training process on each training node until the training is stable and recording the overall power consumption of all training nodes, until N training nodes are added and the overall power consumption of the N training nodes is obtained when the training is stable.
[0093] In one embodiment, the method of recording the overall power consumption N times may be: (1) fixing the batch data size of a single training process, starting a training process on a training node, and recording the overall power consumption of a server node when the training is stable, which is recorded as ; (2) Keep the batch data size of a single training process unchanged, start a training process on each of the two training nodes, and when the training is stable, record the overall power consumption of the two server nodes at this time, which is recorded as ; (3) Similarly, gradually record the overall power consumption when adding a training node , , ...., until N training nodes are added, a training process is started on each of the N training nodes, and when the training is stable, the overall power consumption of the N server nodes is recorded. Finally, the total power consumption when starting different numbers of training processes is obtained. , , , , ..., .
[0094] Furthermore, the distributed model training device associates the overall power consumption recorded each time with the corresponding number of training processes to obtain a mapping table between the overall power consumption and the number of training processes.
[0095] The embodiment of the present invention performs power calibration on a large-scale training model, determines the overall power consumption of the model when different numbers of training processes are started during training, and generates a mapping table between the overall power consumption and the number of training processes. The number of training processes for each round of training is further determined based on the predicted value of power generation in a preset time period in the future and the mapping table, so as to dynamically adjust the number of training processes for distributed training to achieve flexible training. It can make full use of renewable energy to train distributed models under the condition of fluctuating power supply and optimize the efficiency of distributed model training.
[0096] Further, the adjusting the number of training processes started in the distributed model training process based on the power generation prediction value in the future preset time period and the mapping table to train the distributed model includes:
[0097] Based on the predicted value of power generation in the future preset time period, determine the power consumption of the model training provided in the future preset time period;
[0098] Matching the target number of training processes associated with the model training power consumption in the mapping table;
[0099] Initialize the distributed training environment according to the target number of training processes;
[0100] The distributed model is trained in the distributed training environment.
[0101] The following content can be referred to Figure 2 See Step 4 in the previous section.
[0102] Specifically, the distributed model training device determines the power consumption of the model training provided in the future preset time period based on the predicted value of the power generation in the future preset time period.
[0103] Furthermore, if the power consumption of model training is less than the overall power consumption of a training node, that is, the minimum power consumption, the distributed model training device will suspend training, wait for a period of time, and then re-predict the power generation forecast value in the future preset time period to determine whether it is sufficient to support the distributed model training.
[0104] Furthermore, if the power consumption of model training is greater than or equal to the overall power consumption of a training node, the distributed model training device matches the number of training processes associated with the model training power consumption in the mapping table, which is the target number of training processes that can be supported by this round of model training.
[0105] Furthermore, the distributed model training device initializes the distributed training environment according to the target number of training processes.
[0106] Furthermore, the distributed model training device adjusts the learning rate in a distributed training environment and starts training the distributed model. Furthermore, the model is saved at the end of this round of adjustment training.
[0107] The embodiment of the present invention determines the number of training processes for each round of training based on the power generation prediction value and mapping table within a preset time period in the future, and dynamically adjusts the number of training processes for distributed training to achieve flexible training. It can make full use of renewable energy to train distributed models under the condition of fluctuating power supply and optimize the efficiency of distributed model training.
[0108] Further, after adjusting the number of training processes started in the distributed model training process based on the power generation prediction value in the future preset time period to train the distributed model, it also includes:
[0109] Evaluate whether the distributed model after this round of training meets the accuracy requirements;
[0110] If the distributed model after the current round of training meets the requirements, the distributed model after the current round of training is saved and the distributed model training process ends;
[0111] If the distributed model after this round of training does not meet the standard, the step of performing time series prediction based on the first multidimensional environmental data and the second multidimensional environmental data to obtain the predicted value of power generation within a preset time period in the future is re-executed until the distributed model after this round of training meets the standard, the distributed model after this round of training is saved, and the distributed model training process is ended.
[0112] The following content can be referred to Figure 2 See Step 4 in the previous section.
[0113] Specifically, the distributed model training device evaluates whether the distributed model after this round of training meets the accuracy standards. Whether the distributed model meets the accuracy standards can be comprehensively evaluated from aspects such as performance indicator evaluation, model performance evaluation, system stability evaluation, and benefit evaluation.
[0114] Furthermore, if the distributed model after this round of training meets the requirements, the distributed model training device saves the distributed model after this round of training and ends the distributed model training process.
[0115] Furthermore, if the distributed model after this round of training does not meet the standards, the distributed model training device will re-execute the step of inputting the second multidimensional environmental data into the time series prediction model to obtain the predicted value of power generation within a preset future time period output by the time series prediction model until the distributed model after this round of training meets the standards, save the distributed model after this round of training, and end the distributed model training process.
[0116] The embodiment of the present invention evaluates the training effect of the distributed model. If it does not meet the standard, it continues to determine the number of training processes for distributed training and executes the next round of training until the model meets the standard, so as to ensure that the distributed model after multiple rounds of training meets the expected standard. Through an effective feedback loop and dynamic adjustment of the number of training processes, the model performance and model training efficiency are improved to achieve the best training results.
[0117] Furthermore, performing time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period includes:
[0118] Integrating the first multi-dimensional environment data and the second multi-dimensional environment data to obtain target multi-dimensional environment data;
[0119] Inputting the target multi-dimensional environmental data into a time series prediction model to obtain a power generation prediction value within a future preset time period output by the time series prediction model;
[0120] The time series prediction model is trained in the following way:
[0121] Acquire a first historical environmental data sample and a historical power generation data sample;
[0122] Inputting the first historical environment data sample into the data generation model to obtain a second historical environment data sample output by the data generation model;
[0123] Integrate the first historical environment data sample and the second historical environment data sample into a target historical environment data sample;
[0124] Based on the historical power generation data sample, determining label data of the target historical environment data sample;
[0125] Model training is performed based on the second historical environment data sample and the corresponding label data to obtain the time series prediction model.
[0126] Specifically, the distributed model training device integrates the first multidimensional environmental data and the second multidimensional environmental data to obtain target multidimensional environmental data, that is, integrates the original multidimensional environmental data with the newly generated second multidimensional environmental data, and performs subsequent time series prediction through more data.
[0127] Furthermore, the distributed model training device inputs the target multi-dimensional environmental data into the time series prediction model to obtain the power generation prediction value within a future preset time period output by the time series prediction model.
[0128] The following describes the process of training the time series prediction model.
[0129] The distributed model training device obtains the first historical environment data samples and the historical power generation data samples within the historical time period, and normalizes the obtained first historical environment data samples and the historical power generation data samples.
[0130] Furthermore, the distributed model training device inputs the first historical environment sample into the data generation model to obtain a second historical environment data sample output by the data generation model.
[0131] Furthermore, the distributed model training device integrates the first historical environment data sample and the second historical environment data sample into a target historical environment data sample.
[0132] Furthermore, the distributed model training device integrates the target historical environmental data samples into an environmental data sample sequence in time sequence.
[0133] Furthermore, the distributed model training device determines the historical power generation data samples as label data of the environmental data sample sequence according to the time sequence.
[0134] Furthermore, the distributed model training device performs model training based on the environmental data sample sequence and its corresponding label data to obtain a time series prediction model.
[0135] In one embodiment, according to the internal structure of the time series prediction model, the training method of the time series prediction model may include: (1) obtaining an environmental data sample sequence and its corresponding label data; (2) passing the environmental data sample sequence through two linear connection layers to obtain 128-dimensional data, which is the extracted feature; (3) sending the obtained feature to a long short-term memory network (Long Short-Term Memory, LSTM), setting the time step to 32, and obtaining the vector output by the hidden layer in the LSTM; (4) passing the vector output by the hidden layer through a continuous convolution layer and a linear connection layer, and the linear connection layer outputs the predicted value of power generation at the next moment; (5) performing mean square error MSE on the predicted value of power generation and the label data, and back propagating; (6) optimizing the weights of the neural network through the Adam optimizer to minimize the value of the loss function. Through the iterative training in the above manner, a trained time series prediction model can be obtained.
[0136] The embodiment of the present invention uses a trained time series prediction model to perform time series prediction based on relatively comprehensive environmental data, and accurately predict the power generation forecast value within a preset time period in the future. Furthermore, based on the accurate power generation forecast value, the number of training processes started during distributed model training can be dynamically adjusted to make full use of renewable energy to realize the training of distributed models under the condition of fluctuating power supply and optimize the efficiency of distributed model training.
[0137] Furthermore, the data generation model is trained in the following way:
[0138] Obtain multi-dimensional environmental data samples;
[0139] Randomly setting multiple parameters in the multidimensional environmental data sample to zero to obtain an initial environmental data sample;
[0140] Inputting the initial environment data sample into a generator to obtain a target environment data sample output by the generator;
[0141] Based on the multidimensional environment data sample and the target environment data sample, authenticity discrimination is performed by a discriminator to obtain a true prediction probability of the target environment data sample;
[0142] Calculating a loss value based on the true predicted probability of the target environment data sample;
[0143] Based on the loss value, the gradient of the generator parameters and the gradient of the discriminator parameters are calculated by the back propagation algorithm;
[0144] By means of an optimization algorithm, the generator parameters are updated according to the gradient of the generator parameters, and the discriminator parameters are updated according to the gradient of the discriminator parameters;
[0145] The steps of inputting the initial environment data sample into the generator to obtain the target environment data sample output by the generator are iteratively executed until both the generator and the discriminator meet the standards, thereby obtaining a trained data generation model.
[0146] It should be noted that the generative adversarial network consists of a generator and a discriminator. The generator is generally a codec structure that extracts information based on the input and reconstructs the output. The discriminator distinguishes between the label and the reconstructed output. When the discriminator cannot accurately distinguish, it can be considered that the output of the generator can simulate the real label.
[0147] Specifically, the distributed model training device selects one or more data sets with sufficient data volume and data dimension as training sets, that is, obtains multidimensional environmental data samples, and uses the obtained multidimensional environmental data samples as training sets, wherein the constructed data set should try to include parameters such as longitude and latitude, time, light intensity, air humidity, air pressure, cloud cover, air quality, wind direction and speed, and historical power generation.
[0148] Furthermore, the distributed model training device unifies the data format of the multi-dimensional environmental data samples.
[0149] Furthermore, the distributed model training device performs data preprocessing on the multidimensional environmental data samples in a unified data format. The data preprocessing specifically involves randomly setting some parameters to zero to obtain initial environmental data samples. This method can simulate actual environmental data. It should be noted that parameters such as longitude, latitude and time can be kept all the time and not set to zero.
[0150] Furthermore, the distributed model training device generates target environment data samples through a generator based on the initial environment data samples.
[0151] Furthermore, the distributed model training device performs authenticity discrimination through a discriminator based on the multi-dimensional environmental data samples and the target environmental data samples to obtain the true prediction probability of the target environmental data samples.
[0152] It should be noted that since the adversarial generative network has two parts and both parts have outputs, the error calculation can be performed between the two parts of the output and the label (i.e., the original real data-multidimensional environmental data sample), and the appropriate loss function can be selected, such as the Mean-Square Error (MSE) and Binary Cross-Entropy function, to jointly perform supervised training to accelerate model convergence.
[0153] Furthermore, the distributed model training device calculates the loss value through the loss function according to the true prediction probability of the target environment data sample.
[0154] Furthermore, the distributed model training device calculates the gradient of the generator parameters and the gradient of the discriminator parameters through the back propagation algorithm based on the loss value.
[0155] Furthermore, the distributed model training device updates the generator parameters according to the gradient of the generator parameters and updates the discriminator parameters according to the gradient of the discriminator parameters through an optimization algorithm, such as an Adaptive Moment Estimation (Adam) optimization algorithm.
[0156] Furthermore, the distributed model training device iteratively executes the steps of generating target environment samples through the generator based on the initial environment data samples until both the generator and the discriminator meet the standards, obtains a trained data generation model, saves the model and exits the model training process.
[0157] It should be noted that if the discriminator cannot distinguish between the generated data and the original data, it can be considered that the generator can generate actual environmental data based on insufficient environmental data, and can be received and used by the embodiment of the present invention to save the model.
[0158] The data generation model trained in the embodiment of the present invention can simulate real environmental data and integrate it into the original environmental data, which helps to expand the data set or enhance the diversity of data to generate more comprehensive environmental data, and then perform time series prediction through more comprehensive environmental data to accurately predict the power generation forecast value within a preset time period in the future, allowing the use of fewer environmental parameter capture devices to obtain real environmental data, reducing hardware equipment requirements and costs, and allowing more accurate power generation forecast values to be predicted under conditions with less real environmental data, which can improve the flexibility of predicting power generation.
[0159] The distributed model training device provided by the present invention is described below. The distributed model training device described below and the distributed model training method described above can be referenced to each other.
[0160] Reference Figure 3 , Figure 3 It is a structural schematic diagram of the distributed model training device provided by the present invention.
[0161] The distributed model training device comprises:
[0162] The acquisition module 310 is used to acquire first multi-dimensional environmental data.
[0163] The data generation module 320 is used to input the first multi-dimensional environmental data into a data generation model to obtain second multi-dimensional environmental data output by the data generation model; the data generation model is obtained by training based on a generative adversarial network.
[0164] The power generation prediction module 330 is used to perform time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a power generation prediction value within a future preset time period.
[0165] The distributed model training module 340 is used to adjust the number of training processes started during the distributed model training process based on the power generation prediction value within the future preset time period, so as to train the distributed model.
[0166] The distributed model training device provided by the present invention obtains first multi-dimensional environmental data of multiple dimensions, generates new data with similar statistical characteristics by generating an adversarial network learning data distribution, and obtains second multi-dimensional environmental data, which helps to expand the data set or enhance the diversity of data to generate more comprehensive environmental data, and perform time series prediction through the more comprehensive environmental data to accurately predict the power generation prediction value within a preset time period in the future. Further, based on the accurate power generation prediction value, the number of training processes started during distributed model training can be dynamically adjusted to make full use of renewable energy to realize the training of distributed models under the state of power supply fluctuations and optimize the efficiency of distributed model training.
[0167] Furthermore, the distributed model training module 340 is also used for:
[0168] Fixed batch size for a single training process;
[0169] Start a training process on a training node, and record the overall power consumption of the training node until the training is stable;
[0170] Add a training node and start a training process on each training node until the training is stable and record the overall power consumption of all training nodes;
[0171] Iteratively perform the steps of adding a training node and starting a training process on each training node until the overall power consumption of all training nodes is recorded when the training is stable, until N training nodes are added, and the overall power consumption of the N training nodes when the training is stable is obtained;
[0172] Associate the overall power consumption recorded each time with the corresponding number of training processes to obtain a mapping table;
[0173] Based on the predicted value of power generation within the future preset time period and the mapping table, the number of training processes started during the distributed model training process is adjusted to train the distributed model.
[0174] Furthermore, the distributed model training module 340 is also used for:
[0175] Based on the predicted value of power generation in the future preset time period, determine the power consumption of the model training provided in the future preset time period;
[0176] Matching the target number of training processes associated with the model training power consumption in the mapping table;
[0177] Initialize the distributed training environment according to the target number of training processes;
[0178] The distributed model is trained in the distributed training environment.
[0179] Furthermore, the distributed model training module 340 is also used for:
[0180] Evaluate whether the distributed model after this round of training meets the accuracy requirements;
[0181] If the distributed model after the current round of training meets the requirements, the distributed model after the current round of training is saved and the distributed model training process ends;
[0182] If the distributed model after this round of training does not meet the standard, the step of performing time series prediction based on the first multidimensional environmental data and the second multidimensional environmental data to obtain the predicted value of power generation within a preset time period in the future is re-executed until the distributed model after this round of training meets the standard, the distributed model after this round of training is saved, and the distributed model training process is ended.
[0183] Furthermore, the distributed model training device is also used for:
[0184] Integrating the first multi-dimensional environment data and the second multi-dimensional environment data to obtain target multi-dimensional environment data;
[0185] Inputting the target multi-dimensional environmental data into a time series prediction model to obtain a power generation prediction value within a future preset time period output by the time series prediction model;
[0186] The time series prediction model is trained in the following way:
[0187] Acquire a first historical environmental data sample and a historical power generation data sample;
[0188] Inputting the first historical environment data sample into the data generation model to obtain a second historical environment data sample output by the data generation model;
[0189] Integrate the first historical environment data sample and the second historical environment data sample into a target historical environment data sample;
[0190] Based on the historical power generation data sample, determining label data of the target historical environment data sample;
[0191] Model training is performed based on the second historical environment data sample and the corresponding label data to obtain the time series prediction model.
[0192] Furthermore, the distributed model training device is also used for:
[0193] Obtain multi-dimensional environmental data samples;
[0194] Randomly setting multiple parameters in the multidimensional environmental data sample to zero to obtain an initial environmental data sample;
[0195] Inputting the initial environment data sample into a generator to obtain a target environment data sample output by the generator;
[0196] Based on the multidimensional environment data sample and the target environment data sample, authenticity discrimination is performed by a discriminator to obtain a true prediction probability of the target environment data sample;
[0197] Calculating a loss value based on the true predicted probability of the target environment data sample;
[0198] Based on the loss value, the gradient of the generator parameters and the gradient of the discriminator parameters are calculated by the back propagation algorithm;
[0199] By means of an optimization algorithm, the generator parameters are updated according to the gradient of the generator parameters, and the discriminator parameters are updated according to the gradient of the discriminator parameters;
[0200] The steps of inputting the initial environment data sample into the generator to obtain the target environment data sample output by the generator are iteratively executed until both the generator and the discriminator meet the standards, thereby obtaining a trained data generation model.
[0201] Figure 4 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the distributed model training method, which includes: obtaining first multi-dimensional environmental data; inputting the first multi-dimensional environmental data into a data generation model to obtain second multi-dimensional environmental data output by the data generation model; the data generation model is obtained by training based on a generative adversarial network; performing time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period; adjusting the number of training processes started during the distributed model training process based on the predicted value of power generation within the future preset time period to train the distributed model.
[0202] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0203] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the distributed model training method provided by the above-mentioned embodiments, and the method includes: obtaining first multidimensional environmental data; inputting the first multidimensional environmental data into a data generation model to obtain second multidimensional environmental data output by the data generation model; the data generation model is obtained by training based on a generative adversarial network; performing time series prediction based on the first multidimensional environmental data and the second multidimensional environmental data to obtain a predicted value of power generation within a preset time period in the future; based on the predicted value of power generation within the preset time period in the future, adjusting the number of training processes started during the distributed model training process to train the distributed model.
[0204] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the distributed model training method provided in the above-mentioned embodiments, the method comprising: obtaining first multidimensional environmental data; inputting the first multidimensional environmental data into a data generation model to obtain second multidimensional environmental data output by the data generation model; the data generation model is obtained by training based on a generative adversarial network; performing time series prediction based on the first multidimensional environmental data and the second multidimensional environmental data to obtain a predicted value of power generation within a preset time period in the future; and adjusting the number of training processes started during the distributed model training process based on the predicted value of power generation within the preset time period in the future to train the distributed model.
[0205] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. Those of ordinary skill in the art may understand and implement the present invention without creative effort.
[0206] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0207] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A distributed model training method, characterized in that: include: Acquiring first multi-dimensional environmental data; Inputting the first multi-dimensional environmental data into a data generation model to obtain second multi-dimensional environmental data output by the data generation model; The data generation model is obtained by training based on a generative adversarial network; Perform time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period; Based on the predicted value of power generation within the future preset time period, adjusting the number of training processes started during the distributed model training process to train the distributed model; There are N training nodes in the distributed model training process; the number of training processes started in the distributed model training process is adjusted based on the power generation prediction value in the future preset time period to train the distributed model, including: Fixed batch size for a single training process; Start a training process on a training node, and record the overall power consumption of the training node until the training is stable; Add a training node and start a training process on each training node until the training is stable and record the overall power consumption of all training nodes; Iteratively perform the steps of adding a training node and starting a training process on each training node until the overall power consumption of all training nodes is recorded when the training is stable, until N training nodes are added, and the overall power consumption of the N training nodes when the training is stable is obtained; Associate the overall power consumption recorded each time with the corresponding number of training processes to obtain a mapping table; Based on the predicted value of power generation in the future preset time period and the mapping table, adjusting the number of training processes started during the distributed model training process to train the distributed model; The performing time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period includes: Integrating the first multi-dimensional environment data and the second multi-dimensional environment data to obtain target multi-dimensional environment data; Inputting the target multi-dimensional environmental data into a time series prediction model to obtain a power generation prediction value within a future preset time period output by the time series prediction model; The data generation model includes a generator and a discriminator; the data generation model is trained in the following way: Obtain multi-dimensional environmental data samples; Randomly setting multiple parameters in the multidimensional environmental data sample to zero to obtain an initial environmental data sample; Inputting the initial environment data sample into a generator to obtain a target environment data sample output by the generator; Based on the multidimensional environment data sample and the target environment data sample, authenticity discrimination is performed by a discriminator to obtain a true prediction probability of the target environment data sample; Calculating a loss value based on the true predicted probability of the target environment data sample; Based on the loss value, the gradient of the generator parameters and the gradient of the discriminator parameters are calculated by the back propagation algorithm; By means of an optimization algorithm, the generator parameters are updated according to the gradient of the generator parameters, and the discriminator parameters are updated according to the gradient of the discriminator parameters; The steps of inputting the initial environment data sample into the generator to obtain the target environment data sample output by the generator are iteratively executed until both the generator and the discriminator meet the standards, thereby obtaining a trained data generation model.
2. The distributed model training method according to claim 1, characterized in that: The step of adjusting the number of training processes started during the distributed model training process based on the power generation prediction value within the future preset time period and the mapping table to train the distributed model includes: Based on the predicted value of power generation in the future preset time period, determine the power consumption of the model training provided in the future preset time period; Matching the target number of training processes associated with the model training power consumption in the mapping table; Initialize the distributed training environment according to the target number of training processes; The distributed model is trained in the distributed training environment.
3. The distributed model training method according to claim 1, characterized in that: After adjusting the number of training processes started in the distributed model training process based on the power generation prediction value in the future preset time period to train the distributed model, the method further includes: Evaluate whether the distributed model after this round of training meets the accuracy requirements; If the distributed model after the current round of training meets the requirements, the distributed model after the current round of training is saved and the distributed model training process ends; If the distributed model after this round of training does not meet the standard, the step of performing time series prediction based on the first multidimensional environmental data and the second multidimensional environmental data to obtain the predicted value of power generation within a preset time period in the future is re-executed until the distributed model after this round of training meets the standard, the distributed model after this round of training is saved, and the distributed model training process is ended.
4. The distributed model training method according to claim 1, characterized in that: The time series prediction model is trained in the following way: Acquire a first historical environmental data sample and a historical power generation data sample; Inputting the first historical environment data sample into the data generation model to obtain a second historical environment data sample output by the data generation model; Integrate the first historical environment data sample and the second historical environment data sample into a target historical environment data sample; Based on the historical power generation data sample, determining label data of the target historical environment data sample; Model training is performed based on the second historical environment data sample and the corresponding label data to obtain the time series prediction model.
5. A distributed model training device, used to implement the distributed model training method according to any one of claims 1 to 4, characterized in that: include: An acquisition module, used for acquiring first multi-dimensional environmental data; A data generation module, used for inputting the first multi-dimensional environment data into a data generation model to obtain second multi-dimensional environment data output by the data generation model; The data generation model is obtained by training based on a generative adversarial network; A power generation prediction module, used to perform time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a power generation prediction value within a future preset time period; A distributed model training module, used to adjust the number of training processes started during the distributed model training process based on the power generation prediction value within the future preset time period, so as to train the distributed model; There are N training nodes in the distributed model training process; the number of training processes started in the distributed model training process is adjusted based on the power generation prediction value in the future preset time period to train the distributed model, including: Fixed batch size for a single training process; Start a training process on a training node, and record the overall power consumption of the training node until the training is stable; Add a training node and start a training process on each training node until the training is stable and record the overall power consumption of all training nodes; Iteratively perform the steps of adding a training node and starting a training process on each training node until the overall power consumption of all training nodes is recorded when the training is stable, until N training nodes are added, and the overall power consumption of the N training nodes when the training is stable is obtained; Associate the overall power consumption recorded each time with the corresponding number of training processes to obtain a mapping table; Based on the predicted value of power generation in the future preset time period and the mapping table, adjusting the number of training processes started during the distributed model training process to train the distributed model; The performing time series prediction based on the first multi-dimensional environmental data and the second multi-dimensional environmental data to obtain a predicted value of power generation within a future preset time period includes: Integrating the first multi-dimensional environment data and the second multi-dimensional environment data to obtain target multi-dimensional environment data; Inputting the target multi-dimensional environmental data into a time series prediction model to obtain a power generation prediction value within a future preset time period output by the time series prediction model; The data generation model includes a generator and a discriminator; the data generation model is trained in the following way: Obtain multi-dimensional environmental data samples; Randomly setting multiple parameters in the multidimensional environmental data sample to zero to obtain an initial environmental data sample; Inputting the initial environment data sample into a generator to obtain a target environment data sample output by the generator; Based on the multidimensional environment data sample and the target environment data sample, authenticity discrimination is performed by a discriminator to obtain a true prediction probability of the target environment data sample; Calculating a loss value based on the true predicted probability of the target environment data sample; Based on the loss value, the gradient of the generator parameters and the gradient of the discriminator parameters are calculated by the back propagation algorithm; By means of an optimization algorithm, the generator parameters are updated according to the gradient of the generator parameters, and the discriminator parameters are updated according to the gradient of the discriminator parameters; The steps of inputting the initial environment data sample into the generator to obtain the target environment data sample output by the generator are iteratively executed until both the generator and the discriminator meet the standards, thereby obtaining a trained data generation model.
6. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the distributed model training method as described in any one of claims 1 to 4 are implemented.
7. A computer-readable storage medium, the computer-readable storage medium comprising a non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the distributed model training method as described in any one of claims 1 to 4 are implemented.
8. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the steps of the distributed model training method as described in any one of claims 1 to 4 are implemented.