Energy internet of things load prediction method based on multi-target Transform-LSTM optimization

Abstract

The invention discloses an energy internet of things load prediction method based on multi-target Transform-LSTM (Long Short Term Memory) optimization. Historical power load data are collected from a historical database of the energy internet of things, feature reconstruction is carried out, after data normalization, the historical power load data serve as an input data set of a power load prediction offline optimization module, and the prediction average absolute percentage error of model training and the number of model parameters serve as optimization targets. And designing a Transform-LSTM optimization module based on multi-objective optimization, obtaining an optimal model considering prediction performance and model complexity, and deploying the model to an energy Internet of Things real-time power load prediction system on line for load prediction. According to the method, not only can Transform-LSTM automatic generation and optimization design for an energy internet of things power load prediction system be realized, but also indexes such as load prediction precision and model lightweight are considered, and an efficient solution is provided for real-time load prediction of the energy internet of things.

Description

Technical Field

[0001] The present invention belongs to the technical field of intelligent load forecasting in the field of energy economic dispatching of the energy Internet of Things, and particularly relates to a load forecasting method for the energy Internet of Things optimized based on multi-objective Transformer-LSTM. Background Art

[0002] With the large-scale deployment of smart meters and distributed sensors in the energy Internet of Things, the acquisition granularity and real-time performance of power load data have been significantly improved. However, the strong nonlinearity, time-variation, and uncertainty presented by the load sequence pose severe challenges to traditional forecasting methods. Under the combined action of flexible load random access on the user side, distributed energy dynamic response, and complex electricity consumption behaviors, the load curve has evolved from regular periodic fluctuations to a chaotic system of multimodal coupling. Although time series analysis techniques provide a basic framework for power load forecasting, there are still difficulties in practical applications, such as strong nonlinearity weakening the model representation ability, time-varying characteristics causing long-term forecasting instability, and uncertainty superposition reducing reliability. Traditional time series analysis techniques are difficult to achieve high-performance forecasting results.

[0003] To address the challenges of strong nonlinearity, time-variation, and uncertainty of power loads in the energy Internet of Things, load forecasting technologies based on deep learning have become a research hotspot. Deep learning models such as Transformer and Long Short-Term Memory (LSTM) can automatically extract complex time series features in the load sequence. Compared with traditional statistical models, they have significant advantages in forecasting accuracy and nonlinear fitting ability. However, although deep learning has improved the theoretical upper limit of load forecasting, its practical application still faces the following key bottlenecks: (1) The network structures and hyperparameters of existing deep learning models need to be repeatedly debugged relying on expert experience, lacking a systematic architecture search mechanism. When the load pattern mutates due to seasonal migration or the access of new loads, the original model needs to be re-optimized in a time-consuming and laborious trial-and-error manner, and it is difficult to achieve rapid migration and deployment in dynamic scenarios. (2) The designed deep learning models usually only take forecasting accuracy as a single optimization goal, ignoring the constraints of model parameter quantity and computational overhead. Currently, there is a lack of multi-objective automatic deep learning technologies that simultaneously consider forecasting performance and model lightweight, and they cannot be deployed in real time in edge devices with limited computational resources, severely restricting the engineering implementation value.

[0004] In the current research on power load forecasting in the energy Internet of Things, the Transformer, LSTM models, and their hybrid architecture (Transformer-LSTM) have become one of the main technical means. Some studies capture the long-term spatio-temporal dependencies of the load sequence through the multi-head attention mechanism of the Transformer, or extract local temporal dynamic features using the gated structure of the LSTM. The Transformer-LSTM hybrid model attempts to integrate the advantages of both to improve the prediction accuracy. However, the key structural parameters such as the inter-layer connection method, the number of attention heads, and the number of LSTM units in the Transformer-LSTM need to be repeatedly debugged relying on expert experience, lacking automated design criteria. When the load pattern shifts due to the access of distributed energy or the adjustment of demand response strategies, the model needs to re-perform time-consuming structural optimization and hyperparameter search, and it is difficult to adapt to the dynamic data environment. Although existing studies partially relieve the model lightweight pressure through techniques such as knowledge distillation and pruning quantization, they rely on offline compression and the accuracy loss is significant. Therefore, how to construct a multi-objective automatic deep learning method that takes into account both prediction performance and lightweight objectives under the Transformer-LSTM framework, and realizes the collaborative optimization of the model architecture, hyperparameters, and computational overhead, has become the key challenge to break through the high-precision real-time load forecasting technology barrier in the energy Internet of Things. Summary of the Invention

[0005] The purpose of the present invention is to provide a load forecasting method for the energy Internet of Things based on multi-objective Transformer-LSTM optimization in view of the deficiencies of the prior art.

[0006] The purpose of the present invention is achieved through the following technical solutions: The first aspect of the embodiments of the present invention provides a load forecasting method for the energy Internet of Things based on multi-objective Transformer-LSTM optimization, including the following steps:

[0007] Step1: Construct a time series feature extraction mechanism based on the historical database of the energy Internet of Things and a sliding window to construct a training data set and a test data set;

[0008] Step2: Define an initial parameter group and establish a multi-objective optimization function for the Transformer-LSTM;

[0009] Step3: Based on the defined initial parameter group, adopt a hybrid real number coding method to encode the model architecture and hyperparameters in the Transformer-LSTM to generate an initial population S it ;

[0010] Step4: Based on the training set and the established multi-objective optimization function, calculate the population S itThe objective function value of each individual in the population is calculated, and then the non-dominated sorting and crowding distance of each individual in the population are calculated based on the objective function value of each individual.

[0011] Step 5: According to the non-dominated sorting result and crowding distance of each individual, the population is selected. According to the selected individuals, crossover and mutation operations are performed to obtain the offspring population C. it , and calculate C it The objective function value of each individual in the parent population S is used to calculate the non-dominated sorting and crowding distance of each individual. it and offspring population C it Mix and obtain a new population N it ;

[0012] Step 6: According to the non-dominated sorting result and the crowding distance, N it Sort and set it = it + 1, and select the first NS individuals to form a new parent population S it ;

[0013] Step 7: Termination condition judgment: Determine whether the current number of iterations it reaches the preset maximum number of iterations IT max If the upper limit is not reached, the current parent population S it As the parent population of the next generation, continue to perform the operation and repeat Step 4 to Step 6; when the iteration termination condition is reached, perform the solution set extraction operation: first screen S it The non-dominated individuals in the population construct a Pareto frontier solution set, which is then analyzed based on the multi-objective function values. Finally, the optimal compromise solution is selected, and its corresponding network architecture and hyperparameter encoding are decoded to determine the final configuration of the optimal Transformer-LSTM model.

[0014] Step 8: Use the optimal Transformer-LSTM model to perform online power load forecasting for the Energy Internet of Things and evaluate the forecasting model based on the forecast accuracy index.

[0015] Compared with the prior art, the effective effects of the present invention are:

[0016] 1. The present invention comprehensively considers the two goals of prediction performance and number of model parameters of the energy Internet of Things power load forecasting system, and designs a Transformer-LSTM automatic optimization method based on a multi-objective optimization strategy. It effectively realizes the multi-objective optimization function for the energy Internet of Things load forecasting application scenario and takes into account high prediction accuracy and low model complexity. Through the multi-objective optimization mechanism of deeply mining the evolution law of the time series characteristics of load data and dynamically adjusting the model structure parameters, the comprehensive performance of the energy Internet of Things load forecasting system is significantly improved.

[0017] 2. The performance indicators of the present invention, such as the root mean square error, average relative error, average absolute error, and coefficient of determination in the power load forecasting of the energy Internet of Things, are better; while ensuring high prediction accuracy, the number of parameters of the Transformer-LSTM model obtained is significantly reduced, and the model architecture and parameters do not depend on manual empirical trial and error, realizing automated design, and providing a solution with engineering practical value for the real-time deployment and online prediction of the energy Internet of Things. Description of the Drawings

[0018] Figure 1 is the flowchart of the energy Internet of Things load forecasting method based on multi-objective Transformer-LSTM optimization of the present invention

[0019] Figure 2 is the schematic diagram of the corresponding relationship between the individual hybrid real number coding and the Transformer-LSTM architecture and hyperparameters of the present invention;

[0020] Figure 3 is the schematic diagram of the crossover operation process of the present invention;

[0021] Figure 4 is the schematic diagram of the Transformer-LSTM model structure of the present invention;

[0022] Figure 5 is the optimal Parato frontier diagram obtained by the load forecasting method of the present invention;

[0023] Figure 6 is the schematic diagram of the root mean square error comparison result of the prediction method of the present invention with the convolutional neural network (CNN), long short-term memory neural network (LSTM), and convolutional long short-term memory neural network (CNN-LSTM);

[0024] Figure 7 is the schematic diagram of the average relative error comparison result of the prediction method of the present invention with CNN, LSTM, and CNN-LSTM;

[0025] Figure 8 is the schematic diagram of the average absolute error comparison result of the prediction method of the present invention with CNN, LSTM, and CNN-LSTM;

[0026] Figure 9 is the schematic diagram of the coefficient of determination comparison result of the prediction method of the present invention with CNN, LSTM, and CNN-LSTM;

[0027] Figure 10 is the schematic diagram of the comparison result of the number of model parameters of the prediction method of the present invention with CNN, LSTM, and CNN-LSTM. Specific embodiments

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings of the embodiments of the present invention. It should be noted that the embodiments are only some examples of the present invention, not all possible implementation manners. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative efforts belong to the protection scope of the present invention.

[0029] The present invention discloses a load forecasting method for an energy Internet of Things based on multi-objective Transformer-LSTM optimization. Historical power load data is collected from the historical database of the energy Internet of Things. A time series feature extraction mechanism is constructed based on a sliding window to reconstruct the features of the power load data. After data normalization, it is used as the input data set of the offline optimization module for power load forecasting. The prediction mean absolute percentage error and the number of model parameters of model training are respectively used as optimization objectives. A Transformer-LSTM optimization module based on multi-objective optimization is designed to obtain an optimal model that takes into account both prediction performance and model complexity. This model is deployed online into the real-time power load forecasting system of the energy Internet of Things for load forecasting. The present invention can not only realize the automatic generation and optimized design of Transformer-LSTM for the power load forecasting system of the energy Internet of Things, but also take into account indicators such as load forecasting accuracy and model lightweight, providing an efficient solution for the real-time load forecasting of the energy Internet of Things.

[0030] The following further illustrates the present invention with reference to the accompanying drawings, and the purpose and effect of the present invention will be more obvious.

[0031] The load forecasting method for the energy Internet of Things based on multi-objective Transformer-LSTM optimization of the present invention is based on Figure 2The corresponding relationship between the individual hybrid real number coding, the Transformer-LSTM architecture, and the hyperparameters is shown. The number of self-attention heads of the Transformer-LSTM model, the minimum batch size during training, the learning rate decay factor, the learning rate decay period, the type of neural network optimizer selected for the model, etc. are used as optimization variables. A new hybrid real number coding mechanism is designed to encode the Transformer-LSTM model into an individual that can represent the model, and the number of model parameters and the prediction accuracy of the Transformer-LSTM model during power load prediction are used as two optimization objectives. An individual selection operation, a crossover operation, and a mutation operation are designed, and the selection of better individuals is completed through non-dominated sorting and crowding distance calculation. Finally, a Transformer-LSTM model with low model complexity and high prediction accuracy is obtained. The obtained model is used to test the power load data collected by an energy Internet of Things system in a certain place in January 2024, and good results are obtained.

[0032] See Figure 1 , the described energy Internet of Things load prediction method based on multi-objective Transformer-LSTM optimization specifically includes the following steps:

[0033] Step1: Construct a time series feature extraction mechanism based on the energy Internet of Things historical database and a sliding window to construct a training dataset and a test dataset;

[0034] The historical power load data collected from the energy Internet of Things historical database is transformed into data in a supervised learning format through a sliding window construction method;

[0035] The original one-dimensional power load time series data is sliced into multiple supervised learning sample format data with a fixed length to provide a structured input for subsequent model training. The historical power load data is transformed into an input dataset and an output dataset through a sliding window construction method, and its construction formula is:

[0036]

[0037] Among them, D represents the set of collected power load data, D(i + j - 1) and D(i + w) respectively represent the (i + j - 1)-th data and the (i + w)-th data in D, N samples represents the total number of samples, w represents the sliding window size, and X and Y respectively represent the input and output datasets.

[0038] Then, the input and output data are respectively divided into a model training set {P train , T train} and a model test set {P test , T test}, and its specific formula is:

[0039]

[0040] Among them, P train and T train respectively represent the input data set and the output data set of the model training set, and P test and T test respectively represent the input data set and the output data set of the model test set. N train and N samples respectively represent the number of training set samples and the total number of samples, and fe represents the input feature dimension.

[0041] Finally, normalize each group of data respectively, and its calculation formula is:

[0042]

[0043] Among them, Z oi represents the value after normalization of the i-th group of data in the data set, and Z i represents the value before normalization of the i-th group of data. Z imin and Z imax respectively represent the minimum value and the maximum value of the i-th group of data, and b represents the number of features to be normalized.

[0044] It should be noted that N train and N samples can be determined according to the actual data. In this embodiment, N samples = 1344, N train = 1008.

[0045] Step2: Define the initial parameter group and establish the multi-objective optimization function of Transformer-LSTM;

[0046] In this embodiment, the initial parameter group includes the population size NS, the number of iterations IT max , the crossover rate Cr, the mutation rate Mu, and the deep learning network architecture parameters, the maximum position encoding length Maxpos, and the number of model training times Maxepoch. The objective function of the multi-objective Transformer-LSTM is the mean absolute percentage error of the prediction result of the Transformer-LSTM model training and the number of model parameters; it should be understood that the parameters can be set according to the actual situation. For example, in this embodiment, the above parameters are set as NS = 100, IT max = 100, Cr = 0.9, Mu = 0.2, Maxpos = 512, Maxepoch = 400;

[0047] Furthermore, the objective function of the multi-objective Transformer-LSTM is the mean absolute percentage error of the prediction results and the number of model parameters for the training of the Transformer-LSTM model; where the mean absolute percentage error of the prediction results for the training of the Transformer-LSTM model refers to the model training set {P train ,T train} obtained by processing the historical power load data collected from the historical database of the energy Internet of Things. After that, the mean absolute percentage error of the prediction results evaluated by the model trained using the model training set is calculated by the following formula:

[0048]

[0049] where M represents the number of samples, A t represents the actual value, and F t represents the predicted value.

[0050] Step3: Based on the defined initial parameter group, adopt the hybrid real number coding method to code the model architecture and hyperparameters in the Transformer-LSTM to generate the initial population S it ;

[0051] Specifically, randomly generate a population S it containing NS individuals, that is, S it ={s1, s2, …, s NS}, 1 ≤ it ≤ IT max , where it represents the current iteration number. Among them, s1, s2, s NS respectively represent the first individual, the second individual, and the NSth individual in the population. The specific expression of each individual in the population is s i ={M1, M2, M3}, 1 ≤ i ≤ NS. M1, M2, and M3 respectively represent the three basic modules of the Transformer-LSTM network architecture. The first module represents the overall architecture of the neural network, the second module represents the self-attention layer in the Transformer-LSTM model, and the third module represents the LSTM layer in the Transformer-LSTM model.

[0052] Combined with Figure 4, The specific encoding of the first module M1 with hybrid real number encoding is {1, Numh, Mbs, Opt, Ldf, Ldp, L2r, Df, Inlr}, where the first encoding 1 is the module type code, which is used to identify the type of the module during decoding; Numh represents the number of self-attention heads during model training, Mbs represents the minimum batch size during model training; Opt represents the neural network optimizer selected during Transformer-LSTM model training; Ldf represents the learning rate decay factor during model training; Ldp represents the learning rate decay period during model training; L2r represents the L2 regularization coefficient during model training; Df represents the dropout factor of the dropout layer in the model architecture; Inlr represents the initial learning rate during model training. For the second module M2 and the third module M3 with hybrid real number encoding, the maximum number of each individual self-attention layer module and LSTM layer module is marked as Maxln. For each individual, any number of self-attention layers and LSTM layers within the range of 1 to Maxln are randomly generated; during encoding, the self-attention layers and LSTM layers of the individual are respectively integrated into the self-attention module M2 and the LSTM module M3, and the specific encoding in the M2 and M3 modules is {L1, L2, …, L n}, 1 ≤ n ≤ Maxln where L1, L2, L n respectively represent the first layer, the second layer and the nth layer in the module; the specific encoding of the self-attention layer and the LSTM layer is {2, Atm, WeiI, SabI} and {3, Lnd, Actf, WeiI, LsbI}, where the first digits 2 and 3 represent the module type code; Atm represents the mask type of the self-attention layer; ActF represents the type of the activation function after the LSTM layer; WeiI is the type of the weight initializer of the self-attention layer and the LSTM layer; SabI and LsbI respectively represent the types of the bias initializers of the self-attention layer and the LSTM layer.

[0053] Arrange the three modules M1, M2, and M3 according to the classic structure of Transformer-LSTM, with the overall network architecture module ranked first, followed by the self-attention layer module, and finally the LSTM layer module.

[0054] In the above hybrid real number encoding, the encoding methods of each variable are as follows:

[0055] Numh = Int(Numh lb , Numh ub ) (5)

[0056] Mbs = Int(Mbs lb , Mbs ub ) (6)

[0057] Opt = Int(1, 3) (7)

[0058]

[0059] Ldf = Rand(Ldf lb , Ldf ub ) (9)

[0060] Ldp = Int(Ldp lb , Ldp ub ) (10)

[0061] L2r = Rand(L2r lb , L2r ub ) (11)

[0062] Df = Rand(Df lb , Df ub ) (12)

[0063] Inlr = Rand(Inlr lb , Inlr ub ) (13)

[0064] Atm = Int(1, 2) (14)

[0065]

[0066] Actf = Int(1, 4) (16)

[0067]

[0068] Lnd = Int(Lnd lb , Lnd ub ) (18)

[0069] Weil = Int(1, 4) (19)

[0070]

[0071] SabI = Int(1, 3) (21)

[0072]

[0073] LsbI = Int(1, 3) (23)

[0074]

[0075] Among them, Int represents the random integer generation function, Numh represents the number of self-attention heads during model training, and the specific value of Numh is shown in Equation (5), where Numb lb and Numbub respectively represent the upper and lower limits of the Numb value; Mbs represents the minimum number of samples used to update the model parameters in each training step during model training, that is, the minimum batch size. The specific value of Mbs is shown in Equation (6), where Mbs lb and Mbs ub respectively represent the upper and lower limits of the Mbs value; Opt represents the type of neural network optimizer selected by the Transformer-LSTM model. Opt includes a total of three different processing methods. The definition of the Opt value is shown in Equation (7), and Int(1, 3) means to randomly generate an integer within the range of 1 to 3; the relationship between the Opt value and the selection of the neural network optimizer is shown in Equation (8); Adam represents the Adaptive Moment Estimation optimizer, SGD represents the Stochastic Gradient Descent optimizer, and RMSProp represents the Root Mean Square Propagation optimizer; Rand represents the random real number generation function; Ldf represents the learning rate decay factor during model training. The specific value of Ldf is shown in Equation (9), where Ldf lb and Ldf ub respectively represent the upper and lower limits of the Ldf value; Ldp represents the learning rate decay period during model training. The specific value of Ldp is shown in Equation (10), where Ldp lb and Ldp ub respectively represent the upper and lower limits of the Ldp value; L2r represents the L2 regularization factor during model training. The specific value of L2r is shown in Equation (11), where L2r lb and L2r ub respectively represent the upper and lower limits of the L2r value; Df represents the dropout factor of the dropout layer in the model architecture. The specific value of Df is shown in Equation (12), where Df lb and Df ub respectively represent the upper and lower limits of the Df value; Inlr represents the initial learning rate during model training. The specific value of Inlr is shown in Equation (13), where Inlr lb and Inlr ubrespectively represent the upper and lower limits of the Inlr value; Atm represents the mask type selected by the self-attention layer, and the integer encoding of Atm is shown in Equation (14); the corresponding relationship between the integer encoding of Atm and the mask type is shown in Equation (15); Actf represents the activation function type after the LSTM layer in the Transformer-LSTM model, and the integer encoding of Actf is shown in Equation (16); the corresponding relationship between the integer encoding of Actf and the activation function type is shown in Equation (17), ReLU represents the rectified linear unit function, Tanh represents the hyperbolic tangent function, Sigmoid represents the Sigmoid logistic function, and Softmax represents the Softmax normalization exponential function; Lnd represents the number of hidden neurons in the LSTM layer, and the specific value of Lnd is shown in Equation (18), where Lnd lb and Lnd ub respectively represent the upper and lower limits of the Lnd value; WeiI represents the weight initialization function type of the self-attention layer and the LSTM layer, and the integer encoding of WeiI is shown in Equation (19); the corresponding relationship between the integer encoding of WeiI and the weight initialization function type is shown in Equation (20), where glorot and He respectively represent Glorot initialization and He initialization, narrow-normal represents narrow normal distribution initialization, and ones represents all-ones initialization; SabI represents the bias initialization function type of the self-attention layer, and the integer encoding of SabI is shown in Equation (21); the corresponding relationship between the SabI integer encoding and the weight initialization function type is shown in Equation (22), where zeros represents all-zeros initialization, narrow-normal represents narrow normal distribution initialization, and ones represents all-ones initialization; LsbI represents the bias initialization function type of the self-attention layer, and the integer encoding of LsbI is shown in Equation (23); the corresponding relationship between the integer encoding of LsbI and the weight initialization function type is shown in Equation (24), where unit-forget-gate represents unit forget gate initialization, narrow-normal represents narrow normal distribution initialization, and ones represents all-ones initialization. In this embodiment, the specific settings of Numh are: Numb lb = 2, Numb ub = 8; the specific settings of Mbs are: Mbs lb = 32, Mbs ub = 128; the specific settings of Ldf are: Ldf lb = 0.01, Ldf ub = 0.5; the specific settings of Ldp are: Ldp lb = 60, Ldp ub = 150; the specific settings of L2r are: L2r lb = 0.0001, L2rub = 0.001; The specific settings of Df are: Df lb = 0.0001, Df ub = 0.001; The specific settings of Inlr are: Inlr lb = 0.005, Inlr ub = 0.04; The specific settings of Lnd are: Lnd lb = 8, Lnd ub = 64;

[0076] Exemplarily, in this embodiment, the number of individuals in the final population is NS = 100; among which the representative individuals are s1 = [1, 2, 210, 1, 0.1868, 145, 0.000623, 0.02683, 0.000964; 2, 1, 2, 1; 3, 17, 2, 2, 4], s2 = [1, 2, 210, 1, 0.18679, 145, 0.000623, 0.0268265, 0.000964; 2, 1, 2, 1; 3, 18, 1, 2, 1], s3 = [1, 3, 106, 1, 0.4316, 66, 0.0006165, 0.01142, 0.000734; 2, 2, 3, 1; 3, 20, 1, 4, 3]; In addition, in this embodiment, the data set is divided into a model training set and a model test set according to a ratio of 3:1; it should be understood that the data set can be divided according to other ratios according to the needs of model training.

[0077] Step4: Based on the training set and the established multi-objective optimization function, calculate the objective function values of each individual in the population S it in the population, and then calculate the non-dominated sorting and crowding distance of each individual in the population according to the objective function values of each individual.

[0078] Step4.1: Conduct fitness evaluation on all individuals in the population S it for the optimization objectives of the multi-objective Transformer-LSTM, and calculate two objective function values;

[0079] Step4.2: After obtaining the m objective function values f it = {s1, s2,..., s NS} of each individual in the population, perform non-dominated sorting operation on the population. The non-dominated sorting is to construct the Pareto front of the multi-objective space through the solution set. The specific process is as follows: First, according to the dominance relationship criterion, judge any two individuals s k (s i ) in the population, k = 1, 2,..., m, 1 ≤ i ≤ NS, and then perform non-dominated sorting operation on the population. The non-dominated sorting is to construct the Pareto front of the multi-objective space through the solution set. The specific process is as follows: First, according to the dominance relationship criterion, judge any two individuals s i and s jThe dominance relationship, when for any k ∈ [1, m], f k (s i ) ≤ f k (s j ) and there exists l ∈ [1, m] such that f l (s i ) < f l (s j ), then s i dominates s j ; then, initialize the non-dominated sorting operation parameters, and introduce two key parameters Dsi and nsi for each individual s i . Among them, Dsi represents the set of individuals dominated by s i , and nsi represents the number of individuals that dominate s i , that is, the dominance counter value, and initialize the frontier set P = {P1, P2,...}; finally, perform the layering operation. For each individual s it in the population S i , calculate Dsi and nsi; extract the first frontier layer P1: store all individuals with the dominated counter nsi = 0 into P1; iteratively generate subsequent frontiers: starting from P1, perform the following operations on each individual in the current frontier: traverse each individual in the dominance set of this individual, subtract 1 from the dominated counter value nsi of these dominated individuals. If the counter value nsi of a certain dominated individual is reduced to 0, add it to the next frontier layer; repeat the iteration until all individuals are layered: based on the individuals in the previous layer, repeat the operations of extracting the second frontier layer and subsequent frontier layers, layer by layer to generate new frontiers until all individuals are assigned to a certain level.

[0080] Step4.3: Calculate the crowding distance for all individuals in the population. The specific steps of the crowding distance calculation are as follows: First, select a non-dominated frontier layer P u = {s1, s2,..., s u} in the population. For each individual s i , 1 ≤ i ≤ u, which contains m objective function values f k (s i ), k = 1, 2,..., m. For each objective function f k , calculate the maximum and minimum values of the objective values within the frontier layer P u . The calculation formulas are shown in equations (25)-(26):

[0081]

[0082] Among them, and respectively represent the frontier layer P uThe maximum and minimum values of the k-th objective function within

[0083] For individual s i perform a normalization operation on the k-th objective value, and its calculation formula is as shown in Equation (27):

[0084]

[0085] where represents the objective function value after normalization.

[0086] Sort the individuals in the frontier layer P u in ascending order according to any objective function value to obtain a sorted index sequence; for each objective function f k define the crowding distance of the boundary individuals as infinity, as shown in Equation (28):

[0087] CrowdDist(s k,first ) = CrowdDist(s k,last ) = +∞ (28)

[0088] where s k,first represents the individual ranked first according to the k-th objective function value, and s k,last represents the individual ranked last according to the k-th objective function value, and CrowdDist represents the crowding distance array.

[0089] For the individuals s i sorted in the middle position, calculate the sum of the adjacent distances on all objective functions, and its calculation formula is as shown in Equation (29):

[0090]

[0091] where s i+1 and s i-1 are the individuals adjacent to s k after sorting for the objective function f i respectively.

[0092] After completing the calculation of the crowding distance of the frontier P u , enter the next frontier P u+1 , continue to calculate the crowding distance until the crowding distances of all frontiers are calculated in sequence; finally, re-sort the individuals within each frontier according to the crowding distance, so as to realize the sorting of all individuals in the population S it .

[0093] Step5: According to the non-dominated sorting results and crowding distances of each individual, perform a selection operation on the population, and according to the selected individuals, perform crossover and mutation operations to obtain the offspring population C it, and calculate C it Calculate the objective function value of each individual in it , calculate the non-dominated sorting and crowding distance of each individual, and combine the parent population S it and the offspring population C it to obtain a new population N it ;

[0094] Combine Figure 3 , in this embodiment, the individuals in the population perform selection operation, crossover operation and mutation operation to generate an offspring population C it , and combine the parent population S it with the offspring population C it to form a new population N it , the specific process is as follows: randomly select two groups of individuals from the population S it , each group contains two individuals. Perform a binary tournament selection operation on the individuals within each group, and select the undominated and large crowding distance individuals as the parents of each group of individuals, denoted as s1 and s2. Among them, preferentially select the undominated and large crowding distance individuals as the parents of each group of individuals. If the number of dominated times of the individuals is the same, select the individual with a large crowding distance as the parent of each group of individuals, and then perform a crossover operation to obtain offspring individuals; the steps of the crossover operation are as follows: First, separate each type of module of the parent individuals one by one to ensure that the same type of modules between individuals are crossed, and then perform a simulated binomial crossover on the corresponding parameters of each module. If the number of layers of the same type of module of the parent individuals is different, take the smaller number of layers as the crossover layer L min , and perform a crossover operation on the first L min layers of this type of module of the parent. The specific process of the simulated binomial crossover is shown in Equation (30):

[0095]

[0096] where x L and x U represent the upper and lower bounds of the variable, r represents a random number between 0 and 1, η represents the distribution index of the simulated binomial crossover used to control the crossover intensity, s1 and s2 are the parent individuals, c1 and c2 are the offspring individuals obtained after the crossover operation, β i , α i and β qi , i ∈ {1, 2}, represent the intermediate variables for calculating c1 and c2; perform a crossover operation on the parent population S it to obtain the offspring population C it , and then perform a mutation operation. The mutation operation is based on the mutation rate in the offspring population C itSelect a part of the individuals for mutation. There are two ways of mutation, including the mutation of the parameters in the modules of the individual and the mutation of the structure in the modules. The parameter mutation of the module means randomly selecting a module in the individual, regenerating the parameters of this type of module and replacing the original parameters of the corresponding module of the individual; the structure mutation of the module is divided into the addition and deletion of layers in the module, and the structure mutation only occurs in the self-attention layer module and the LSTM layer module. The mutation operation of the individual is shown in Equation (31):

[0097]

[0098] Among them, parameter represents parameter mutation, structure represents structure mutation, add represents the addition of layers, delete represents the deletion of layers, rand1 and rand2 respectively represent two random numbers between 0 and 1; during the mutation process, the minimum number of layers in the self-attention layer module and the LSTM layer module is set to 1; repeat the crossover and mutation operations until the size of the offspring population C it reaches NS; then combine the parent population S it with the offspring population C it to form a new population N it . There are a total of 2NS individuals in the new population.

[0099] Step6: Sort N it according to the non-dominated sorting result and the crowding distance. At the same time, set it = it + 1, and select the first NS individuals from it to form a new parent population S it ;

[0100] Specifically, after calculating the objective function values of each individual in the new population, performing the non-dominated sorting operation and the crowding distance calculation on it, select the first NS non-dominated solutions as the new parent population according to the elitist retention strategy, which is S it ;

[0101] Step7: Judgment of termination condition: Judge whether the current iteration number it reaches the preset maximum iteration number IT max . If it does not reach the upper limit, use the current parent population S it as the parent population of the next generation to continue the operation, and repeat Step4 to Step6; when the iteration termination condition is reached, perform the solution set extraction operation: First, screen the non-dominated individuals in the S it population to construct the Pareto front solution set. Then, analyze the Pareto solution set based on the multi-objective function values. Finally, select the optimal compromise solution, and decode the corresponding network architecture and hyperparameter encoding to determine the final configuration scheme of the optimal Transformer-LSTM model;

[0102] Furthermore, the optimal compromise solution is selected and determined as the final configuration scheme of the optimal Transformer-LSTM model, which specifically includes: First, judge the relationship between it and IT max . If the current iteration number it is less than the maximum iteration number IT max , then increment the value of it by 1 and continue the iteration process; conversely, when the maximum iteration number is reached, the following operations need to be performed: First, identify the non-dominated solutions in the population S it to form the Pareto front solution set, and then screen out the optimal solution from it, and normalize the values of the optimization objective function in their respective solution sets; then add the normalized values of the objective function to obtain the final value of the objective function. The Pareto optimal individual is the individual with the smallest final objective function value in the Pareto optimal solution set. Finally, the Transformer-LSTM model structure and hyperparameters corresponding to this solution are determined as the optimal configuration scheme.

[0103] In this embodiment, after 100 iterations, the encoded Pareto optimal individual obtained is s best = [1, 3, 62, 1, 0.3612, 72, 0.000218, 0.0389, 0.00068; 2, 1, 2, 3; 3, 24, 2, 3, 2]; The specific values of the two objectives are [0.85885, 50.3].

[0104] Step8: Use the optimal Transformer-LSTM model to perform online power load forecasting for the energy Internet of Things and evaluate the forecasting model according to the forecasting accuracy index.

[0105] The model forecasting accuracy indexes include root mean square error RMSE, mean absolute percentage error MAPE, mean absolute error MAE, and coefficient of determination R 2 , and the specific calculation formulas are shown in formula (32):

[0106]

[0107] where M represents the number of samples, A t represents the true value, F t represents the predicted value, represents the mean value of the true values.

[0108] Figure 5 is the optimal Parato front graph obtained by the load forecasting method of the present invention. Figures 6 - 10They are respectively the schematic diagrams of the comparison results of the prediction method of the present invention with the convolutional neural network (CNN), long short-term memory neural network (LSTM), and convolutional long short-term memory neural network (CNN-LSTM) in terms of root mean square error, average relative error, average absolute error, coefficient of determination, number of model parameters, etc. The present invention can not only realize the automatic generation and optimization design of Transformer-LSTM for the power load prediction system of the energy Internet of Things, but also take into account indicators such as load prediction accuracy and model lightweight, providing an efficient solution for the real-time load prediction of the energy Internet of Things.

[0109] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention.

Claims

1. A load forecasting method for the energy Internet of Things optimized based on multi-objective Transformer-LSTM, characterized in that, It includes the following steps: Step1: Construct a training dataset and a test dataset based on the historical database of the energy Internet of Things and a sliding window to construct a time series feature extraction mechanism; Step2: Define an initial parameter group and establish a multi-objective optimization function for Transformer-LSTM; Step 3: Based on the defined initial parameter group, use the hybrid real number coding method to encode the model architecture and hyperparameters in Transformer-LSTM to generate the initial population S it ; Step4: Based on the training set and the established multi-objective optimization function, calculate the objective function values of each individual in population S it and then, according to the objective function values of each individual, calculate the non-dominated sorting and crowding distance of each individual in the population; Step 5: According to the non-dominated sorting results and crowding distances of each individual, perform a selection operation on the population. Based on the selected individuals, perform crossover and mutation operations to obtain the offspring population C it , and calculate the objective function values of each individual in C it , calculate the non-dominated sorting and crowding distances of each individual, and mix the parent population S it and the offspring population C it to obtain the new population N it ; Step6: Sort N according to the non-dominated sorting result and crowding distance it At the same time, let it = it + 1, and select the top NS individuals from it to form a new parental population S it ; Step7: Termination condition judgment: Determine whether the current iteration number it reaches the preset maximum iteration number IT max , if it does not reach the upper limit, then use the current parental population S it as the parental population of the next generation and continue to perform operations, repeating Step4 to Step6; when the iteration termination condition is reached, perform the solution set extraction operation: First, screen the non-dominated individuals within the S it population to construct the Pareto front solution set, and then analyze the Pareto solution set based on the multi-objective function values. Finally, select the optimal compromise solution, and decode the corresponding network architecture and hyperparameter encoding to determine the final configuration plan of the optimal Transformer-LSTM model; Step8: Use the optimal Transformer-LSTM model to perform online power load forecasting for the energy Internet of Things and evaluate the forecasting model according to the forecasting accuracy index.

2. The load forecasting method for the energy Internet of Things optimized based on multi-objective Transformer-LSTM according to claim 1, characterized in that, The historical power load data collected from the historical database of the energy Internet of Things is transformed into data in a supervised learning format through a sliding window construction method; The original one-dimensional power load time series data is sliced into data in the format of multiple supervised learning samples with a fixed length, providing a structured input for subsequent model training; The historical power load data is transformed into an input dataset and an output dataset through a sliding window construction method, and the construction formula is: Among them, D represents the set of collected power load data, D(i + j - 1) and D(i + w) represent the (i + j - 1)-th data and the (i + w)-th data in D respectively, N samples represents the total number of samples, w represents the sliding window size, and X and Y represent the input and output data sets respectively; Then, the input and output data are respectively divided into a model training set {P train , T train} and a model test set {P test , T test}, and the specific formula is as follows: Among them, P train and T train respectively represent the input data set and the output data set of the model training set. P test and T test respectively represent the input data set and the output data set of the model test set. N train and N samples respectively represent the number of training set samples and the total number of samples. fe represents the input feature dimension; Finally, normalize each group of data respectively, and its calculation formula is: Among them, Z oi represents the value after normalization of the i-th group of data in the data set, and Z i represents the value before normalization of the i-th group of data. Z imin and Z imax represent the minimum and maximum values of the i-th group of data respectively, and b represents the number of features to be normalized.

3. The load forecasting method for the energy Internet of Things optimized based on multi-objective Transformer-LSTM according to claim 2, wherein The initial parameter set includes the population size NS, the number of iterations IT max , the crossover rate Cr, the mutation rate Mu, and the deep learning network architecture parameters, the maximum position encoding length Maxpos, and the number of model training times Maxepoch; The objective function of the multi-objective Transformer-LSTM is the mean absolute percentage error of the prediction results of the Transformer-LSTM model training and the number of model parameters; where the mean absolute percentage error of the prediction results of the Transformer-LSTM model training refers to the model training set {P train ,T train} obtained by processing the historical power load data collected from the historical database of the energy Internet of Things, and the mean absolute percentage error of the prediction results obtained by evaluating the model training set with the model trained using the model training set. The calculation formula is as follows: Among them, M represents the number of samples, A t represents the actual value, F t represents the predicted value.

4. The load forecasting method for the energy Internet of Things optimized based on multi-objective Transformer-LSTM according to claim 3, wherein According to the initial parameter group, the model architecture and hyperparameters in Transformer-LSTM are encoded by using a hybrid real number coding method to generate an initial population S it , specifically including: Randomly generate a population S containing NS individuals it , that is, S it = {s1, s2, …, s NS}, 1 ≤ it ≤ IT max , where it represents the current iteration number, and where s1, s2, s NS respectively represent the 1st individual, the 2nd individual, and the NSth individual of the population. The specific expression of each individual in the population is s i = {M1, M2, M3}, 1 ≤ i ≤ NS, where M1, M2, and M3 respectively represent the three basic modules of the Transformer-LSTM network architecture. The first module represents the overall architecture of the neural network, the second module represents the self-attention layer in the Transformer-LSTM model, and the third module represents the LSTM layer in the Transformer-LSTM model; The specific coding of the first module M1 of the hybrid real number coding is {1, Numh, Mbs, Opt, Ldf, Ldp, L2r, Df, Inlr}, where the first coding 1 is the module type code, which is used to identify the type of the module during decoding; Numh represents the number of self-attention heads during model training, Mbs represents the minimum batch size during model training; Opt represents the neural network optimizer selected during Transformer-LSTM model training; Ldf represents the learning rate decay factor during model training; Ldp represents the learning rate decay period during model training; L2r represents the L2 regularization coefficient during model training; Df represents the dropout factor of the dropout layer in the model architecture; Inlr represents the initial learning rate during model training; For the second module M2 and the third module M3 with hybrid real - number encoding, mark the maximum number of each individual self - attention layer module and LSTM layer module as Maxln. For each individual, randomly generate any number of self - attention layers and LSTM layers within the range of 1 to Maxln; during encoding, integrate the self - attention layers and LSTM layers of the individual into the self - attention module M2 and the LSTM module M3 respectively. The specific encoding in M2 and M3 modules is {L1, L2, …, L n}, 1 ≤ n ≤ Maxln where L1, L2, L n represent the first layer, the second layer and the nth layer in the module respectively; the specific encoding of the self - attention layer and the LSTM layer is {2, Atm, WeiI, SabI} and {3, Lnd, Actf, WeiI, LsbI}, where the leading 2 and 3 are module type codes; Atm represents the mask type of the self - attention layer; ActF represents the type of the activation function after the LSTM layer; WeiI is the type of the weight initializer for the self - attention layer and the LSTM layer; SabI and LsbI represent the initializer types of the biases of the self - attention layer and the LSTM layer respectively; Arrange the three modules M1, M2, and M3 according to the classical structure of Transformer-LSTM, with the overall network architecture module ranked first, followed by the self-attention layer module, and finally the LSTM layer module; In the above hybrid real number coding, the coding method of each variable is: Numh = Int(Numh lb , Numh ub ) (5) Mbs = Int(Mbs lb , Mbs ub ) (6) Opt = Int(1, 3) (7) Ldf = Rand(Ldf lb , Ldf ub ) (9) Ldp = Int(Ldp lb , Ldp ub ) (10) L2r = Rand(L2r lb ,L2r ub ) (11) Df = Rand(Df lb , Df ub ) (12) Inlr = Rand(Inlr lb , Inlr ub ) (13) Atm = Int(1, 2) (14) Actf = Int(1, 4) (16) Lnd = Int(Lnd lb , Lnd ub ) (18) Weil = Int(1, 4) (19) SabI = Int(1, 3) (21) LsbI = Int(1, 3) (23) Among them, Int represents the random integer generation function, Numh represents the number of self-attention heads during model training, and the specific value of Numh is shown in Equation (5), where Numb lb and Numb ub respectively represent the upper and lower limits of the value of Numb; Mbs represents the minimum number of samples used to update the model parameters in each training step during model training, that is, the minimum batch size, and the specific value of Mbs is shown in Equation (6), where Mbs lb and Mbs ub respectively represent the upper and lower limits of the value of Mbs; Opt represents the type of neural network optimizer selected by the Transformer-LSTM model. Opt includes a total of three different processing methods, and the value of Opt is defined as shown in Equation (7). Int(1,3) means to randomly generate an integer in the range of 1 to 3; the relationship between the value of Opt and the selection of the neural network optimizer is shown in Equation (8); Adam represents the adaptive moment estimation optimizer, SGD represents the stochastic gradient descent optimizer, and RMSProp represents the root mean square propagation optimizer; Rand represents the random real number generation function; Ldf represents the learning rate decay factor during model training, and the specific value of Ldf is shown in Equation (9), where Ldf lb and Ldf ub respectively represent the upper and lower limits of the value of Ldf; Ldp represents the learning rate decay period during model training, and the specific value of Ldp is shown in Equation (10), where Ldp lb and Ldp ub respectively represent the upper and lower limits of the value of Ldp; L2r represents the L2 regularization factor during model training, and the specific value of L2r is shown in Equation (11), where L2r lb and L2r ub respectively represent the upper and lower limits of the value of L2r; Df represents the dropout factor of the dropout layer in the model architecture, and the specific value of Df is shown in Equation (12), where Df lb and Df ub respectively represent the upper and lower limits of the value of Df; Inlr represents the initial learning rate during model training, and the specific value of Inlr is shown in Equation (13), where Inlr lb and Inlr ub respectively represent the upper and lower limits of the Inlr value; Atm represents the mask type selected by the self-attention layer, and the integer encoding of Atm is shown in Equation (14); the correspondence between the integer encoding of Atm and the mask type is shown in Equation (15); Actf represents the activation function type after the LSTM layer in the Transformer-LSTM model, and the integer encoding of Actf is shown in Equation (16); the correspondence between the integer encoding of Actf and the activation function type is shown in Equation (17), where ReLU represents the rectified linear unit function, Tanh represents the hyperbolic tangent function, Sigmoid represents the Sigmoid logistic function, and Softmax represents the Softmax normalization exponential function; Lnd represents the number of hidden neurons in the LSTM layer, and the specific value of Lnd is shown in Equation (18), where Lnd lb and Lnd ub respectively represent the upper and lower limits of the Lnd value; WeiI represents the weight initialization function type of the self-attention layer and the LSTM layer, and the integer encoding of WeiI is shown in Equation (19); the correspondence between the integer encoding of WeiI and the weight initialization function type is shown in Equation (20), where glorot and He respectively represent Glorot initialization and He initialization, narrow-normal represents narrow normal distribution initialization, and ones represents all-ones initialization; SabI represents the bias initialization function type of the self-attention layer, and the integer encoding of SabI is shown in Equation (21); the correspondence between the integer encoding of SabI and the weight initialization function type is shown in Equation (22), where zeros represents all-zeros initialization, narrow-normal represents narrow normal distribution initialization, and ones represents all-ones initialization; LsbI represents the bias initialization function type of the self-attention layer, and the integer encoding of LsbI is shown in Equation (23); the correspondence between the integer encoding of LsbI and the weight initialization function type is shown in Equation (24), where unit-forget-gate represents unit forget gate initialization, narrow-normal represents narrow normal distribution initialization, and ones represents all-ones initialization; Initialize all individuals s in population S according to the above parameter selection range it where 1 ≤ i ≤ NS i .

5. A method for predicting the load of an energy Internet of Things optimized based on multi-objective Transformer-LSTM according to claim 4, characterized in that In Step 4, based on the training set and the established multi-objective optimization function, calculate the population S it the objective function value of each individual in, and then calculate the non-dominated sorting and crowding distance of each individual in the population according to the objective function value of each individual; It includes the following sub-steps: Step4.1: Evaluate the fitness of all individuals in population S according to the optimization objectives of the multi-objective Transformer-LSTM, and calculate the values of two objective functions; it ​ Step 4.2: After obtaining the m objective function values \(f_{k}(s_{i})\) of each individual in the population \(S = \{s_{1}, s_{2}, \ldots, s_{NS}\}\), where \(k = 1, 2, \ldots, m\) and \(1\leq i\leq NS\), perform non - dominated sorting operation on the population. The non - dominated sorting is to construct the Pareto front of the multi - objective space through the solution set. The specific process is as follows: it =\{s1, s2, \ldots, s NS}\} for each individual's m objective function values \(f k (s i ), k = 1, 2, \ldots, m, 1\leq i\leq NS, then perform non - dominated sorting operation on the population. The non - dominated sorting is to construct the Pareto front of the multi - objective space through the solution set. The specific process is as follows: First, according to the domination relation criterion, judge the domination relation between any two individuals s i and s j in the population. When for any k ∈ [1, m], f k (s i ) ≤ f k (s j ) and there exists l ∈ [1, m] such that f l (s i ) < f l (s j ), then s i dominates s j ; Then, initialize the non-dominated sorting operation parameters, and introduce two key parameters Dsi and nsi for each individual s i . Among them, Dsi represents the set of individuals dominated by s i , and nsi represents the number of individuals that dominate s i , that is, the domination counter value, and initialize the front set P = {P1, P2,...}; Finally, perform the layering operation. For each individual s it in the population S i , calculate Dsi and nsi; Extract the first front layer P1: Store all individuals with the domination counter nsi = 0 into P1; Iteratively generate subsequent fronts: Starting from P1, perform the following operations on each individual in the current front: Traverse each individual in the domination set of this individual, subtract 1 from the domination counter value nsi of these dominated individuals. If the counter value nsi of a certain dominated individual is reduced to 0, add it to the next front layer; Repeat the iteration until all individuals are layered: Based on the individuals in the previous layer, repeat the operations of extracting the second front layer and subsequent front layers, layer by layer generating new fronts until all individuals are assigned to a certain level; Step4.3: Calculate the crowding distance for all individuals in the population, and the specific steps of the crowding distance calculation are as follows: First, select a non-dominated front layer \(P\) in the population u =\(\{s_1, s_2, \ldots, s\) u \}\), and each individual \(s\) i , \(1\leq i\leq u\), contains \(m\) objective function values \(f\) k (s i ), \(k = 1, 2, \ldots, m\). For each objective function \(f\) k , calculate the maximum and minimum values of the objective values within the front layer \(P\) u . The calculation formulas are shown in Eqs. (25)-(26) as follows: Among them, and respectively represent the maximum and minimum values of the k-th objective function in the leading edge layer P u ; For individual s i perform a normalization operation on the k-th target value, and the calculation formula is shown in Equation (27): Among them, represents the objective function value after normalization; For the leading edge layer P u Sort the individuals in it in ascending order according to any objective function value to obtain the sorted index sequence; for each objective function f k Define the crowding distance of the boundary individuals as infinity, as shown in Equation (28): CrowdDist(s k,first ) = CrowdDist(s k,last ) = +∞ (28) Among them, s k,first represents the individual ranked first according to the k-th objective function value, and s k,last represents the individual ranked last according to the k-th objective function value. CrowdDist represents the crowding distance array; For the individual s in the middle position after sorting i , calculate the sum of its adjacent distances on all objective functions. The calculation formula is shown in Equation (29): where s i+1 and s i-1 are the individuals adjacent to the sorted objective function f k and s i respectively; Complete the leading edge P u After calculating the crowding distance, enter the next leading edge P u+1 , and continue to calculate the crowding distance until the crowding distances of all leading edges are calculated in sequence; finally, re - sort the individuals within each leading edge according to the crowding distance, so as to sort all individuals in the population S it .

6. The intelligent load forecasting method based on multi-objective Transformer-LSTM automatic optimization according to claim 5, wherein, The described Step5 includes the following process: randomly select two groups of individuals from the population S it where each group contains two individuals. Perform binary tournament selection operations on the individuals within each group, and select the non-dominated individuals with a large crowding distance as the parents of each group of individuals, denoted as s1 and s2. Among them, preferentially select the non-dominated individuals with a large crowding distance as the parents of each group of individuals. If the number of times an individual is dominated is the same, select the individual with a large crowding distance as the parent of each group of individuals. Subsequently, perform a crossover operation to obtain offspring individuals; the steps of the crossover operation are as follows: First, each type of module of the parental individuals is separated one by one to ensure that the same-type modules between individuals are crossed. Then, binomial crossover is simulated for the corresponding parameters of each module. If the number of layers of the same-type modules of the parental individuals is different, the smaller number of layers is taken as the crossover layer number L min , and the first L min layers of this type of module of the parental generation are respectively subjected to the crossover operation; the specific process of simulating binomial crossover is shown in Equation (30): where x L and x U represent the upper and lower bounds of the variable, r represents a random number between 0 and 1, η represents the distribution exponent for simulating binomial crossover to control the crossover intensity, s1 and s2 are parent individuals, c1 and c2 are offspring individuals obtained after the crossover operation, β i , α i and β qi , i ∈ {1, 2}, represent the intermediate variables for calculating c1 and c2; perform the crossover operation on the parent population S it to obtain the offspring population C it , and then perform the mutation operation. The mutation operation selects a part of the individuals in the offspring population C it for mutation according to the mutation rate. There are two ways of mutation, including the mutation of the parameters in the module of the individual and the mutation of the structure in the module. The parameter mutation of the module means randomly selecting a module in the individual, regenerating the parameters of this type of module and replacing the original parameters of the corresponding module of the individual; the structure mutation of the module is divided into the addition and deletion of layers in the module, and the structure mutation only occurs in the self-attention layer module and the LSTM layer module. The mutation operation of the individual is shown in Equation (31): Among them, parameter represents parameter mutation, structure represents structural mutation, add represents the addition of layers, delete represents the deletion of layers, rand1 and rand2 respectively represent two random numbers between 0 and 1; during the mutation process, the minimum number of layers in the self-attention layer module and the LSTM layer module is set to 1; repeat the crossover and mutation operations until the scale of the offspring population C it reaches NS; then combine the parent population S it with the offspring population C it to form a new population N it . There are a total of 2NS individuals in the new population.

7. The load forecasting method for the energy Internet of Things optimized based on multi-objective Transformer-LSTM according to claim 6, wherein Select the top NS individuals from the new population to form a new parental population S it , specifically including: After calculating the objective function values of each individual in the new population, performing non-dominated sorting operation and crowding distance calculation on them, the top NS non-dominated solutions are selected as the new parental population according to the elitist retention strategy, which is S it .

8. The intelligent load forecasting method based on multi-objective Transformer-LSTM automatic optimization according to claim 7, wherein The screening S it Construct a Pareto front solution set with non-dominated individuals within the population, then analyze the Pareto solution set based on multi-objective function values, and finally select the optimal compromise solution, specifically including: First, judge the relationship between it and IT max If the current iteration number it is less than the maximum iteration number IT max , then increment the value of it by 1 and continue the iteration process; conversely, when the maximum iteration number is reached, the following operations need to be performed: First, identify the non-dominated solutions in the population S it to form the Pareto front solution set, then screen out the optimal solution from it, and finally determine the Transformer-LSTM model structure and hyperparameters corresponding to this solution as the optimal configuration scheme; Select the Pareto optimal compromise solution from the Pareto optimal solution set. First, normalize the optimization objective function values in their respective solution sets; then add the normalized objective function values to obtain the final objective function value, and the Pareto optimal compromise solution is the individual with the smallest final objective function value in the Pareto optimal solution set.

9. The intelligent load forecasting method based on multi-objective Transformer-LSTM automatic optimization according to claim 8, wherein, The model prediction accuracy metrics include Root Mean Square Error (RMSE), Mean Average Percentage Error (MAPE), Mean Absolute Error (MAE), and Coefficient of Determination (R). 2 , and the specific calculation formulas are shown in Formula (32) as follows: Among them, M represents the total number of samples, A t represents the true value, F t represents the predicted value, represents the mean value of the true values.

Images