Time series forecasting modeling methods, forecasting methods, equipment, media and products

Abstract

The present application discloses a time series prediction modeling method, a prediction method, a device, a medium and a product, and relates to the field of time series prediction. The modeling method includes constructing a time point set and a local domain set for each time series; constructing a prediction model including a first embedding module, a second embedding module and an attention module; training the prediction model using the time point set and the local domain set of each time series to obtain a target prediction model; wherein the first embedding module embeds each time point in the time point set into a high-dimensional space; the second embedding module embeds each local domain in the local domain set into a high-dimensional space; the attention module generates a query vector, a key vector and a value vector based on the embedding result, and calculates the point-local domain attention dependency result based on the query vector, the key vector and the value vector. The present application solves the problem that traditional Transformer is difficult to effectively capture the complex dependencies of time series.

Description

Technical Field

[0001] The present application belongs to the technical field of time series prediction, and in particular, relates to a time series prediction modeling method, prediction method, device, medium and product. Background Art

[0002] Power transformers generate heat during operation. The greater the load, the more heat is generated. The oil temperature of the power transformer is an important indicator reflecting the load and operating status of the power transformer. By using the power transformer oil temperature data obtained through continuous observation and changing over time, future indicators such as the temperature (such as oil temperature) and load of the power transformer can be predicted, providing a basis for power load adjustment control, transformer cooling system control, and equipment failure warning.

[0003] The temperature and load data of power transformers are arranged in chronological order. Based on the analysis of historical time series, future temperature and load can be predicted. However, since the temperature and load data of power transformers have time-varying, non-stationary and non-linear characteristics, time series prediction faces difficulties such as non-stationarity, non-linearity and complex dependency structure.

[0004] At present, the Transformer model based on the attention mechanism has become the most promising method for time series prediction due to its advantage of multi-head attention mechanism in capturing the correlation between elements of the time series. However, the attention mechanism of the Transformer model uses single-point elements of the time series as token input, and does not make sufficient use of the local morphology, continuous features, and temporal features of the time series, and cannot effectively capture the complex dependencies of the time series. Summary of the invention

[0005] The purpose of this application is to provide a time series prediction modeling method, prediction method, device, medium and product to solve the problem that the traditional Transformer model based on the attention mechanism is difficult to effectively capture the complex dependencies of the time series, resulting in poor prediction performance of the power transformer temperature and load time series.

[0006] The present application solves the above technical problems through the following technical solutions: a time series prediction modeling method, comprising:

[0007] Constructing a time series data set; wherein the time series data set is a time series data set of power transformer temperature and load, and the time series data set includes a plurality of time series of equal length;

[0008] For each time series in the time series data set, construct a time point set and a local domain set of the time series; wherein each time point in the time point set includes the position, time and attribute value of the time point, and each local domain in the local domain set includes the position, time and attribute value of n consecutive time points, 1<n<N, N represents the length of the time series;

[0009] Constructing a prediction model, wherein the prediction model includes a first embedding module, a second embedding module, and an attention module;

[0010] The prediction model is trained using a time point set and a local domain set of each time series in the time series data set to obtain a target prediction model;

[0011] In the prediction model, each time point in the time point set is embedded into a high-dimensional space using the first embedding module to obtain an embedding vector of the time point; each local domain in the local domain set is embedded into a high-dimensional space using the second embedding module to obtain an embedding vector of the local domain;

[0012] The attention module is used to generate a query vector according to the embedding vector of each time point corresponding to the time point set, and a key vector and a value vector are generated according to the embedding vector of each local domain corresponding to the local domain set; and a point-local domain attention dependency result between the time point set and the local domain set is calculated according to the query vector, the key vector and the value vector.

[0013] Further, using the first embedding module to embed each time point in the time point set into a high-dimensional space includes:

[0014] Embed the position of each time point into a high-dimensional space to obtain the position embedding vector of the time point;

[0015] Embed the time of each time point into a high-dimensional space to obtain the time embedding vector of the time point;

[0016] Embed the attribute value of each time point into a high-dimensional space to obtain the attribute value embedding vector of the time point;

[0017] An embedding vector of a time point is calculated according to the position embedding vector, the time embedding vector and the attribute value embedding vector of the time point.

[0018] Furthermore, the specific embedding formula of the position embedding vector of the time point is:

[0019] = ;

[0020] , ;

[0021] in, Indicates The position embedding vector of each time point; Indicates The embedding value of the position of each time point in the even-dimensional high-dimensional space; Indicates The embedding value of the position of each time point in the odd dimension of the high-dimensional space; Indicates The location of a point in time; Represents the dimension of the high-dimensional space, and the dimension number starts from 0;

[0022] The specific embedding formula of the time embedding vector at the time point is:

[0023] ; ;

[0024] in, Indicates The time embedding vector of each time point, Indicates The time point time, M represents the number of time. Indicates The time point A kind of embedding vector that maps time to a high-dimensional space, Indicates time The index of Represents an index The weight matrix of the linear transformation of ;

[0025] The specific embedding formula of the attribute value embedding vector at the time point is:

[0026] ;

[0027] in, Indicates The attribute value embedding vector at each time point, Indicates The attribute value at a time point, Represents attribute value The weight matrix of the linear transformation of ;

[0028] The specific calculation formula of the embedding vector at the time point is:

[0029] ;

[0030] in, Indicates The embedding vector of each time point.

[0031] Further, using the second embedding module to embed each local domain in the local domain set into a high-dimensional space includes:

[0032] Embed the position of the last time point in the local domain into a high-dimensional space to obtain a position embedding vector of the last time point; use the position embedding vector of the last time point as the position embedding vector of the local domain;

[0033] Embed the time of the last time point in the local domain into a high-dimensional space to obtain a time embedding vector of the last time point; use the time embedding vector of the last time point as the time embedding vector of the local domain;

[0034] Embed the attribute values ​​of the local domain at n consecutive time points into a high-dimensional space to obtain an attribute value embedding vector of the local domain;

[0035] An embedding vector of the local domain is calculated according to the position embedding vector, the time embedding vector and the attribute value embedding vector of the local domain.

[0036] Furthermore, the specific embedding formula of the attribute value embedding vector of the local domain is:

[0037] ;

[0038] ;

[0039] in, Indicates The attribute value embedding vector of the local domain, Indicates taking the maximum value, represents the correction factor, Indicates the intermediate quantity, Indicated by The value vector composed of the attribute values ​​of n consecutive time points in a local domain, Represents a value vector The weight matrix of the linear transformation of ;

[0040] The specific calculation formula of the embedding vector of the local domain is:

[0041] ;

[0042] in, Indicates The embedding vector of the local domain, Indicates The position embedding vector of the local domain, Indicates The temporal embedding vector of the local domain.

[0043] Furthermore, the attention module is used to calculate the point-local domain attention dependency result between the time point set and the local domain set using a scaled dot product attention function.

[0044] Based on the same concept, the present application also provides a time series prediction method, including:

[0045] Calculate the point-local domain attention dependency results according to the time series prediction modeling method described above;

[0046] The temperature and load time series of the power transformer are predicted based on the point-local domain attention dependency results.

[0047] Based on the same concept, the present application also provides an electronic device, including a memory, a processor, and a computer program / instructions stored in the memory, wherein the processor executes the computer program / instructions to implement the time series prediction modeling method or time series prediction method as described above.

[0048] Based on the same concept, the present application also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the time series prediction modeling method or time series prediction method as described above is implemented.

[0049] Based on the same concept, the present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the time series prediction modeling method or time series prediction method as described above.

[0050] Beneficial Effects

[0051] Compared with the prior art, the advantages of this application are:

[0052] The present application constructs a time point set and a local domain set for each time series, and embeds each time point of the time point set and each local domain of the local domain set into a high-dimensional space. The local domain set contains the local morphology, continuous features and time series features of the time series, which solves the problem that the attention mechanism of the traditional Transformer model only uses time point information but lacks feature information such as local morphology, continuous features and time series features, and improves the prediction accuracy of the temperature and load time series of the power transformer; the point-local domain attention dependency result obtained in the present application expresses the complex dependency relationship between the time point in the time series and the local features such as local morphology, continuous features and time series features before the time point, which solves the problem that the traditional Transformer model is difficult to effectively capture the complex dependency relationship of the time series, and improves the prediction performance of the temperature and load time series of the power transformer. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0054] Figure 1 It is a flow chart of the time series prediction modeling method in the embodiment of the present application;

[0055] Figure 2 It is a point-local domain attention dependency result calculation graph of the time series in the embodiment of the present application;

[0056] FIG3 (a) is a heat map of the point-local domain attention scores of the first layer of attention of the encoder in an embodiment of the present application;

[0057] FIG3( b ) is a heat map of the point-local domain attention scores of the second layer of attention of the encoder in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following is a clear and complete description of the technical solutions in this application in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0059] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0060] Figure 1 The flowchart of the time series prediction modeling method provided by the embodiment of the present application is shown. Figure 1 As shown, the time series forecasting modeling method includes the following steps:

[0061] Step S1: Construct a time series dataset.

[0062] The time series data set is a power transformer temperature and load time series data set. The temperature and load time series of the power transformer are collected, and a time series data set is constructed based on the collected temperature and load time series. The length of each time series in the time series data set is the same.

[0063] Step S2: Construct a set of time points and a set of local domains for each time series.

[0064] The specific construction process of the time point set of each time series is: obtain all the time points of the time series, and construct a time point set based on all the time points; wherein each time point in the time point set includes the position, time and attribute value of the time point, the position of the time point refers to the order of the time point in the time series, and the time of the time point is expressed in 8-digit year, month and day and 6-digit time, for example, 2024 / 11 / 30 11:00:00 (i.e., November 30, 2024 11:00:00), and the attribute value of the time point refers to the observed values ​​of multiple variables of the research object at that time point, such as the observed values ​​of 7 variables (load of high demand full load, load of high demand low load, load of medium demand full load, load of medium demand low load, load of low demand full load, load of low demand low load, and transformer oil temperature) at a certain time point for a power transformer.

[0065] The specific construction process of the local domain set of each time series is as follows: for each time point in the time series, obtain the position, time and attribute value of the time point and the position, time and attribute value of the n-1 time points before the time point, construct the local domain of the time point from the position, time and attribute values ​​of these n time points, and then construct the local domain set from the local domain of each time point. Where 1<n<N, N represents the length of the time series, and n is an integer.

[0066] Step S3: Build a prediction model.

[0067] The prediction model includes a first embedding module, a second embedding module and an attention module; wherein the first embedding module is used to embed each time point in the time point set into a high-dimensional space to obtain an embedding vector of the time point ; The second embedding module is used to embed each local domain in the local domain set into a high-dimensional space to obtain an embedding vector of the local domain ; The attention module is used to generate a query vector based on the embedding vector of each time point corresponding to the time point set , generate a key vector based on the embedding vector of each local domain corresponding to the local domain set Sum value vector ; and according to the query vector , key vector Sum value vector Calculate the point-local domain attention dependency results between the time point set and the local domain set, such as Figure 2 As shown, Figure 2 In, x i Represents the first A vector of a time point, which is composed of the position, time and attribute value of the time point. The prediction model of this embodiment adopts a Transformer model, which includes an encoder and a decoder. The encoder includes a first embedding module, a second embedding module and an attention module. The encoder also includes an adder and a feedforward layer. The adder and the feedforward layer are both existing structures of the traditional Transformer model, and the decoder is also an existing structure of the traditional Transformer model.

[0068] In a specific implementation of the present application, the first embedding module uses position embedding, time embedding and value embedding methods to embed each time point in the time point set into a high-dimensional space, specifically including:

[0069] Step S3.11: embed the position of each time point into a high-dimensional space to obtain a position embedding vector of the time point; wherein the specific embedding formula of the position embedding vector of the time point is:

[0070] (1)

[0071] (2)

[0072] = (3)

[0073] in, Indicates The embedding value of the position of each time point in the even-dimensional high-dimensional space; Indicates The location of a point in time; Represents the dimension of the high-dimensional space, and the dimension number starts from 0; , Respectively represent the even and odd dimensions of the high-dimensional space; Indicates The embedding value of the position of each time point in the odd dimension of the high-dimensional space; Indicates The position embedding vector of each time point.

[0074] Step S3.12: embed the time of each time point into a high-dimensional space to obtain a time embedding vector of the time point; wherein the specific embedding formula of the time embedding vector of the time point is:

[0075] (4)

[0076] (5)

[0077] in, Indicates The time point Types of time; M represents the number of types of time, including year, day, month, day, weekday, hour, and minute, that is, M = 5; Indicates time Index, index size of 5 times They are 366 (a year is 365.25 days, rounded to 366), 31, 7, 24, 60; Indicates The time point An embedding vector that maps time to a high-dimensional space; Represents an index The weight matrix of the linear transformation, the weight matrix The size is , Indicates by index Take a row of the weight matrix; Indicates The time embedding vector of each time point. The weight matrix The initial value of each element is assigned by uniform distribution sampling, and then the weight matrix is ​​adjusted by self-learning Each element of the weight matrix The value of each element of is learned during the automatic optimization training process using a time series dataset.

[0078] For example, the time point of the 5th hour on the 2nd day of 2024 can be expressed as: 2024 / 01 / 01 05:00:00, the index of the year-day is 2, and the index size of the year-day is 366.

[0079] Step S3.13: embed the attribute value of each time point into the high-dimensional space to obtain the attribute value embedding vector of the time point; wherein the specific embedding formula of the attribute value embedding vector of the time point is:

[0080] (6)

[0081] in, Indicates The attribute value embedding vector at each time point, Indicates The attribute value at a time point, Represents attribute value The weight matrix of the linear transformation of . The weight matrix The size is , Indicates The dimension of the attribute value at each time point, the weight matrix It is learned during the automatic optimization training process using time series datasets.

[0082] Step S3.14: Calculate the embedding vector of the time point according to the position embedding vector, the time embedding vector and the attribute value embedding vector of the time point; wherein the specific calculation formula of the embedding vector of the time point is:

[0083] (7)

[0084] in, Indicates The embedding vector of each time point.

[0085] In a specific implementation of the present application, the second embedding module embeds each local domain in the local domain set into a high-dimensional space, including:

[0086] Step S3.21: embed the position of the last time point in the local domain into the high-dimensional space to obtain the position embedding vector of the last time point; use the position embedding vector of the last time point as the position embedding vector of the local domain The calculation formula of the position embedding vector at the last time point is shown in Formula (1) to Formula (3).

[0087] Step S3.22: embed the time of the last time point in the local domain into the high-dimensional space to obtain the time embedding vector of the last time point; use the time embedding vector of the last time point as the time embedding vector of the local domain The calculation formula of the time embedding vector at the last time point is shown in formula (4) and formula (5).

[0088] Step S3.23: embed the attribute values ​​of the local domain at n consecutive time points into the high-dimensional space to obtain the attribute value embedding vector of the local domain; wherein the specific embedding formula of the attribute value embedding vector of the local domain is:

[0089] (8)

[0090] (9)

[0091] in, Indicates the intermediate quantity, Yes Embedding vectors mapped to high-dimensional space; Indicated by The value vector consisting of the attribute values ​​of n consecutive time points in a local domain , value vector The size is , Indicates The attribute value of the last time point of a local domain, n represents the length of the local domain; Represents a value vector The weight matrix of the linear transformation, the weight matrix The size is × , the weight matrix It is learned during the automatic optimization training process using time series datasets; Indicates The attribute value embedding vector of a local domain; Indicates taking the maximum value, Indicates the correction factor. represents the Leaky ReLU activation function, is 0.01.

[0092] Step S3.24: Calculate the embedding vector of the local domain according to the position embedding vector, the time embedding vector and the attribute value embedding vector of the local domain; wherein the specific calculation formula of the embedding vector of the local domain is:

[0093] (10)

[0094] in, Indicates The embedding vector of the local domain.

[0095] The attention module generates a query vector based on the embedding vector of each time point in the time point set, and generates a key vector and a value vector based on the embedding vector of each local domain in the local domain set. ( ) is combined into a query vector , the embedding vector of each local domain in the local domain set ( ) to form a key vector , value vector .

[0096] In a specific implementation of the present application, the attention module uses a scaled dot product attention function to calculate the point-local domain attention dependency result between the time point set and the local domain set. The specific formula is:

[0097] (11)

[0098] in, represents the scaled dot product attention function, and the superscript T represents the transpose.

[0099] Step S4: Train the prediction model using the time point set and local domain set of each time series in the time series data set to obtain a target prediction model.

[0100] During the training process, each time point in the time point set is embedded into a high-dimensional space using the first embedding module to obtain an embedding vector of the time point; each local domain in the local domain set is embedded into a high-dimensional space using the second embedding module to obtain an embedding vector of the local domain;

[0101] The attention module is used to generate a query vector according to the embedding vector of each time point corresponding to the time point set, and a key vector and a value vector are generated according to the embedding vector of each local domain corresponding to the local domain set;

[0102] The attention module is used to calculate the point-local domain attention dependency results between the time point set and the local domain set based on the query vector, key vector and value vector.

[0103] Example 2

[0104] In order to verify the effectiveness of the method of this application, the temperature and load prediction of the power transformer is taken as an example. The temperature and load of the power transformer are observed according to a cycle of 1 hour. The observation time interval is from July 1, 2016 to June 26, 2018. There are a total of 17421 observation records, that is, there are 17421 time points. A power transformer temperature and load time series is formed by N consecutive time points, and then the power transformer temperature and load time series data set is obtained. The power transformer temperature and load time series includes 8 fields, namely 1 observation time field (date), 6 power load attribute fields and 1 power temperature attribute field. The 6 power load attributes include high demand full load load (HUFL), high demand low load load (HULL), medium demand full load load (MUFL), medium demand low load load (MULL), low demand full load load (LUFL), low demand low load load (LULL), and 1 power temperature attribute is transformer oil temperature (OT).

[0105] According to steps S1 to S4 in Example 1 of the present application, the point-local domain attention score result between the time point set and the local domain set of the power transformer temperature and load time series can be calculated, that is, the correlation between the query vector (Q) and the key vector (K). Each sample in the power transformer temperature and load time series data set is a temperature and load time series. In this embodiment, the length of each temperature and load time series is N=96, that is, each temperature and load time series includes 96 time points. The power transformer temperature and load time series data set is divided into a training set and a test set. The total number of samples in the test set is 3389, 106 batches, 32 samples per batch, and each batch has 2 point-local domain attention score (i.e., the correlation between the query vector and the key vector) heat maps, a total of 212 heat maps.

[0106] The prediction model in this application adopts the Transformer model, which includes an encoder and a decoder. The encoder includes an input layer, a first embedding module, a second embedding module, an attention module, an adder, and a feedforward layer. The encoder is used to learn the deep features of the time series under the multi-head self-attention mechanism, and the decoder is used to restore the features. Figure 3 (a) and Figure 3 (b) respectively show the heat map of the point-local domain attention scores of the first batch of samples in the test set in the first layer of attention of the encoder and the heat map of the point-local domain attention scores of the second layer of attention of the encoder. The numbers on the ribbons in Figure 3 (a) and Figure 3 (b) represent the point-local domain attention scores. It can be seen from Figure 3 (a) and Figure 3 (b) that the point-local domain attention scores of the encoder's first-layer attention are generally high at the 1st to 48th time points (hours), indicating that the correlation between medium- and short-term time points is mainly captured, and the correlation captured at 49th to 96th time points (hours) gradually decreases; the point-local domain attention scores of the encoder's second-layer attention not only capture the correlation with longer time at the 49th to 96th time points (hours) as a whole, but also show that there is a relatively regular correlation between each time point (vertical arrangement band), and there is a piecewise continuous point-local domain equally spaced correlation (diagonal arrangement band) between the time points of the query vector and the time points of the key vector, reflecting the periodicity of time association in the time series.

[0107] According to the point-local domain attention calculation results of the power transformer temperature and load time series of this application, the temperature and load prediction for up to 96 time points (4 days and 96 hours in total) are further performed according to the encoder-decoder structure of Transformer, and the difference between the predicted value and the true value is calculated to obtain the mean square error (MSE) and mean absolute error (MAE). The mean square error (MSE) and mean absolute error (MAE) obtained by this application are compared with the traditional Transformer model, Informer model, and Autoformer model. The comparison results are shown in Table 1.

[0108] Table 1 Comparison results of mean square error (MSE) and mean absolute error (MAE) of each model

[0109]

[0110] From the comparison results in Table 1, it can be seen that the prediction results of the present application not only have a smaller prediction error than the traditional Transformer model, but also have a smaller prediction error than the improved models Informer and Autoformer of the Transformer model, indicating that the point-local domain attention of the power transformer temperature and load time series calculated by the embodiment of the present application can more effectively capture the complex dependencies of the time series, and can more accurately and reliably predict the temperature and load of the power transformer for up to 96 hours in the future, providing a more effective basis for power load regulation, transformer cooling system internal control, and equipment fault warning.

[0111] Example 3

[0112] A time series prediction method provided in an embodiment of the present application includes the following steps:

[0113] Step B1: Calculate the point-local domain attention dependency result according to the time series prediction modeling method of Example 1 of the present application;

[0114] Step B2: Predict the temperature and load time series of the power transformer according to the point-local domain attention dependency result calculated in step B1.

[0115] Example 4

[0116] An embodiment of the present application also provides an electronic device, which includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the time series prediction modeling method or time series prediction method in the embodiment of the present application.

[0117] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0118] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0119] Although not shown, the embodiments of the present application also provide a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the time series prediction modeling method or the time series prediction method in the embodiments of the present application is implemented.

[0120] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0121] Although not shown, the embodiments of the present application also provide a computer program product, including: a computer program / instruction, which, when executed by a processor, implements the time series prediction modeling method or time series prediction method in the embodiments of the present application.

[0122] The above disclosure is only a specific implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or modifications within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A time series prediction modeling method, characterized in that: The modeling method comprises: Constructing a time series data set; wherein the time series data set is a time series data set of power transformer temperature and load, and the time series data set includes a plurality of time series of equal length; For each time series in the time series data set, construct a time point set and a local domain set of the time series; wherein each time point in the time point set includes the position, time and attribute value of the time point, and each local domain in the local domain set includes the position, time and attribute value of n consecutive time points, 1<n<N, N represents the length of the time series; Constructing a prediction model, wherein the prediction model includes a first embedding module, a second embedding module, and an attention module; The prediction model is trained using a time point set and a local domain set of each time series in the time series data set to obtain a target prediction model; In the prediction model, each time point in the time point set is embedded into a high-dimensional space using the first embedding module to obtain an embedding vector of the time point; each local domain in the local domain set is embedded into a high-dimensional space using the second embedding module to obtain an embedding vector of the local domain; The attention module is used to generate a query vector according to the embedding vector of each time point corresponding to the time point set, and a key vector and a value vector are generated according to the embedding vector of each local domain corresponding to the local domain set; and a point-local domain attention dependency result between the time point set and the local domain set is calculated according to the query vector, the key vector and the value vector.

2. The time series prediction modeling method according to claim 1, characterized in that: Embedding each time point in the set of time points into a high-dimensional space using the first embedding module includes: Embed the position of each time point into a high-dimensional space to obtain the position embedding vector of the time point; Embed the time of each time point into a high-dimensional space to obtain the time embedding vector of the time point; Embed the attribute value at each time point into a high-dimensional space to obtain the attribute value embedding vector at that time point; An embedding vector of a time point is calculated according to the position embedding vector, the time embedding vector and the attribute value embedding vector of the time point.

3. The time series prediction modeling method according to claim 2, characterized in that: The specific embedding formula of the position embedding vector at the time point is: = ； ， ； in, Indicates The position embedding vector of each time point; Indicates The embedding value of the position of each time point in the even-dimensional high-dimensional space; Indicates The embedding value of the position of each time point in the odd dimension of the high-dimensional space; Indicates The location of a point in time; Represents the dimension of the high-dimensional space, and the dimension number starts from 0; The specific embedding formula of the time embedding vector at the time point is: ； ； in, Indicates The time embedding vector of each time point, Indicates The time point time, M represents the number of time. Indicates The time point A kind of embedding vector that maps time to a high-dimensional space, Indicates time The index of Represents an index The weight matrix of the linear transformation of ; The specific embedding formula of the attribute value embedding vector at the time point is: ； in, Indicates The attribute value embedding vector at each time point, Indicates The attribute value at a time point, Represents attribute value The weight matrix of the linear transformation of ; The specific calculation formula of the embedding vector at the time point is: ； in, Indicates The embedding vector of each time point.

4. The time series prediction modeling method according to claim 1, characterized in that: Embedding each local domain in the local domain set into a high-dimensional space using the second embedding module, comprising: Embed the position of the last time point in the local domain into a high-dimensional space to obtain a position embedding vector of the last time point; use the position embedding vector of the last time point as the position embedding vector of the local domain; Embed the time of the last time point in the local domain into a high-dimensional space to obtain a time embedding vector of the last time point; use the time embedding vector of the last time point as the time embedding vector of the local domain; Embed the attribute values ​​of the local domain at n consecutive time points into a high-dimensional space to obtain an attribute value embedding vector of the local domain; An embedding vector of the local domain is calculated according to the position embedding vector, the time embedding vector and the attribute value embedding vector of the local domain.

5. The time series prediction modeling method according to claim 4, characterized in that: The specific embedding formula of the attribute value embedding vector of the local domain is: ； ； in, Indicates The attribute value embedding vector of the local domain, Indicates taking the maximum value, represents the correction factor, Indicates the intermediate quantity, Indicated by The value vector composed of the attribute values ​​of n consecutive time points in a local domain, Represents a value vector The weight matrix of the linear transformation of ; The specific calculation formula of the embedding vector of the local domain is: ； in, Indicates The embedding vector of the local domain, Indicates The position embedding vector of the local domain, Indicates The temporal embedding vector of the local domain.

6. The time series prediction modeling method according to any one of claims 1 to 5, characterized in that: The attention module is used to calculate the point-local domain attention dependency result between the time point set and the local domain set using a scaled dot product attention function.

7. A time series prediction method, characterized in that: The prediction method comprises: Calculate the point-local domain attention dependency result according to the time series prediction modeling method described in any one of claims 1 to 6; The temperature and load time series of the power transformer are predicted based on the point-local domain attention dependency results.

8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the time series prediction modeling method as described in any one of claims 1 to 6 or the time series prediction method as described in claim 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instruction is executed by a processor, it implements the time series prediction modeling method as described in any one of claims 1 to 6 or the time series prediction method as described in claim 7.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, it implements the time series prediction modeling method as described in any one of claims 1 to 6 or the time series prediction method as described in claim 7.

Images