Method and device for predicting multivariate time series

Abstract

The embodiments of this specification provide a method and device for predicting multivariate time series. For the historical time series of each business entity, time series coding is performed on the one hand, and period coding is performed on the other hand, so as to mine the correlation between data from both time series and period. In the time series prediction process, the data of time series coding and period coding are fused and decoded to obtain the corresponding decoded tensor, and the decoded tensor is used together with the time series feature tensor to predict multivariate time series. This method can adaptively mine the correlation between time series and period, and provide more accurate time series prediction results more flexibly.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer technology, and more particularly, to a method and apparatus for predicting multivariate time series. Background Art

[0002] A time series (or dynamic series) is a series of values ​​of the same statistical indicator arranged in the order of their occurrence. The main purpose of time series analysis is to predict the future based on existing historical data. Time series prediction can be applied to various scenarios, such as the time series prediction of customer flow in commodity supermarkets, the time series prediction of funds in financial services, the prediction of computing resource flow required in cloud computing, logistics demand, and power consumption prediction in smart grids, etc. The prediction results can serve business decisions, for example.

[0003] Multivariate time series refers to a series of values ​​of the same statistical indicator in multiple time periods of the same length arranged in the order of their occurrence. Multivariate series prediction is needed in many scenarios, such as: time series prediction of sales of multiple commodities; time series prediction of funds from multiple channels in finance; prediction of the flow of multiple computing resources required in cloud computing; and so on. Classical prediction methods include methods such as the VAR vector autoregression model based on the concept of cointegration in econometrics. These methods are usually based on strong statistical assumptions. However, the data distribution in the real world is more complex. For example, in financial scenarios, the data distribution between different financial data is intricate, and different time series often have different periodic characteristics. Therefore, how to mine the complex relationship between multivariate time series is an important issue in the process of multivariate time series prediction. Summary of the invention

[0004] One or more embodiments of this specification describe a method and apparatus for predicting a multivariate time series, so as to solve one or more problems mentioned in the background technology.

[0005] According to a first aspect, a method for predicting a multivariate time series is provided, which is used to predict the time series of multiple business entities, and the method comprises: extracting a time series feature tensor from each historical multivariate time series corresponding to each piece of historical business data sampled from the multiple business entities, the time series feature tensor corresponding to three dimensions of business data record, time, and feature value; performing time series period processing on the time series feature tensor according to the periodicity of the time dimension to obtain a periodic coding tensor in the three dimensions; performing time series coding on each piece of historical business data through the time series coding module to obtain a time series coding tensor in the three dimensions; performing fusion and decoding processing on the time series coding tensor and the periodic coding tensor based on the time series decoding module to obtain a decoded tensor corresponding to the three dimensions; predicting the multivariate time series corresponding to the multiple business entities through the prediction module based on the time series feature tensor and the decoding tensor.

[0006] In one embodiment, the eigenvalue dimension of the time series feature tensor includes multidimensional time embedding features obtained by embedding time information and business features in historical multivariate time series.

[0007] In one embodiment, the time information corresponds to multiple time levels.

[0008] In one embodiment, the time series feature tensor is subjected to time series periodic processing according to the periodicity of the time dimension to obtain a periodic coding tensor in the three dimensions, including: dividing the time series feature tensor along the time dimension according to a predetermined periodic window to reorganize the time series feature tensor into a periodic feature tensor; processing each two-dimensional periodic feature tensor corresponding to each periodic unit in the periodic feature tensor in sequence through a first recurrent neural network, and the obtained coding tensors are arranged in order as the periodic coding tensor.

[0009] In a further embodiment, compared with the timing feature tensor, the periodic feature tensor corresponds to a first number of periodic units in the time dimension, and the number of units in the historical business data record dimension is increased to a second number, the first number being the quotient of the number of time units of the timing feature tensor in the time dimension divided by the predetermined period window, and the second number being the product of the number of business entities and the predetermined period window.

[0010] In one embodiment, a single historical time series is composed of business data of multiple time units; the time encoding of each piece of historical business data to obtain the time encoding tensor in the three dimensions includes: sequentially processing the two-dimensional business tensor composed of the business data of the multiple business entities in each time unit through a second recurrent neural network to obtain each two-dimensional time encoding tensor; and obtaining the time encoding tensor based on the sequential arrangement of each two-dimensional time encoding tensor along the time dimension.

[0011] In one embodiment, the fusion and decoding processing of the time coding tensor and the period coding tensor to obtain the decoding tensor corresponding to the three dimensions includes: fusing the time coding tensor and the period coding tensor along the time dimension according to each time unit to obtain a fused tensor corresponding to the three dimensions; using a third recurrent neural network to process the two-dimensional tensors corresponding to each time unit in the fused tensor in turn according to the time dimension to obtain the decoding tensor.

[0012] In a further embodiment, the fusion of the temporal coding tensor and the periodic coding tensor on a single time unit is achieved by splicing.

[0013] In another further embodiment, the fusion of the time coding tensor and the period coding tensor on a single time unit is achieved in the following manner: processing the two-dimensional time coding tensor on the single time unit based on the self-attention mechanism to obtain a single two-dimensional attention tensor; fusing the single two-dimensional attention tensor with the two-dimensional period coding tensor of the period coding tensor on the single time unit to obtain a single two-dimensional fused tensor.

[0014] In a further embodiment, a single two-dimensional attention tensor is fused with a single two-dimensional periodic code by one of the following methods: concatenating the single two-dimensional periodic code tensor with a single two-dimensional attention vector; or multiplying the single two-dimensional attention vector with the elements in the single two-dimensional periodic code tensor one by one.

[0015] In one embodiment, predicting the multivariate time series corresponding to the multiple business entities based on the timing feature tensor and the decoding tensor includes: concatenating the timing feature tensor and the decoding tensor into a prediction tensor; and processing the prediction tensor based on an attention mechanism to obtain a two-dimensional multivariate time series corresponding to two dimensions of time and feature.

[0016] According to a second aspect, a device for predicting a multivariate time series is provided, which is used to simultaneously predict the time series of multiple business entities through a prediction model, wherein the prediction model includes a time series encoding module, a time series cycle processing module, a time series decoding module and a prediction module; the device includes:

[0017] A feature extraction unit is configured to extract a time series feature tensor from each historical multivariate time series corresponding to each piece of historical business data sampled from the plurality of business entities, wherein the time series feature tensor corresponds to three dimensions: business data record, time, and feature value;

[0018] A periodic coding unit, configured to perform time series periodic processing on the time series feature tensor according to the periodicity of the time dimension to obtain a periodic coding tensor in the three dimensions;

[0019] A time series encoding unit, configured to perform time series encoding on each piece of historical business data to obtain a time series encoding tensor in the three dimensions;

[0020] A decoding unit, configured to fuse and decode the temporal coding tensor and the periodic coding tensor to obtain a decoding tensor corresponding to the three dimensions;

[0021] The prediction unit is configured to predict the multivariate time series corresponding to the plurality of business entities according to the time series feature tensor and the decoding tensor.

[0022] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0023] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method of the first aspect is implemented.

[0024] Through the method and device provided by the embodiments of this specification, in the process of predicting a multivariate time series, after extracting the corresponding time series features from each historical time series corresponding to each piece of historical business data sampled from each business entity, on the one hand, a three-dimensional coding tensor related to the time series period is determined based on the time series features, and on the other hand, a three-dimensional coding tensor of the time series is determined based on the time series features, and then the three-dimensional coding tensor related to the period and the three-dimensional coding tensor of the time series are fused, and the fusion result is decoded to obtain a three-dimensional decoded tensor. Then, the decoded tensor is fused with the extracted initial time series features through an attention mechanism to obtain a multivariate time series predicted for multiple business entities. This technical conception fully exploits the periodicity of the time series and the correlation between each time series, avoiding the standardized process through strong assumptions, so that it can adapt to various complex time series distribution situations in reality and improve the accuracy and flexibility of multivariate time series prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0026] Figure 1A schematic diagram of an implementation architecture of this specification is shown;

[0027] Figure 2 A flowchart of an improved method for predicting multivariate time series is shown;

[0028] Figure 3 A schematic diagram showing a specific example of a three-dimensional tensor;

[0029] Figure 4 A schematic diagram of a decoding processing architecture according to a specific example is shown;

[0030] Figure 5 A schematic block diagram of an apparatus for predicting a multivariate time series according to an embodiment is shown. DETAILED DESCRIPTION

[0031] The technical solution provided in this specification is described below in conjunction with the accompanying drawings.

[0032] It can be understood that a business subject can be an entity that can independently distinguish and count the corresponding business, such as a store, a category of goods, goods or stores in units of regions, and so on. A single business subject can correspond to the business volume (or other business data) on a predetermined business indicator within a single time period. The business volume of multiple time periods can constitute a time series, which can also be referred to as a "time series". Among them, the various time periods of the sampling time series can be continuous or intermittent, and this specification does not limit this. For example, in one embodiment, a single time period is one week, and data from multiple consecutive weeks constitute a time series. In another embodiment, a single time period is one Monday, and data from multiple Mondays constitute a time series, but the data between two adjacent Mondays is intermittent. A time series that has been generated before a certain time point can be called a historical time series relative to that time point and the time points thereafter.

[0033] The time series of the same business indicator of multiple business entities in the same multiple time periods are called multivariate time series. The generated multivariate time series are called historical multivariate time series. Multivariate time series can also be regarded as tensors. The process of processing historical multivariate time series and predicting future multivariate time series is the process of predicting multivariate time series. Among them, the business entities in the process of predicting multivariate time series are different according to different specific business scenarios. For example, in the shopping platform scenario, multiple business entities can be various commodities, the corresponding business indicators are, for example, sales, and the corresponding time period can be a week, a day, a month, etc., then the specific sales (i.e., business volume) in multiple time periods can constitute a time series. Similarly, in the investment and financial management scenario, multiple business entities can be various investment channels or various financial products, such as insurance, funds, fixed deposits, stocks, etc., or insurance types, fund types, bank fixed deposit product types, stock entities, etc., and the business indicators described by the time series can be investment amounts, investment ratios, number of investment users, etc. The specific values ​​(business volumes) corresponding to a single business indicator constitute a time series. In other business scenarios, time series can also describe the business volume of other business entities on other business indicators. For example, for the same type of goods (such as brushes, toothpaste, etc.), the business volume on business indicators such as click volume, sales volume, and page views can form a time series, which will not be repeated here.

[0034] Conventional technology can estimate the dynamic relationship of joint endogenous variables in multivariate time series prediction scenarios through vector autoregression models (VAR models) based on econometrics, etc., without prior constraints. Taking the VAR model as an example, the model is established based on the statistical properties of the data, and each endogenous variable in the system is constructed as a function of the lagged values ​​of all endogenous variables in the system to construct the model, thereby generalizing the univariate autoregression model to a "vector" autoregression model composed of multivariate time series variables. However, the VAR model relies on the need to satisfy the "cointegration" relationship between multivariate time series - if the time series considered have the same single integral order, and a certain linear combination (cointegration vector) reduces the single integral order of the combined time series, then these time series are said to have a significant cointegration relationship. Therefore, the VAR model requires strong statistical assumptions. However, the data distribution in the real world is more complex, and some time series data often satisfy some periodic characteristics.

[0035] In view of this, this specification provides a technical concept for predicting multivariate time series, which builds a new prediction model architecture based on neural networks, attention mechanisms, etc., which not only analyzes the characteristics of the time series itself, but also mines the periodic characteristics contained in the time series. Optionally, the attention mechanism can be used to fuse data according to their importance. This technical concept can adaptively mine various associations in multivariate time series data, thereby better adapting to the complex data distribution in reality and providing better time series prediction effects.

[0036] Figure 1 The following is a schematic diagram of an implementation architecture based on the technical concept of this specification. Figure 1 As shown, under this implementation architecture, it is assumed that corresponding time series can be predicted for multiple business entities, that is, multivariate time series can be predicted. First, each of these multiple business entities can correspond to a historical time series. Figure 1 It can be seen that the processing of historical time series is divided into two aspects. On the one hand, each historical time series is encoded (such as through a timing coding module such as an RNN network) to obtain a timing coding tensor. The timing coding tensor contains the timing characteristics of each historical time series. On the other hand, a two-dimensional timing feature tensor can be obtained through embedding through a single historical time series, and the two-dimensional timing feature tensors corresponding to multiple business entities can constitute a three-dimensional timing feature tensor. Furthermore, the timing feature sequence is processed through a timing period processing module (such as an RNN network) to obtain a period coding tensor. The period coding tensor fully exploits the periodic characteristics of each historical time series.

[0037] It can be understood that both the time-series coding tensor and the period coding tensor have three dimensions: historical business data record dimension, time dimension, and eigenvalue dimension (output dimension of the corresponding neuron), that is, they are all three-dimensional tensors. Among them, the time-series coding tensor can correspond to the time point or time period corresponding to the time series (such as time-steps) as the time unit in the time dimension.

[0038] Then, the time-series coding tensor and the cycle coding tensor can be decoded to obtain a decoded tensor. In a possible design, during the decoding process of the time-series coding tensor and the cycle coding tensor, different weights are given to each data element based on the attention mechanism. Figure 1As shown, in one embodiment, the time series coding tensor can be processed with a self-attention mechanism to mine the mutual correlation therein and obtain an attention tensor. The attention tensor is used to decode the periodic coding tensor. For example, the periodic coding tensor can be expanded in the time dimension and cyclically decoded, such as by a recurrent neural network, decoding a two-dimensional periodic coding tensor corresponding to a time unit at a time. During decoding, the attention vector can be fused with the corresponding two-dimensional tensor (such as concat, etc.), and the spliced ​​tensor can be processed using a decoding network (such as an RNN network) to obtain a decoded two-dimensional tensor. The individual decoding results are spliced ​​together according to the time step dimension to form a three-dimensional decoding tensor.

[0039] Furthermore, the decoding tensor can be more deeply fused with the time series feature tensor to output a multivariate time series predicted for multiple business entities, namely the predicted time series.

[0040] This processing method fully exploits the periodic features in the time series, and thus obtains the prediction results of the multivariate time series together with the time series feature tensor of the historical time series, and exploits more data correlations reflected in the time series, thereby improving the adaptability and prediction accuracy of multivariate predictions in various complex distributions. Optionally, an attention mechanism can be introduced to explore the importance of mutual contribution of features, and to consider various features more carefully, further improving the accuracy of the prediction results.

[0041] Reference below Figure 2 A specific example is shown to describe the technical concept of this specification in detail.

[0042] Figure 2 The process of predicting multivariate time series of an embodiment is shown. The execution subject of the process can be a computer, device or server with certain computing capabilities. The process can be used to predict the time series of multiple business entities for predetermined businesses.

[0043] like Figure 2As shown, the process of predicting multivariate time series can include the following steps: Step 201, extracting time series feature tensors from each historical multivariate time series corresponding to each piece of historical business data sampled from multiple business entities, and the time series feature tensors correspond to three dimensions: business data record, time, and feature value; Step 202, using the time series period processing module to process the time series feature tensor according to the periodicity of the time dimension, and obtain the periodic coding tensor in three dimensions; Step 203, using the time series coding module to perform time series coding on each piece of historical business data, and obtain the time series coding tensor in three dimensions; Step 204, based on the time series decoding module, the fusion and decoding processing of the time series coding tensor and the period coding tensor is performed to obtain the decoding tensor corresponding to the three dimensions; Step 205, predicting the multivariate time series corresponding to multiple business entities according to the time series feature tensor and the decoding tensor. First, in step 201, extract the time series feature tensor from each historical multivariate time series corresponding to each piece of business data sampled from multiple business entities. It can be understood that in the multivariate time series prediction process, the corresponding time series can be predicted for multiple business entities at the same time. The initial data (ie, input data) for prediction are the historical time series corresponding to each business entity.

[0044] The business data corresponding to each of the multiple historical predetermined time units (such as time points, time periods, etc.) arranged in sequence by a single business subject can constitute a single historical time series. A single predetermined time unit can be set according to business needs, such as one day, one week, one month, etc. Multiple time units can be continuous or discontinuous. Taking a single predetermined time unit of 1 day as an example, if the sales volume for multiple consecutive days is sampled, then for a single merchant as the business subject, the sales volume sampled for multiple consecutive days (such as 60 days) can be used as its historical time series, and the multiple time units are continuous; if only the sales volume on weekends is sampled, then for a single merchant as the business subject, the sales volume sampled for multiple weekends (such as 52 weekends) can be used as its historical time series, and the multiple time units are discontinuous. The historical time series can be represented by a multi-dimensional vector, array, or by a set, etc., which is not limited here.

[0045] A historical multivariate time series can correspond to a combination of historical time series collected by multiple business entities at a certain time point. For example, multiple business entities are product 1, product 2, and product 3, each of which corresponds to a time series at time point 1 (a 11 , a 12 , a 13 )、(b 11 , b 12 , b 13 )、(c 11 , c 12 , c 13), then the corresponding multivariate time series can be a very boring splicing result (a 11 , a 12 , a 13 , b 11 , b 12 , b 13 , c 11 , c 12 , c 13 ).

[0046] A single piece of business data can be data composed of multiple sampling historical multivariate time series of the above multiple business entities. In other words, a single piece of business data can include multiple historical multivariate time series corresponding to multiple time points. Each multivariate time series can correspond to a different collection time, so that, when arranged in the time dimension, it can constitute two-dimensional business data. As shown in Table 1, it is a schematic diagram of a single piece of business data.

[0047] Table 1. Schematic diagram of business data of multivariate time series

[0048]

[0049] In Table 1, each row can correspond to a time point of data collection, forming a time dimension in the direction of the column (such as vertically). Each column can be used as a feature, thereby forming a time series dimension along the direction of the row (horizontally). It is worth noting that in the time series dimension, two columns (one or more columns in time) of time information are also included. The time information here is used to describe the time of multivariate time series collection of the corresponding row (such as the aforementioned time point 1, etc.). For example, there are two time information corresponding to the column, one is the time of the day (such as time of day), and the other is the time of the week (such as day of week). The first row of business data (such as corresponding to time point 1) can include time information (9 o'clock, Monday) and multivariate time series. Similarly, the second and third rows of business data can also include time information and corresponding multivariate time series. In this way, a row of data can be called a time unit (time-steps), but in fact there is no necessary time ordering relationship between rows.

[0050] According to Table 1, a single piece of business data can have two dimensions (corresponding to rows and columns, respectively). Then, multiple pieces of business data can be arranged together to form three-dimensional data (corresponding to three dimensions). Furthermore, the corresponding time series features can be extracted through the historical time series. If the time series features are represented by tensors, they are called time series feature tensors.

[0051] Since a single piece of business data contains both time information and multi-dimensional time series features, the corresponding time series features can include time-related features and time series-related features. In order to better describe the time features in business data, the time information can be embedded (embedding) to obtain time embedding features of a predetermined dimension as time-related features. For example, in Table 1, the time information "9 o'clock" and "Monday" in the first row are embedded to obtain multi-dimensional (such as 6-dimensional) time features, such as (t 11 , t 12 , t 13 , t 14 , t 15 , t 16 ), etc. Time series related features can be the multivariate time series itself, or embedded features obtained by embedding the multivariate time series, collectively referred to as business features of historical multivariate time series. Time embedding features and business features together constitute the feature dimension in the time series feature tensor. For example, a row of business data corresponds to (t 11 , t 12 , t 13 , t 14 , t 15 , t 16 , a 11 , a 12 , a 13 , b 11 , b 12 , b 13 , c 11 , c 12 , c 13 ).

[0052] In this way, each line of a business data constitutes a tensor in two dimensions: time and feature. After extracting features from multiple business data, a time series feature tensor in three dimensions: business data record, time, and feature is formed. For example, it can be recorded as: (batch-size, time-steps, feat-dim), where batch-size, time-steps, and feat-dim correspond to the business data record dimension, time dimension, and feature dimension, respectively. Figure 3 A schematic diagram of a three-dimensional tensor is shown. Figure 3 As shown, a three-dimensional tensor may include data in three dimensions. A two-dimensional tensor along the time dimension corresponds to a time unit (time-step), and a two-dimensional tensor along the business data record corresponds to a piece of business data.

[0053] Next, through step 202, the time series feature tensor is processed according to the periodicity of the time dimension to obtain a periodic coding tensor in three dimensions. It is easy to understand that things usually show a certain periodicity in the process of movement and change. Periodicity is a natural law. For example, time has periodicity such as day and night, four seasons, the rotation of the earth has periodicity, and people's living habits and consumption habits can also have periodicity. Furthermore, the time series used to describe the business can also have periodicity. In order to explore this periodicity, the time series feature tensor can be encoded to obtain a periodic coding tensor.

[0054] In order to explore the periodicity of the time series feature tensor, the time series feature tensor can be segmented in the time dimension. For example, several time units (hereinafter referred to as period windows) can be divided into one period. That is to say, data is reorganized according to a predetermined period window in the time dimension, and a period window can contain multiple time units. Among them, how many time units (i.e., period windows) a period window contains can be determined by the user based on experience, assuming that it is recorded as period-window-size, such as the corresponding multiple rows in Table 1. The time series feature tensor corresponding to the historical multivariate time series can be divided into n periods in total, such as n-period, for the convenience of description, it can be called the first quantity. The first quantity is determined according to the initial number of time units and the predetermined period window, such as the quotient of time-steps divided by period-window-size.

[0055] In this way, after the time series feature tensor is processed by period division, the original time dimension is replaced by the period unit instead of the time unit, thus becoming a period feature tensor. Since the total dimension of the time series feature tensor remains unchanged, the dimension of the time dimension is reduced, and the shape of the tensor changes. For example, the number of units in the business data record dimension can be increased. In other words, it is equivalent to flattening the data in the time dimension along the business data record dimension. For example, the three-dimensional tensor can be changed from (batch-size, time-steps, feat-dim) to (batch-size*period-window-size, n-period, feat-dim). Among them, the number of units in the business data record dimension changes from batch-size to batch-size*period-window-size, such as recorded as the second number. The second number can be the product of the number of business entities and the predetermined period window. In this way, compared with the time series feature tensor, the total number of elements of the period feature tensor remains unchanged, but it is deformed while maintaining the overall dimension unchanged by taking fewer values ​​in the time dimension (thinning of the three-dimensional tensor) and more values ​​in the business data record dimension (thickening of the three-dimensional tensor).

[0056] In this way, one period unit in the time dimension corresponds to data from one or more time periods. On the one hand, the data from one period unit can describe the changing pattern of data within the period, for example, the sales volume of a certain commodity is higher at the time points around the three meals, and lower at other time points. On the other hand, the continuous data of multiple period units may describe the long-term changing characteristics of the business, and the earlier period affects the data of the later period. Therefore, the period feature tensor can be processed according to the periodicity of the time dimension to extract deep features related to the period. Here, this deep feature can be called a coding feature.

[0057] In one embodiment, the processing of the periodic feature tensor according to the periodicity of the time dimension can be implemented, for example, by a recurrent neural network (RNN), referred to herein as a first recurrent neural network. The neurons of the first recurrent neural network can process data corresponding to one periodic unit at a time. It can be understood that one unit of the periodic dimension includes a two-dimensional matrix tensor of (batch-size*period-window-size, feat-dim), and therefore, the neurons of the first recurrent neural network can process one two-dimensional periodic feature tensor at a time.

[0058] The periodic encoding tensor obtained by encoding the periodic feature tensor by the first recurrent neural network can still be a three-dimensional tensor, for example, a three-dimensional tensor in the three dimensions of business data, time, and features, such as (batch-size, time-steps, num-cells 1 ). Among them, num-cells represents the output dimension of the recurrent neural network, that is, the encoding feature dimension, and the superscript 1 represents the first recurrent neural network to distinguish it from other recurrent neural networks. Here, the time unit time-steps is restored after the encoding time dimension, which can be understood as restoring the shape of the periodic encoding tensor to be consistent with the time series feature tensor.

[0059] On the other hand, through step 203, each piece of historical business data is time-series encoded to obtain a time-series encoding tensor in three dimensions. Since the historical multivariate time series describes the time-series changes of each business entity, it can also be time-series encoded to mine the chronological correlation features between time series. This step 203 is implemented, for example, by a second recurrent neural network.

[0060] According to one embodiment, the historical multivariate time series in each piece of business data constitutes a three-dimensional data array, and the second recurrent network can process these data along the time dimension. For example, the data corresponding to each time unit can be processed in turn by the second recurrent neural network to obtain a two-dimensional coded tensor in the two dimensions of business data and features. In this way, N time units can obtain corresponding N two-dimensional coded tensors. The N two-dimensional coded tensors are arranged in sequence to form a (batch-size, time-steps, num-cells) array. 2 ), corresponding to the three dimensions of business data, time, and coding features, as a time series coding tensor.

[0061] According to another embodiment, each piece of business data may be arranged into a three-dimensional time series tensor, and the three-dimensional time series tensor may be processed by an unrolled recurrent neural network (Unrolling-RNN). Among them, one recurrent unit of the unrolled recurrent neural network may correspond to one time unit, and is used to process the two-dimensional data corresponding to the corresponding time unit. In this way, a (batch-size, time-steps, num-cells) 2 ) in the three dimensions of business data record, time, and coding features as a time series coding tensor.

[0062] The above num-cells represents the output dimension of the recurrent neural network, that is, the encoding feature dimension, and the superscript 2 can represent a recurrent neural network for encoding historical time series, or a second recurrent neural network, to distinguish it from other recurrent neural networks. In other embodiments, time series features can also be encoded by other networks such as convolutional neural networks, which will not be described in detail here.

[0063] In the above steps 201, 202, and 203, by encoding each piece of business data according to the period or encoding it itself, the data before and after correlation, the periodicity of the time series characteristics, the periodic correlation of the time series, etc. in the historical time series are mined to obtain the corresponding periodic coding tensor and time series coding tensor. It is worth noting that steps 201 and 202 are periodic coding steps for historical multivariate time series data, and step 203 is a time series coding step for historical multivariate time series data. For the historical multivariate time series in multiple business data, the periodic coding step and the sequence coding step can be performed in parallel or in a reversed order, which is not limited here.

[0064] Next, in step 204, the time series coding tensor and the period coding tensor are fused and decoded to obtain a decoded tensor corresponding to three dimensions. It can be understood that coding is an abstraction of historical multivariate time series to extract hidden features in historical multivariate time series. In order to predict multivariate time series, it is also necessary to decode the coding tensor to obtain the corresponding decoded tensor. The time series coding tensor and the period coding tensor reflect the changing laws of business data in the historical multivariate time series from different angles. Therefore, in the decoding process, the two can be fused.

[0065] The fusion of the temporal coding tensor and the periodic coding tensor can be performed in the time dimension. For example, the two-dimensional tensors corresponding to the same time unit (such as the first one) are fused together. The fused tensor is then processed by a pre-trained decoder or decoding network to obtain a decoded tensor.

[0066] According to a possible design, the fusion of the time-series coding tensor and the period coding tensor is performed, for example, by splicing, corresponding weighted summation, etc. Among them, in the splicing method, for example, splicing can be performed on the feature dimension, that is, in a single time unit, the two-dimensional tensor in the time-series coding tensor and the two-dimensional tensor in the period coding tensor are expanded on the feature dimension, so that the feature dimension increases. For example, the obtained fused tensor is (batch-size, time-steps, num-cells 1 +num-cells 2 ). In the corresponding weighted summation method, the feature dimensions of the two are usually the same. Then, in a single time unit, the two-dimensional tensor in the time-series coding tensor and the two-dimensional tensor in the periodic coding tensor can be weighted summed accordingly, which is equivalent to the weighted summation of the corresponding features in the time-series coding tensor and the corresponding features in the periodic coding tensor for each feature in the fusion tensor.

[0067] It can be understood that the fused tensor is a three-dimensional tensor. Therefore, in an optional implementation, a decoder or a decoding neural network can be constructed by a convolutional neural network, etc., and the fused tensor can be processed to obtain a decoding tensor. In addition, along the time step dimension, it can be regarded as a series of data composed of multiple two-dimensional tensors. Therefore, in another optional implementation, each two-dimensional tensor can be processed in sequence along the time dimension by a recurrent neural network to obtain a decoding tensor. In order to distinguish it from the recurrent neural network in the previous article, the recurrent neural network here can be called the third recurrent neural network.

[0068] In another possible design, in order to further explore the importance features in the time series data during the decoding process, the time series coding tensor can also be processed based on the self-attention mechanism to obtain each two-dimensional time series coding tensor corresponding to each time unit to obtain each two-dimensional attention tensor. Then, the third recurrent neural network is used to process each two-dimensional attention tensor in turn. During the processing, for a single two-dimensional attention tensor, the two-dimensional periodic coding tensor of the periodic coding tensor on the corresponding time unit is also integrated to fuse the periodic features of the time series data. The single two-dimensional attention tensor and the corresponding single two-dimensional periodic coding tensor can be fused in one of the following ways: splicing the single two-dimensional periodic coding tensor with the single two-dimensional attention tensor; determining the weight parameters of each element in the single two-dimensional attention tensor based on the decoding of the single two-dimensional periodic coding tensor, so as to correct each element in the single two-dimensional attention tensor according to the corresponding weight parameters. For example, the single two-dimensional periodic coding tensor is multiplied one by one with each element in the single two-dimensional attention tensor.

[0069] Figure 4 A schematic diagram of a decoding architecture is shown. Figure 4 As shown, the decoding operation can be implemented by a recurrent neural network. Under this decoding architecture, the time-series coding tensor is split into individual two-dimensional time-series coding tensors along the time dimension. Then each of the split two-dimensional periodic coding tensors is processed by the attention mechanism to obtain the corresponding attention tensor. The attention mechanism is a processing mechanism that can quickly capture the information contained in the data. The attention tensor here is equivalent to extracting important information from the time-series coding tensor. Optionally, a self-attention mechanism can be used here. The self-attention mechanism is an improvement on the attention mechanism, which reduces the dependence on external information and is better at capturing the internal correlation of data or features. The attention vector determined by the self-attention mechanism can describe the correlation between the elements inside the corresponding two-dimensional tensor.

[0070] Afterwards, each attention tensor is processed in sequence by a recurrent neural network (such as the third recurrent neural network) to perform time series decoding. In the process of processing a single attention tensor, the two-dimensional periodic coding tensor on the corresponding time unit is fused. The attention tensor can be fused with the corresponding two-dimensional periodic coding tensor, and after fusion, it is decoded as a whole tensor to obtain a two-dimensional decoding tensor corresponding to the corresponding time unit. The two-dimensional decoding tensors corresponding to each time unit are arranged together to form a decoding tensor for the time series coding tensor.

[0071] The decoded tensor contains the time period characteristics of each business entity and the correlation between them in time sequence. The format of the decoded tensor is (batch-size, time-steps, num-cells 3). Among them, num-cells 3 Represents the feature dimension in the decoded tensor.

[0072] Further, through step 205, the multivariate time series corresponding to each business entity is predicted according to the time series feature tensor and the decoding tensor. Under the technical concept of this specification, in the process of predicting the multivariate time series using the time series feature tensor, the decoding tensor determined by step 204 is added, and the multivariate time series is predicted together with the time series feature tensor and the decoding tensor.

[0073] It can be understood that the form of the time series feature tensor is (batch-size, time-steps, feat-dim), corresponding to the three dimensions of business data record, time, and feature value, and the form of the decoding tensor is (batch-size, time-steps, num-cells 3 ) corresponds to the three dimensions of business data record, time, and feature value, while the predicted multivariate time series can correspond to the business volume of each business subject on the predetermined business along time, corresponding to the data on the feature dimension. Compared with the time series feature tensor, a predicted multivariate time series corresponds to the feature data of a single business data at a single time point. For example, such as a row of data in Table 1. Optionally, the predicted multivariate time series may correspond to other business feature data in Table 1 except time information. Therefore, for multiple business subjects, each predicted multivariate time series constitutes a prediction tensor of two dimensions: business subject and time.

[0074] According to one embodiment, the time series feature tensor and the decoding tensor may be spliced ​​together to predict a multivariate time series. For example, the time series feature tensor and the decoding tensor are spliced ​​in the feature or time dimension, and then processed by a fully connected neural network, an activation function, etc. to output a corresponding multivariate time series tensor.

[0075] The predicted multivariate time series tensor can be a one-dimensional tensor, in which case its form is as follows (a x1 , a x2 , a x3 , b x1 , b x2 , b x3 , c x1 , c x2 , c x3 ). The predicted multivariate time series tensor can also be split into two-dimensional tensors, in which case its form is 3×3 [(a x1 , a x2 , a x3 ), (b x1 , b x2 , b x3 ), (cx1 , c x2 , c x3 )] T wait.

[0076] In a specific example, the timing feature tensor and the decoding tensor can be processed by the attention mechanism to obtain the corresponding prediction tensor. For example, the timing feature tensor and the decoding tensor can be spliced ​​in the feature dimension, and the spliced ​​tensor can be used as the input data of the attention mechanism for synchronous processing to obtain a prediction tensor. According to another specific example, one of the data in the timing feature tensor and the decoding tensor can be processed by the attention mechanism to obtain weight data, and then the weight data is used to fuse the tensor of the other item to obtain a prediction tensor. According to another specific example, one of the tensors in the timing feature tensor and the decoding tensor can also be processed by the attention mechanism to obtain the attention tensor spliced ​​into the tensor on the other side, and the spliced ​​tensor is processed to obtain a prediction tensor. In other specific examples, there can also be other fusion methods for the timing feature tensor and the decoding tensor, which will not be repeated here.

[0077] According to the above description, the prediction operation of multivariate time series can be implemented through one or more of attention networks, fully connected neural networks, convolutional neural networks, etc.

[0078] It is worth noting that in the above steps, in the process of data processing, the three dimensions of business data records, time, and feature values ​​may undergo some changes, such as the number of values ​​becoming more or less, the meaning of a single value unit changing, such as the time unit becoming larger or smaller, the meaning of the feature value changing, etc., but overall, changes occur in the three dimensions. Although the above description process is collectively referred to as the three dimensions, this change can be understood based on the above description.

[0079] Among the possible designs, Figure 2 The process shown can be implemented with the assistance of a machine learning model, and the corresponding machine learning model is called a prediction model. The prediction model here is named according to the function of the machine learning model, and the name does not limit the essence of the model. Under the technical concept of this specification, the prediction model may include at least a timing encoding module, a timing cycle processing module, a timing decoding module, and a prediction module, which correspond to the operations of steps 202 to 205, respectively. The operation of step 201 can be implemented by a feature extraction module, or by other computer-assisted processing, which is not limited here.

[0080] According to the above description, the parameters in the prediction model may include various parameters in the time series encoding module, the time series cycle processing module, the time series decoding module and the prediction module. In the prediction model training stage, these parameters are pending parameters and can be adjusted by parameter adjustment methods such as gradient descent method and Newton method. Among them, when training the prediction model, a single training sample can correspond to the time series data consisting of multiple historical multivariate time series collected at multiple time points for multiple business entities as input data, and a multivariate time series subsequently generated by these business entities is used as label data. The prediction tensor obtained by processing the input data by the prediction model is compared with the label data to determine the model loss. Among them, the model loss can be determined by at least one of the mean absolute error MAE, the mean square error MSE, the quantile loss error, etc.

[0081] The above process of predicting multivariate time series can be used in a variety of forecasting and analysis scenarios, for example: sales forecasting between multiple categories on a shopping platform, and sales between different commodities can achieve joint modeling and simultaneous forecasting of sales of multiple commodities through the above process; in the multi-indicator capacity estimation scenario of the application system for technical risks on the cloud computing platform, the changes in multiple capacity indicators of the application system in the future can be predicted in advance, serving the decision-making on expanding or reducing the indicator capacity; for the prediction of the amount of liquidity funds from multiple channels for calculating future customer funds, the multiple funding channels and / or the amount of funds required in the future can be predicted in advance, serving the liquidity management decision-making system; in the consumer finance scenario, for the prediction of interbank fund consumption, the amount of funds that consumer credit users need to consume in various scenarios can be predicted in advance, serving the interbank fund management system of consumer credit; and so on.

[0082] Looking back at the above process, the technical concept provided in this specification is to perform time series coding on the one hand and period coding on the other hand for the historical multivariate time series of each business entity, so as to mine the correlation between the data from both the time series and period aspects. In the time series prediction process, the data of time series coding and period coding are fused and decoded to obtain the corresponding decoding tensor, and the decoding tensor is used together with the time series feature tensor to predict the multivariate time series. This method utilizes designs such as time series coding, time series period processing, and time series decoding to avoid reliance on strong statistical assumptions, and can adaptively mine the correlation between time series and periods, and provide more accurate time series prediction results more flexibly.

[0083] According to another embodiment, a device for predicting a multivariate time series is also provided, which can be used to predict a multivariate time series on a predetermined business for multiple business entities.

[0084] Figure 5 FIG. 4 shows a schematic diagram of a framework of an apparatus for predicting a multivariate time series according to an embodiment. Figure 5 As shown, the device 500 includes:

[0085] The feature extraction unit 501 is configured to extract a time series feature tensor from each historical multivariate time series corresponding to each piece of historical business data sampled from the plurality of business entities, wherein the time series feature tensor corresponds to three dimensions: business data record, time, and feature value;

[0086] The periodic coding unit 502 is configured to perform periodic processing on the time series feature tensor according to the periodicity of the time dimension to obtain a periodic coding tensor in three dimensions;

[0087] A time series coding unit 503 is configured to perform time series coding on each piece of historical business data to obtain a time series coding tensor in three dimensions;

[0088] A decoding unit 504 is configured to fuse and decode the temporal coding tensor and the periodic coding tensor to obtain a decoding tensor corresponding to three dimensions;

[0089] The prediction unit 505 is configured to predict the multivariate time series corresponding to the multiple business entities according to the time series feature tensor and the decoding tensor.

[0090] In one embodiment, the eigenvalue dimension of the time series feature tensor includes multidimensional time embedding features obtained by embedding time information and business features in historical multivariate time series.

[0091] In a further embodiment, each time unit corresponds to multiple levels of time information.

[0092] In one embodiment, the periodic encoding unit 502 is further configured to: split the time series feature tensor along the time dimension according to a predetermined periodic window to reorganize the time series feature tensor into a periodic feature tensor; process each two-dimensional periodic feature tensor corresponding to each periodic unit in the periodic feature tensor in sequence through a first recurrent neural network, and arrange the obtained encoding tensors in order as a periodic encoding tensor.

[0093] In a further embodiment, compared with the timing feature tensor, the periodic feature tensor corresponds to a first number of periodic units in the time dimension, and the number of units in the business data record dimension is increased to a second number, and the first number is the quotient of the number of time units of the timing feature tensor in the time dimension divided by a predetermined period window.

[0094] In one embodiment, a single historical time series is composed of business data of multiple time points / time periods; the time series coding unit 503 is also configured to: sequentially process a two-dimensional business tensor composed of business data of multiple business entities at each time unit through a second recurrent neural network to obtain each two-dimensional time series coding tensor; and obtain a time series coding tensor based on the sequential arrangement of each two-dimensional time series coding tensor along the time dimension.

[0095] In one embodiment, the decoding unit 504 is further configured to:

[0096] The time-series coding tensor and the period coding tensor are fused along the time dimension according to each time unit to obtain a fused tensor corresponding to three dimensions;

[0097] The third recurrent neural network is used to process the two-dimensional tensors corresponding to each time unit in the fusion tensor in sequence according to the time dimension to obtain a decoding tensor.

[0098] In one embodiment, the fusion of the temporal coding tensor and the periodic coding tensor on a single time unit is achieved by splicing.

[0099] In a further embodiment,

[0100] The fusion of the time-series coded tensor and the periodic coded tensor on a single time unit is achieved in the following way:

[0101] The two-dimensional temporal coding tensor on a single time unit is processed based on the self-attention mechanism to obtain a single two-dimensional attention tensor; the single two-dimensional attention tensor is fused with the two-dimensional periodic coding tensor of the periodic coding tensor on a single time unit to obtain a single two-dimensional fused tensor.

[0102] In another further embodiment, a single two-dimensional attention tensor is fused with a single two-dimensional periodic code by one of the following methods: concatenating the single two-dimensional periodic code tensor with a single two-dimensional attention vector; or multiplying the single two-dimensional attention vector with the elements in the single two-dimensional periodic code tensor one by one.

[0103] In yet another further embodiment, the prediction unit 505 is further configured to: concatenate the time series feature tensor and the decoding tensor into a prediction tensor; and process the prediction tensor based on an attention mechanism to obtain a two-dimensional multivariate time series corresponding to two dimensions of time and feature.

[0104] It is worth mentioning that Figure 5 The device 500 shown is Figure 2 The method described corresponds to Figure 2 The corresponding description in the method embodiment is also applicable to the device 500 and will not be repeated here.

[0105] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described by et al.

[0106] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 2 The method described by et al.

[0107] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0108] The specific implementation methods described above further explain in detail the purpose, technical solutions and beneficial effects of the technical concept of this specification. It should be understood that the above description is only the specific implementation method of the technical concept of this specification, and is not used to limit the protection scope of the technical concept of this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of this specification should be included in the protection scope of the technical concept of this specification.

Claims

1. A method for predicting multivariate time series, for predicting time series of multiple business entities, wherein the time series is a sequence consisting of one of the following: click volume, pageview volume, customer flow, computing resource flow required for cloud computing, power consumption in smart grid, and logistics flow; the method comprises: Extracting a time series feature tensor from each historical multivariate time series corresponding to each piece of historical business data sampled from the multiple business entities, wherein the time series feature tensor corresponds to three dimensions: business data record, time, and feature value; Performing time series periodic processing on the time series feature tensor according to the periodicity of the time dimension to obtain a periodic coding tensor in the three dimensions; Performing time series coding on each piece of historical business data to obtain a time series coding tensor in the three dimensions; The temporal coding tensor and the periodic coding tensor are fused and decoded to obtain a decoding tensor corresponding to the three dimensions; According to the time series feature tensor and the decoded tensor, multivariate time series corresponding to multiple business entities are predicted.

2. The method according to claim 1, wherein: The eigenvalue dimension of the time series feature tensor includes multidimensional time embedding features obtained by embedding time information and business features in historical multivariate time series.

3. According to the method of claim 2, the time information corresponds to multiple time levels.

4. The method according to claim 1, wherein: The step of performing time series periodic processing on the time series feature tensor according to the periodicity of the time dimension to obtain the periodic coding tensor in the three dimensions includes: Splitting the time series feature tensor along the time dimension according to a predetermined period window to reorganize the time series feature tensor into a period feature tensor; The two-dimensional periodic feature tensors corresponding to each periodic unit in the periodic feature tensor are processed in sequence by the first recurrent neural network, and the obtained encoding tensors are arranged in order as the periodic encoding tensor.

5. The method according to claim 4, wherein: Compared with the timing feature tensor, the periodic feature tensor corresponds to a first number of periodic units in the time dimension, and the number of units in the business data record dimension is increased to a second number, the first number being the quotient of the number of time units of the timing feature tensor in the time dimension divided by the predetermined period window, and the second number being the product of the number of business entities and the predetermined period window.

6. The method according to claim 1, wherein: A single historical time series is composed of business data of multiple time units; the time series encoding of each piece of historical business data to obtain the time series encoding tensor in the three dimensions includes: Processing the two-dimensional business tensors composed of the business data of the plurality of business entities at each time unit in sequence through a second recurrent neural network to obtain each two-dimensional time series coding tensor; Based on the sequential arrangement of each two-dimensional time-series coding tensor along the time dimension, the time-series coding tensor is obtained.

7. The method according to claim 1, wherein: The fusion and decoding processing of the temporal coding tensor and the periodic coding tensor to obtain the decoded tensor corresponding to the three dimensions includes: The time-series coding tensor and the periodic coding tensor are fused along the time dimension according to each time unit to obtain a fused tensor corresponding to the three dimensions; The two-dimensional tensors corresponding to each time unit in the fused tensor are processed in sequence according to the time dimension by using the third recurrent neural network to obtain the decoded tensor.

8. The method according to claim 7, wherein: The fusion of the temporal coding tensor and the periodic coding tensor in a single time unit is achieved by splicing.

9. The method according to claim 7, wherein: The fusion of the temporal coding tensor and the periodic coding tensor in a single time unit is achieved in the following way: Processing the two-dimensional temporal coding tensor on the single time unit based on a self-attention mechanism to obtain a single two-dimensional attention tensor; The single two-dimensional attention tensor is fused with the two-dimensional periodic coding tensor of the periodic coding tensor on the single time unit to obtain a single two-dimensional fused tensor.

10. The method according to claim 9, wherein: A single 2D attention tensor is fused with a single 2D periodic encoding in one of the following ways: Concatenate a single 2D period encoding tensor with a single 2D attention vector; Multiply a single 2D attention vector element-by-element with a single 2D period encoding tensor.

11. The method according to claim 1, wherein: The predicting, according to the time series feature tensor and the decoding tensor, the multivariate time series corresponding to the plurality of business entities comprises: splicing the time series feature tensor and the decoding tensor into a prediction tensor; The prediction tensor is processed based on the attention mechanism to obtain a two-dimensional multivariate time series corresponding to two dimensions of time and eigenvalue as a multivariate time series predicted for the multiple business entities.

12. A device for predicting multivariate time series, used for simultaneously predicting the time series of multiple business entities, wherein the time series is a sequence consisting of one of the following: click volume, pageview volume, customer flow, computing resource flow required for cloud computing, power consumption in smart grids, and logistics flow; the device comprises: A feature extraction unit is configured to extract a time series feature tensor from each historical multivariate time series corresponding to each piece of historical business data sampled from the plurality of business entities, wherein the time series feature tensor corresponds to three dimensions: business data record, time, and feature value; A periodic coding unit, configured to perform time series periodic processing on the time series feature tensor according to the periodicity of the time dimension to obtain a periodic coding tensor in the three dimensions; A time series encoding unit, configured to perform time series encoding on each piece of historical business data to obtain a time series encoding tensor in the three dimensions; A decoding unit, configured to fuse and decode the temporal coding tensor and the periodic coding tensor to obtain a decoding tensor corresponding to the three dimensions; The prediction unit is configured to predict the multivariate time series corresponding to the plurality of business entities according to the time series feature tensor and the decoding tensor.

13. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 11.

14. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 11 is implemented.

Images