Time series prediction method and device for layered services

Abstract

The embodiments of this specification provide a time series prediction method and device for layered services, which are used to predict the time series composed of the business volume of business entities of a single business at multiple levels, wherein a single level corresponds to at least one business entity. According to one implementation, after obtaining the historical time series corresponding to each business entity in multiple levels, each historical time series can be encoded to obtain the corresponding encoding vectors, and then a multivariate Gaussian distribution satisfied by each encoding vector is constructed, and the multivariate Gaussian distribution is converted into a non-parametric complex distribution, and further the sampling sequence obtained by sampling the parameterized complex distribution is determined to determine each prediction sequence for each business entity. This method can improve the accuracy of the prediction results.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of computer technology, and in particular, to a method and apparatus for predicting time series for layered services. Background Art

[0002] A time series (or dynamic series) is a series of values ​​of the same statistical indicator arranged in the order of their occurrence. The main purpose of time series analysis is to predict the future based on existing historical data. Time series prediction can be applied to various scenarios, such as the time series prediction of customer flow in commodity supermarkets, the time series prediction of funds in financial services, the prediction of computing resource flow required in cloud computing, logistics demand, and power consumption in smart grids, etc. The prediction results can serve business decisions, for example. With the development of artificial intelligence, machine learning models can be used for time series analysis.

[0003] Hierarchical time series is a special kind of time series that can describe related businesses from multiple levels, making the time series have a hierarchical structure. For example, the time series of sales of large supermarkets can have a hierarchical structure according to the hierarchical structure of their geographical locations, such as being divided into the following multiple levels: national total sales time series; sales time series of each province and city; sales time series of each store; and so on. In fact, there may also be correlations between different time series. How to mine such correlations and make full use of the time series information of the hierarchical structure is an important issue in time series prediction. Summary of the invention

[0004] One or more embodiments of the present specification describe a method and device for time series prediction for layered services, to solve one or more problems mentioned in the background technology.

[0005] According to a first aspect, a time series prediction method for layered business is provided, which is used to predict the time series composed of the business volume of business entities on multiple levels on predetermined business indicators, wherein a single level corresponds to at least one business entity; the method comprises: obtaining each historical time series corresponding to each business entity in the multiple levels, and a single historical time series corresponds to each business volume corresponding to a single business entity in a plurality of predetermined time periods arranged in sequence; encoding each historical time series to obtain corresponding encoding vectors; constructing a joint probability distribution satisfied by each encoding vector; processing the joint probability distribution through a standardized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector; based on each sampling sequence obtained by sampling each business entity according to the non-parametric complex distribution, determining each prediction sequence for each business entity, wherein a single prediction sequence describes each business volume of a single business entity in a plurality of subsequent predetermined time periods, and each prediction sequence satisfies the following consistency constraint: for a single time period, the business volume of a single business entity at a higher level is consistent with the sum of the business volume of at least one business entity corresponding to it at a lower level.

[0006] In one possible design, the joint probability distribution is a multivariate Gaussian distribution, and the processing of the multivariate Gaussian distribution via a standardized flow model includes: starting from the multivariate Gaussian distribution, converting the multivariate Gaussian distribution into a complex distribution by applying a series of reversible transformation functions, wherein the reversible transformation function repeatedly replaces the current variable with a new variable according to the substitution theorem, and finally obtains the probability distribution of the target variable as a non-parametric complex distribution.

[0007] In one embodiment, the encoding of each historical time series is performed by a single recurrent neural network, or an unfolded recurrent neural network consisting of multiple recurrent neural networks corresponding to each business entity one by one.

[0008] In one embodiment, the multivariate Gaussian distribution corresponds to a first mean matrix and a first covariance matrix, and a single sampling sequence is determined in the following manner: sampling a first vector in a standardized multivariate Gaussian distribution, wherein the mean matrix of the standardized multivariate Gaussian distribution is a 0 matrix, and the covariance matrix is ​​a unit matrix; superimposing the product of the quadratic root of the first covariance matrix and the first vector with the first mean matrix to obtain a superposition result; and determining the single sampling sequence by inversely transforming the superposition result based on the inverse function of the reversible transformation function.

[0009] In one embodiment, the sampling results obtained by sampling each business subject according to the non-parametric complex distribution, determining each prediction sequence corresponding to each business subject includes: sampling each business subject based on the non-parametric complex distribution, and arranging the corresponding sampling sequences in sequence to form a sampling matrix as a preliminary prediction result; adjusting the preliminary prediction results to obtain each prediction sequence, so that each prediction sequence satisfies a consistency constraint.

[0010] In a further embodiment, adjusting the preliminary prediction results to obtain various prediction sequences includes: encoding the preliminary prediction results to obtain various prediction sequences corresponding to various business entities at the lowest level; and determining various prediction sequences corresponding to various business entities at other levels through consistency constraints.

[0011] In a further embodiment, the operation of encoding the preliminary prediction results is implemented through a multi-head attention mechanism, including: splitting the preliminary prediction results into h parts of data, where h is the number of attention heads under the multi-head attention mechanism; performing independent linear transformations under different parameters on the h parts of data to obtain h processing results; based on the splicing of the h processing results, obtaining each prediction sequence corresponding to each business entity at the lowest level.

[0012] In another further embodiment, the consistency constraint is represented by a business architecture matrix, and the prediction sequences corresponding to the business entities in other hierarchical levels are determined based on the product of the business architecture matrix and the coding matrix, wherein the coding matrix is ​​a matrix composed of the prediction sequences corresponding to the business entities in the lowest hierarchical level.

[0013] In a further embodiment, the business architecture matrix is ​​formed by concatenating a unit matrix describing each business subject at the lowest level and a sum matrix corresponding to each business subject at other levels, wherein each business subject at the lowest level corresponds one-to-one to the rows / columns of the corresponding unit matrix, and a single business subject at other levels corresponds to a single row / column in the sum matrix, and a single row / column in the sum matrix is ​​formed by superimposing each row / column of the corresponding unit matrix on each business subject corresponding to the single business subject at the lowest level.

[0014] According to a second aspect, there is also provided a time series prediction device for layered services, which is used to predict the time series of business volumes of predetermined services of business entities at multiple levels, wherein a single level corresponds to at least one business entity; the device comprises:

[0015] An acquisition unit is configured to acquire each historical time series corresponding to each business subject in the multi-level hierarchy, wherein each historical time series corresponds to each business volume corresponding to each business subject in a plurality of predetermined time periods arranged in sequence;

[0016] An encoding unit configured to encode each historical time series to obtain corresponding encoding vectors;

[0017] A mapping unit configured to construct a joint probability distribution satisfied by each encoding vector;

[0018] A conversion unit configured to process the joint probability distribution via a standardized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector;

[0019] The prediction unit is configured to determine each prediction sequence for each business subject based on each sampling sequence obtained by sampling each business subject according to the non-parametric complex distribution, wherein a single prediction sequence describes each business volume of a single business subject in a plurality of subsequent predetermined time periods, and each prediction sequence satisfies a consistency constraint, wherein the consistency constraint includes: for a single time period, the business volume of a single business subject at a higher level is consistent with the sum of the business volumes of at least one business subject corresponding to it at a lower level.

[0020] According to a third aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed in a computer, the computer is caused to execute the method of the first aspect.

[0021] According to a fourth aspect, a computing device is provided, comprising a memory and a processor, wherein an executable code is stored in the memory, and when the processor executes the executable code, the method of the first aspect is implemented.

[0022] Through the method and device provided in the embodiments of this specification, in the time series prediction process for layered services, the time series of each level are mapped to a high-order vector and a joint probability distribution is constructed. Then, the non-parametric complex distribution between each coding vector is mined based on the joint probability distribution through the annotated flow model, and the complex distribution relationship between different time series in the layered time series is characterized, so that the model in the technical solution can learn the non-Gaussian linear characteristics in the layered time series, thereby considering the complex correlation of the time series at each level. When resampling according to this complex distribution to obtain the corresponding predicted time series, the correlation between business data at different levels can be fully utilized to achieve information fusion, thereby improving the accuracy of the prediction results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0024] Figure 1 A specific architectural diagram showing hierarchical timing issues;

[0025] Figure 2 A flow chart of an improved time series prediction method for layered services is shown;

[0026] Figure 3 A schematic diagram showing encoding of a historical time series by an encoding module according to a specific example;

[0027] Figure 4 A schematic diagram of a standardized stream processing architecture according to a specific example is shown;

[0028] Figure 5 A schematic diagram of a standardized stream processing architecture according to the RealNVP architecture is shown;

[0029] Figure 6 A schematic diagram of a processing architecture for determining a prediction result using a sampling result according to a specific example is shown;

[0030] Figure 7 A schematic block diagram of a time series prediction device for layered services according to an embodiment is shown. DETAILED DESCRIPTION

[0031] The technical solution provided in this specification is described below in conjunction with the accompanying drawings.

[0032] Figure 1 A specific architecture diagram showing the hierarchical timing problem. Figure 1 As shown in the figure, in this specific application scenario, three business levels are involved. The lowest business level is the level corresponding to specific business entities (such as business entities D, E, F, G, etc.), the higher level is the level corresponding to business entities with certain comprehensive significance (such as business entities B, C, etc.), and the higher level can be the level corresponding to more comprehensive business entities (such as business entity A, etc.). The business entity here can be an entity that can independently distinguish and count the corresponding business, such as a store, a category of goods, goods or stores in a region, and so on. Figure 1 A hierarchical business problem with a three-level structure is shown in FIG. 1 . In practice, the hierarchical business structure may have more (eg, 8 levels) or fewer levels (eg, 2 levels), which is not limited here.

[0033] A single business subject can correspond to the business volume of the predetermined business indicator in a single time period. The business volume derived from the predetermined business indicator in multiple time periods constitutes a time series. Figure 1 For example, business entity A (such as a large supermarket with stores across the country) corresponds to time series y. A，t (such as describing sales business), business entity B (such as a large supermarket store in North China) corresponds to the time series y B，t , business entity D (such as a large supermarket store in South China) corresponds to the time series y D，t , etc. Among them, each time period of the sampling time series can be continuous or discontinuous, and this specification does not limit this. For example, in one embodiment, a single time period is a week, and data from multiple consecutive weeks constitute a time series. In another embodiment, a single time period is a Monday, and data from multiple Mondays constitute a time series, but the time between two adjacent Mondays is discontinuous.

[0034] Among them, according to different specific business scenarios, the business subjects at each level are also different, and the corresponding business indicators are also different. For example, in the shopping platform scenario, the lowest level can correspond to each merchant, and the corresponding business indicator is, for example, sales volume, and the corresponding time period can be one week, one day, one month, etc. The higher level can correspond to a small category of business subjects, such as merchants in categories such as kitchen appliances, newborn clothes, and home decoration materials, and the higher level can correspond to a larger category of business subjects, such as merchants in categories such as electrical appliances, clothing, and building materials. Similarly, in the investment and financial management scenario, the highest level can correspond to various investment channels, such as insurance, funds, fixed deposits, stocks, etc., and the higher level can correspond to various investment objects, such as insurance companies, fund units, banks, stock subject categories, etc. The lower level corresponds to various specific investment methods, such as insurance types, fund types, bank fixed deposit product types, stock subjects, etc. The business volume described by the time series is business indicators such as investment amount, investment ratio, and number of investment users. In other business scenarios, the business subjects of each level can also be in other forms, which will not be repeated here.

[0035] According to the meaning of layered business, the time series corresponding to business entities at adjacent levels have the following consistency relationship: within a single time period, the business volume corresponding to a single business entity at a higher level is the sum of the business volumes of its corresponding business entities at lower levels. Furthermore, when the time periods corresponding to the time series are consistent, the time series corresponding to a single business entity at a higher level can be the sum of the time series corresponding to its corresponding business entities at lower levels. Figure 1 As shown, there are: y A，t =y B，t +y C，t ;y B，t=y D，t +y E，t ;y C，t =y F，t +y G，t .

[0036] In such Figure 1 In the hierarchical time series forecasting process shown, the future time series can be predicted based on the historical time series of each business entity. For example, based on the historical time series corresponding to the sales volume of each business entity in the past three months, the time series corresponding to the sales volume of each business entity in the next 30 days can be predicted.

[0037] To this end, the time series of each business entity can be predicted separately, and then adjusted to meet the consistency requirements. For example, the time series corresponding to the sales of business entities A, B, C, D, E, F, and G in the past three months can be independently predicted, and then the prediction results can be uniformly adjusted so that the final prediction results meet the consistency constraints. For example, the time series predicted for A is the sum of the time series predicted for B and C, the time series predicted for B is the sum of the time series predicted for D and E, and the time series predicted for C is the sum of the time series predicted for F and G.

[0038] In fact, the time series of various business entities may also be correlated. For example, when the sales of maternal and infant products continue to rise, the sales of a single maternal and infant store with low historical sales will also continue to rise in the future. For another example, when the proportion of insurance, fund, and stock investment continues to rise, the proportion of fixed deposit investment decreases. Therefore, this specification provides a time series prediction solution for layered businesses, which establishes a non-parametric complex distribution between the historical time series of various business entities, explores the correlation between the time series of various business entities, and thus more accurately predicts the future time series of various business entities.

[0039] Reference below Figure 2 A specific example is shown to describe the technical concept of this specification.

[0040] Figure 2 The following is a time series prediction process for layered services according to an embodiment. The execution subject of the process may be a computer, device or server with certain computing capabilities. The process may be implemented with the assistance of a machine learning model. The corresponding machine learning model may be called a prediction model according to its function. It can be understood that Figure 2 In the illustrated embodiment, the hierarchy of the layered service may include at least two levels, for example, a first level (such as Figure 1 The level where B and C are located) and the second level (such as Figure 1In practice, hierarchical trees are usually more complex.

[0041] like Figure 2 As shown, the time series prediction process for layered services may include the following steps: Step 201, obtaining each historical time series corresponding to each business subject in multiple hierarchical levels, wherein a single historical time series corresponds to each business volume corresponding to a single business subject in a plurality of predetermined time periods arranged in sequence; Step 202, encoding each historical time series to obtain corresponding encoding vectors; Step 203, determining the joint probability distribution satisfied by each encoding vector; Step 204, processing the joint probability distribution via a standardized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector; Step 205, based on each sampling sequence obtained by sampling each business subject according to the non-parametric complex distribution, determining each prediction sequence for each business subject, wherein a single prediction sequence describes each business volume of a single business subject in a plurality of subsequent predetermined time periods, and each prediction sequence satisfies a consistency constraint, and the consistency constraint includes: for a single time period, the business volume of a single business subject at a higher hierarchical level is consistent with the sum of the business volume of at least one business subject corresponding to it at a lower hierarchical level.

[0042] First, in step 201, each historical time series corresponding to each business subject in the multi-level hierarchy of the layered business is obtained. It can be understood that the layered business can correspond to multiple levels, such as Figure 1 There are three levels in each level. There is at least one business subject in each level, and each business subject in a lower level corresponds to a higher-level business subject in a higher level, for example, a more general business subject in a higher level. On the other hand, a single higher-level business subject corresponds to at least one lower-level business subject in a lower level.

[0043] The business volumes corresponding to the predetermined business indicators of a single business subject in multiple predetermined time periods arranged in sequence can constitute a single historical time series. A single predetermined time period can be set according to business needs, such as one day, one week, one month, etc. Multiple time periods can be continuous or discontinuous in time. Taking a single predetermined time period of 1 day as an example, assuming that multiple time periods are continuous, such as sampling sales volume for multiple consecutive days, then for a single merchant as a business subject, the sales volume value sampled for multiple consecutive days (such as 60 days) can be used as its historical time series. Assuming that multiple time periods are continuous, such as only sampling sales volume on weekends, then for a single merchant as a business subject, the sales volume value sampled for multiple weekends (such as 52 weekends) can be used as its historical time series. The historical time series can be represented by a multi-dimensional vector, array, or by a set, etc., which is not limited here. Each historical time series can be represented by a matrix, etc., or by each vector.

[0044] In an optional embodiment, a single business volume data in a historical time series can also be represented by data of multiple dimensions, and a single dimension of a single historical time series can be expanded into multi-dimensional data. As an example, if a single business volume data corresponds to 2-dimensional data (such as data of two dimensions of passenger flow and sales volume), the business volume of 60 time periods can correspond to a 120-dimensional vector or array, or to a 2×60-dimensional matrix, and so on.

[0045] Next, in step 202, each historical time series is encoded to obtain corresponding encoding vectors. It can be understood that for a single historical time series, the specific business volume of a single business entity in a certain period can be described. In order to represent it abstractly, the historical time series can be encoded to mine the deep correlation between each business volume in the time series. In an optional implementation, step 202 can be implemented by a reasonable model, such as a recurrent neural network (RNN), a neural network based on an attention mechanism, BERT, etc.

[0046] Since historical time series usually have forward and backward correlation and periodicity, they can be processed using a recurrent neural network (RNN). In one embodiment, each historical time series can be processed separately by a recurrent neural network. In this case, the historical time series of a single business entity can be regarded as input features corresponding to multiple time points. Through the processing of the recurrent unit of the recurrent neural network, the processing result can be obtained as the encoding result for the corresponding historical time series. In another embodiment, the matrix composed of multiple historical time series can also be processed by an unrolled recurrent neural network (Unrolling RNN). Figure 3As shown, for example, each historical time series can be described in matrix form, and the recurrent unit of the recurrent neural network is unfolded and flattened into multiple copies (i.e., Unrolling RNN). In this way, a single recurrent unit corresponds to a single business entity, and the output result of a single recurrent unit corresponds to the encoding result of a historical time series. For example, N+1 recurrent units output the encoding results of N+1 business entities.

[0047] In an optional implementation, when the data of a single business subject in a single time period is multi-dimensional, the single input data processed by a single loop unit can be the corresponding multi-dimensional data. The encoding result for an encoding object can be in vector form, for example, called an encoding vector. The encoding vector can have a higher dimension than the time series to better describe the time series.

[0048] Then, according to step 203, determine the joint probability distribution satisfied by each coding vector. Among them, the joint probability distribution is referred to as the joint distribution, which is the probability distribution of random variables composed of two or more random variables. The joint probability distribution is, for example, a multivariate Gaussian distribution, a multivariate exponential distribution, a multivariate Poisson distribution, a multivariate student-T distribution, and the like. The joint probability distribution in this specification can be a distribution applicable to data statistical scenarios such as a multivariate Gaussian distribution and a multivariate student-T distribution. This article takes the multivariate Gaussian distribution as an example for description. The multivariate Gaussian distribution can be regarded as a joint Gaussian distribution of data of multiple dimensions. For example, N+1 coding vectors can be regarded as data of N+1 dimensions, and more coding vectors can be regarded as data of more dimensions. In step 203, the corresponding joint probability distribution can be constructed by mining the multidimensional relationship between each coding vector.

[0049] As an example, assume that the multivariate Gaussian distribution satisfied by each encoding vector is expressed as follows:

[0050]

[0051] Where z is the standardized parameter that satisfies the Gaussian distribution (0, 1), the multivariate Gaussian distribution represented by z is the standard multivariate Gaussian distribution, x represents the variable matrix composed of various coding vectors, μ x is the mean matrix of the multivariate Gaussian distribution, Σ represents the covariance matrix corresponding to the variable matrix x, and the i-th row and j-th column are the variables x i and x j The covariance of (e.g., the covariance of the i-th coding vector and the j-th coding vector), n is the total number of service entities in each level of the layered service. Figure 1 In the architecture of the layered service shown, n = 7. When the variables are independent of each other, Σ is a diagonal matrix, z, μ x ,Σ satisfy:

[0052]

[0053] as well as:

[0054]

[0055] In a standardized multivariate Gaussian distribution, σ i 2 =1, Σ is the n×n dimensional unit matrix, each encoding vector corresponds to the standard Gaussian distribution N(0, I), I represents the unit matrix. x , Σ are the distribution parameters of the joint Gaussian distribution constructed for each encoding vector.

[0056] The process of determining the multivariate Gaussian distribution satisfied by each encoding vector can also be viewed as the mean matrix μ of the multivariate Gaussian distribution x , the process of determining the covariance matrix Σ. By extension, the process of determining other joint probability distributions for each coding vector can also be regarded as the process of determining other parameters of other joint probability distributions besides the independent variables.

[0057] Furthermore, in step 204, the joint probability distribution is processed via a normalized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector.

[0058] Among them, Normalized Flow (also called normalized flow) is a generative model. In probability and statistics theory, a generative model refers to a model that can randomly generate observational data, especially given certain implicit parameters. It assigns a joint probability distribution to the observed value and the labeled data sequence. In machine learning, generative models can be used to directly model data (for example, data sampling according to the probability density function of a variable), or to establish conditional probability distributions between variables. The conditional probability distribution can be formed by the generative model according to Bayes' theorem. Conventional generative models such as GAN and VAE require explicit learning of the probability density function p(x) (where x∈D) of the actual data x. In practice, this probability density function is usually not clear. For example, in models with hidden variables, it is difficult to calculate p(x)=∫p(x|z)p(z)dz because it is difficult to traverse all possible values ​​of the hidden code z. Normalized flow is a powerful statistical tool for density estimation, which can overcome this problem through a series of reversible transformations with the help of its deep generative model.

[0059] Normalized flows usually start from simple distributions (such as Gaussian distribution) and transform simple distributions into complex distributions by applying a series of reversible transformation functions. Through a series of transformations, the current variables are repeatedly replaced with new variables according to the substitution theorem, and finally the probability distribution of the target variable is obtained. The idea of ​​substitution is as follows: given a random variable z and its known probability density function z~π(z), a new random variable is constructed using a 1:1 mapping function x=f(z). Function f is reversible, so z=f -1 (x). This process is called variable substitution.

[0060] For multiple variables, we have:

[0061] z~π(z),x=f(z),z=f -1 (x)

[0062]

[0063] Here, z and x are matrices composed of multiple scalars, and det represents the determinant of the square matrix.

[0064] Figure 4 A schematic diagram of standardized flow transformation is shown. z0 follows a Gaussian distribution, which is composed of random variables z i =f i (z i -1) The traversed path can be called a flow and is represented by a continuous distribution π i The whole chain formed is called normalized flow or standardized flow. According to the calculation requirements, the transformation function f i It should satisfy two properties: it is easy to invert and its Jacobian is easy to compute.

[0065] The Jacobian matrix is ​​a gradient matrix that describes the linear change of a mapping before and after the mapping, which represents the best linear approximation of a multivariate vector function. The computability of the Jacobian determinant means that the distribution is continuously differentiable. Models based on standardized flow are such as RealNVP (real-valued non-volume preserving). Taking the RealNVP architecture as an example, Figure 5 Figure 2 shows a flow model diagram of a RealNVP architecture. Figure 5 As shown in the figure, the RealNVP architecture can be regarded as a multi-scale architecture. The multi-scale architecture introduces convolutional layers. After each convolutional layer is processed, half of the channels are directly applied to the output layer, while the other half of the channels are processed by the flow operation. Each time, half of the channels undergo a more complex transformation. Figure 5 In the example, the input x passes through the convolution coupling layer to get z1, and then z1 is taken to the final output z 1：d , and the other half d+1：dRepeat the previous step, pass the convolution coupling layer, take half of it as the final output, and repeat the above steps for the other half, stacking n layers. RealNVP's multi-scale architecture can reduce the complexity of the model and introduce regularization effects.

[0066] By processing the joint probability distribution through the standardized flow model, a simpler distribution can be converted into a non-parametric complex distribution, thereby adapting to the more complex distribution forms between the historical time series of various business entities.

[0067] Next, through step 205, based on the sampling sequences obtained by sampling each business subject according to the non-parametric complex distribution, each prediction sequence for each business subject is determined. In practice, sampling can be performed for each business subject that needs to predict the time series to obtain each prediction sequence. Among them, the business subjects that need to predict the time series can be all business subjects in the layered business, or can be part of the business subjects, which is not limited here. Among them, the dimension of the sampling variable z can be consistent with or inconsistent with the dimension of the historical time series.

[0068] It can be understood that due to some statistical characteristics of the hierarchical structure, in the historical time series, the time series of each business subject at each level is based on the description of the business scenario in the previous article, and the business volume (or time series) of each business subject is mutually constrained. That is, the business volume (or time series) of a business subject in a high-level level is the sum of the business volume (or time series) of each business subject corresponding to it in a certain lower level. For example Figure 1 In the example shown, the time series corresponding to business subject B is the sum of the time series corresponding to business subjects D and E. This can also be called a consistency constraint. Figure 2 In the illustrated embodiment, the consistency constraint includes: the sum of the prediction sequence corresponding to the first business subject and the prediction sequences of the corresponding N business subjects.

[0069] Since the business volumes of business entities at other levels are determined by the business volumes of business entities at the lowest level, in a possible design, the time series of business entities at the lowest level can be sampled first during the sampling process, and then the corresponding sampling sequences are summed up as the sampling sequence of the higher level. Figure 1 In the above example, the prediction sequence corresponding to business entities D, E, F, and G is directly sampled from the joint probability distribution, and then the prediction sequence y corresponding to business entity B is determined based on the prediction sequence corresponding to business entities D and E. B，t =:y D，t +y E，t , determine the prediction sequence y corresponding to business subject C based on the prediction sequence corresponding to business subjects F and G C，t =y F，t +y G，t, and then determine the prediction sequence y corresponding to business subject A based on the prediction sequences corresponding to business subjects B and C A，t =y B，t +y C，t In this way, it can be ensured that each prediction sequence satisfies the consistency constraint.

[0070] According to another possible design, in order to ensure the randomness of sampling, the corresponding time series can be sampled from the above non-parametric complex distribution as the sampling sequence for each business entity. In this way, the sampling results (such as the sampling matrix composed of each sampling sequence) can be used as the preliminary prediction results, and the consistency constraints between the various business entities can be achieved by adjusting the preliminary prediction results through conventional technical means, which will not be repeated here. For example, according to the consistency constraint, a time series matrix y is adjusted as small as possible based on the preliminary prediction results, so that the time series matrix y becomes the prediction matrix Y composed of the final predicted time series. For example, solve so that The smallest modulus y, where The sampling matrix composed of the sampling sequence, for example, the timing matrix y and the sampling matrix The modulus of the difference is minimized, The modulus here can be represented by a first-order norm, a second-order norm, etc., and is not limited here.

[0071] According to another possible design, in order to further utilize the correlation in the sampling results, an encoding-decoding architecture that introduces an attention mechanism can be used. Specifically, each sampling sequence obtained by sampling is processed by an encoder of the attention mechanism to obtain a prediction sequence for each business entity in the lowest level. The prediction sequence incorporates the correlation of the sampling sequences of each business entity in the sampling results. After that, it passes through a decoder that meets the consistency constraint to determine the prediction sequence of each business entity in other levels other than the lowest level, and obtain the prediction sequence of the business entity in each level.

[0072] The essence of the Attention mechanism can be described as a mapping from a query to a series of (key-value) pairs. Specifically, the query and each key can be similarly calculated to obtain weights. Common similarity functions include dot product, concatenation, perceptron, etc., and then an activation function (such as softmax) is used to normalize these weights; finally, the weights and the corresponding key values ​​are weighted and summed to obtain the final attention. There are many variants of the Attention mechanism, such as recurrent attention, self-attention, global attention, multi-head attention, etc.

[0073] Figure 6The attention architecture shown in is multi-head attention. Under the multi-head attention mechanism, Query, Key, and Value first undergo a linear transformation and then are input into the scaled dot product attention. This is done h times (split the data into h parts, where the data is, for example, the sampling results), which is the so-called multi-head, and each time counts as one head. Moreover, the parameters of the linear transformation of Q, K, and V are different each time. Then the h-times scaled dot product attention results are concatenated, and the value obtained by performing a linear transformation again is used as the result of the multi-head attention. It can be seen that the difference of multi-head attention is that it performs h calculations instead of just one. The multi-head attention mechanism can extract relevant implicit features from multiple angles, thereby more accurately describing the importance of other-order sampling results in the lowest level.

[0074] Through the attention mechanism encoding network, the prediction sequence of each business subject in the lowest level can be obtained. Then, the prediction sequence of each business subject in each level can be further obtained through the decoder that satisfies the consistency constraint. The decoder can be the aforementioned consistency constraint relation, such as Figure 1 The consistency constraint y in B，t =y D，t +y E，t ,y C，t =y F，t +y G，t ,y A，t =g B，t +y C，t etc. Figure 6 In the decoder, the decoder may also include a decoding logic determined based on the service architecture matrix S of the corresponding hierarchical service. Specifically, the prediction sequence of each service subject in the lowest level may be regarded as a column vector y', such as Figure 1 y' in =(y D ,y E ,y F ,y G ) T , a single element in the column vector substantially represents a corresponding prediction sequence. Then the decoder can be y = S × y'.

[0075] The business architecture matrix S can be determined in the following way: since the business subjects at the lowest level are independent of each other, they can be described by the unit matrix, and a row / column of the unit matrix is ​​used as the expression vector of a lowest-level business subject; the expression vectors of each business subject corresponding to the other-order business subjects at the lowest level are added to obtain the sum matrix; the concatenation matrix of the sum matrix and the unit matrix is ​​used as the business architecture matrix S. The concatenation is performed according to the direction of the expression vector used to express the business subject in the unit matrix.

[0076] by Figure 1 Taking the implementation architecture shown as an example, the lowest-level business entities D, E, F, and G can be described by a 4×4 dimensional diagonal matrix, namely:

[0077]

[0078] Assuming that each row of the above unit matrix represents a business entity, the business entities at higher levels can be represented by the vectors corresponding to the business entities at the lowest level. Figure 1 The event sequence of business entities A, B, and C in the higher level can be represented by business entities D, E, F, and G in the lowest level. C，t =y F，t +y G，t Get the vector (0, 0, 1, 1) corresponding to C, according to y B，t =y D，t +y E，t Get the vector (1, 1, 0, 0) corresponding to B, and according to y A，t =y B，t +y C，t The vector corresponding to A is (1, 1, 1, 1). In this way, the matrix composed of the vectors corresponding to A, B, and C can be called a sum matrix, for example, a sum matrix, such as:

[0079]

[0080] Since each row of the unit matrix represents a business entity, the business architecture matrix S can be obtained by splicing rows and increasing the number of rows:

[0081]

[0082] The business architecture matrix Figure 6 It is consistent with the example shown. It can be seen that the product of S and y' can represent the sum of the corresponding prediction sequences of the lowest level. For example, the product of the first row of S and y' indicates that business subject A is the sum of the prediction sequences of all the lowest-level business subjects, that is, it satisfies the consistency constraint. In other embodiments, there may be other decoding logic or matrix processing logic, which will not be repeated here.

[0083] In this way, the randomness of sampling is guaranteed, and the consistency of the final prediction results is also guaranteed.

[0084] Among them, the above prediction process can be implemented by a prediction model, and each step is implemented as each module of the prediction model. For example, step 202 is implemented by a coding module, step 203 is implemented by a mapping module, step 204 is implemented by a distribution transfer module, and step 205 is implemented by a sampling module. At this time, in the prediction model training process, the parameters of the coding module, the parameters of the mapping module or the joint probability distribution parameters (optionally, sampling parameters can also be included), the parameters of the distribution transfer module, etc. (for example, attention modules, etc.) can all be used as pending parameters, and the historical time series of each business subject in the hierarchical business structure are used as sample data, and the subsequent time series of the historical time series are used as sample labels to determine the model loss and adjust each pending parameter. The model loss can be measured in various reasonable ways such as variance and cross entropy, which are not limited here. The adjustment of model parameters can adopt various suitable methods. Among them, in a continuously differentiable prediction model, gradient descent method, Newton method and other gradient-related methods can be used to adjust each pending parameter in the direction of reducing model loss. Using gradient is also a better parameter adjustment method.

[0085] It is worth noting that, when the prediction model is trained using the gradient correlation method, in order to ensure that the prediction process is differentiable and continuous, the non-parametric complex distribution obtained in step 204 can be sampled using the reparameterization technique. According to the reparameterization technique, from the distribution N(μ, σ 2 ), which is equivalent to sampling a variable ε from the standard distribution N(0,1), and then setting the sampling result Y = μ + ε × σ. Specifically, in step 205, the variable z can be first sampled from the standard joint probability distribution, and then converted to the sampling result from the non-parametric complex distribution by using the reversibility of the standardized flow model. For example, sampling z~N(0,I) in the case of multivariate Gaussian distribution, the conversion to the sampling result from the non-parametric complex distribution is: The process of sampling from the standard joint probability distribution is not described here. It can be described in matrix form: each row / column corresponds to a business entity (corresponding to a vector ), each column / row corresponds to a future time period.

[0086] The continuity of the sampling process is reflected in the fact that it is differentiable (differentiable, that is, the gradient can be calculated). In other words, if the Jacobian of the output of the normalized flow model can be calculated, it can be ensured that the sampling process is for a continuous distribution, and the model parameters can be adjusted using gradient-related methods. Taking the RealNVP architecture as an example, the output Jacobian is, for example:

[0087]

[0088] Transformed to Gaussian distribution, the Jacobian matrix of affine coupling is still a triangular matrix, but the diagonal is not all 1, and it can be expressed as a block matrix, for example:

[0089]

[0090] O is the zero matrix, s is the output of the convolutional network, j is used to represent the diagonal elements of the diagonal matrix s, and the corresponding Jacobian determinant is Σs j , can be calculated. This ensures that the model parameters are adjusted using gradient-related optimization methods during model training.

[0091] Looking back at the above process, the technical concept provided in this specification constructs a joint probability distribution (such as a multivariate Gaussian distribution) by utilizing the historical time series of each business entity, integrates the time series information of business entities at all levels, fully fuses information, and mines the mutual influence of data between business entities. The joint probability distribution is converted into a non-parametric complex distribution, which can adapt to various complex distribution forms, thereby improving the accuracy of time series prediction.

[0092] According to another embodiment, a time series prediction device for layered services is also provided. The device is used to predict the time series consisting of the business volume of business entities of multiple layers on a predetermined business through a prediction model, wherein the prediction model includes a coding module, a mapping module, a distribution transfer module and a sampling module, and a single layer corresponds to at least one business entity.

[0093] Figure 7 FIG. 7 shows a time series prediction device 700 for layered services according to an embodiment. Figure 7 As shown, the device 700 includes: an acquisition unit 701, configured to acquire each historical time series corresponding to each business subject in multiple hierarchical levels, and a single historical time series corresponds to each business volume corresponding to a single business subject in multiple predetermined time periods arranged in sequence; an encoding unit 702, configured to encode each historical time series to obtain corresponding encoding vectors; a mapping unit 703, configured to determine the joint probability distribution satisfied by each encoding vector; a conversion unit 704, configured to process the joint probability distribution via a standardized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector; a prediction unit 705, configured to determine each prediction sequence for each business subject based on each sampling sequence obtained by sampling each business subject according to the non-parametric complex distribution, wherein a single prediction sequence describes each business volume of a single business subject in multiple subsequent predetermined time periods, and each prediction sequence satisfies a consistency constraint, and the consistency constraint includes: for a single time period, the business volume of a single business subject at a higher hierarchical level is consistent with the sum of the business volume of at least one business subject corresponding to it at a lower hierarchical level.

[0094] According to a possible design, the joint probability distribution is a multivariate Gaussian distribution, and the conversion unit 704 is further configured to: starting from the multivariate Gaussian distribution, the multivariate Gaussian distribution is converted into a complex distribution by applying a series of reversible transformation functions, wherein the reversible transformation function repeatedly replaces the current variable with the new variable according to the substitution theorem, and finally obtains the probability distribution of the target variable as a non-parametric complex distribution.

[0095] In a further optional implementation, the multivariate Gaussian distribution corresponds to a first mean matrix and a first covariance matrix, and the apparatus 700 further includes a sampling unit (not shown) configured to determine a single sampling sequence in the following manner:

[0096] Sampling a first vector in a standardized multivariate Gaussian distribution, wherein the mean matrix of the standardized multivariate Gaussian distribution is a zero matrix and the covariance matrix is ​​a unit matrix;

[0097] Superimposing the product of the quadratic root of the first covariance matrix and the first vector with the first mean matrix to obtain a superposition result;

[0098] A single sampling sequence is determined based on an inverse transformation of the superposition result by an inverse function of the reversible transformation function.

[0099] According to one embodiment, the prediction unit 705 is further configured to:

[0100] Sampling each business entity based on non-parametric complex distribution, and arranging the corresponding sampling sequences in sequence to form a sampling matrix as the preliminary prediction result;

[0101] The preliminary prediction results are adjusted to obtain various prediction sequences, so that each prediction sequence satisfies the consistency constraint.

[0102] In a further embodiment, the prediction unit 705 adjusts the preliminary prediction results to obtain various prediction sequences including: encoding the preliminary prediction results to obtain various prediction sequences corresponding to various business entities at the lowest level; and determining various prediction sequences corresponding to various business entities at other levels through consistency constraints.

[0103] Furthermore, the consistency constraint can be represented by a business architecture matrix, and the prediction sequences corresponding to each business subject in other hierarchical levels can be determined based on the product of the business architecture matrix and the coding matrix, wherein the coding matrix is ​​a matrix composed of the prediction sequences corresponding to each business subject in the lowest hierarchical level.

[0104] Among them, the business architecture matrix can be composed of a unit matrix describing each business entity at the lowest level and a sum matrix corresponding to each business entity at other levels. Each business entity at the lowest level corresponds one-to-one to the rows / columns of the corresponding unit matrix, and a single business entity at other levels corresponds to a single row / column in the sum matrix. The single row / column in the sum matrix is ​​formed by superimposing the rows / columns of the corresponding unit matrix on each business entity corresponding to the single business entity at the lowest level.

[0105] It is worth mentioning that Figure 2 The device 700 shown is Figure 2 The method described corresponds to Figure 2 The corresponding description in the method embodiment is also applicable to the device 700 and will not be repeated here.

[0106] According to another embodiment, there is also provided a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute a combination of Figure 2 The method described by et al.

[0107] According to another embodiment of the present invention, a computing device is provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the Figure 2 The method described by et al.

[0108] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of this specification can be implemented by hardware, software, firmware, or any combination thereof. When implemented by software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0109] The specific implementation methods described above further explain in detail the purpose, technical solutions and beneficial effects of the technical concept of this specification. It should be understood that the above description is only the specific implementation method of the technical concept of this specification, and is not used to limit the protection scope of the technical concept of this specification. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the embodiments of this specification should be included in the protection scope of the technical concept of this specification.

Claims

1. A time series prediction method for hierarchical business, used to predict the time series of business volume of business entities at multiple levels on predetermined business indicators, wherein the predetermined business indicator is one of the following: computing resource flow required for cloud computing, power consumption in smart grids, and logistics demand; wherein: A single layer corresponds to at least one business entity; the method comprises: Acquire each historical time series corresponding to each business subject in the multi-level hierarchy, wherein each historical time series corresponds to each business volume of a single business subject in a plurality of predetermined time periods arranged in sequence; Encode each historical time series to obtain the corresponding encoding vectors; Determine the joint probability distribution satisfied by each encoding vector; Processing the joint probability distribution via a standardized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector; Based on the sampling sequences obtained by sampling each business subject according to the non-parametric complex distribution, each prediction sequence for each business subject is determined, wherein a single prediction sequence describes the business volume of a single business subject in multiple subsequent predetermined time periods, and each prediction sequence satisfies the following consistency constraint: for a single time period, the business volume of a single business subject at a higher level is consistent with the sum of the business volume of at least one business subject corresponding to it at a lower level.

2. The method according to claim 1, wherein: The joint probability distribution is a multivariate Gaussian distribution, and the processing of the joint probability distribution via a standardized flow model includes: Starting from the multivariate Gaussian distribution, the multivariate Gaussian distribution is converted into a complex distribution by applying a series of reversible transformation functions, wherein the reversible transformation function repeatedly replaces the current variable with the new variable according to the substitution theorem, and finally obtains the probability distribution of the target variable as a non-parametric complex distribution.

3. The method according to claim 1, wherein: The encoding of each historical time series is implemented by a single recurrent neural network, or an expanded recurrent neural network consisting of multiple recurrent neural networks corresponding to each business entity one by one.

4. The method according to claim 2, wherein: The multivariate Gaussian distribution corresponds to a first mean matrix and a first covariance matrix, and a single sampling sequence is determined in the following manner: Sampling a first vector in a standardized multivariate Gaussian distribution, wherein a mean matrix of the standardized multivariate Gaussian distribution is a zero matrix and a covariance matrix is ​​a unit matrix; Superimposing the product of the second root of the first covariance matrix and the first vector with the first mean matrix to obtain a superposition result; The single sampling sequence is determined based on inverse transformation of the superposition result by an inverse function of the reversible transformation function.

5. The method according to claim 1, wherein: The sampling results obtained by sampling each business subject according to the non-parametric complex distribution respectively, and determining each prediction sequence corresponding to each business subject one by one comprises: Sampling each business entity based on the non-parametric complex distribution, and obtaining corresponding sampling sequences, arranging them in sequence to form a sampling matrix as a preliminary prediction result; The preliminary prediction results are adjusted to obtain various prediction sequences, so that each prediction sequence satisfies the consistency constraint.

6. The method according to claim 5, wherein: The adjusting the preliminary prediction results to obtain each prediction sequence comprises: Encoding the preliminary prediction results to obtain prediction sequences corresponding to the business entities at the lowest level; The prediction sequences corresponding to each business entity in other hierarchical levels are determined through consistency constraints.

7. The method according to claim 6, wherein: The operation of encoding the preliminary prediction result is implemented by a multi-head attention mechanism, including: Split the preliminary prediction results into h pieces of data, where h is the number of attention heads in the multi-head attention mechanism; Performing independent linear transformations under different parameters on the h pieces of data to obtain h processing results; Based on the concatenation of h processing results, the prediction sequences corresponding to the business entities at the lowest level are obtained.

8. The method according to claim 6, wherein: The consistency constraint is represented by a business architecture matrix, and the prediction sequences corresponding to the business entities in other hierarchical levels are determined based on the product of the business architecture matrix and the coding matrix, wherein the coding matrix is ​​a matrix composed of the prediction sequences corresponding to the business entities in the lowest hierarchical level.

9. The method according to claim 8, wherein: The business architecture matrix is ​​composed of a unit matrix describing each business subject at the lowest level and a sum matrix corresponding to each business subject at other levels. Each business subject at the lowest level corresponds one-to-one to the rows / columns of the corresponding unit matrix, and a single business subject at other levels corresponds to a single row / column in the sum matrix. The single row / column in the sum matrix is ​​formed by superimposing the rows / columns of the corresponding unit matrix on each business subject corresponding to the single business subject at the lowest level.

10. A time series prediction device for hierarchical business, used to predict the time series of business volume of business entities at multiple levels on predetermined business indicators, wherein the predetermined business indicator is one of the following: computing resource flow required for cloud computing, power consumption in smart grids, and logistics demand; wherein, A single layer corresponds to at least one business entity; the device comprises: An acquisition unit is configured to acquire each historical time series corresponding to each business subject in the multi-level hierarchy, wherein each historical time series corresponds to each business volume corresponding to each business subject in a plurality of predetermined time periods arranged in sequence; An encoding unit configured to encode each historical time series to obtain corresponding encoding vectors; A mapping unit configured to determine a joint probability distribution satisfied by each encoding vector; A conversion unit configured to process the joint probability distribution via a standardized flow model to obtain a non-parametric complex distribution satisfied by each encoding vector; The prediction unit is configured to determine each prediction sequence for each business subject based on each sampling sequence obtained by sampling each business subject according to the non-parametric complex distribution, wherein a single prediction sequence describes each business volume of a single business subject in a plurality of subsequent predetermined time periods, and each prediction sequence satisfies the following consistency constraint: for a single time period, the business volume of a single business subject at a higher level is consistent with the sum of the business volumes of at least one business subject corresponding to it at a lower level.

11. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method according to any one of claims 1 to 9.

12. A computing device comprising a memory and a processor, characterized in that: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 9 is implemented.

Images