Prediction model construction method and apparatus, and prediction model calling method and apparatus

Abstract

Embodiments of the present application provide a prediction model construction method and apparatus, and a prediction model calling method and apparatus, for use in performing information interaction from different dimensions in a modeling process of a prediction model for predicting a multivariate time series, to enrich information content in the multivariate time series, and suppressing, in the information interaction process, noise generated between the dimensions and noise of the dimensions themselves, thereby obtaining a prediction model having higher prediction precision. When predicting a multivariate time series, a prediction model can perform full information interaction on the multivariate time series, and also has the function of suppressing different types of noise, so that a more accurate prediction result can be obtained, thereby providing a reliable basis for making a rational decision on the basis of the prediction result.

Description

Method and device for constructing and calling prediction model

[0001] The present application claims priority to the Chinese patent application No. 202311514572.7, filed on November 13, 2023, and titled "Method and device for constructing and calling prediction model", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the technical field of computer, in particular to a method for constructing a prediction model, a method for calling the prediction model and a device. BACKGROUND

[0003] Multivariate time series (also can be called multivariable time series or multi-feature time series) refers to data composed of multiple features collected in chronological order. For example, device A includes temperature sensor 1, vibration sensor, …, sensor 10, and data of each sensor is collected every hour within 24 hours. The multivariate time series can be represented as a matrix A with a size of 24*10. The aij in the matrix A can refer to the data collected by sensor j at the i-th sampling time of device A, i = 1, 2, …, 24, j = 1, 2, …, 10. There is usually a certain correlation between the “elements” in the multivariate time series. Therefore, the prediction of the multivariate time series is of great significance to actual production and life. The prediction of the multivariate time series refers to learning the change rule of multiple related variables over time to predict the future situation of the multiple variables. The more accurate the prediction result is, the more reasonable the decision based on the prediction result will be. Therefore, it is worth studying to accurately predict the multivariate time series.

[0004] Currently, the prediction of the multivariate time series does not consider suppressing noise, and the prediction result is not accurate enough due to noise interference. Even if the noise interference is considered and the noise is suppressed, for example, the Dlinear model, different variables need to be split and a corresponding prediction model needs to be constructed for each variable. Such a prediction model cannot accurately predict the multivariate time series, and it can only suppress the noise caused by the interaction between different variables, but cannot suppress the noise included in each variable itself, so the prediction result is still not accurate.

[0005] Therefore, it is urgent to provide a prediction model that can accurately predict the future situation of multiple “elements” of the multivariate time series, so that it is possible to make a reasonable decision based on the accurate prediction result.

[0006] SUMMARY

[0007] Based on this, the application provides a prediction model construction and calling method and device. The constructed prediction model can accurately predict multivariate time series and provide reliable data basis for reasonable decision-making.

[0008] In a first aspect, the application provides a prediction model construction method. The prediction model in the method is used to predict multivariate time series. The construction process of the prediction model can include: on the one hand, information interaction is performed in different dimensions to enrich information content in the multivariate time series; on the other hand, noise generated between the dimensions and noise self-borne by the dimensions is inhibited in the process of information interaction, so that the prediction accuracy of the constructed prediction model is higher. As an example, the prediction model construction method can include: first, a plurality of first training samples are obtained, each of the plurality of first training samples includes a first multivariate time series and a second multivariate time series, the second multivariate time series is a multivariate time series collected after the first multivariate time series in the first training sample to which the second multivariate time series belongs; then, based on the first multivariate time series, the second multivariate time series, an initial model and an information bottleneck constraint condition, a prediction model is obtained, the initial model includes a model performing information interaction in at least one dimension, the information bottleneck constraint condition can be used to at least inhibit noise in the first multivariate time series, and the prediction model is a trained model used to predict a multivariate time series after a known multivariate time series. In this way, in the prediction model training process, the multivariate time series in the training sample is fully subjected to information interaction, and various noises are inhibited based on the information bottleneck constraint condition, so that the constructed prediction model can more accurately predict the multivariate time series.

[0009] In a possible implementation, the information bottleneck constraint condition can also be used to enhance learning of useful information in the multivariate time series, so that a prediction model with more accurate prediction effect is further constructed, and it becomes possible to more accurately predict the multivariate time series based on the prediction model.

[0010] In a possible implementation, taking the first dimension as an example of the at least one dimension, the process of obtaining the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition in the construction process of the prediction model is exemplarily described. The process of obtaining the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition can include N times of updating the initial model in the first dimension, and N is an integer greater than or equal to 1.

[0011] As an example, each time of the updating of the initial model in the first dimension is similar to the process of the updating of the initial model in the first dimension. The process of the updating of the initial model in the first dimension can include: first, inputting the first multivariate time series into the initial model to output a third multivariate time series; then, obtaining first mutual information according to a first intermediate result of the first multivariate time series, the first intermediate result being a high-dimensional mapping of the first multivariate time series in the first dimension, the first mutual information being used to indicate the influence of noise from other dimensions except the first dimension on the prediction process and the useful information learned from other dimensions except the first dimension; obtaining second mutual information according to the second multivariate time series and the first intermediate result, the second mutual information being used to indicate the connection between the representation and the prediction; then, determining a first loss based on the first mutual information and the second mutual information, and updating the initial model based on the first loss to obtain an updated initial model; and updating the updated initial model based on the second multivariate time series and the third multivariate time series. Alternatively, the initial model can be updated based on the second multivariate time series and the third multivariate time series to obtain an updated initial model, and then the updated initial model can be updated based on the first mutual information and the second mutual information. In this implementation, after the N times of updating of the initial model in the first dimension, the updated initial model in the first dimension at the Nth time can be used as the constructed prediction model. In this way, the influence of noise from other features on the training process can be suppressed to obtain a more accurate prediction model.

[0012] For example, the initial model can include a first encoding module and a first decoding module that interact information in the first dimension, and the output of the first encoding module is used as the input of the first decoding module. Then, inputting the first multivariate time series into the initial model to output the third multivariate time series can include: inputting the first multivariate time series into the first encoding module in the initial model, and the output of the first encoding module is the above-mentioned first intermediate result; and inputting the first intermediate result into the first decoding module, and the first decoding module outputs the third multivariate time series.

[0013] In another possible implementation, taking an example of a second dimension in addition to the first dimension in at least one dimension, the process of obtaining the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition in the process of constructing the prediction model is exemplarily described. Wherein, the process of obtaining the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition can include N times of updating of the initial model in the first dimension and M times of updating of the initial model in the second dimension, M being an integer greater than or equal to 1. Wherein, the N times of updating of the initial model in the first dimension is described above.

[0014] As an example, since the M times of updating the initial model in the second dimension, each time of updating the initial model in the second dimension is similar. The process of updating the initial model in the second dimension can include: first, inputting the first multivariate time series into the initial model to output a fourth multivariate time series; then, obtaining a third mutual information according to the first multivariate time series and a second intermediate result, the second intermediate result being a high-dimensional mapping of the first multivariate time series in the second dimension, the third mutual information being used to indicate the influence of noise from each dimension on the prediction process, the each dimension including the second dimension; obtaining a fourth mutual information according to the second multivariate time series and the second intermediate result, the fourth mutual information being used to indicate the connection between the representation and the prediction; then, determining a second loss based on the third mutual information and the fourth mutual information, and updating the initial model based on the second loss to obtain an updated initial model; updating the updated initial model based on the second multivariate time series and the fourth multivariate time series. Alternatively, the initial model can be updated based on the second multivariate time series and the fourth multivariate time series to obtain an updated initial model first; then the updated initial model is updated based on the third mutual information and the fourth mutual information. In this implementation, after the N times of updating the initial model in the first dimension and the M times of updating the initial model in the second dimension are performed, the last updated initial model can be used as the constructed prediction model. In this way, the influence of noise in each dimension on the training process can be suppressed, and a more accurate prediction model can be obtained by more comprehensively suppressing noise.

[0015] For example, the initial model can also include a second encoding module and a second decoding module for information interaction in the second dimension, and the output of the second encoding module is used as the input of the second decoding module. Then, inputting the first multivariate time series into the initial model to output a fourth multivariate time series can include: inputting the first multivariate time series into the second encoding module in the initial model, and the output of the second encoding module is the above-mentioned second intermediate result; inputting the second intermediate result into the second decoding module, and the second decoding module outputs the fourth multivariate time series.

[0016] In yet another possible implementation, the process of obtaining the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition in the process of constructing the prediction model is exemplarily described in the case of at least one dimension including the second dimension. The obtaining of the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition can include L times of updating the initial model in the second dimension, where L is an integer greater than or equal to 1. The L times of updating the initial model in the second dimension is described above and will not be repeated here.

[0017] As an example, in the step of obtaining the prediction model based on the first multivariate time series, the second multivariate time series, the initial model and the information bottleneck constraint condition in the process of constructing the prediction model, the updating of the initial model in the second dimension and the updating of the initial model in the first dimension can be performed according to a preset rule, and the initial model after the previous updating is used as the initial model in the subsequent updating. The preset rule can include but is not limited to, for example, the updating of the initial model in the second dimension and the updating of the initial model in the first dimension can be performed alternately, and the alternating frequency can be 1 or any other positive integer. Taking the case where the alternating frequency is 1 as an example, the updating of the initial model in the first dimension can be performed once after the updating of the initial model in the second dimension is performed once; or, the updating of the initial model in the first dimension can be performed N times after the updating of the initial model in the second dimension is performed M times; or, the updating of the initial model in the second dimension can be performed M times after the updating of the initial model in the first dimension is performed N times.

[0018] As an example, the first dimension can be a variable dimension, a feature dimension or a channel dimension, and the second dimension can be a time dimension or a frequency dimension.

[0019] In a possible implementation, after the prediction model is constructed, the method can further include obtaining a sixth multivariate time series based on a fifth multivariate time series and the prediction model. The fifth multivariate time series is a multivariate time series to be predicted, and the sixth multivariate time series is a prediction result of the prediction model for a multivariate time series that can appear after the fifth multivariate time series. If the prediction performance of the prediction model is good enough, then the multivariate time series collected after the fifth multivariate time series is similar enough to the sixth multivariate time series.

[0020] As an example, the fifth multivariate time series and the first multivariate time series are respectively taken as the input of the prediction model in the prediction stage and the training stage, the size of the fifth multivariate time series and the first multivariate time series can be the same, and the size of the sixth multivariate time series and the second multivariate time series is the same if the prediction is directly based on the constructed prediction model; if there is a specific requirement for the size of the output multivariate time series, the constructed prediction model can be adjusted, and in this case, the size of the sixth multivariate time series and the second multivariate time series can also be different.

[0021] In a second aspect, the application further provides a calling method of a prediction model. In the method, when a first multivariate time series to be predicted is generated, a prediction model constructed based on the method provided in the embodiments of the application is obtained, the prediction model is used to predict the first multivariate time series, and a second multivariate time series is obtained. As an example, the method can include the following steps. First, a prediction model is obtained, the prediction model is a trained model for predicting a multivariate time series, and the prediction model is constrained by an information bottleneck constraint condition in the training process. The information bottleneck constraint condition is used to suppress the multivariate time series in the training sample. Then, based on the prediction model and the first multivariate time series, a second multivariate time series is obtained, the second multivariate time series is the prediction result of the prediction model for the multivariate time series that appears after the first multivariate time series. In this way, since the prediction model can fully interact with the first multivariate time series and has the function of suppressing various noises, the prediction result (i.e., the second multivariate time series) obtained by using the prediction model for the first multivariate time series is accurate and close to the real multivariate time series that appears after the first multivariate time series, thereby providing a reliable basis for making a reasonable decision based on the prediction result.

[0022] In a possible implementation, the information bottleneck constraint condition can also be used to enhance the learning of useful information in the multivariate time series, so that it is possible to make more accurate prediction of the multivariate time series based on the prediction model.

[0023] It should be noted that the related description of the second aspect can refer to the corresponding description of the first aspect, for example, the specific implementation and construction process of the prediction model in the second aspect can refer to the corresponding description of the first aspect.

[0024] In a third aspect, the present application also provides a device for constructing a prediction model, which can include an acquisition unit and a training unit. The acquisition unit is configured to acquire a plurality of first training samples, each of the plurality of first training samples including a first multivariate time series and a second multivariate time series, the second multivariate time series being a multivariate time series collected after the first multivariate time series in the first training sample to which the second multivariate time series belongs; and the training unit is configured to obtain a prediction model based on the first multivariate time series, the second multivariate time series, an initial model, and an information bottleneck constraint condition, the initial model including a model for information interaction in at least one dimension, the information bottleneck constraint condition being used to suppress noise in the first multivariate time series, and the prediction model being a trained model for predicting a multivariate time series after a known multivariate time series.

[0025] In a possible implementation, the information bottleneck constraint condition can also be used to enhance learning of useful information in the multivariate time series.

[0026] In a possible implementation, the at least one dimension includes a first dimension, and the training unit is specifically configured to perform N times of updating the initial model in the first dimension, N being an integer greater than or equal to 1.

[0027] As an example, in the N times of updating the initial model in the first dimension, the process of updating the initial model in the first dimension once includes: inputting the first multivariate time series into the initial model to output a third multivariate time series, the initial model including a first encoding module and a first decoding module for information interaction in the first dimension, the output of the first encoding module being used as the input of the first decoding module; obtaining first mutual information according to the first multivariate time series and a first intermediate result output by the first encoding module, the first intermediate result being a high-dimensional mapping of the first multivariate time series in the first dimension, the first mutual information being used to indicate the influence of noise from other dimensions except the first dimension on the prediction process and being used to indicate useful information learned from other dimensions except the first dimension; obtaining second mutual information according to the second multivariate time series and the first intermediate result, the second mutual information being used to indicate the connection between the representation and the prediction; updating the initial model based on the first mutual information and the second mutual information to obtain an updated initial model; and updating the updated initial model based on the first multivariate time series and the third multivariate time series.

[0028] In another possible implementation manner, the at least one dimension further includes a second dimension, and the training unit is further configured to: perform M times of updating the initial model in the second dimension, where M is an integer greater than or equal to 1.

[0029] As an example, in the M times of updating the initial model in the second dimension, one time of updating the initial model in the second dimension includes: inputting the first multivariate time series into the initial model to output a fourth multivariate time series, the initial model further including a second encoding module and a second decoding module that perform information interaction in the second dimension, an output of the second encoding module serving as an input of the second decoding module; obtaining third mutual information according to the first multivariate time series and a second intermediate result output by the second encoding module, the second intermediate result being a high-dimensional mapping of the first multivariate time series in the second dimension, the third mutual information being used to indicate an influence of noise from each dimension, including the second dimension, on a prediction process; obtaining fourth mutual information according to the second multivariate time series and the second intermediate result, the fourth mutual information being used to indicate a relationship between representation and prediction; updating the initial model based on the third mutual information and the fourth mutual information to obtain an updated initial model; and updating the updated initial model based on the first multivariate time series and the fourth multivariate time series.

[0030] As an example, the updating the initial model in the second dimension and the updating the initial model in the first dimension are alternately performed according to a preset rule, and an initial model in a previous time of updating serves as an initial model in a next time of updating.

[0031] As an example, the first dimension is a feature dimension or a channel dimension, and the second dimension is a time dimension or a frequency dimension.

[0032] In yet another possible implementation manner, the at least one dimension includes a second dimension, and the training unit is specifically configured to: perform L times of updating the initial model in the second dimension, where L is an integer greater than or equal to 1.

[0033] In a possible implementation manner, the apparatus can further include a prediction unit. The prediction unit is configured to obtain a sixth multivariate time series based on a fifth multivariate time series and the prediction model, the sixth multivariate time series being a prediction result of the prediction model on a multivariate time series that appears after the fifth multivariate time series.

[0034] As an example, the fifth multivariate time series has the same size as the first multivariate time series, and the sixth multivariate time series has the same size as or different from the second multivariate time series.

[0035] It should be noted that the related descriptions of the construction device of the prediction model in the third aspect are referred to the corresponding descriptions in the first aspect.

[0036] In the fourth aspect, the present application further provides a calling device of a prediction model, which can include an acquisition unit and a prediction unit. The acquisition unit is configured to acquire a prediction model, the prediction model being a trained model for predicting multivariate time series, and the training process of the prediction model being constrained by an information bottleneck constraint condition, the information bottleneck constraint condition being configured to suppress noise of the multivariate time series in the training sample. The prediction unit is configured to obtain a second multivariate time series based on the prediction model and a first multivariate time series, the second multivariate time series being a prediction result of the prediction model for a multivariate time series occurring after the first multivariate time series.

[0037] In a possible implementation, the information bottleneck constraint condition can also be used to enhance learning of useful information in the multivariate time series.

[0038] It should be noted that the related descriptions of the calling device of the prediction model in the fourth aspect are referred to the corresponding descriptions in the second aspect.

[0039] In the fifth aspect, the present application provides a communication device, which includes a processor and a memory. The processor is configured to execute instructions stored in the memory, so that the communication device implements the method corresponding to the first aspect, the second aspect and possible implementation manners thereof.

[0040] In the sixth aspect, the present application further provides a storage medium, which stores instructions. When the instructions are run on a processor, the method corresponding to the first aspect, the second aspect and possible implementation manners thereof is implemented.

[0041] In the seventh aspect, the present application further provides a program product, which includes a program. When the program is run on a processor, the method corresponding to the first aspect, the second aspect and possible implementation manners thereof is implemented.

[0042] In the eighth aspect, the present application provides a chip, which includes a memory and a processor. The memory is configured to store instructions, and the processor is configured to call and run the instructions from the memory, so as to implement the method corresponding to the first aspect, the second aspect and possible implementation manners thereof. BRIEF DESCRIPTION OF DRAWINGS

[0043] FIG. 1 is a flow diagram of a method 100 for constructing a prediction model according to an embodiment of the present application;

[0044] FIG. 2 is a schematic diagram of a framework for information interaction of an initial model according to an embodiment of the present application;

[0045] FIG. 3 is a schematic diagram of initial model training corresponding to Channel Attention according to an embodiment of the present application;

[0046] FIG. 4 is a schematic diagram of initial model training corresponding to Temporal Attention according to an embodiment of the present application;

[0047] FIG. 5 is a flow diagram of a method 200 for calling a prediction model according to an embodiment of the present application;

[0048] FIG. 6 is a schematic diagram of a structure of a communication apparatus 600 according to an embodiment of the present application;

[0049] FIG. 7 is a schematic diagram of a structure of a communication apparatus 700 according to an embodiment of the present application;

[0050] FIG. 8 is a schematic diagram of a structure of a communication device 800 according to an embodiment of the present application. DETAILED DESCRIPTION

[0051] Prediction of multivariate time series has wide applications in many fields, such as finance, economy, meteorology, transportation, etc. If decision makers know the prediction results of multiple variables involved in a known multivariate time series in the future, they can understand the trend of the multiple variables in the future, which serves as an important reference for making corresponding decisions. For example, in the financial field, investors need to predict the future trend of multiple variables such as stock prices, exchange rates, and interest rates, in order to make investment decisions. For another example, in the meteorological field, meteorological forecasters need to predict the future trend of multiple variables such as temperature, humidity, and rainfall, in order to provide accurate weather forecasts. For another example, in the transportation field, transportation planners need to predict the future trend of multiple variables such as traffic flow and congestion, in order to develop reasonable transportation planning. For another example, in grid-connected applications of industrial entities, industrial entities need to predict the load power consumption, in order to determine the buying and selling of power from the power grid, to maximize their own profits under the premise of meeting their own production. It can be seen that accurate prediction of multivariate time series has important significance in practical applications.

[0052] Currently, the prediction of multivariate time series is not accurate enough.

[0053] For example, for the Frequency Enhanced Decomposed Transformer (FEDFormer), the main principle can be understood as follows: relying on the attention mechanism, the point-wise relationship is captured, the enhancement and attention operations are converted to the frequency domain by using the time-to-frequency domain conversion, which can optimize the prediction performance to a certain extent and also reduce the computational complexity of the model. However, the FEDFormer has the following problems when predicting a multivariate time series: Problem one, it does not consider suppressing noise in the multivariate time series, and the prediction result is not accurate enough; Problem two, in order to reduce the computational complexity, the time domain data with a large vector length is converted to frequency domain data with a small vector length, and then operations are performed in the frequency domain. The sampling in the frequency domain is random, which reduces the information content of the multivariate time series, making the prediction result not accurate enough; Problem three, the FEDFormer itself has a certain complexity, and a large amount of data is needed for training to achieve a certain prediction accuracy.

[0054] For example, for Dlinear, the main principle can be understood as follows: a Multilayer Perceptron (MLP) structure is used for modeling to capture the point-to-point relationship. In specific implementation, the multivariate time series is n features collected at T time points, both n and T are integers greater than 1, Dlinear splits the multivariate time series in the feature dimension, and the multivariate time series is divided into n “unit time series”, n is the number of “units” in the multivariate time series, and each “unit time series” only indicates the data of one feature at multiple time points; for each “unit time series”, a model is established separately, and each model only needs to consider using the MLP structure to capture the mapping from the input T time points to the output future K time points, K is an integer greater than or equal to 1, K can be equal to T, and K can also be not equal to T. Although Dlinear predicts the multivariate time series by modeling each feature separately, it can prevent the prediction of each feature corresponding to the “unit time series” from introducing noise from other features, but it has the following problems: Problem one, separate modeling in the feature dimension does not introduce information from other features, resulting in a lack of contribution from other features that are associated with each feature, and the prediction result is likely to be inaccurate; Problem two, although it is ensured that no noise from other features is introduced, each feature corresponding to the “unit time series” will inevitably have its own noise, and this method does not consider suppressing the noise of the feature itself in the “unit time series” corresponding to each feature.

[0055] Moreover, whether it is FEDFormer or Dlinear, the modeling is only in the time dimension, and not in the feature dimension.

[0056] Based on this, the embodiment of the present application provides a method for constructing a prediction model for predicting a multivariate time series. In the modeling process of the prediction model, information interaction is performed from different dimensions to enrich the information content in the multivariate time series. In the process of information interaction, noise generated between the dimensions and noise carried by the dimensions themselves is suppressed to obtain a prediction model with higher prediction accuracy.

[0057] In a possible implementation, the embodiment of the present application provides a method for constructing a prediction model. In the method, first, a plurality of first training samples are obtained. Each of the plurality of first training samples includes a first multivariate time series and a second multivariate time series. The second multivariate time series is a multivariate time series collected after the first multivariate time series in the first training sample to which the second multivariate time series belongs. Then, based on the first multivariate time series, the second multivariate time series, an initial model, and an information bottleneck constraint condition, a prediction model is obtained. The initial model includes a model that performs information interaction in at least one dimension. The information bottleneck constraint condition is used to suppress noise in the first multivariate time series. The prediction model is a trained model used to predict a multivariate time series after a known multivariate time series. In this way, in the training process of the prediction model, sufficient information interaction is performed on the multivariate time series in the training sample, and various noises are suppressed based on the information bottleneck constraint condition, so that the constructed prediction model can more accurately predict the multivariate time series.

[0058] It should be noted that the subject implementing the method for constructing the prediction model can be the prediction model construction device provided by the embodiment of the present application. The prediction model construction device can be carried in an electronic device or a functional module of an electronic device.

[0059] In another possible implementation, the embodiment of the present application further provides a method for calling a prediction model. In the method, when a first multivariate time series to be predicted is generated, a prediction model constructed based on the method provided by the embodiment of the present application can be obtained. The prediction model is used to predict the first multivariate time series to obtain a second multivariate time series. Since the prediction model can perform sufficient information interaction on the first multivariate time series and has the function of suppressing various noises, the prediction result (i.e., the second multivariate time series) obtained by using the prediction model to predict the first multivariate time series is accurate and close to the real multivariate time series that appears after the first multivariate time series. This provides a reliable basis for making a reasonable decision based on the prediction result.

[0060] It should be noted that the subject of the calling method of the prediction model can be the calling device of the prediction model provided by the embodiments of the present application, and the calling device of the prediction model can be carried in an electronic device or a functional module of the electronic device. The subject of the calling method of the prediction model can be the same as or different from the subject of the construction method of the prediction model.

[0061] Before introducing the method provided by the embodiments of the present application, the concepts that can be involved in the embodiments of the present application are described.

[0062] The attention mechanism generally refers to a mechanism for extracting part of information in all information for information perception. The mechanism is derived from the study of human vision. In human cognition, due to the bottleneck of information processing, humans selectively focus on part of all information and ignore other visible information, so the attention mechanism appears. For neural networks, the attention mechanism can be understood as layer-by-layer aggregation and update of information. For prediction of multi-element time series, the attention mechanism is to refresh the information of a position by using the information of other positions.

[0063] Multi-view (MV) refers to learning from multiple perspectives, so that the model better understands things from multiple aspects, thereby improving the performance of the model. The understanding of multi-view can include the following two aspects: aspect one, multi-view means multiple sources, for example, face, fingerprint, etc. can be used as input of different sources in human body identification; aspect two, multi-view means multiple feature subsets, for example, an image can be described from different features such as color and text. For prediction of multi-element time series, multi-view can be understood as modeling from different angles of data, for example, the prediction model of multi-element time series can perform information interaction in the time dimension and the feature (also referred to as element, variable or channel) dimension.

[0064] Mutual information is a measure of information quantity in information theory, which can be understood as the amount of information contained in a random variable about another random variable, or it can also be understood as the uncertainty of a random variable due to the known of another random variable.

[0065] Information Bottleneck (IB) is an information theory method for explaining and analyzing the complexity of data. The theory believes that the information in the data can be divided into two parts: useful information and useless information. Useful information refers to the part that can be used to predict or explain the data, while useless information refers to the part that cannot provide any predictive or explanatory value. The core idea of information bottleneck is to extract the key information (key information can also be understood as as much useful information as possible) in the data by minimizing the difference between useful information and useless information. This difference is called "information bottleneck" because it is the bottleneck that limits data compression and prediction ability.

[0066] The following describes a method for constructing a prediction model provided by an embodiment of the present application.

[0067] FIG. 1 is a flowchart of a method 100 for constructing a prediction model provided by an embodiment of the present application. The method 100 can be applied to a prediction model construction device, which can be, for example, the prediction model construction device 600 shown in FIG. 6.

[0068] As shown in FIG. 1, the method 100 may, for example, include the following S101-S102:

[0069] S101, obtaining a plurality of first training samples, each of the plurality of first training samples including a first multivariate time series and a second multivariate time series, the second multivariate time series being a multivariate time series collected after the first multivariate time series in the first training sample.

[0070] As an example, S101 may, for example, include obtaining historical data of a preset time length (such as 30 days), which can be data collected and saved in a preset time length according to a preset time granularity (such as 15 minutes); dividing the historical data in the time dimension according to a preset step (such as 192) to obtain a plurality of first training samples, each of which can include a first multivariate time series and a second multivariate time series, the first multivariate time series can be the first T (such as 96) multivariate time series in the first training sample, and the second multivariate time series can be the multivariate time series in the first training sample other than the first multivariate time series.

[0071] Taking the time granularity in the historical data as 15 minutes (i.e., collecting the data of each feature once every 15 minutes), and taking 30 days as a preset time length as an example, the historical data includes feature data of 96*30=2880 time points; assuming that the prediction model uses a day (i.e., 96 time points) of multivariate time series to predict a day of multivariate time series in the future, then the input and output of the prediction model need 96*2=192 consecutive time points of feature data, so when the 2880 time points of feature data in the historical data are divided, 15 first training samples are obtained by taking 192 as a preset step length, and each first training sample includes 192 consecutive time points of feature data; in each first training sample, the feature data of the first 96 time points constitutes a first multivariate time series, which is used as the input when the prediction model is trained using the first training sample, and the feature data of the last 96 time points constitutes a second multivariate time series, which is used as the theoretical output corresponding to the input first multivariate time series when the prediction model is trained using the first training sample. It can be understood that the label of the training data in the usual training sample.

[0072] It can be seen that the first training sample obtained in S101 provides a data basis for implementing the training step in S102.

[0073] In S102, a prediction model is obtained based on the first multivariate time series, the second multivariate time series, an initial model, and an information bottleneck constraint condition. The initial model includes a model that performs information interaction in at least one dimension. The information bottleneck constraint condition is used to suppress noise in the first multivariate time series. The prediction model is a trained model used to predict a multivariate time series after a known multivariate time series.

[0074] As an example, S102 may, for example, include: S11, training the initial model based on the i-th first training sample to obtain an updated initial model, i=1, 2, …, A, A being the number of first training samples; S12, determining whether a training stop condition is met, if yes, executing S14, otherwise executing S13; S13, i=i+1 and returning to execute S11; S14, ending the training process and determining the updated initial model as the prediction model in S102. In this way, the initial model is trained multiple times, the initial model is updated continuously, the prediction performance of the initial model is continuously improved, the initial model obtained when the training stop condition is met is recorded as the prediction model, and accurate prediction of the multivariate time series is realized. The training stop condition may, for example, include but is not limited to: i being equal to A, or the prediction accuracy of the initial model being greater than a preconfigured accuracy threshold.

[0075] In the embodiments of the present application, in order to enable the constructed prediction model to interact information of the input multivariate time series, to improve the richness of the data input by the prediction model, and better explore the correlation between features, the architecture of the initial model is a model architecture that includes information interaction of at least one dimension. For example, the architecture of the initial model can include but is not limited to Multi-View Attention architecture or MLP architecture. In the following, the architecture of the initial model is taken as an example of Multi-View Attention architecture. The at least one dimension can include time dimension, frequency dimension, variable dimension, feature dimension or channel dimension. In the following, the at least one dimension is taken as the first dimension, the second dimension or the first dimension+the second dimension, the first dimension is the variable dimension, the feature dimension or the channel dimension, and the second dimension is the time dimension or the frequency dimension.

[0076] For information interaction of the initial model, refer to FIG. 2. In FIG. 2, the multivariate time series is a 6*3 sequence, including feature data of 3 features at 6 time points. The 6 can be understood as the sequence length, and the 3 can be understood as the number of features. When the first multivariate time series passes through the Multi-View Attention architecture, information interaction is performed from different angles. For example, information interaction can be performed from the time dimension (Temporal Attention in FIG. 2), and / or information interaction can be performed from the feature dimension (Channel Attention in FIG. 2). The Temporal Attention can be understood as information interaction using data of different time points of the same feature. For example, in FIG. 2, for a column of features represented by black, the data of each time point is interacted with all or part of the data of other time points of the feature to obtain updated feature data of the time point. The number of time points after information interaction is not limited. The Channel Attention can be understood as information interaction using data of different features at the same (or different) time point. For example, in FIG. 2, for a feature data in the first row, information interaction is performed with all or part of the feature data in the first row, and / or information interaction is performed with all or part of the feature data in each row to obtain updated feature data of the feature. The number of features after information interaction is not limited.

[0077] The effect of information interaction can be understood as discovering the correlation between data in the multivariate time series, and making the information amount of the information extracted sufficient rich during information extraction, so as to more accurately map to the downstream task (i.e., more accurate prediction of the future).

[0078] The Multi-View Attention architecture includes N times of Channel Attention and M times of Temporal Attention. The execution order of Channel Attention and Temporal Attention can be flexibly designed based on requirements. For example, Channel Attention and Temporal Attention can be alternately executed, which can be regular or irregular. When alternately executed regularly, the alternation frequency can be 1 or an integer greater than 1. For another example, Channel Attention and Temporal Attention can be executed in sequence. Channel Attention can be executed first, followed by Temporal Attention, or vice versa. For another example, Channel Attention and Temporal Attention can be executed simultaneously without interfering with each other. After N times of Channel Attention and M times of Temporal Attention are executed, the initial models obtained are fused. In FIG. 2, the Multi-View Attention architecture includes N times of Channel Attention and M times of Temporal Attention. Channel Attention and Temporal Attention are alternately executed at an alternation frequency of 1. M and N are integers greater than or equal to 1. The values of M and N can be the same or different.

[0079] In the process of implementing information fusion on the input initial model by interacting information of different dimensions, an information bottleneck constraint condition is designed as a constraint on the initial model training process to suppress noise. The information bottleneck constraint condition corresponds to different constraint conditions for different dimensions. Optionally, the information bottleneck constraint condition can also be used to enhance the learning of useful information. Hereinafter, the information bottleneck constraint condition capable of suppressing noise and enhancing the learning of useful information is taken as an example for description.

[0080] In a possible implementation, if the at least one dimension includes a first dimension, S102 can include N times of updating the initial model in the first dimension, which can be understood as S102 includes performing N times of Channel Attention in the scenario of FIG. 2. The updating process of the initial model in the first dimension each time is similar, and the implementation of S102 in this implementation is described below by taking the updating process of the initial model in the first dimension once as an example.

[0081] As an example, the updating process of the initial model in the first dimension once in S102 can include: S21, inputting the first multivariate time series into the initial model to output a third multivariate time series; S22, obtaining first mutual information according to the first multivariate time series and a first intermediate result, the first intermediate result being a high-dimensional mapping of the first multivariate time series in the first dimension, the first mutual information being used to indicate an influence of noise from other dimensions except the first dimension on a prediction process and being used to indicate useful information learned from other dimensions except the first dimension; S23, obtaining second mutual information according to the second multivariate time series and the first intermediate result, the second mutual information being used to indicate a connection between representation and prediction; S24, determining a first loss based on the first mutual information and the second mutual information, and updating the initial model based on the first loss. Optionally, the process can further include: S25, updating the initial model based on the second multivariate time series and the third multivariate time series. In S25, the “initial model” to be updated can be the “initial model” obtained after S24 is updated.

[0082] It can be understood that the initial model can include a first encoding module and a first decoding module that perform information interaction in the first dimension, the output of the first encoding module being used as the input of the first decoding module, for example, the first intermediate result is the input of the first encoding module and is also the output of the first decoding module. The first encoding module and the first decoding module can implement information interaction corresponding to the first dimension, for example, the process of the first encoding module performing high-dimensional mapping on the input first multivariate time series to obtain the first intermediate result involves information interaction in the first dimension.

[0083] It should be noted that the updating of the initial model based on the second multivariate time series and the third multivariate time series in S25 can be, for example, updating the initial model based on a difference between the second multivariate time series and the third multivariate time series.

[0084] It should be noted that the execution order of S22 and S23 is not limited in the embodiments of the present application. The execution order of S24 and S25 is not limited in the embodiments of the present application.

[0085] For example, taking the first dimension as the variable dimension, the feature dimension or the channel dimension as an example, as shown in FIG. 3, the initial model 10 can include a first encoding module 11 and a first decoding module (also referred to as a first prediction module) 12 for information interaction in the first dimension. The first training sample includes a first multivariate time series X and a second multivariate time series Y. The first multivariate time series X is input into the first encoding module 11, and the first encoding module 11 outputs a first intermediate result Zi. The first intermediate result Zi is input into the first decoding module 12, and the first decoding module 12 outputs a third multivariate time series Yi’. In FIG. 3, Xi represents the i-th feature in the first multivariate time series X, Xj represents other features in the first multivariate time series X except the i-th feature, and Zi is a representation of Xi (also referred to as a high-dimensional mapping). In Channel Attention, the role of the information bottleneck constraint condition 1 corresponding to the first dimension can be understood as follows: on the one hand, the influence of noise from other features on the training process is suppressed, and on the other hand, useful information learned from other features is increased. For example, suppressing noise and increasing learned useful information can be achieved through mutual information between Zi and Xj (i.e., the first mutual information described above). Mutual information between Zi and Yi (i.e., the second mutual information described above) is used to indicate the relationship between the representation and the prediction. The information bottleneck constraint condition 1 can be expressed by the following formula (1), for example: IB L = I(Zi; Yi) - β*I(Zi; Xj) …… Formula (1)

[0086] where I(Zi; Yi) can correspond to the second mutual information in S23, I(Zi; Xj) can correspond to the first mutual information in S22, β is a hyperparameter, which can be flexibly set based on actual requirements, and L IB is the basis for updating the initial model in S24 in this Channel Attention, which can be understood as a first loss determined based on the first mutual information and the second mutual information.

[0087] In this example, after performing N times of Channel Attention, the initial model updated after S24 or S25 in the Nth Channel Attention can be used as the prediction model in S102.

[0088] As another example, if the at least one dimension includes a second dimension in addition to the first dimension, S102 can include N times of updating the initial model in the first dimension and M times of updating the initial model in the second dimension, which can be understood as S102 including performing N times of Channel Attention and M times of Temporal Attention in the scenario of FIG. 2. The process of each time of updating the initial model in the first dimension can refer to S21-S25 described above. The process of each time of updating the initial model in the second dimension is similar, and the process of one time of updating the initial model in the second dimension is taken as an example below to introduce the implementation of S102 in this implementation.

[0089] In S102, the process of one time of updating the initial model in the second dimension can include: S31, inputting the first multivariate time series into the initial model to output a fourth multivariate time series; S32, obtaining third mutual information according to the first multivariate time series and a second intermediate result, the second intermediate result being a high-dimensional mapping of the first multivariate time series in the second dimension, the third mutual information being used to indicate the influence of noise from each dimension on the prediction process, the each dimension including the second dimension; S33, obtaining fourth mutual information according to the second multivariate time series and the second intermediate result, the fourth mutual information being used to indicate the connection between the representation and the prediction; S34, determining a second loss based on the third mutual information and the fourth mutual information, and updating the initial model based on the second loss. Optionally, the process can further include: S35, updating the initial model based on the second multivariate time series and the fourth multivariate time series.

[0090] It can be understood that the initial model can further include a second encoding module and a second decoding module for information interaction in the second dimension, the output of the second encoding module being the input of the second decoding module, for example, the second intermediate result described above is the output of the second encoding module and the input of the second decoding module. The second encoding module and the second decoding module can implement information interaction in the second dimension, for example, the process of the second encoding module performing high-dimensional mapping on the input first multivariate time series to obtain the second intermediate result involves information interaction in the second dimension.

[0091] It should be noted that the updating of the initial model based on the second multivariate time series and the fourth multivariate time series in S35 can be, for example, updating the initial model based on the difference between the second multivariate time series and the fourth multivariate time series.

[0092] It should be noted that the execution order of S32 and S33 is not limited in the embodiments of the present application. The execution order of S34 and S35 is not limited in the embodiments of the present application.

[0093] It should be noted that the difference between S33 and S23 is that S23 suppresses the influence of noise from other features on the training process based on the first loss, but S33 can suppress the influence of noise from each feature, including the current feature and other features, on the training process based on the second loss, which can more comprehensively suppress noise, so that the prediction model can more comprehensively suppress noise for the multi-element time series to be inferred, and it is possible to make the prediction model more accurate.

[0094] For example, taking the first dimension as the variable dimension, the feature dimension or the channel dimension, and the second dimension as the time dimension or the frequency dimension, as shown in FIG. 4, the initial model 10 can further include a second encoding module 21 and a second decoding module (which can also be referred to as a second prediction module) 22 for information interaction in the second dimension. The first training sample includes a first multi-element time series X and a second multi-element time series Y. The first multi-element time series X is input into the second encoding module 21, and the second encoding module 21 outputs a second intermediate result Z. The second intermediate result Z is input into the second decoding module 22, and the second decoding module 22 outputs a fourth multi-element time series Y'. In FIG. 4, Z is a representation of X. The role of the information bottleneck constraint condition 2 corresponding to the second dimension in the Temporal Attention can be understood as follows: on the one hand, suppressing the influence of noise from other features on the training process, and on the other hand, suppressing the influence of noise from the feature itself on the training process. For example, suppressing noise from other features and suppressing noise from the feature itself can be achieved by reducing the mutual information between Z and X (i.e., the third mutual information described above). The mutual information between Z and Y (i.e., the fourth mutual information) is used to indicate the relationship between the representation and the prediction. The information bottleneck constraint condition 2 can be expressed by the following formula (2), for example: IB L = I(Z; Y) - β * I(Z; X) …… Formula (2)

[0095] Wherein, I(Z; Y) can correspond to the fourth mutual information in S33, I(Z; X) can correspond to the third mutual information in S32, β is a hyperparameter, which can be flexibly set based on actual needs, and L IB is the basis for updating the initial model in S34 this time, which can be understood as a second loss determined based on the third mutual information and the fourth mutual information.

[0096] In this example, after performing the N times of Channel Attention and the M times of Temporal Attention, the initial model updated by the last Attention can be used as the prediction model in S102. In this example, the update of the initial model in the second dimension and the update of the initial model in the first dimension can be performed according to a preset rule. In either the update of the initial model in the second dimension or the update of the initial model in the first dimension, the initial model updated by the previous update can be used as the initial model in the subsequent update.

[0097] In another possible implementation, if the at least one dimension includes the second dimension, S102 can include L times of update of the initial model in the second dimension. In the scenario of FIG. 2, S102 can include performing L times of Temporal Attention. The process of updating the initial model in the second dimension each time is similar, and the process of updating the initial model in the first dimension once can be understood with reference to the description of S31-S35 above. L can be an integer greater than or equal to 1, and L can be the same as or different from N.

[0098] In this implementation, after performing the L times of Temporal Attention, the initial model updated in the Lth Temporal Attention can be used as the prediction model in S102.

[0099] In some implementations, after the construction apparatus of the prediction model performs S101-S102, the obtained prediction model can be integrated into a device or a server that can be loaded by a user. When the calling apparatus of the prediction model has a prediction task of a multivariate time series, the calling apparatus of the prediction model can load the prediction model from the device or the server, and perform prediction on the multivariate time series by means of the prediction model.

[0100] As an example, after S102, the method 100 can further include S103: obtaining a sixth multivariate time series based on a fifth multivariate time series and the prediction model, the sixth multivariate time series being a prediction result of the prediction model on a multivariate time series that appears after the fifth multivariate time series. The size of the fifth multivariate time series can be the same as that of the first multivariate time series, and the size of the sixth multivariate time series can be the same as or different from that of the second multivariate time series.

[0101] Generally, when the prediction is directly based on the prediction model constructed in S102, the size of the sixth multivariate time series is the same as that of the second multivariate time series.

[0102] Taking the prediction task of the multi-view time series in the scenario of "load prediction" as an example, when constructing a high-precision prediction model in the method 100, the historical data can come from at least one data set, including the load data of the at least one data set at a certain site for one year, the time granularity is 15 minutes, and a plurality of numerical weather features that can affect the load of the site for one year are also included, the training process is described with reference to S101-S102 in the method 100, and a load prediction model is obtained. When using the load prediction model, the load data X 1:T ∈R T×D of the historical T time points (a time point appears every 15 minutes) can be used to predict the load situation X T+1:T+K ∈R K×D of the future K time points, where D is the number of features involved in the multi-view time series. If T is 96 in the first multi-view time series and K is 96 in the second multi-view time series during training, the historical multi-view time series of T = 96 time points (96 * 15 minutes = one day) can be used to predict the load situation for one day when using the load prediction model. For example, the multi-view time series collected every 15 minutes from 3 pm today to 3 pm yesterday is used as the input of the load prediction model, and the prediction result output by the load prediction model can indicate the load situation from 3 pm today to 3 pm tomorrow.

[0103] When a prediction result of a specific scale is needed and the specific scale is different from the scale of the second multi-view time series, the structure of the prediction model can be adjusted so that the prediction result is a multi-view time series of a specific scale. The prediction model can include Multi-View Attention and MLP, and the MLP is used to simply adjust the data output by the Multi-View Attention. Therefore, the effect of adjusting the prediction result to a specific scale can be achieved by adjusting the structure of the MLP.

[0104] The prediction model (also referred to as MTS-VIB) constructed by the embodiments of the present application is compared with other models capable of predicting multi-view time series. Through verification, in multiple cross-validations performed on the same data set, the prediction performance of the prediction model constructed by the embodiments of the present application is better than that of other models.

[0105] The prediction model constructed by the embodiments of the present application can be applied to various fields of predicting multivariate time series, such as influenza epidemic cases, traffic, weather, electricity, transformer box oil temperature and other fields involving multivariate time series prediction tasks. Through verification, among various models for predicting multivariate time series based on open source datasets ILI, Traffic, Weather, Electricity, ETTm2, ETTm1 and ETTh2, the prediction model constructed by the embodiments of the present application has the best prediction performance.

[0106] It can be seen that in the method 100, the prediction model is trained by multi-view information interaction and information bottleneck constraint, which can extract more useful information and suppress various noises, so as to perform high-precision prediction on multivariate time series. Moreover, the prediction model is not limited to the selection of the initial model, and has strong generalization.

[0107] Next, a calling method of a prediction model provided by the embodiments of the present application is introduced.

[0108] FIG. 5 is a flowchart of a calling method 200 of a prediction model provided by the embodiments of the present application. The method 200 can be applied to a prediction model calling device, such as the prediction model calling device 700 shown in FIG. 7.

[0109] As shown in FIG. 5, the method 200 may, for example, include the following S201-S202:

[0110] S201, obtaining a prediction model, the prediction model being a trained model for predicting multivariate time series, and the prediction model being constrained by an information bottleneck constraint condition in a training process, the information bottleneck constraint condition being used to suppress noise of the multivariate time series in a training sample.

[0111] S202, obtaining a second multivariate time series based on the prediction model and a first multivariate time series, the second multivariate time series being a prediction result of the prediction model on the multivariate time series appearing after the first multivariate time series.

[0112] Optionally, the information bottleneck constraint condition can also be used to enhance learning of useful information in the first multivariate time series.

[0113] It should be noted that the prediction model can refer to the related description of the prediction model in the method 100. S201 and S202 can refer to the related description of S103, the first multivariate time series can correspond to the fifth multivariate time series of S103, and the second multivariate time series corresponds to the sixth multivariate time series of S103.

[0114] It can be seen that, in the method 200, since the prediction model can perform sufficient information interaction on the first multivariate time series, and has the functions of enhancing learning of useful information in the first multivariate time series and suppressing various noises, the prediction result (i.e., the second multivariate time series) obtained by using the prediction model on the first multivariate time series is accurate, close to the real multivariate time series appearing after the first multivariate time series, and provides a reliable basis for the user to make a reasonable decision based on the prediction result.

[0115] Correspondingly, the embodiment of the application further provides a communication device 600 (which can also be referred to as a construction device 600 of the prediction model), as shown in FIG. 6. The communication device 600 can include an acquisition unit 601 and a training unit 602. Wherein:

[0116] The acquisition unit 601 is configured to acquire a plurality of first training samples, each of the plurality of first training samples including a first multivariate time series and a second multivariate time series, the second multivariate time series being a multivariate time series collected after the first multivariate time series in the first training sample to which the second multivariate time series belongs. The acquisition unit 601 can perform S101 shown in FIG. 1.

[0117] The training unit 602 is configured to obtain a prediction model based on the first multivariate time series, the second multivariate time series, an initial model, and an information bottleneck constraint condition, the initial model including a model performing information interaction in at least one dimension, the information bottleneck constraint condition being used to suppress noise in the first multivariate time series, and the prediction model being a trained model used to predict a multivariate time series after a known multivariate time series. The training unit 602 can perform S102 shown in FIG. 1.

[0118] In a possible implementation, the information bottleneck constraint condition is further used to enhance learning of useful information in the first multivariate time series.

[0119] In a possible implementation, the at least one dimension includes a first dimension, and the training unit 602 is specifically configured to perform N times of updating of the initial model in the first dimension, N being an integer greater than or equal to 1.

[0120] As an example, in the process of performing N times of updating the initial model in the first dimension, the training unit 602 performs, in one of the N times, the following process of updating the initial model in the first dimension: inputting the first multivariate time series into the initial model to output a third multivariate time series, the initial model comprising a first encoding module and a first decoding module that interact information in the first dimension, the output of the first encoding module serving as the input of the first decoding module; obtaining first mutual information according to the first multivariate time series and a first intermediate result output by the first encoding module, the first intermediate result being a high-dimensional mapping of the first multivariate time series in the first dimension, the first mutual information being used to indicate the influence of noise from other dimensions except the first dimension on the prediction process and being used to indicate useful information learned from other dimensions except the first dimension; obtaining second mutual information according to the second multivariate time series and the first intermediate result, the second mutual information being used to indicate the connection between the representation and the prediction; updating the initial model based on the first mutual information and the second mutual information to obtain an updated initial model; and updating the updated initial model based on the first multivariate time series and the third multivariate time series.

[0121] In another possible implementation, the at least one dimension further comprises a second dimension, and the training unit 602 is further configured to perform M times of updating the initial model in the second dimension, M being an integer greater than or equal to 1.

[0122] As an example, in the process of performing M times of updating the initial model in the second dimension, the training unit 602 performs, in one of the M times, the following process of updating the initial model in the second dimension: inputting the first multivariate time series into the initial model to output a fourth multivariate time series, the initial model further comprising a second encoding module and a second decoding module that interact information in the second dimension, the output of the second encoding module serving as the input of the second decoding module; obtaining third mutual information according to the first multivariate time series and a second intermediate result output by the second encoding module, the second intermediate result being a high-dimensional mapping of the first multivariate time series in the second dimension, the third mutual information being used to indicate the influence of noise from each dimension on the prediction process, the each dimension including the second dimension; obtaining fourth mutual information according to the second multivariate time series and the second intermediate result, the fourth mutual information being used to indicate the connection between the representation and the prediction; updating the initial model based on the third mutual information and the fourth mutual information to obtain an updated initial model; and updating the updated initial model based on the first multivariate time series and the fourth multivariate time series.

[0123] As an example, the updating the initial model in the second dimension and the updating the initial model in the first dimension are alternately performed according to a preset rule, and the initial model after the previous updating is used as the initial model in the subsequent updating operation.

[0124] As an example, the first dimension is a feature dimension or a channel dimension, and the second dimension is a time dimension or a frequency dimension.

[0125] In yet another possible implementation, the at least one dimension includes a second dimension, and the training module is specifically configured to: perform L times of updating the initial model in the second dimension, where L is an integer greater than or equal to 1.

[0126] In a possible implementation, the apparatus 600 can further include a prediction unit. The prediction unit is configured to obtain a sixth multivariate time sequence based on a fifth multivariate time sequence and the prediction model, where the sixth multivariate time sequence is a prediction result of the prediction model on a multivariate time sequence occurring after the fifth multivariate time sequence.

[0127] As an example, the fifth multivariate time sequence has the same size as the first multivariate time sequence, and the sixth multivariate time sequence has the same size as or different from the second multivariate time sequence.

[0128] It should be noted that various specific embodiments of the communication apparatus 600 can refer to the related description of the method 100 in FIG. 1, and details are not described herein again.

[0129] Correspondingly, the embodiment of the application further provides a communication apparatus 700 (which can also be referred to as a calling apparatus 700 of a prediction model), as shown in FIG. 7. The communication apparatus 700 can include an acquisition unit 701 and a prediction unit 702. Wherein:

[0130] The acquisition unit 701 is configured to acquire a prediction model, where the prediction model is a trained model used for predicting a multivariate time sequence, and the prediction model is subjected to a constraint of an information bottleneck constraint condition in a training process of the prediction model, and the information bottleneck constraint condition is used to suppress noise of the multivariate time sequence in a training sample. The acquisition unit 701 can perform S201 shown in FIG. 5.

[0131] The prediction unit 702 is configured to obtain a second multivariate time sequence based on the prediction model and a first multivariate time sequence, where the second multivariate time sequence is a prediction result of the prediction model on a multivariate time sequence occurring after the first multivariate time sequence. The prediction unit 702 can perform S202 shown in FIG. 5.

[0132] Optionally, the information bottleneck constraint condition can also be used to enhance learning of useful information in the first multivariate time series.

[0133] It should be noted that various specific embodiments of the communication device 700 can refer to the related description of the corresponding method 200 in FIG. 5, and the present embodiment will not be repeated.

[0134] Referring to FIG. 8, the present embodiment provides a communication device 800. The communication device 800 can be any of the above-mentioned execution subjects. The communication device 800 can implement the functions of the above-mentioned embodiments. The communication device 800 includes at least one processor 801, a bus system 802, a memory 803, and at least one communication interface 804.

[0135] The communication device 800 is a hardware structure device, which can be used to implement the functional modules in the communication device 600 shown in FIG. 6. For example, those skilled in the art can conceive that the acquisition unit 601 and the training unit 602 in the communication device 600 shown in FIG. 6 are implemented by the at least one processor 801 calling the code in the memory 803. For another example, those skilled in the art can conceive that the acquisition unit 701 and the prediction unit 702 in the communication device 700 shown in FIG. 7 can be implemented by the at least one processor 801 calling the code in the memory 803.

[0136] Optionally, the communication device 800 can be a network device or a control entity implementing the embodiments of the present application.

[0137] Optionally, the processor 801 can be a general central processing unit (CPU), a network processor (NP), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of programs of the present application.

[0138] The bus system 802 can include a path for transmitting information between the above-mentioned components.

[0139] The communication interface 804 is used for communication with other devices or communication networks.

[0140] The memory 803 can be a read-only memory (ROM) or other type of static storage device that can store static information and instructions, a random access memory (RAM), or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage or other magnetic storage devices, or any other medium capable of storing desired program code in the form of instructions or data structures and that can be accessed by a computer, but is not limited to this. The memory can exist independently, and is connected to the processor through the bus. The memory can also be integrated with the processor.

[0141] The memory 803 is configured to store application program codes for implementing the solutions of the present application, and the processor 801 is configured to control the execution of the application program codes. The processor 801 is configured to execute the application program codes stored in the memory 803, so as to implement the functions in the method of the present application.

[0142] In a specific implementation, as an example, the processor 801 can include one or more CPUs, such as CPU0 and CPU1 in FIG. 8.

[0143] In a specific implementation, as an example, the communication device 800 can include multiple processors, such as the processor 801 and the processor 807 in FIG. 8. Each of the processors can be a single-CPU processor or a multi-CPU processor. The processor herein can refer to one or more devices, circuits, and / or processing cores for processing data (for example, computer program instructions).

[0144] It should be understood that the communication apparatuses or communication devices in various product forms described above each have any function implemented by the execution subject in the method embodiments, which will not be described herein.

[0145] The chip system provided by the embodiment of the present application comprises a processor and an interface circuit. The interface circuit is configured to receive instructions and transmit the instructions to the processor. The processor, for example, can be a specific implementation form of the packet processing apparatus in the embodiment of the present application, and can be configured to execute the method 100 or the method 200. The processor is coupled with a memory, and the memory is configured to store programs or instructions. When the programs or instructions are executed by the processor, the chip system implements the method in any method embodiment.

[0146] Optionally, the processor in the chip system can be one or more. The processor can be implemented by hardware or software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, or the like. When implemented by software, the processor can be a general-purpose processor, and the processor is implemented by reading software codes stored in the memory.

[0147] Optionally, the memory in the chip system can also be one or more. The memory can be integrated with the processor or arranged separately from the processor, which is not limited in the present application. For example, the memory can be a non-transient processor, for example, a read-only memory (ROM), which can be integrated on the same chip as the processor or arranged on different chips respectively, and the type of the memory and the arrangement manner of the memory and the processor are not limited in the present application.

[0148] For example, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a micro controller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0149] In addition, the embodiment of the present application further provides a storage medium, and the storage medium stores program codes or instructions. When the program codes or instructions are run on the processor, the processor executes the method in any implementation manner in the above embodiment.

[0150] In addition, the embodiment of the present application further provides a program product, when the program product is executed on a processor, the processor is caused to execute the method in any one of the implementation manners of the foregoing method 100 or method 200.

[0151] It should be understood that the "determining B according to A" mentioned in the embodiment of the present application does not mean that B is determined only according to A, but B can also be determined according to A and / or other information.

[0152] It should be understood that the network architecture and the service scenario described in the embodiment of the present application are for more clearly illustrating the technical solutions of the embodiment of the present application, and do not constitute a limitation on the technical solutions provided by the embodiment of the present application. It can be known by those skilled in the art that, with the evolution of network architecture and the appearance of new service scenarios, the technical solutions provided by the embodiment of the present application are also applicable to similar technical problems.

[0153] The ordinal numbers "1", "2", "3", "first", "second", and "third" and the like in the present application are used to distinguish a plurality of objects, and are not used to limit the order of the plurality of objects.

[0154] The "A and / or B" mentioned in the present application should be understood as including the following cases: only A, only B, or both A and B.

[0155] From the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the foregoing embodiment methods can be implemented by means of software plus a general hardware platform. Based on such understanding, the technical solutions of the present application can be embodied in the form of a software product. The computer software product can be stored in a storage medium, such as a read-only memory (English: read-only memory, ROM) / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network communication device such as a router) execute the methods described in the various embodiments or some parts of the embodiments of the present application.

[0156] The various embodiments described in this specification are described using a numbering of embodiments so that relative relationships can be described between the various embodiments. Each of the various embodiments were developed in response to one or more of the then-existing problems in the art and, as such, the one or more of the then-existing objects was addressed by one or more of the embodiments. However, the numbering of embodiments should not be construed as a limitation on the scope of the disclosure, but merely as an identification of embodiments deserved in this specification. Embodiments can include "step Sx" where x is a number, means that the claim can include, but does not require, the recited step. Functionality can be added or removed therefrom.

[0157] The above descriptions are only the preferred embodiments of the present application, not intended to limit the protection scope of the present application. It should be pointed out that, for those skilled in the art, without departing from the present application, a number of improvements and refinements can be made, which should also be considered as the protection scope of the present application.

Claims

1. A method for constructing a prediction model, characterized in that, The method comprises: obtaining a plurality of first training samples, each of the plurality of first training samples comprising a first multivariate time series and a second multivariate time series, the second multivariate time series being a multivariate time series collected after the first multivariate time series in the first training sample to which the second multivariate time series belongs; obtaining a prediction model based on the first multivariate time series, the second multivariate time series, an initial model, and an information bottleneck constraint condition, the initial model comprising a model for information interaction in at least one dimension, the information bottleneck constraint condition being used to suppress noise in the first multivariate time series, the prediction model being a trained model for predicting a multivariate time series after a known multivariate time series.

2. The method of claim 1, wherein, The information bottleneck constraint condition is also used to enhance learning of useful information in the first multivariate time series.

3. The method according to claim 1 or 2, characterized in that, The at least one dimension comprises a first dimension, and the obtaining of the prediction model based on the first multivariate time series, the second multivariate time series, the initial model, and the information bottleneck constraint condition comprises: N times of updating the initial model in the first dimension, N being an integer greater than or equal to 1.

4. The method of claim 3, wherein, In the N times of updating the initial model in the first dimension, a process of updating the initial model in the first dimension once comprises: inputting the first multivariate time series into the initial model to output a third multivariate time series, the initial model comprising a first encoding module and a first decoding module for information interaction in the first dimension, an output of the first encoding module being used as an input of the first decoding module; obtaining first mutual information according to the first multivariate time series and a first intermediate result, the first intermediate result being a high-dimensional mapping of the first multivariate time series in the first dimension, the first mutual information being used to indicate an influence of noise from other dimensions except the first dimension on a prediction process, and the first mutual information being also used to indicate useful information learned from other dimensions except the first dimension; obtaining second mutual information according to the second multivariate time series and the first intermediate result, the second mutual information being used to indicate a connection between a representation and a prediction; updating the initial model based on the first mutual information and the second mutual information to obtain an updated initial model; and updating the updated initial model based on the second multivariate time series and the third multivariate time series.

5. The method according to claim 3 or 4, characterized in that, The at least one dimension further comprises a second dimension, and the obtaining of the prediction model based on the first multivariate time series, the second multivariate time series, the initial model, and the information bottleneck constraint condition further comprises: M times of updating the initial model in the second dimension, M being an integer greater than or equal to 1.

6. The method of claim 5, wherein, In the M times of updating the initial model in the second dimension, a process of updating the initial model in the second dimension once comprises: inputting the first multivariate time series into the initial model to output a fourth multivariate time series; According to the first multi-dimensional time sequence and the second intermediate result, a third mutual information is obtained, the second intermediate result being a high-dimensional mapping of the first multi-dimensional time sequence in the second dimension, the third mutual information being used to indicate an influence of noise from each dimension on a prediction process, the each dimension including the second dimension; According to the second multi-dimensional time sequence and the second intermediate result, a fourth mutual information is obtained, the fourth mutual information being used to indicate a link between representation and prediction; Based on the third mutual information and the fourth mutual information, the initial model is updated to obtain an updated initial model; Based on the second multi-dimensional time sequence and the fourth multi-dimensional time sequence, the updated initial model is updated.

7. The method according to claim 5 or 6, characterized in that, The updating of the initial model in the second dimension is performed according to a preset rule, and a previously updated initial model is used as an initial model in a subsequent updating operation.

8. The method according to any one of claims 5-7, characterized in that, The first dimension is a variable dimension, a feature dimension or a channel dimension, and the second dimension is a time dimension or a frequency dimension.

9. The method of claim 1 or 2, wherein, The at least one dimension includes a second dimension, and the obtaining of the prediction model based on the first multi-dimensional time sequence, the second multi-dimensional time sequence, an initial model and an information bottleneck constraint condition includes: L times of updating of the initial model in the second dimension, L being an integer greater than or equal to 1.

10. The method according to any one of claims 1 to 9, characterized in that, The method further includes: Based on a fifth multi-dimensional time sequence and the prediction model, a sixth multi-dimensional time sequence is obtained, the sixth multi-dimensional time sequence being a prediction result of the prediction model on a multi-dimensional time sequence appearing after the fifth multi-dimensional time sequence.

11. The method of claim 10, wherein, The fifth multi-dimensional time sequence has the same size as the first multi-dimensional time sequence, and the sixth multi-dimensional time sequence has the same size as or different from the second multi-dimensional time sequence.

12. A method for invoking a prediction model, characterized in that, The method further includes: An initial model is obtained, the initial model being a model trained for prediction on a multi-dimensional time sequence, the initial model being constrained by an information bottleneck constraint condition in a training process of the initial model, the information bottleneck constraint condition being used to suppress noise of a multi-dimensional time sequence in a training sample; Based on the prediction model and a first multi-dimensional time sequence, a second multi-dimensional time sequence is obtained, the second multi-dimensional time sequence being a prediction result of the prediction model on a multi-dimensional time sequence appearing after the first multi-dimensional time sequence.

13. A device for constructing a prediction model, characterized by The method further includes: An obtaining unit is configured to obtain a plurality of first training samples, each of the plurality of first training samples including a first multi-dimensional time sequence and a second multi-dimensional time sequence, the second multi-dimensional time sequence being a multi-dimensional time sequence collected after the first multi-dimensional time sequence in the first training sample to which the second multi-dimensional time sequence belongs; The training unit is configured to obtain a prediction model based on the first multivariate time series, the second multivariate time series, an initial model and an information bottleneck constraint condition, the initial model including a model for information interaction in at least one dimension, the information bottleneck constraint condition being used to suppress noise in the first multivariate time series, and the prediction model being a trained model for predicting a multivariate time series after a known multivariate time series.

14. The apparatus of claim 13, wherein, The information bottleneck constraint condition is further used to enhance learning of useful information in the first multivariate time series.

15. The apparatus of claim 13 or 14, wherein, The at least one dimension includes a first dimension, and the training unit is specifically configured to perform N times of updating the initial model in the first dimension, N being an integer greater than or equal to 1.

16. The apparatus of claim 15, wherein, In the N times of updating the initial model in the first dimension, the process of updating the initial model in the first dimension once includes: inputting the first multivariate time series into the initial model to output a third multivariate time series; obtaining first mutual information according to the first multivariate time series and a first intermediate result, the first intermediate result being a high-dimensional mapping of the first multivariate time series in the first dimension, the first mutual information being used to indicate an influence of noise from other dimensions except the first dimension on a prediction process and being used to indicate useful information learned from other dimensions except the first dimension; obtaining second mutual information according to the second multivariate time series and the first intermediate result, the second mutual information being used to indicate a connection between a representation and a prediction; updating the initial model based on the first mutual information and the second mutual information to obtain an updated initial model; and updating the updated initial model based on the first multivariate time series and the third multivariate time series.

17. The apparatus of claim 15 or 16, wherein, The at least one dimension further includes a second dimension, and the training unit is further configured to perform M times of updating the initial model in the second dimension, M being an integer greater than or equal to 1.

18. The apparatus of claim 17, wherein, In the M times of updating the initial model in the second dimension, the process of updating the initial model in the second dimension once includes: inputting the first multivariate time series into the initial model to output a fourth multivariate time series; obtaining third mutual information according to the first multivariate time series and a second intermediate result, the second intermediate result being a high-dimensional mapping of the first multivariate time series in the second dimension, the third mutual information being used to indicate an influence of noise from each dimension, including the second dimension, on the prediction process; obtaining fourth mutual information according to the second multivariate time series and the second intermediate result, the fourth mutual information being used to indicate the connection between the representation and the prediction; updating the initial model based on the third mutual information and the fourth mutual information to obtain an updated initial model; and updating the updated initial model based on the first multivariate time series and the fourth multivariate time series.

19. The apparatus of claim 17 or 18, wherein, The updating of the initial model in the second dimension is performed alternately with the updating of the initial model in the first dimension according to a preset rule, and the initial model after the previous updating is used as the initial model in the subsequent updating.

20. The apparatus of any of claims 17-19, wherein, The first dimension is a feature dimension or a channel dimension, and the second dimension is a time dimension or a frequency dimension.

21. The apparatus of claim 13 or 14, wherein, The at least one dimension includes a second dimension, and the training unit is specifically configured to: perform L times of updating of the initial model in the second dimension, where L is an integer greater than or equal to 1.

22. The apparatus of any of claims 13-21, wherein, The device further includes: The prediction unit is configured to obtain a sixth multivariate time sequence based on a fifth multivariate time sequence and the prediction model, the sixth multivariate time sequence being a prediction result of the prediction model for a multivariate time sequence occurring after the fifth multivariate time sequence.

23. The apparatus of claim 22, wherein, The fifth multivariate time sequence has the same size as the first multivariate time sequence, and the sixth multivariate time sequence has the same size as or different from the second multivariate time sequence.

24. A device for invoking a prediction model, characterized in that, The device further includes: The obtaining unit is configured to obtain a prediction model, the prediction model being a trained model used for predicting a multivariate time sequence, and the prediction model being subjected to a constraint of an information bottleneck constraint condition in a training process of the prediction model, the information bottleneck constraint condition being used for suppressing noise of the multivariate time sequence in a training sample. The prediction unit is configured to obtain a second multivariate time sequence based on the prediction model and a first multivariate time sequence, the second multivariate time sequence being a prediction result of the prediction model for a multivariate time sequence occurring after the first multivariate time sequence.

25. A communications device, characterized by The communication device includes a memory and a processor; The memory is configured to store instructions; The processor is configured to execute the instructions in the memory, and execute the method in any one of claims 1-12.

26. A storage medium characterized by The storage medium includes instructions, when the instructions run on the processor, cause the processor to execute the method in any one of the above claims 1-12. The storage medium includes instructions, when the instructions run on the processor, cause the processor to execute the method in any one of the above claims 1-12.

Images