Long time sequence prediction method and device based on linear neural network

Abstract

The invention provides a long time sequence prediction method and device based on a linear neural network, and the method comprises the steps: collecting time sequence data under at least one channel related to a to-be-predicted target; generating a trend sequence and a residual sequence under each channel according to the time sequence data under each channel, dividing the trend sequence into a plurality of trend subsequences with the same length, and dividing the residual sequence into a plurality of residual subsequences with the same length; and inputting the trend sequence, the trend sub-sequence, the residual sequence and the residual sub-sequence under the at least one channel into a pre-trained long time sequence prediction model based on a linear neural network to obtain a prediction result of the to-be-predicted target generated by the long time sequence prediction model. According to the invention, long-time-sequence prediction service can be provided for various fields. In addition, the trend characteristics of the time sequence are considered, and the accuracy of model prediction can be improved.

Description

Technical Field

[0001] The present application relates to the field of neural network technology, and in particular to a long time series prediction method and device based on a linear neural network. Background Art

[0002] Long-term time series forecasting (LTSF) technology has applications in many fields, such as building energy consumption forecasting, traffic flow forecasting, weather forecasting, product and parts demand forecasting in the manufacturing sector, and even price forecasting in the financial sector. The linear long-term time series forecasting (LTSF-Linear) model has attracted widespread attention due to its low time complexity (O(L)) and compact model. Existing time series forecasting methods are mostly based on the characteristics of data in a specific field. The lack of a universal method for extracting data features makes the data features used by various methods incompatible, making these long-term time series forecasting technologies incompatible across fields.

[0003] For example, the Chinese patent application with publication number CN117436325A discloses a method for predicting air conditioning energy consumption, which can reduce related energy consumption, reduce energy consumption costs and improve energy utilization efficiency. The Chinese patent application with publication number CN112150209A discloses a drug sales forecasting model, which provides a reference for the purchase volume of medicinal materials, avoids the loss of medicinal properties of medicinal materials due to long-term storage and saves money. The forecasting schemes of the above-mentioned prior art are all targeted at specific fields. For example, CN117436325A is targeted at energy consumption forecasting, and CN112150209A is used for drug sales forecasting. They each need to perform feature extraction within the field before prediction, so the forecasting model lacks versatility. In addition, none of the above-mentioned prior art processes the trend characteristics of time series. Summary of the Invention

[0004] At least one embodiment of the present application provides a long time series prediction method and device based on a linear neural network, which implements a long time series prediction solution that can be applied to various fields and can improve the prediction performance of the model.

[0005] According to the first aspect of the present application, at least one embodiment provides a long time series prediction method based on a linear neural network, comprising:

[0006] Collecting time series data of at least one channel related to the target to be predicted;

[0007] For the time series data of each channel, a trend sequence and a residual sequence are generated under each channel, the trend sequence is divided into multiple trend subsequences of the same length, and the residual sequence is divided into multiple residual subsequences of the same length;

[0008] Inputting the trend sequence, trend subsequence, residual sequence and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model;

[0009] The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer:

[0010] The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel;

[0011] The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

[0012] Optionally, when the number of the single-head prediction modules is 1, the prediction result of the single-head prediction module is the prediction result of the target to be predicted.

[0013] Optionally, when the number of the single-head prediction modules is greater than or equal to 2, the long time series prediction model further includes a gateway layer and a fusion module;

[0014] The gateway layer is used to generate the weight of each single-head prediction module according to the timestamp characteristics of the time series data;

[0015] The fusion module is used to fuse the prediction results of the at least one single-head prediction module according to the weight of each single-head prediction module to obtain the prediction result of the target to be predicted.

[0016] Optionally, for the time series data of each channel, generate the trend series and residual series under each channel, including:

[0017] Using a moving average method, processing the time series data to obtain the trend sequence;

[0018] The difference between the time series data and the trend series is calculated to obtain the residual series.

[0019] Optionally, the above method further includes:

[0020] The steps of pre-training the long time series prediction model specifically include:

[0021] Acquire a training set, wherein the training set includes historical time series samples under the at least one channel and true values ​​corresponding to the historical time series samples;

[0022] Generate a historical trend sequence and a historical residual sequence for each channel based on the historical time series samples for each channel, split the historical trend sequence into multiple historical trend subsequences of the same length, and split the historical residual sequence into multiple historical residual subsequences of the same length;

[0023] Inputting the historical trend sequence, historical trend subsequence, historical residual sequence and historical residual subsequence under the at least one channel into the long time series prediction model to obtain a predicted value of the target to be predicted generated by the long time series prediction model;

[0024] A loss function is constructed based on the difference between the predicted value of the target to be predicted and the true value corresponding to the historical time series sample; the long time series prediction model is trained by backpropagation of the loss function to obtain a trained long time series prediction model.

[0025] According to the second aspect of the present application, at least one embodiment provides a long time series prediction device based on a linear neural network, comprising:

[0026] A data acquisition module, used to collect time series data under at least one channel related to the target to be predicted;

[0027] A sequence processing module is used to generate a trend sequence and a residual sequence under each channel for the time series data of each channel, split the trend sequence into multiple trend subsequences of the same length, and split the residual sequence into multiple residual subsequences of the same length;

[0028] A prediction module is configured to input the trend sequence, trend subsequence, residual sequence, and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model;

[0029] The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer:

[0030] The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel;

[0031] The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

[0032] Optionally, when the number of the single-head prediction modules is 1, the prediction result of the single-head prediction module is the prediction result of the target to be predicted.

[0033] Optionally, when the number of the single-head prediction modules is greater than or equal to 2, the long time series prediction model further includes a gateway layer and a fusion module;

[0034] The gateway layer is used to generate the weight of each single-head prediction module according to the timestamp characteristics of the time series data;

[0035] The fusion module is used to fuse the prediction results of the at least one single-head prediction module according to the weight of each single-head prediction module to obtain the prediction result of the target to be predicted.

[0036] Optionally, the sequence processing module is further configured to:

[0037] Using a moving average method, processing the time series data to obtain the trend sequence;

[0038] The difference between the time series data and the trend series is calculated to obtain the residual series.

[0039] Optionally, the above device further includes:

[0040] The training module is used to pre-train the long time series prediction model, specifically including:

[0041] Acquire a training set, wherein the training set includes historical time series samples under the at least one channel and true values ​​corresponding to the historical time series samples;

[0042] Generate a historical trend sequence and a historical residual sequence for each channel based on the historical time series samples for each channel, split the historical trend sequence into multiple historical trend subsequences of the same length, and split the historical residual sequence into multiple historical residual subsequences of the same length;

[0043] Inputting the historical trend sequence, historical trend subsequence, historical residual sequence and historical residual subsequence under the at least one channel into the long time series prediction model to obtain a predicted value of the target to be predicted generated by the long time series prediction model;

[0044] A loss function is constructed based on the difference between the predicted value of the target to be predicted and the true value corresponding to the historical time series sample; the long time series prediction model is trained by backpropagation of the loss function to obtain a trained long time series prediction model.

[0045] According to the third aspect of the present application, at least one embodiment provides a long time series prediction device based on a linear neural network, comprising a processor, a memory, and a program or instruction stored on the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method described in any one of the first aspects.

[0046] According to the fourth aspect of the present application, at least one embodiment provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the steps of any method of the first aspect are implemented.

[0047] According to a fifth aspect of the present application, at least one embodiment provides a computer program product, comprising computer instructions, which, when executed by a processor, implement the steps of the method described in any one of the first aspects.

[0048] Compared with the prior art, the long time series prediction method and device based on linear neural network provided by the embodiment of the present application can provide long time series prediction services for a variety of fields. In addition, the embodiment of the present application takes into account the trend characteristics of the time series, which is conducive to improving the accuracy of model prediction. The embodiment of the present application can provide long time series prediction services based on historical data in respective application scenarios without doing feature engineering. Such as predicting building power consumption, predicting traffic flow, predicting weather, predicting the number of influenza patients, etc. The embodiment of the present application can be used for prediction tasks of various management systems in smart cities, such as energy consumption prediction services for office buildings, traffic flow prediction on roads, future weather forecasts, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0050] Figure 1A schematic diagram of a flow chart of a long time series prediction method according to an embodiment of the present application;

[0051] Figure 2 A schematic diagram of the structure of a long time series prediction model according to an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of a single-head prediction module according to an embodiment of the present application;

[0053] Figure 4 Another structural diagram of the long time series prediction model according to an embodiment of the present application;

[0054] Figure 5 This is an example diagram of a process for training a long time series prediction model according to an embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a long time series prediction device according to an embodiment of the present application;

[0056] Figure 7 This is another structural diagram of the long time series prediction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0058] It should be understood that references throughout this specification to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic associated with the embodiment is included in at least one embodiment of the present application. Therefore, the appearance of "in one embodiment" or "in an embodiment" throughout this specification does not necessarily refer to the same embodiment. Furthermore, these particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. The terms "first," "second," and so on, used in the specification and claims of this application are used to distinguish similar items and are not necessarily used to describe a particular order or sequential sequence. It should be understood that such usage is interchangeable where appropriate, such that the embodiments of the present application described herein can, for example, be implemented in an order other than that illustrated or described herein. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, product, or apparatus. The term "and / or" used in the specification and claims refers to at least one of the connected items.

[0059] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0060] The following description provides examples and does not limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. The various examples may appropriately omit, substitute, or add various procedures or components. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. In addition, features described with reference to certain examples may be combined in other examples.

[0061] To address the poor versatility of existing long-term time series prediction solutions, the present invention provides a linear neural network-based long-term time series method based on the LTSF-Linear model, which can be used as an embedding layer to extract features from various data types. This method constructs a linear neural network structure (prediction model) that can provide long-term time series prediction services for a variety of fields. In addition, this method takes into account the trend characteristics of the time series, which is conducive to improving the accuracy of the model prediction.

[0062] like Figure 1 As shown, the long time series method based on linear neural network provided in the embodiment of the present application includes the following steps:

[0063] Step 11: Collect time series data under at least one channel related to the target to be predicted.

[0064] Here, the target to be predicted can be a target in various fields, for example, it can be a building energy consumption forecast, traffic flow forecast, weather forecast, demand forecast for products and parts in the manufacturing field, or even price forecast in the financial field, etc., and the embodiments of the present application do not specifically limit this. In order to predict the target to be predicted, it is usually necessary to collect time series data under at least one channel related to the target to be predicted, thereby obtaining time series data under at least one channel. Each channel corresponds to a data type.

[0065] Step 12: for the time series data of each channel, generate a trend sequence and a residual sequence under each channel, divide the trend sequence into multiple trend subsequences of the same length, and divide the residual sequence into multiple residual subsequences of the same length.

[0066] Here, for each channel, the embodiment of the present application calculates the trend sequence and residual sequence for that channel based on the time series data for that channel. Specifically, the time series data can be processed using a moving average method to obtain the trend sequence, and then the difference between the time series data and the trend sequence can be calculated to obtain the residual sequence.

[0067] For example, when extracting trend series and residual series from original time series data, the moving average method is used to extract the trend. The residual is obtained after subtracting the trend from the original time series data, which includes some periodic data and residuals.

[0068] The core idea of ​​the moving average method is to use the average value of local data to represent the trend of the overall data, thereby eliminating noise and fluctuations in the data and better reflecting the long-term trend of the data. Suppose a time series data x1, x2, ..., x n , the moving average method can extract trends by calculating the moving average of a window size of m, where m represents the length of the window. The mathematical expression of the moving average is:

[0069]

[0070] Among them, MA t represents the moving average at time t, x i Represents the data value of the original time series data at time i. When calculating the moving average, it is necessary to slide the data according to the window size m to smooth the data and extract trend information.

[0071] Trend sequences and residual sequences are also a type of time series data. After obtaining the trend sequence and residual sequence for each channel, the trend sequence is further divided into multiple trend subsequences of equal length, and the residual sequence is further divided into multiple residual subsequences of equal length. For example, the sequence may be divided into K trend subsequences and K residual subsequences. K is an integer greater than or equal to 2. Assuming the length of the time series data for each channel is 336, the length of each subsequence is 336 / K.

[0072] Step 13: Input the trend sequence, trend subsequence, residual sequence and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model.

[0073] Here, a long time series prediction model based on a linear neural network is pre-trained. In step 13, the trend sequence, trend subsequence, residual sequence, and residual subsequence under the at least one channel obtained in step 12 are input into the long time series prediction model, and the long time series prediction model generates a prediction result for the template to be predicted.

[0074] In an embodiment of the present application, the long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted. Each single-head prediction module includes a linear embedding layer and a linear prediction layer, wherein:

[0075] The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel.

[0076] For example, the linear embedding layer generates the features of the trend sequence, trend subsequence, residual sequence, and residual subsequence for channel 1. The features of the trend sequence and each trend subsequence for channel 1 are then fused to obtain the trend fusion feature for channel 1. The features of the residual sequence and each residual subsequence for channel 1 are fused to obtain the residual fusion feature for channel 1. Similarly, the above process is performed for channels 2, 3, ..., and M, respectively, to obtain the trend fusion feature and residual fusion feature for each channel. Here, it is assumed that there are M channels.

[0077] The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

[0078] For example, the information linear prediction layer generates a trend prediction result for channel 1 based on the trend fusion features of channel 1, and generates a residual prediction result for channel 1 based on the residual fusion features of channel 1. Similarly, the above processing is performed for channels 2, 3, ..., and M, respectively, to obtain the trend fusion features and residual fusion features for each channel. Then, the trend prediction results and residual prediction results of channels 1, 2, 3, ..., and M are fused to obtain the prediction result of the single-head prediction module.

[0079] In the embodiment of the present application, when the number of single-head prediction modules in the long time series prediction model is 1, the prediction result of the single-head prediction module is the prediction result of the target to be predicted. At this time, the structure of the long time series prediction model is as follows: Figure 2 shown. Figure 2 This is a schematic diagram of the structure of a long-term time series prediction model that includes only one single-head prediction module. This single-head prediction module generates prediction results based on time series data (sequence 1, sequence 2, ..., sequence M) from at least one input channel. In addition, the prediction results can be further processed by a post-processing module to generate output for specific scenarios. In this case, the long-term time series prediction model can also include a post-processing module.

[0080] Figure 3 A schematic diagram of the single-head prediction module is provided. Here, the trend sequence and residual sequence data under each channel are divided into K subsequences. Assume that the time series data under each channel is 336. Figure 3In the example, sequence 1, sequence 2, sequence 3, ..., sequence M are the time series data of channel 1, channel 2, channel 3, ..., and channel M, respectively. First, the sequence processing module extracts the trend sequence and residual sequence from sequence 1, sequence 2, sequence 3, ..., and sequence M (S31). The trend sequence and residual sequence are then divided into K subsequences (S32). Each subsequence then passes through an embedding module (e.g., embedding 1, embedding 2, ..., embedding K) in the linear embedding layer to generate subsequence features (S33). Similarly, the trend sequence also passes through embedding module K+1 (e.g., embedding K+1) to generate trend sequence features, and the residual sequence also passes through embedding module K+1 to generate residual sequence features (S34). Here, the feature lengths generated by each embedding module are the same as those of the input sequence. Then, according to the order of the trend subsequences in the trend sequence, the features of all trend subsequences under the same channel are spliced ​​together, and then spliced ​​with the features of the trend sequence to obtain a trend fusion feature with a length of 2*336; according to the order of the residual subsequences in the residual sequence, the features of all residual subsequences under the same channel are spliced ​​together, and then spliced ​​with the features of the residual sequence to obtain a residual fusion feature with a length of 2*336 (S35). The trend fusion feature under each channel is input into the linear prediction layer to generate the trend prediction result under the channel; the residual fusion feature under each channel is input into the linear prediction layer to generate the residual prediction result under the channel. Then, the trend prediction results and residual prediction results under all channels are fused to obtain the prediction result of the single-head prediction module (S36-S38). During the training process, by comparing the prediction result with the true value, a loss function is constructed based on the difference between the two, and the network parameters are updated through gradient backpropagation (S39), for example, the parameters of the linear embedding layer and the linear prediction layer are updated.

[0081] When using a long-term prediction model that includes a single-head prediction module for prediction, once the long-term prediction model is trained, it can be used for prediction. During prediction, the time series data is input into the long-term prediction model, along with the number of time steps you wish to predict. The model then outputs a result, for example, predicting values ​​for 96, 192, 336, or 720 time steps into the future.

[0082] In the embodiment of the present application, the linear embedding layer and the linear prediction layer are used to extract features from the time series data. A mathematical expression is as follows:

[0083] Y=A i X+b i

[0084] Among them, A i represents the weight of linear layer i, bi Denotes the bias of linear layer i, and Y denotes the output of linear layer i. Linear layer i can be the embedding module in the above-mentioned linear embedding layer or the above-mentioned linear prediction layer.

[0085] In order to improve the prediction accuracy of the model, the embodiment of the present application can combine multiple single-head prediction modules to form a multi-head model. Figure 4 A schematic diagram of the structure of a long time series prediction model including multiple single-head prediction modules is provided. In the embodiment of the present application, when the number of the single-head prediction modules is greater than or equal to 2, the long time series prediction model may further include a gateway layer and a fusion module. The structure of each single-head prediction module can be referred to Figure 3 shown.

[0086] Here, the gateway layer is used to generate the weight of each single-head prediction module based on the timestamp feature of the time series data. The timestamp feature is used to represent the temporal position in a time period, for example, an hour in a natural day (hour / day), a day in a week (day / week), a day in a month (day / month), a day in a year (day / year), a month in a year (month / year), a week in a year (week / year), etc. The timestamp feature is scaled to a preset range after regularization calculation. The timestamp feature reflects the periodicity of the data.

[0087] The fusion module is used to fuse the prediction results of the at least one single-head prediction module according to the weight of each single-head prediction module to obtain the prediction result of the target to be predicted.

[0088] The timestamp features passed through the gateway layer generate a weight for each single-head prediction module. The gateway layer is an allocator that dynamically decides which single-head prediction module to assign it to based on the characteristics of the input sample.

[0089] The prediction results generated by each head according to the weight are added together to generate the final prediction result. Let G(x) i and E i (x) are the outputs of the gateway layer and the i-th single-head prediction module, respectively. Then for the current input x, the output is the weighted sum of all single-head prediction modules. Here, it is assumed that there are n single-head prediction modules:

[0090]

[0091] Similarly, when using a long-term prediction model that includes multiple single-head prediction modules for prediction, after the long-term prediction model is trained, it can be used for prediction. During prediction, the time series data is input into the long-term prediction model, and the number of time steps to be predicted can also be input. The long-term prediction model will then output a result, for example, predicting the value for the next 96, 192, 336, or 720 time steps.

[0092] The following describes the training process of the long time series prediction model in the embodiment of the present application. Figure 5 As shown, the following steps are included.

[0093] Step 51: Acquire a training set, where the training set includes historical time series samples under the at least one channel and true values ​​corresponding to the historical time series samples.

[0094] Step 52: Generate a historical trend sequence and a historical residual sequence for each channel based on the historical time series samples for each channel, split the historical trend sequence into multiple historical trend subsequences of the same length, and split the historical residual sequence into multiple historical residual subsequences of the same length.

[0095] Step 53: Input the historical trend sequence, historical trend subsequence, historical residual sequence and historical residual subsequence under the at least one channel into the long time series prediction model to obtain the predicted value of the target to be predicted generated by the long time series prediction model.

[0096] Step 54: construct a loss function based on the difference between the predicted value of the target to be predicted and the true value corresponding to the historical time series sample.

[0097] Step 55, using the loss function to back-propagate and train the long time series prediction model, updating the channels in the linear embedding layer and the linear prediction layer until the training end condition is met, thereby obtaining a trained long time series prediction model.

[0098] From the above, it can be seen that the embodiment of the present application can provide long-term prediction services for a variety of fields by constructing a linear neural network structure (long-term prediction model). In addition, the embodiment of the present application takes into account the trend characteristics of the time series, which is conducive to improving the accuracy of model prediction. The embodiment of the present application can provide long-term prediction services based on historical data in respective application scenarios without doing feature engineering. Such as predicting building power consumption, predicting traffic flow, predicting weather, predicting the number of influenza patients, etc. The embodiment of the present application can be used for prediction tasks of various management systems in smart cities, such as energy consumption prediction services for office buildings, traffic flow prediction on roads, future weather forecasts, etc.

[0099] Please refer to Figure 6, a structure of a long time series prediction device based on a linear neural network provided in an embodiment of the present application includes:

[0100] A data acquisition module 61 is used to acquire time series data under at least one channel related to the target to be predicted;

[0101] A sequence processing module 62 is configured to generate a trend sequence and a residual sequence for each channel for the time series data of each channel, and to divide the trend sequence into a plurality of trend subsequences of the same length, and to divide the residual sequence into a plurality of residual subsequences of the same length;

[0102] A prediction module 63 is configured to input the trend sequence, trend subsequence, residual sequence, and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model;

[0103] The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer:

[0104] The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel;

[0105] The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

[0106] Optionally, when the number of the single-head prediction modules is 1, the prediction result of the single-head prediction module is the prediction result of the target to be predicted.

[0107] Optionally, when the number of the single-head prediction modules is greater than or equal to 2, the long time series prediction model further includes a gateway layer and a fusion module;

[0108] The gateway layer is used to generate the weight of each single-head prediction module according to the timestamp characteristics of the time series data;

[0109] The fusion module is used to fuse the prediction results of the at least one single-head prediction module according to the weight of each single-head prediction module to obtain the prediction result of the target to be predicted.

[0110] Optionally, the sequence processing module is further configured to:

[0111] Using a moving average method, processing the time series data to obtain the trend sequence;

[0112] The difference between the time series data and the trend series is calculated to obtain the residual series.

[0113] Optionally, the long time series prediction device further includes:

[0114] The training module is used to pre-train the long time series prediction model, specifically including:

[0115] Acquire a training set, wherein the training set includes historical time series samples under the at least one channel and true values ​​corresponding to the historical time series samples;

[0116] Generate a historical trend sequence and a historical residual sequence for each channel based on the historical time series samples for each channel, split the historical trend sequence into multiple historical trend subsequences of the same length, and split the historical residual sequence into multiple historical residual subsequences of the same length;

[0117] Inputting the historical trend sequence, historical trend subsequence, historical residual sequence and historical residual subsequence under the at least one channel into the long time series prediction model to obtain a predicted value of the target to be predicted generated by the long time series prediction model;

[0118] A loss function is constructed based on the difference between the predicted value of the target to be predicted and the true value corresponding to the historical time series sample; the long time series prediction model is trained by backpropagation of the loss function to obtain a trained long time series prediction model.

[0119] It should be noted that the various systems provided in the above embodiments are devices corresponding to the above-mentioned long time series prediction method based on linear neural network. The implementation methods in the above-mentioned embodiments are applicable to the embodiments of the device and can achieve the same technical effects. The above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0120] Please refer to Figure 7, a structural diagram of another long time series prediction device based on a linear neural network provided in an embodiment of the present application, the device includes: a processor 701, a transceiver 702, a memory 703, a user interface 704 and a bus interface.

[0121] In the embodiment of the present application, the device further includes: a program stored in the memory 703 and executable on the processor 701 .

[0122] The transceiver 702 is configured to send and receive data under the control of the processor;

[0123] The processor 701 is configured to read the computer program in the memory and perform the following operations:

[0124] Collecting time series data of at least one channel related to the target to be predicted;

[0125] For the time series data of each channel, a trend sequence and a residual sequence are generated under each channel, the trend sequence is divided into multiple trend subsequences of the same length, and the residual sequence is divided into multiple residual subsequences of the same length;

[0126] Inputting the trend sequence, trend subsequence, residual sequence and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model;

[0127] The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer:

[0128] The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel;

[0129] The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

[0130] It can be understood that in the embodiment of the present application, when the computer program is executed by the processor 701, each process of the above-mentioned embodiment of the long time series prediction method based on linear neural network can be implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0131] exist Figure 7 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 701 and memory represented by memory 703. The bus architecture may also link together various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 702 may be a plurality of components, i.e., a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium. For different user devices, the user interface 704 may also be an interface capable of connecting external or internal devices as required, and the connected devices include but are not limited to a keypad, a display, a speaker, a microphone, a joystick, etc.

[0132] The processor 701 is responsible for managing the bus architecture and general processing, and the memory 703 can store data used by the processor 701 when performing operations.

[0133] It should be noted that the device in this embodiment is a device corresponding to the above-mentioned long time series prediction method based on linear neural network, and the implementation methods in the above-mentioned embodiments are all applicable to the embodiments of the device, and can also achieve the same technical effects. In the device, the transceiver 702 and the memory 703, as well as the transceiver 702 and the processor 701 can be connected through a bus interface communication, the function of the processor 701 can also be implemented by the transceiver 702, and the function of the transceiver 702 can also be implemented by the processor 701. It should be noted that the above-mentioned device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned method embodiment, and can achieve the same technical effects. The parts and beneficial effects that are the same as those in the method embodiment will not be specifically described here.

[0134] In some embodiments of the present application, a computer-readable storage medium is further provided, on which a program is stored. When the program is executed by a processor, the following steps are implemented:

[0135] Collecting time series data of at least one channel related to the target to be predicted;

[0136] For the time series data of each channel, a trend sequence and a residual sequence are generated under each channel, the trend sequence is divided into multiple trend subsequences of the same length, and the residual sequence is divided into multiple residual subsequences of the same length;

[0137] Inputting the trend sequence, trend subsequence, residual sequence and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model;

[0138] The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer:

[0139] The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel;

[0140] The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

[0141] When this program is executed by the processor, it can implement all the implementation methods of the above-mentioned long time series prediction method based on linear neural network, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0142] An embodiment of the present application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the various processes of the above-mentioned long time series prediction method embodiment based on a linear neural network are implemented, and the same technical effect can be achieved. To avoid repetition, they will not be repeated here.

[0143] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0144] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0145] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0146] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0147] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0148] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, or an optical disk.

[0149] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A long time series prediction method based on linear neural network, characterized in that: include: Collecting time series data of at least one channel related to the target to be predicted; For the time series data of each channel, a trend sequence and a residual sequence are generated under each channel, the trend sequence is divided into multiple trend subsequences of the same length, and the residual sequence is divided into multiple residual subsequences of the same length; Inputting the trend sequence, trend subsequence, residual sequence and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model; The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer: The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel; The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

2. The method according to claim 1, wherein When the number of the single-head prediction module is 1, the prediction result of the single-head prediction module is the prediction result of the target to be predicted.

3. The method according to claim 1, wherein When the number of the single-head prediction modules is greater than or equal to 2, the long time series prediction model further includes a gateway layer and a fusion module; The gateway layer is used to generate the weight of each single-head prediction module according to the timestamp characteristics of the time series data; The fusion module is used to fuse the prediction results of the at least one single-head prediction module according to the weight of each single-head prediction module to obtain the prediction result of the target to be predicted.

4. The method according to claim 1, wherein For the time series data of each channel, generate the trend sequence and residual sequence under each channel, including: Using a moving average method, processing the time series data to obtain the trend sequence; The difference between the time series data and the trend series is calculated to obtain the residual series.

5. The method according to claim 1, wherein Also includes: The steps of pre-training the long time series prediction model specifically include: Acquire a training set, wherein the training set includes historical time series samples under the at least one channel and true values ​​corresponding to the historical time series samples; Generate a historical trend sequence and a historical residual sequence for each channel based on the historical time series samples for each channel, split the historical trend sequence into multiple historical trend subsequences of the same length, and split the historical residual sequence into multiple historical residual subsequences of the same length; Inputting the historical trend sequence, historical trend subsequence, historical residual sequence and historical residual subsequence under the at least one channel into the long time series prediction model to obtain a predicted value of the target to be predicted generated by the long time series prediction model; A loss function is constructed based on the difference between the predicted value of the target to be predicted and the true value corresponding to the historical time series sample; the long time series prediction model is trained by backpropagation of the loss function to obtain a trained long time series prediction model.

6. A long time series prediction device based on linear neural network, characterized in that: include: A data acquisition module, used to collect time series data under at least one channel related to the target to be predicted; A sequence processing module is used to generate a trend sequence and a residual sequence under each channel for the time series data of each channel, split the trend sequence into multiple trend subsequences of the same length, and split the residual sequence into multiple residual subsequences of the same length; A prediction module is configured to input the trend sequence, trend subsequence, residual sequence, and residual subsequence under the at least one channel into a pre-trained long time series prediction model based on a linear neural network to obtain a prediction result of the target to be predicted generated by the long time series prediction model; The long time series prediction model includes at least one single-head prediction module, and the prediction results of the at least one single-head prediction module are fused to obtain the prediction result of the target to be predicted; wherein each single-head prediction module includes a linear embedding layer and a linear prediction layer: The linear embedding layer is used to generate the features of the trend sequence, trend subsequence, residual sequence and residual subsequence under each channel respectively; the features of the trend sequence and each trend subsequence under the same channel are fused respectively to obtain the trend fusion feature under each channel; the features of the residual sequence and each residual subsequence under the same channel are fused respectively to obtain the residual fusion feature under each channel; The linear prediction layer is used to generate a trend prediction result under each channel according to the trend fusion features under each channel; generate a residual prediction result under each channel according to the residual fusion features under each channel; and fuse the trend prediction results and residual prediction results under all channels to obtain the prediction result of the single-head prediction module.

7. The device according to claim 6, characterized in that When the number of the single-head prediction module is 1, the prediction result of the single-head prediction module is the prediction result of the target to be predicted.

8. The device according to claim 6, wherein When the number of the single-head prediction modules is greater than or equal to 2, the long time series prediction model further includes a gateway layer and a fusion module; The gateway layer is used to generate the weight of each single-head prediction module according to the timestamp characteristics of the time series data; The fusion module is used to fuse the prediction results of the at least one single-head prediction module according to the weight of each single-head prediction module to obtain the prediction result of the target to be predicted.

9. The device according to claim 6, wherein The sequence processing module is further configured to: Using a moving average method, processing the time series data to obtain the trend sequence; The difference between the time series data and the trend series is calculated to obtain the residual series.

10. The device according to claim 6, wherein Also includes: The training module is used to pre-train the long time series prediction model, specifically including: Acquire a training set, wherein the training set includes historical time series samples under the at least one channel and true values ​​corresponding to the historical time series samples; Generate a historical trend sequence and a historical residual sequence for each channel based on the historical time series samples for each channel, split the historical trend sequence into multiple historical trend subsequences of the same length, and split the historical residual sequence into multiple historical residual subsequences of the same length; Inputting the historical trend sequence, historical trend subsequence, historical residual sequence and historical residual subsequence under the at least one channel into the long time series prediction model to obtain a predicted value of the target to be predicted generated by the long time series prediction model; A loss function is constructed based on the difference between the predicted value of the target to be predicted and the true value corresponding to the historical time series sample; the long time series prediction model is trained by backpropagation of the loss function to obtain a trained long time series prediction model.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the steps of the method according to any one of claims 1 to 5.

Images