Prediction method and device of time series data, electronic equipment and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a time sequence data prediction method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring to-be-predicted time series data; inputting to-be-predicted time sequence data into the target prediction model, capturing a time sequence relationship of the time sequence data by using the time sequence capturing module, and performing feature transformation on output of the time sequence capturing module by using the feature transformation module, performing residual connection on the output of the feature transformation module and the input of the time sequence capture module by using a residual module, and taking a residual connection result as a target result output by the target prediction model; and obtaining a target result output by the target prediction model, and determining a prediction result corresponding to the to-be-predicted time sequence data according to the target result. The accuracy of the time sequence data prediction result can be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and storage medium for predicting time series data. Background Art

[0002] With the development of computer technology, the use of time series prediction methods to predict the development trend of future things has provided convenience for people's work and life. Time series prediction refers to the compilation and analysis of time series, and the analogy or extension based on the development process, direction and trend reflected in the time series, so as to predict the level that may be achieved at the next time point or the next period of time, such as the prediction of the cooling capacity of shopping mall refrigeration stations and the prediction of power grid load. Although the existing technology has achieved good results in the prediction of time series, there is still the problem of inaccurate prediction. Summary of the invention

[0003] In view of this, embodiments of the present application provide a method, device, electronic device and storage medium for predicting time series data to solve the problem of inaccurate prediction results in the prior art for predicting time series data.

[0004] In a first aspect of an embodiment of the present application, a method for predicting time series data is provided, the method comprising: Acquire the time series data to be predicted; input the time series data to be predicted into the target prediction model, the target prediction model includes a time series capture module, a feature transformation module and a residual module, the time series capture module is used to capture the time series relationship of the time series data to be predicted, the feature transformation module is used to perform feature transformation on the output of the time series capture module, the residual module is used to perform residual connection between the output of the feature transformation module and the input of the time series capture module, and use the result of the residual connection as the target result output by the target prediction model; obtain the target result output by the target prediction model, and determine the prediction result corresponding to the time series data to be predicted according to the target result.

[0005] According to a second aspect of an embodiment of the present application, a device for predicting time series data is provided, the device comprising: An acquisition module is configured to acquire time series data to be predicted; an input module is configured to input the time series data to be predicted into a target prediction model, the target prediction model includes a time series capture module, a feature transformation module and a residual module, the time series capture module is used to capture the time series relationship of the time series data to be predicted, the feature transformation module is used to perform feature transformation on the output of the time series capture module, the residual module is used to perform residual connection between the output of the feature transformation module and the input of the time series capture module, and the result of the residual connection is used as the target result output by the target prediction model; a determination module is configured to acquire the target result output by the target prediction model, and determine the prediction result corresponding to the time series data to be predicted according to the target result.

[0006] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the computer program.

[0007] According to a fourth aspect of an embodiment of the present application, a readable storage medium is provided, which stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0008] Compared with the prior art, the embodiments of the present invention have the following beneficial effects: Obtain the time series data to be predicted; input the time series data to be predicted into the target prediction model, firstly extract and capture the feature information related to the time dimension from the time series data to be predicted through the time series capture module, so that the subsequent prediction can be more in line with the actual situation of the data, so as to improve the accuracy of the prediction, and secondly, through the transformation processing of the feature transformation module, the target prediction model can focus on more critical and effective information in the data, improve the fitting ability of complex data, and then improve the prediction effect, and further improve the prediction accuracy, and finally use the residual module to connect the transformed feature data output by the feature transformation module with the initial input data of the time series capture module, so as to integrate data information at different stages, so that the model can not only use the advanced features transformed by the feature transformation module, but also take into account some basic information in the original input data, so that the final prediction result integrates effective information at multiple levels, further improves the accuracy and reliability of the prediction result, and avoids the problem of inaccurate prediction results in the prediction of time series data in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0010] Figure 1 It is a flowchart of a method for predicting time series data provided by an embodiment of the present application; Figure 2 It is a structural schematic diagram of a time series data prediction device provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0012] A method, device, electronic device and storage medium for predicting time series data according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0013] Figure 1 FIG. 1 is a flow chart of a method for predicting time series data provided by an embodiment of the present application. Figure 1 As shown, the method includes: S101, obtaining time series data to be predicted; S102, inputting the time series data to be predicted into the target prediction model, the target prediction model includes a time series capture module, a feature transformation module and a residual module, the time series capture module is used to capture the time series relationship of the time series data to be predicted, the feature transformation module is used to perform feature transformation on the output of the time series capture module, the residual module is used to perform residual connection between the output of the feature transformation module and the input of the time series capture module, and the result of the residual connection is used as the target result output by the target prediction model; S103, obtaining a target result output by the target prediction model, and determining a prediction result corresponding to the time series data to be predicted according to the target result.

[0014] Specifically, the time series data prediction method of this embodiment can be executed by the client or the server, or can be executed by the client and the server together. The following description is made by taking the client as an example.

[0015] It can be understood that time series data forecasting is a method of inferring future data values ​​based on historical time series data. A time series is a sequence of observations arranged in chronological order. These observations can be various data such as sales, stock prices, traffic flow, customer flow, etc. For example, a store records the sales of each month in the past few years. These monthly sales data are arranged in chronological order to form a time series. By analyzing these historical sales data to predict the sales of the next few months is time series data forecasting.

[0016] In some examples, after obtaining the time series data to be predicted, the time series data to be predicted is input into the target prediction model, wherein the target prediction model includes a time series capture module, a feature transformation module, and a residual module. The time series capture module is used to capture the time series relationship inherent in the time series data to be predicted. For example, for time series data that changes over time, such as stock prices, the time series capture module can find the correlation between the order, periodicity, trend, etc. of prices at different time points, which is equivalent to extracting feature information related to the time dimension from the data so that subsequent predictions can be based on a basis that is more in line with the actual situation, thereby improving the accuracy of the prediction. Specifically, the time series capture module can adopt a unidirectional gated recurrent unit (Gated Recurrent Unit, GRU), a bidirectional GRU, or a long short-term memory network (Long-Short Term Memory, LSTM) and other structures. In this example, a bidirectional GRU structure is preferably used as the time series capture module, that is, the time series capture module includes a bidirectional GRU network to capture the bidirectional dependency relationship (including forward and reverse time series relationships) of the time series data to be predicted, enhance the modeling ability of the time series, and improve the accuracy of the prediction.

[0017] In some examples, the feature transformation module receives the results output from the time series capture module, that is, the data from which the time series relationship features have been extracted, and then performs feature transformation operations on these data. Feature transformation can be in various forms, such as through some linear or nonlinear transformation methods (such as linear transformation through matrix multiplication, nonlinear transformation using activation functions, etc.), the original feature representation is transformed into a new feature representation that is more conducive to subsequent prediction and can better reflect the essential characteristics of the data, so that the target prediction model can focus on more critical and effective information in the data, improve the ability to fit complex data, and thus improve the prediction effect.

[0018] In some examples, the residual module performs a residual connection between the transformed feature data output by the feature transformation module and the initial input data of the time series capture module, and finally uses the result of this residual connection as the target result output by the entire target prediction model. In the residual network in deep learning, the residual connection helps to alleviate the problem of gradient vanishing or gradient explosion during model training. Especially when the model structure is relatively complex, the residual connection can ensure that the gradient is more effectively back-propagated, so that the model can be better trained and converge to a better parameter value more quickly. On the other hand, the residual connection can integrate data information at different stages, so that the model can not only use the advanced features transformed by the feature transformation module, but also take into account some basic information in the original input data, so that the final prediction result integrates effective information at multiple levels, further improving the accuracy and reliability of the prediction.

[0019] In some examples, finally, the target result output by the target prediction model is obtained, and the prediction result corresponding to the time series data to be predicted is determined based on the target result. Specifically, the prediction result corresponding to the time series data to be predicted is determined based on the target result, including: using the output linear layer to map the target result to the target dimension; if the prediction task corresponding to the time series data to be predicted is a classification task, then the category probability distribution is output; if the prediction task corresponding to the time series data to be predicted is a regression task, then the prediction result is output.

[0020] It should be noted that for different prediction tasks, the corresponding labels need to be processed in different ways. Continuing with the above example, in the classification task, the labels represent different discrete categories, such as judging the action state of a person (standing, walking, running, etc.), or distinguishing which category a device belongs to (computer, mobile phone, printer, etc.). These categories are independent and non-continuous. At this time, these discrete category labels can be converted into one-hot encoding or category indexing. The category index simply assigns a numerical sequence to each category, which facilitates the model to distinguish and operate different categories during training and processing. The labels in the regression task are continuous values, such as predicting the cooling demand of a building (which may be a specific value such as kilowatts) or predicting the energy consumption of a device (a specific value such as kilowatt-hours), and its value can be any real number within a certain range. For such continuous value labels, it is sufficient to directly retain the original value, so that when the model is trained, the predicted value output by the model is also continuous and can be consistent with the numerical form of the label.

[0021] As an example, for example, to predict the weather conditions of a certain day in the future, the weather conditions are divided into the categories of "sunny", "cloudy", "light rain" and "heavy rain". At this time, the target prediction model outputs a target result. This target result may be a vector of a specific dimension, such as a vector of length 10. The output linear layer is used to map this target result to a target dimension suitable for the weather classification task. For example, after the transformation of the linear layer, the original vector of length 10 is converted into a vector of length 4 (4 weather categories). Finally, based on this vector of length 4, through specific calculations (such as using the softmax function, etc.), the category probability distribution is output. Assuming that the obtained vector is [0.2, 0.3, 0.4, 0.1], it means that the probability of predicting that this day is "light rain" is 0.4, the probability of being "sunny" is 0.2, the probability of being "cloudy" is 0.3, and the probability of being "heavy rain" is 0.1. Based on this probability distribution, we can determine that the final predicted weather category is most likely "light rain", thereby completing the weather classification prediction task corresponding to the time series data to be predicted.

[0022] For another example, to predict the average temperature of each day in the next week, the target prediction model outputs a target result with a dimension of 5. The output linear layer is used to map this target result to the target dimension. For temperature prediction, it is mapped to a numerical dimension (because the specific temperature value is output in the end). After the linear layer transformation, the vector is converted into a specific value (for example, through appropriate weight and bias calculation operations). Finally, the model directly outputs this value, such as 18.5℃, which means that the predicted average temperature for each day in the future is 18.5℃, thereby completing the regression task of temperature prediction corresponding to the time series data to be predicted.

[0023] According to the technical solution provided in the embodiment of the present application, the time series data to be predicted is obtained; the time series data to be predicted is input into the target prediction model, and firstly, the feature information related to the time dimension is extracted and captured from the time series data to be predicted by the time series capture module, so that the subsequent prediction can be more in line with the actual situation of the data, so as to improve the accuracy of the prediction; secondly, the transformation processing of the feature transformation module can enable the target prediction model to focus on more critical and more effective information in the data, improve the fitting ability of complex data, and then improve the prediction effect, and further improve the prediction accuracy; finally, the residual module is used to perform residual connection on the transformed feature data output by the feature transformation module and the initial input data of the time series capture module, so as to integrate data information at different stages, so that the model can not only utilize the advanced features transformed by the feature transformation module, but also take into account some basic information in the original input data, so that the final prediction result integrates effective information at multiple levels, further improves the accuracy and reliability of the prediction result, and avoids the problem of inaccurate prediction results in the prediction of time series data in the prior art.

[0024] In some embodiments, before inputting the time series data to be predicted into the target prediction model, it also includes: obtaining the sample time series data and the sample label results corresponding to the sample time series data, and inputting the target sample time series features obtained based on the sample time series data into the target prediction model to be trained; obtaining the target sample results output by the target prediction model to be trained, and determining the sample prediction results corresponding to the sample time series data based on the target sample results; calculating the loss value between the sample prediction results and the sample label results, and using the loss value to update the parameters of the target prediction model to be trained to obtain the target prediction model.

[0025] Specifically, before inputting the time series data to be predicted into the target prediction model, we must first obtain the sample time series data and the corresponding sample label results. The sample time series data is the historical time series data used to train the model, and the sample label results are the known correct results corresponding to these sample time series data.

[0026] Furthermore, the sample label time series features obtained based on the sample time series data are input into the target prediction model to be trained. The specific steps of obtaining the sample label time series features will be described in detail in the subsequent embodiments, and will not be repeated here. After receiving the input target sample time series features, the target prediction model to be trained will undergo a series of operations and processing (time series capture module, feature transformation module, and residual module feature processing and calculation) and finally output the target sample results.

[0027] Furthermore, the loss value between the sample prediction result and the sample label result is calculated. The loss value is an indicator to measure the degree of difference between the model prediction result and the true result. Specifically, loss functions such as mean square error (for regression tasks) and cross entropy loss (for classification tasks) can be used. This embodiment does not specifically limit it here. Finally, the calculated loss value is used to update the parameters of the target prediction model to be trained through an optimization algorithm (such as gradient descent algorithm and its variants, stochastic gradient descent, Adam optimizer, etc.). That is, various parameters in the model (such as weights, biases and other parameters in the neural network) are adjusted so that the prediction result can be closer to the true sample label result when the model makes a prediction next time. After multiple such iterative training processes (continuously inputting different sample time series data, repeating the above steps of calculating loss values ​​and updating parameters), the trained target prediction model is finally obtained.

[0028] According to the technical solution provided in the embodiments of the present application, sample time series data and sample label results corresponding to the sample time series data are obtained, and the target sample time series features obtained based on the sample time series data are input into the target prediction model to be trained; the target sample results output by the target prediction model to be trained are obtained, and the sample prediction results corresponding to the sample time series data are determined based on the target sample results; the loss value between the sample prediction results and the sample label results is calculated, and the parameters of the target prediction model to be trained are updated using the loss value to obtain the target prediction model, so as to improve the prediction accuracy of the target prediction model in subsequent processing.

[0029] In some embodiments, the target prediction model includes at least two prediction sub-modules, each prediction sub-module includes a time series capture module, a feature transformation module and a residual module; after the target sample time series features obtained based on the sample time series data are input into the target prediction model to be trained, it also includes: in the first prediction sub-module, the time series relationship of the target sample time series features is captured by the time series capture module to obtain the first time series features; the first time series features are transformed by the feature transformation module to obtain the second time series features; the target sample time series features and the second time series features are residually connected by the residual module to obtain the third time series features; the third time series features are used as the input of the next prediction sub-module, and the processing flow of the time series capture module, the feature transformation module and the residual module is repeatedly executed until the result output by the last prediction sub-module is obtained.

[0030] Specifically, the target prediction model includes at least two prediction submodules, each of which includes a time series capture module, a feature transformation module and a residual module. When the target sample time series features obtained based on the sample time series data are input into the target prediction model to be trained, they first enter the time series capture module of the first prediction submodule, and after being processed by this module, the first time series features are obtained. Then, the first time series features enter the feature transformation module of the first prediction submodule. The feature transformation module performs further transformation operations on the first time series features to obtain the second time series features. Finally, the residual module performs a residual connection on the target sample time series features (that is, the original input data features) initially input into the first prediction submodule and the second time series features output by the feature transformation module to obtain the third time series features.

[0031] It can be understood that after obtaining the third time series feature output by the first prediction submodule, it will not stop there, but will use the third time series feature as the input of the next prediction submodule, and then repeat the above-mentioned series of processing procedures of time series capture, feature processing and residual connection in the next prediction submodule. Such operations will be carried out in sequence between each prediction submodule, and data will be continuously transferred and processed until all prediction submodules are processed, and finally the result output by the last prediction submodule is obtained. Through this method of progressive processing by multiple prediction submodules with the same structure, the target prediction model can perform in-depth and detailed feature extraction and analysis of time series data, thereby further improving the accuracy and reliability of time series data prediction.

[0032] In some embodiments, the feature transformation module includes a first feature transformation module and a second feature transformation module, and the feature transformation module is used to perform feature transformation on the first time series feature to obtain the second time series feature, including: using the first feature transformation module to perform feature transformation processing on the first time series feature to obtain the initial transformation feature; using the second feature transformation module to perform feature transformation processing on the initial transformation feature to obtain the second time series feature.

[0033] Specifically, the feature transformation module includes a first feature transformation module and a second feature transformation module. The entire feature transformation process of the feature transformation module is to first use the first feature transformation module to process the first time series feature to obtain an initial transformation feature, and then further process the initial transformation feature through the second feature transformation module to finally obtain the second time series feature.

[0034] In some embodiments, the first feature transformation module includes a first fully connected layer, a first normalization layer and a first function layer; the first feature transformation module is used to perform feature transformation processing on the first time series feature to obtain an initial transformation feature, including: using the first fully connected layer to perform linear transformation on the first time series feature to obtain a first transformation feature; using the first normalization layer to perform layer normalization processing on the first transformation feature to obtain an initial normalized transformation feature; using the first activation function of the first function layer to perform nonlinear transformation on the initial normalized transformation feature to obtain an initial transformation feature.

[0035] Specifically, the core calculation of the fully connected layer involves linear transformation. Here, the role of the first fully connected layer is to perform a linear transformation on the first time series feature and convert it into the first transformed feature. By connecting the neurons in the fully connected layer with all neurons in the previous layer and calculating the corresponding weight matrix, the numerical representation of the input feature is changed to give it a new linear structure, laying the foundation for subsequent further processing.

[0036] It can be understood that after obtaining the first transformation feature, the first normalization layer performs layer normalization on the first transformation feature, normalizes the data of the feature, makes the mean and variance of the data within a suitable range, and then obtains the initial normalized transformation feature. This helps to improve the stability of the entire model training, avoids problems such as gradient disappearance or gradient explosion that are prone to occur during neural network training, and can also speed up the model training to a certain extent.

[0037] It can be understood that the first activation function in the first function layer (such as Sigmoid, ReLU, Tanh and other activation functions) performs nonlinear transformation on the initial normalized transformation features, and finally obtains the initial transformation features. The introduction of the activation function is to give the features nonlinear capabilities, because it is difficult to fit the complex actual data relationship only by the previous linear transformation. Different activation functions have different characteristics. The first activation function performs nonlinear mapping on the normalized features according to its own characteristics, so that the processed initial transformation features can better adapt to the subsequent application requirements in the entire feature processing and related tasks.

[0038] In some embodiments, the second feature transformation module includes a second fully connected layer, a second normalization layer and a second function layer; the second feature transformation module is used to perform feature transformation processing on the initial transformation features to obtain second time series features, including: using the second fully connected layer to perform linear transformation on the initial transformation features to obtain second transformation features; using the second normalization layer to perform layer normalization processing on the second transformation features to obtain normalized transformation features; using the second activation function of the second function layer to perform nonlinear transformation on the normalized transformation features to obtain second time series features.

[0039] Specifically, the second fully connected layer is used to perform a linear transformation on the initial transformation features to obtain the second transformation features, and the first fully connected layer in the first feature transformation module and the second fully connected layer in the second feature transformation module are linearly transformed respectively. Through two linear transformations, the structure of the features can be adjusted from different angles. The first linear transformation (first fully connected layer) is a preliminary dimensional adjustment or information integration of the original first time series features, while the second linear transformation (second fully connected layer) can further explore the relationship between features based on the initial transformation features, so that the features can better represent the complex information in the data.

[0040] It can be understood that the second transformation feature is layer-normalized using the second normalization layer to obtain the normalized transformation feature, which avoids problems such as gradient vanishing or gradient explosion after the data is processed through multiple layers. For example, in a deep neural network, as the number of layers increases, the distribution of data may change, resulting in an unstable training process. By performing normalization in both stages, the distribution of data can be effectively controlled, making the model more stable during training and better able to generalize to unseen data.

[0041] It is understandable that in neural networks, nonlinear transformations are very important because actual data distributions and relationships are often nonlinear. By superimposing two nonlinear transformations, the model can fit more complex functional relationships. The first nonlinear transformation will capture some simple nonlinear patterns, and the second nonlinear transformation can further refine and expand these patterns on this basis, so that the final second time series features can better express the nonlinear characteristics in the data.

[0042] In this way, the feature transformation process is divided into two stages. The first feature transformation module is used to obtain the initial transformation features, and then the second feature transformation module is used to further process the second time series features. This staged approach is similar to a process of gradually extracting and refining information. The first stage can preliminarily screen and transform the useful information in the first time series features and remove some noise or redundant information. The second stage can further mine and refine more advanced and abstract features based on the preliminary processing, so that the final second time series features are more representative and discriminative, which is conducive to subsequent classification, prediction and other tasks.

[0043] In some embodiments, before the target sample time series features obtained based on the sample time series data are input into the target prediction model to be trained, it also includes: performing data preprocessing on the sample time series data to obtain initial sample time series features; mapping the initial sample time series features to the hidden layer dimension to obtain linear transformation features; performing layer normalization on the linear transformation features to obtain linear normalized features; and performing nonlinear transformation on the linear normalized features using a third activation function to obtain target sample time series features.

[0044] Specifically, data preprocessing is performed on the sample time series data to obtain initial sample time series features, including: performing frequency conversion on the sample time series data to convert the sampling frequency of the sample time series data into a target frequency; performing data cleaning and standardization on the frequency-converted sample time series data, and performing feature enhancement on the standardized sample time series data to obtain initial sample time series features.

[0045] In some examples, in practical applications, time series data often comes from multiple different sensors or other data sources, and the time intervals (i.e., frequencies) for collecting data from these data sources may be different. For example, some sensors may collect data once per second, while others may collect data every five seconds. If these data are to be comprehensively analyzed and subsequently processed, they need to be unified to the same time frequency, so that the data are comparable and consistent in the time dimension, which is convenient for subsequent data processing. In addition, during the data collection process, outliers may be generated due to various reasons such as sensor failure and external interference. If these outliers are not processed, they will have a negative impact on the training and analysis of subsequent models, such as causing deviations in model fitting and reducing the accuracy of the model. At the same time, the original data may contain many features, but not all features are closely related to the specific tasks to be solved. Too many irrelevant features will increase the complexity and computational complexity of the model, and may also introduce unnecessary noise. Therefore, it is necessary to process outliers in the sample time series data.

[0046] In addition, different features often have different numerical dimensions (that is, units and value ranges, etc.). For example, in a data set containing multiple features such as temperature, pressure, and length, the temperature may be in the range of tens of degrees Celsius, the pressure may be in the range of hundreds of Pascals, and the length may be in the range of several meters. If features with large dimensional differences are used directly for model training, it may cause the model to attach an unbalanced degree of importance to different features during the training process, because features with large values ​​may dominate the calculation process, masking the role of features with small values ​​but may be equally important. By standardizing features and eliminating dimensional differences, the model can treat each feature more fairly and reasonably, thereby improving the generalization ability of the model and enabling it to perform better when facing different test data. Specifically, standardization methods (such as StandardScaler) can be used to adjust the features to a distribution with a mean of 0 and a variance of 1. Each feature is processed according to the following formula:

[0047] in, is the feature mean, is the characteristic standard deviation.

[0048] In addition, random noise, random slicing or sliding window enhancement can be introduced to enhance the features of the standardized sample time series data to obtain the final initial sample time series features, thereby improving its performance during training and its generalization ability to new data.

[0049] Furthermore, the initial sample time series features are mapped to the hidden layer dimensions to obtain linear transformation features, which can recombin and abstract the original features and mine the potential relationships in the original data. This linear transformation is the basis for building complex feature representations, which helps to extract more advanced and representative feature information, so that the model can better understand the data. The linear normalized features obtained by layer normalization can effectively avoid the problem of gradient disappearance or gradient explosion. During the training of a neural network, the distribution of data will change with the increase in the number of layers. If it is not controlled, it may cause gradient problems, making training difficult or slow to converge. Layer normalization adjusts the mean and variance of the data to make the data distribution more stable, thereby improving the stability of model training. The normalized features can also enable the model to learn effective information in the data faster, reduce fluctuations during training, and improve training efficiency. Finally, the target sample time series features are obtained by nonlinear transformation using the third activation function (such as Sigmoid, ReLU, Tanh, etc.), which enables the model to fit complex nonlinear relationships. Since real data is often distributed nonlinearly, nonlinear transformation enables features to better capture complex patterns in the data and enhance the ability of features in expressing complex data relationships.

[0050] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the process of the embodiments of the present application.

[0051] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described in detail here.

[0052] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0053] Figure 2 Schematic diagram of a time series data prediction device provided in an embodiment of the present application. Figure 2 As shown, the device comprises: An acquisition module 201 is configured to acquire time series data to be predicted; The input module 202 is configured to input the time series data to be predicted into the target prediction model, the target prediction model includes a time series capture module, a feature transformation module and a residual module, the time series capture module is used to capture the time series relationship of the time series data to be predicted, the feature transformation module is used to perform feature transformation on the output of the time series capture module, and the residual module is used to perform residual connection between the output of the feature transformation module and the input of the time series capture module, and use the result of the residual connection as the target result output by the target prediction model; The determination module 203 is configured to obtain the target result output by the target prediction model, and determine the prediction result corresponding to the time series data to be predicted according to the target result.

[0054] In some embodiments, the prediction device for time series data also includes a training module 204, which is configured to obtain sample time series data and sample label results corresponding to the sample time series data, and input the target sample time series features obtained based on the sample time series data into the target prediction model to be trained; obtain the target sample results output by the target prediction model to be trained, and determine the sample prediction results corresponding to the sample time series data based on the target sample results; calculate the loss value between the sample prediction results and the sample label results, and use the loss value to update the parameters of the target prediction model to be trained to obtain the target prediction model.

[0055] In some embodiments, the target prediction model includes at least two prediction sub-modules, each prediction sub-module includes a time series capture module, a feature transformation module and a residual module, and the training module 204 is also configured to, in the first prediction sub-module, use the time series capture module to capture the time series relationship of the target sample time series features to obtain the first time series features; use the feature transformation module to perform feature transformation on the first time series features to obtain the second time series features; use the residual module to perform residual connection on the target sample time series features and the second time series features to obtain the third time series features; use the third time series features as the input of the next prediction sub-module, and repeat the processing flow of the time series capture module, the feature transformation module and the residual module until the result output by the last prediction sub-module is obtained.

[0056] In some embodiments, the feature transformation module includes a first feature transformation module and a second feature transformation module, and the training module 204 is also configured to use the first feature transformation module to perform feature transformation processing on the first time series feature to obtain an initial transformation feature; and use the second feature transformation module to perform feature transformation processing on the initial transformation feature to obtain a second time series feature.

[0057] In some embodiments, the first feature transformation module includes a first fully connected layer, a first normalization layer and a first function layer, and the training module 204 is also configured to use the first fully connected layer to perform a linear transformation on the first time series feature to obtain a first transformation feature; use the first normalization layer to perform layer normalization on the first transformation feature to obtain an initial normalized transformation feature; use the first activation function of the first function layer to perform a nonlinear transformation on the initial normalized transformation feature to obtain an initial transformation feature.

[0058] In some embodiments, the second feature transformation module includes a second fully connected layer, a second normalization layer and a second function layer, and the training module 204 is also configured to use the second fully connected layer to perform linear transformation on the initial transformation features to obtain second transformation features; use the second normalization layer to perform layer normalization on the second transformation features to obtain normalized transformation features; use the second activation function of the second function layer to perform nonlinear transformation on the normalized transformation features to obtain second time series features.

[0059] In some embodiments, the training module 204 is also configured to perform data preprocessing on the sample time series data to obtain initial sample time series features; map the initial sample time series features to the hidden layer dimension to obtain linear transformation features; perform layer normalization on the linear transformation features to obtain linear normalized features; use a third activation function to perform nonlinear transformation on the linear normalized features to obtain target sample time series features.

[0060] Figure 3Schematic diagram of an electronic device 3 provided in an embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment includes: a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 301 executes the computer program 303, the functions of the modules / units in the above-mentioned device embodiments are implemented.

[0061] The electronic device 3 may be a desktop computer, a notebook, a PDA, a cloud server, or other electronic device. The electronic device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will appreciate that Figure 3 The electronic device 3 is merely an example and does not limit the electronic device 3 , and may include more or less components than those shown in the figure, or different components.

[0062] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0063] The memory 302 may be an internal storage unit of the electronic device 3, for example, a hard disk or memory of the electronic device 3. The memory 302 may also be an external storage device of the electronic device 3, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 3. The memory 302 may also include both an internal storage unit of the electronic device 3 and an external storage device. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0064] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units.

[0065] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. The computer program may include computer program code, which may be in source code form, object code form, executable file or some intermediate form. The readable storage medium may include: any entity or device that can carry computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0066] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for predicting time series data, characterized in that: include: Get the time series data to be predicted; The time series data to be predicted is input into a target prediction model, wherein the target prediction model comprises a time series capture module, a feature transformation module and a residual module, wherein the time series capture module is used to capture the time series relationship of the time series data to be predicted, the feature transformation module is used to perform feature transformation on the output of the time series capture module, and the residual module is used to perform residual connection between the output of the feature transformation module and the input of the time series capture module, and the result of the residual connection is used as the target result output by the target prediction model; The target result output by the target prediction model is obtained, and a prediction result corresponding to the time series data to be predicted is determined according to the target result.

2. The method according to claim 1, characterized in that Before inputting the time series data to be predicted into the target prediction model, the method further includes: Obtaining sample time series data and sample label results corresponding to the sample time series data, and inputting target sample time series features obtained based on the sample time series data into a target prediction model to be trained; Obtaining a target sample result output by the target prediction model to be trained, and determining a sample prediction result corresponding to the sample time series data according to the target sample result; The loss value between the sample prediction result and the sample label result is calculated, and the parameters of the target prediction model to be trained are updated using the loss value to obtain the target prediction model.

3. The method according to claim 2, characterized in that The target prediction model includes at least two prediction submodules, each of which includes a time series capture module, a feature transformation module and a residual module; after inputting the target sample time series features obtained based on the sample time series data into the target prediction model to be trained, it also includes: In the first prediction submodule, the time series capture module is used to capture the time series relationship of the target sample time series feature to obtain a first time series feature; Using the feature transformation module to perform feature transformation on the first time series feature to obtain a second time series feature; Using the residual module to perform residual connection on the target sample time series feature and the second time series feature to obtain a third time series feature; The third time series feature is used as the input of the next prediction submodule, and the processing flow of the time series capture module, the feature transformation module and the residual module is repeatedly executed until the result output by the last prediction submodule is obtained.

4. The method according to claim 3, characterized in that The feature transformation module includes a first feature transformation module and a second feature transformation module, and the feature transformation of the first time series feature by using the feature transformation module to obtain the second time series feature includes: Using the first feature transformation module to perform feature transformation processing on the first time series feature to obtain an initial transformation feature; The second feature transformation module is used to perform feature transformation processing on the initial transformation feature to obtain the second time series feature.

5. The method according to claim 4, characterized in that The first feature transformation module includes a first fully connected layer, a first normalization layer, and a first function layer; the first feature transformation module is used to perform feature transformation processing on the first time series feature to obtain an initial transformation feature, including: Using the first fully connected layer to perform a linear transformation on the first time series feature to obtain a first transformed feature; Using the first normalization layer to perform layer normalization processing on the first transformation features to obtain initial normalized transformation features; The initial normalized transformation features are nonlinearly transformed using the first activation function of the first function layer to obtain the initial transformation features.

6. The method according to claim 4, characterized in that The second feature transformation module includes a second fully connected layer, a second normalization layer, and a second function layer; the using the second feature transformation module to perform feature transformation processing on the initial transformation feature to obtain the second time series feature includes: Using the second fully connected layer to perform a linear transformation on the initial transformation feature to obtain a second transformation feature; Using the second normalization layer to perform layer normalization processing on the second transformed features to obtain normalized transformed features; The normalized transformation feature is nonlinearly transformed using the second activation function of the second function layer to obtain the second time series feature.

7. The method according to claim 2, characterized in that Before inputting the target sample time series features obtained based on the sample time series data into the target prediction model to be trained, the method further includes: Performing data preprocessing on the sample time series data to obtain initial sample time series features; Mapping the initial sample time series features to the hidden layer dimension to obtain linear transformation features; Performing layer normalization processing on the linear transformation features to obtain linear normalized features; The linear normalized features are nonlinearly transformed using a third activation function to obtain the target sample time series features.

8. A prediction device for time series data, characterized in that: include: An acquisition module, configured to acquire time series data to be predicted; An input module is configured to input the time series data to be predicted into a target prediction model, wherein the target prediction model includes a time series capture module, a feature transformation module, and a residual module, wherein the time series capture module is used to capture the time series relationship of the time series data to be predicted, the feature transformation module is used to perform feature transformation on the output of the time series capture module, and the residual module is used to perform residual connection between the output of the feature transformation module and the input of the time series capture module, and use the result of the residual connection as the target result output by the target prediction model; The determination module is configured to obtain the target result output by the target prediction model, and determine the prediction result corresponding to the time series data to be predicted according to the target result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images