Time sequence prediction method and device and electronic equipment

Abstract

The invention discloses a time sequence prediction method and device and electronic equipment. The method comprises the steps that original time series data to be predicted are acquired, and the original time series data are time series data generated when data interaction is carried out between different data sources; a model upper layer and a model lower layer in a preset model are adopted to predict the original time sequence data to obtain predicted time sequence data, the model lower layer is used for determining trend sequence data corresponding to the original time sequence data, and a first linear regression neural network layer in the model upper layer is used for predicting the trend sequence data to obtain predicted time sequence data; a second linear regression neural network in the upper layer of the model is used for predicting remainder sequence data corresponding to the original time sequence data, and the remainder sequence data is obtained through the original time sequence data and the trend sequence data. The technical problem that a traditional time sequence prediction method is difficult to capture a long-term dependency relationship in data, and consequently the prediction precision is poor is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and more specifically, to a time series prediction method, device and electronic device. Background Art

[0002] With the rapid development of information technology and the advent of the big data era, time series data plays a vital role in various fields. Time series forecasting aims to use historical data to predict future trends and changes, providing important basis for corporate decision-making, scientific research, etc. Whether it is commodity sales trend analysis in the retail industry, load demand forecasting in the power industry, climate pattern forecasting in meteorology, or even disease development trend research in the medical and health field, it involves a large amount of complex time series data analysis and forecasting.

[0003] Traditional time series prediction methods are usually based on statistical models or machine learning methods, such as autoregressive model (AR), moving average model (MA), autoregressive moving average model (ARMA), support vector machine (SVM), etc. These methods have certain effects in dealing with simple linear time series problems, but they have some limitations when facing nonlinear, high-dimensional, periodic and trend changes, and data scenarios with strong noise interference, such as limited ability to extract and express data features, and difficulty in capturing long-term dependencies and nonlinear associations in the data, resulting in poor prediction accuracy.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a time series prediction method, device and electronic device to at least solve the technical problem that traditional time series prediction methods are difficult to capture long-term dependencies in data, resulting in poor prediction accuracy.

[0006] According to one aspect of an embodiment of the present application, a time series prediction method is provided, including: obtaining original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; using an upper model layer and a lower model layer in a preset model to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network layer in the upper model layer is used to predict the remainder series data corresponding to the original time series data, wherein the remainder series data is obtained through the original time series data and the trend series data.

[0007] Optionally, the preset model is trained in the following manner: obtaining historical time series data generated by the data source during data interaction, and historical true label values ​​corresponding to the historical time series data; inputting the historical time series data and the historical true label values ​​into the initial preset model for training to obtain historical output data; inputting the historical time series data and the historical output data into a fusion loss function to obtain a first historical loss value; after determining the first historical loss value, adjusting the parameters of the initial preset model, and predicting the historical time series data according to the initial preset model after the parameters are adjusted to obtain adjusted historical output data; inputting the historical time series data and the adjusted historical output data into the fusion loss function to obtain a second historical loss value, and continuing to adjust the parameters of the initial preset model when the second historical loss value does not meet the preset conditions, until the loss value corresponding to the historical output data output by the initial preset model after the parameters are adjusted meets the preset conditions, and the training is stopped to obtain the preset model.

[0008] Optionally, the fusion loss function is determined in the following manner: obtaining the data volume of historical time series data; determining a first loss function based on the data volume, historical true label values ​​and historical output data; determining a second loss function based on the data volume, historical true label values, historical output data and a logarithmic hyperbolic cosine function; determining a fusion loss function based on the first loss function and the second loss function.

[0009] Optionally, adjusting the parameters of the initial preset model includes: determining the adversarial gradient of the fusion loss function; determining the first-order moving average momentum and the second-order moving average momentum based on the adversarial gradient; determining the parameter correction value of the initial preset model based on the first-order moving average momentum and the second-order moving average momentum; and adjusting the parameters of the initial preset model based on the parameter correction value.

[0010] Optionally, determining the adversarial gradient of the fusion loss function includes: determining the original gradient of the fusion loss function; determining the perturbation data based on the original gradient and the perturbation parameter; calculating the perturbation data and the historical time series data through the fusion loss function to obtain the historical perturbation loss value; back-propagating the fusion loss function after obtaining the historical perturbation loss value to obtain the perturbation gradient; determining the adversarial gradient of the fusion loss function based on the original gradient and the perturbation gradient, wherein the adversarial gradient is the adversarial gradient during the first round of training of the initial preset model.

[0011] Optionally, determining the first-order moving average momentum and the second-order moving average momentum based on the adversarial gradient includes: obtaining a first-order momentum exponential decay rate parameter, a second first-order moving average momentum obtained during a second round of training, and an adversarial gradient, wherein the second round of training is a round of training before the first round of training; determining the first-order moving average momentum based on the first-order momentum exponential decay rate parameter, the second first-order moving average momentum, and the adversarial gradient; obtaining a second-order momentum exponential decay rate parameter, a second second-order moving average momentum obtained during the second round of training, and the adversarial gradient; determining the second-order moving average momentum based on the second-order momentum exponential decay rate parameter, the second second-order moving average momentum, and the adversarial gradient.

[0012] Optionally, the parameters of the initial preset model are adjusted according to the parameter correction value, including: obtaining the first parameter of the initial preset model in the first round of training, and obtaining the original gradient of the fusion loss function; when the product of the first parameter and the adversarial gradient is less than the preset parameter, determining the projection of the parameter correction value in the first parameter direction of the fusion loss function; adjusting the first parameter according to the first parameter, the projection of the parameter correction value in the first parameter direction of the fusion loss function and the learning rate; when the product of the first parameter and the adversarial gradient is greater than or equal to the preset parameter, adjusting the first parameter according to the first parameter, the parameter correction value and the learning rate.

[0013] According to another aspect of an embodiment of the present application, a time series prediction device is also provided, including: an acquisition module, used to acquire original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; a prediction module, used to predict the original time series data using an upper model layer and a lower model layer in a preset model to obtain predicted time series data, wherein the lower model layer is used to determine trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network layer in the upper model layer is used to predict the remainder sequence data corresponding to the original time series data, wherein the remainder sequence data is obtained through the original time series data and the trend sequence data.

[0014] According to another aspect of the embodiments of the present application, there is also provided an electronic device, comprising: a memory for storing program instructions; a processor, connected to the memory, for executing program instructions to implement the following functions: obtaining original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; using an upper model layer and a lower model layer in a preset model to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network layer in the upper model layer is used to predict the remainder series data corresponding to the original time series data, wherein the remainder series data is obtained through the original time series data and the trend series data.

[0015] According to another aspect of the embodiments of the present application, a non-volatile storage medium is provided, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the above-mentioned time series prediction method by running the computer program.

[0016] According to another aspect of the embodiments of the present application, a computer program product is also provided, including computer instructions, which implement the above-mentioned time series prediction method when executed by a processor.

[0017] In an embodiment of the present application, by obtaining original time series data to be predicted, wherein the original time series data is the time series data generated when different data sources interact with each other; the upper model layer and the lower model layer in the preset model are used to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network in the upper model layer is used to predict the remainder sequence data corresponding to the original time series data, wherein the remainder sequence data is obtained through the original time series data and the trend series data, thereby achieving the purpose of improving the time prediction accuracy, thereby achieving the technical effect of combining trend prediction and remainder prediction to enhance the model's processing ability for complex time series data, and thus solving the technical problem that traditional time series prediction methods are difficult to capture long-term dependencies in data, resulting in poor prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a time series prediction method according to an embodiment of the present application;

[0020] Figure 2 is a flow chart of a time series prediction method according to an embodiment of the present application;

[0021] Figure 3 It is a structural diagram of a time series prediction device according to an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] First, some nouns or terms that appear in the process of explaining the embodiments of the present application are subject to the following explanations:

[0025] Time series prediction: An important direction in the field of computer science and intelligent data, it is a method of using data from the past to predict information in the future. The basic idea of ​​time series prediction is to use the time series as a basis, analyze its direction and degree of change, and thus speculate on the level that may be reached in the future. The purpose of time series analysis is to describe the past change characteristics of the time series, discover the operating rules of the time series, and thus predict the trend of the time series in the future.

[0026] A Linear Neural Network is a neural network model with a relatively simple structure. It usually contains one or more fully connected layers. Each neuron in each layer is connected to all neurons in the previous layer, and its activation function is linear. In a linear neural network, the output is the result of the weighted sum of the input signal plus the bias term, and then obtained through a linear transformation.

[0027] In order to solve the problem of poor time series prediction accuracy in the related art, the present application embodiment provides a time series prediction method, which can be run on Figure 1 In the computer terminal shown, the computer terminal is explained below.

[0028] The time series prediction method embodiment provided in the embodiment of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computer terminal for implementing a time series prediction method. Figure 1 As shown, the computer terminal 10 may include one or more (102a, 102b, ..., 102n are used to illustrate) processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory 104 for storing data, and a transmission module 106 for communication functions connected via a wired and / or wireless network. In addition, it may also include: a display, a keyboard, a cursor control device, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and a BUS bus. Those skilled in the art can understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuits". The data processing circuits may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuit may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computer terminal 10. As described in the embodiments of the present application, the data processing circuit acts as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0030] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the time series prediction method in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, realizing the above-mentioned time series prediction method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computer terminal 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0031] The transmission module 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission module 106 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission module 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0032] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computer terminal 10 .

[0033] It should be noted that, in some optional embodiments, the above Figure 1 The computer terminal shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of hardware elements and software elements. It should be noted that Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computer terminal described above.

[0034] In the above-mentioned operating environment, an embodiment of the present application provides an embodiment of a time series prediction method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0035] Figure 2 is a flowchart of a time series prediction method according to an embodiment of the present application, such as Figure 2As shown, the method comprises the following steps:

[0036] Step S202 , obtaining original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other.

[0037] In the above step S202, time series data is a series of data points arranged in chronological order, which records the trend and characteristics of a phenomenon over time. In data interaction across data sources, these time series data may come from multiple different systems, platforms or devices. The above data sources may include, for example, sensor data, database records, log files, API call results, file storage, etc. For data that needs to be obtained through API or specific protocols, it is necessary to design or determine a suitable data interface. Use ETL (extraction, transformation, loading) tools or write custom scripts to extract data from various data sources. Integrate the data extracted from different data sources into a unified format and structure for subsequent processing and analysis.

[0038] After extracting the data, it is necessary to clean and preprocess the data. Specifically, data cleaning can include dealing with missing values, outliers, duplicate data, and other issues to ensure the accuracy and completeness of the data. Preprocessing includes time alignment and data standardization. The reason for time alignment is that different data sources may have different timestamp recording methods, so time alignment is required to ensure that all time series data are in the same time frame. When performing data standardization, the data is converted into a format suitable for analysis, such as converting a string timestamp into the timestamp type in a time series database.

[0039] Step S204, using the upper layer and the lower layer of the model in the preset model to predict the original time series data to obtain predicted time series data, wherein the lower layer of the model is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper layer of the model is used to predict the trend series data, and the second linear regression neural network in the upper layer of the model is used to predict the remainder sequence data corresponding to the original time series data, wherein the remainder sequence data is obtained through the original time series data and the trend sequence data.

[0040] In the above step S204, the lower layer of the model is a linear decomposition layer that decomposes the input (original) time series data. This layer uses a one-dimensional average pooling with a determined window size, step size and edge filling to perform moving average on the input time series data, and the result is used as trend sequence data. The main goal of the lower layer of the model is to identify and extract the trend component in the original time series data. The trend refers to the rising or falling trend presented by the time series data in the long term. The output of the lower layer of the model is trend sequence data, which represents the long-term trend part in the original time series data. The original time series data is composed of multiple components, including but not limited to trend, seasonality, and periodicity. After the trend sequence is extracted by the lower layer of the model, the remainder sequence data can be further calculated. The remainder sequence is obtained by subtracting the trend sequence from the original time series. This sequence contains all other components in the original data that are not explained by the trend part, such as seasonal fluctuations and periodic changes.

[0041] The upper layer of the model is responsible for further prediction based on the separated trend and residue sequences. It usually consists of two main parts: the first linear regression neural network layer and the second linear regression neural network layer, which process the trend sequence data and the residue sequence data respectively in a fully connected network manner. The predicted time series data output by the final model is obtained by adding the prediction of the trend sequence by the first linear regression neural network layer and the prediction of the residue sequence by the second linear regression neural network layer. This method combines information at different levels and aims to improve the accuracy and robustness of the overall prediction.

[0042] Through the above steps S202 to S204, the purpose of improving the time prediction accuracy is achieved, thereby achieving the technical effect of combining trend prediction and residue prediction to enhance the model's ability to process complex time series data, thereby solving the technical problem that traditional time series prediction methods are difficult to capture long-term dependencies in data, resulting in poor prediction accuracy. The following is an explanation.

[0043] In step S204 of the above-mentioned time series prediction method, the preset model is trained in the following manner: obtaining historical time series data generated by the data source during data interaction, and historical true label values ​​corresponding to the historical time series data; inputting the historical time series data and the historical true label values ​​into the initial preset model for training to obtain historical output data; inputting the historical time series data and the historical output data into the fusion loss function to obtain a first historical loss value; after determining the first historical loss value, adjusting the parameters of the initial preset model, and predicting the historical time series data according to the initial preset model after the parameters are adjusted to obtain the adjusted historical output data; inputting the historical time series data and the adjusted historical output data into the fusion loss function to obtain a second historical loss value, and continuing to adjust the parameters of the initial preset model when the second historical loss value does not meet the preset conditions, until the loss value corresponding to the historical output data output by the initial preset model after the parameters are adjusted meets the preset conditions, and the training is stopped to obtain the preset model.

[0044] In the embodiment of the present application, the training process of the preset model includes the following steps:

[0045] 1. Data collection: Collect historical time series data from data sources. These data are generated through data interaction in a certain period of time in the past. For example, they can include observations, records, or measurements of time series, and ensure that the collected data covers a long enough time range to contain enough historical information. At the same time, collect historical true label values ​​corresponding to these historical time series data. These label values ​​are usually the actual observations of the time series data at a certain point in the future, which are used to evaluate the prediction accuracy of the model.

[0046] 2. Initial preset model: Prepare an initial preset model, which can be any machine learning or deep learning model suitable for time series prediction, such as linear regression, time series decomposition model, recurrent neural network (RNN), long short-term memory network (LSTM), etc.

[0047] 3. Model training (first round): Input historical time series data and historical true label values ​​into the initial preset model for training. The model generates predicted values ​​(i.e. historical output data) through its internal algorithm (such as forward propagation) based on the input data and labels.

[0048] 4. Loss calculation (first round): The historical time series data and the historical output data generated by the model are input into the fusion loss function. The fusion loss function is a function that evaluates the prediction accuracy of the model, which calculates the difference between the predicted value and the actual label value. The fusion loss function outputs the first historical loss value, which reflects the prediction performance of the model in the first round of training.

[0049] 5. Model parameter adjustment: According to the first historical loss value, the parameters of the initial preset model are adjusted through optimization algorithms such as back propagation. The purpose of parameter adjustment is to reduce the loss value, that is, to improve the prediction accuracy of the model.

[0050] 6. Model training (iteration): Use the model with adjusted parameters to predict the historical time series data again to obtain the adjusted historical output data. Input the new historical time series data and the adjusted historical output data into the fusion loss function again to calculate the second historical loss value. If the second historical loss value does not meet the preset conditions (such as being lower than a certain threshold or the loss value decreases less than a certain value after multiple rounds of training), continue to adjust the model parameters and repeat the above process.

[0051] 7. Training stop: When the loss value corresponding to the historical output data output by the model after parameter adjustment meets the preset conditions, the training process is stopped. At this point, the model is considered to have fully learned the patterns in the historical data and can be used for future prediction tasks.

[0052] 8. Obtaining the preset model: After the above training process, the model obtained is the preset model. The preset model has been optimized based on historical data and has the ability to predict future time series data.

[0053] In the above steps, the fusion loss function is determined by: obtaining the data volume of historical time series data; determining the first loss function based on the data volume, historical true label values ​​and historical output data; determining the second loss function based on the data volume, historical true label values, historical output data and logarithmic hyperbolic cosine function; determining the fusion loss function based on the first loss function and the second loss function.

[0054] In an embodiment of the present application, in each batch, the input data pairs are trained and learned, and the amount of historical time series data, i.e., the total number of data points, is obtained. The first loss function is determined based on the amount of data, the historical true label value (i.e., the actual observed value) and the historical output data (i.e., the predicted value of the model). The first loss function is usually used to directly measure the difference between the model prediction value and the actual value. In the embodiment of the present application, the first loss function is the L1 loss function. Common first loss functions may also include mean square error (MSE), mean absolute error (MAE), etc. In addition to the first loss function, it is also necessary to determine the second loss function, which is determined based on the amount of data, the historical true label value, the historical output data, and the logarithmic hyperbolic cosine function (such as the log-cosh function). The logarithmic hyperbolic cosine function is a smooth loss function that has a smaller penalty for outliers (i.e., points far from the mean value), which helps the model to perform more stably on data sets with noise or outliers. According to the first loss function and the second loss function, a fusion loss function is formed by combining in some way (such as weighted sum). The purpose of the fusion loss function is to consider multiple aspects of prediction errors at the same time in order to more comprehensively evaluate and optimize the model. The choice of weights may be based on experience, cross-validation, or other optimization strategies to ensure that the fusion loss function can guide the model to train in the desired direction.

[0055] Specifically, the formula of the fusion loss function is as follows:

[0056]

[0057] Among them, BatchLoss is the fusion loss function, N is the size of a batch, that is, the amount of data mentioned above, y i is the true label value of the i-th historical sample data input in this batch, z i is the output value of the neural network model for the i-th sample data input in this batch (i.e. the historical output data mentioned above), α is the weight parameter, δ is the smoothing L1 hyperparameter, and log-cosh function is the logarithmic hyperbolic cosine function.

[0058] In the above steps, the parameters of the initial preset model are adjusted, including: determining the adversarial gradient of the fusion loss function; determining the first-order moving average momentum and the second-order moving average momentum based on the adversarial gradient; determining the parameter correction value of the initial preset model based on the first-order moving average momentum and the second-order moving average momentum; and adjusting the parameters of the initial preset model based on the parameter correction value.

[0059] In an embodiment of the present application, the first-order moving average momentum accumulates past gradient information, helps to accelerate the update of the gradient in the relevant direction, and suppresses oscillation. The second-order moving average momentum takes into account the accumulation of the square of the gradient, which is used to adaptively adjust the learning rate of each parameter. The update of these momentums is usually based on the current gradient (i.e., the adversarial gradient of the fusion loss function). In each step of the optimization algorithm, the correction value of the model parameters can be calculated based on the current first-order moving average and second-order moving average momentum. This correction value is intended to adjust the model parameters in an optimal way to minimize the loss function. According to the calculated parameter correction value, the parameters of the initial preset model are updated. This update step is the core of the iterative process. By repeating this process continuously, the model parameters gradually converge to the optimal solution (or approximate optimal solution) that can minimize the loss function.

[0060] In the above steps, the adversarial gradient of the fusion loss function is determined, including: determining the original gradient of the fusion loss function; determining the perturbation data based on the original gradient and the perturbation parameter; calculating the perturbation data and the historical time series data through the fusion loss function to obtain the historical perturbation loss value; back-propagating the fusion loss function after obtaining the historical perturbation loss value to obtain the perturbation gradient; determining the adversarial gradient of the fusion loss function based on the original gradient and the perturbation gradient, wherein the adversarial gradient is the adversarial gradient during the first round of training of the initial preset model.

[0061] In an embodiment of the present application, the original gradient of the fusion loss function with respect to the model parameters is calculated. For example, it can be accomplished by applying the fusion loss function to the real labels of the model prediction and historical time series data, and then executing the back propagation algorithm to obtain the original gradient. Through the perturbation generation mechanism, perturbation data is generated according to the perturbation parameters, and the perturbation data is intended to test the sensitivity of the model to changes in the input data. The perturbation data is combined with the historical time series data and input into the model together, and the fusion loss function is used to calculate the loss value, which reflects the performance of the model when it is affected by the perturbation. The fusion loss function containing the perturbation data is back-propagated to obtain the gradient of the model parameters with respect to the perturbation data, i.e., the perturbation gradient. The perturbation gradient indicates the direction of change of the model parameters in reducing the loss caused by the perturbation. In the embodiment of the present application, the adversarial gradient refers to a quantity that combines the original gradient and the perturbation gradient, which is used to consider the sensitivity of the model to perturbations during the optimization process.

[0062] Specifically, according to the loss function value calculated by the forward propagation result, adversarial learning is added after the neural network obtains the gradient through back propagation. That is, the perturbation data r is obtained by the calculation formula of the following symbol function according to the original gradient of the model fusion loss function:

[0063] r=ε*sign(g)

[0064] Among them, ε is the perturbation parameter, g is the original gradient of the fusion loss function of the model, and sign(g) is the result of taking the sign function of the original gradient. The sign function is used to ensure that the direction of the perturbation is consistent with the direction of the gradient. The above perturbation data r is added to the original vector data of the model (i.e., historical time series data), and the forward loss function value (i.e., historical perturbation loss value) is calculated for the obtained vector data, and the perturbed gradient (i.e., perturbation gradient) is obtained by backpropagation, the original weight is restored, and the perturbation gradient is accumulated to the original gradient to obtain the final adversarial gradient.

[0065] In the above steps, the first-order moving average momentum and the second-order moving average momentum are determined according to the adversarial gradient, including: obtaining the first-order momentum exponential decay rate parameter, the second first-order moving average momentum obtained in the second round of training, and the adversarial gradient, wherein the second round of training is the previous round of training of the first round of training; determining the first-order moving average momentum according to the first-order momentum exponential decay rate parameter, the second first-order moving average momentum, and the adversarial gradient; obtaining the second-order momentum exponential decay rate parameter, the second second-order moving average momentum obtained in the second round of training, and the adversarial gradient; determining the second-order moving average momentum according to the second-order momentum exponential decay rate parameter, the second second-order moving average momentum, and the adversarial gradient.

[0066] In the embodiment of the present application, when searching for the optimal parameters of the model according to the adversarial gradient, a projection-based momentum gradient optimizer is used to first calculate the optimal parameters of the model according to the current adversarial gradient g. t Calculate the first-order moving average momentum, the formula is as follows:

[0067] m t =β1*m t-1 +(1-β1)*g t

[0068] Where β1 is the first-order momentum exponential decay rate parameter (between 0 and 1), m t-1 is the first-order moving average momentum of the previous training step (i.e., the second first-order moving average momentum obtained in the second round of training above), with an initial value of 0, m t is the first-order moving average momentum corresponding to the first round of training, that is, the current first-order moving average momentum, where the first round of training can be the current training round, and the second round of training can be the previous training of the current training round. Then calculate the second-order moving average momentum, the formula is as follows:

[0069]

[0070] Where β2 is the second-order momentum exponential decay rate parameter (between 0 and 1), v t-1 is the second-order moving average momentum of the previous training step (i.e., the second second-order moving average momentum obtained in the second round of training above), and its initial value is 0. tis the second-order moving average momentum corresponding to the first round of training, that is, the current second-order moving average momentum. The projection-based model parameter correction value P is calculated based on the first-order momentum and the second-order momentum. t , the formula is as follows:

[0071]

[0072] Among them, ε1 is a weighted adjustment parameter between 0 and 1.

[0073] In the above steps, the parameters of the initial preset model are adjusted according to the parameter correction value, including: obtaining the first parameter of the initial preset model in the first round of training, and obtaining the original gradient of the fusion loss function; when the product of the first parameter and the adversarial gradient is less than the preset parameter, determining the projection of the parameter correction value in the first parameter direction of the fusion loss function; adjusting the first parameter according to the projection of the first parameter and the parameter correction value in the first parameter direction of the fusion loss function and the learning rate; when the product of the first parameter and the adversarial gradient is greater than or equal to the preset parameter, adjusting the first parameter according to the first parameter, the parameter correction value and the learning rate.

[0074] In the embodiment of the present application, taking the first round of training (i.e., the current training round) as an example, the current parameter θ of the deep learning model t , that is, the first parameter mentioned above, the parameter θ for the next training step t+1 Adjust when θ t ·g t When it is less than the adjustment threshold parameter δ1 (i.e. the above preset parameter), the adjustment formula is as follows:

[0075] θ t+1 =θ t -ηΠP t

[0076] Among them, ΠP t YesP t The projection in the direction of the current parameter (i.e., the first parameter) of the fusion loss function, η is the learning rate.

[0077] Otherwise, when θ t ·g t When it is greater than or equal to the adjustment threshold parameter δ1, the adjustment formula is as follows:

[0078] θ t+1 =θ t -ηP t

[0079] According to the above-mentioned gradient optimization method based on projection combined with first-order and second-order moving average momentum, the optimal parameter model of the deep learning model is found.

[0080] When performing model inference, load the preset model obtained from the previous training, perform regression inference on the input time series data, set the model to eval mode, turn off automatic gradient calculation, infer based on the neural network model, and organize the results after identification and prediction to meet the needs of subsequent management decisions and ensure the availability of prediction and recognition results.

[0081] The time series prediction method provided by the embodiment of the present application introduces a new fusion loss function, a projection-based momentum gradient optimizer and an adversarial learning training mechanism while using two linear neural networks with relatively simple structures to predict time series, and designs a new deep learning time series prediction model, which effectively solves the problem of time series prediction in simple neural networks with fewer parameters, and improves the accuracy of prediction and the effect of time series prediction. Specifically, the present application decomposes the time series data into trend series data and remainder series data, and uses two simple linear neural networks to predict them separately. In the deep learning training stage, the fusion loss function of the α average of the smooth L1 loss function and the logarithmic hyperbolic cosine loss function is adopted, which combines the robustness of the former to abnormal processing and the smoothness of the latter to gradient processing, taking into account the sensitivity of the model to small errors and large errors, and can better balance the accuracy and stability of the prediction results, which helps to reduce the risk of overfitting, improve the generalization ability of the model, and enhance the adaptability of the model in different scenarios. At the same time, adversarial learning is added to increase the perturbation consistent with the change in the gradient direction to improve the generalization ability of the model. Based on the idea of ​​knowledge transfer, the projection-based momentum gradient optimizer Adamp is introduced into the linear neural network for structured data time series prediction, and good generalization performance is achieved by combining the projection-based model parameter correction value with the first-order and second-order momentum gradient optimization method, thereby achieving more effective and accurate time series prediction performance as a whole.

[0082] The time series prediction method provided in the embodiment of the present application can be applied to a variety of intelligent time data prediction scenarios, including energy management and scheduling, supply chain and logistics management, operation control, etc. For example, in energy management and scheduling, the method can be used for power load prediction. According to historical power consumption data, combined with factors such as weather and holidays, the future power grid load can be accurately predicted, thereby optimizing power production plans, allocating resources, and improving power grid operation efficiency. It can also be used for natural gas demand prediction: by modeling time series data that affect natural gas demand such as seasonal consumption patterns and climate factors, the supply strategy can be planned in advance. For another example, in supply chain and logistics management, the method can be used for sales forecasting, based on historical sales records and external environmental variables (such as promotional activities), predicting the sales volume of products in future time periods, in order to formulate inventory strategies and reduce backlogs or out-of-stock situations. When predicting logistics demand, time series analysis is used to estimate freight volume, transportation tool usage requirements, etc., to achieve reasonable allocation of logistics resources and cost control. For another example, in operation control, the method can be used for operator network fault prediction, by monitoring the health status of network equipment, predicting potential faults and maintenance requirements, which helps operators take measures in advance to reduce service interruptions and improve network reliability. In operator network traffic forecasting, operators need to accurately predict network traffic in order to reasonably allocate resources, optimize network configuration, and plan future network expansion. Time series forecasting can help predict future data usage patterns, including user activity, application traffic, and network congestion.

[0083] During the experimental verification, the authoritative time series public dataset WSP (the meteorological wind speed prediction dataset of Kaggle, a world-renowned artificial intelligence data science community platform) and PEMS07 (the high-speed traffic flow monitoring dataset of the California Department of Transportation) were used to compare the original DLinear method with this method. The mean square error was used as the regression prediction accuracy evaluation indicator. The results are shown in the following table:

[0084] WSP PEMS07 Original method 7.5775 2.6890 This method 3.8793 0.9940

[0085] It can be seen from the above table that for the two authoritative time series prediction public data sets, the mean square error between the regression prediction value and the true value of this method is smaller than that of the original method. In comparison, this method has a good improvement over the original method and has higher recognition and prediction accuracy.

[0086] Figure 3 is a structural diagram of a time series prediction device according to an embodiment of the present application, such as Figure 3 As shown, the device comprises:

[0087] An acquisition module 30 is used to acquire original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other;

[0088] The prediction module 32 is used to use the upper layer and the lower layer of the model in the preset model to predict the original time series data to obtain the predicted time series data, wherein the lower layer of the model is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper layer of the model is used to predict the trend series data, and the second linear regression neural network in the upper layer of the model is used to predict the remainder sequence data corresponding to the original time series data, wherein the remainder sequence data is obtained through the original time series data and the trend sequence data.

[0089] Through the acquisition module 30 and the prediction module 32 in the above-mentioned time series prediction device, the purpose of improving the time prediction accuracy is achieved, thereby realizing the technical effect of combining trend prediction and residue prediction to enhance the model's processing ability for complex time series data, thereby solving the technical problem that traditional time series prediction methods are difficult to capture long-term dependencies in the data, resulting in poor prediction accuracy.

[0090] The above-mentioned time series prediction device also includes a training module 34, which is used to train a preset model. Specifically, the preset model is trained in the following manner: obtaining historical time series data generated by the data source during data interaction, and historical real label values ​​corresponding to the historical time series data; inputting the historical time series data and the historical real label values ​​into the initial preset model for training to obtain historical output data; inputting the historical time series data and the historical output data into a fusion loss function to obtain a first historical loss value; after determining the first historical loss value, adjusting the parameters of the initial preset model, and predicting the historical time series data according to the initial preset model after the parameters are adjusted to obtain the adjusted historical output data; inputting the historical time series data and the adjusted historical output data into the fusion loss function to obtain a second historical loss value, and continuing to adjust the parameters of the initial preset model when the second historical loss value does not meet the preset conditions, until the loss value corresponding to the historical output data output by the initial preset model after the parameters are adjusted meets the preset conditions, and then stopping the training to obtain the preset model.

[0091] In the above-mentioned time series prediction device, it also includes a determination module 36, which is also used to obtain the data volume of historical time series data; determine the first loss function based on the data volume, historical true label values ​​and historical output data; determine the second loss function based on the data volume, historical true label values, historical output data and logarithmic hyperbolic cosine function; determine the fusion loss function based on the first loss function and the second loss function.

[0092] In the above-mentioned time series prediction device, it also includes an adjustment module 38, which is used to determine the adversarial gradient of the fusion loss function; determine the first-order moving average momentum and the second-order moving average momentum based on the adversarial gradient; determine the parameter correction value of the initial preset model based on the first-order moving average momentum and the second-order moving average momentum; adjust the parameters of the initial preset model based on the parameter correction value.

[0093] In the determination module in the above-mentioned time series prediction device, the determination module is also used to determine the original gradient of the fusion loss function; determine the disturbance data based on the original gradient and the disturbance parameter; calculate the disturbance data and the historical time series data through the fusion loss function to obtain the historical disturbance loss value; backpropagate the fusion loss function after obtaining the historical disturbance loss value to obtain the disturbance gradient; determine the adversarial gradient of the fusion loss function based on the original gradient and the disturbance gradient, wherein the adversarial gradient is the adversarial gradient during the first round of training of the initial preset model.

[0094] In the determination module in the above-mentioned time series prediction device, the determination module is also used to obtain the first-order momentum exponential decay rate parameter, the second first-order moving average momentum obtained during the second round of training, and the adversarial gradient, wherein the second round of training is the previous round of training of the first round of training; determine the first-order moving average momentum based on the first-order momentum exponential decay rate parameter, the second first-order moving average momentum, and the adversarial gradient; obtain the second-order momentum exponential decay rate parameter, the second second-order moving average momentum obtained during the second round of training, and the adversarial gradient; determine the second-order moving average momentum based on the second-order momentum exponential decay rate parameter, the second second-order moving average momentum, and the adversarial gradient.

[0095] In the adjustment module in the above-mentioned time series prediction device, the adjustment module is also used to obtain the first parameter of the initial preset model in the first round of training, and to obtain the original gradient of the fusion loss function; when the product of the first parameter and the adversarial gradient is less than the preset parameter, determine the projection of the parameter correction value in the first parameter direction of the fusion loss function; adjust the first parameter based on the projection of the first parameter and the parameter correction value in the first parameter direction of the fusion loss function and the learning rate; when the product of the first parameter and the adversarial gradient is greater than or equal to the preset parameter, adjust the first parameter based on the first parameter, the parameter correction value and the learning rate.

[0096] It should be noted that Figure 3 The time series forecasting device shown is used to perform Figure 2 The time series prediction method shown in the figure, therefore the relevant explanations in the above time series prediction method are also applicable to the time series prediction device, and will not be repeated here.

[0097] An embodiment of the present application also provides an electronic device, which includes a memory and a processor, wherein the memory is used to store program instructions; the processor is connected to the memory, and is used to execute program instructions to implement the following functions: obtaining original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; using the upper model layer and the lower model layer in the preset model to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network layer in the upper model layer is used to predict the remainder series data corresponding to the original time series data, wherein the remainder series data is obtained through the original time series data and the trend series data.

[0098] It should be noted that the above electronic equipment is used to execute Figure 2 The time series prediction method shown, therefore the relevant explanations in the above time series prediction method are also applicable to the electronic device and will not be repeated here.

[0099] An embodiment of the present application also provides a non-volatile storage medium, which includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the following time series prediction method by running the computer program: obtaining original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; using the upper model layer and the lower model layer in the preset model to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network layer in the upper model layer is used to predict the remainder series data corresponding to the original time series data, wherein the remainder series data is obtained through the original time series data and the trend series data.

[0100] It should be noted that the above non-volatile storage medium is used to execute Figure 2 The time series prediction method shown, therefore the relevant explanations in the above time series prediction method are also applicable to the non-volatile storage medium, and will not be repeated here.

[0101] An embodiment of the present application also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the time series prediction method in each embodiment of the present application.

[0102] The embodiments of the present application also provide a computer program, which, when executed by a processor, implements the steps of the time series prediction method in each embodiment of the present application.

[0103] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0104] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0105] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0106] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0107] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0109] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A time series prediction method, characterized in that: include: Acquire original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; The upper model layer and the lower model layer in the preset model are used to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network in the upper model layer is used to predict the remainder sequence data corresponding to the original time series data, wherein the remainder sequence data is obtained through the original time series data and the trend series data.

2. The method according to claim 1, characterized in that The preset model is trained in the following way: Acquire historical time series data generated by the data source during data interaction, and historical real label values ​​corresponding to the historical time series data; Inputting the historical time series data and the historical true label values ​​into an initial preset model for training to obtain historical output data; Inputting the historical time series data and the historical output data into a fusion loss function to obtain a first historical loss value; After determining the first historical loss value, adjusting the parameters of the initial preset model, predicting the historical time series data according to the initial preset model after the parameter adjustment, and obtaining adjusted historical output data; The historical time series data and the adjusted historical output data are input into the fusion loss function to obtain a second historical loss value. When the second historical loss value does not meet the preset conditions, the parameters of the initial preset model continue to be adjusted until the loss value corresponding to the historical output data output by the initial preset model after the parameter adjustment meets the preset conditions, and the training is stopped to obtain the preset model.

3. The method according to claim 2, characterized in that The fusion loss function is determined by: Obtaining the data volume of the historical time series data; Determining a first loss function according to the data volume, the historical true label value and the historical output data; Determine a second loss function according to the data volume, the historical true label value, the historical output data and the logarithmic hyperbolic cosine function; The fusion loss function is determined according to the first loss function and the second loss function.

4. The method according to claim 2, characterized in that: Adjusting the parameters of the initial preset model includes: Determining the adversarial gradient of the fusion loss function; Determining a first-order moving average momentum and a second-order moving average momentum according to the adversarial gradient; Determining a parameter correction value of the initial preset model according to the first-order moving average momentum and the second-order moving average momentum; The parameters of the initial preset model are adjusted according to the parameter correction values.

5. The method according to claim 4, characterized in that Determining the adversarial gradient of the fusion loss function includes: Determining the original gradient of the fusion loss function; Determining disturbance data according to the original gradient and the disturbance parameter; Calculating the disturbance data and the historical time series data by using the fusion loss function to obtain a historical disturbance loss value; Back-propagating the fusion loss function after obtaining the historical perturbation loss value to obtain the perturbation gradient; The adversarial gradient of the fusion loss function is determined based on the original gradient and the perturbation gradient, wherein the adversarial gradient is the adversarial gradient when the initial preset model is trained for the first round.

6. The method according to claim 5, characterized in that Determining a first-order moving average momentum and a second-order moving average momentum according to the adversarial gradient includes: Obtaining a first-order momentum exponential decay rate parameter, a second first-order moving average momentum obtained during a second round of training, and the adversarial gradient, wherein the second round of training is a previous round of training of the first round of training; Determining the first-order moving average momentum according to the first-order momentum exponential decay rate parameter, the second first-order moving average momentum and the adversarial gradient; Obtaining a second-order momentum exponential decay rate parameter, a second second-order moving average momentum obtained during a second round of training, and the adversarial gradient; The second-order moving average momentum is determined according to the second-order momentum exponential decay rate parameter, the second second-order moving average momentum and the adversarial gradient.

7. The method according to claim 4, characterized in that Adjusting the parameters of the initial preset model according to the parameter correction value includes: Obtaining a first parameter of the initial preset model during a first round of training, and obtaining an original gradient of the fusion loss function; When the product of the first parameter and the adversarial gradient is less than a preset parameter, determining a projection of the parameter correction value in the first parameter direction of the fusion loss function; Adjust the first parameter according to the first parameter, the projection of the parameter correction value in the first parameter direction of the fusion loss function, and the learning rate; When the product of the first parameter and the adversarial gradient is greater than or equal to a preset parameter, the first parameter is adjusted according to the first parameter, the parameter correction value and the learning rate.

8. A time series prediction device, characterized in that: include: An acquisition module, used to acquire original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; A prediction module is used to use the upper layer and lower layer of the model in the preset model to predict the original time series data to obtain predicted time series data, wherein the lower layer of the model is used to determine the trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper layer of the model is used to predict the trend series data, and the second linear regression neural network in the upper layer of the model is used to predict the remainder sequence data corresponding to the original time series data, wherein the remainder sequence data is obtained through the original time series data and the trend sequence data.

9. An electronic device, characterized in that: include: A memory for storing program instructions; A processor is connected to the memory and is used to execute program instructions to implement the following functions: obtaining original time series data to be predicted, wherein the original time series data is time series data generated when different data sources interact with each other; using an upper model layer and a lower model layer in a preset model to predict the original time series data to obtain predicted time series data, wherein the lower model layer is used to determine trend series data corresponding to the original time series data, the first linear regression neural network layer in the upper model layer is used to predict the trend series data, and the second linear regression neural network in the upper model layer is used to predict the remainder series data corresponding to the original time series data, wherein the remainder series data is obtained through the original time series data and the trend series data.

10. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored computer program, wherein the device where the non-volatile storage medium is located executes the time series prediction method according to any one of claims 1 to 7 by running the computer program.

11. A computer program product comprising computer instructions, characterized in that: When the computer instructions are executed by a processor, the time series prediction method described in any one of claims 1 to 7 is implemented.

Images