Data prediction method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to the technical field of data processing, in particular to a data prediction method and device, equipment and a storage medium, and aims at achieving rapid prediction of time series data. The method comprises the steps of performing data processing on historical time series data to obtain a corresponding time series data learning sample; judging whether the time series data learning sample is periodic data or not through a preset function; when the time series data learning sample is periodic data, at least one hyper-parameter combination is determined, and the hyper-parameter combination is used in the data prediction model; training the data prediction model corresponding to the at least one hyper-parameter combination by using a time series data learning sample to obtain at least one trained data prediction model; through error degree calculation, selecting a target data prediction model from the at least one trained data prediction model; and performing data prediction on the input time series data through the target data prediction model to obtain a data prediction result.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and specifically, to a data prediction method, device, equipment and storage medium. Background Art

[0002] Trend prediction of time series data is an important research direction in the field of artificial intelligence today. This technology is applied to various fields that require data analysis. In the existing technology, when predicting the trend of periodic data, technicians need to analyze the fluctuation and development rules of historical data, set appropriate algorithm parameters, and continuously adjust the parameters to compare the prediction model effect, and then apply the intelligent algorithm to the trend prediction of time series data.

[0003] In the existing technology, historical data must be analyzed and then the corresponding algorithm parameters are set. This has a high professional threshold and consumes a lot of manpower and material resources, which affects the widespread application of trend prediction algorithms. Summary of the invention

[0004] The embodiments of the present application provide a data prediction method, apparatus, device and storage medium, aiming to achieve rapid prediction of time series data.

[0005] A first aspect of an embodiment of the present application provides a data prediction method, the method comprising: Perform data processing on historical time series data to obtain corresponding time series data learning samples; Determine whether the time series data learning sample is periodic data by using a preset function; In the case where the time series data learning sample is periodic data, determining at least one hyperparameter combination according to period information of the time series data learning sample, wherein the hyperparameter combination is used in a preset data prediction model; Using the time series data learning sample, training the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model; Selecting a target data prediction model from at least one trained data prediction model by error calculation; The target data prediction model is used to perform data prediction on the input time series data to obtain a data prediction result.

[0006] Optionally, the performing data processing on the historical time series data to obtain corresponding time series data learning samples includes: Performing alignment processing on the timestamps of the historical time series data; Filling missing data in the historical time series data; The historical time series data is processed to eliminate abnormal points to obtain the time series data learning samples.

[0007] Optionally, judging whether the time series data learning sample is periodic data by using a preset function includes: Calculating the autocorrelation value between the values ​​of every other preset time period in the time series data learning sample by the preset function; In the case where there is a positive correlation between the autocorrelation values, determining that the time series data learning sample is periodic data; When the autocorrelation value approaches zero, it is determined that the time series data learning sample does not have periodicity.

[0008] Optionally, determining at least one hyperparameter combination according to the period information of the time series data learning sample includes: reading the time series data learning sample and the period information; Read a list of possible hyperparameter values ​​corresponding to the data prediction model; According to the period information, each parameter value in the list of possible hyperparameter values ​​is combined to obtain the at least one hyperparameter combination.

[0009] Optionally, the using the time series data learning sample to train the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model includes: Splitting the time series data learning sample into at least one combination of a training data set and a test data set; The data prediction model corresponding to the at least one hyperparameter combination is trained using at least one training data set to obtain at least one data prediction model.

[0010] Optionally, splitting the time series data learning sample into at least one combination of a training data set and a test data set includes: Preset a fixed-size window; The window is slid to split the time series data learning sample into at least one combination of the training data set and the test data set.

[0011] Optionally, the selecting a target data prediction model from at least one trained data prediction model through error degree calculation includes: Calculating the error percentage of the data prediction model by splitting a test data set from the time series data learning sample; Determine the average error of each data prediction model on each data set combination according to the error percentage; The data prediction model with the smallest average error is used as the target prediction model.

[0012] A second aspect of an embodiment of the present application provides a data prediction device, the device comprising: The data processing module is used to process the historical time series data to obtain the corresponding time series data learning samples; A data judgment module, used to judge whether the time series data learning sample is periodic data through a preset function; A hyperparameter combination determination module, used for determining at least one hyperparameter combination according to the period information of the time series data learning sample when the time series data learning sample is periodic data, and the hyperparameter combination is used in a preset data prediction model; A model training module, used to train the data prediction model corresponding to the at least one hyperparameter combination using the time series data learning sample to obtain at least one trained data prediction model; A target prediction model determination module, configured to select a target data prediction model from at least one trained data prediction model by calculating an error degree; The prediction result acquisition module is used to perform data prediction on the input time series data through the target data prediction model to obtain the data prediction result.

[0013] Optionally, the data processing module includes: A first data processing submodule, configured to perform alignment processing on the timestamps of the historical time series data; A second data processing submodule is used to fill missing data in the historical time series data; The third data processing submodule is used to perform outlier elimination processing on the historical time series data to obtain the time series data learning samples.

[0014] Optionally, the data determination module includes: An autocorrelation value calculation submodule, used to calculate the autocorrelation value between the values ​​of every other preset time period in the time series data learning sample by using the preset function; A first judgment submodule, configured to determine that the time series data learning sample is periodic data when there is a positive correlation between the autocorrelation values; The second judgment submodule is used to determine that the time series data learning sample does not have periodicity when the autocorrelation value approaches zero.

[0015] Optionally, the hyper-parameter combination determination module includes: an information reading submodule, configured to read the time series data learning sample and the period information; A list reading submodule, used to read a list of possible hyperparameter values ​​corresponding to the data prediction model; The hyperparameter combination determination submodule is used to combine the parameter values ​​in the hyperparameter possible value list according to the period information to obtain the at least one hyperparameter combination.

[0016] Optionally, the model training module includes: A data set division submodule, used to divide the time series data learning sample into at least one combination of a training data set and a test data set; The model training submodule is used to train the data prediction model corresponding to the at least one hyperparameter combination through at least one training data set to obtain at least one data prediction model.

[0017] Optionally, the data set division submodule includes: The window setting submodule is used to preset a window of a fixed size; The window sliding submodule is used to slide the window to split the time series data learning sample into at least one combination of the training data set and the test data set.

[0018] Optionally, the target prediction model determination module includes: An error calculation submodule, used to calculate the error percentage of the data prediction model by using a test data set separated from the time series data learning sample; An average error calculation submodule, used to determine the average error of each of the data prediction models on each data set combination according to the error percentage; The target prediction model determination submodule is used to use the data prediction model with the smallest average error as the target prediction model.

[0019] A third aspect of an embodiment of the present application provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps in the method described in the first aspect of the present application are implemented.

[0020] A fourth aspect of an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect of the present application are implemented.

[0021] The data prediction method provided in the present application is used to process historical time series data to obtain corresponding time series data learning samples; a preset function is used to determine whether the time series data learning samples are periodic data; when the time series data learning samples are periodic data, at least one hyperparameter combination is determined based on the periodic information of the time series data learning samples, and the hyperparameter combination is used in a preset data prediction model; the time series data learning samples are used to train the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model; a target data prediction model is selected from at least one trained data prediction model through error calculation; and the input time series data is predicted through the target data prediction model to obtain a data prediction result.

[0022] In this method, historical time series data is collected and processed to obtain time series data learning samples. Through the periodic information in the periodic time series data, multiple hyperparameter combinations of the data prediction model are determined, and the data prediction models with different hyperparameter combinations are trained separately. Then, the error degree of each trained data prediction model is calculated, and then the optimal data prediction model is selected. The time series data is predicted by the optimal data prediction model. Through this automatic parameter adjustment method, the optimal prediction model can be customized for different data sets, saving manpower and material resources and realizing the rapid prediction of time series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0024] Figure 1 is a flow chart of a data prediction method proposed in one embodiment of the present application; Figure 2 is a schematic diagram of the ACF value proposed in one embodiment of the present application; Figure 3 This is a schematic diagram of data set division proposed in an embodiment of the present application; Figure 4 This is a flow chart of automatic parameter adjustment of a trend prediction algorithm proposed in an embodiment of the present application; Figure 5 It is a schematic diagram of the flow of a data prediction method proposed in an embodiment of the present application; Figure 6 It is a time series data rising and falling trend prediction chart proposed in an embodiment of the present application; Figure 7It is a one-day period trend prediction chart of time series data proposed in an embodiment of the present application; Figure 8 This is a trend prediction result diagram proposed in an embodiment of the present application; Fig. 9 is a schematic diagram of a data prediction device proposed in an embodiment of the present application; Fig.10 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] refer to Figure 1 , Figure 1 is a flow chart of a data prediction method proposed in one embodiment of the present application. Figure 1 As shown, the method comprises the following steps: S11: Process the historical time series data to obtain corresponding time series data learning samples.

[0027] In this embodiment, the historical time series data is the time series data recorded by each platform, and the time series data learning samples are data samples obtained after processing the time series data.

[0028] In this embodiment, in order to predict time series data, it is first necessary to perform various data processing such as missing data filling, granularity alignment, and discrete point elimination on the historical time series data to obtain a continuous, standardized, and high-quality time series data learning sample.

[0029] In this embodiment, the specific steps of processing the historical time series data to obtain the corresponding time series data learning samples include: S11-1: Align the timestamps of the historical time series data.

[0030] In this embodiment, the timestamps of the historical time series data are aligned to ensure that the data points in the historical time series data maintain the same statistical frequency granularity.

[0031] For example, align the timestamp upward to the whole minute, ensuring that the timestamp of all data is a multiple of the whole minute.

[0032] S11-2: Filling missing data in the historical time series data.

[0033] In this embodiment, missing data is filled in the historical time series data to ensure the integrity of the historical time series data.

[0034] In this embodiment, if there are missing breakpoints in the historical time series data, the missing data points are smoothly filled based on the data before and after the breakpoints.

[0035] For example, suppose that there are several data points missing between the mth data value Vm and the nth data value Vn, let: (1) The data filled between m and n are: (2) S11-3: Perform outlier elimination processing on the historical time series data to obtain the time series data learning samples.

[0036] In this embodiment, abnormal points in historical time series data are eliminated to ensure data stability and obtain time series data learning samples.

[0037] For example, the Isolation Forest algorithm and 3 times the standard deviation are used to detect and remove abnormal outliers in time series data.

[0038] S12: Determine whether the time series data learning sample is periodic data through a preset function.

[0039] In this embodiment, the periodic data refers to data that changes periodically.

[0040] In this embodiment, it is first necessary to determine whether the time series data learning sample is periodic data. A preset function is used to determine whether the time series data learning sample is periodic data. Only periodic data can be used for data prediction. Non-periodic data does not have periodic patterns and the prediction process cannot be executed.

[0041] In this embodiment, the specific steps of determining whether the time series data learning sample is periodic data by using a preset function include: S12-1: Calculate the autocorrelation value between the values ​​in every preset time period in the time series data learning sample by using the preset function.

[0042] In this embodiment, the autocorrelation value represents the correlation between the values ​​observed at intervals of time.

[0043] In this embodiment, in order to determine whether the time series data in the time series data learning sample is periodic data, the autocorrelation value between the numerical values ​​of each preset time period in the time series data learning sample is first calculated through a preset function, and then the correlation between the numerical values ​​observed at every time interval is determined.

[0044] S12-2: When there is a positive correlation between the autocorrelation values, determine that the time series data learning sample is periodic data.

[0045] In this embodiment, when there is a positive correlation between the correlation values, it means that the data in the time series data learning sample are similar at intervals, and the time series data learning sample is determined to be periodic data.

[0046] S12-3: When the autocorrelation value approaches zero, determine that the time series data learning sample does not have periodicity.

[0047] In this embodiment, when the autocorrelation value approaches zero, it means that there is no correlation between the data in the time series data learning sample. At this time, it is determined that the time series data learning sample does not have periodicity.

[0048] For example, the ACF (Autocorrelation Function) autocorrelation function is used to calculate the autocorrelation value of the processed time series data. Assuming that the time series data has a fluctuation pattern with a period interval of h, Xt and Xt-h represent the current value and the value of the previous period respectively, and the autocorrelation function ACF is used to measure the correlation between the observed values ​​Xt and Xt-h every h time units in the time series. The autocorrelation value ACF calculation formula is: (3) The covariance calculation formula is:

[0049] Standard deviation calculation formula:

[0050] Here, x and y are observations in the time series data.

[0051] Assuming that the statistical granularity of the time series data is 1, calculate the autocorrelation value ACF(h) corresponding to h=1,2…n respectively. The ACF value can be used to describe the correlation between two observations of the time series data at each time interval h as follows: Positive correlation: The ACF value is between 0 and 1, indicating that the data in adjacent periods rise or fall synchronously. The closer the rise and fall rates are, the closer the ACF value is to 1. Negative correlation: The ACF value is between 0 and -1, indicating that the data rise and fall trends in adjacent periods are exactly opposite. When one period rises, the other period falls. The closer the absolute values ​​of the rise and fall rates are, the closer the ACF value is to -1. Uncorrelated: When the rising and falling pace of two periodic data is chaotic (sometimes rising and falling synchronously, and sometimes going in opposite directions), it can be positive or negative. The more chaotic the pace is, the closer it is to 0.

[0052] In this embodiment, the periodic characteristics of the time series data are determined by the value of ACF(h).

[0053] In this embodiment, reference Figure 2 , Figure 2 is a schematic diagram of the ACF value proposed in an embodiment of the present application, such as Figure 2 As shown in , if there is a positive correlation between the autocorrelation values, the current time series data is determined to be periodic, and the h value corresponding to the autocorrelation value closest to 1 is extracted as the possible period of the time series data. The positive peak position of the autocorrelation value is usually used to determine the possible period. Figure 2 The ACF value of a certain time series data shown in the figure, in addition to ACF (h=0) representing the time series data itself, it can be seen that when h=1 day, the autocorrelation ACF value is the largest and closest to 1, which means that the time series data fluctuates with a periodic frequency of 1 day.

[0054] In this embodiment, if all ACF(h) values ​​of the current time series data tend to 0, the time series data is determined to be stationary, that is, the data has no obvious periodic pattern but randomly oscillates around a fixed value. The process of this patent method determines this situation as unsuitable for trend prediction, and the process ends directly. In this case, the time series data is suitable for anomaly detection using methods such as outlier detection.

[0055] In this embodiment, the h value can be adjusted according to the actual usage scenario.

[0056] S13: When the time series data learning sample is periodic data, determine at least one hyperparameter combination according to the period information of the time series data learning sample, and the hyperparameter combination is used in a preset data prediction model.

[0057] In this embodiment, the data prediction model is a model for trend prediction of time series data. It is a model trained by using historical time series data as a training set and a basic deep learning model as a basic architecture. The deep learning model can use any neural network model, and there is no restriction here. The model has the ability to predict time series data. Hyperparameters are parameters in the data prediction model. These parameters do not change with training and are pre-set parameters. There are multiple combinations of multiple hyperparameters in the prediction model.

[0058] In this embodiment, when the time series data learning sample is periodic data, at least one hyperparameter combination is determined based on the periodic information of the time series data learning sample, and the periodic information characterizes the periodic characteristics of the time series data learning sample, and different periodic characteristics are adapted to different hyperparameter combinations. Values ​​are taken from a pre-stored list of possible values ​​of the algorithm hyperparameters for free combination until all possible hyperparameter combinations are completed.

[0059] In this embodiment, the specific steps of determining at least one hyperparameter combination according to the period information of the time series data learning sample include: S13-1: Read the timing data learning sample and the period information.

[0060] In this embodiment, the periodic information is time series feature information extracted from the time series data learning sample through the autocorrelation function, which represents the fluctuation pattern of the data.

[0061] In this embodiment, firstly, the time series data learning sample and the period information corresponding to the time series data learning sample are read.

[0062] S13-2: Read a list of possible hyperparameter values ​​corresponding to the data prediction model.

[0063] In this embodiment, the list of possible hyperparameter values ​​is a list of possible values ​​of important hyperparameters in the data prediction model.

[0064] In this embodiment, a list of possible hyperparameter values ​​corresponding to the data prediction model is read.

[0065] For example, the possible values ​​of the hyperparameters are listed in the following table:

[0066] Table 1 S13-3: According to the period information, combine the parameter values ​​in the list of possible hyperparameter values ​​to obtain the at least one hyperparameter combination.

[0067] In this embodiment, the fluctuation period of the time series data is determined based on the obtained period information, and then the parameter values ​​in the list of possible hyperparameter values ​​are combined to obtain at least one hyperparameter combination.

[0068] In this embodiment, different hyperparameter combinations can be adapted for time series data with different periodic characteristics (daily, weekly, monthly, etc.) when performing trend forecasting. This can be verified and summarized based on industry characteristics and actual data samples, and a list of possible values ​​for each periodic characteristic can be customized and preset. Each possible hyperparameter combination is automatically repeated, that is, each possible hyperparameter value is traversed to generate multiple hyperparameter combinations.

[0069] For example, the distribution probabilities of three periodic curve coefficients and the probability distributions of the growth rate changes of the two trend lines before and after the four overall trend segmentation points are preset, and 3*4=12 parameter combinations will be automatically traversed.

[0070] S14: Using the time series data learning samples, train the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model.

[0071] In this embodiment, the time series data learning sample is used to train the data prediction model corresponding to at least one hyperparameter combination to obtain at least one trained data prediction model. During training, the time series data learning sample is first split into a training data set and a test data set. The training data set is used to train the model, and the test data set is used to verify the prediction effect of the model and calculate the error degree.

[0072] In this embodiment, the specific steps of using the time series data learning sample to train the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model include: S14-1: Split the time series data learning sample into at least one combination of a training data set and a test data set.

[0073] In this embodiment, when training the data prediction model, the time series data learning sample is first split into a training data set and a test data set. Each split can split different data into different training data sets and test data sets, thereby obtaining a variety of combinations of training data sets and test data sets.

[0074] In this embodiment, the specific steps of splitting the time series data learning sample into at least one combination of a training data set and a test data set include: S14-2: Preset a window of a fixed size.

[0075] In this embodiment, a window of a fixed size is preset, and the window can select data by sliding.

[0076] S14-3: Slide the window to split the time series data learning sample into at least one combination of the training data set and the test data set.

[0077] In this embodiment, the time series data learning samples are split into a combination of multiple training data sets and test data sets through a sliding window.

[0078] refer to Figure 3 , Figure 3 is a schematic diagram of data set division proposed in an embodiment of the present application, such as Figure 3 As shown in the figure, in order to improve the generalization ability of the prediction model and avoid overfitting of the prediction model, which leads to too high false alarm rate when using the prediction model for time series data anomaly detection, the time series data needs to be randomly sliced ​​and divided into multiple training data sets and test data sets. The training set is used to train the generated model, and the test set is used to verify the generalization ability and accuracy of the model.

[0079] To avoid overfitting, the time series data is divided into a training set and a test set using a sliding window backtesting verification method. The training set is used for model training, and the test set is used to verify model performance. The sliding window technology ensures that the model has good adaptability to data in different time periods by continuously adjusting the distribution of training data and test data. This method is different from the traditional random segmentation of data sets and can ensure the temporal continuity and dependency of time series data.

[0080] S14-2: Train the data prediction model corresponding to the at least one hyperparameter combination using at least one training data set to obtain at least one data prediction model.

[0081] In this embodiment, the data prediction model of each hyperparameter combination is trained by each training data set to obtain multiple data prediction models. Each divided training data set is trained in parallel using each hyperparameter combination, and the training is cyclically trained until each training data set uses each hyperparameter combination to train a data prediction model.

[0082] When training the model, each hyperparameter combination is trained in parallel using the divided training set. During the data analysis process, according to the general time series data analysis and prediction framework, the characteristics of the time series data are decomposed into a combination of multiple influencing factors, including long-term trends, cyclical fluctuations, external interference factors (such as holidays, etc.), and random residuals. Each factor is processed independently to ensure that the model can fully consider various data characteristics during the analysis process.

[0083] For example, three training data sets are divided, and there are 12 hyperparameter combinations, then 36 data prediction models are trained.

[0084] S15: Select a target data prediction model from at least one trained data prediction model through error calculation.

[0085] In this embodiment, the error degree calculation is to determine the error degree between the prediction of the data prediction model and the actual data.

[0086] In this embodiment, the error between the data predicted by each data prediction model and the data in the test data set is calculated using the test data set to obtain the error of each data prediction model, and then the average error of each data prediction model on all data sets is calculated, and then the data prediction model with the smallest error is selected as the target data prediction model.

[0087] In this embodiment, the specific steps of selecting a target data prediction model from at least one trained data prediction model by calculating the error degree include: S15-1: Calculate the error percentage of the data prediction model by splitting a test data set from the time series data learning sample.

[0088] In this embodiment, the error percentage of the data prediction model is calculated by splitting the test data set from the time series data learning sample.

[0089] S15-2: Determine the average error of each data prediction model on each data set combination based on the error percentage.

[0090] In this embodiment, after obtaining the error percentage of the data prediction model on each data set, the average error of each data prediction model on each data set combination is determined. The average error can reflect the actual prediction level of a data prediction model.

[0091] S15-3: Use the data prediction model with the smallest average error as the target prediction model.

[0092] In this embodiment, the data prediction model with the smallest average error is used as the target prediction model. The error between the data predicted by this model and the data in the actual test set is the smallest, so the target prediction model predicts the most accurately.

[0093] For example, the test set data corresponding to each training set is used to calculate the error percentage of each prediction model, and then the average error of the prediction model obtained by each hyperparameter combination on all partitioned data set combinations is calculated. The model with the smallest error represents the lowest degree of deviation from the actual data points, and this hyperparameter combination can be determined as the optimal hyperparameter combination.

[0094] The error calculation formula is:

[0095] Among them, y represents the predicted value, Represents the observed value.

[0096] The above formula calculates the absolute error between each observation and the predicted value, adds up all the absolute error values ​​to get the total error, and divides the total error by the sum of all observations to get the ratio of error to total value.

[0097] In this example, using this evaluation score, the performance of the model can be expressed as a percentage of mean absolute error, which is very useful for interpreting and understanding the actual impact of the error. For example, if the score is 15%, it means that the model's predicted values ​​deviate from the actual values ​​by 15% on average. Compared to metrics such as Mean Squared Error (MSE), which are more affected by large error values, this method is less affected by large errors because it only considers the absolute value of the error. This method has an advantage over Mean Absolute Percentage Error (MAPE) because MAPE cannot handle when the actual observation value is 0, while this method does not have this problem.

[0098] refer to Figure 4 , Figure 4 is a flowchart of automatic parameter adjustment of a trend prediction algorithm proposed in an embodiment of the present application, such as Figure 4 As shown, first prepare the time series data, read the periodic characteristics of the time series data, then read the list of possible values ​​of the key hyperparameters, and then generate hyperparameter combinations, then divide the training set into the test set, cyclically train each parameter combination, and then evaluate the score of each model, calculate the average score, and then select the optimal parameters.

[0099] In this embodiment, through the above method, the data prediction model corresponding to the optimal parameter combination is used as the target prediction model.

[0100] S16: Performing data prediction on the input time series data through the target data prediction model to obtain a data prediction result.

[0101] In this embodiment, the target data prediction model is used to predict the input time series data to obtain a data prediction result.

[0102] For example, it is possible to predict the degradation of indicator data such as the CPU usage of the infrastructure, the number of Paas connections, and the duration monitoring indicators of business processing in time-series periodic indicator data in the monitoring scenario.

[0103] In this embodiment, by determining the optimal data prediction model through the above method, future time series data can be predicted quickly and accurately.

[0104] refer to Figure 5 , Figure 5 : is a flow chart of a data prediction method proposed in an embodiment of the present application. Figure 5As shown, first input the time series data, perform sample processing on the time series data, then calculate the autocorrelation value of the data, determine the characteristics of the time series data, and when the time series data is periodic data, generate a hyperparameter combination based on the information of the time series data, and then divide the time series data into a training set and a test set, train the model and evaluate the score, and then generate the optimal parameter combination. When the time series data is stationary data, stop executing.

[0105] In another embodiment of the present application, 15 days (2024-01-01 00:00:00 ~ 2024-01-15 23:59:59) of time series data samples are input for trend prediction, such as Figure 4 As shown, Figure 4 A schematic diagram of original values ​​of time series data proposed in an embodiment of the present application.

[0106] In this embodiment, since the sample quality is good and no missing data is found, the data processing step is automatically skipped and feature identification is entered. By comparing the ACF values ​​corresponding to various periodic intervals, it is found that when the periodic interval h=1 day, the corresponding autocorrelation value ACF(h) value is the largest, so it is determined that the periodic feature of the time series data is 1 day. Figure 2 The fluctuation law of the original data can be known.

[0107] In this embodiment, the prediction result of the rising and falling trend of the time series data is as follows: Figure 6 , Figure 6 is a time series data up and down trend prediction diagram proposed in an embodiment of the present application, such as Figure 6 As shown, it is found that there is basically no fluctuation, that is, there is no upward or downward trend in the time series data.

[0108] In this embodiment, it has been identified that the period of the time series data is 1 day, and the periodic fluctuation law of the time series data within 1 day is predicted as follows: Figure 7 As shown, Figure 7 This is a one-day periodic trend prediction chart for time series data proposed in an embodiment of the present application.

[0109] In this embodiment, reference Figure 8 , Figure 8 is a trend prediction result diagram proposed in an embodiment of the present application, such as Figure 8As shown. Use 15 days of samples (2024-01-01 00:00:00 ~ 2024-01-15 23:59:59) for training, and then predict the trend line of one day (2024-01-16 00:00:00 ~ 2024-01-16 23:59:59). Then use the actual data on January 16 to superimpose the prediction line and the prediction threshold tolerance interval for verification. It can be seen that except for a few discrete points, most of the actual data falls within the prediction tolerance interval. Here, this method considers this to be a normal phenomenon. The appropriate threshold interval should not be too wide to cause too many missed alarms, nor too narrow to cause too many false alarms. It can be used in conjunction with alarm conditions when using the prediction model for time series data anomaly detection in the future, for example: an alarm will only be issued when three consecutive data points exceed the threshold tolerance interval.

[0110] In the above embodiments of the present application, the autocorrelation function is used to extract the periodic characteristics of the time series data, and the number of times can also be used to extract the periodic characteristics in other ways, such as Fourier function, as long as the periodic law of the time series data can be automatically decomposed. When generating hyperparameter combinations in the automatic parameter adjustment link, the multiplication mode is used to traverse all possible parameter combinations, and the addition mode or other random modes can be used to generate hyperparameter combinations according to actual conditions. When evaluating the error degree of the model, the error degree algorithm can be replaced by other algorithms, as long as the degree of deviation between the actual data and the prediction model can be evaluated, or the accuracy score can be used to judge the ability of the model.

[0111] In the above embodiments of the present application, the selection of model hyperparameters is carried out through adaptive selection of different data features, which can effectively cope with various periodic time series data, save labor costs, and through parallel training and verification, the optimal parameter configuration can be quickly found in multiple hyperparameter combinations to improve prediction efficiency. Verification ensures the prediction ability of the model in different time periods and avoids the overfitting problem of a single model. Automatically customize the optimal prediction model according to different data sets, reduce the tediousness and difficulty of parameter adjustment, and improve the speed and accuracy of time series data prediction.

[0112] Based on the same inventive concept, an embodiment of the present application provides a data prediction device. Fig. 9 , Fig. 9 FIG. 9 is a schematic diagram of a data prediction device 900 proposed in an embodiment of the present application. Fig. 9 As shown, the device comprises: The data processing module 901 is used to process the historical time series data to obtain corresponding time series data learning samples; A data judgment module 902 is used to judge whether the time series data learning sample is periodic data through a preset function; A hyperparameter combination determination module 903 is used to determine at least one hyperparameter combination according to the period information of the time series data learning sample when the time series data learning sample is periodic data, and the hyperparameter combination is used in a preset data prediction model; A model training module 904 is used to train the data prediction model corresponding to the at least one hyperparameter combination using the time series data learning sample to obtain at least one trained data prediction model; A target prediction model determination module 905 is used to select a target data prediction model from at least one trained data prediction model by calculating the error degree; The prediction result obtaining module 906 is used to perform data prediction on the input time series data through the target data prediction model to obtain data prediction results.

[0113] Optionally, the data processing module includes: A first data processing submodule, configured to perform alignment processing on the timestamps of the historical time series data; A second data processing submodule is used to fill missing data in the historical time series data; The third data processing submodule is used to perform outlier elimination processing on the historical time series data to obtain the time series data learning samples.

[0114] Optionally, the data determination module includes: An autocorrelation value calculation submodule, used to calculate the autocorrelation value between the values ​​of every other preset time period in the time series data learning sample by using the preset function; A first judgment submodule, configured to determine that the time series data learning sample is periodic data when there is a positive correlation between the autocorrelation values; The second judgment submodule is used to determine that the time series data learning sample does not have periodicity when the autocorrelation value approaches zero.

[0115] Optionally, the hyper-parameter combination determination module includes: an information reading submodule, configured to read the time series data learning sample and the period information; A list reading submodule, used to read a list of possible hyperparameter values ​​corresponding to the data prediction model; The hyperparameter combination determination submodule is used to combine the parameter values ​​in the hyperparameter possible value list according to the period information to obtain the at least one hyperparameter combination.

[0116] Optionally, the model training module includes: A data set division submodule, used to divide the time series data learning sample into at least one combination of a training data set and a test data set; The model training submodule is used to train the data prediction model corresponding to the at least one hyperparameter combination through at least one training data set to obtain at least one data prediction model.

[0117] Optionally, the data set division submodule includes: The window setting submodule is used to preset a window of a fixed size; The window sliding submodule is used to slide the window to split the time series data learning sample into at least one combination of the training data set and the test data set.

[0118] Optionally, the target prediction model determination module includes: An error calculation submodule, used to calculate the error percentage of the data prediction model by using a test data set separated from the time series data learning sample; An average error calculation submodule, used to determine the average error of each of the data prediction models on each data set combination according to the error percentage; The target prediction model determination submodule is used to use the data prediction model with the smallest average error as the target prediction model.

[0119] Based on the same inventive concept, another embodiment of the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the data prediction method as described in any of the above embodiments of the present application.

[0120] Based on the same inventive concept, another embodiment of the present application provides an electronic device, Fig.10 It is a schematic diagram of an electronic device 1000 proposed in one embodiment of the present application, including a memory 1002, a processor 1001 and a computer program stored in the memory and executable on the processor, and the processor, when executed, implements the steps in the data prediction method described in any of the above embodiments of the present application.

[0121] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0122] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0123] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0124] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0128] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0129] The data prediction method, device, equipment and storage medium provided by the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data prediction method, characterized in that: The method comprises: Perform data processing on historical time series data to obtain corresponding time series data learning samples; Determine whether the time series data learning sample is periodic data by using a preset function; In the case where the time series data learning sample is periodic data, determining at least one hyperparameter combination according to period information of the time series data learning sample, wherein the hyperparameter combination is used in a preset data prediction model; Using the time series data learning sample, training the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model; Selecting a target data prediction model from at least one trained data prediction model by error calculation; The target data prediction model is used to perform data prediction on the input time series data to obtain a data prediction result.

2. The data prediction method according to claim 1, characterized in that: The data processing of the historical time series data to obtain the corresponding time series data learning samples includes: Performing alignment processing on the timestamps of the historical time series data; Filling missing data in the historical time series data; The historical time series data is processed to eliminate abnormal points to obtain the time series data learning samples.

3. The data prediction method according to claim 1, characterized in that: The determining whether the time series data learning sample is periodic data by using a preset function includes: Calculating the autocorrelation value between the values ​​of every other preset time period in the time series data learning sample by the preset function; In the case where there is a positive correlation between the autocorrelation values, determining that the time series data learning sample is periodic data; When the autocorrelation value approaches zero, it is determined that the time series data learning sample does not have periodicity.

4. The data prediction method according to claim 1, characterized in that: The determining at least one hyperparameter combination according to the period information of the time series data learning sample includes: Reading the time series data learning sample and the period information; Read a list of possible hyperparameter values ​​corresponding to the data prediction model; According to the period information, each parameter value in the list of possible hyperparameter values ​​is combined to obtain the at least one hyperparameter combination.

5. The data prediction method according to claim 1, characterized in that: The using the time series data learning sample to train the data prediction model corresponding to the at least one hyperparameter combination to obtain at least one trained data prediction model includes: Splitting the time series data learning sample into at least one combination of a training data set and a test data set; The data prediction model corresponding to the at least one hyperparameter combination is trained using at least one training data set to obtain at least one data prediction model.

6. The data prediction method according to claim 5, characterized in that: The step of splitting the time series data learning sample into at least one combination of a training data set and a test data set includes: Preset a fixed-size window; The window is slid to split the time series data learning sample into at least one combination of the training data set and the test data set.

7. The data prediction method according to claim 1, characterized in that: The step of selecting a target data prediction model from at least one trained data prediction model through error degree calculation includes: Calculating the error percentage of the data prediction model by splitting a test data set from the time series data learning sample; Determine the average error of each data prediction model on each data set combination according to the error percentage; The data prediction model with the smallest average error is used as the target prediction model.

8. A data prediction device, characterized in that: The device comprises: The data processing module is used to process the historical time series data to obtain the corresponding time series data learning samples; A data judgment module, used to judge whether the time series data learning sample is periodic data through a preset function; A hyperparameter combination determination module, used for determining at least one hyperparameter combination according to the period information of the time series data learning sample when the time series data learning sample is periodic data, and the hyperparameter combination is used in a preset data prediction model; A model training module, used to train the data prediction model corresponding to the at least one hyperparameter combination using the time series data learning sample to obtain at least one trained data prediction model; A target prediction model determination module, configured to select a target data prediction model from at least one trained data prediction model by calculating an error degree; The prediction result acquisition module is used to perform data prediction on the input time series data through the target data prediction model to obtain the data prediction result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

Images