Data prediction method and device and storage medium

Abstract

The invention discloses a data prediction method and device and a storage medium, and the method comprises the steps: predicting the data change trend of a data sequence through a first data prediction model and a second data prediction model, and obtaining a first data prediction result and a second data prediction result; respectively calculating deviation values between data prediction results output by the first data prediction model and the second data prediction model in a preset historical time period and an actual data result, and obtaining a prediction deviation of the first data prediction model and a prediction deviation of the second data prediction model; and based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model, performing weighted fusion on the first data prediction result and the second data prediction result to obtain a target data prediction result. The data prediction capabilities of a plurality of different data prediction models can be integrated, the output results of the plurality of different data prediction models are effectively fused, and the prediction accuracy of complex data is improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data prediction method, device and storage medium. Background Art

[0002] Trend prediction is an important analytical task in the field of time series data analysis. Taking commodity price data as an example, decision makers hope to judge whether commodity prices will rise, remain stable, or fall in the future by analyzing existing data.

[0003] At present, trend prediction algorithms include the autoregressive integrated moving average model (ARIMA), the exponential smoothing (Holt-Winters) model, and the neural network model trained based on machine learning methods.

[0004] However, the data in data prediction scenarios are generally more complex. The prediction scheme corresponding to the prediction model often has a good prediction effect on one type of data, but a very poor prediction effect on another type of data. How to improve the prediction accuracy of these complex data is an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present application at least provides a data prediction method, device and storage medium.

[0006] The first aspect of the present application provides a data prediction method, the method comprising: obtaining a data sequence to be predicted, the data sequence being composed of data collected at multiple different times; using a first data prediction model and a second data prediction model to respectively predict the data change trend of the data sequence, to obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model; and respectively calculating the deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model within a preset historical time period and the actual data results, to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; wherein the actual data result refers to the real data collected at the time corresponding to the data prediction result; based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model, calculating the result weighting parameter; weighted fusion of the first data prediction result and the second data prediction result according to the result weighting parameter, to obtain the target data prediction result corresponding to the data sequence.

[0007] In one embodiment, the deviation values ​​between the data prediction results and the actual data results output by the first data prediction model and the second data prediction model within a preset historical time period are calculated respectively to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model, including: obtaining each data prediction result output by the first data prediction model within the preset historical time period, and obtaining the actual data result corresponding to each data prediction result; respectively calculating the difference between each data prediction result and the actual data result corresponding to it, and calculating the sum of squares corresponding to each difference, to obtain the prediction deviation of the first data prediction model; and, obtaining each data prediction result output by the second data prediction model within the preset historical time period, and obtaining the actual data result corresponding to each data prediction result; respectively calculating the difference between each data prediction result and the actual data result corresponding to it, and calculating the sum of squares corresponding to each difference, to obtain the prediction deviation of the second data prediction model.

[0008] In one embodiment, a result weighting parameter is calculated based on a prediction deviation of a first data prediction model and a prediction deviation of a second data prediction model, including: summing the inverse of the prediction deviation of the first data prediction model and the inverse of the prediction deviation of the second data prediction model to obtain a summed value; calculating the ratio between the inverse of the prediction deviation of the first data prediction model and the summed value to obtain a result weighting parameter corresponding to the first data prediction model; and, calculating the ratio between the inverse of the prediction deviation of the second data prediction model and the summed value to obtain a result weighting parameter corresponding to the second data prediction model.

[0009] In one embodiment, the first data prediction model includes an autoregressive integrated moving average model, and the second data prediction model includes an exponential smoothing model; the first data prediction model and the second data prediction model are used to respectively predict the data change trend of the data series to obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model, including: using the autoregressive integrated moving average model to predict the data change trend of the data series to obtain the first data prediction result output by the autoregressive integrated moving average model; and using the exponential smoothing model to predict the data change trend of the data series to obtain the second data prediction result output by the exponential smoothing model.

[0010] In one embodiment, the exponential smoothing model includes an additive model and a multiplicative model, and the exponential smoothing model is used to predict the data change trend of the data sequence to obtain a second data prediction result output by the exponential smoothing model, including: determining whether the data sequence is suitable for the additive model or the multiplicative model based on the data characteristics of the data sequence; using the additive model or the multiplicative model suitable for the data sequence to predict the data change trend of the data sequence to obtain the second data prediction result output by the exponential smoothing model.

[0011] In one embodiment, obtaining a data sequence to be predicted includes: collecting raw data at different times; preprocessing the raw data and storing the preprocessed raw data in a preset data warehouse; and in response to a data prediction instruction, obtaining the preprocessed raw data from the data warehouse to obtain the data sequence to be predicted.

[0012] In one embodiment, the method also includes: obtaining real data collected at the time corresponding to the target data prediction result, and obtaining the actual data result corresponding to the target data prediction result; and, obtaining data alarm rules; detecting whether the target data prediction result and / or the actual data result corresponding to the target data prediction result satisfies the data alarm rules; if the data alarm rules are satisfied, sending an alarm message to the alarm terminal corresponding to the data alarm rule.

[0013] In one embodiment, obtaining data alarm rules includes: obtaining business requirements of a current data prediction scenario; and generating data alarm rules corresponding to the current data prediction scenario based on the business requirements.

[0014] The second aspect of the present application provides a data prediction device, which includes: a data acquisition module, which is used to acquire a data sequence to be predicted, and the data sequence is composed of data collected at multiple different times; a data prediction module, which is used to use a first data prediction model and a second data prediction model to respectively predict the data change trend of the data sequence, and obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model; a deviation calculation module, which is used to respectively calculate the deviation value between the data prediction results output by the first data prediction model and the second data prediction model within a preset historical time period and the actual data results, and obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; wherein the actual data result refers to the real data collected at the time corresponding to the data prediction result; a weight calculation module, which is used to calculate the result weighting parameter based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; a weighted fusion module, which is used to weightedly fuse the first data prediction result and the second data prediction result according to the result weighting parameter, and obtain the target data prediction result corresponding to the data sequence.

[0015] A third aspect of the present application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned data prediction method.

[0016] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned data prediction method when the program instructions are executed by a processor.

[0017] The above scheme uses the first data prediction model and the second data prediction model to respectively predict the data change trend of the data sequence, thereby obtaining the first data prediction result output by the first data prediction model and the second data prediction result output by the second data prediction model; and respectively calculates the deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model in a preset historical time period and the actual data results, thereby obtaining the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; calculates the result weighting parameter based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; and performs weighted fusion of the first data prediction result and the second data prediction result according to the result weighting parameter to obtain the target data prediction result corresponding to the data sequence, thereby comprehensively integrating the data prediction capabilities of multiple different data prediction models, and effectively fusion the output results of multiple different data prediction models, thereby improving the prediction accuracy of complex data.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0020] Figure 1 It is a schematic diagram of an implementation environment of a solution provided by an embodiment of the present application;

[0021] Figure 2 is a flow chart of a data prediction method shown in an exemplary embodiment of the present application;

[0022] Figure 3 is a schematic diagram of data prediction and data alarm shown in an exemplary embodiment of the present application;

[0023] Figure 4 is a block diagram of a data prediction device shown in an exemplary embodiment of the present application;

[0024] Figure 5 is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present application;

[0025] Figure 6 It is a schematic diagram of the structure of a computer-readable storage medium shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0026] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0027] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0028] The term "and / or" in this article is only an association information describing the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the previous and next associated objects are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0029] The data prediction method provided in the embodiments of the present application is described below.

[0030] Please refer to Figure 1 , which shows a schematic diagram of a solution implementation environment provided by an embodiment of the present application. The solution implementation environment may include a terminal 110 and a server 120, and the terminal 110 and the server 120 are connected to each other in communication.

[0031] The number of the terminal 110 may be one or more. The terminal 110 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0032] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), as well as big data and artificial intelligence platforms.

[0033] In one example, the server 120 may perform data prediction processing on the data sequence to be predicted obtained from the terminal 110 to obtain a target data prediction result. Of course, the server 120 may store the target data prediction result locally, transmit it back to the terminal 110, or transmit it to other terminals 110.

[0034] In one example, a client running a target application is installed in the terminal 110. For example, the target application may be an application with a data prediction function, and the target application performs data prediction processing on the data sequence to be predicted to obtain a target data prediction result. The server 120 may be a background server of the target application, which is used to provide background services for the client of the target application.

[0035] In the data prediction method provided in the embodiment of the present application, the execution subject of each step can be the terminal 110, such as the client of the target application installed and running in the terminal 110, or the server 120, or the terminal 110 and the server 120 can interact and cooperate to execute, that is, part of the steps of the method are executed by the terminal 110 and the other part of the steps are executed by the server 120.

[0036] See also Figure 2 , Figure 2 is a flowchart of a data prediction method shown in an exemplary embodiment of the present application. The data prediction method can be applied to Figure 1 It should be understood that the method can also be applied to other exemplary implementation environments and be specifically executed by devices in other implementation environments, and this embodiment does not limit the implementation environment to which the method is applicable.

[0037] like Figure 2 As shown, the data prediction method at least includes steps S210 to S250, which are described in detail as follows:

[0038] Step S210: Obtain a data sequence to be predicted, where the data sequence is composed of data collected at multiple different times.

[0039] It should be noted that different data prediction scenarios will result in different data in the acquired data sequence.

[0040] The following are examples of various data prediction scenarios:

[0041] In the network and server management scenario, the operation data of the network devices and servers are obtained to obtain the data sequence to be predicted. The operation data of the network devices and servers include but are not limited to one or more of the central processing unit (CPU) usage rate, memory usage, network traffic, etc.

[0042] In the application management scenario, the application's operating data is obtained to obtain the data sequence to be predicted, wherein the application's operating data includes but is not limited to one or more of the number of database connections, request response time, error log, etc.

[0043] In the database management scenario, the operation data of the database is obtained to obtain the data sequence to be predicted, wherein the operation data of the database includes but is not limited to one or more of query performance, index failure, and number of database connections.

[0044] In the IoT device management scenario, the operation data of the IoT device is obtained to obtain the data sequence to be predicted. The operation data of the IoT device includes but is not limited to one or more of the data collected by the temperature sensor, the data collected by the humidity sensor, the data collected by the pressure sensor, etc.

[0045] In the financial transaction management scenario, real-time data and rules of financial transactions are obtained to obtain the data sequence to be predicted, such as obtaining one or more data such as transaction volume, abnormal transaction behavior, etc.

[0046] In industrial production management scenarios, various data parameters on the production line are obtained to obtain the data sequence to be predicted, such as obtaining one or more data parameters such as temperature, humidity, pressure, quality, etc.

[0047] Acquire data collected at multiple different times to obtain the data sequence to be predicted.

[0048] Exemplarily, raw data at different times are collected; the raw data are preprocessed and the preprocessed raw data are stored in a preset data warehouse; in response to a data prediction instruction, the preprocessed raw data are obtained from the data warehouse to obtain a data sequence to be predicted.

[0049] Among them, preprocessing includes but is not limited to noise data cleaning, data integration, etc. This application does not limit the specific implementation method of preprocessing.

[0050] The preprocessed raw data is stored in the preset data warehouse, thereby generating reliable data samples for subsequent data prediction and analysis. After receiving the data prediction instruction, the preprocessed raw data is obtained from the data warehouse to obtain the data sequence to be predicted.

[0051] In some implementations, each data has a corresponding collection time, a time range for data acquisition is determined, and data with a collection time within the time range is acquired to obtain a data sequence to be predicted.

[0052] Among them, the data prediction instruction may carry a time range for data acquisition. If the data prediction instruction does not carry a time range for data acquisition, the time range for data acquisition can be determined based on one or more of the type of data, the source of the data, the accuracy required for data prediction, the time length required for data prediction, the prediction accuracy of the data prediction model, the remaining available computing resources of the execution entity, etc.

[0053] For example, by analyzing the data prediction results within the historical time period, the optimal time range for data acquisition corresponding to each different type of data and / or data from different sources is determined; or, the data prediction instruction carries the accuracy of the data prediction requirement. The higher the accuracy of the data prediction requirement, the larger the time range for data acquisition, and vice versa, the smaller the time range for data acquisition; or, the data prediction instruction carries the time length required for data prediction. The longer the time length required for data prediction, that is, the farther the time interval between the prediction and the current data is, the larger the time range for data acquisition is, and vice versa, the smaller the time range for data acquisition is; or, the prediction accuracy of the data prediction model is obtained. The lower the prediction accuracy, the larger the time range for data acquisition is, and vice versa, the smaller the time range for data acquisition is; or, the remaining available computing resources of the execution subject are obtained. The more remaining available computing resources, the larger the time range for data acquisition is, and vice versa, the smaller the time range for data acquisition is.

[0054] Step S220: using the first data prediction model and the second data prediction model to respectively predict the data change trend of the data sequence, and obtaining a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model.

[0055] Both the first data prediction model and the second data prediction model have the ability to analyze data change trends of data sequences, and can predict data change trends at future moments based on the input data sequences to obtain data prediction results.

[0056] Among them, the first data prediction model and the second data prediction model include but are not limited to at least two of an autoregressive integrated moving average model, an exponential smoothing model, a neural network model based on deep learning, etc.

[0057] The data sequence is respectively input into the first data prediction model and the second data prediction model to perform data change trend analysis to obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model.

[0058] Optionally, if the time range for data acquisition is determined based on the prediction accuracy of the data prediction model, the prediction accuracy of the first data prediction model and the prediction accuracy of the second data prediction model can be weightedly calculated, and the time range for data acquisition can be determined based on the prediction accuracy obtained by the weighted calculation. In this case, the first data prediction model and the second data prediction model correspond to the same data sequence. The data acquisition time range corresponding to the first data prediction model can also be determined based on the prediction accuracy of the first data prediction model, so that data acquisition is performed according to the data acquisition time range corresponding to the first data prediction model to obtain the data sequence corresponding to the first data prediction model, and the data acquisition time range corresponding to the second data prediction model can be determined based on the prediction accuracy of the second data prediction model, so that data acquisition is performed according to the data acquisition time range corresponding to the second data prediction model to obtain the data sequence corresponding to the second data prediction model. In this case, the first data prediction model and the second data prediction model may correspond to different data sequences.

[0059] Step S230: Calculate the deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model within a preset historical time period and the actual data results respectively, and obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; wherein the actual data result refers to the real data collected at the time corresponding to the data prediction result.

[0060] The preset historical time period refers to the time period before the current moment, and the prediction deviation is used to characterize the prediction accuracy of the data prediction model.

[0061] Specifically, after obtaining the data prediction result, the real data collected at the time corresponding to the data prediction result is determined to observe the actual data value corresponding to the data prediction result to obtain the actual data result. Then, the deviation value between the data prediction result output within the preset historical time period and the actual data result is used to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model.

[0062] Exemplarily, the deviation values ​​between the data prediction results and the actual data results output by the first data prediction model and the second data prediction model within a preset historical time period are calculated respectively to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model, including: obtaining each data prediction result output by the first data prediction model within the preset historical time period, and obtaining the actual data result corresponding to each data prediction result; respectively calculating the difference between each data prediction result and the actual data result corresponding to it, and calculating the sum of squares corresponding to each difference, to obtain the prediction deviation of the first data prediction model; and, obtaining each data prediction result output by the second data prediction model within the preset historical time period, and obtaining the actual data result corresponding to each data prediction result; respectively calculating the difference between each data prediction result and the actual data result corresponding to it, and calculating the sum of squares corresponding to each difference, to obtain the prediction deviation of the second data prediction model.

[0063] For example, the calculation method of the prediction deviation of the first data prediction model can refer to the following formula 1:

[0064]

[0065] Among them, E ARIMA is the prediction deviation of the first data prediction model, is the prediction result of the i-th data, x m-k+i is the actual data result at time m-k+i, m is the total number of samples, and k is the seasonal length of the data series.

[0066] The total number of data and seasonal length of the data sequence are used to determine the actual data result at the moment corresponding to the data prediction result, so as to calculate the difference between the data prediction result and the corresponding actual data result, and then calculate the sum of squares corresponding to each difference to obtain the prediction deviation of the second data prediction model.

[0067] Similarly, the calculation method of the prediction deviation of the second data prediction model can refer to the following formula 2:

[0068]

[0069] Among them, E HW is the prediction deviation of the second data prediction model, is the prediction result of the i-th data, y m-k+i is the actual data result at time m-k+i, m is the total number of samples, and k is the seasonal length of the data series.

[0070] Optionally, in addition to calculating the sum of squares corresponding to each difference to obtain the prediction deviation of the data prediction model, other methods can be selected to calculate the prediction deviation of the data prediction model. For example, after obtaining the difference between each data prediction result and its corresponding actual data result, the largest difference is selected as the prediction deviation of the data prediction model. The specific calculation method of the prediction deviation can be flexibly selected according to actual conditions, and this application does not limit this.

[0071] In some embodiments, since the data prediction model has different prediction accuracies for different data, calculating the prediction deviation of the data prediction model based on the difference between each data prediction result and its corresponding actual data result includes: determining the data similarity between the currently output data prediction result (the first data prediction result or the second data prediction result) and the data prediction result or the actual data result within a preset historical time period, setting a weighting coefficient corresponding to the difference between each data prediction result and its corresponding actual data result within the preset historical time period according to the data similarity, performing weighted calculation on each difference according to the weighting coefficient corresponding to each difference, obtaining the final prediction deviation of the data prediction model, and improving the accuracy of the prediction deviation calculation.

[0072] Among them, the data fluctuation of the reference data sequence corresponding to the currently output data prediction result can be obtained, and the data fluctuation of the reference data sequence corresponding to the data prediction result within a preset historical time period can be obtained, and the data similarity can be determined according to the similarity between the above two data fluctuations; the difference between the currently output data prediction result and the actual data result can also be calculated, and the data similarity can be determined according to the size of the difference, which is not limited in this application.

[0073] Step S240: Calculate result weighting parameters based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model.

[0074] According to the size relationship between the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model, the result weighting parameters corresponding to the first data prediction model and the second data prediction model are set respectively.

[0075] Exemplarily, the inverse of the prediction deviation of the first data prediction model and the inverse of the prediction deviation of the second data prediction model are summed to obtain a sum value; the ratio between the inverse of the prediction deviation of the first data prediction model and the sum value is calculated to obtain a result weighting parameter corresponding to the first data prediction model; and, the ratio between the inverse of the prediction deviation of the second data prediction model and the sum value is calculated to obtain a result weighting parameter corresponding to the second data prediction model.

[0076] For example, the calculation method of the result weighting parameter of the first data prediction model can refer to the following formula 3:

[0077]

[0078] Among them, E ARIMA is the prediction deviation of the first data prediction model, E HW is the prediction deviation of the second data prediction model, W ARIMA A weighting parameter for the result of the first data prediction model;

[0079] The calculation method of the result weighting parameter of the second data prediction model can refer to the following formula 4:

[0080]

[0081] Among them, W HW The result weighting parameters are for the second data prediction model.

[0082] It should be noted that in addition to the above-mentioned calculation method, other methods can also be used to calculate the result weighting parameters, such as calculating the ratio between the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model, and calculating the result weighting parameters based on the ratio. This application does not limit this.

[0083] Step S250: weighted fusion of the first data prediction result and the second data prediction result is performed according to the result weighting parameter to obtain a target data prediction result corresponding to the data sequence.

[0084] A weighted sum calculation is performed on the first data prediction result and the second data prediction result according to the obtained result weighting parameter, and the calculation result is used as the target data prediction result corresponding to the data sequence.

[0085] For example, the specific calculation method for weighted fusion of the first data prediction result and the second data prediction result can be referred to the following formula 5:

[0086]

[0087] Among them, F i Predict the results for the target data, is the first data prediction result, is the second data prediction result, W ARIMA is the result weighting parameter of the first data prediction model, W HW The result weighting parameters are for the second data prediction model.

[0088] Next, some embodiments of the present application are described in detail.

[0089] In some embodiments, the first data prediction model includes an autoregressive integrated moving average model, and the second data prediction model includes an exponential smoothing model.

[0090] The data change trend of the data sequence is predicted using the autoregressive integrated moving average model to obtain the first data prediction result output by the autoregressive integrated moving average model; the data change trend of the data sequence is predicted using the exponential smoothing model to obtain the second data prediction result output by the exponential smoothing model.

[0091] The data prediction process of the Auto regressive Integrated Moving Average Model (ARIMA) is described in detail:

[0092] The autocorrelation and partial autocorrelation of the data sequence are analyzed to obtain the differential order corresponding to the data sequence; the data sequence is differentiated based on the differential order to convert the non-stationary data sequence into a stable differential sequence; the model parameters of the autoregressive integrated moving average model for the differential sequence are estimated using the maximum likelihood estimation method to obtain the fitted autoregressive integrated moving average model; based on the fitted autoregressive integrated moving average model, the differential sequence is de-differentiated to obtain the first data prediction result.

[0093] Specifically, the ARIMA model is mainly composed of three parts, namely, the autoregressive model (AR), the difference process (Integrated) and the moving average model (MA). The ARIMA model is expressed as: ARIMA (p, d, q) = AR (p) + I (d) + MA (q); where AR (p) represents the autoregressive model, I (d) represents the difference model, and MA (q) represents the moving average model;

[0094] The formula of the autoregressive model AR(p) is expressed as the following formula 6:

[0095]

[0096] Among them, X t is the current value of the prediction, μ is the constant term, p is the order, r i is the autocorrelation coefficient, ε t It's an error.

[0097] The above formula 6 uses the data from the 1st to t-1th order in the data sequence to predict the current performance.

[0098] The formula of the difference model I(d) is expressed as the following formula 7:

[0099] ΔX t =X t -X t-1 Formula 7

[0100] Among them, X t is the predicted current value, X t-1 is the data value of the t-1th order.

[0101] The formula of the moving average model MA(q) is expressed as the following formula 8:

[0102]

[0103] Among them, q is the order of the lag term, that is, how many periods of white noise terms in the past are considered to predict the current value; θ i is the coefficient of the white noise term in the i-th period.

[0104] The MA(q) model is able to predict future values ​​based on past white noise terms, which are considered to be random fluctuations in the series.

[0105] The specific prediction steps of the ARIMA model include: using the unit root test model (Augmented Dickey-Fuller test, ADF) to test the stationarity of the data series. If the P value of the test result is less than the significance level (usually 0.05), the data series is judged to be stationary, otherwise the data series is not stationary.

[0106] Stabilize the non-stationary data series: determine the difference order d; establish the difference sequence based on the number of differences d.

[0107] The p-order and q-order are determined based on the autocorrelation function (ACF) parameter and the partial autocorrelation function (PACF) coefficient.

[0108] The implementation of ACF and PACF can be found in Table 1 below:

[0109]

[0110] Table 1

[0111] Among them, truncation refers to the property that ACF or PACF is 0 after a certain order, which means that after a certain order of time series data, the impact of past data on future data can be ignored. p and q are determined by the number of autocorrelation ACF and partial autocorrelation coefficient PACF that are significantly non-zero:

[0112] The ACF related formula is shown in the following formula 9:

[0113]

[0114] Among them, Cov(Xt ,X t+k ) is the covariance of the observations at time point t and time point t+k lagged by k periods, Var(X t ) is the time series X t The variance of .

[0115] The PACF related formula is shown in the following formula 10:

[0116]

[0117] Among them, Cov represents covariance calculation, E represents expectation, that is, the average value, and Var represents variance calculation.

[0118] Since there is human subjectivity in judging the parameters of ARIMA through autocorrelation analysis and partial autocorrelation analysis, in order to avoid this influence, the optimal parameters p and q can be automatically found based on the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC).

[0119] The AIC and BIC related formulas are shown in the following formulas 11 and 12:

[0120] AIC=2K-2ln(L) Formula 11

[0121] BIC = Kln(n)-2ln(L) Formula 12

[0122] Where n is the number of samples, L is the maximum likelihood function, and K is the number of estimated parameters in the model.

[0123] The parameters that minimize the AIC and BIC values ​​are comprehensively selected as the optimal parameter values;

[0124] By analyzing the autocorrelation diagram (ACF) and partial autocorrelation diagram (PACF) of the data sequence, the difference order d is determined. Then, based on the differenced sequence, the maximum likelihood estimation method is used to determine the model parameter value that maximizes the probability of the observed data appearing, and the parameters of the fitted ARIMA model are obtained. Finally, based on the fitted ARIMA model, the original difference sequence is reversed to obtain the final prediction value, that is, the first data prediction result.

[0125] The data prediction process of the exponential smoothing model (Holt-Winters) is explained in detail:

[0126] The exponential smoothing model includes an additive model and a multiplicative model. The exponential smoothing model is used to predict the data change trend of the data sequence to obtain a second data prediction result output by the exponential smoothing model, including: determining whether the data sequence is suitable for the additive model or the multiplicative model based on the data characteristics of the data sequence; using the additive model or the multiplicative model suitable for the data sequence to predict the data change trend of the data sequence to obtain the second data prediction result output by the exponential smoothing model.

[0127] The data characteristics of the data sequence include, but are not limited to, one or more of the type of data, the fluctuation of the data, the amount of data contained in the data sequence, etc.

[0128] Determine whether the current data series is suitable for an additive model or a multiplicative model, so as to use the corresponding exponential smoothing model to analyze the data series and improve the accuracy of data prediction.

[0129] Taking the multiplication model as an example, it mainly includes initialization steps and update steps:

[0130] initialization:

[0131] The initial levels L0 and L0 are determined using the following formulas 13 and 14:

[0132]

[0133] Where k is the seasonal length, V i is the i-th actual data result.

[0134] The initial trend T0 is determined using the following formula 15:

[0135]

[0136] The initial seasonal S is calculated using the following formula 16: i Adjustment:

[0137]

[0138] renew:

[0139] Update the level L using the following formula 17 i :

[0140]

[0141] Among them, S i-k is the seasonal value corresponding to ik, L i-1 is the i-1 level, T i-1 is the trend of i-1, and α is the smoothing coefficient.

[0142] Update the trend T using the following formula 18i :

[0143] T i =β*(L i -L i-1 )+(1-β)*T i-1 ,i∈[k,2k-1] Formula 18

[0144] Among them, β is the trend parameter.

[0145] Update the seasonality using the following formula 19:

[0146]

[0147] Among them, γ is the seasonal parameter.

[0148] Then, the multiplication model is used to perform data prediction to obtain a second data prediction result, as shown in the following formula 20:

[0149] Y i =(L i +T i )*S i ,i∈[k,m) Formula 20

[0150] Among them, Y i is the second data prediction result of the i-th output, and m is the total number of samples.

[0151] In the present application, the prediction capabilities of the above-mentioned first data prediction model and the second data prediction model can be evaluated. For example, one or more algorithms such as root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE) are used to evaluate the accuracy of the output target data prediction result, or the accuracy of the output first data prediction result and the second data prediction result are evaluated separately:

[0152] For example, the root mean square error RMSE is expressed as the following formula 21:

[0153]

[0154] in, is the target data prediction result, the first data prediction result or the second data prediction result, x i is the actual data result, and m is the total number of samples.

[0155] The mean absolute error MAE is expressed as the following formula 22:

[0156]

[0157] The mean absolute percentage error MAPE is expressed as the following formula 23:

[0158]

[0159] The smaller the value of the above-mentioned root mean square error, mean absolute error and / or mean absolute percentage error is, the higher the accuracy of the data prediction is.

[0160] In some embodiments, the method also includes: obtaining real data collected at the time corresponding to the target data prediction result, and obtaining the actual data result corresponding to the target data prediction result; and, obtaining data alarm rules; detecting whether the target data prediction result and / or the actual data result corresponding to the target data prediction result satisfies the data alarm rules; if the data alarm rules are satisfied, sending an alarm message to the alarm terminal corresponding to the data alarm rule.

[0161] Acquire the real data collected at the time corresponding to the target data prediction result, obtain the actual data result corresponding to the target data prediction result, and store it in the data storage medium. Then, obtain the actual data result corresponding to the target data prediction result from the data storage medium through the data storage connection method implemented by the natural programming language; then connect the data prediction module through the preset programming code to obtain the target data prediction result; input the target data prediction result and the actual data result corresponding to the target data prediction result as input parameters into the alarm detection program.

[0162] The alarm detection program is provided with data alarm rules, which can be preset based on experience, or can be based on the business needs of the current data prediction scenario; the data alarm rules corresponding to the current data prediction scenario are generated based on the business needs, and this application does not limit this.

[0163] Detect whether the target data prediction result and / or the actual data result corresponding to the target data prediction result meets the data alarm rule; if the data alarm rule is met, send an alarm message to the alarm terminal corresponding to the data alarm rule.

[0164] Exemplarily, the code written in a natural programming language is connected to a message notification module of a software capable of realizing a message notification function, so as to realize notification of alarm information.

[0165] For an example, see Figure 3 , Figure 3 A schematic diagram of data prediction and data warning is shown in an exemplary embodiment of the present application, as shown in FIG. Figure 3As shown, the data obtained from the basic layer (including one or more of the business database, platform point data, client point data, device point data, etc.) is stored in the data warehouse, and the data in the data warehouse is input into the data prediction module as a sample (data sequence). The autoregressive integral moving average model and the exponential smoothing model of the data prediction module perform data change trend analysis on the input samples to obtain data prediction results, and the data prediction results output by the autoregressive integral moving average model and the exponential smoothing model are weighted and fused to obtain the target data prediction result set. Then, the data alarm module obtains the actual data result set from the data warehouse and the target data prediction result set from the data prediction module to detect whether the target data prediction result and / or the actual data result corresponding to the target data prediction result meet the data alarm rule, and sends an alarm message to the alarm terminal corresponding to the data alarm rule. Among them, the alarm rules include one or more of null value constraints, difference constraints, maximum value constraints, minimum value constraints, average value constraints, difference percentage constraints, etc.

[0166] Through the above embodiments, abnormal situations can be discovered and handled in a timely manner to ensure the stable operation of the system corresponding to the data.

[0167] It should be noted that the embodiments of the present application take the first data prediction model and the second data prediction model as examples, but in actual application scenarios, a larger number of data prediction models can also be considered, such as including a first data prediction model, a second data prediction model and a third data prediction model, and combining the data prediction results output by the first data prediction model, the second data prediction model and the third data prediction model to obtain the final target data prediction result.

[0168] The data prediction method provided in the present application uses a first data prediction model and a second data prediction model to respectively predict the data change trend of a data sequence, thereby obtaining a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model; and respectively calculates the deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model in a preset historical time period and the actual data results, thereby obtaining a prediction deviation of the first data prediction model and a prediction deviation of the second data prediction model; calculates a result weighting parameter based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; and performs weighted fusion of the first data prediction result and the second data prediction result according to the result weighting parameter to obtain a target data prediction result corresponding to the data sequence, thereby comprehensively integrating the data prediction capabilities of multiple different data prediction models, and effectively fusion the output results of multiple different data prediction models, thereby improving the prediction accuracy of complex data.

[0169] Figure 4FIG. 1 is a block diagram of a data prediction device shown in an exemplary embodiment of the present application. Figure 4 As shown, the exemplary data prediction device 400 includes: a data acquisition module 410, a data prediction module 420, a deviation calculation module 430, a weight calculation module 440 and a weighted fusion module 450. Specifically:

[0170] The data acquisition module 410 is used to acquire a data sequence to be predicted, where the data sequence is composed of data collected at multiple different times;

[0171] A data prediction module 420 is used to use the first data prediction model and the second data prediction model to respectively predict the data change trend of the data sequence, and obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model;

[0172] The deviation calculation module 430 is used to calculate the deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model in a preset historical time period and the actual data results, respectively, to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; wherein the actual data result refers to the real data collected at the time corresponding to the data prediction result;

[0173] A weight calculation module 440, configured to calculate a result weighting parameter based on a prediction deviation of the first data prediction model and a prediction deviation of the second data prediction model;

[0174] The weighted fusion module 450 is used to perform weighted fusion on the first data prediction result and the second data prediction result according to the result weighting parameter to obtain the target data prediction result corresponding to the data sequence.

[0175] It should be noted that the data prediction device provided in the above embodiment and the data prediction method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the data prediction device provided in the above embodiment can allocate the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0176] See also Figure 5 , Figure 51 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 500 includes a memory 501 and a processor 502, and the processor 502 is used to execute program instructions stored in the memory 501 to implement the steps in any of the above data prediction method embodiments. In a specific implementation scenario, the electronic device 500 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 500 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0177] Specifically, the processor 502 is used to control itself and the memory 501 to implement the steps in any of the above data prediction method embodiments. The processor 502 can also be called a central processing unit (CPU). The processor 502 may be an integrated circuit chip with signal processing capabilities. The processor 502 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 502 can be implemented by an integrated circuit chip.

[0178] See also Figure 6 , Figure 6 The computer-readable storage medium 600 stores program instructions 610 that can be executed by a processor, and the program instructions 610 are used to implement the steps in any of the above data prediction method embodiments.

[0179] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0180] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0182] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

Claims

1. A data prediction method, characterized in that: The method comprises: Acquire a data sequence to be predicted, wherein the data sequence is composed of data collected at multiple different times; Using a first data prediction model and a second data prediction model to respectively predict the data change trend of the data sequence, to obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model; and, Calculating the deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model in a preset historical time period and the actual data results respectively, to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; wherein the actual data result refers to the real data collected at the time corresponding to the data prediction result; Calculating a result weighting parameter based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model; The first data prediction result and the second data prediction result are weighted and fused according to the result weighting parameter to obtain the target data prediction result corresponding to the data sequence.

2. The method according to claim 1, characterized in that The respectively calculating deviation values ​​between the data prediction results output by the first data prediction model and the second data prediction model in a preset historical time period and the actual data results to obtain the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model includes: Obtaining each data prediction result output by the first data prediction model within the preset historical time period, and obtaining the actual data result corresponding to each data prediction result; respectively calculating the difference between each data prediction result and the actual data result corresponding to it, and calculating the sum of squares corresponding to each difference, to obtain the prediction deviation of the first data prediction model; and, Obtain each data prediction result output by the second data prediction model within the preset historical time period, and obtain the actual data result corresponding to each data prediction result; respectively calculate the difference between each data prediction result and its corresponding actual data result, and calculate the sum of squares corresponding to each difference to obtain the prediction deviation of the second data prediction model.

3. The method according to claim 2, characterized in that The calculating result weighting parameter based on the prediction deviation of the first data prediction model and the prediction deviation of the second data prediction model includes: Summing the inverse of the prediction deviation of the first data prediction model and the inverse of the prediction deviation of the second data prediction model to obtain a sum value; Calculate the ratio between the inverse of the prediction deviation of the first data prediction model and the sum value to obtain the result weighting parameter corresponding to the first data prediction model; and calculate the ratio between the inverse of the prediction deviation of the second data prediction model and the sum value to obtain the result weighting parameter corresponding to the second data prediction model.

4. The method according to claim 1, characterized in that: The first data prediction model includes an autoregressive integrated moving average model, and the second data prediction model includes an exponential smoothing model; the first data prediction model and the second data prediction model are used to respectively predict the data change trend of the data sequence to obtain a first data prediction result output by the first data prediction model and a second data prediction result output by the second data prediction model, including: Predicting the data change trend of the data sequence using an autoregressive integrated moving average model to obtain a first data prediction result output by the autoregressive integrated moving average model; and, An exponential smoothing model is used to predict the data change trend of the data sequence to obtain a second data prediction result output by the exponential smoothing model.

5. The method according to claim 4, characterized in that The exponential smoothing model includes an additive model and a multiplicative model. The exponential smoothing model is used to predict the data change trend of the data sequence to obtain a second data prediction result output by the exponential smoothing model, including: Based on the data features of the data sequence, determining whether the data sequence is applicable to the additive model or the multiplicative model; The addition model or the multiplication model applicable to the data sequence is used to predict the data change trend of the data sequence to obtain a second data prediction result output by the exponential smoothing model.

6. The method according to claim 1, characterized in that The step of obtaining a data sequence to be predicted includes: Collect raw data at different times; Preprocessing the raw data, and storing the preprocessed raw data in a preset data warehouse; In response to the data prediction instruction, the preprocessed original data is obtained from the data warehouse to obtain a data sequence to be predicted.

7. The method according to claim 1, characterized in that The method further comprises: Acquire the real data collected at the time corresponding to the target data prediction result, and obtain the actual data result corresponding to the target data prediction result; and obtain the data alarm rule; Detecting whether the target data prediction result and / or the actual data result corresponding to the target data prediction result satisfies the data alarm rule; If the data alarm rule is met, an alarm message is sent to an alarm terminal corresponding to the data alarm rule.

8. The method according to claim 7, characterized in that The data alarm rules are obtained, including: Obtain the business needs of the current data prediction scenario; Generate data alarm rules corresponding to the current data prediction scenario based on the business needs.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the steps in the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, and the program instructions can be executed by a processor to implement the steps in the method according to any one of claims 1 to 8.

Images