Data monitoring method and system based on time sequence model

Abstract

The invention belongs to the technical field of big data processing, and particularly discloses a data monitoring method and system based on a time sequence model, and the method comprises the steps: obtaining the historical big data of contract performance, and constructing a data set through a data cleaning technology; constructing a time sequence model based on the historical big data; predicting real-time contract performance data by using the trained time sequence model to obtain a performance time prediction result, and obtaining a real-time time deviation based on the performance time prediction result and the planned performance time; for different contract data types and cooperative objects, performing weighted smoothing on historical time deviation data by adopting an exponential weighted moving average algorithm, and generating a self-adaptive dynamic threshold value by adjusting a weight coefficient and a risk tolerance coefficient; and when the real-time time deviation exceeds a dynamic threshold value, an early warning mechanism is triggered to carry out early warning prompt. According to the invention, the accuracy, real-time performance and intelligence of contract performance data processing can be improved.

Description

Technical Field

[0001] This application belongs to the field of big data processing, and more specifically, relates to a data monitoring method and system based on a time series model. Background Technology

[0002] In industries such as supply chain and logistics, real-time monitoring of key operational nodes in the contract performance process is crucial. With the widespread adoption of industrial data acquisition technologies, the overall contract performance data of the system is experiencing a massive growth trend. Traditional system status monitoring methods that rely on regular manual inspections are inefficient and prone to oversights when faced with large-scale, multi-dimensional contract data, failing to promptly identify potential performance risks.

[0003] While existing technologies attempt to process data through automation, they face technical bottlenecks in areas such as cleaning massive amounts of operational data, recognizing complex time-series patterns, and adapting dynamic models, making it difficult to meet the stringent requirements for accuracy, real-time performance, and intelligence in current data processing.

[0004] Therefore, it is urgent to combine big data processing technology to build an automated monitoring solution that can efficiently process massive amounts of contract data and accurately predict the status change trends of key nodes, so as to ensure the stable and efficient operation of industrial systems. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this application is to provide a data monitoring method and system based on a time series model, which aims to solve the problem that the data processing bottleneck in the prior art makes it difficult to meet the requirements of accuracy, real-time performance and intelligence.

[0006] To achieve the above objectives, in a first aspect, this application provides a data monitoring method based on a time series model, comprising: Obtain historical big data on contract performance and construct a dataset using data cleaning techniques; A time series model is constructed based on historical big data. The time series model is constructed based on seasonal difference autoregressive moving average and is determined through difference stabilization, parameter space search, model fitting, model selection and model validation. A trained time series model is used to predict real-time contract performance data to obtain performance time prediction results. Based on the performance time prediction results and the planned performance time, the real-time time deviation is obtained. For different contract types and partners, an exponentially weighted moving average algorithm is used to smooth historical time deviation data, and an adaptive dynamic threshold is generated by adjusting the weighting coefficient and risk tolerance coefficient. When the real-time time deviation exceeds the dynamic threshold, an early warning mechanism is triggered to provide an early warning.

[0007] Optionally, the method for constructing the time series model includes: The dataset is divided into training, validation and test sets by time series analysis. A seasonal difference autoregressive moving average model is constructed to capture the long-term trend and random fluctuation characteristics of contract performance samples through non-seasonal parameters and the periodic characteristics of contract performance samples through seasonal parameters. The seasonal difference autoregressive moving average model is subjected to difference stabilization, parameter space search, model fitting, model selection, and model validation to obtain the time series model.

[0008] Optionally, the process of difference stabilization, parameter space search, model fitting, model selection, and model validation of the time series model specifically includes: The original time series data corresponding to the training set is differentially processed to eliminate data trends and periodic fluctuations, resulting in a stationary sequence; Define the parameter search space and narrow down the parameter range of seasonal and non-seasonal parameters by combining data characteristics; Traverse the entire parameter search space, fit the model in parallel, and evaluate the fit by calculating the log-likelihood function; Based on the information criterion, the fitted models on the training set are initially screened, and the top few models with the best indicators are selected as candidate models. The prediction error of each candidate model is calculated through the validation set, and the model with the smallest error is selected as the optimal output model. The optimal output model is tested to verify its stability. If the stability fails, the parameters or methods are adjusted and the model is retrained until the final time series model is obtained.

[0009] Optionally, the specific process of the difference stabilization process includes: Obtain the preprocessed training set time series of the contract performance time series; The training set time is subjected to non-seasonal differencing to eliminate sequence trends, resulting in non-seasonal differencing results; The non-seasonal difference results are subjected to seasonal difference processing to eliminate periodic fluctuations, resulting in seasonal difference results. Verify whether the seasonal difference result meets the stationarity requirement. If it does not meet the requirement, return to the non-seasonal difference processing, adjust the difference order, and reprocess. Output processed time series data that meets the stationarity requirement for subsequent model training.

[0010] Optionally, the specific process of model fitting includes: Iterate through the preset parameter combinations and perform model training in parallel for each parameter combination; For the current parameter combination, the model is fitted using historical performance time series data from the dataset to obtain the corresponding predicted value sequence; The model predictions are determined based on the predicted value sequence, the model residuals are determined based on the model predictions and the actual observed values, and the residual distribution characteristics are constructed based on the variance of the residual values. The model fit index is calculated based on the number of samples in the predicted value sequence, the variance of the residual values, and the residual distribution characteristics to evaluate the model fit effect of each parameter combination.

[0011] Optionally, the specific process of model selection includes: Based on the information criterion, all models fitted on the training set are initially evaluated, and the candidate model with the best comprehensive evaluation result is selected. The candidate models are used to make predictions using the validation set, and the prediction accuracy index of each candidate model is calculated. Based on the prediction accuracy index, the candidate models are compared on the validation set to determine their prediction accuracy performance, and the model with the smallest prediction error is selected as the optimal output model.

[0012] Optionally, the specific process of model validation includes: The optimal output model is used to calculate the prediction accuracy index through the test set, and then compared and analyzed with the performance on the validation set. The model is deemed to have met the reliability standard when the difference between the prediction error of the test set and the validation set does not exceed a preset threshold. When the difference between the prediction error of the test set and the validation set exceeds a preset threshold, the model is determined to have failed the reliability verification, and optimization measures are taken to continue iterative optimization until the model passes the reliability verification.

[0013] Optionally, the optimization measures include at least one of the following: Expand the search range for model parameters, increase the number of candidate models, and retrain. Adjust the difference order or correct the seasonal cycle parameters; Improve data preprocessing methods and optimize the handling of missing and outlier values; The data was grouped according to contract type, and a dedicated prediction model was built for each group.

[0014] Secondly, this application also provides a data monitoring system based on a time series model, comprising: The data assurance module is used to acquire historical big data on contract performance and to construct datasets using data cleaning techniques. The data modeling module is used to construct time series models based on historical big data. The time series models are constructed based on seasonal differenced autoregressive moving averages and are determined through differenced stabilization, parameter space search, model fitting, model selection, and model validation. The real-time computing module is used to predict real-time contract performance data using a trained time series model, obtain the performance time prediction result, and obtain the real-time time deviation based on the performance time prediction result and the planned performance time. The risk management module is used to apply an exponentially weighted moving average algorithm to smooth historical time deviation data for different contract types and partners, and to generate an adaptive dynamic threshold by adjusting the weighting coefficient and risk tolerance coefficient. When the real-time time deviation exceeds the dynamic threshold, an early warning mechanism is triggered to provide an early warning.

[0015] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0017] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0018] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0019] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application significantly improves the accuracy, real-time performance, and efficiency of contract performance data monitoring by using a time series model based on seasonal differential autoregressive moving average (SARIMA) and an adaptive dynamic threshold mechanism. First, by using differential stabilization processing and parameter space search to optimize model fitting, the trend and seasonal characteristics of performance data can be accurately captured, improving the accuracy of prediction results. Second, by dynamically adjusting the threshold through an exponentially weighted moving average algorithm, combined with the characteristics of different contract data types and partners, personalized risk monitoring is achieved, enhancing the system's adaptability and real-time response capability. Finally, the model screening and verification process ensures the robustness of the algorithm, while the automated early warning mechanism significantly reduces the cost of manual intervention. Overall, through intelligent data processing and real-time prediction, the analysis cycle is effectively shortened, monitoring efficiency is improved, and timely and reliable data support is provided for contract performance risk management.

[0020] (2) The big data-driven SARIMA model constructed in this application automatically learns parameters through scientific data partitioning and multi-criteria optimization mechanism, which significantly improves the prediction accuracy and generalization ability of the model on the test set, and can more accurately grasp the contract performance time pattern and achieve accurate prediction.

[0021] (3) This application changes the traditional contract management model that relies on manual monitoring and experience-based judgment. It achieves full automation and intelligence in contract performance monitoring. From data collection and model training to anomaly warning, no large amount of manual intervention is required, which greatly reduces the probability of human error. At the same time, enterprise managers can obtain performance status information in real time, which significantly improves the efficiency of contract management.

[0022] (4) Given the non-stationarity and time-varying nature of contract performance data, traditional fixed thresholds and static models are difficult to adapt to complex and ever-changing business scenarios. The dynamic threshold generation method and adaptive model optimization mechanism of this application can quickly respond to factors such as changes in the performance mode of partners and policy adjustments, effectively avoid false alarms, and ensure that high early warning accuracy and effectiveness can still be maintained under various business changes. Attached Figure Description

[0023] Figure 1 This is one of the flowcharts illustrating the data monitoring method based on a time series model provided in this application embodiment; Figure 2 This is the second flowchart illustrating the data monitoring method based on a time series model provided in this application embodiment; Figure 3 This is a schematic diagram of the structure of the data monitoring device based on the time series model provided in the embodiments of this application; Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0026] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0027] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0028] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0029] The embodiments of this application are described below with reference to the accompanying drawings.

[0030] Reference Figure 1 This application provides a data monitoring method based on a time series model, including: S101. Obtain historical big data on contract performance and construct a dataset using data cleaning techniques; S102. Construct a time series model based on historical big data; the time series model is constructed based on seasonal difference autoregressive moving average and is determined through difference stabilization, parameter space search, model fitting, model selection and model validation; S103. Use the trained time series model to predict real-time contract performance data to obtain the performance time prediction result, and obtain the real-time time deviation based on the performance time prediction result and the planned performance time; S104. For different contract data types and partners, an exponentially weighted moving average algorithm is used to smooth historical time deviation data, and an adaptive dynamic threshold is generated by adjusting the weight coefficient and risk tolerance coefficient. When the real-time time deviation exceeds the dynamic threshold, an early warning mechanism is triggered to provide an early warning.

[0031] Specifically, firstly, through step S101, historical big data on contract performance is collected, focusing on key contract performance time nodes (including planned performance time y). plan To address the common issues of missing and anomalies in historical actual performance times (y), high-quality datasets are constructed using big data cleaning techniques. This provides a reliable data foundation for subsequent model training and analysis, ensuring the validity and accuracy of model inputs. Specifically, this includes: Missing value handling: Linear interpolation is used for intelligent repair of missing data. Assume the time series data is y1, y2, ..., y... n If y i Missing data, based on known data before and after. and A linear relationship is established, as shown in the following formula:

[0032] in, This represents the index number of the data point in the time series. For the first Data points.

[0033] This technology achieves accurate filling of missing values ​​by constructing a linear relationship. It solves the problems of low efficiency and high subjectivity of traditional manual data entry, ensuring data integrity and avoiding model training bias caused by missing data.

[0034] Outlier Handling: Based on big data statistical analysis technology, the 3σ principle is used to intelligently identify data points that deviate from the mean by more than three standard deviations. Through data correction or deletion strategies, noise in the performance data (such as abnormal performance records with extreme delays or advances) is filtered out to ensure that the performance data input into the model conforms to the actual distribution characteristics of the business, laying a reliable data foundation for subsequent time series analysis.

[0035] Secondly, through step S102, after obtaining a high-quality historical dataset, the focus is on building a time series model that can accurately capture the changing patterns of performance time.

[0036] In this embodiment, given that contract performance data often exhibits significant seasonality, the Seasonally Differential Autoregressive Moving Average (SARIMA) model is chosen as the core modeling tool. First, the data is differentially stabilized to eliminate trends and seasonality, ensuring it meets the model assumptions. Then, a systematic parameter space search (e.g., using AIC or BIC criteria) is conducted to explore model performance under different parameter combinations (p, d, q) and seasonal parameter combinations (P, D, Q). Next, the selected parameter combinations are fitted to the model. Based on the fit, residual analysis, model selection (comparing the merits of different models), and rigorous model validation (e.g., using a test set to evaluate prediction accuracy) are performed to ultimately determine the optimal SARIMA model structure.

[0037] Furthermore, step S103 is the core application of the monitoring solution, aiming to apply the trained SARIMA model to actual operations. By accessing the latest contract performance data stream in real time, and utilizing the trained time series model, predictions are made for each key performance node, outputting an estimated performance time point, i.e., the performance time prediction result.

[0038] Subsequently, this forecast result is directly compared with the planned performance time explicitly stipulated in the contract, and the time difference between the two, i.e., the real-time time deviation, is calculated. This real-time time deviation is a key indicator for judging whether there is a risk in performance; a positive value indicates that the forecast will be delayed, and a negative value indicates that the forecast will be brought forward. By continuously performing this forecasting and deviation calculation process, the real-time status of contract performance can be dynamically monitored.

[0039] Finally, to effectively manage performance deviations and avoid false alarms, step S104 introduces an adaptive dynamic threshold mechanism. Considering that different types of contracts (such as procurement contracts of varying value, service contracts in different industries) and different partners (such as long-term stable partners and new partners) may have significant differences in the variability of performance time, a simple fixed threshold is difficult to apply.

[0040] Therefore, for each type of contract data or each partner, historical time deviation data is collected and processed using an exponentially weighted moving average algorithm. The algorithm assigns higher weights to recent deviation data, making the calculated smoothed deviation value more reflective of the current trend, while also retaining some memory of historical data.

[0041] It should be further explained that by adjusting the weighting coefficients of the exponentially weighted moving average algorithm and introducing a risk tolerance coefficient, an adaptive dynamic threshold is generated that dynamically adjusts over time, contract type, and partnership. When the real-time time deviation calculated in step S103 exceeds this dynamic threshold, a preset early warning mechanism will be immediately triggered, sending warning prompts to relevant responsible parties through various means such as email, SMS, and system pop-ups, enabling them to intervene in a timely manner and take countermeasures, thereby controlling potential performance risks.

[0042] Optionally, the method for constructing the time series model includes: The dataset is divided into training, validation and test sets by time series analysis. A seasonal decomposition autoregression integrated moving average model is constructed to capture the long-term trend and random fluctuation characteristics of contract performance samples through non-seasonal parameters and the periodic characteristics of contract performance samples through seasonal parameters. The seasonal decomposition autoregression integrated moving average model is processed through difference stabilization, parameter space search, model fitting, model selection, and model validation to obtain the time series model.

[0043] Specifically, relying on a big data computing framework, a SARIMA model suitable for massive contract performance data is constructed to achieve in-depth mining of trend, seasonal, and periodic characteristics in performance time series, providing model support for accurate prediction. This includes: Data partitioning: The time-series data output by the data quality assurance module is partitioned chronologically into a training set (70%), a validation set (15%), and a test set (15%) to ensure that the temporal sequence is not disrupted. The training set data, containing complete historical contract performance cycle data, is used to train and generate model parameters. The validation set data is used to evaluate the model's generalization ability during parameter optimization to avoid overfitting. The test set data is completely independent of the training / validation process and is used to evaluate the final model performance.

[0044] Model architecture: A seasonal decomposition autoregressive integrated moving average (SARIMA) model is adopted, which is in the form of SARIMA(p,d,q)(P,D,Q). s This model captures the long-term trend and random fluctuations of contract performance time through non-seasonal parameters (autoregressive order p, differencing order d, and moving average order q), and adapts to the periodic patterns of different types of contract performance (such as monthly settlement cycle and quarterly delivery cycle) through seasonal parameters (seasonal autoregressive order P, seasonal differencing order D, seasonal moving average order Q, and seasonal cycle s), thereby solving the technical problem that traditional models cannot handle multi-dimensional time features simultaneously.

[0045] Furthermore, the processes of difference stabilization, parameter space search, model fitting, model selection, and model validation for the time series model specifically include: The original time series data corresponding to the training set is differentially processed to eliminate data trends and periodic fluctuations, resulting in a stationary sequence; Define the parameter search space and narrow down the parameter range of seasonal and non-seasonal parameters by combining data characteristics; Traverse the entire parameter search space, fit the model in parallel, and evaluate the fit by calculating the log-likelihood function; Based on the information criterion, the fitted models on the training set are initially screened, and the top few models with the best indicators are selected as candidate models. The prediction error of each candidate model is calculated through the validation set, and the model with the smallest error is selected as the optimal output model. The optimal output model is tested to verify its stability. If the stability fails, the parameters or methods are adjusted and the model is retrained until the final time series model is obtained.

[0046] Specifically, this application's embodiments learn patterns from historical contract performance data to construct a SARIMA (Seasonally Differential Autoregressive Moving Average) model capable of accurately predicting future performance time trends. The entire process follows a rigorous machine learning modeling procedure to ensure the model's accuracy and reliability.

[0047] First, differencing is used to achieve data stationarity. Raw contract performance time series data (e.g., the number of days between the actual and planned delivery dates) often contain obvious trends (such as overall delay trends) and periodic fluctuations (such as seasonal effects or patterns within a fixed period). To meet the SARIMA model's requirement for input data stationarity (i.e., the statistical properties of the data do not change over time), the original series needs to be differencing. This includes: non-seasonal differencing, which eliminates linear trends by calculating period-by-period differencing of the series; and seasonal differencing, which eliminates periodic fluctuations by calculating differencing at seasonal intervals. After appropriate differencing orders d (non-seasonal) and D (seasonal), a stationary time series is obtained.

[0048] Secondly, the parameter search space is defined and narrowed. A SARIMA model consists of multiple parameters, including the order of the non-seasonal component (p, d, q) and the order of the seasonal component (P, D, Q, s). Then, by combining preliminary data analysis or business experience, the possible values ​​of these parameters are narrowed down, reducing the number of parameter combinations that need to be evaluated subsequently and improving search efficiency.

[0049] Then, the parameters are traversed and the model is fitted in parallel. Within the defined parameter search space, all or most possible combinations of (p, q, P, Q) need to be evaluated. For each parameter combination, a SARIMA model is trained using historical data. During the fitting process, the model learns the autoregressive and moving average relationships in the data. To initially assess the fit of each candidate model, its log-likelihood value is calculated. A higher log-likelihood value generally indicates a better fit of the model to the training data.

[0050] Next, initial screening is performed based on information criteria, followed by a validation set for further screening. All parameter combinations are iterated through, and the logarithmic similarity is calculated. Then, information criteria (such as the Akaike Information Criterion (AIC) or the Bayesian Information Criterion (BIC)) are used to initially screen candidate models. Information criteria, while measuring the model's fit, penalize model complexity, helping to avoid overfitting. The top few models with the smallest information criterion values ​​are selected as the candidate pool.

[0051] Then, a separate validation set is used to calculate the prediction error of these candidate models on the validation set; in this embodiment, the root mean square error (RMSE) is used. The model with the smallest prediction error on the validation set is selected as the optimal candidate model for the current stage. This step aims to ensure that the model not only fits the training data well but also has good generalization ability and performs well on unseen data.

[0052] Finally, an independent test set is used to verify the model's stability and determine the final model. To definitively confirm the stability and generalization ability of the selected model, it is applied to a completely independent test set to evaluate its performance on the test set, specifically checking whether its prediction error is consistent with its performance on the validation set, and whether the model exhibits instability or a sharp decline in performance.

[0053] If the model performs stably and as expected on the test set, it is selected as the final output model. If the test results indicate insufficient model stability or poor performance, it is necessary to return to the previous steps, adjust the parameter search space, improve the differencing method, try different model structures, or re-collect / process the data, and then retrain and validate until a final time series model that performs stably and reliably on an independent test set is obtained.

[0054] Specifically, the differential stabilization process includes: Obtain the preprocessed training set time series of the contract performance time series; The training set time is subjected to non-seasonal differencing to eliminate sequence trends, resulting in non-seasonal differencing results; The non-seasonal difference results are subjected to seasonal difference processing to eliminate periodic fluctuations, resulting in seasonal difference results. Verify whether the seasonal difference result meets the stationarity requirement. If it does not meet the requirement, return to the non-seasonal difference processing, adjust the difference order, and reprocess. Output processed time series data that meets the stationarity requirement for subsequent model training.

[0055] Specifically, this step transforms the original non-stationary sequence into a stationary sequence that meets the requirements of the SARIMA model, ensuring the effectiveness of subsequent parameter estimation and avoiding model bias caused by data trends / periods. The specific implementation is as follows: (1) Input data: Obtain the preprocessed training set time series Y (train) ={y1,y2,...,y n} This includes the contract performance time (such as the number of days between the port handling fee payment date and the end date of the operation).

[0056] (2) Non-seasonal differencing: This involves differentiating the input training data to eliminate sequential trends (e.g., increased payment cycles due to business growth).

[0057] in, It is the result after differencing. It is a difference operator, where d is the difference order (0, 1, or 2). It starts with the 0th order difference. If the difference passes the subsequent stationarity test, it proceeds to the next step. Otherwise, it increases the difference order and continues to differ until the difference order reaches 2.

[0058] (3) Seasonal differencing: The results after non-seasonal differencing are then subjected to seasonal differencing to eliminate periodic fluctuations (such as the performance peaks before and after the monthly settlement date):

[0059] in, It is the result after differencing. It is a seasonal difference operator, where s is the seasonal period (e.g., the monthly period of a port is s=30 days), and D is the seasonal difference order (0, 1, or 2). It starts with the 0th order difference. If the difference passes the subsequent stationarity test, it proceeds to the next step. Otherwise, it increases the difference order and continues to differ until the difference order reaches 2.

[0060] (4) Stationarity test: The ADF test (Augmented Dickey-Fuller Test) can be used for verification. Stability.

[0061] Optionally, the parameter space in SARIMA is initialized as follows: This step defines a reasonable parameter search space for the parameters in the SARIMA model based on the characteristics of the contract performance data, narrows the parameter range based on data features, and reduces unnecessary computation: (1) Non-seasonal parameter: autoregression order Moving average order Difference order .

[0062] (2) Seasonal parameter: seasonal autoregression order Seasonal moving average order Seasonal difference order .

[0063] (3) Seasonal cycle s: Identify significant cycles through the autocorrelation function (ACF) plot, such as the general port operation agreement performance data s=30 days.

[0064] (4) Parameter space generation: Generate all possible parameter combinations (p,d,q,P,D,Q,s) through Cartesian product, with a total number of combinations of . kind.

[0065] Optionally, the specific process of model fitting includes: Iterate through the preset parameter combinations and perform model training in parallel for each parameter combination; For the current parameter combination, the model is fitted using historical performance time series data from the dataset to obtain the corresponding predicted value sequence; The model predictions are determined based on the predicted value sequence, the model residuals are determined based on the model predictions and the actual observed values, and the residual distribution characteristics are constructed based on the variance of the residual values. The model fit index is calculated based on the number of samples in the predicted value sequence, the variance of the residual values, and the residual distribution characteristics to evaluate the model fit effect of each parameter combination.

[0066] Specifically, this step involves the model fitting and likelihood function calculation process. It traverses the entire parameter space of the model and performs model fitting and likelihood function calculation in parallel for each parameter combination (p, d, q, P, D, Q, s). (1) Model fitting: For the current parameter combination (p,d,q,P,D,Q,s), use historical performance time series data Y (train) ={y1,y2,...,y n} Fitting the SARIMA model yields the predicted value sequence {ŷ1,ŷ2,...,ŷ n} .

[0067] (2) Model residual calculation: Model residual ,calculate The variance is denoted as ; (3) Calculation of the log-likelihood function of the model: Assume that the residuals of the model follow a normal distribution. The log-likelihood function of the model is as follows:

[0068] The log-likelihood function reflects the degree to which the model fits the data; the larger the value, the better the fit.

[0069] Optionally, the specific process of model selection includes: Based on the information criterion, all models fitted on the training set are initially evaluated, and the candidate model with the best comprehensive evaluation result is selected. The candidate models are used to make predictions using the validation set, and the prediction accuracy index of each candidate model is calculated. Based on the prediction accuracy index, the candidate models are compared on the validation set to determine their prediction accuracy performance, and the model with the smallest prediction error is selected as the optimal output model.

[0070] Specifically, this step involves the evaluation and screening of multi-criteria models. The models fitted in the above steps are evaluated and screened using the following methods: (1) Initial screening of the training set: The model is evaluated using the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC). The calculation formulas for the two criteria are as follows:

[0071]

[0072] in, k It refers to the number of model parameters, including non-seasonal parameters (p+q+d) and seasonal parameters (P+Q+D+1), i.e. k =p+q+d+P+Q+D+1; n is the model sample size, i.e., the size of the model training dataset; ln( L ) is the log-likelihood function value of the model.

[0073] After completing the AIC / BIC calculation, the top 5 candidate models with the smallest AIC / BIC values ​​are selected.

[0074] (2) Validation set screening: For the selected candidate models, input the validation set data to make predictions, and then calculate the root mean square error (RMSE):

[0075] in, It is validation set data. This is the model's prediction data. After calculating the RMSE of all candidate models, the model with the smallest RMSE value is selected as the optimal model output.

[0076] Optionally, the specific process of model validation includes: The optimal output model is used to calculate the prediction accuracy index through the test set, and then compared and analyzed with the performance on the validation set. The model is deemed to have met the reliability standard when the difference between the prediction error of the test set and the validation set does not exceed a preset threshold. When the difference between the prediction error of the test set and the validation set exceeds a preset threshold, the model is determined to have failed the reliability verification, and optimization measures are taken to continue iterative optimization until the model passes the reliability verification.

[0077] The optimization measures include at least one of the following: Expand the search range for model parameters, increase the number of candidate models, and retrain. Adjust the difference order or correct the seasonal cycle parameters; Improve data preprocessing methods and optimize the handling of missing and outlier values; The data was grouped according to contract type, and a dedicated prediction model was built for each group.

[0078] Specifically, in this embodiment, the optimal model output from the above steps is validated for model reliability using a test dataset.

[0079] (1) Use the optimal model to predict the test set data and calculate the root mean square error (RMSE):

[0080] (2) Reliability judgment: Compare the prediction error on the test dataset with the prediction error on the validation dataset. If the following conditions are met, the model is considered to have met the stability criteria:

[0081] Here, γ is usually taken as 0.2, that is, the error of the test set does not exceed 20% of that of the validation set.

[0082] If the model fails reliability verification, it may be due to the following reasons, and targeted measures can be taken: (1) The model underfits because it does not fully extract data features. On the one hand, it may be that the model did not fully consider generalization ability when screening through the validation dataset. This can be addressed by expanding the model parameter search space and increasing the number of candidate models before retraining and screening the model. On the other hand, it may be due to omission of higher-order autoregressive terms or errors in seasonal cycle identification. This can be addressed by increasing the difference order or adjusting the seasonal cycle s (e.g., s=30→s=90).

[0083] (2) Data quality issues affect model training performance. The methods for handling missing values ​​and identifying outliers during data preprocessing can be further improved. For example, the handling of missing values ​​could be upgraded to using a random forest algorithm to predict missing continuous values, thus compensating for the shortcomings of linear interpolation.

[0084] (3) The data is too complex to be fitted by a single model. Different types of contracts may have different seasonal cycles, and a single model may not be able to fit them perfectly. It is possible to classify the contracts according to their execution characteristics, and then group the data and train a dedicated model for each group.

[0085] Optionally, the embodiments of this application utilize big data-based performance time prediction technology and dynamic threshold-driven anomaly detection and early warning, specifically implemented through the following methods: By using a pre-trained SARIMA model to perform batch predictions on real-time collected contract performance data, the system quickly generates performance time prediction results. Combined with the planned performance time, it calculates the time deviation Δ, providing real-time data support for anomaly detection and meeting enterprises' real-time monitoring needs for contract performance.

[0086] in, It is the predicted fulfillment time. The contractually agreed performance time is represented by Δ, which is the deviation from the performance time. A value greater than zero indicates a risk of delay.

[0087] Furthermore, a dynamic threshold system is constructed to address the differences in performance characteristics among different contract types and partners, enabling accurate identification and real-time early warning of performance anomalies, thus resolving the issue that traditional fixed thresholds cannot adapt to complex business scenarios. Specifically, this includes: Threshold Calculation: An exponentially weighted moving average algorithm is introduced to perform weighted smoothing on historical time deviation data to generate a dynamic threshold. The calculation formula is as follows:

[0088] The relevant parameters are explained below: It is the deviation between the actual performance time of the contract and the planned performance time at the previous moment.

[0089] λ is a weighting coefficient used to control the influence of recent data (the larger the λ value, the higher the weight of recent data). The value can be determined based on historical experience: when λ is close to 0, the threshold changes slowly, which is suitable for partners with high performance stability; when λ is close to 1, the threshold responds quickly to recent changes, which is suitable for new partners or high volatility scenarios.

[0090] k is an adjustment coefficient used to amplify or reduce the threshold, reflecting the company's tolerance for different contract types and partner performance risks. Multiple levels of k values ​​can be set according to risk preferences: k=1 for strict risk control scenarios; k=1.5 for ordinary risk control scenarios; and k=2 for lenient risk control scenarios.

[0091] It is the final calculated dynamic threshold, which is dynamically adjusted according to the distribution of historical performance data. It can adapt to the performance stability of different partners (such as the low volatility of high-quality customers and the high volatility of new customers), avoiding the problems of missed detection or false detection caused by fixed threshold.

[0092] Intelligent early warning: When the real-time calculation module outputs Δ> When triggered, a real-time big data early warning mechanism can be established, pushing warning notifications containing information such as contract number, abnormal nodes, and time deviations to relevant personnel in real time via message queues. This application's embodiment realizes a shift from "post-event processing" to "pre-event early warning," ensuring that enterprises can intervene promptly before or in the early stages of performance risks, thereby reducing losses from breach of contract.

[0093] Reference Figure 2 , Figure 2 This is a complete flowchart of the application, including the following steps: Data preprocessing: Preliminary processing of raw contract performance data, including cleaning, formatting, and standardization, to ensure data quality.

[0094] Data partitioning: The preprocessed data is divided into three subsets: training set, validation set, and test set. The training set is used for model training, the validation set is used for model parameter tuning and preliminary evaluation, and the test set is used for final model performance evaluation.

[0095] Model Fitting + AIC / BIC Initial Screening: Multiple candidate models are fitted using the training set data, and the AIC (Akaike Information Criterion) and BIC (Bayes Information Criterion) values ​​of each model are calculated for initial screening.

[0096] RMSE fine screening: For models that pass the initial screening of AIC / BIC, further fine screening is carried out using validation set data, mainly by calculating the root mean square error (RMSE) to determine the optimal model.

[0097] Model reliability verification: The selected best model is verified using independent test set data to ensure the stability and accuracy of the model in practical applications.

[0098] Model Deployment: If the model passes reliability verification, it is deployed to the production environment and put into operation.

[0099] Real-time prediction of contract performance deviation: Utilize the deployed model to make real-time predictions of new contract performance data and calculate the prediction deviation.

[0100] Dynamic threshold calculation: Calculate dynamic thresholds based on historical data and business needs to better identify abnormal situations.

[0101] Intelligent early warning: When the prediction deviation exceeds the set dynamic threshold, the system will issue an intelligent early warning to notify relevant personnel to take measures.

[0102] Adjusting data preprocessing methods: If the model performs poorly during the validation or testing phase, you may need to go back to this step and adjust the data preprocessing methods to improve data quality and model performance.

[0103] Adjust training parameters: Adjust the model's training parameters appropriately based on the model's performance to optimize model performance.

[0104] Candidate models: During the model fitting and initial screening stages, multiple candidate models may be generated. These models will be further screened and validated in subsequent steps.

[0105] Optimal Model: Through a series of screening and verification steps, an optimal model is finally determined for actual contract performance monitoring.

[0106] Prediction bias: The difference between actual contract performance data and model predictions, used to determine whether there are potential risks or problems.

[0107] Dynamic threshold: A threshold that is dynamically adjusted according to the actual situation, used to determine whether the prediction deviation has reached a level that requires an early warning.

[0108] Intelligent early warning: Based on prediction deviations and dynamic thresholds, it automatically issues early warning signals to remind relevant personnel to pay attention to potential performance risks.

[0109] Reference Figure 3 This application also provides a data monitoring system based on a time series model, comprising: Data assurance module 310 is used to acquire historical big data on contract performance and to construct a dataset using data cleaning technology; The data modeling module 320 is used to construct a time series model based on historical big data. The time series model is constructed based on seasonal difference autoregressive moving average and is determined through difference stabilization processing, parameter space search, model fitting, model selection, and model validation. The real-time computing module 330 is used to predict real-time contract performance data using a trained time series model, obtain the performance time prediction result, and obtain the real-time time deviation based on the performance time prediction result and the planned performance time. The risk management module 340 is used to apply an exponentially weighted moving average algorithm to smooth historical time deviation data for different contract types and partners, and to generate an adaptive dynamic threshold by adjusting the weighting coefficient and risk tolerance coefficient. When the real-time time deviation exceeds the dynamic threshold, an early warning mechanism is triggered to provide an early warning.

[0110] Optionally, the method for constructing the time series model includes: The dataset is divided into training, validation and test sets by time series analysis. A seasonal decomposition autoregression integrated moving average model is constructed to capture the long-term trend and random fluctuation characteristics of contract performance samples through non-seasonal parameters and the periodic characteristics of contract performance samples through seasonal parameters. The seasonal decomposition autoregression integrated moving average model is processed through difference stabilization, parameter space search, model fitting, model selection, and model validation to obtain the time series model.

[0111] Optionally, the process of difference stabilization, parameter space search, model fitting, model selection, and model validation of the time series model specifically includes: The original time series data corresponding to the training set is differentially processed to eliminate data trends and periodic fluctuations, resulting in a stationary sequence; Define the parameter search space and narrow down the parameter range of seasonal and non-seasonal parameters by combining data characteristics; Traverse the entire parameter search space, fit the model in parallel, and evaluate the fit by calculating the log-likelihood function; Based on the information criterion, the fitted models on the training set are initially screened, and the top few models with the best indicators are selected as candidate models. The prediction error of each candidate model is calculated through the validation set, and the model with the smallest error is selected as the optimal output model. The optimal output model is tested to verify its stability. If the stability fails, the parameters or methods are adjusted and the model is retrained until the final time series model is obtained.

[0112] Optionally, the specific process of the difference stabilization process includes: Obtain the preprocessed training set time series of the contract performance time series; The training set time is subjected to non-seasonal differencing to eliminate sequence trends, resulting in non-seasonal differencing results; The non-seasonal difference results are subjected to seasonal difference processing to eliminate periodic fluctuations, resulting in seasonal difference results. Verify whether the seasonal difference result meets the stationarity requirement. If it does not meet the requirement, return to the non-seasonal difference processing, adjust the difference order, and reprocess. Output processed time series data that meets the stationarity requirement for subsequent model training.

[0113] Optionally, the specific process of model fitting includes: Iterate through the preset parameter combinations and perform model training in parallel for each parameter combination; For the current parameter combination, the model is fitted using historical performance time series data from the dataset to obtain the corresponding predicted value sequence; The model predictions are determined based on the predicted value sequence, the model residuals are determined based on the model predictions and the actual observed values, and the residual distribution characteristics are constructed based on the variance of the residual values. The model fit index is calculated based on the number of samples in the predicted value sequence, the variance of the residual values, and the residual distribution characteristics to evaluate the model fit effect of each parameter combination.

[0114] Optionally, the specific process of model selection includes: Based on the information criterion, all models fitted on the training set are initially evaluated, and the candidate model with the best comprehensive evaluation result is selected. The candidate models are used to make predictions using the validation set, and the prediction accuracy index of each candidate model is calculated. Based on the prediction accuracy index, the candidate models are compared on the validation set to determine their prediction accuracy performance, and the model with the smallest prediction error is selected as the optimal output model.

[0115] Optionally, the specific process of model validation includes: The optimal output model is used to calculate the prediction accuracy index through the test set, and then compared and analyzed with the performance on the validation set. The model is deemed to have met the reliability standard when the difference between the prediction error of the test set and the validation set does not exceed a preset threshold. When the difference between the prediction error of the test set and the validation set exceeds a preset threshold, the model is determined to have failed the reliability verification, and optimization measures are taken to continue iterative optimization until the model passes the reliability verification.

[0116] Optionally, the optimization measures include at least one of the following: Expand the search range for model parameters, increase the number of candidate models, and retrain. Adjust the difference order or correct the seasonal cycle parameters; Improve data preprocessing methods and optimize the handling of missing and outlier values; The data was grouped according to contract type, and a dedicated prediction model was built for each group.

[0117] It is understood that the detailed functional implementation of each of the above units / modules can be found in the description in the aforementioned method embodiments, and will not be repeated here.

[0118] It should be understood that the above-described device is used to execute the methods in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above methods. The working process of the device can be referred to the corresponding process in the above methods, and will not be repeated here.

[0119] Reference Figure 4Based on the methods in the above embodiments, this application provides an electronic device that may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. The processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions stored in the memory 430 to execute the methods in the above embodiments.

[0120] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0121] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0122] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0123] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0124] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0125] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0126] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0127] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data monitoring method based on a time series model, characterized in that, include: Obtain historical big data on contract performance and construct a dataset using data cleaning techniques; A time series model is constructed based on historical big data. The time series model is constructed based on seasonal difference autoregressive moving average and is determined through difference stabilization, parameter space search, model fitting, model selection and model validation. A trained time series model is used to predict real-time contract performance data to obtain performance time prediction results. Based on the performance time prediction results and the planned performance time, the real-time time deviation is obtained. For different contract types and partners, an exponentially weighted moving average algorithm is used to smooth historical time deviation data, and an adaptive dynamic threshold is generated by adjusting the weighting coefficient and risk tolerance coefficient. When the real-time time deviation exceeds the dynamic threshold, an early warning mechanism is triggered to provide an early warning.

2. The data monitoring method based on a time series model according to claim 1, characterized in that, The method for constructing the time series model includes: The dataset is divided into training, validation and test sets by time series analysis. A seasonal difference autoregressive moving average model is constructed to capture the long-term trend and random fluctuation characteristics of contract performance samples through non-seasonal parameters and the periodic characteristics of contract performance samples through seasonal parameters. The seasonal difference autoregressive moving average model is subjected to difference stabilization, parameter space search, model fitting, model selection, and model validation to obtain the time series model.

3. The data monitoring method based on a time series model according to claim 2, characterized in that, The process of difference stabilization, parameter space search, model fitting, model selection, and model validation of the time series model specifically includes: The original time series data corresponding to the training set is differentially processed to eliminate data trends and periodic fluctuations, resulting in a stationary sequence; Define the parameter search space and narrow down the parameter range of seasonal and non-seasonal parameters by combining data characteristics; Traverse the entire parameter search space, fit the model in parallel, and evaluate the fit by calculating the log-likelihood function; Based on the information criterion, the fitted models on the training set are initially screened, and the top few models with the best indicators are selected as candidate models. The prediction error of each candidate model is calculated through the validation set, and the model with the smallest error is selected as the optimal output model. The optimal output model is tested to verify its stability. If the stability fails, the parameters or methods are adjusted and the model is retrained until the final time series model is obtained.

4. The data monitoring method based on a time series model according to claim 2, characterized in that, The specific process of the difference stabilization process includes: Obtain the preprocessed training set time series of the contract performance time series; The training set time is subjected to non-seasonal differencing to eliminate sequence trends, resulting in non-seasonal differencing results; The non-seasonal difference results are subjected to seasonal difference processing to eliminate periodic fluctuations, resulting in seasonal difference results. Verify whether the seasonal difference result meets the stationarity requirement. If it does not meet the requirement, return to the non-seasonal difference processing, adjust the difference order, and reprocess. Output processed time series data that meets the stationarity requirement for subsequent model training.

5. The data monitoring method based on a time series model according to claim 2, characterized in that, The specific process of model fitting includes: Iterate through the preset parameter combinations and perform model training in parallel for each parameter combination; For the current parameter combination, the model is fitted using historical performance time series data from the dataset to obtain the corresponding predicted value sequence; The model predictions are determined based on the predicted value sequence, the model residuals are determined based on the model predictions and the actual observed values, and the residual distribution characteristics are constructed based on the variance of the residual values. The model fit index is calculated based on the number of samples in the predicted value sequence, the variance of the residual values, and the residual distribution characteristics to evaluate the model fit effect of each parameter combination.

6. The data monitoring method based on a time series model according to claim 2, characterized in that, The specific process of model selection includes: Based on the information criterion, all models fitted on the training set are initially evaluated, and the candidate model with the best comprehensive evaluation result is selected. The candidate models are used to make predictions using the validation set, and the prediction accuracy index of each candidate model is calculated. Based on the prediction accuracy index, the candidate models are compared on the validation set to determine their prediction accuracy performance, and the model with the smallest prediction error is selected as the optimal output model.

7. The data monitoring method based on a time series model according to claim 2, characterized in that, The specific process of model validation includes: The optimal output model is used to calculate the prediction accuracy index through the test set, and then compared and analyzed with the performance on the validation set. The model is deemed to have met the reliability standard when the difference between the prediction error of the test set and the validation set does not exceed a preset threshold. When the difference between the prediction error of the test set and the validation set exceeds a preset threshold, the model is determined to have failed the reliability verification, and optimization measures are taken to continue iterative optimization until the model passes the reliability verification.

8. The data monitoring method based on a time series model according to claim 7, characterized in that, The optimization measures include at least one of the following: Expand the search range for model parameters, increase the number of candidate models, and retrain. Adjust the difference order or correct the seasonal cycle parameters; Improve data preprocessing methods and optimize the handling of missing and outlier values; The data was grouped according to contract type, and a dedicated prediction model was built for each group.

9. A data monitoring system based on a time series model, characterized in that, include: The data assurance module is used to acquire historical big data on contract performance and to construct datasets using data cleaning techniques. The data modeling module is used to construct time series models based on historical big data. The time series models are constructed based on seasonal differenced autoregressive moving averages and are determined through differenced stabilization, parameter space search, model fitting, model selection, and model validation. The real-time computing module is used to predict real-time contract performance data using a trained time series model, obtain the performance time prediction result, and obtain the real-time time deviation based on the performance time prediction result and the planned performance time. The risk management module is used to apply an exponentially weighted moving average algorithm to smooth historical time deviation data for different contract types and partners, and to generate adaptive dynamic thresholds by adjusting the weighting coefficients and risk tolerance coefficients. When the real-time time deviation exceeds the dynamic threshold, an early warning mechanism is triggered to provide an early warning.

10. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-8.

Images