Data prediction method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a data prediction method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers.The method comprises the steps that in response to a data prediction request, prediction time is obtained to determine corresponding recent time; calling a first prediction model to predict the data of the recent time, determining a predicted abnormal sample as a pseudo-positive sample, determining an account-adjusted sample of the recent time as a true-positive sample, and carrying out new sample combination to obtain a new positive sample; calling a change point detection model, adding change point features to the newly added positive samples, determining weights of the newly added positive samples, and performing incremental training on the first prediction model based on the newly added positive samples, the weights and the change point features to obtain a second prediction model; updating the change point features, and performing periodic total training on the second prediction model to obtain a target prediction model; and inputting the prediction time into the target prediction model, and outputting prediction result data. And the model prediction accuracy and the user working efficiency are improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data prediction method, apparatus, electronic device, and computer-readable medium. Background Technology

[0002] Currently, in real-world business scenarios, especially those involving cash flow (such as billing, settlement, and reconciliation processes), the volume of data processed is often very large and has strong time-sensitive characteristics. This necessitates ensuring the model's effectiveness while maintaining high recall and precision. For example, in the scenario of settlement statement anomaly identification, the monthly data volume for a single line can range from millions to tens of millions, and the reconciliation rate (i.e., the positive sample rate) can vary from a few percent to nearly 50%. Current models need to identify and issue medium-to-high-risk settlement statements to the business for verification weekly, and output high-precision, anomaly-free settlement statements for automatic settlement. Current predictive models lack the capacity to handle high-volume data and exhibit significant performance fluctuations. They cannot guarantee that the optimal parameter combination validated on historical data will maintain good performance on the training model for predicting future data, impacting user efficiency and business processing accuracy. Summary of the Invention

[0003] In view of this, embodiments of this application provide a data prediction method, apparatus, electronic device, and computer-readable medium, which can solve the problems that existing prediction models have insufficient processing capabilities for high-volume data and large fluctuations in performance. They cannot guarantee that the optimal parameter combination verified on historical data will still maintain good performance on the training model for predicting future data, thus affecting user work efficiency and business processing accuracy.

[0004] To achieve the above objectives, according to one aspect of the embodiments of this application, a data prediction method is provided, comprising:

[0005] In response to a data prediction request, the corresponding prediction time is obtained, and then the corresponding near-term time is determined based on the prediction time.

[0006] The first prediction model is invoked to predict recent data, and outliers in the prediction results are identified as pseudo-positive samples.

[0007] The adjusted accounts samples corresponding to recent times are identified as true positive samples. Based on the false positive samples and the true positive samples, the new samples are merged to obtain new positive samples.

[0008] The change point detection model is invoked to add change point features to the newly added positive samples, and the weights of the newly added positive samples are determined. Based on the newly added positive samples, weights, and change point features, the first prediction model is incrementally trained to obtain the second prediction model.

[0009] Update the variable point features to perform periodic full training on the second prediction model, thereby obtaining the target prediction model;

[0010] Input the prediction time into the target prediction model to output the prediction result data.

[0011] Optionally, a change point detection model is invoked to add change point features to newly added positive samples, including:

[0012] Call the change point detection model to obtain the feature importance of each feature in the newly added positive samples;

[0013] Based on feature importance, important features are selected, and change point detection is performed based on these important features to obtain change point detection result data.

[0014] Based on the change point detection results, change point features are added to newly added positive samples.

[0015] Optionally, change point detection is performed based on important features to obtain change point detection result data, including:

[0016] Identify the categorical features among the important features and convert the categorical features into corresponding numerical data;

[0017] Based on a preset window size, assign corresponding windows to the numerical data and calculate the frequency of occurrence of the numerical data in each window;

[0018] Based on the frequency of occurrence, change point detection result data is obtained.

[0019] Optionally, based on the frequency of occurrence, change point detection result data is obtained, including:

[0020] Based on the frequency of occurrence, determine the cumulative sum of residuals for each numerical data point based on the cumulative time series;

[0021] Based on the cumulative residuals and preset thresholds, determine the numerical data of the point of change;

[0022] Based on the numerical data of the changing points, change point detection result data is generated.

[0023] Optionally, the preset threshold is determined through the following steps:

[0024] Determine the period of the training data for the target prediction model, and calculate the offset standard deviation based on the period;

[0025] The preset threshold is determined based on the offset standard deviation and preset weights.

[0026] Optionally, the weights of newly added positive samples are determined, including:

[0027] The weight of the true positive samples in the new samples is set as the first weight, and the weight of the false positive samples is set as the second weight, where the first weight is greater than the second weight.

[0028] Optionally, the first prediction model is incrementally trained based on the newly added positive samples, weights, and change point features to obtain a second prediction model, including:

[0029] The newly added positive samples are merged with the original training data to form a new training set;

[0030] Based on the weights, update the weights of the samples in the new training set;

[0031] The first prediction model is incrementally trained based on the new training set, updated weights, and change point features to obtain the second prediction model.

[0032] Optionally, update the variable point features, including:

[0033] Generate change point features for the samples in the new training set that have changed points, and update the change point features for the samples in the new training set.

[0034] In addition, this application also provides a data prediction apparatus, comprising:

[0035] The acquisition unit is configured to respond to a data prediction request, acquire the corresponding prediction time, and then determine the corresponding near time based on the prediction time.

[0036] The prediction unit is configured to invoke a first prediction model to predict recent data and identify outlier samples in the prediction results as pseudo-positive samples.

[0037] The sample merging unit is configured to identify the adjusted samples corresponding to the most recent time as true positive samples, and merge the new samples based on the pseudo-positive samples and the true-positive samples to obtain new positive samples;

[0038] The first training unit is configured to call the change point detection model to add change point features to the new positive samples, determine the weights of the new positive samples, and incrementally train the first prediction model based on the new positive samples, weights, and change point features to obtain the second prediction model.

[0039] The second training unit is configured to update the variable point features to periodically train the second prediction model in full, thereby obtaining the target prediction model.

[0040] The output unit is configured to input the prediction time into the target prediction model to output prediction result data.

[0041] Optionally, the first training unit is further configured as follows:

[0042] Call the change point detection model to obtain the feature importance of each feature in the newly added positive samples;

[0043] Based on feature importance, important features are selected, and change point detection is performed based on these important features to obtain change point detection result data.

[0044] Based on the change point detection results, change point features are added to newly added positive samples.

[0045] Optionally, the first training unit is further configured as follows:

[0046] Identify the categorical features among the important features and convert the categorical features into corresponding numerical data;

[0047] Based on a preset window size, assign corresponding windows to the numerical data and calculate the frequency of occurrence of the numerical data in each window;

[0048] Based on the frequency of occurrence, change point detection result data is obtained.

[0049] Optionally, the first training unit is further configured as follows:

[0050] Based on the frequency of occurrence, determine the cumulative sum of residuals for each numerical data point based on the cumulative time series;

[0051] Based on the cumulative residuals and preset thresholds, determine the numerical data of the point of change;

[0052] Based on the numerical data of the changing points, change point detection result data is generated.

[0053] Optionally, the preset threshold is determined through the following steps:

[0054] Determine the period of the training data for the target prediction model, and calculate the offset standard deviation based on the period;

[0055] The preset threshold is determined based on the offset standard deviation and preset weights.

[0056] Optionally, the first training unit is further configured as follows:

[0057] The weight of the true positive samples in the new samples is set as the first weight, and the weight of the false positive samples is set as the second weight, where the first weight is greater than the second weight.

[0058] Optionally, the first training unit is further configured as follows:

[0059] The newly added positive samples are merged with the original training data to form a new training set;

[0060] Based on the weights, update the weights of the samples in the new training set;

[0061] The first prediction model is incrementally trained based on the new training set, updated weights, and change point features to obtain the second prediction model.

[0062] Optionally, the second training unit is further configured as follows:

[0063] Generate change point features for the samples in the new training set that have changed points, and update the change point features for the samples in the new training set.

[0064] In addition, this application also provides a data prediction electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the data prediction method as described above.

[0065] In addition, this application also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the data prediction method described above.

[0066] One embodiment of the above invention has the following advantages or beneficial effects: In response to a data prediction request, this application obtains the corresponding prediction time, and then determines the corresponding recent time based on the prediction time; it calls a first prediction model to predict the data of the recent time, identifying abnormal samples in the prediction results as pseudo-positive samples; it identifies the adjusted samples corresponding to the recent time as true-positive samples, and merges new samples based on the pseudo-positive samples and true-positive samples to obtain new positive samples; it calls a change point detection model to add change point features to the new positive samples, determines the weight of the new positive samples, and incrementally trains the first prediction model based on the new positive samples, weights, and change point features to obtain a second prediction model; it updates the change point features to periodically train the second prediction model to obtain a target prediction model; and it inputs the prediction time into the target prediction model to output prediction result data. This can efficiently improve the prediction effect of the prediction model while ensuring recall and precision, increasing the accuracy of model prediction and improving user work efficiency.

[0067] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0068] The accompanying drawings are provided to better understand this application and do not constitute an undue limitation thereof. Wherein:

[0069] Figure 1 This is a schematic diagram of the main flow of a data prediction method provided according to an embodiment of this application;

[0070] Figure 2 This is a schematic diagram of the main flow of a data prediction method provided according to an embodiment of this application;

[0071] Figure 3 This is a schematic diagram illustrating an application scenario of a data prediction method provided according to an embodiment of this application;

[0072] Figure 4 This is a schematic diagram of the main units of a data prediction device according to an embodiment of this application;

[0073] Figure 5 This is an exemplary system architecture diagram to which embodiments of this application can be applied;

[0074] Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing terminal devices or servers in the embodiments of this application. Detailed Implementation

[0075] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of this application, including various details to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions of this disclosure all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0076] Figure 1 This is a schematic diagram of the main flow of a data prediction method provided according to an embodiment of this application, as shown below. Figure 1 As shown, data prediction methods include:

[0077] Step S101: In response to the data prediction request, obtain the corresponding prediction time, and then determine the corresponding near time based on the prediction time.

[0078] In this embodiment, the execution entity of the data prediction method (e.g., a server) can receive data prediction requests via wired or wireless connections. The data prediction request could be, for example, a request to predict data for June. After receiving the prediction request, the execution entity can obtain the corresponding prediction time. For example, the prediction time could be June, and based on the prediction time, the corresponding near-term time, such as May, can be determined.

[0079] Step S102: Invoke the first prediction model to predict recent data and identify abnormal samples in the prediction results as pseudo-positive samples.

[0080] The first prediction model could be, for example, a prediction model trained using historical data for rolling predictions, with a fixed time window and parameters. Using this first prediction model to predict data for the near future (May), outliers in the prediction results are treated as pseudo-positive samples.

[0081] Step S103: Determine the adjusted accounts sample corresponding to the recent time as the true positive sample. Based on the false positive sample and the true positive sample, merge the new sample to obtain the new positive sample.

[0082] For example, historical data for March and April has been adjusted, resulting in complete positive and negative samples; adjustments for May are incomplete, resulting in incomplete positive and negative samples. The data for June is then predicted. The adjusted samples for recent periods, such as May, are identified as true positive samples. These true positive and false positive samples from recent periods, such as May, are then merged with new samples to obtain new positive samples.

[0083] Step S104: Call the change point detection model to add change point features to the newly added positive samples, determine the weight of the newly added positive samples, and perform incremental training on the first prediction model based on the newly added positive samples, weights and change point features to obtain the second prediction model.

[0084] The executing entity can invoke the change point detection model to add change point features to the samples with change points in the newly added positive samples for subsequent model training. The newly added positive samples are weighted according to the preset weights of true-positive and false-positive samples. Based on the weighted allocation of the newly added positive samples and the newly added positive samples labeled with change point features, the first prediction model is subjected to short-term incremental training to obtain the second prediction model.

[0085] Specifically, determining the weight of the newly added positive samples includes: setting the weight of true positive samples in the newly added samples as a first weight, and setting the weight of false positive samples as a second weight, wherein the first weight is greater than the second weight. This application does not specifically limit the values ​​of the first and second weights in its embodiments.

[0086] Specifically, the first prediction model is incrementally trained based on the newly added positive samples, weights, and change point features to obtain the second prediction model, including: merging the newly added positive samples with the original training data to form a new training set; updating the weights of the samples in the new training set based on the weights; and incrementally training the first prediction model based on the new training set, the updated weights, and change point features to obtain the second prediction model.

[0087] As another implementation method, when updating the model with new data in the initial stage, the main body generates change point features for the new data, then merges them with pseudo-positive samples, assigns them corresponding weights, and uses only this part of the data for incremental training of the model to obtain the second prediction model.

[0088] Step S105: Update the change point features to perform periodic full training on the second prediction model, thereby obtaining the target prediction model.

[0089] Specifically, updating the change point features includes: generating change point features for samples in the new training set that have experienced change points and updating the change point features of the samples in the new training set.

[0090] For example, after accumulating a certain amount of new data, the executing entity can re-detect change points on the accumulated amount of new data, generate change point features for all data (historical data + new data) in the new training set, and perform full training on the second prediction model to continuously update the second prediction model, which is a fixed training version, such as the Catboost anomaly detection model, in order to obtain the target prediction model.

[0091] A target prediction model is a model that can accurately predict the prediction time corresponding to a data prediction request.

[0092] Step S106: Input the prediction time into the target prediction model to output the prediction result data.

[0093] Forecast result data, such as whether there are anomalies at the forecast time, or results of reducing order volume or amount.

[0094] This embodiment responds to a data prediction request, obtains the corresponding prediction time, and then determines the corresponding recent time based on the prediction time. It then calls a first prediction model to predict the data for the recent time, identifying anomalous samples in the prediction results as pseudo-positive samples. It identifies the adjusted accounts samples corresponding to the recent time as true-positive samples. Based on the pseudo-positive and true-positive samples, it merges new samples to obtain new positive samples. It then calls a change point detection model to add change point features to the new positive samples, determines the weights of the new positive samples, and incrementally trains the first prediction model based on the new positive samples, weights, and change point features to obtain a second prediction model. Finally, it updates the change point features and periodically performs full training on the second prediction model to obtain the target prediction model. The prediction time is then input into the target prediction model to output the prediction result data. This approach can efficiently improve the prediction performance of the prediction model while maintaining recall and precision, increasing the accuracy of model predictions and improving user work efficiency.

[0095] Figure 2This is a schematic diagram of the main flow of a data prediction method provided according to an embodiment of this application, as shown below. Figure 2 As shown, data prediction methods include:

[0096] Step S201: In response to the data prediction request, obtain the corresponding prediction time, and then determine the corresponding near time based on the prediction time.

[0097] In fields involving cash flow, data forecasting requests can be, for example, requests to predict whether there are anomalies at the forecast time, whether order volume needs to be reduced, or whether the amount needs to be reduced. The forecast time can be the time corresponding to the time identifier carried in the data forecasting request, such as June. A recent time can be, for example, a time immediately preceding the forecast time, such as May. This application does not specifically limit the recent time in its embodiments.

[0098] Step S202: Invoke the first prediction model to predict recent data and identify abnormal samples in the prediction results as pseudo-positive samples.

[0099] Abnormal samples in the prediction results are data samples with abnormalities in the data for the near future.

[0100] Step S203: Determine the adjusted sample corresponding to the recent time as the true positive sample. Based on the false positive sample and the true positive sample, merge the new sample to obtain the new positive sample.

[0101] When the prediction period is June, the implementing entity can identify the adjusted samples without anomalies corresponding to recent times, such as May, as true positive samples. The pseudo-positive samples with anomalies in the prediction results for recent times are then merged with the identified true positive samples to obtain new positive samples.

[0102] Step S204: Call the change point detection model to obtain the feature importance of each feature in the newly added positive samples.

[0103] Change point detection models, such as the CatBoost model, are used. Specifically, the CatBoost model consists of trees; assuming there are m trees, each split is based on a single feature, and each tree is composed of a series of splits. For feature i, its importance is the sum of the gains it brings across all tree splits. The feature importance of each feature in newly added positive samples can be obtained using CatBoost's `model.get_importance()` method.

[0104] Step S205: Based on feature importance, important features are selected, and change point detection is performed based on the important features to obtain change point detection result data.

[0105] For example, the obtained features are sorted in descending order of importance, and the top 20 features (accounting for about 30% of all features) are selected for subsequent change point detection to obtain change point detection result data, that is, to determine which features have changed.

[0106] Specifically, change point detection is performed based on important features to obtain change point detection result data, including: determining the categorical features among the important features and converting the categorical features into corresponding numerical data; allocating corresponding windows to the numerical data based on a preset window size and calculating the frequency of occurrence of the numerical data in each window; and obtaining change point detection result data based on the frequency of occurrence.

[0107] CUSUM, a statistical process control tool for detecting small offsets, is a cumulative sum control chart primarily used to monitor quality issues in the production process to ensure product compliance with standards. Since over 80% of features in real-world scenarios are categorical, to perform change point detection on the categorical data among the selected key features, it's necessary to convert the categorical data into numerical data and generate a sliding window of categorical frequencies. CUSUM is then applied to this frequency data to perform change point detection on the categorical features, yielding the change point detection results.

[0108] Specifically, based on the frequency of occurrence, change point detection result data is obtained, including: determining the cumulative sum of residuals for each numerical data based on the cumulative time series based on the frequency of occurrence; determining the numerical data where a change point occurs based on a preset threshold of the cumulative sum of residuals; and generating change point detection result data based on the numerical data where a change point occurs.

[0109] In this embodiment of the application, for each processed important feature frequency data, the residual of its cumulative time series data is calculated. When the cumulative sum of the residuals exceeds a certain preset threshold, CUSUM will trigger a signal indicating that the process in the important feature frequency data may have changed. Then, based on the changed important feature frequency data, change point detection result data is accurately generated.

[0110] Specifically, the preset threshold is determined through the following steps: determining the period of the training data of the target prediction model, and calculating the offset standard deviation based on the period; and determining the preset threshold based on the offset standard deviation and the preset weights.

[0111] For example, the preset threshold setting uses the corresponding period of the training data as a benchmark and calculates the CUSUM standard deviation for that period. Set the threshold as follows: Where k is 2, meaning that change point detection will capture changes that exceed 2 standard deviations from the mean.

[0112] Step S206: Based on the change point detection results data, add change point features to the newly added positive samples.

[0113] The samples that show changes in the newly added positive samples are marked with change point features to facilitate more accurate subsequent model training.

[0114] Step S207: Determine the weights of the newly added positive samples, and incrementally train the first prediction model based on the newly added positive samples, weights, and change point features to obtain the second prediction model.

[0115] The newly added positive samples include true positive samples and pseudo positive samples. Preset weights are assigned to both true and pseudo positive samples, with the weight of true positive samples being greater than that of pseudo positive samples. The executing entity can then perform incremental training on the first prediction model based on the weight assignments and the newly added positive samples marked with change point features, to obtain a second prediction model. This improves the incremental training effect of the first prediction model, thereby enhancing the accuracy of the trained second prediction model when making data predictions.

[0116] Step S208: Update the change point features to perform periodic full training on the second prediction model, thereby obtaining the target prediction model.

[0117] Updating change point features refers to updating the change point features of the entire training sample (including historical samples and newly added positive samples), thereby conducting long-term periodic full training on the second prediction model, and finally obtaining the target prediction model that can accurately predict data.

[0118] Step S209: Input the prediction time into the target prediction model to output the prediction result data.

[0119] The prediction result data may include the result data of whether there are anomalies at the prediction time, obtained by the target prediction model based on the input prediction time. This application does not specifically limit the prediction result data.

[0120] The embodiments of this application can efficiently improve the prediction performance of the prediction model while ensuring recall and precision, thereby increasing the accuracy of model predictions and improving user efficiency. It can reduce the impact of noise or other unstable factors on the prediction of models trained on non-stationary data, thus improving model performance. It can efficiently improve the prediction performance of long-term models while maintaining recall and precision.

[0121] Figure 3This is a schematic diagram illustrating an application scenario of the data prediction method provided in one embodiment of this application. The embodiment of this application includes two core stages in practical application. The first stage uses a fixed model to predict recent data to obtain pseudo-positive samples, then merges new samples and adjusts the weights of true and pseudo samples to maximize the utilization effect of data from different periods in prediction. The second stage uses a change point detection model to capture external features reflecting the temporal characteristics of the data. Finally, combining the new samples and dynamic temporal features supplemented in the first two stages, the fixed model is updated to improve the prediction effect of the long-term model. Figure 3 As shown, on the timeline, historical data (March-April) is in the following state: adjustment completed, positive and negative samples complete; recent data (May) is in the following state: adjustment incomplete, positive and negative samples incomplete; and predicted data (June). The anomaly detection model (Catboost), i.e., the first prediction model in this embodiment, is used to perform rolling predictions on the historical data (March-April) to fix the model parameters. The model with fixed parameters (the anomaly detection model (Catboost)) is then used to predict the recent data (May) to obtain new samples (including pseudo-positive and true-positive samples). These new samples are then combined with historical data and input change point features to update and train the fixed model, ultimately resulting in the Updated model (Catboost), i.e., the target prediction model of this application. The Updated model (Catboost) is then used to accurately predict the data for the prediction time (June) and output the prediction results.

[0122] As a general example of this application, the following is illustrated:

[0123] Phase 1, Sample Augmentation:

[0124] In billing and settlement scenarios, the business performs weekly advance payments and monthly settlements for each billing result. The entire verification and settlement process takes two months to complete. Due to the nearly ten million data points per month for a single business line and over a hundred features, the initial model was based on the business's monthly settlement logic, using historical data from the past three months to predict the data for one month (four weeks in total), and the weekly predictions were automatically distributed.

[0125] Basic Model Training: Data Cleaning: Each settlement order has multiple dimensions of features (waybill information, merchant information, carrier information, package information, product information, etc.), with over 80% of the features being categorical. The following are the basic processing steps for the input data of the Catboost model. Feature Selection: First, in feature selection, features with only one unique value are removed from the categorical features. Features with a null value rate exceeding 80% are also removed or supplemented through interpolation. Format Conversion: Due to the large data volume, numerous features, and the complexity and proportion of categorical features, considering both model efficiency and categorical feature processing efficiency, Catboost will be used as the basic training model in this scenario. First, based on real-world data, features are divided into numerical features (num_features) and categorical features (cate_features). To avoid Catboost's inability to handle NaN (Not a Number) and empty strings, NaN and empty strings for categorical features are filled with "Missing". Another advantage of this approach is that for categorical features with many missing values, empty values ​​are also considered as a category. Hyperparameter search: Model performance is influenced not only by feature effectiveness but also by model parameters. Model parameters: In Catboost, iterations, learning rate, and depth are the most fundamental parameters that directly affect model performance. To prevent overfitting and improve training efficiency, regularization and training device parameters also need to be set. The parameters are explained below: iterations: The number of iterations during training. A larger value results in a more complex model; care should be taken to avoid excessive values ​​that could lead to overfitting. learning rate: The step size for each iteration. A smaller learning rate may require more iterations but may be more robust. `depth`: The depth of the tree. A larger value indicates a more complex model, but may lead to overfitting. `l2_leaf_reg`: For large datasets or complex models, sets the coefficient of the L2 regularization term to prevent overfitting. `task_type`: The type of training device. Setting it to GPU significantly accelerates model training. Searching for optimal parameters: Custom evaluation metrics: In the basic model's performance evaluation, precision and recall both measure the number of predicted items, i.e., the precision and recall of a single item. However, in actual business scenarios, it is necessary to pay more attention to the precision and recall of abnormal item amounts. Therefore, custom evaluation metrics representing the amount prediction performance are `money_presision` and `money_recall`, and the model performance score is defined as: `score = a1 × money`. precision + a1×money recall(a1 and a2 represent the weights assigned to precision and recall, respectively; different weights can be customized for different business needs). Hyperopt hyperparameter optimization: Compared to ordinary grid search, Hyperopt is more efficient because it uses Bayesian optimization, which can adjust the hyperparameter values ​​for the next test based on the results of previous trials; and it supports parallel computing. Here, the parameter search space is:

[0126]

[0127]

[0128]

[0129] Set the number of training loops: 3 training loops in total. The total score is the sum of the scores from the 3 training loops. The overall score is the average of the 3 training loops. Since Hyperopt minimizes the optimal parameters by default, in order to maximize money_presision and money_recall, a negative sign is needed to set the parameters to be maximized: score = -((TotalScore) / 3).

[0130] Rolling Prediction Evaluation: To ensure the stability of the long-term model training cycle and performance, rolling predictions of historical data are necessary to fix the time window and parameters used by the prediction model. Using the three months preceding the predicted data as the training time window, weekly training results for April to June are output on a rolling basis, and then fixed parameters are selected based on the actual training results. In the direct feed scenario, the initial fixed parameters for the anomaly detection Catboost model are set to {depth=6, learning_rate=0.17}.

[0131] New Sample Processing: Pseudo-sample Output: When predicting data for the first week of June, the May data, which has not yet been adjusted, is considered recent data. Using the pre-trained Catboost model for March and April, the May data is predicted, and outlier samples in the prediction results are treated as pseudo-positive samples. Sample Merging: The adjusted portion of the recent data (in this case, May) is considered true-positive, defined as `true_positive`. The pseudo-positive samples predicted as outliers in the previous step, after deduplication with `true_positive`, are defined as `false_positive`. These two positive samples are merged as new positive samples, which are then added to the model for incremental training.

[0132] Phase 2, Change Point Detection:

[0133] Change point detection is a method for identifying time points in time series data where statistical characteristics change significantly. In the context of settlement anomaly identification, when key factors for determining whether to adjust accounts (such as settlement rules and adjustment logic) change significantly, this method can capture whether certain features of the sample have changed significantly at a certain point in time. The change point information is then processed and added to the model as external features to optimize the model's prediction performance for future samples. The following is a specific scheme for using change point detection in the Catboost model to introduce time features and sample change characteristics. Data preprocessing and feature selection: Considering the data volume and model efficiency, change point detection here only targets features with practical business significance and significant contributions to the model's performance. Therefore, important features need to be selected before change point detection. Calculation logic: The Catboost model is composed of trees. Assuming there are m trees, each split is based on a single feature, and each tree consists of a series of splits. For feature i, its importance is the sum of the gains it brings in all tree splits. The specific calculation formula is as follows:

[0134]

[0135] Where Gain(j,k) is the gain from the j-th split in the k-th tree. It represents all splits of the k-th tree based on feature i.

[0136] Obtain important features: Use CatBoost's model.get_importance() to obtain the importance of each feature, sort the obtained feature importance in descending order, and select the top 20 features (accounting for about 30% of all features) for subsequent change point detection.

[0137] Feature transformation: Since over 80% of the features in real-world scenarios are categorical features, in order to perform change point detection on the categorical data of the above important features, it is necessary to convert them into numerical values ​​and generate a sliding window categorical frequency. By using CUSUM (a statistical process control tool for detecting small shifts, mainly used to monitor quality issues in the production process to ensure that products meet standards) on this frequency data, change point detection of categorical features can be achieved.

[0138] Numericalized categorical features: Categorical features are transformed into numerical data, and a unique integer value is assigned to each category. For example, for the vehicle type "carrier_type", its possible values ​​are ['1-2-3-4', '1-2', '1-3-5-8-0'], which can be mapped to [1, 2, 3].

[0139] Calculate the frequency of categories in the sliding window: Since the training period of the fixed model is 3 months and the prediction is weekly, the window size is set to 3 months. For each window position, the frequency of occurrence of each category feature in that window is calculated and used as the feature for change point detection. At this time, the category data has been transformed into a series of frequency data, which meets CUSUM's requirement for continuous data.

[0140] Data sorting: For time series problems, order is key, so sort the data by date for subsequent change point detection.

[0141] Change point detection is performed using the CUSUM algorithm, which is better able to detect small, gradual changes in time series data and is computationally simple and easy to implement. Therefore, this approach uses CUSUM to detect change points on the important features selected earlier. The core idea of ​​this algorithm is to accumulate historical information (i.e., calculate the cumulative deviation between data points and the mean) to determine whether a feature has undergone statistically significant changes over time.

[0142] Calculation logic: For each processed important feature frequency data, calculate the residual of its cumulative time series data. When the cumulative sum of the residuals exceeds a certain preset threshold, CUSUM will trigger a signal indicating that the process in the data may have changed.

[0143] Threshold setting: Using the corresponding period of the training data as a benchmark, calculate the CUSUM standard deviation for that period. Set the threshold as follows: Where k is 2, meaning that change point detection will capture changes that exceed 2 standard deviations from the mean.

[0144] Variable point filtering: Let S be the variable point. t Let be the CUSUM value at time t. When |S0|>h, time t is considered a change point, where h is a preset threshold.

[0145] New Feature Creation: Based on the results of change point detection, the following three features are generated to reflect the temporal changes of newly added data.

[0146] Time since the last anomaly: This feature helps the model understand the time interval since the last anomaly.

[0147] Create a new feature D t , and initialize it to 0. Traverse the CUSUM sequence, and whenever a change point (i.e., |S0|>h) appears, then set D. t Reset to 0. For non-variable points, set D... t Set as the time step since the last change point (i.e., the time elapsed since the last change point), let T be... t Given the time of the previous change point, then feature D t It can be represented as: Dt =tT t .

[0148] CUSUM value: directly convert the CUSUM sequence S t As a new feature, namely new feature C t =S t S t It is the cumulative sum of the residuals from the first data point to the data point at time t, i.e. , where y i is the value of the i-th data point, and μ is the mean of the data points during the stable period of this feature.

[0149] Is it a change point? This feature is a binary feature generated based on the change point detection results. The specific formula is:

[0150]

[0151] Where h is a preset threshold for each feature, when |S t |When the time exceeds h, it is considered that a change point has occurred at time t, so V(t) is 1; otherwise, it is 0.

[0152] The third stage, model update:

[0153] Data Preparation: Merging Data: Merge the historical data used by the Catboost model (i.e., the original training data) with the recently added data (including real samples and pseudo-positive samples) into a new training set. Assigning Weights: Assign weights to each sample in the new dataset. As before, the weight of a real sample is ω1, and the weight of a pseudo-positive sample is ω2, where ω1 > ω2. Adding Change Point Features: Based on the previous change point detection, add change point features to the new data.

[0154] Model Training: Incremental training is performed on the model in the short term, while full training is conducted in the long term to update change point features. Incremental Training: When updating the model with new data in the initial stage, change point features are generated for the new data, then merged with pseudo-positive samples, assigned appropriate weights, and only this part of the data is used for "incremental training" of the model. Periodic Full Training: After accumulating a certain amount of new data, change point detection is performed again, change point features are generated for all data (historical + new), and full training is conducted to continuously update the Catboost anomaly detection model as a fixed training version.

[0155] Due to the application of change point detection, the data prediction capability of the basic model is significantly improved by capturing the temporal characteristics of important features. By predicting the change points of important features in the newly added data, and supplementing with both true and false samples, incremental training is performed on the basic Catboost model (i.e., the first prediction model in this embodiment), resulting in significant optimization of the final performance. Compared to the basic Catboost model, this solution improves model precision by 6% and recall for reducing order volume and amount by over 2%. In actual capacity settlement scenarios, with tens of millions of orders per month, this level of model improvement will have a greater positive impact on business. This embodiment utilizes false sample processing and change point detection to increase the effective use of recent data and capture the temporal characteristics of important features in a timely manner, significantly improving the prediction effect and stability of settlement order anomaly identification in logistics scenarios. It solves the prediction reliability problem of classification models on time series data, is more in line with actual application scenarios, and uses intelligent means to help capacity settlement personnel improve verification efficiency and accuracy.

[0156] Figure 4 This is a schematic diagram of the main units of a data prediction apparatus according to an embodiment of this application. Figure 4 As shown, the data prediction device 400 includes an acquisition unit 401, a prediction unit 402, a sample merging unit 403, a first training unit 404, a second training unit 405, and an output unit 406.

[0157] The acquisition unit 401 is configured to acquire the corresponding prediction time in response to a data prediction request, and then determine the corresponding recent time based on the prediction time.

[0158] Prediction unit 402 is configured to invoke a first prediction model to predict recent data in order to identify outlier samples in the prediction results as pseudo-positive samples.

[0159] The sample merging unit 403 is configured to determine the adjusted sample corresponding to the recent time as the true positive sample, and merge the new sample based on the false positive sample and the true positive sample to obtain the new positive sample.

[0160] The first training unit 404 is configured to call the change point detection model to add change point features to the newly added positive samples, determine the weights of the newly added positive samples, and perform incremental training on the first prediction model based on the newly added positive samples, weights, and change point features to obtain the second prediction model.

[0161] The second training unit 405 is configured to update the variable point features to periodically train the second prediction model to obtain the target prediction model.

[0162] Output unit 406 is configured to input the prediction time into the target prediction model to output prediction result data.

[0163] In some embodiments, the first training unit 404 is further configured to: invoke a change point detection model to obtain the feature importance of each feature in the newly added positive samples; based on the feature importance, select important features, perform change point detection based on the important features to obtain change point detection result data; and add change point features to the newly added positive samples based on the change point detection result data.

[0164] In some embodiments, the first training unit 404 is further configured to: determine the categorical features among the important features, convert the categorical features into corresponding numerical data; allocate corresponding windows for the numerical data based on a preset window size, calculate the frequency of occurrence of the numerical data in each window; and obtain change point detection result data based on the frequency of occurrence.

[0165] In some embodiments, the first training unit 404 is further configured to: determine the cumulative residual sum of each numerical data based on the cumulative time series according to the frequency of occurrence; determine the numerical data where a change point occurs according to a preset threshold of the cumulative residual sum; and generate change point detection result data based on the numerical data where a change point occurs.

[0166] In some embodiments, the preset threshold is determined by the following steps: determining the period of the training data of the target prediction model, calculating the offset standard deviation based on the period; and determining the preset threshold based on the offset standard deviation and the preset weight.

[0167] In some embodiments, the first training unit 404 is further configured to: set the weight of the true positive samples in the newly added samples as the first weight, and set the weight of the false positive samples as the second weight, wherein the first weight is greater than the second weight.

[0168] In some embodiments, the first training unit 404 is further configured to: merge the newly added positive samples with the original training data to form a new training set; update the weights of the samples in the new training set based on the weights; and incrementally train the first prediction model based on the new training set, the updated weights, and the change point features to obtain a second prediction model.

[0169] In some embodiments, the second training unit 405 is further configured to generate change point features for samples in the new training set that have changed points and update the change point features of the samples in the new training set.

[0170] It should be noted that the data prediction method and data prediction device in this application are related in terms of specific implementation, so repeated content will not be described again.

[0171] Figure 5An exemplary system architecture 500 is shown that can be applied to the data prediction method or data prediction apparatus of the embodiments of this application.

[0172] like Figure 5 As shown, system architecture 500 may include terminal devices 501, 502, and 503, a network 504, and a server 505. Network 504 serves as the medium for providing communication links between terminal devices 501, 502, and 503 and server 505. Network 504 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0173] Users can use terminal devices 501, 502, and 503 to interact with server 505 via network 504 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 501, 502, and 503, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0174] Terminal devices 501, 502, and 503 can be various electronic devices with a data prediction and processing screen that support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0175] Server 505 can be a server providing various services, such as a backend management server (for example only) supporting data prediction requests submitted by users using terminal devices 501, 502, and 503. The backend management server can respond to the data prediction request by obtaining the corresponding prediction time, and then determining the corresponding recent time based on the prediction time; calling the first prediction model to predict the data for the recent time, identifying anomalous samples in the prediction results as pseudo-positive samples; identifying the adjusted samples corresponding to the recent time as true-positive samples; merging new samples based on the pseudo-positive and true-positive samples to obtain new positive samples; calling the change point detection model to add change point features to the new positive samples, determining the weights of the new positive samples; incrementally training the first prediction model based on the new positive samples, weights, and change point features to obtain a second prediction model; updating the change point features to periodically perform full training on the second prediction model to obtain the target prediction model; and inputting the prediction time into the target prediction model to output the prediction result data. It can efficiently improve the predictive performance of the prediction model while ensuring recall and precision, increase the accuracy of model predictions, and improve user work efficiency.

[0176] It should be noted that the data prediction method provided in this application embodiment is generally executed by server 505, and correspondingly, the data prediction device is generally set in server 505.

[0177] It should be understood that Figure 5 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0178] The following is for reference. Figure 6 It shows a schematic diagram of the structure of a computer system 600 suitable for implementing a terminal device according to the embodiments of this application. Figure 6 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0179] like Figure 6 As shown, the computer system 600 includes a central processing unit (CPU) 601, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 602 or programs loaded from storage section 608 into random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the computer system 600. The CPU 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0180] The following components are connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 610 as needed so that computer programs read from it can be installed into storage section 608 as needed.

[0181] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 609, and / or installed from removable medium 611. When the computer program is executed by central processing unit (CPU) 601, it performs the functions defined above in the system of this application.

[0182] It should be noted that the computer-readable medium shown in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0183] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0184] The units described in the embodiments of this application can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor can be described as including an acquisition unit, a prediction unit, a sample merging unit, a first training unit, a second training unit, and an output unit. The names of these units do not necessarily limit the specific unit itself.

[0185] In another aspect, this application also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to respond to a data prediction request, obtain the corresponding prediction time, and then determine the corresponding recent time based on the prediction time; invoke a first prediction model to predict the data for the recent time, identifying abnormal samples in the prediction results as pseudo-positive samples; identify the adjusted samples corresponding to the recent time as true-positive samples, and merge new samples based on the pseudo-positive samples and true-positive samples to obtain new positive samples; invoke a change point detection model to add change point features to the new positive samples, determine the weights of the new positive samples, and incrementally train the first prediction model based on the new positive samples, weights, and change point features to obtain a second prediction model; update the change point features to periodically perform full training on the second prediction model to obtain a target prediction model; and input the prediction time into the target prediction model to output prediction result data.

[0186] The technical solution of this application embodiment can efficiently improve the prediction effect of the prediction model, improve the accuracy of model prediction, and improve user work efficiency while ensuring recall and precision.

[0187] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A data prediction method, characterized in that, include: In response to a data prediction request, the corresponding prediction time is obtained, and then the corresponding near time is determined based on the prediction time; The first prediction model is invoked to predict the recent data, and abnormal samples in the prediction results are identified as pseudo-positive samples. The adjusted accounts sample corresponding to the recent time is determined as the true positive sample. Based on the false positive sample and the true positive sample, the new sample is merged to obtain the new positive sample. The change point detection model is invoked to add change point features to the newly added positive samples, the weights of the newly added positive samples are determined, and the first prediction model is incrementally trained based on the newly added positive samples, the weights and the change point features to obtain the second prediction model. The variable point features are updated to perform periodic full training on the second prediction model, thereby obtaining the target prediction model; The predicted time is input into the target prediction model to output the prediction result data.

2. The method according to claim 1, characterized in that, The step of calling the change point detection model to add change point features to the newly added positive samples includes: Invoke the change point detection model to obtain the feature importance of each feature in the newly added positive samples; Based on the importance of the aforementioned features, important features are selected, and change point detection is performed based on these important features to obtain change point detection result data. Based on the change point detection results, change point features are added to the newly added positive samples.

3. The method according to claim 2, characterized in that, The step of performing change point detection based on the aforementioned important features to obtain change point detection result data includes: Identify the categorical features among the important features, and convert the categorical features into corresponding numerical data; Based on a preset window size, a corresponding window is assigned to the numerical data, and the frequency of occurrence of the numerical data in each window is calculated. Based on the frequency of occurrence, change point detection result data is obtained.

4. The method according to claim 3, characterized in that, The process of obtaining change point detection result data based on the occurrence frequency includes: Based on the frequency of occurrence, determine the cumulative residual sum for each numerical data point based on the cumulative time series; Based on the accumulated residuals and the preset threshold, the numerical data of the point of change is determined; Based on the numerical data of the changing points, change point detection result data is generated.

5. The method according to claim 4, characterized in that, The preset threshold is determined through the following steps: Determine the period of the training data for the target prediction model, and calculate the offset standard deviation based on the period. A preset threshold is determined based on the offset standard deviation and the preset weight.

6. The method according to claim 1, characterized in that, Determining the weight of the newly added positive sample includes: The weight of the true positive samples in the newly added samples is set as the first weight, and the weight of the false positive samples is set as the second weight, wherein the first weight is greater than the second weight.

7. The method according to claim 1, characterized in that, The step of incrementally training the first prediction model based on the newly added positive samples, the weights, and the change point features to obtain the second prediction model includes: The newly added positive samples are merged with the original training data to form a new training set; Based on the weights, update the weights of the samples in the new training set; The first prediction model is incrementally trained based on the new training set, the updated weights, and the change point features to obtain the second prediction model.

8. The method according to claim 7, characterized in that, Updating the variable point feature includes: Generate change point features for the samples in the new training set that have changed points, and update the change point features of the samples in the new training set.

9. A data prediction device, characterized in that, include: The acquisition unit is configured to acquire the corresponding prediction time in response to a data prediction request, and then determine the corresponding recent time based on the prediction time. The prediction unit is configured to invoke a first prediction model to predict the recent data and identify anomalous samples in the prediction results as pseudo-positive samples. The sample merging unit is configured to determine the adjusted sample corresponding to the recent time as the true positive sample, and merge the new sample based on the false positive sample and the true positive sample to obtain the new positive sample; The first training unit is configured to call the change point detection model to add change point features to the newly added positive samples, determine the weights of the newly added positive samples, and incrementally train the first prediction model based on the newly added positive samples, the weights, and the change point features to obtain the second prediction model. The second training unit is configured to update the variable point features to perform periodic full training on the second prediction model, thereby obtaining the target prediction model. The output unit is configured to input the prediction time into the target prediction model to output prediction result data.

10. The apparatus according to claim 9, characterized in that, The first training unit is further configured to: Invoke the change point detection model to obtain the feature importance of each feature in the newly added positive samples; Based on the importance of the aforementioned features, important features are selected, and change point detection is performed based on these important features to obtain change point detection result data. Based on the change point detection results, change point features are added to the newly added positive samples.

11. A data prediction electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-8.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-8.

Images