Score calibration method and device for score card prediction model

Abstract

The invention relates to a score calibration method for a score card prediction model. The method comprises the following steps: training a classifier by utilizing first user multi-dimensional data and a behavior prediction result; predicting second user multi-dimensional data by using the classifier to obtain a prediction result; and obtaining a calibration probability value between the main scorecard prediction model and the standby scorecard prediction model according to the prediction result. According to the method, the main model and the standby model which are used as the scoring card prediction model are realized, if the external data stops providing service, the main model can be switched to the standby model, the score of the main model and the score of the standby model are calibrated to be one scale, and downstream business calling is not perceived. The invention further relates to equipment and a storage medium.

Description

Technical Field

[0001] The present application relates to the technical field of big data machine learning model algorithms, and in particular to a score calibration method and device for a scorecard prediction model. Background Art

[0002] The scorecard prediction model is a machine learning model used to predict the results of user behavior. Its goal is to identify possible factors before some user behavior states such as credit overdue, transaction willingness, and credit qualification occur. As the main model and backup model of the scorecard prediction model, if the external data stops providing services, the main model can be switched to the backup model. Therefore, how to make the downstream business calls unaware when switching between the main model and the backup model is a technical problem that needs to be solved urgently. Summary of the invention

[0003] In view of the above-mentioned problems existing in the switching between the main model and the backup model of the existing scoring card prediction model, the present application provides a score calibration method and device for the scoring card prediction model.

[0004] In a first aspect, the present application provides a score calibration method for a scorecard prediction model, the method comprising:

[0005] Training a classifier using the multi-dimensional data of the first user and the behavior prediction result;

[0006] Using the classifier to predict the second user's multidimensional data to obtain a prediction result;

[0007] According to the prediction result, a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained.

[0008] Furthermore, the method of training a classifier using the user multi-dimensional data and the behavior prediction results specifically includes:

[0009] training a first classifier using the first user multidimensional data and a first behavior prediction result, wherein the first behavior prediction result is obtained by inputting the first user multidimensional data into the primary scorecard prediction model;

[0010] The second classifier is trained using the first user multidimensional data and the second behavior prediction result, where the second behavior prediction result is obtained by inputting the first user multidimensional data into the backup scorecard prediction model.

[0011] Furthermore, the using the classifier to predict the user multi-dimensional data to obtain a prediction result specifically includes:

[0012] Using the first classifier to predict the second user multi-dimensional data to obtain a first prediction result;

[0013] The second classifier is used to predict the second user multi-dimensional data to obtain a second prediction result.

[0014] Further, obtaining a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result specifically includes:

[0015] Training a first converter using the first prediction result;

[0016] Training a second converter using the second prediction result;

[0017] Inputting the third user multi-dimensional data into the first converter to obtain a first result;

[0018] Inputting the third user multi-dimensional data into the second converter to obtain a second result;

[0019] A calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained according to the first result, the second result and the transformation function.

[0020] Further, obtaining the calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the first result, the second result and the transformation function specifically includes:

[0021] The first result and the second result are input into the transformation function to obtain a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model.

[0022] Further, obtaining a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result specifically includes:

[0023] Performing strategic distribution on the first prediction result and the second prediction result;

[0024] Determining the linear transformation coefficient of the primary scorecard prediction model and the linear transformation coefficient of the backup model according to the result of the strategy distribution;

[0025] A calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained according to the linear transformation coefficient of the primary scoring card prediction model and the linear transformation coefficient of the backup model.

[0026] Further, the determining of the linear transformation coefficient of the primary scorecard prediction model and the linear transformation coefficient of the backup model according to the result of the strategy distribution specifically includes:

[0027] When the values ​​of the same policy item in the results of the policy distribution are the same, obtaining the decision threshold of the primary scorecard prediction model and the decision threshold of the backup model;

[0028] According to the decision threshold of the primary scoring card prediction model and the decision threshold of the backup model, a linear transformation coefficient of the primary scoring card prediction model and a linear transformation coefficient of the backup model are obtained.

[0029] In a second aspect, the present application also provides a score calibration device for a scorecard prediction model, the device comprising:

[0030] A first processing module, used for training a classifier using the multi-dimensional data of the first user and the behavior prediction result;

[0031] A second processing module, used to predict the second user's multi-dimensional data using the classifier to obtain a prediction result;

[0032] The third processing module is used to obtain a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result.

[0033] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the score calibration method for a scoring card prediction model described in any one of the first aspects is implemented.

[0034] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the score calibration method for a score card prediction model described in any one of the first aspects is implemented.

[0035] The present application provides a score calibration method for a scoring card prediction model, including training a classifier using multi-dimensional data and behavior prediction results of a first user; using the classifier to predict multi-dimensional data of a second user to obtain a prediction result; and obtaining a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result. The present invention implements a primary model and a backup model as a scoring card prediction model. If external data stops providing services, the primary model can be switched to the backup model. The scores of the primary model and the backup model are calibrated to a scale, and downstream business calls are unaware. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0037] In the attached picture:

[0038] Figure 1 is a schematic diagram of a calibration curve provided in an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of probability distribution of a master-slave model provided in another embodiment of the present application;

[0040] Figure 3 is a schematic diagram of the strategy distribution of the main model provided by another embodiment of the present application;

[0041] Figure 4 is a schematic diagram of strategy distribution of a backup model provided by another embodiment of the present application;

[0042] Figure 5 is a schematic diagram of probability distribution of a calibrated primary and standby model provided in another embodiment of the present application;

[0043] Figure 6 It is a schematic diagram of the strategy distribution of the main model provided in the embodiment of the present application;

[0044] Figure 7 is a schematic diagram of strategy distribution of a backup model provided by another embodiment of the present application;

[0045] Figure 8 is a schematic diagram of a clustering model scenario provided by another embodiment of the present application;

[0046] Fig. 9 is a schematic diagram of a downgrade standby strategy provided by another embodiment of the present application;

[0047] Fig.10 is a schematic diagram of probability distribution of a master-slave model provided in another embodiment of the present application;

[0048] Fig.11 It is a schematic diagram of the strategy distribution of the main model provided in the embodiment of the present application;

[0049] Fig.12 is a schematic diagram of strategy distribution of a backup model provided by another embodiment of the present application;

[0050] Fig.13 is a schematic diagram of probability distribution of a calibrated primary and standby model provided in another embodiment of the present application;

[0051] Fig.14 is a schematic diagram of the strategy distribution of the main model provided by another embodiment of the present application;

[0052] Fig.15 is a schematic diagram of strategy distribution of a backup model provided by another embodiment of the present application;

[0053] Fig.16It is a schematic diagram of the strategy distribution of the main model provided in the embodiment of the present application;

[0054] Fig.17 is a schematic diagram of strategy distribution of a backup model provided by another embodiment of the present application;

[0055] Fig.18 is a schematic diagram of the strategy distribution of the main model provided by another embodiment of the present application;

[0056] Fig.19 It is a schematic diagram of the strategy distribution of the backup model provided in another embodiment of the present application.

[0057] Fig. 20 is a schematic diagram of a score calibration method for a scorecard prediction model provided by another embodiment of the present application;

[0058] Fig.21 It is a schematic diagram of a score calibration device for a score card prediction model provided in another embodiment of the present application. DETAILED DESCRIPTION

[0059] The present invention will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present invention can be combined with each other without conflict.

[0060] The following detailed description is an exemplary description, which is intended to provide further detailed description of the present invention. Unless otherwise specified, all technical terms used in the present invention have the same meaning as those generally understood by those skilled in the art to which the present invention belongs. The terms used in the present invention are only for describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present invention.

[0061] In machine learning modeling, the probability score output by the machine learning model is usually converted into an integer score, which is called model score calibration. In fact, this stage may be more appropriately called scaling. However, some books do not strictly distinguish between calibration and scaling, and collectively refer to them as risk calibration or score calibration.

[0062] The concept of score calibration, in the practical application of machine learning models, in most cases, focuses on the ranking of classification models, and rarely cares about the specific numerical value of the output probability. In other words, focus on relative values ​​and ignore absolute values.

[0063] On the one hand, some classifiers (such as SVM) can only directly label the category but cannot give confidence. On the other hand, in some scenarios, we hope to get the real probability. For example, in the field of credit risk control, the predicted customer default probability (Probability of Default, PD) is compared with the real default probability, that is, the probability of model risk can represent the real risk level. In this way, we can perform more accurate risk pricing.

[0064] This leads to the concept of calibration, which is understood as the statistical consistency between the predicted distribution and the true distribution (observation).

[0065] For a perfectly calibrated (binary) classifier, if the classifier predicts that a sample belongs to the positive class with a probability of 0.8, then it should be said that there is an 80% confidence level that the sample belongs to the positive class, or that 80 out of 100 samples with a probability of 0.8 do belong to the positive class.

[0066] Since the true conditional probability cannot be known, the labels of the observed samples are usually used as statistics instead, and reliability curve diagrams are used to intuitively show how much the output results of the current model deviate from the true results.

[0067] like Figure 1 As shown in the figure, if the data points almost all fall on the diagonal line, it means that the model is well calibrated; conversely, if the deviation from the diagonal line is more obvious, the calibration is worse.

[0068] Reliabilitydiagrams(Hartmannetal.2002)aresimplygraphsoftheObservedfrequenc yofanevent plottedagainst theForecastprobabilityofanevent.

[0069] Thiseffectivelytellstheuserhowoften(asapercentage)aforecastprobabilityactuallyoccurred.

[0070] Therefore, the horizontal axis is the predicted probability of an event, and the vertical axis is the actual frequency of the event, which can show the relationship between "the predicted probability of an event VS the actual frequency of occurrence". For an ideal prediction system, the two are completely consistent, that is, the diagonal line.

[0071] The steps for drawing the reliability curve are as follows:

[0072] step1. Horizontal axis: Arrange the predicted probabilities in ascending order and select a threshold, where [0, threshold] is used as a box.

[0073] step2. Vertical axis: calculate the hit rate (hitrate) in this box, that is, the positive sample rate.

[0074]

[0075] step3. Select multiple thresholds, repeat the calculation to obtain multiple points, and connect them into lines.

[0076] Let's use the scenario of tossing a coin to evaluate the probability of the front side facing up to illustrate. After tossing the records, we get 1W (or more) samples with real 0 (tails facing up) and 1 (heads facing up). Then, we train a binary classification model based on the various features of the coin and the front and back labels, score this batch of samples, and sort them in ascending order. Then, take a score as the threshold (such as 0.5), and count the actual proportion of 1 in this batch of samples with scores ranging from 0 to 0.5 as the "real" probability. Finally, compare the "real" probability with the predicted probability to detect consistency.

[0077] Existing technical solutions

[0078] In view of the above technical background, the existing technical solutions include the following two most common probability calibration methods:

[0079] Plattscaling uses the LR model to fit the values ​​​​of the model output (not

[0080] The reliability diagram is used for fitting, which is suitable for situations with small sample sizes, such as credit risk control scenarios.

[0081] Isotonicregression fits the data in the reliabilitydiagram and is suitable for situations with a large number of samples, such as search recommendation scenarios. When the sample size is small, isotonicregression is prone to overfitting.

[0082] Scheme: Platt calibration scheme

[0083] Assume that there is currently one LR model as the main model and another LR model as the backup model to simulate scenarios 1 and 2. At this time, the ranking of the main model and the backup model is the same, but the absolute values ​​are different, corresponding to different risk levels, such as Figure 2-4 The main model and the backup model need to be calibrated to the same scale.

[0084] The prediction result of SVM can be converted into a posterior probability value through the sigmoid function. The implementation process is as follows:

[0085] Step 1. Use sample feature X and target variable y to train a classifier model1. (Classifier type is not limited)

[0086] step2. Use model1 to predict the sample and get the prediction result out.

[0087] Step 3. Use the predicted result out as the new feature X', and then use the sample label y to train an LR.

[0088] step4. The probability value finally output by LR is the predicted probability after platt's scaling.

[0089] Execute step 3 on the active model and the standby model respectively to obtain the calibrated fractional active model_platts and standby model_platts, as shown in Figure 5-7 As shown in the figure, the difference between the two scores is almost 0, so they have similar risk levels. At the same time, since the calibration function is monotonic, the sorting and discrimination will not be affected before and after the calibration.

[0090] pass Figure 5 Probability distribution comparison, Figure 6-7 From the red range column in the strategy distribution, we can see that the difference between the two scores is almost 0 after Platt calibration, but there is still a difference. For the same cutoff, there is a significant difference in the "Cumulative Capture Rate" column.

[0091] As shown in step 3 of Platt's implementation process, the label y of the sample is required for implementation. In actual business, the label y of the sample may not be available.

[0092] After Platt calibration, the score distribution is not normal.

[0093] The “percentage of positive samples in the cumulative captured samples” in the red box in the strategy distribution of Table 3 and Table 4

[0094] The main model is 15.4364% with a cutoff of 0.058, and the backup model is 15.8571% with a cutoff of 0.086. It can be seen that for the same risk level and the same proportion of captured positive samples, the cutoff is not the same.

[0095] The scorecard prediction model is a machine learning model used to predict the results of various user behaviors (such as credit delinquency, transaction willingness, credit qualification). The goal of this type of model is to identify factors that may affect the behavior results before the behavior results occur, allowing marketing, risk control and other practitioners to take measures in advance and intervene in the user.

[0096] The main objectives of the scorecard prediction model include but are not limited to: Optimize user experience: Through accurate data analysis and personalized strategies, ensure that users enjoy a smooth, convenient and safe experience throughout the transaction process, thereby improving user satisfaction and loyalty. Improve decision-making efficiency and accuracy: With the help of advanced algorithms and models, quickly identify and evaluate potential market opportunities and risks, provide data support for management, and help efficient and wise decision-making. Enhance customer segmentation and precision marketing: Through deep insights into customer behavior, achieve more refined customer classification, and then implement personalized marketing strategies to improve the input-output ratio of marketing resources. Promote business growth and market share expansion: Through accurate market positioning and efficient marketing strategies, effectively attract new customers, while deepening cooperation with existing customers, and promoting the sustainable and healthy growth of the business. Strengthen risk identification and prevention and control capabilities: Establish a comprehensive risk monitoring system, monitor potential risk points in real time, warn in advance and take effective measures to ensure the safety and stability of business operations. Optimize cost structure and operational efficiency: Use data analysis to optimize resource allocation, reduce unnecessary waste, and improve internal operational efficiency to create more value for the company.

[0097] The data used to train the scorecard prediction model usually includes: user basic attribute data, user behavior data. External data: multi-head data, third-party payment data, payment settlement data, Internet vertical data, SDK data, operator data, and government data.

[0098] Feature engineering refers to extracting useful features from raw data that can help the model better understand and predict the outcome state. Feature engineering may include: Time series features: such as time of day, day of the week, etc. Statistical features: such as mean, variance, maximum, minimum, etc. Aggregate features: such as hourly and daily data summaries.

[0099] Scorecard predictions can use a variety of machine learning techniques, including but not limited to:

[0100] Supervised learning: such as logistic regression, support vector machine (SVM), random forest, gradient boosting tree (such as XGBoost or LightGBM), etc. Unsupervised learning: such as clustering algorithm (K-means), anomaly detection (such as IsolationForest), etc. Deep learning: such as long short-term memory network (LSTM), convolutional neural network (CNN), etc.

[0101] Training a scorecard prediction model usually includes the following steps: Sample design: determine the research objectives and select a representative set of samples; consider the diversity of samples to ensure that the model can work effectively in different scenarios. Feature engineering: clean data, handle missing values, convert non-numerical features, and scale features. Data preparation: divide the data set into training sets, validation sets, and test sets; ensure the quality and completeness of the data set. Feature selection: identify which features are most valuable for prediction; remove redundant or noisy features to improve the generalization ability of the model. Model training: use training set data to fit the model and adjust model parameters to optimize performance. Model evaluation: use test set data to evaluate the performance of the model; common evaluation indicators include accuracy, precision, recall, F1 score, AUC-ROC curve, etc.; adjust the model according to the evaluation results until a satisfactory performance level is achieved.

[0102] Once the model is trained and deployed, it can be run in real time or periodically to predict possible future user states.

[0103] Scorecard prediction models have a wide range of applications, including but not limited to financial marketing, financial risk control, insurance marketing, education marketing, medical diagnosis, quantitative investment, etc. By effectively utilizing these models, organizations can significantly improve the stability and efficiency of their business.

[0104] This calibration scheme can achieve the same capture rate of the two models under the same cutoff. In the process of score calibration, no other restrictions such as sample y labels are required, which is easy to implement. It can calibrate irregular distribution to normal distribution at the same scale. It can achieve the same proportion of positive samples captured by the two models under the same cutoff.

[0105] Figure 8 As shown, application scenario 1: used to unify the score cutoff points under multiple clustering models

[0106] If a single model does not perform well on all data, we will first segment the data and then build multiple sub-models for each data.

[0107] There is a need to calibrate scores to the same scale for several reasons.

[0108] 1. Multiple sets of strategies need to be formulated for multiple branch models, which will greatly increase the workload of strategy personnel and is not conducive to strategy maintenance and adjustment.

[0109] 2. The scores output by different models are not comparable, and their distributions are different. In order to output a unified final score after fusion.

[0110] 3. Each clustering model is equivalent to a piecewise function, and there are jumps between the scores. Calibration can ensure the continuity of each score.

[0111] Fig. 9 As shown, application scenario 2: cutoff for downgrade standby strategy

[0112] When using external data for modeling, a degradation strategy is usually adopted in consideration of the potential instability in external data collection. That is, after removing the external data, a model is built as a standby model for the active model. If the external data stops providing services one day, you can switch to the standby model. At the same time, in order to make downstream business calls imperceptible, the scores of the active and standby models are calibrated to a scale. This ensures that policy personnel only need to formulate a set of cutoff plans without adjustment, and only need to make necessary policy switching logs and before and after fluctuation monitoring.

[0113] Solution 1: Gaussian copula calibration solution for application scenarios 1 and 2

[0114] Assume that there is currently one LR model as the main model and another LR model as the backup model to simulate application scenarios 1 and 2. At this time, the ranking of the main model and the backup model is the same, but the absolute values ​​are different, corresponding to different risk levels, such as Figure 10-12 The main model and the backup model need to be calibrated to the same scale.

[0115] Gaussian copula transformation plus sigmoid function can transform the Gaussian copula transformation result into a probability value between 0 and 1. The implementation process is as follows:

[0116] Step 1. Use sample feature X and target variable y to train a classifier model1 [main] and model2 [backup]. (Classifier type is not limited)

[0117] step2. Use model1 [main] and model2 [backup] to predict the sample and get the prediction result out

[0118]

main

backup

[0119] Step 3. Use the prediction results out [main] and out [standby] as the input of the Gaussian copula model, and train one converter, converter [main] and converter [standby] respectively.

[0120] Step 4. The outputs of the converter [main] and the converter [backup] are transformed with the same sigmoid function, and the final output probability value is the final calibration probability.

[0121] After executing the above process, the calibrated fractional main model_Gaussiancopula and backup model_Gaussiancopula are obtained, as shown in Figure 13-15 As shown in the figure. From the probability distribution and strategy distribution, we can see that the two scores are completely the same, so they have similar risk levels. At the same time, since the calibration function is monotonic, the sorting and discrimination will not be affected before and after the calibration.

[0122] Execute step 3 and step 4 on the active model and the backup model respectively to obtain the calibrated fractional active model_Gaussiancopula and backup model_Gaussiancopula, as shown in Figure 13-15 As shown in the figure. From the probability distribution and strategy distribution, we can see that the two scores are completely the same, so they have similar risk levels. At the same time, since the calibration function is monotonic, the sorting and discrimination will not be affected before and after the calibration.

[0123] Solution 2: For application scenarios 1 and 2, a linear transformation calibration solution whose risk levels are similar but not completely equal.

[0124] The probability values ​​predicted by the primary and backup models are linearly transformed to integer scores with completely equal risk levels. The implementation process is as follows:

[0125] Step 1. Use sample feature X and target variable y to train a classifier model1 [main] and model2 [backup]. (Classifier type is not limited)

[0126] step2. Use model1 [main] and model2 [backup] to predict the sample and get the prediction result out

[0127]

main

backup

[0128] Step 3. Make a strategic distribution of the prediction results out [primary] and out [backup], see Table 9 and Table 10.

[0129] Step 4. When the percentage of positive samples in the cumulative captured samples is 15%, the cutoff of the main model is 0.072, and the cutoff of the backup model is 0.11. The linear transformation coefficient of the main model is 100, and the linear transformation coefficient of the backup model is (0.072*100) / 0.11=65.4545.

[0130] step5. Subtract the prediction result of out [main] from 100 and the prediction result of out [backup] from 65.4545.

[0131] After the above implementation process, the calibrated fractional main model _linear and backup model are obtained.

[0132] _linear, such as Figure 16-17 As shown. Figure 18-19 From the strategy distribution, we can see that the two scores have exactly the same risk level. At the same time, since the linear transformation calibration is monotonic, the sorting and discrimination will not be affected before and after the calibration.

[0133] The present invention can achieve the same capture rate of two models under the same cutoff. In the score calibration process, no other restrictions such as the y label of the sample are required, which is easy to implement. It can calibrate the irregular distribution to a normal distribution under the same scale. It can achieve the same proportion of positive samples captured by two models under the same cutoff.

[0134] This calibration solution can achieve the same capture rate of the two models under the same cutoff, and more perfectly solve the business problems of scenarios 1 and 2 in the background technology. In the score calibration process, no other restrictions such as the y label of the sample are required, which is easy to implement. It can calibrate the irregular distribution to a normal distribution at the same scale, which is easy to apply. It can achieve the same proportion of positive samples captured by the two models under the same cutoff, and there is no business deviation.

[0135] The following will be combined with the attached Fig. 20 , a score calibration method for a scorecard prediction model provided in an embodiment of the present application comprises the following steps:

[0136] Training a classifier using the multi-dimensional data of the first user and the behavior prediction result;

[0137] Using the classifier to predict the second user's multidimensional data to obtain a prediction result;

[0138] According to the prediction result, a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained.

[0139] Furthermore, the method of training a classifier using the user multi-dimensional data and the behavior prediction results specifically includes:

[0140] training a first classifier using the first user multidimensional data and a first behavior prediction result, wherein the first behavior prediction result is obtained by inputting the first user multidimensional data into the primary scorecard prediction model;

[0141] The second classifier is trained using the first user multidimensional data and the second behavior prediction result, where the second behavior prediction result is obtained by inputting the first user multidimensional data into the backup scorecard prediction model.

[0142] Furthermore, the using the classifier to predict the user multi-dimensional data to obtain a prediction result specifically includes:

[0143] Using the first classifier to predict the second user multi-dimensional data to obtain a first prediction result;

[0144] The second classifier is used to predict the second user multi-dimensional data to obtain a second prediction result.

[0145] Further, obtaining the calibration probability value between the primary scorecard prediction model and the backup scorecard prediction model according to the prediction result, the converter and the transformation function specifically includes:

[0146] Training a first converter using the first prediction result;

[0147] Training a second converter using the second prediction result;

[0148] Inputting the third user multi-dimensional data into the first converter to obtain a first result;

[0149] Inputting the third user multi-dimensional data into the second converter to obtain a second result;

[0150] A calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained according to the first result, the second result and the transformation function.

[0151] Further, obtaining the calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the first result, the second result and the transformation function specifically includes:

[0152] The first result and the second result are input into the transformation function to obtain a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model.

[0153] Further, obtaining a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result specifically includes:

[0154] Performing strategic distribution on the first prediction result and the second prediction result;

[0155] Determining the linear transformation coefficient of the primary scorecard prediction model and the linear transformation coefficient of the backup model according to the result of the strategy distribution;

[0156] A calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained according to the linear transformation coefficient of the primary scoring card prediction model and the linear transformation coefficient of the backup model.

[0157] Further, the determining of the linear transformation coefficient of the primary scorecard prediction model and the linear transformation coefficient of the backup model according to the result of the strategy distribution specifically includes:

[0158] When the values ​​of the same policy item in the results of the policy distribution are the same, obtaining the decision threshold of the primary scorecard prediction model and the decision threshold of the backup model;

[0159] According to the decision threshold of the primary scoring card prediction model and the decision threshold of the backup model, a linear transformation coefficient of the primary scoring card prediction model and a linear transformation coefficient of the backup model are obtained.

[0160] The present embodiment provides a score calibration method for a scoring card prediction model, including training a classifier using multi-dimensional data and behavior prediction results of a first user; using the classifier to predict multi-dimensional data of a second user to obtain a prediction result; and obtaining a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model based on the prediction result. The present invention implements a primary model and a backup model as a scoring card prediction model. If external data stops providing services, the primary model can be switched to the backup model. The scores of the primary model and the backup model are calibrated to one scale, and downstream business calls are unaware.

[0161] The following will be combined with the attached Fig.21 , a score calibration device for a scorecard prediction model provided in an embodiment of the present application, the device comprising:

[0162] A first processing module, used for training a classifier using the multi-dimensional data of the first user and the behavior prediction result;

[0163] A second processing module, used to predict the second user's multi-dimensional data using the classifier to obtain a prediction result;

[0164] The third processing module is used to obtain a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result.

[0165] In addition, an embodiment of the present application includes a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements a score calibration method for a scoring card prediction model described in any one of the above technical solutions.

[0166] An embodiment of the present application also includes a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the score calibration method for a scoring card prediction model described in any one of the above technical solutions is implemented.

[0167] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the specific implementation modes of the present invention can still be modified or replaced by equivalents, and any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A score calibration method for a scorecard prediction model, characterized in that: The method comprises: Training a classifier using the multi-dimensional data and behavior results of the first user; Using the classifier to predict the second user's multidimensional data to obtain a prediction result; According to the prediction result, a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained.

2. The method according to claim 1, characterized in that The method of training a classifier using user multi-dimensional data and behavior prediction results specifically includes: training a first classifier using the first user multidimensional data and a first behavior prediction result, wherein the first behavior prediction result is obtained by inputting the first user multidimensional data into the primary scorecard prediction model; The second classifier is trained using the first user multidimensional data and the second behavior prediction result, where the second behavior prediction result is obtained by inputting the first user multidimensional data into the backup scorecard prediction model.

3. The method according to claim 2, characterized in that The using the classifier to predict the user multi-dimensional data to obtain a prediction result specifically includes: Using the first classifier to predict the second user multi-dimensional data to obtain a first prediction result; The second classifier is used to predict the second user multi-dimensional data to obtain a second prediction result.

4. The method according to claim 3, characterized in that The obtaining, according to the prediction result, the converter and the transformation function, a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model specifically includes: Training a first converter using the first prediction result; Training a second converter using the second prediction result; Inputting the third user multi-dimensional data into the first converter to obtain a first result; Inputting the third user multi-dimensional data into the second converter to obtain a second result; A calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained according to the first result, the second result and the transformation function.

5. The method according to claim 4, characterized in that The obtaining, according to the first result, the second result and the transformation function, a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model specifically includes: The first result and the second result are input into the transformation function to obtain a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model.

6. The method according to claim 3, characterized in that The obtaining, according to the prediction result, a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model specifically includes: Performing strategic distribution on the first prediction result and the second prediction result; Determining the linear transformation coefficient of the primary scorecard prediction model and the linear transformation coefficient of the backup model according to the result of the strategy distribution; A calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model is obtained according to the linear transformation coefficient of the primary scoring card prediction model and the linear transformation coefficient of the backup model.

7. The method according to claim 6, characterized in that The step of determining the linear transformation coefficient of the primary scorecard prediction model and the linear transformation coefficient of the backup model according to the result of the strategy distribution specifically includes: When the values ​​of the same policy item in the results of the policy distribution are the same, obtaining the decision threshold of the primary scorecard prediction model and the decision threshold of the backup model; According to the decision threshold of the primary scoring card prediction model and the decision threshold of the backup model, a linear transformation coefficient of the primary scoring card prediction model and a linear transformation coefficient of the backup model are obtained.

8. A score calibration device for a scorecard prediction model, characterized in that: The device comprises: A first processing module, used for training a classifier using the multi-dimensional data of the first user and the behavior prediction result; A second processing module, used to predict the second user's multi-dimensional data using the classifier to obtain a prediction result; The third processing module is used to obtain a calibration probability value between the primary scoring card prediction model and the backup scoring card prediction model according to the prediction result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the score calibration method for a scorecard prediction model according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the score calibration method for a scorecard prediction model according to any one of claims 1 to 7 is implemented.

Images