A method, device and storage medium for realizing cost optimization and model building

Abstract

The present application relates to the field of credit risk control, and in particular to a method, device and storage medium for cost optimization and model construction. Using a training set and an out-of-time sample set, a base model is built through traditional logistic regression or a tree model; using the base model and a preset lift value, samples of different score segments are selected for superposition training to obtain a model set and a first processing result; using the first processing result, the model set is mapped to a unified scale according to the overdue rate to obtain a trained base model. It can be understood that the technical solution provided by the present invention utilizes the uniqueness of the data source and improves the accuracy of the model in a targeted manner according to the needs of the business scenario, so that the prediction results are more accurate, which can help financial institutions to reduce procurement budgets while improving the risk control effect of the model, thereby achieving the effect of reducing costs and improving efficiency.

Description

Technical Field

[0001] The present application relates to the field of credit risk control, and in particular to a method, device and storage medium for implementing cost optimization and building a model. Background Art

[0002] At present, the credit customers of banks, consumer finance companies, small loan companies and Internet financial companies (hereinafter referred to as "financial institutions") are mainly divided into new customers and old customers. In each link of the entire life cycle of credit, including but not limited to pre-loan anti-fraud, pre-loan credit assessment, loan disbursement, mid-loan risk warning and post-loan collection, proprietary and external data are used to build anti-fraud models, A card models, loan disbursement risk models, B card models and C card models, etc. For new customers, financial institutions at all levels do not have enough proprietary data to support risk control. At the same time, old customers also need to supplement external data to improve non-control capabilities, so a large amount of external data needs to be introduced, and the proportion of external data is 0.1% to 0.2% of its loan scale (eg a medium-sized Internet finance company with an annual loan scale of 100 billion yuan has an external data procurement quota of 100 million to 200 million yuan).

[0003] At present, various institutions mainly purchase external data by introducing standard models and through joint modeling. These models are composed of data sources one by one, and each data source has its own price. The final model price will be the cumulative value of the data source price.

[0004] With the increase in data sources on the market, in order to achieve better results, these institutions can only choose better data sources within a limited procurement budget and give up some data sources that are effective for certain customer groups or have higher prices. In addition, the traditional risk control method is to unconditionally superimpose the data source and reflect the effect of the data source in the model, which makes the accuracy of the risk prediction model slightly deviated, making it difficult to meet the current risk assessment needs. Summary of the invention

[0005] In view of this, the object of the present invention is to provide a method, device and storage medium for cost optimization implementation and model construction to solve the technical problems existing in the prior art.

[0006] According to a first aspect of an embodiment of the present invention, a method for constructing a cost optimization model is provided, the method comprising:

[0007] A training set and an out-of-time sample set are obtained; the training set includes: credit risk control data to be processed;

[0008] Using the training set and the out-of-time sample set, a base model is built through traditional logistic regression or a tree model;

[0009] Using the base model and the preset lift value, samples of different score segments are selected for superposition training to obtain a model set and a first processing result;

[0010] Using the first processing result, the model set is mapped to a uniform scale according to the overdue rate to obtain a trained base model.

[0011] Furthermore, the method further comprises:

[0012] The base model consists of two or three data sources of comparable effectiveness.

[0013] Furthermore, before using the base model and the preset lift value to select samples of different score segments for superposition training to obtain the model set and the first processing result, the method further includes:

[0014] Divide the data in the training set into a preset number of bins with equal frequency.

[0015] Furthermore, the base model and the preset lift value are used to select samples of different score segments for superposition training to obtain a model set and a first processing result, including:

[0016] Using the base model and the preset lift value, and using samples of different score segments, the data to be lifted in the data of the preset bins divided into equal frequency groups by the customers in the training set are superimposed for training;

[0017] If the data to be lifted in the preset binned data is tail lift data, superimposing training is performed on the tail lift data using the first recognition data, so that the tail lift data after training reaches a standard threshold;

[0018] If the data to be improved in the preset binned data is head lift data, superimposing training is performed on the head lift data using the second recognition data, so that the head lift data after training reaches a standard threshold;

[0019] If the data in the preset bins need to be ranked higher, the third recognition data is used to perform superimposed training on the data that need to be ranked higher, so that the data in the preset bins after the ranking higher reaches the standard threshold.

[0020] Further, the first processing result is used to map the model set to a unified scale according to the overdue rate to obtain a trained base model, including:

[0021] The first processing result is mapped to a uniform scale using a preset sigmoid function to obtain a trained base model.

[0022] According to a second aspect of an embodiment of the present invention, a method for implementing cost optimization is provided, the method comprising:

[0023] A test set and an out-of-time sample set are obtained; the test set includes: credit risk control data to be processed;

[0024] Inputting the test set and the out-of-time sample set into the trained base model to obtain a model score;

[0025] Wherein, the trained base model is constructed by using any of the cost optimization model construction methods described above.

[0026] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising: a processor, a memory and a communication bus, wherein the processor and the memory communicate with each other via the communication bus;

[0027] The memory is used to store computer programs;

[0028] The processor is used to execute the program stored in the memory to implement any of the above-mentioned methods for constructing a cost optimization model, or to implement any of the above-mentioned methods for implementing cost optimization.

[0029] According to a fourth aspect of an embodiment of the present invention, there is provided a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the computer program implements any one of the above-mentioned methods for constructing a cost optimization model, or implements any one of the above-mentioned methods for implementing cost optimization.

[0030] The technical solution provided by the embodiments of the present invention may have the following beneficial effects:

[0031] It can be understood that the technical solution provided by the present invention, by obtaining a training set and a time-out sample set; using the training set and the time-out sample set, building a base model through traditional logistic regression or tree model; using the base model and the preset lift value, selecting samples of different score segments for superposition training, obtaining a model set and a first processing result; using the first processing result, mapping the model set to a unified scale according to the overdue rate, and obtaining a trained base model. It can be understood that the technical solution provided by the present invention, using the base model and the preset lift value, selecting samples of different score segments for targeted superposition training, using the uniqueness of the data source, according to the needs of the business scenario, targetedly improving the accuracy of the model, making the prediction result more accurate; according to the characteristics of the data domain and the deficiencies of the base model, targetedly superimposing specific data sources on some samples, so that within a lower price budget, the effects of more data sources are contributed to the model, giving full play to the maximum value of each data source, thereby greatly improving the model effect, to meet the current risk assessment needs, and can also help financial institutions, while improving the model risk control effect, reduce procurement budgets, and play a cost-reducing and efficiency-enhancing effect.

[0032] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0034] Figure 1 is a schematic diagram of steps of a method for constructing a cost optimization model according to an exemplary embodiment;

[0035] Figure 2 is a schematic diagram showing how to divide customers into preset bin data with equal frequency according to an exemplary embodiment;

[0036] Figure 3 is a schematic diagram of tail LIFT lifting according to an exemplary embodiment;

[0037] Figure 4 is a schematic diagram of head LIFT lifting according to an exemplary embodiment;

[0038] Figure 5 is a schematic diagram of ranking improvement according to an exemplary embodiment;

[0039] Figure 6 is a schematic diagram of a process of model calibration of a method for building a cost optimization model according to an exemplary embodiment;

[0040] Figure 7 is a schematic diagram of steps of a method for implementing cost optimization according to an exemplary embodiment;

[0041] Figure 8 The figure is a schematic diagram showing the composition of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0042] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0043] Embodiment 1

[0044] See also Figure 1 , Figure 1 The figure is a schematic diagram of steps of a method for constructing a cost optimization model according to an exemplary embodiment, the method comprising:

[0045] S1. Obtain a training set and an out-of-time sample set; the training set includes: credit risk control data to be processed;

[0046] S2. Using the training set and the out-of-time sample set, a base model is built by traditional logistic regression or tree model;

[0047] S3. Using the base model and the preset lift value, select samples of different score segments for superposition training to obtain a model set and a first processing result;

[0048] S4. Using the first processing result, the model set is mapped to a uniform scale according to the overdue rate to obtain a trained base model.

[0049] In specific implementation, it can be understood that the technical solution provided by the present invention obtains a training set and a time-out sample set; uses the training set and the time-out sample set to build a base model through traditional logistic regression or tree model; uses the base model and the preset lift value to select samples of different score segments for superposition training to obtain a model set and a first processing result; uses the first processing result to map the model set to a unified scale according to the overdue rate to obtain a trained base model. It can be understood that the technical solution provided by the present invention uses the base model and the preset lift value to select samples of different score segments for targeted superposition training, uses the uniqueness of the data source, and improves the accuracy of the model in a targeted manner according to the needs of the business scenario, so that the prediction result is more accurate; according to the characteristics of the data domain and the deficiencies of the base model, superposition of specific data sources is carried out on some samples in a targeted manner, so that within a lower price budget, the effects of more data sources are contributed to the model, and the maximum value of each data source is fully utilized, thereby greatly improving the model effect to meet the current risk assessment needs.

[0050] In one embodiment, the purpose of this application is to maximize the value of each data source within a limited procurement budget, thereby significantly improving the model effect.

[0051] First, let's understand the main indicators for evaluating the effectiveness of the model, including the model's basic indicators KS value and AUC value. When these indicators are OK, we will observe and refine the indicators more accurately according to the application scenarios of the model: 1. Sorting ability (customer sorting), 2. Model head LIFT (screening good customers) and 3. Tail LIFT (excluding bad customers).

[0052] These indicators have a certain correlation. For example, after superimposing a data source, both the sorting and the head-to-tail LIFT will be improved. However, the model for this link is mainly used to exclude bad customers, so the sorting is not that important.

[0053] Secondly, we need to understand the classification of data sources on the market:

[0054] Strong ability to identify bad customers (e.g., multiple data, network and payment security data, etc.);

[0055] Strong ability to identify good customers (for example: customer high-end consumption data, game recharge data, air travel information data);

[0056] Ability to distinguish between good and bad customers (for example, data from large Internet companies, which contains more information, has a higher coverage rate, and is relatively stable);

[0057] For the models of each link mentioned in the above question, use these data to superimpose them to achieve the model indicators you are concerned about.

[0058] In a specific implementation, as described in step S1, obtaining the training set and the out-of-time sample set includes:

[0059] The data set is divided into a training set (Train), a test set (Test), and an out-of-time sample set (OOT). Train and Test are mainly used to ensure that the model does not overfit or underfit, and OOT is mainly used to ensure that the model does not have a large attenuation in the time dimension, that is, the model is relatively stable.

[0060] As described in step S2, in the specific implementation, the base model is built by traditional logistic regression (Logistic Regression) or tree model (Xgboost, LightGBM and Catboost).

[0061] As described in step S3, in the specific implementation, the base model and the preset lift value are used to select samples of different score segments for superposition training to obtain a model set and a first processing result, including:

[0062] S.3.1 Using the base model and the preset lift value, and using samples of different score segments, superimpose training on the data to be lifted in the data of the preset bins divided by equal frequency of the customers in the training set;

[0063] S.3.2 If the data to be lifted in the preset binned data is tail lift data, superimpose training is performed on the tail lift data using the first recognition data so that the tail lift data after training reaches the standard threshold;

[0064] S.3.3 If the data to be lifted in the preset binned data is head lift data, superimposing training is performed on the head lift data using the second recognition data so that the head lift data after training reaches a standard threshold;

[0065] S.3.4 If the data in the preset bins need to be sorted and improved, the third recognition data is used to perform superimposed training on the data that need to be sorted and improved, so that the data in the preset bins after sorting and improvement reaches the standard threshold.

[0066] In the specific implementation, after the base model is built, the output is a score of 300 to 850, where the score is related to the overdue rate, that is, as the score increases, the customer's overdue rate will decrease (the overdue rate refers to a certain score range, the number of bad customers in this score range / the total number of customers in this score range).

[0067] In this way, we get a basic model, which is also an essential expenditure in the external data procurement budget.

[0068] After this model is built, we will divide the customers into 10 or 20 boxes with equal frequency and calculate the overdue rate of each box. For example, Figure 2 .

[0069] As described in step S.3.2, for specific implementation, refer to Figure 3 It can be seen that in the tail score range of 350 to 415, the LIFT of the model is only 1.84. It is generally believed that for a 10-point bin, the tail LIFT is expected to be above 2.5, which is considered good.

[0070] At this time, we will select a data source with a stronger ability to distinguish bad customers based on our business experience. Based on the data from large manufacturers, we will further distinguish the gray areas that cannot be distinguished by the data from large manufacturers. Specifically, we will select the score range of 350 to 554, that is, the first identification data, and use the data with a stronger ability to distinguish bad customers for superposition, and no operation will be performed on 554 to 850.

[0071] As described in step S.3.3, in the specific implementation, Figure 4 It can be seen that in the tail score range of 791 to 850, the LIFT of the model is only 0.58. It is generally believed that for a 10-point bin, it is considered relatively good if the tail LIFT can be within 0.5.

[0072] At this time, we will select a data source with a stronger ability to distinguish good customers based on our business experience. On the basis of the data from large manufacturers, we will further distinguish the gray areas that cannot be distinguished by the data from large manufacturers. Specifically, we will select people with scores between 693 and 850, that is, the second identification data, and superimpose it with data with a stronger ability to distinguish bad customers. The score range between 300 and 693 will not be processed.

[0073] As described in step S3.3, in the specific implementation, Figure 5 It can be seen that there are many people in the score range of 629 to 693, but the sorting is not very good, and even inverted, so we need to further sort the people in this section, so we choose data with better sorting for sorting, for example: consumption data in payment data can distinguish a person's consumption ability well according to consumption amount, frequency and shopping preferences.

[0074] In the specific implementation, according to the characteristics of the data domain and the deficiencies of the base model, specific data sources are superimposed on some samples in a targeted manner, so as to contribute the effects of more data sources to the model within a lower price budget.

[0075] In specific implementation, as in step S4, using the first processing result, the model set is mapped to a unified scale according to the overdue rate to obtain a trained base model, including:

[0076] See also Figure 6 Through the above operations, we will get many sub-models, because the base model is equivalent to the one trained with the full amount of samples. After the base model, we select samples with different score ranges for superposition training, so there will be many models. The output value of each model is a score of 300 to 850. We need to map these models to the final unified scale according to the overdue rate to ensure the consistency of the results of each model. For example, 400 points from model 1 and 400 points from model 2 correspond to the same overdue rate.

[0077] Assume that there is a model 1 and a model 2 with scores score1 and score2. At this time, the ranking of score1 and score2 is exactly the same, but the absolute value is different. We need to calibrate score1 and score2 to the same scale.

[0078] Platt first proposed that the prediction result of SVM can be converted into a posterior probability value through the sigmoid function. The implementation process is as follows:

[0079] Step 1. Use sample features score1 and score2 as X and target variable y to train a classifier model1. (Classifier type is not limited, egLR)

[0080] step 2. Use model1 to predict the sample and get the prediction result out.

[0081] Step 3. Use the predicted result out as the new feature X', and then use the sample label y to train an LR.

[0082] Step 4. The probability value finally output by LR is the predicted probability after Platt's scaling.

[0083] We execute step 3 for score1 and score2 respectively, and get the calibrated scores score1_cal and score2_cal, so they have the same risk level. Similarly, we can get the calibration values ​​of all sub-models. Since the calibration function is monotonic, the sorting and discrimination will not be affected before and after calibration.

[0084] Finally, we obtained a model score, the effect of which was composed of 11 data sources. The effect was far better than the traditional model built with 5 data sources, but the price was lower than the traditional model.

[0085] See also Figure 7 , Figure 7 1 is a schematic diagram of steps of a method for implementing cost optimization according to an exemplary embodiment, the method comprising:

[0086] A test set and an out-of-time sample set are obtained; the test set includes: credit risk control data to be processed;

[0087] Inputting the test set and the out-of-time sample set into the trained base model to obtain a model score;

[0088] Wherein, the trained base model is constructed by using any of the cost optimization model construction methods described above.

[0089] During the specific implementation, the model scores obtained can help financial institutions to reduce procurement budgets while improving the risk control effect of the model, thereby achieving the effect of reducing costs and improving efficiency.

[0090] See also Figure 8 , Figure 8 FIG. 1 is a schematic diagram showing a composition of an electronic device according to an exemplary embodiment, wherein the electronic device includes:

[0091] It includes: a processor 82, a memory 81 and a communication bus, wherein the processor 82 and the memory 81 communicate with each other via the communication bus;

[0092] The memory 81 is used to store computer programs;

[0093] The processor 82 is used to execute the program stored in the memory to implement any of the above-mentioned methods for constructing a cost optimization model, or to implement any of the above-mentioned methods for implementing cost optimization.

[0094] In addition, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the steps of any of the above methods. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD); the storage medium may also include a combination of the above-mentioned types of memory.

[0095] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0096] It should be noted that, in the description of the present invention, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0097] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0098] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0099] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0100] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0101] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0102] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0103] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for constructing a cost optimization model, characterized in that: The method comprises: A training set and an out-of-time sample set are obtained; the training set includes: credit risk control data to be processed; Using the training set and the out-of-time sample set, a base model is built through traditional logistic regression or tree model; Using the base model and the preset lift value, samples of different score segments are selected for superposition training to obtain a model set and a first processing result; Using the first processing result, mapping the model set to a unified scale according to the overdue rate to obtain a trained base model; Before the method uses the base model and the preset lift value to select samples of different score segments for superposition training to obtain the model set and the first processing result, it also includes: Divide the data in the training set into a preset number of bins with equal frequency; The method uses the base model and the preset lift value to select samples of different score segments for superposition training to obtain a model set and a first processing result, including: Using the base model and the preset lift value, and using samples of different score segments, the data to be lifted in the data of the training set divided into preset bins with equal frequency by customers is superimposed for training; If the data to be lifted in the preset binned data is tail lift data, superimposing training is performed on the tail lift data using the first recognition data, so that the tail lift data after training reaches a standard threshold; If the data to be improved in the preset binned data is head lift data, superimposing training is performed on the head lift data using the second recognition data, so that the head lift data after training reaches a standard threshold; If the data in the preset bins need to be ranked higher, the third recognition data is used to perform superimposed training on the data that need to be ranked higher, so that the data in the preset bins after the ranking higher reaches the standard threshold; According to the characteristics of the data domain and the deficiencies of the base model, specific data sources are superimposed on some samples in a targeted manner, so as to contribute the effects of more data sources to the model within a lower price budget.

2. The method according to claim 1, characterized in that The method further comprises: The base model consists of two or three data sources of comparable effectiveness.

3. The method according to claim 1, characterized in that The method of using the first processing result to map the model set to a uniform scale according to the overdue rate to obtain a trained base model includes: The first processing result is mapped to a uniform scale using a preset sigmoid function to obtain a trained base model.

4. A method for implementing cost optimization, characterized in that: The method comprises: A test set and an out-of-time sample set are obtained; the test set includes: credit risk control data to be processed; Inputting the test set and the out-of-time sample set into the trained base model to obtain a model score; Wherein, the trained base model is constructed by using a cost optimization model construction method as described in any one of claims 1 to 3.

5. An electronic device, characterized in that: include: A processor, a memory and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory is used to store computer programs; The processor is used to execute the program stored in the memory to implement a method for constructing a cost optimization model as described in any one of claims 1 to 3, or to implement a method for implementing cost optimization as described in claim 4.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements a method for constructing a cost optimization model as described in any one of claims 1 to 3, or implements a method for implementing cost optimization as described in claim 4.

Images