Risk ranking scoring index optimization method and system based on multi-statistic fusion

Abstract

The invention provides a risk ranking scoring index optimization method and system based on multi-statistic fusion, and relates to the field of credit risk control, and the method comprises the steps: model index calculation, index standardization processing, index weight distribution, determining the weight coefficient of each index according to expert experience in the initial stage, and determining the weight coefficient of each index according to the expert experience; gradually correcting a weight coefficient by adopting results of a large number of historical models through a manual judgment mode; sequencing coefficient calculation: performing weighted summation calculation to obtain a sequencing coefficient; sequencing score calculation: performing specific linear regression conversion on the sequencing coefficient to obtain a sequencing score; formulating sorting score thresholds, and setting a series of sorting score thresholds according to historical data and business requirements to judge whether the indexes are qualified or not; and code packaging is carried out to form a standardized tool which can be universally used. The objective of the invention is to solve the problem that the model index sorting performance cannot be quantitatively evaluated in the risk control field, and a new decision support tool is provided for risk management.

Description

Technical Field

[0001] The present invention relates to the field of credit risk control technology, and in particular to a risk ranking scoring index optimization method and system based on multi-statistics fusion. Background Art

[0002] In the field of risk control, sortability refers to the ability of a model to effectively sort observations from high risk to low risk or from low risk to high risk. A model with good sortability can clearly distinguish risk levels and maintain consistency in sorting.

[0003] AUC and KS represent the difference in the cumulative distribution of good and bad samples, and are fixed values ​​after sample binning. However, the lift curve represents the changing trend of the difference in sample distribution. Its monotonicity and adjacent discrimination are still artificial "experience values" in the industry. It is not a specific value that can be quantified, but a reflection of the professional experience of relevant personnel. This may lead to different people having different "experience values" for the same model.

[0004] In the risk control scenario, the ranking ability of the model is directly related to the accurate identification and management of risky customers. A risk control model with good ranking can more accurately distinguish high-risk and low-risk customers, thereby improving the effectiveness and accuracy of credit decisions. Traditional model evaluation methods, such as AUC and KS, can measure the model's distinguishing ability to a certain extent, but cannot comprehensively evaluate the model's performance and ranking consistency under different bins. For example, risk control business and model developers often mention the strong or weak ranking of a certain feature or model, but this is more based on the experience of business or model personnel. There is no quantifiable indicator to measure the comparison of ranking between multiple features or models. For example, the ranking of a certain model is stronger than that of a certain model. This kind of judgment is also based on experience.

[0005] Therefore, it is extremely important to design a statistic to describe the ranking ability of a feature or model, conduct a quantitative evaluation of the feature or model, and provide decision support for risk management. Summary of the invention

[0006] In view of the technical problems existing in the prior art, the present invention provides a risk ranking scoring indicator optimization method and system based on multi-statistical fusion. The purpose of the present invention is to solve the limitations of the ranking capabilities of traditional measurement models and explore more effective, more intuitive, more multi-dimensional and more quantitative optimization methods suitable for the calculation of risk ranking scoring indicators.

[0007] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0008] According to a first aspect of the present invention, a risk ranking scoring index optimization method based on multi-statistics fusion is provided, comprising the following steps:

[0009] Step 1. Obtain the model score and overdue label of the risk control scenario, divide the model score into boxes, and calculate the lift value of the bad sample ratio in each box, that is, the bad sample rate of the box / the overall bad sample rate;

[0010] Step 2. Construct the same number of arithmetic progressions according to the number of model score bins, and calculate the Spearman correlation coefficient corr between the lift list after the risk control model score bins and the arithmetic progression list;

[0011] Step 3. Calculate the area under the ROC curve (AUC) of the model score and the maximum interval distance (KS) between the true positive rate and the false positive rate curves;

[0012] Step 4. According to the lift value of the bad sample ratio of the model score bin, calculate the adjacent discrimination index of the model score, recorded as diff;

[0013] Step 5. Normalize the above corr, AUC, KS, and diff indicators to eliminate the dimension effect;

[0014] Step 6. Use expert experience to determine the weight coefficient of each indicator, and then use historical experience to gradually revise the indicator weight coefficient;

[0015] Step 7. Perform weighted sum calculation on each obtained indicator and the determined indicator weight coefficient to obtain the ranking coefficient;

[0016] Step 8. The ranking coefficient is converted into a final ranking score through linear regression. If the ranking score is greater than the preset value, it means that the ranking is good.

[0017] Based on the above technical solution, the present invention can also make the following improvements.

[0018] Preferably, in step 1, the scores are divided into boxes and the lift value of each box is calculated, and the calculation formula is as follows:

[0019]

[0020] Among them, bin_badrate i is the bad sample ratio of the i-th box, total_badrate is the bad sample ratio of the whole sample, and the calculated result is lift i is the lift value of the i-th box.

[0021] Preferably, the AUC value is calculated in step 3, and the formula is as follows:

[0022]

[0023] Among them, |D + |、|D - | is the total number of good and bad users, is the number of user pairs that the model corrects, The logarithm of the order of good and bad user teams is the same.

[0024] Preferably, the KS value is calculated in step 3, and the formula is as follows:

[0025] KS=max(TPR-FPR|)

[0026] Among them, TPR represents the true positive rate, that is, the proportion of all bad users predicted to be bad users, and TPR represents the false positive rate, that is, the proportion of all good users predicted to be good users.

[0027] Preferably, in step 4, the adjacent discrimination index diff is calculated using the following formula:

[0028]

[0029] Where n is the number of bins, lift i is the lift value of the i-th box.

[0030] Preferably, the normalization processing of the Spearman correlation coefficient corr, AUC, and KS in step 5 is as follows:

[0031]

[0032] Among them, values ​​are the original values, values min is the theoretical minimum value of this value, values max is the theoretical maximum value of this value, where the value range of corr is [0,1], the value range of AUC is [0.5,1], and the value range of KS is [0,1]; the adjacent discrimination index diff of the model score is normalized by sigmoid, and the formula is as follows:

[0033]

[0034] Among them, -z is the specific original indicator value diff.

[0035] Preferably, in step 6, a weight coefficient is assigned to each indicator, and the sum of all weight coefficients is 1, that is, Σw i =1, where the weight of each indicator is w i , determined by expert experience, data-driven, and optimization methods, and the ranking coefficient is calculated by weighted summation. The formula is as follows:

[0036] coeff ranking =∑index i *w i

[0037] Among them, coeff ranking Represents the ranking coefficient, index i Represent the normalized corr, AUC, KS, diff, w i Represent the weight coefficients corresponding to corr, AUC, KS, and diff respectively.

[0038] Preferably, in step 8, the sorting coefficient coeff ranking Convert to get ranking score ranking , the formula is as follows:

[0039]

[0040] Among them, A and B are two constant parameters.

[0041] According to a second aspect of the present invention, a risk ranking scoring index optimization system based on multi-statistics fusion is provided, comprising:

[0042] The parameter acquisition module is used to obtain the Spearman correlation coefficient corr, AUC, KS, and adjacent discrimination index diff between the lift list and the arithmetic sequence list after sample binning;

[0043] The preprocessing module is used to normalize all indicators and eliminate the impact of dimensions;

[0044] The first data processing module is used to determine the weight coefficient of each indicator by expert experience in the initial stage, and then gradually correct the weight coefficient by manual judgment using the results of a large number of historical models;

[0045] The second data processing module calculates the ranking coefficient by weighted summing the obtained indicators and the determined indicator weight coefficients;

[0046] The third data processing module converts the ranking coefficient into a ranking score through linear regression to ensure the rationality and interpretability of the score;

[0047] The ranking scoring module is used to convert the ranking coefficient into a final ranking score through linear regression. If the ranking score is greater than a preset value, it means that the ranking is good.

[0048] According to the third aspect of the present invention, the present invention provides an electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that when the processor executes the program, the method for optimizing a risk ranking scoring indicator based on multi-statistical fusion is implemented.

[0049] Technical effects and advantages of the present invention:

[0050] The present invention provides a risk ranking scoring index optimization method and system based on multi-statistic fusion, quantifies lift monotonicity through Spearman correlation coefficient, and avoids the empirical orientation of lift monotonicity in traditional scenarios. The lift adjacent discrimination is quantified by accumulating and averaging the lift differences between adjacent intervals, and avoids the empirical orientation of lift adjacent discrimination in traditional scenarios.

[0051] The present invention uses the experience of practitioners and historical data correction to perform weighted summation of the Spearman correlation coefficient corr, AUC, KS, and lift adjacent discrimination index diff between the lift list and the arithmetic progression list after sample binning, and generates a new index coeff to measure the ranking ability of the model. ranking , using one indicator instead of four indicators can reflect the ranking ability of the model in a more multidimensional, comprehensive and intuitive way.

[0052] The present invention designs a statistic to describe the ranking ability of a feature or model, conducts quantitative evaluation of the feature or model, improves the qualitative efficiency of practitioners on the ranking ability, and provides powerful decision support for risk managers. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 It is a flowchart of the steps of a risk ranking scoring indicator optimization method based on multi-statistics fusion provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] It should be noted that when the industry studies ranking, it will comprehensively consider multiple risk control indicators, such as AUC, KS, and lift. Among them, AUC represents the area under the ROC curve. The larger the AUC, the stronger the model's overall ability to distinguish risks; KS represents the maximum interval between the TPR and FPR curves. The larger the KS, the stronger the model's overall ability to distinguish risks; lift indicates that the model's ability to predict the "response" in the target is better than a multiple of random selection. The curve connected by the lift value points of each box is called a lift curve. The more monotonic the lift is and the greater the lift value gap between adjacent boxes, the stronger the model's ability to distinguish risks. The monotonicity of lift refers to the trend of the lift value changing with the bin value under different bins.

[0056] In an ideal model, as the binning value decreases, that is, the prediction of the positive class becomes more stringent, the lift value should be monotonically decreasing. In the evaluation process of the above indicators, AUC and KS represent the difference in the cumulative distribution of good and bad samples, which is a fixed value after the sample is binned, but the lift curve represents the trend of the difference in sample distribution. Its monotonicity and adjacent discrimination are still artificial "experience values" in the industry. It is not a specific value that can be quantified, but a reflection of the experience of relevant personnel. This will lead to different people having different "experience values" for the same model. By designing a statistic to describe the size of the ranking ability of a feature or model, the feature or model can be quantitatively evaluated to improve the qualitative efficiency of practitioners on the size of the ranking, and provide strong decision support for risk managers.

[0057] It is understandable that, based on the defects in the background technology, the embodiment of the present invention proposes a risk ranking scoring index optimization method based on multi-statistics fusion, specifically as follows: Figure 1 As shown, the method comprises the following steps:

[0058] Step 1. Obtain the model score and overdue label of the risk control scenario, divide the model score into boxes, and calculate the lift value of the bad sample ratio in each box, that is, the bad sample rate of the box / the overall bad sample rate;

[0059] After dividing the scores into boxes, the lift value of each box is calculated. The calculation formula is as follows:

[0060]

[0061] Among them, bin_badrate i is the bad sample ratio of the i-th box, total_badrate is the bad sample ratio of the whole sample, and the calculated result is lift i is the lift value of the i-th box.

[0062] Step 2. Construct the same number of arithmetic progressions according to the number of model score bins, and calculate the Spearman correlation coefficient corr between the lift list after the risk control model score bins and the arithmetic progression list;

[0063] Calculate the Spearman correlation coefficient corr between the lift list and the arithmetic sequence list after the risk control model score is binned. The formula is as follows:

[0064]

[0065] Among them, d i is the difference in rank of the ith pair, and n is the number of observation pairs.

[0066] Step 3. Calculate the area under the ROC curve (AUC) of the model score and the maximum interval distance (KS) between the true positive rate and the false positive rate curves;

[0067] The AUC value is calculated using the following formula:

[0068]

[0069] Among them, |D + |、|D - | is the total number of good and bad users, is the number of user pairs that the model corrects, The order of good and bad users is the same as the logarithm;

[0070] Calculate the KS value using the following formula:

[0071] KS=max(|TPR-FPR|)

[0072] Among them, TPR represents the true positive rate, that is, the proportion of all bad users predicted to be bad users, and TPR represents the false positive rate, that is, the proportion of all good users predicted to be good users.

[0073] Step 4. According to the lift value of the bad sample ratio of the model score bin, calculate the adjacent discrimination index of the model score, recorded as diff;

[0074] Calculate the adjacent discrimination index diff, the formula is as follows:

[0075]

[0076] Where n is the number of bins, lift i is the lift value of the i-th box.

[0077] Step 5. Normalize the above corr, AUC, KS, and diff indicators to eliminate the dimension effect;

[0078] The normalization formula for Spearman's correlation coefficient corr, AUC, and KS is as follows:

[0079]

[0080] Among them, values ​​are the original values, values min is the theoretical minimum value of this value, values max is the theoretical maximum value of this value, where the value range of corr is [0,1], the value range of AUC is [0.5,1], and the value range of KS is [0,1];

[0081] In addition, the indicator diff is normalized using sigmoid, and the formula is as follows:

[0082]

[0083] Among them, -z is the specific original indicator value diff.

[0084] Step 6. Use expert experience to determine the weight coefficient of each indicator, and then use historical experience to gradually revise the indicator weight coefficient;

[0085] In this implementation step, a weight coefficient is assigned to each indicator, and the sum of all weight coefficients is 1, that is, ∑w i =1, where the weight of each indicator is w i , which can be determined by expert experience, data-driven, and computational methods.

[0086] Step 7. Perform weighted sum calculation on each obtained indicator and the determined indicator weight coefficient to obtain the ranking coefficient;

[0087] The ranking coefficient is calculated by weighted summation, and the formula is as follows:

[0088] coeff ranking =∑index i *w i

[0089] Among them, coeff ranking Represents the ranking coefficient, index i Represent the normalized corr, AUC, KS, diff, w i Represent the weight coefficients corresponding to corr, AUC, KS, and diff respectively.

[0090] Step 8. The ranking coefficient is converted into a final ranking score through linear regression. If the ranking score is greater than the preset value, it means that the ranking is good.

[0091] Through linear regression, the ranking coefficient coeff ranking Convert to get ranking score ranking , the formula is as follows:

[0092]

[0093] Among them, A and B are two constant parameters.

[0094] It should be noted that the ranking score threshold can be formulated based on historical data and business needs to set a series of ranking score thresholds to determine whether the indicators are qualified; code encapsulation is performed to form a standardized tool that can be used in a generalized manner. For example, in the field of credit risk control, a ranking score of the risk control model greater than 500 indicates good ranking. Therefore, in this embodiment, the preset value can be selected as 500 points.

[0095] In summary, the risk ranking scoring index optimization method based on multi-statistic fusion provided by the embodiment of the present invention, by designing a statistic to describe the ranking ability of a feature or model, quantitatively evaluates the feature or model, and integrates each independent professional index into a quantifiable unified index, thus solving the limitations of the traditional method of measuring the ranking ability of the model. It improves the qualitative efficiency of practitioners in the ranking size and provides strong decision support for risk managers.

[0096] The embodiment of the present invention also provides a risk ranking scoring index optimization system based on multi-statistics fusion, including:

[0097] The parameter acquisition module is used to obtain the Spearman correlation coefficient corr, AUC, KS, and adjacent discrimination index diff between the lift list and the arithmetic sequence list after sample binning;

[0098] The preprocessing module is used to normalize all indicators and eliminate the impact of dimensions;

[0099] The first data processing module is used to determine the weight coefficient of each indicator by expert experience in the initial stage, and then gradually revise the weight coefficient by manual judgment using the results of a large number of historical models to ensure the scientific nature of the weight;

[0100] The second data processing module calculates the ranking coefficient by weighted summing the obtained indicators and the determined indicator weight coefficients;

[0101] The third data processing module converts the ranking coefficient into a ranking score through linear regression;

[0102] The ranking scoring module is used to set a series of ranking scoring thresholds based on historical data and business needs to determine whether the indicators are qualified; encapsulate the code to form a standardized tool that can be generalized, so that the ranking scoring calculation process can be reused.

[0103] It can be understood that the risk ranking scoring index optimization system based on multi-statistical fusion provided by the present invention corresponds to the risk ranking scoring index optimization method based on multi-statistical fusion provided by the aforementioned embodiments, and the relevant technical features of the risk ranking scoring index optimization system based on multi-statistical fusion can refer to the relevant technical features of the risk ranking scoring index optimization method based on multi-statistical fusion, which will not be repeated here.

[0104] In addition, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the program, the above-mentioned risk ranking scoring indicator optimization method based on multi-statistical fusion is implemented.

[0105] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0106] Finally, it should be noted that the purpose of the present invention is to solve the pain point that the ranking of model indicators cannot be quantitatively evaluated in the field of risk control. By quantitatively evaluating the ranking ability of risk indicators, a new decision support tool is provided for risk management.

[0107] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0108] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0109] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A risk ranking scoring index optimization method based on multi-statistics fusion, characterized in that: The following steps are involved: Step 1. Obtain the model score and overdue label of the risk control scenario, divide the model score into boxes, and calculate the lift value of the bad sample ratio in each box, that is, the bad sample rate of the box / the overall bad sample rate; Step 2. Construct the same number of arithmetic progressions according to the number of model score bins, and calculate the Spearman correlation coefficient corr between the lift list after the risk control model score bins and the arithmetic progression list; Step 3. Calculate the area under the ROC curve (AUC) of the model score and the maximum interval distance (KS) between the true positive rate and the false positive rate curves; Step 4. According to the lift value of the bad sample ratio of the model score bin, calculate the adjacent discrimination index of the model score, recorded as diff; Step 5. Normalize the above corr, AUC, KS, and diff indicators to eliminate the dimension effect; Step 6. Use expert experience to determine the weight coefficient of each indicator, and then use historical experience to gradually revise the indicator weight coefficient; Step 7. Perform weighted sum calculation on each obtained indicator and the determined indicator weight coefficient to obtain the ranking coefficient; Step 8. The ranking coefficient is converted into a final ranking score through linear regression. If the ranking score is greater than the preset value, it means that the ranking is good.

2. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that: In step 1, the scores are divided into boxes and the lift value of each box is calculated. The calculation formula is as follows: Among them, bin_badrate i is the bad sample ratio of the i-th box, total_badrate is the bad sample ratio of the whole sample, and the calculated result is lift i is the lift value of the i-th box.

3. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that: In step 3, the AUC value is calculated using the following formula: Among them, |D + |、|D - | is the total number of good and bad users, is the number of user pairs that the model corrects, It is the same logarithm as the order of good and bad user teams.

4. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that ,In step 3, calculate the KS value, the formula is as follows: KS=max(TPR-FPR|) Among them, TPR represents the true positive rate, that is, the proportion of all bad users predicted to be bad users, and TPR represents the false positive rate, that is, the proportion of all good users predicted to be good users.

5. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that ,In step 4, the adjacent discrimination index diff is calculated, and the formula is as follows: Where n is the number of bins, lift i is the lift value of the i-th box.

6. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that: The normalization processing of Spearman correlation coefficient corr, AUC, and KS in step 5 is as follows: In the formula, values ​​are the original values, values min is the theoretical minimum value of this value, values max is the theoretical maximum value, where the value range of corr is [0,1], the value range of AUC is [0.5,1], and the value range of KS is [0,1].

7. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that: In step 6, a weight coefficient is assigned to each indicator so that the sum of all weight coefficients is 1, that is, Σw i =1, where the weight of each indicator is w i , using expert experience, data-driven, and optimization methods to determine, and calculating the ranking coefficient through weighted summation. The formula is as follows: coeff ranking =∑index i *w i Among them, coeff ranking Represents the ranking coefficient, index i Represent the normalized corr, AUC, KS, diff, w i Represent the weight coefficients corresponding to corr, AUC, KS, and diff respectively.

8. The risk ranking scoring index optimization method based on multi-statistics fusion according to claim 1 is characterized in that: In step 8, the sorting coefficient coeff ranking Convert to get ranking score ranking , the formula is as follows: Among them, A and B are two constant parameters.

9. The risk ranking scoring index optimization system based on multi-statistics fusion is characterized by: include: The parameter acquisition module is used to obtain the Spearman correlation coefficient corr, AUC, KS, and adjacent discrimination index diff between the lift list and the arithmetic sequence list after sample binning; The preprocessing module is used to normalize all indicators and eliminate the impact of dimensions; The first data processing module is used to determine the weight coefficient of each indicator by expert experience in the initial stage, and then gradually revise the weight coefficient by manual judgment using the results of a large number of historical models to ensure the scientific nature of the weight; The second data processing module calculates the ranking coefficient by weighted summing the obtained indicators and the determined indicator weight coefficients; The third data processing module converts the ranking coefficient into a ranking score through linear regression; The ranking scoring module is used to convert the ranking coefficient into a final ranking score through linear regression. If the ranking score is greater than a preset value, it means that the ranking is good.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, a risk ranking scoring index optimization method based on multi-statistics fusion as described in any one of claims 1 to 8 is implemented.

Images