Evaluation method and device of credit risk control data source, electronic equipment and computer program product

Abstract

The invention discloses an evaluation method and device of a credit and loan risk control data source, electronic equipment and a computer program product, and the method comprises the steps: obtaining credit and loan risk control data and evaluation configuration information of a user, the credit and loan risk control data comprising credit and loan risk control data of an internal data source and a plurality of external data sources; processing the credit risk control data of the plurality of external data sources according to a basic index evaluation strategy corresponding to the evaluation configuration information to obtain a basic index score of each external data source; constructing a risk prediction model based on internal and external credit risk control data, and determining a model gain score of each external data source in combination with a model gain evaluation strategy corresponding to the evaluation configuration information; and determining and outputting a final evaluation result according to the basic index score and the model gain score of each external data source. According to the method, the accuracy, the flexibility and the evaluation efficiency of credit risk control data source evaluation are improved, transverse comparison between external data sources is realized, and decision-making and use of application personnel are facilitated.

Description

Technical Field

[0001] The present application relates to the field of credit risk control technology, and in particular to a method, device, electronic equipment, and computer program product for evaluating a credit risk control data source. Background Art

[0002] In the field of credit risk management, financial institutions and third-party service providers usually need to conduct risk assessment and management on a large amount of third-party credit data, which involves consideration of multiple aspects such as the borrower's credit history, repayment ability, and financial status.

[0003] There are two main existing implementation solutions:

[0004] 1) Manual test evaluation: Using statistical analysis software such as SAS and Python as analysis tools, external data testing is performed by manually reviewing data and generating reports. This method is inefficient and prone to errors.

[0005] 2) Automated test evaluation: Solidify the test process and automatically generate test reports. This method often lacks flexibility and cannot adjust the analysis strategy according to the characteristics of different data sets. It cannot adapt well to different scenarios and specific needs. Summary of the invention

[0006] The embodiments of the present application provide a method, device, electronic device, and computer program product for evaluating a credit risk control data source to improve the efficiency, accuracy, and flexibility of evaluating an external data source in a credit risk control scenario.

[0007] The present application embodiment adopts the following technical solutions:

[0008] In a first aspect, an embodiment of the present application provides a method for evaluating a credit risk control data source, wherein the method for evaluating a credit risk control data source comprises:

[0009] Acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources;

[0010] Processing the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source;

[0011] Building a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determining a model gain score of each external data source based on the risk prediction model and using a model gain evaluation strategy corresponding to the evaluation configuration information;

[0012] Based on the basic indicator score and model gain score of each external data source, the final evaluation result of each external data source is determined and output.

[0013] Optionally, the basic indicator evaluation strategy includes multiple indicator dimensions, and the basic indicator evaluation strategy corresponding to the evaluation configuration information processes the credit risk control data of multiple external data sources respectively to obtain the basic indicator score of each external data source, including:

[0014] Based on the credit risk control data from multiple external data sources, calculate the indicator score of each external data source in each indicator dimension;

[0015] The indicator scores of each external data source in each indicator dimension are summed up in the data source dimension to obtain the basic indicator score of each external data source.

[0016] Optionally, the basic indicator evaluation strategy further includes indicator weights, indicator value intervals, and corresponding interval scores. The step of calculating the indicator score of each external data source in each indicator dimension based on the credit risk control data of multiple external data sources includes:

[0017] Determine the interval distribution ratio of the credit risk control data in each indicator dimension according to the value of the credit risk control data corresponding to each indicator dimension and the indicator value interval;

[0018] The indicator score for each indicator dimension is calculated based on the indicator weight, indicator value interval, interval score corresponding to each indicator dimension and the interval distribution ratio of credit risk control data in each indicator dimension.

[0019] Optionally, the constructing a risk prediction model based on the credit risk control data from the internal data source and the credit risk control data from multiple external data sources includes:

[0020] Training a first risk prediction model using the credit risk control data from the internal data source;

[0021] Merging the credit risk control data from the internal data source with the credit risk control data from multiple external data sources to obtain multiple merged credit risk control data;

[0022] The second risk prediction model is trained using each integrated credit risk control data.

[0023] Optionally, the model gain evaluation strategy includes a model gain evaluation index, and the step of determining the model gain score of each external data source based on the risk prediction model and using the model gain evaluation strategy corresponding to the evaluation configuration information includes:

[0024] Using the model gain evaluation index to respectively evaluate the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source;

[0025] The model gain score of each external data source is calculated based on the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source.

[0026] Optionally, the model gain evaluation strategy also includes an index value improvement interval and a corresponding interval score of the model gain evaluation index, and the model gain score of each external data source is calculated according to the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source, including:

[0027] According to the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each of the external data sources, respectively calculate the index improvement value of the model gain evaluation index corresponding to each of the external data sources;

[0028] The model gain score of each external data source is calculated according to the indicator improvement value, the indicator value improvement interval and the corresponding interval score of the model gain evaluation indicator corresponding to each of the external data sources.

[0029] Optionally, the determining and outputting of the final evaluation result of each external data source according to the basic indicator score and the model gain score of each external data source includes:

[0030] Calculate the comprehensive score of each external data source based on the basic indicator score and model gain score of each external data source;

[0031] Based on the comprehensive score of each external data source, an evaluation report is generated and outputted using a preset report generation model, wherein the evaluation report includes an evaluation report of each external data source and / or a comparative analysis report between external data sources.

[0032] Optionally, after determining and outputting the final evaluation result of each external data source according to the basic indicator score and model gain score of each external data source, the credit risk control data source evaluation method further includes:

[0033] Receive user's evaluation configuration adjustment information;

[0034] The evaluation report is updated and output according to the evaluation configuration adjustment information.

[0035] In a second aspect, an embodiment of the present application further provides an evaluation device for a credit risk control data source, characterized in that the evaluation device for a credit risk control data source comprises:

[0036] An acquisition unit, used to acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources;

[0037] A first scoring unit is used to process the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source;

[0038] A second scoring unit is used to build a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determine the model gain score of each external data source based on the risk prediction model using the model gain evaluation strategy corresponding to the evaluation configuration information;

[0039] The determination unit is used to determine and output the final evaluation result of each external data source based on the basic indicator score and model gain score of each external data source.

[0040] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0041] Processor; and

[0042] A memory arranged to store computer executable instructions, which, when executed, cause the processor to execute any of the aforementioned credit risk control data source evaluation methods.

[0043] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes any of the aforementioned credit risk control data source evaluation methods.

[0044] At least one of the above technical solutions adopted in the embodiment of the present application can achieve the following beneficial effects: the evaluation method of the credit risk control data source of the embodiment of the present application first obtains the credit risk control data and the evaluation configuration information of the user, the credit risk control data includes the credit risk control data of the internal data source and the credit risk control data of multiple external data sources; then the credit risk control data of multiple external data sources are processed respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain the basic indicator score of each external data source; then a risk prediction model is constructed based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and based on the risk prediction model, the model gain score of each external data source is determined by using the model gain evaluation strategy corresponding to the evaluation configuration information; finally, according to the basic indicator score and model gain score of each external data source, the final evaluation result of each external data source is determined and output. The evaluation method of the credit risk control data source of the embodiment of the present application improves the accuracy and flexibility of the evaluation of different external data sources in the credit risk control scenario, realizes the horizontal comparison function between different external data sources, is conducive to the decision-making and use of application personnel, and the whole process can be automatically realized without manual operation, thereby improving the evaluation and application efficiency of the data source. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 This is a flow chart of a method for evaluating a credit risk control data source in an embodiment of the present application;

[0047] Figure 2 This is a schematic diagram of the structure of an evaluation device for a credit risk control data source in an embodiment of the present application;

[0048] Figure 3 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0050] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0051] This application embodiment provides a method for evaluating a credit risk control data source, such as Figure 1 As shown, a flow chart of a method for evaluating a credit risk control data source in an embodiment of the present application is provided. The method for evaluating a credit risk control data source at least includes the following steps S110 to S140:

[0052] Step S110, obtaining credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources.

[0053] The credit risk control data source to be evaluated in the embodiment of the present application mainly refers to the third-party data source. The third-party data source refers to the processed data with its own characteristics formed by the external third-party platform based on long-term business accumulation. These data may include multiple loan data, payment data, credit score data, etc. They play an important role in financial risk control, market analysis and other fields.

[0054] When evaluating the credit risk control data source, it is necessary to first obtain the required credit risk control data. Credit risk control data can be specifically divided into internal data sources and credit risk control data from multiple external data sources. Internal data sources mainly refer to the data generated and accumulated by the credit institution itself during its business operations. These data are usually highly exclusive and real-time, and are an important basis for credit institutions to understand the credit status of borrowers and assess loan risks. External data sources refer to the above-mentioned three-party data sources. The sources of external credit risk control data are diverse, so it is necessary to comprehensively evaluate and analyze the advantages and disadvantages of these external data sources.

[0055] In addition, it is also necessary to obtain the user's evaluation configuration information, that is, the business party with data source evaluation needs can flexibly configure the corresponding evaluation strategy according to its own business needs and usage scenarios. For example, it can configure the key indicators to be evaluated and the output of the evaluation results, etc.

[0056] Step S120, processing the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source.

[0057] The evaluation of external data sources in the embodiment of the present application is mainly divided into two aspects. One aspect is the evaluation of basic indicators. Since different evaluation configuration information will affect the actual evaluation strategy adopted, it is necessary to determine the current basic indicator evaluation strategy based on the evaluation configuration information, and use the basic indicator evaluation strategy to perform statistical analysis on the credit risk control data of each external data source, so as to obtain the basic indicator score of each external data source.

[0058] Step S130, building a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and based on the risk prediction model, determining the model gain score of each external data source using the model gain evaluation strategy corresponding to the evaluation configuration information.

[0059] Another aspect of the evaluation of external data sources in the embodiments of the present application is the evaluation of model gain effects. The evaluation of model gain effects mainly evaluates the ability of each external data source to improve the predictive effect of risk control related models. Therefore, it is necessary to first use the credit risk control data of internal data sources and multiple external data sources to build a risk prediction model, and then based on the risk prediction model, use the model gain evaluation strategy corresponding to the above-mentioned evaluation configuration information to calculate the model gain score of each external data source.

[0060] Step S140, according to the basic indicator score and model gain score of each external data source, determine and output the final evaluation result of each external data source.

[0061] The scores of the above basic indicators reflect the quality of the data of the external data source on some key indicators, and the scores of the model gain reflect the ability of the external data source to improve the prediction effect of the existing risk control model. Therefore, the embodiment of the present application can comprehensively determine the final evaluation results of each external data source by combining the basic indicator scores and model gain scores of each external data source and output them for interpretation and decision-making by the business party.

[0062] The evaluation method of the credit risk control data source in the embodiment of the present application improves the accuracy and flexibility of the evaluation of different external data sources in the credit risk control scenario, realizes the horizontal comparison function between different external data sources, is beneficial to the decision-making of application personnel, and the whole process can be realized automatically without manual operation, thereby improving the evaluation and application efficiency of the data source.

[0063] In some embodiments of the present application, when obtaining credit risk control data, a data reading engine can be used for reading, which is compatible with CSV and XLSX formats and supports batch data import, merging and table splicing.

[0064] In addition, due to the diversity of data sources, traditional methods do not take into account the possible data quality issues such as missing data, incorrect data entry, inconsistent formats, etc., which directly affects the accuracy of risk assessment. In addition, the traditional risk assessment process often requires manual data cleaning, statistical analysis and other work, which is not only time-consuming and labor-intensive, but also prone to human errors.

[0065] Based on this, the embodiment of the present application can also design a data parser that can identify and manually set data types and structures, and automatically map them to the system's internal data model. If the automatically identified field type is incorrect, manual modification is supported. For example, if "date of birth" is identified as a string format, it can be manually modified to a date format. Furthermore, a data verification mechanism can be introduced to check for data format errors and potential data quality issues, thereby ensuring the accuracy of subsequent data processing.

[0066] In some embodiments of the present application, the basic indicator evaluation strategy includes multiple indicator dimensions, and the basic indicator evaluation strategy corresponding to the evaluation configuration information processes the credit risk control data of multiple external data sources respectively to obtain the basic indicator score of each external data source, including: calculating the indicator score of each external data source in each indicator dimension respectively according to the credit risk control data of multiple external data sources; summing the indicator score of each external data source in each indicator dimension in the data source dimension to obtain the basic indicator score of each external data source.

[0067] The embodiment of the present application may predefine a variety of evaluation indicators according to the data characteristics of different external data sources and business requirements, for example, the following indicator dimensions may be included:

[0068] 1) Missing rate: Calculate the proportion of missing values ​​in each column of data.

[0069] 2) Concentration: Evaluate the concentration of data distribution by calculating the value with the largest proportion in each column of data.

[0070] 3) Correlation: Use Pearson correlation coefficient or Spearman rank correlation coefficient to analyze the linear and nonlinear relationships between variables.

[0071] 4) IV value (information value): evaluates the contribution of a single variable to the predictive power of the target variable. By using supervised methods such as decision tree binning and chi-square binning to bin the feature variables, the weight of evidence WOE of each bin is calculated. It can be expressed as follows:

[0072] bin_woe=ln((bin_bad / total_bad)*(total_good / bin_good))

[0073] Among them, bin_bad is the number of bad samples in a certain bin of the variable, bin_good is the number of good samples, total_bad is the total number of bad samples, and total_good is the total number of good samples.

[0074] Finally, the variable IV value is calculated, which can be expressed as follows:

[0075] bin_iv=(bin_bad / total_bad)-(total_good / bin_good)*bin_woe

[0076] IV = sum(bin_iv)

[0077] 5) Stability: Through time series analysis, detect the changing trend of data over time and ensure the long-term stability of data.

[0078] 6) Query rate: 1-missing rate.

[0079] 7) Accuracy / coverage (blacklist category): the number of hits on the blacklist / the number of hits on the blacklist in the sample.

[0080] 8) False rejection rate = number of blacklisted customers / number of customers who passed in the early stage and are good customers after the loan.

[0081] 9) Effective difference rate = number of hits in the blacklist / number of passed and bad samples in the sample.

[0082] 10) Invalid difference rate = number of hits on the blacklist / other rejection rates in the sample.

[0083] It should be noted that the above-mentioned evaluation indicators are merely examples of the embodiments of the present application. Those skilled in the art can flexibly set the specific evaluation indicators according to actual needs, and no specific limitation is made here.

[0084] Based on the above evaluation indicators, the indicator score of each external data source in each indicator dimension can be calculated separately according to the credit risk control data of each external data source, and finally summarized in the data source dimension to obtain the basic indicator score of each data source.

[0085] In specific calculations, the embodiment of the present application can use statistics and machine learning algorithms to calculate the index value, and the calculation method of each index is already built into the system, and the method and related thresholds can be adjusted through their respective pages without writing code for calculation. For example, for the calculation of correlation, algorithms such as Pearson and Spearman can be selected for calculation, and corresponding screening thresholds can be set.

[0086] In some embodiments of the present application, the basic indicator evaluation strategy also includes indicator weights, indicator value intervals and corresponding interval scores. The calculation of the indicator scores of each external data source in each indicator dimension based on the credit risk control data of multiple external data sources includes: determining the interval distribution ratio of the credit risk control data in each indicator dimension based on the numerical value of the credit risk control data corresponding to each indicator dimension and the indicator value interval; calculating the indicator score of each indicator dimension based on the indicator weight, indicator value interval, interval score and the interval distribution ratio of the credit risk control data in each indicator dimension.

[0087] The basic indicator evaluation strategy of the embodiment of the present application also includes indicator weights, indicator value intervals and corresponding interval scores. The score of each indicator dimension is calculated based on the indicator weight corresponding to each indicator dimension, the indicator value interval in which the data variable falls, the interval score and the proportion of the data variable falling in the indicator value interval.

[0088] As shown in Table 1 below, taking variable data as an example, the full score is 100 points. For the stability index, 80% of the variables (number) fall in the interval of (0, 0.05], then the score of this interval is 24 points. The sum of the scores of the stability index in all intervals is the total score of the stability index. Finally, the sum of all indicator scores of the current data source is the total score of the basic indicators of the data source.

[0089] Table 1

[0090]

[0091] It should be noted that Table 1 above is only an example of a specific implementation of the embodiment of the present application. The specific setting of indicator weights, division of indicator value intervals, and setting of interval scores can be flexibly adjusted by technical personnel in this field according to actual needs and are not specifically limited here.

[0092] In some embodiments of the present application, constructing a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources includes: training a first risk prediction model using the credit risk control data of the internal data source; fusing the credit risk control data of the internal data source with the credit risk control data of multiple external data sources to obtain multiple fused credit risk control data; and training a second risk prediction model using each fused credit risk control data.

[0093] When evaluating the model gain effect of external data sources, the risk prediction model can be first constructed using the external data sources. The construction of the model here is divided into two aspects. On the one hand, it is based on the existing internal data source to build a separate model, that is, to use the existing internal data source to train a first risk prediction model. On the other hand, the data of the internal data source is fused with the data of each external data source, and multiple fused data sources are used to build models separately to obtain multiple second risk prediction models.

[0094] The above-mentioned risk prediction model can be, for example, a model related to credit risk control, such as predicting the user's default probability or the expected probability of overdue payment. The specific network structure to be used for training can be flexibly selected by technicians in this field in combination with existing technologies, and no specific limitation is made here.

[0095] In some embodiments of the present application, the model gain evaluation strategy includes a model gain evaluation indicator, and the model gain score of each external data source is determined based on the risk prediction model and using the model gain evaluation strategy corresponding to the evaluation configuration information, including: using the model gain evaluation indicator to respectively evaluate the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source; and calculating the model gain score of each external data source based on the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source.

[0096] The embodiment of the present application can use a pre-defined model gain evaluation index to evaluate the prediction effect of the model. The model gain evaluation index here can include, for example, the AUC index and the KS index. The AUC index is the area under the curve (Area Under Curve). The higher the AUC value, the stronger the model's ability to rank positive samples before negative samples, that is, the higher the prediction accuracy of the model. The KS index is used to measure the model's ability to distinguish between positive and negative samples, and calculates the maximum difference between the cumulative distribution of positive and negative samples. The larger the KS value, the stronger the risk differentiation ability of the model.

[0097] By comparing the index values ​​of the model gain evaluation index of the second risk prediction model corresponding to each external data source with the index values ​​of the model gain evaluation index of the first risk prediction model, the gain effect of each external data source on the risk prediction model can be analyzed and the model gain score can be calculated.

[0098] In some embodiments of the present application, the model gain evaluation strategy also includes an indicator value improvement interval and a corresponding interval score of the model gain evaluation indicator. The calculation of the model gain score of each external data source based on the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each of the external data sources includes: calculating the indicator improvement value of the model gain evaluation indicator corresponding to each of the external data sources according to the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each of the external data sources; calculating the model gain score of each external data source according to the indicator improvement value, indicator value improvement interval and the corresponding interval score of the model gain evaluation indicator corresponding to each of the external data sources.

[0099] The model gain evaluation strategy of the embodiment of the present application also includes the index value improvement interval of the model gain evaluation index and the corresponding interval score, that is, the improvement effect of the model is divided into multiple intervals, and different intervals correspond to different gain scores, thereby improving the accuracy of the model gain effect evaluation.

[0100] As shown in Table 2, for example, KS is improved by 0.015 points, which falls within the interval of (0.01, 0.02], and the corresponding KS gain score is 40 points. AUC is improved by 0.025, which falls within the interval of (0.02, 0.03], and the corresponding AUC gain score is 70 points. Then the model gain score = 40*0.5+70*0.5=55.

[0101] Table 2

[0102] Model Improvement KS Gain Score AUC Gain Score [0,0.01] 10 10 (0.01,0.02] 40 40 (0.02,0.03] 70 70 (0.03,0.04] 90 90 (0.04,inf) 100 100

[0103] It should be noted that Table 2 above is only an example of a specific implementation of the embodiment of the present application. The specific division of the index value interval and the setting of the interval score can be flexibly adjusted by technical personnel in this field according to actual needs and are not specifically limited here.

[0104] In some embodiments of the present application, determining and outputting the final evaluation result of each external data source based on the basic indicator score and the model gain score of each external data source includes: calculating a comprehensive score of each external data source based on the basic indicator score and the model gain score of each external data source; based on the comprehensive score of each external data source, generating and outputting an evaluation report using a preset report generation model, wherein the evaluation report includes an evaluation report of each external data source and / or a comparative analysis report between external data sources.

[0105] After the basic indicator score and model gain score of each external data source are calculated based on the above embodiment, the basic indicator score and model gain score of each external data source can be weighted and summed to obtain a comprehensive score of each external data source. For example, it can be expressed as follows:

[0106] Comprehensive score of data source = data source basic indicator score * α + data source model gain score * β

[0107] α+β=1

[0108] Considering that the process of generating existing data source evaluation reports is usually very complicated, it is necessary to manually summarize the analysis results and write a detailed report, which is time-consuming and difficult to ensure the consistency and completeness of the report. Based on this, the embodiment of the present application can automatically generate a corresponding evaluation report based on the above evaluation results, and can generate an evaluation report for a single data source, or a comparative analysis evaluation report between multiple data sources.

[0109] Specifically, when generating an evaluation report, you can preset a report template and design the sections and formats of the final output of the report, such as the sample training and test set data distribution section, the indicator compliance rate section for each variable, and the test conclusion section. Deploy the artificial intelligence big model offline to automatically generate an evaluation report based on the preset report template and indicator calculation results. The report can include the specific value of each indicator, the comparison result with the set threshold, the data column identification that exceeds the threshold, and the recommended measures. In addition, graphical displays, such as bar charts, line charts, and heat maps, can also be provided to help intuitively understand the data quality status.

[0110] The embodiment of the present application adopts an artificial intelligence big model to generate an evaluation report for a data source. On the one hand, it can improve the flexibility of the conclusions of the evaluation report, because the big model can flexibly adjust the analysis logic according to a variety of preset evaluation scenarios and indicators, and can easily update its internal logic and rules to adapt to new testing standards and requirements, showing extremely high flexibility.

[0111] On the other hand, the large model has a strong ability to capture key points when generating evaluation reports. It can efficiently analyze massive amounts of data through deep learning algorithms, quickly identify key issues in the evaluation process, and give special attention and explanations in the report, which helps application personnel quickly locate problems and provide strong guidance for subsequent repair and optimization work.

[0112] On the other hand, since the large model has learned a large amount of natural language text during the training process and has mastered a wealth of language expression skills and logical structures, the large model of the embodiment of the present application can use more appropriate language and sentence patterns to clearly explain the test results and problems when generating the conclusion of the evaluation report. At the same time, it can also adjust the difficulty and professionalism of the speech according to the different backgrounds and needs of the application personnel to ensure that the report conclusions are both accurate and easy to understand.

[0113] In addition, the embodiment of the present application generates an evaluation report for the data source through an offline deployed artificial intelligence big model, which can greatly save manpower, improve the efficiency, accuracy and comprehensiveness of report generation, and can flexibly adapt to the needs of complex and changeable data sources and business scenarios through continuous learning and optimization.

[0114] In order to improve the presentation effect of the report, the embodiment of the present application can also divide the indicators, for example, they can be divided into several categories as shown in Table 3 below, and the indicator calculation results of each data source under different categories of indicator dimensions are displayed respectively, so that users can clearly and intuitively see the indicator scores under each category of indicators.

[0115] Table 3

[0116] List indicators Variable Class Indicators Comprehensive scoring indicators Missing rate Missing rate Missing rate Concentration Concentration Concentration IV IV IV Detection rate Relevance Relevance Accuracy stability stability False Rejection Rate KS Effective difference rate Invalid difference rate stability

[0117] Furthermore, for the generation of comparative analysis and evaluation reports between multiple data sources, we can first compare the third-party data of the same data source horizontally based on the classification of data sources such as multiple-head category, payment category, app list, etc., and display the indicator ranking and comprehensive ranking of the same data source separately, which is convenient for decision makers to use. On the one hand, we can rank according to the indicators, and rank all the same data sources for each key indicator such as missing rate, concentration, correlation, IV value, KS value and stability, etc., to show which data source performs best on this indicator. On the other hand, we can rank according to the comprehensive score, score the data source based on the comprehensive performance of all key indicators, and finally rank the overall data source according to the scoring results to compare the advantages and disadvantages of the data source. Decision makers can adjust the weights of key indicators to change the ranking rules. In this way, the problem that the existing evaluation method cannot quantify and compare the advantages and disadvantages of the same data source and the effect is not intuitive enough is solved.

[0118] Of course, the specific method for generating the above evaluation report can be flexibly selected by those skilled in the art in combination with the existing technology, and is not specifically limited here.

[0119] In some embodiments of the present application, after determining and outputting the final evaluation result of each external data source based on the basic indicator score and model gain score of each external data source, the evaluation method of the credit risk control data source also includes: receiving the user's evaluation configuration adjustment information; updating the evaluation report according to the evaluation configuration adjustment information and outputting it.

[0120] The embodiment of the present application provides an adjustment function for evaluation configuration information. According to the various indicators automatically calculated by the aforementioned embodiment, the thresholds of various indicators can be manually adjusted to adjust the content of the report output, so that the generated evaluation report is more suitable for interpretation and decision-making by business personnel, and the flexibility of generating the evaluation report is improved. Thus, the problem that the existing credit risk control data evaluation method only supports preset evaluation indicators and thresholds, lacks flexibility, and cannot adapt well to different scenarios and specific needs is solved.

[0121] In the embodiment of the present application, a variable filtering process can also be set during the calculation stage, each process is configured with different calculation indicators, and the number of passes and the pass rate of each step are displayed.

[0122] The present application embodiment also provides a credit risk control data source evaluation device 200, such as Figure 2 As shown, a schematic diagram of the structure of an evaluation device for a credit risk control data source in an embodiment of the present application is provided. The evaluation device 200 for a credit risk control data source includes: an acquisition unit 210, a first scoring unit 220, a second scoring unit 230, and a determination unit 240, wherein:

[0123] An acquisition unit 210 is used to acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources;

[0124] A first scoring unit 220 is used to process the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source;

[0125] A second scoring unit 230 is used to build a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determine the model gain score of each external data source based on the risk prediction model using the model gain evaluation strategy corresponding to the evaluation configuration information;

[0126] The determination unit 240 is used to determine and output the final evaluation result of each external data source according to the basic indicator score and the model gain score of each external data source.

[0127] In some embodiments of the present application, the basic indicator evaluation strategy includes multiple indicator dimensions, and the first scoring unit 220 is specifically used to: calculate the indicator score of each external data source in each indicator dimension based on the credit risk control data of multiple external data sources; sum the indicator score of each external data source in each indicator dimension in the data source dimension to obtain the basic indicator score of each external data source.

[0128] In some embodiments of the present application, the basic indicator evaluation strategy also includes indicator weights, indicator value intervals and corresponding interval scores, and the first scoring unit 220 is specifically used to: determine the interval distribution ratio of the credit risk control data in each indicator dimension according to the numerical value of the credit risk control data corresponding to each indicator dimension and the indicator value interval; calculate the indicator score of each indicator dimension according to the indicator weight, indicator value interval, interval score and the interval distribution ratio of the credit risk control data in each indicator dimension.

[0129] In some embodiments of the present application, the second scoring unit 230 is specifically used to: train a first risk prediction model using the credit risk control data of the internal data source; fuse the credit risk control data of the internal data source with the credit risk control data of multiple external data sources to obtain multiple fused credit risk control data; and train a second risk prediction model using each fused credit risk control data.

[0130] In some embodiments of the present application, the model gain evaluation strategy includes a model gain evaluation index, and the second scoring unit 230 is specifically used to: use the model gain evaluation index to respectively evaluate the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source; calculate the model gain score of each external data source based on the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source.

[0131] In some embodiments of the present application, the model gain evaluation strategy also includes an indicator value improvement interval and a corresponding interval score of the model gain evaluation indicator, and the second scoring unit 230 is specifically used to: calculate the indicator improvement value of the model gain evaluation indicator corresponding to each of the external data sources according to the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each of the external data sources; calculate the model gain score of each external data source according to the indicator improvement value, indicator value improvement interval and corresponding interval score of the model gain evaluation indicator corresponding to each of the external data sources.

[0132] In some embodiments of the present application, the determination unit 240 is specifically used to: calculate a comprehensive score for each external data source based on the basic indicator score and model gain score of each external data source; based on the comprehensive score of each external data source, generate and output an evaluation report using a preset report generation model, wherein the evaluation report includes an evaluation report for each external data source and / or a comparative analysis report between external data sources.

[0133] In some embodiments of the present application, the evaluation device 200 of the credit risk control data source also includes: a receiving unit, which is used to receive the user's evaluation configuration adjustment information after determining and outputting the final evaluation result of each external data source based on the basic indicator score and model gain score of each external data source; and an updating unit, which is used to update and output the evaluation report based on the evaluation configuration adjustment information.

[0134] It can be understood that the above-mentioned evaluation device of the credit risk control data source can implement the various steps of the evaluation method of the credit risk control data source provided in the aforementioned embodiment. The relevant explanations on the evaluation method of the credit risk control data source are applicable to the evaluation device of the credit risk control data source and will not be repeated here.

[0135] Figure 3 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 3At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. The memory may include a memory, such as a high-speed random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage. Of course, the electronic device may also include hardware required for other services.

[0136] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0137] The memory is used to store the program. Specifically, the program may include a program code, and the program code includes a computer operation instruction. The memory may include a memory and a non-volatile memory, and provides instructions and data to the processor.

[0138] The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it, forming an evaluation device for the credit risk control data source at the logical level. The processor executes the program stored in the memory and is specifically used to perform the following operations:

[0139] Acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources;

[0140] Processing the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source;

[0141] Building a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determining a model gain score of each external data source based on the risk prediction model and using a model gain evaluation strategy corresponding to the evaluation configuration information;

[0142] Based on the basic indicator score and model gain score of each external data source, the final evaluation result of each external data source is determined and output.

[0143] The above application Figure 1 The method performed by the evaluation device of the credit risk control data source disclosed in the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0144] The present application also provides a computer-readable storage medium, which stores one or more programs, wherein the one or more programs include instructions, which, when executed by an electronic device including multiple application programs, enable the electronic device to execute Figure 1 The method performed by the evaluation device of the credit risk control data source in the embodiment shown is specifically used to perform:

[0145] Acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources;

[0146] Processing the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source;

[0147] Building a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determining a model gain score of each external data source based on the risk prediction model and using a model gain evaluation strategy corresponding to the evaluation configuration information;

[0148] Based on the basic indicator score and model gain score of each external data source, the final evaluation result of each external data source is determined and output.

[0149] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0151] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0152] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0153] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0154] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0155] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0156] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0157] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0158] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for evaluating a credit risk control data source, characterized in that: The evaluation method of the credit risk control data source includes: Acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources; Processing the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source; Building a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determining a model gain score of each external data source based on the risk prediction model and using a model gain evaluation strategy corresponding to the evaluation configuration information; Based on the basic indicator score and model gain score of each external data source, the final evaluation result of each external data source is determined and output.

2. The evaluation method for credit risk control data source according to claim 1 is characterized in that: The basic indicator evaluation strategy includes multiple indicator dimensions. The basic indicator evaluation strategy corresponding to the evaluation configuration information processes the credit risk control data of multiple external data sources respectively to obtain the basic indicator score of each external data source, including: Based on the credit risk control data from multiple external data sources, calculate the indicator score of each external data source in each indicator dimension; The indicator scores of each external data source in each indicator dimension are summed up in the data source dimension to obtain the basic indicator score of each external data source.

3. The evaluation method for credit risk control data source according to claim 2 is characterized in that: The basic indicator evaluation strategy also includes indicator weights, indicator value intervals, and corresponding interval scores. The calculation of the indicator score of each external data source in each indicator dimension based on the credit risk control data of multiple external data sources includes: Determine the interval distribution ratio of the credit risk control data in each indicator dimension according to the value of the credit risk control data corresponding to each indicator dimension and the indicator value interval; The indicator score for each indicator dimension is calculated based on the indicator weight, indicator value interval, interval score corresponding to each indicator dimension and the interval distribution ratio of credit risk control data in each indicator dimension.

4. The evaluation method for credit risk control data source according to claim 1 is characterized in that: The step of constructing a risk prediction model based on the credit risk control data from the internal data source and the credit risk control data from multiple external data sources includes: Training a first risk prediction model using the credit risk control data from the internal data source; Merging the credit risk control data from the internal data source with the credit risk control data from multiple external data sources to obtain multiple merged credit risk control data; The second risk prediction model is trained using each integrated credit risk control data.

5. The evaluation method for credit risk control data source according to claim 4 is characterized in that: The model gain evaluation strategy includes a model gain evaluation index, and the model gain score of each external data source is determined based on the risk prediction model using the model gain evaluation strategy corresponding to the evaluation configuration information, including: Using the model gain evaluation index to respectively evaluate the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source; The model gain score of each external data source is calculated based on the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source.

6. The evaluation method for credit risk control data source according to claim 5 is characterized in that: The model gain evaluation strategy also includes an index value improvement interval and a corresponding interval score of the model gain evaluation index. The model gain score of each external data source is calculated based on the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each external data source, including: According to the risk prediction effect of the first risk prediction model and the risk prediction effect of the second risk prediction model corresponding to each of the external data sources, respectively calculate the index improvement value of the model gain evaluation index corresponding to each of the external data sources; The model gain score of each external data source is calculated according to the indicator improvement value, the indicator value improvement interval and the corresponding interval score of the model gain evaluation indicator corresponding to each of the external data sources.

7. The evaluation method for credit risk control data source according to claim 1 is characterized in that: The final evaluation result of each external data source is determined and output according to the basic indicator score and model gain score of each external data source, including: Calculate the comprehensive score of each external data source based on the basic indicator score and model gain score of each external data source; Based on the comprehensive score of each external data source, an evaluation report is generated and outputted using a preset report generation model, wherein the evaluation report includes an evaluation report of each external data source and / or a comparative analysis report between external data sources.

8. The evaluation method for credit risk control data source according to claim 7 is characterized in that: After determining and outputting the final evaluation result of each external data source according to the basic indicator score and model gain score of each external data source, the evaluation method of the credit risk control data source further includes: Receive user's evaluation configuration adjustment information; The evaluation report is updated and output according to the evaluation configuration adjustment information.

9. A device for evaluating a credit risk control data source, characterized in that: The evaluation device of the credit risk control data source includes: An acquisition unit, used to acquire credit risk control data and user evaluation configuration information, wherein the credit risk control data includes credit risk control data from an internal data source and credit risk control data from multiple external data sources; A first scoring unit is used to process the credit risk control data of multiple external data sources respectively according to the basic indicator evaluation strategy corresponding to the evaluation configuration information to obtain a basic indicator score for each external data source; A second scoring unit is used to build a risk prediction model based on the credit risk control data of the internal data source and the credit risk control data of multiple external data sources, and determine the model gain score of each external data source based on the risk prediction model using the model gain evaluation strategy corresponding to the evaluation configuration information; The determination unit is used to determine and output the final evaluation result of each external data source based on the basic indicator score and model gain score of each external data source.

10. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, wherein when the executable instructions are executed, the processor executes the evaluation method of the credit risk control data source according to any one of claims 1 to 8.

11. A computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the evaluation method of the credit risk control data source according to any one of claims 1 to 8.

Images