An automated feature engineering credit risk assessment system and method

By using Copula entropy and transfer entropy test combined with custom conversion functions, the problem of high cost and poor interpretability of artificial feature engineering in traditional credit score card models is solved, and an efficient and interpretable credit score card model construction is achieved. , significantly improving the performance and efficiency of credit risk assessment.

CN114049198BActive Publication Date: 2024-11-26BEIJING DAOKOU JINKE TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111333182.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-11
Publication Date
2024-11-26
Estimated Expiration
2041-11-11

AI Technical Summary

Technical Problem

The construction of traditional credit score card models requires a lot of manual feature engineering, resulting in high cost and waste of time, and it is difficult to find nonlinear but causal connections, affecting the interpretability and operational efficiency of the model.

Method used

Copula entropy and transfer entropy are used for correlation and causality testing, combined with the automatic derivation of features of custom transformation functions, a credit score card model with causal relationship is constructed, shortening the modeling time and improving the interpretability of the model.

Benefits of technology

It significantly shortens the modeling time, improves the interpretability and classification effect of the model, improves the efficiency of feature generation, ensures the automatic generation of causal correlation, and significantly improves the performance of credit risk assessment.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114049198B_ABST
    Figure CN114049198B_ABST
Patent Text Reader

Abstract

The present invention provides an automated feature engineering credit risk assessment system and method, belonging to the field of computer science. The system of the present invention includes an original feature extraction module, a new feature derivation module, a new feature verification module, a feature automatic generation module, and a customer credit risk classification model. The method of the present invention includes: collecting customer data to extract original features, constructing a custom conversion function to derive new features, constructing a verification function to verify new features, automatically generating features, constructing a customer credit classification model, generating a sample training classification model according to a feature set, and using the trained model to classify customer data. The credit score card model constructed by the present invention has more interpretable automated features with causal relationships, which improves the modeling time by more than 50%, and the model classification effect is more significant.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to technologies such as computers, big data mining and credit score card models, and to the application of automatically generated causal relationship features in credit score card models, and specifically to an automated feature engineering credit risk evaluation system and method. Background Art

[0002] The credit score card model is a model for defining personal financial rights that has emerged in recent years to ensure the financial security of banks and other financial sectors. It is a decision-making tool for risk management. The model refers to analyzing and mining customer credit behavior data based on the customer's credit history information, using a certain credit scoring model to obtain different levels of credit scores, and determining the amount of money the customer can hold based on the customer's credit score, thereby ensuring the security of repayment and other businesses. As the role of loans and credit cards becomes increasingly prominent in modern society and companies, the credit score card model is becoming more and more important, and its development prospects are immeasurable.

[0003] In the traditional financial context, building a standard credit scorecard model sometimes requires inputting hundreds of feature variables; if experts manually construct features, feature engineering work needs to be re-performed for another application scenario, and a large part of it may be repetitive work, which leads to a large waste of labor and time costs, affecting model development and operational efficiency; and this method is limited by manual experience and may miss some effective features. Therefore, how to effectively utilize and reuse knowledge, reduce labor and time costs, and automatically generate high-utility and comprehensive features is of great significance to the development of standard credit scorecard models.

[0004] With the rise of machine learning and deep learning in recent years, many financial institutions have begun to try to use black box models such as nonlinear models or neural networks for prediction. However, in traditional financial scenarios, the output results of the model are required to be explainable and traceable. Most nonlinear models cannot establish a causal relationship between the input features and the prediction results, and naturally cannot explain the prediction results. Secondly, to complete a credit scorecard model, it is generally necessary to build hundreds of model features. The construction of model features mainly relies on the industry experience of experts. However, limited by past experience, it is difficult to discover some nonlinear but causally related indicators. Feature work needs to be re-performed for different data scenarios, which is not only time-consuming and labor-intensive, but may also cause omissions. In summary, how to automatically generate efficient and explainable features with causal associations is crucial to building an explainable and traceable credit scorecard model, and is a problem that needs to be solved at present. Summary of the invention

[0005] In view of the above problems, the present invention provides an automated feature engineering credit risk assessment system and method, which utilizes the technology of Copula entropy and transfer entropy plus custom conversion function, and can automatically derive the extracted basic features. The constructed credit score card model is more solvable and causal, and the modeling time is greatly shortened. This technical solution is implemented under the premise of complying with relevant laws and regulations on personal information protection and meets the relevant circumstances of legal use.

[0006] An automated feature engineering credit risk assessment method of the present invention comprises the following steps:

[0007] Step 1: Collect customer data, extract original features and add them to feature sets; each customer corresponds to a sample, and set the sample generation method;

[0008] Step 2: Design a custom transformation function to derive new features based on the features of the feature set;

[0009] The custom conversion function is divided into three categories according to different working domains: mapping conversion function, aggregation conversion function, and time window conversion function; wherein the mapping conversion function inputs a single feature or multiple features into a custom mapping function to generate a new feature; the mapping function includes a logarithmic transformation function, a differential operation function, and a Gaussian Copula function; the aggregation conversion function performs aggregation calculations on multiple groups of data of a single feature to generate a new feature; the time window conversion function uses the concept of a sliding time window to operate on the data of a single feature in multiple time windows;

[0010] Step 3, construct a test evaluation function to test the derived features;

[0011] The test includes correlation test and causality test. Copula entropy is used for correlation test, and transfer entropy is used for causality test. If the new feature passes the test of Copula entropy and transfer entropy, it is added to the feature set. Otherwise, the new feature is combined with the features in the feature set and the custom conversion function in step 2 is reused for derivation.

[0012] Step 4: Generate a final feature set, extract feature data from the customer data according to the feature set, and generate samples;

[0013] Step 5: Select a logistic regression model as the customer credit risk classification model and train the model; when evaluating customer credit risk, extract customer data according to the final feature set, input the trained model, and obtain the customer credit risk classification result.

[0014] Accordingly, the present invention provides an automated feature engineering credit risk assessment system, comprising the following modules:

[0015] The original feature extraction module is used to extract database fields from the collected customer data, obtain original features and store them in the feature set;

[0016] New feature derivation module, used to derive new features from features in feature set according to custom conversion functions;

[0017] New feature test module, which performs correlation test and causality test on new features; among them, Copula entropy is used for correlation test, and transfer entropy is used for causality test;

[0018] The feature automatic generation module is used to call the new feature derivation module and the new feature verification module to automatically generate new features, store the new features that pass the correlation test and causality test into the feature set, and for the new features that fail the test, combine the features in the feature set, reuse the new feature derivation model to derive new features, and then use the new feature verification module to test them;

[0019] The customer credit risk classification model uses a logistic regression model to classify customer credit risks. The input of the model is user data obtained based on a feature set, and the output is the customer credit risk classification result.

[0020] Compared with the prior art, the advantages and positive effects of the present invention are: the credit score card model constructed by the method and system of the present invention has more interpretable automated feature engineering with causal relationships, and has been verified by experiments to have improved the modeling time by more than 50%, and the model classification effect is more significant. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is an overall flow chart of the automated feature engineering credit risk assessment method of the present invention;

[0022] Figure 2 is an example diagram of the present invention using a mapping conversion function to derive input features;

[0023] Figure 3 is an example diagram of the present invention using an aggregate transformation function to derive input features;

[0024] Figure 4 This is an example diagram of the present invention using a time window method function to derive input features;

[0025] Figure 5 It is a schematic diagram of the process of automatic feature generation of the present invention;

[0026] Figure 6 It is a comparison chart of the ROC performance of the classification model obtained by the method of the present invention and the existing method on the same data set;

[0027] Figure 7It is a comparison chart of the feature generation time between the method of the present invention and the expert experience feature engineering method;

[0028] Figure 8 It is a schematic diagram of the number of features generated and the feature coverage of the method of the present invention and the expert experience feature engineering method. DETAILED DESCRIPTION

[0029] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0030] like Figure 1 As shown, the automated feature engineering credit risk assessment method of the present invention includes the following six steps.

[0031] Step 1: Define samples and construct original features.

[0032] First, understand the characteristics of the customer base, and aim to reduce the credit delinquency rate while ensuring the approval rate. It is necessary to determine the observation period and performance period of the modeling sample, the definition of good and bad samples, and the sampling method. At the same time, construct the original features of the collected customer data and add them to the feature set.

[0033] In the credit business scenario, samples are divided into two categories: good and bad. Good samples are customers with no overdue payments. On the basis of being able to accept a certain degree of overdue payments, in order to increase profits, the standard for good customers can be appropriately relaxed to within 7 days of overdue payments. Bad samples are usually customers with overdue payments of more than 30 days. Depending on the acceptable level of bad debts, they can also be tightened to 15 days of overdue payments. Generally speaking, a complete account period of a customer is required to define good or bad. Most credit performance is based on a 12-month account period, so the performance period is usually 12 months. Determine whether a customer is a good sample or a bad sample based on the credit performance of the customer during the performance period. The observation period varies depending on the constructed features. Usually, in order to ensure the stability of the customer's behavior, the observation period of the constructed features will not be less than 3 months. At the same time, in order to ensure the timeliness of the features, the observation period will not be more than 21 months. Obtain customer data during the observation period to obtain feature data.

[0034] The original features mainly come from the data source of the modeler itself. When a customer submits an application, the credit data of the People's Bank of China, bank flow data, tax information and invoicing information will be retrieved according to the customer's identity information, and the original feature fields will be generated according to the original field names in the retrieved database. The original feature fields are recorded, and the corresponding field data is recorded for each sample.

[0035] For example, the original fields such as credit card ID, credit limit, credit card account opening date, billing date, account status, etc. recorded in the credit investigation data are used as original feature fields. For bank flow data, tax information, and billing information, data within the observation period can be extracted.

[0036] The extracted original features include credit limit, number of credit cards, credit card account opening time, credit granting time, etc.

[0037] Step 2: Design a custom transformation function to derive new feature indicators from the features in the feature set.

[0038] Setting a custom function is to transform the original feature field and change its dimensional distribution. Custom transformation functions include scaling, differential, logarithmic transformation and other calculation operations, or mapping a text feature into a linear feature, or changing the distribution of the original feature and de-dimensionalizing the original feature. These transformations are all for constructing new features.

[0039] In the embodiment of the present invention, the custom conversion functions are mainly divided into three categories, which are mapping conversion functions, aggregation conversion functions, and time window conversion functions according to the working domain mode of the conversion function.

[0040] Mapping transformation functions act on a single feature or multiple features, returning new features as input to a custom function, such as Figure 2 As shown in the figure, after inputting the two features of credit card and number of credit cards into the constructed Gaussian Copula function, feature 1 is obtained, or the mapping function is changed such as logarithmic transformation and differential operation to obtain feature 2, feature 3, etc. In short, the calculation link of the feature by the mapping conversion function occurs on a single feature field column or between multiple feature field columns.

[0041] Aggregate transformation functions act on multiple sets of data for a single original feature field under a user or a group, such as Figure 3 As shown, the minimum credit limit of different credit cards of the same user can be calculated for this set of data, the minimum credit limit of the user's credit card can be obtained, the total credit limit of the user can be obtained by summing this column, and other similar aggregation calculations can also be performed. In short, the calculation of aggregation conversion functions occurs between multiple sets of data of the same user in a single original feature field.

[0042] The time window conversion function uses the concept of sliding time window. It is a conversion function that operates on multiple time windows on the same feature field, such as Figure 4 As shown, for example, for the credit limit feature, we can find out whether the credit limit of users within a period of time should be increased.

[0043] Step three, construct a test evaluation function to test the derived characteristic indicators.

[0044] The test function evaluates the automatically constructed feature index, and the evaluation results of the test function on the feature index are used for diversion. The indexes that pass the test enter the index pool, and the constructed feature indexes that fail the test form new and expanded feature indexes with the original features, and then the feature test is performed on the expanded features. The test functions constructed by the present invention are divided into two categories: correlation detection and causality detection.

[0045] The correlation detection adopts the Copula entropy test. Copula entropy is a theory of measuring the correlation relationship between multiple variables, which corresponds to the theory of correlation relationship representation - Copula function theory. Copula function represents the correlation relationship, and the Copula entropy obtained therefrom measures the amount of information in the relationship. The characteristics of Copula entropy include symmetry, non-negativity, monotonic transformation invariance, and equivalence to the correlation coefficient when Gaussian variables are used. The present invention calculates the new features and the existing features using the Copula function, calculates the Copula entropy, compares it with the preset threshold, verifies the correlation, retains the new features with strong correlation, and the new features that do not meet the threshold requirements will continue to be combined with the existing features for conversion.

[0046] The causality test function adopts the transfer entropy test. Transfer entropy is a nonlinear generalization of causality (Granger Causality), which is equivalent to the conditional mutual information (CMI) of information theory. In essence, it is a test of conditional independence and is model-independent. Therefore, it is applicable to causality tests in any situation. Transfer entropy, as a widely used causal relationship measure, is more scientific and reasonable than other traditional causal relationship inference methods with empirical model assumptions, and has wider universality. The present invention calculates the transfer entropy of new features and existing features, retains new features whose transfer entropy meets the preset threshold requirements, and new features that do not meet the threshold requirements will continue to be combined and converted with existing features.

[0047] New features that pass the Copula entropy test and the transfer entropy test will be added to the feature set, and features that fail the test will be reconverted. Features that pass the Copula entropy test have a strong correlation with the function results, ensuring the final performance of the model, while the transfer entropy test determines the causal relationship of the features, ensuring that the features are interpretable. The automated features that have been double-tested ensure that the credit scorecard model has high predictive power and is also interpretable.

[0048] Step 4: Use the automated feature generation process to generate features and form the final feature set.

[0049] like Figure 5As shown, the present invention inputs the original features collected in step 1 into the conversion function, and uses the test function to perform correlation test on the feature set after the conversion function. The feature set that passes the test is output to the input feature set, and the features that fail the test are returned to step 2, and the conversion function combines the features with the original variables or the features that pass the test, expands the feature set and tests it. In this way, more input features that are easy to predict and interpretable are generated through cyclic derivation. At the same time, the features that have been derived 5 times but have not passed are retained, and these features can be analyzed using expert experience.

[0050] Step 5: Select the detection model.

[0051] New features are generated through the automated feature generation project in step 4, and are input into a common machine learning model for training together with the labels of the transaction records to obtain a fraud detection model. In the field of Internet finance, credit risk models or anti-fraud models usually need to be interpretable to provide a basis for decision-making. Therefore, the model in the embodiment of the present invention uses the logistic regression model most commonly used by traditional financial institutions.

[0052] Step 6: Extract features from the collected data based on the feature set in step 4 to generate sample data. Use the training data to train the detection model in step 5. When applying, collect customer data, extract the feature set in step 4, and then input the trained detection model to classify the customer's credit.

[0053] Correspondingly, the automated feature engineering credit risk assessment system provided by the present invention includes an original feature extraction module, a new feature derivation module, a new feature verification module, a feature automatic generation module and a customer credit risk classification model.

[0054] The original feature extraction module is used to extract database fields from the collected customer data, obtain original features and store them in the feature set. The specific implementation of the original feature extraction module is recorded in step 1 and will not be repeated here.

[0055] The new feature derivation module is used to derive new features from the features in the feature set according to the custom conversion function. The new feature derivation method and the description of the custom conversion function are recorded in step 2 and will not be repeated here.

[0056] The new feature test module performs correlation test and causality test on the new feature; among them, Copula entropy is used for correlation test, and transfer entropy is used for causality test. The description of feature test is recorded in step 3 and will not be repeated here.

[0057] The feature automatic generation module is used to call the new feature derivation module and the new feature verification module to automatically generate new features. The new features that pass the correlation test and causality test are stored in the feature set. For new features that fail the test, the new feature derivation model is combined with the features in the feature set to derive new features, which are then tested using the new feature verification module.

[0058] The customer credit risk classification model uses a logistic regression model to classify customer credit risks. The input of the model is user data obtained based on a feature set, and the output is the customer credit risk classification result.

[0059] In the embodiment of the present invention, the automatic feature extraction engineering of the present invention is applied to the traditional manual expert experience feature engineering and the original feature engineering on the same data set to compare the customer credit card risk classification. The three methods used in the comparison all use the logistic regression classification model as the customer credit classification model. The ROC performance of the classification models obtained by the three methods is as follows: Figure 6 As shown, the AUC of the ROC curve of the method of the present invention is 0.86, the AUC of the ROC curve of the classification model of artificial expert experience feature engineering is 0.86, and the AUC of the ROC curve of the classification model based on original feature engineering is 0.65. It can be seen that the model performance obtained by the present invention is the best. Although the performance of the classification model of artificial expert experience feature engineering is not much different from that of the model of the present invention, the method of the present invention has more advantages in working time. Figure 7 As shown in Figure 2, the time required for constructing features using different conversion functions in the present invention is 2 hours, which is much more efficient than the 8 hours or even several days of feature engineering time required by experts. In addition, the present invention automatically generates 682 features, such as Figure 8 As shown, 80 of the 120 features constructed by the expert experience feature engineering method can be found in the features automatically generated by the present invention, with a coverage rate of 66%, indicating that the model has causal relationship and solvability while ensuring efficiency and performance.

[0060] Except for the technical features described in the specification, all of them are known technologies to those skilled in the art. The present invention omits the description of the known technologies to avoid redundant description and unnecessary limitation of the present invention. The implementation methods described in the above embodiments do not represent all implementation methods consistent with the present application. On the basis of the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative labor are still within the protection scope of the present invention.

Claims

1. An automated feature engineering credit risk assessment method, characterized in that: The steps include: Step 1: Collect customer data, extract original features and add them to the feature set; Each customer corresponds to a sample, and the sample generation method is set; With the goal of reducing the credit delinquency rate, the sample generation method is set, including: determining the observation period and performance period of the sample, and setting up two types of samples: good and bad. Good samples are customers with no overdue payments and customers who are overdue within 7 days; bad samples are customers who are overdue for more than 15 days. The observation period is set to be at least 3 months and at most 21 months; the performance period is set to 12 months; whether a customer is a good or bad sample is determined based on the customer's credit performance during the performance period; customer data is obtained from the customer during the observation period; the original features extracted include credit limit, number of credit cards, credit card account opening time, and credit granting time. Step 2: Construct a custom transformation function to derive new features based on the features of the feature set; The custom conversion function is divided into three categories according to different working domains: mapping conversion function, aggregation conversion function, and time window conversion function; wherein the mapping conversion function inputs a single feature or multiple features into a custom mapping function to generate a new feature; the mapping function includes a logarithmic transformation function, a differential operation function, and a Gaussian Copula function; the aggregation conversion function performs aggregation calculations on multiple sets of data of a single feature to generate a new feature; the time window conversion function uses the concept of a sliding time window to operate on the data of a single feature in multiple time windows; Among them, the new features derived include: inputting the credit card limit and the number of credit cards into the constructed Gaussian Copula function to obtain feature 1, inputting the logarithmic transformation function to obtain feature 2, and inputting the difference operation function to obtain feature 3; finding the minimum value and total credit limit of different credit card limits for the same user; and finding the credit limit increase of a user over a period of time; Step 3: construct a test function to test the derived features; The test includes correlation test and causality test. Copula entropy is used for correlation test, and transfer entropy is used for causality test. If the new feature passes the Copula entropy and transfer entropy tests, it is added to the feature set. Otherwise, the new feature is combined with the features in the feature set and the custom conversion function in step 2 is reused for derivation. Among them, the new features and existing features are calculated using the Copula function, and the calculated Copula entropy is compared with the preset threshold to test the correlation. New features with strong correlation are retained, and new features that do not meet the threshold requirements will continue to be combined and transformed with existing features; the transfer entropy of the new features and existing features is calculated, and new features whose transfer entropy meets the preset threshold requirements are retained, and new features that do not meet the threshold requirements will continue to be combined and transformed with existing features; Step 4: Generate a final feature set, extract feature data from the customer data according to the feature set, and generate samples; Step 5: Select a logistic regression model as the customer credit risk classification model and train the model; when evaluating customer credit risk, extract customer data according to the final feature set, input the trained model, and obtain the customer credit risk classification result.

2. The method according to claim 1, characterized in that In the step 1, the collected customer data includes the People's Bank of China credit information, bank statements, tax information and invoicing information, and original features are generated based on the original fields in the database recording the customer data.

3. An automated feature engineering credit risk assessment system, characterized by: include: The original feature extraction module is used to extract database fields from the collected customer data, obtain original features and store them in the feature set; With the goal of reducing the credit delinquency rate, the sample generation method is set, including: determining the observation period and performance period of the sample, and setting up two types of samples: good and bad. Good samples are customers with no overdue payments and customers who are overdue within 7 days; bad samples are customers who are overdue for more than 15 days. The observation period is set to be at least 3 months and at most 21 months; the performance period is set to 12 months; whether a customer is a good or bad sample is determined based on the customer's credit performance during the performance period; customer data is obtained from the customer during the observation period; the original features extracted include credit limit, number of credit cards, credit card account opening time, and credit granting time. A new feature derivation module is used to derive new features from features in a feature set based on a custom conversion function; the custom conversion functions stored in the new feature derivation module are divided into three categories according to different working domains: mapping conversion functions, aggregation conversion functions, and time window conversion functions; wherein, the mapping conversion function inputs a single feature or multiple features into a custom mapping function to generate a new feature; the mapping function includes a logarithmic transformation function, a differential operation function, and a Gaussian Copula function; the aggregation conversion function performs aggregation calculations on multiple sets of data of a single feature to generate a new feature; the time window conversion function uses the concept of a sliding time window to operate on the data of a single feature in multiple time windows; wherein, deriving new features includes: inputting the two features of credit card limit and number of credit cards into the constructed Gaussian Copula function to obtain feature 1, inputting the logarithmic transformation function to obtain feature 2, and inputting the differential operation function to obtain feature 3; finding the minimum value and total credit limit of different credit card limits of the same user; finding the credit limit increased by the user within a period of time; The new feature testing module performs correlation and causality tests on new features. Copula entropy is used for correlation testing, and transfer entropy is used for causality testing. Copula functions are used to calculate new features and existing features, and the calculated Copula entropy is compared with the preset threshold to test the correlation. New features with strong correlation are retained, and new features that do not meet the threshold requirements will continue to be combined and transformed with existing features. Transfer entropy is calculated for new features and existing features, and new features whose transfer entropy meets the preset threshold requirements are retained. New features that do not meet the threshold requirements will continue to be combined and transformed with existing features. The feature automatic generation module is used to call the new feature derivation module and the new feature verification module to automatically generate new features. The new features that pass the correlation test and causality test are stored in the feature set. For new features that fail the test, the new feature derivation model is used to derive new features by combining the features in the feature set, and then the new features are tested by the new feature verification module. The customer credit risk classification model uses a logistic regression model to classify customer credit risks. The input of the model is a user data sample obtained based on a feature set, and the output is the customer credit risk classification result.

4. The system according to claim 3, characterized in that The customer data collected by the original feature extraction module includes credit information from the People's Bank of China, bank statements, tax information and invoicing information.

Citation Information

Patent Citations

  • Hybrid machine learning credit scoring model building method

    CN106897918A

  • Online network payment fraud detection system based on automatic feature engineering

    CN111275447A

  • Voltage transformer error prediction method based on transfer entropy and wavelet neural network

    CN111814390A

  • Credit card issuing model and application method thereof

    CN111951099A

  • Fall-down risk assessment method and device

    CN112120703A