Enterprise credit scoring model modeling method and system

Abstract

The invention particularly relates to an enterprise credit scoring model modeling method and system. The enterprise credit scoring model modeling method comprises the steps of obtaining data, normalizing the data, filtering out abnormal values, performing variable correlation test, and performing standardization processing on variables; preliminarily analyzing and processing unbalanced sample data; screening important features, sorting based on XGBOOST feature importance, identifying category features, realizing category feature coding, expanding to generate new combined features, analyzing feature correlation, and reducing the number of the features to a specified dimension; constructing a base model, searching for optimal model parameters, constructing a hierarchical structure model, determining features and weights of all hierarchies, performing performance evaluation on the model, and continuously performing optimization and improvement. According to the enterprise credit scoring model modeling method and system, the evaluation accuracy of the model on the enterprise credit risk is improved, the over-fitting risk of the model is reduced, the interpretability and credibility of the model are improved, and continuous optimization and iteration of the model are realized.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a method and system for building an enterprise credit scoring model. Background Art

[0002] With the rapid development of my country's industrial system and financial technology, financial institutions have put forward higher requirements for corporate credit rating.

[0003] Traditional single modeling methods have limitations. For example, the use of a single expert modeling method may be affected by subjective factors, resulting in biased results. Therefore, a corporate credit scoring modeling method based on a combination of multiple algorithms, machine learning, and expert experience is particularly important.

[0004] Based on the above problems, the present invention proposes a method and system for building an enterprise credit scoring model.

[0005] This technical solution aims to develop an efficient and accurate enterprise credit scoring model based on machine learning, integrating expert experience, and achieving accurate assessment of enterprise credit risk through a systematic process including data preprocessing, sample processing, feature selection and derivation, model building, and model evaluation. This model can be widely used in finance, credit, insurance and other fields to provide enterprises with credit rating, risk assessment and other services. Summary of the invention

[0006] In order to make up for the deficiencies of the prior art, the present invention provides a simple and efficient enterprise credit scoring model building method and system.

[0007] The present invention is achieved through the following technical solutions:

[0008] A method for building a corporate credit scoring model, characterized in that it includes the following steps:

[0009] Step S1: Data preprocessing

[0010] Get data: Get enterprise-related financial data, operating data, and industry data from custom data sources;

[0011] Data normalization: clean the raw data, remove duplicate data, unify the data format, and perform preliminary processing on invalid data including missing values, null values ​​and outliers;

[0012] Outlier filtering: Use statistical methods to identify and filter out outliers in the data;

[0013] Variable correlation test: Use Pearson correlation coefficient or Spearman rank correlation coefficient to quantify the correlation between variables, and use visualization methods to intuitively display the correlation between variables; use correlation analysis to identify and remove highly correlated variables, and use recursive feature elimination (RFE) or model-based feature selection to automatically select custom important variables to reduce redundancy and improve the generalization ability of the model;

[0014] Variable standardization: Customize the standardization method and standardization tool to standardize the variables so that they have a unified dimension and distribution to improve the training efficiency and accuracy of the model;

[0015] In step S1, a box plot, a Z-score statistical method or an interquartile range (IQR) is used to identify outliers in the data; a visualization method is used to intuitively display the distribution of the data so that users can identify outliers; and a data analysis software is used as a filtering tool to automatically filter out outliers.

[0016] In the step S1, a Z-score standardization, a Min-Max standardization, a decimal calibration standardization or a logarithmic transformation method is selected by a user, and the Python NumPy library, the Pandas library or the scikit-learn library is used as a standardization tool to standardize the variables.

[0017] Step S2: Sample processing

[0018] Sample analysis: Conduct preliminary analysis on sample data, use charts to visually display the distribution of data, calculate and count the mean, standard deviation, median and quartile of the statistical data to quantify the distribution characteristics of the data; analyze the number and proportion of samples of different categories to determine whether the samples are balanced;

[0019] Sample balancing: Use oversampling, undersampling or synthetic minority oversampling technology (SMOTE) to process unbalanced sample data;

[0020] Missing value filling: fill missing values ​​using interpolation, mean filling or model prediction methods;

[0021] Sample filtering: Identify and delete duplicate data records by comparing data IDs or timestamps, identify and delete abnormal samples using box plots, Z-score statistics, or isolation forest machine learning algorithms, and delete samples with missing values ​​exceeding a custom threshold.

[0022] In step S2, random oversampling and synthetic minority class oversampling techniques are used to make the ratio of positive and negative samples close by copying or generating minority class samples;

[0023] Or use random undersampling and cluster undersampling to reduce the number of majority class samples by deleting or merging majority class samples, so that the ratio of positive and negative samples is close;

[0024] Alternatively, oversampling and undersampling methods can be combined to achieve sample balancing.

[0025] In step S2, a linear interpolation method or a polynomial interpolation method is used to calculate the missing value through an interpolation formula according to the value of the known data point;

[0026] Or fill missing values ​​with the average value of the feature where the missing value is located;

[0027] For skewed distribution data, fill the missing values ​​with the median of the feature where the missing value is located;

[0028] For discrete features, the missing values ​​are filled with the mode of the feature where the missing value is located;

[0029] Or use regression models or classification models to predict and fill missing values.

[0030] Step S3: Feature selection and derivation

[0031] Feature filtering: Calculate the importance or relevance of features based on statistical tests and machine learning algorithms, and filter out the top M important features that exceed the threshold or are ranked according to the custom threshold or ranking results from the original feature set, and verify the impact of the filtered feature set on the model performance to ensure that no important information is lost, reduce noise and redundancy, and improve the generalization ability of the model;

[0032] Sorting based on XGBOOST feature importance: Configure the parameters of the XGBOOST model, train the model using the XGBOOST algorithm, and obtain the importance scores of the features; sort the features according to the scores and select the top N important features;

[0033] Category feature encoding: Identify the category features in the data set, encode the category features using the one-hot encoding method or the label encoding method, and generate numerical features;

[0034] Feature combination: Identify feature pairs or feature groups with nonlinear relationships, perform Cartesian product operations on two or more features to generate new combined features, or generate combined features through polynomial expansion, and verify the impact of combined features on model performance;

[0035] Feature correlation analysis: Calculate the correlation coefficient matrix between features, identify feature pairs whose correlation coefficients exceed a custom threshold, and choose to retain or delete redundant features to ensure that there is no high correlation between features;

[0036] Feature dimensionality reduction uses principal component analysis (PCA) to perform linear transformation, project the data into a low-dimensional space, and retain the maximum variance information. Alternatively, linear discriminant analysis (LDA) is used to maximize the distance between classes and minimize the distance within classes to find the optimal projection direction. Alternatively, tree-based feature selection methods are used to sort the importance of features in random forests and select features with higher importance. Dimensionality reduction methods are applied to reduce the number of features to the specified dimension, and the impact of the reduced feature set on the model performance is verified to ensure that no important information is lost.

[0037] In step S3, during statistical testing, a chi-square test is used for classification problems, and a t-test is used for regression problems, and features are screened by testing the statistical correlation between features and target variables;

[0038] Random forest is used to customize the screening features by calculating the importance scores of the features; Lasso regression automatically selects important features through L1 regularization and compresses the unimportant feature coefficients to zero.

[0039] In step S3, the Pearson correlation coefficient or the Spearman rank correlation coefficient is used to calculate the correlation coefficient between the features, and a visualization method (such as a heat map, a correlation matrix diagram, etc.) is used to intuitively display the correlation between the features.

[0040] Step S4: Model establishment

[0041] Training set partitioning: custom partitioning of the data set into training set, validation set, and test set;

[0042] Ensemble learning: Use random forest, gradient boosting tree GBDT or XGBOOST ensemble learning algorithms to build multiple base models, and combine the prediction results of each base model to obtain the final prediction result;

[0043] Parameter optimization: Determine the parameter range to be optimized, use grid search, random search or Bayesian optimization method to find the optimal model parameters, and record the optimal parameter combination;

[0044] AHP expert experience modeling: Use the analytic hierarchy process (AHP) to build a hierarchical model, determine the characteristics and weights of each level, use expert experience to adjust and optimize the hierarchical model, and train and verify the adjusted model to evaluate model performance;

[0045] Verification and evaluation: Customize the accuracy, recall or F1 score as the evaluation indicator, use the validation set to evaluate the performance of the model, and record the evaluation results; optimize and adjust the model based on the evaluation results, and perform cross-validation to further verify the stability and reliability of the model;

[0046] In step S4, for the binary classification problem, the AUC value is used to evaluate the performance of the model at different thresholds.

[0047] Step S5: Model evaluation

[0048] Calculate the AUC value: Calculate the AUC value of the model to evaluate the classification performance of the model;

[0049] KS model evaluation: Calculate the KS statistic to evaluate the model's ability to distinguish at different thresholds;

[0050] Feature PSI monitoring: monitor feature stability, calculate the difference between feature distribution in the training set and the validation set (or test set) to obtain the PSI value, and use the PSI value to evaluate the change in feature distribution. The larger the PSI value, the greater the change in feature distribution. When the PSI value exceeds the custom threshold, the user is notified to make adjustments.

[0051] Model expert review: Invite experts to review the business logic, defects, and interpretability of the model, and optimize and improve the model based on the experts' improvement suggestions;

[0052] Continuous model optimization: collect evaluation results and expert suggestions in real time, analyze the problems and deficiencies of the model, determine the optimization direction and implementation plan, iterate and optimize the model, and re-evaluate the model performance after iteration and optimization to verify the optimization effect.

[0053] A corporate credit scoring model building system, comprising:

[0054] The data preprocessing module is responsible for obtaining data from custom data sources, normalizing the data, filtering out outliers, performing variable correlation tests, and standardizing the variables to give them a uniform dimension and distribution.

[0055] The sample processing module is responsible for preliminary analysis of sample data, processing unbalanced sample data, filling missing values, filtering out duplicate data and samples with missing values ​​exceeding a custom threshold;

[0056] The feature selection and derivation module is responsible for screening important features, sorting the feature importance based on XGBOOST, identifying the category features in the data set, implementing category feature encoding, expanding feature pairs or feature groups with nonlinear relationships, generating new combined features, analyzing feature correlation, and reducing the number of features to the specified dimension;

[0057] The model building module is responsible for dividing the training set, using the integrated learning algorithm to build the base model, finding the optimal model parameters, building the hierarchical model, determining the characteristics and weights of each level, and adjusting and optimizing the hierarchical model using expert experience; using the validation set to evaluate, optimize and adjust the model performance;

[0058] The model evaluation module is responsible for calculating the AUC value of the model and evaluating the classification performance of the model; calculating the KS statistic and evaluating the model's ability to distinguish under different thresholds; monitoring feature stability, evaluating changes in feature distribution through PSI values, inviting experts to review the model, collecting evaluation results and expert suggestions in real time, and continuously optimizing and improving the model based on the evaluation results and expert improvement suggestions.

[0059] The beneficial effects of the enterprise credit scoring modeling method and system of the present invention are:

[0060] Improved assessment accuracy: By combining multiple algorithms and supplementing expert experience, the model's assessment accuracy of corporate credit risk has been improved;

[0061] Enhanced model stability: The feature selection and derivation process can ensure the validity and stability of model features and reduce the risk of model overfitting;

[0062] Improved model interpretability: The Analytic Hierarchy Process (AHP) combined with expert experience improved the interpretability and credibility of the model;

[0063] Continuous optimization was achieved: Through feature PSI monitoring and model expert review, continuous optimization and iteration of the model was achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0065] Attached Figure 1 It is a schematic diagram of the modeling method of the enterprise credit scoring model of the present invention. DETAILED DESCRIPTION

[0066] In order to make the technical solutions in the present invention better understood by the persons skilled in the art, the technical solutions in the embodiments of the present invention will be described clearly and completely in combination with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments.

[0067] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative work should fall within the scope of protection of the present invention.

[0068] The enterprise credit scoring model modeling method comprises the following steps:

[0069] Step S1: Data preprocessing

[0070] Data preprocessing is a crucial step in fields such as machine learning, data analysis, and data mining. It is directly related to the effectiveness and accuracy of subsequent model training.

[0071] Acquire data: Obtain enterprise-related financial data, operating data, and industry data from custom data sources; the comprehensiveness, accuracy, and timeliness of the data are crucial for subsequent analysis and decision-making. After acquiring the data, conduct a preliminary assessment of the data's completeness, accuracy, consistency, and timeliness.

[0072] Data normalization: clean the raw data, remove duplicate data, unify the data format, and perform preliminary processing on invalid data including missing values, null values ​​and outliers;

[0073] In order to ensure the consistency and comparability of data, eliminate redundancy and contradictions in the data, and provide a reliable basis for subsequent analysis and modeling, the steps for implementing data normalization in step S1 are as follows:

[0074] Step S101, removing duplicate data: identifying and deleting duplicate data records by comparing the unique identifiers of the data (such as ID, timestamp, etc.);

[0075] Step S102, processing invalid data: identifying and processing invalid data including missing values, null values ​​and abnormal values;

[0076] Missing values ​​were handled using mean filling, median filling, interpolation or deletion methods;

[0077] Step S103, unifying data formats: converting data into a unified custom format, such as a date format, a numerical format, a text format, etc., to ensure the consistency and comparability of the data.

[0078] Outlier filtering: Use statistical methods to identify and filter out outliers in the data;

[0079] Variable correlation test: Highly correlated variables will lead to overfitting of the model, reducing the accuracy and stability of the model. Use the Pearson correlation coefficient or Spearman rank correlation coefficient to quantify the correlation between variables, and use visualization methods such as heat maps and correlation matrix diagrams to intuitively display the correlation between variables; use correlation analysis to identify and remove highly correlated variables, and use recursive feature elimination (RFE) or model-based feature selection (such as random forest, gradient boosting tree, etc.) to automatically select custom important variables to reduce redundancy and improve the generalization ability of the model;

[0080] Variable standardization: Customize the standardization method and standardization tool to standardize the variables so that they have a unified dimension and distribution to improve the training efficiency and accuracy of the model;

[0081] Outliers refer to data points that are significantly different from most data points. They may be caused by data entry errors, measurement errors, or extreme events. The presence of outliers will have a negative impact on the training of the model, so they need to be filtered. In step S1, a box plot, a Z-score statistical method, or an interquartile range (IQR) is used to identify outliers in the data; a visualization method, such as a scatter plot, a histogram, etc., is used to intuitively display the distribution of the data so that users can identify outliers; and a data analysis software is used as a filtering tool to automatically filter out outliers.

[0082] In the step S1, a Z-score standardization, a Min-Max standardization, a decimal calibration standardization or a logarithmic transformation method is selected by a user, and the Python NumPy library, the Pandas library or the scikit-learn library is used as a standardization tool to standardize the variables.

[0083] Z-score standardization: convert the variable into a normal distribution with a mean of 0 and a standard deviation of 1;

[0084] Min-Max Normalization: Transform the variable to range between 0 and 1.

[0085] Other standardization methods: decimal calibration standardization, logarithmic transformation, etc. Choose the appropriate standardization method according to the specific situation of the data.

[0086] Step S2: Sample processing

[0087] Sample analysis: Conduct preliminary analysis on sample data, use charts (such as histograms, scatter plots, box plots, etc.) to intuitively display the distribution of data, calculate and count the mean, standard deviation, median and quartile of the statistical data to quantify the distribution characteristics of the data; analyze the number and proportion of samples of different categories to determine whether the samples are balanced; provide important basis for subsequent data preprocessing, feature selection and model training;

[0088] Sample balancing: Use oversampling, undersampling or synthetic minority oversampling technology (SMOTE) to process unbalanced sample data;

[0089] Missing value filling: fill missing values ​​using interpolation, mean filling or model prediction methods;

[0090] Sample filtering: Identify and delete duplicate data records by comparing data IDs or timestamps, identify and delete abnormal samples using box plots, Z-score statistics, or isolation forest machine learning algorithms, and delete samples with missing values ​​exceeding a custom threshold.

[0091] In practical applications, samples are often unbalanced, that is, the number of samples in some categories is much larger than that in other categories. This imbalance will cause the model to favor the category with a larger number during training, thereby reducing the classification performance of the model. Therefore, it is necessary to process the unbalanced sample data to improve the generalization ability of the model.

[0092] In step S2, random oversampling and synthetic minority oversampling technique (SMOTE) are used to copy or generate minority class samples so that the ratio of positive and negative samples is close;

[0093] Or use random undersampling and cluster undersampling to reduce the number of majority class samples by deleting or merging majority class samples, so that the ratio of positive and negative samples is close;

[0094] Alternatively, oversampling and undersampling methods can be combined to achieve better sample balancing effects.

[0095] During the data collection process, there may be missing values ​​in the data due to various reasons (such as equipment failure, data entry errors, etc.). The existence of missing values ​​will affect the training effect of the model, so the missing values ​​need to be filled.

[0096] In step S2, a linear interpolation method or a polynomial interpolation method is used to calculate the missing value through an interpolation formula according to the value of the known data point;

[0097] Alternatively, fill missing values ​​with the average value of the feature where the missing value lies; this method is simple and easy, but may introduce bias.

[0098] For skewed distribution data, fill the missing values ​​with the median of the feature where the missing value is located;

[0099] For discrete features, the missing values ​​are filled with the mode of the feature where the missing value is located;

[0100] Alternatively, a regression model or a classification model can be used to predict and fill in missing values. This method can make full use of the information in the data and improve the accuracy of filling.

[0101] During the data collection process, some invalid or low-quality samples may be generated, such as duplicate samples, abnormal samples, or samples with too many missing values. These samples will have a negative impact on the training of the model, so these samples need to be removed to improve data quality.

[0102] Step S3: Feature selection and derivation

[0103] Feature filtering: Calculate the importance or relevance of features based on statistical tests and machine learning algorithms, and filter out the top M important features that exceed the threshold or are ranked according to the custom threshold or ranking results from the original feature set, and verify the impact of the filtered feature set on the model performance to ensure that no important information is lost, reduce noise and redundancy, and improve the generalization ability of the model;

[0104] Sorting based on XGBOOST feature importance: Configure the parameters of the XGBOOST model, such as learning rate, maximum depth, etc. Use the XGBOOST algorithm to train the model and obtain the importance scores of the features; sort the features according to the scores and select the top N important features;

[0105] Category feature encoding: Identify the category features in the data set, encode the category features using the One-Hot Encoding method or the Label Encoding method, and generate numerical features;

[0106] Categorical features (such as gender, color, etc.) are usually represented in text form and cannot be directly used as model input. Encoding is the process of converting these text features into numerical features so that they can be processed by machine learning algorithms.

[0107] One-Hot Encoding: Create a new binary feature for each category. If the sample belongs to that category, the corresponding feature is 1, otherwise it is 0.

[0108] Label Encoding: Mapping categories to integers, but this method may introduce order relationships and is not suitable for unordered categories.

[0109] Feature combination: Identify feature pairs or feature groups with nonlinear relationships, perform Cartesian product operations on two or more features to generate new combined features, or generate combined features through polynomial expansion, such as square terms, cross terms, etc. And verify the impact of combined features on model performance;

[0110] Feature combination generates new features by combining multiple features together, which can capture the nonlinear relationship between features and improve the expressiveness of the model.

[0111] Feature correlation analysis: Calculate the correlation coefficient matrix between features, identify feature pairs whose correlation coefficients exceed a custom threshold, and choose to retain or delete redundant features to ensure that there is no high correlation between features, identify redundant relationships between features, and avoid the negative impact of redundant features on model performance.

[0112] Feature dimensionality reduction: aims to reduce the number of features and computational complexity while retaining as much information as possible in the original data. Use the principal component analysis (PCA) method for linear transformation to project the data into a low-dimensional space and retain the maximum variance information, or use the linear discriminant analysis (LDA) method to maximize the distance between classes and minimize the distance within classes to find the optimal projection direction, or use the tree-based feature selection method to sort the importance of features in the random forest and select features with higher importance by custom selection; apply the dimensionality reduction method to reduce the number of features to the specified dimension, and verify the impact of the reduced feature set on the model performance to ensure that no important information is lost;

[0113] In step S3, during statistical testing, a chi-square test is used for classification problems, and a t-test is used for regression problems, and features are screened by testing the statistical correlation between features and target variables;

[0114] Random forest is used to customize the screening features by calculating the importance scores of the features; Lasso regression automatically selects important features through L1 regularization and compresses the unimportant feature coefficients to zero.

[0115] In step S3, the Pearson correlation coefficient or the Spearman rank correlation coefficient is used to calculate the correlation coefficient between the features, and a visualization method (such as a heat map, a correlation matrix diagram, etc.) is used to intuitively display the correlation between the features.

[0116] Step S4: Model establishment

[0117] Training set division: Customize the division of the data set into training set, validation set, and test set; to ensure the generalization ability of the model on different data sets, help prevent model overfitting, and improve the practicality and reliability of the model.

[0118] Ensemble learning: Use random forest, gradient boosted tree (GBDT) or XGBOOST ensemble learning algorithms to build multiple base models (such as decision trees), combine the prediction results of each base model, and get the final prediction result; improve the accuracy and stability of the model by combining the prediction results of multiple models. This method can reduce the bias and variance of a single model and improve the generalization ability of the model.

[0119] Random Forest: Improves the accuracy of the model by building multiple decision trees and combining their prediction results. Random Forest has strong anti-overfitting capabilities and is suitable for high-dimensional data and complex problems.

[0120] Gradient boosted tree (GBDT): Improves performance by gradually building multiple weak classifiers and continuously optimizing the model. GBDT can handle nonlinear data and missing values ​​and is suitable for classification and regression problems.

[0121] XGBOOST: An efficient gradient boosting algorithm that improves the performance of the model by optimizing the loss function and regularization term. XGBOOST supports a variety of loss functions and regularization methods and is suitable for large-scale data and complex problems.

[0122] Parameter optimization: Determine the range of parameters to be optimized, use grid search, random search or Bayesian optimization method to find the optimal model parameters, and record the optimal parameter combination; improve the performance of the model by adjusting the parameters of the model. Reasonable parameter settings can significantly improve the accuracy and stability of the model.

[0123] Grid search: Find the optimal parameters by traversing all possible parameter combinations. This method is comprehensive but computationally intensive and is suitable for situations with a small parameter range.

[0124] Random search: Randomly select a set of parameters in the parameter space for training and keep the best result. This method has a small amount of computation, but may miss the optimal solution.

[0125] Bayesian optimization: Use Bayes’ theorem to find the optimal solution in parameter space. This method can use historical information to guide the search process and improve search efficiency.

[0126] AHP expert experience modeling: Use the analytic hierarchy process (AHP) to build a hierarchical model, determine the characteristics and weights of each level, use expert experience to adjust and optimize the hierarchical model, and train and verify the adjusted model to evaluate model performance; in order to improve the interpretability and credibility of the model.

[0127] Analytic Hierarchy Process (AHP): The importance of each feature is evaluated by constructing a hierarchical model, a judgment matrix, and calculating weights. The AHP method can systematically handle complex decision-making problems.

[0128] Expert experience: Based on the expert’s in-depth understanding of the business and industry experience, the model parameters and features are appropriately adjusted. Expert experience can make up for the model’s shortcomings in data-driven aspects.

[0129] Verification and evaluation: Customize the accuracy, recall or F1 score as the evaluation indicator, use the validation set to evaluate the performance of the model, and record the evaluation results; optimize and adjust the model based on the evaluation results, and perform cross-validation to further verify the stability and reliability of the model;

[0130] Accuracy: This measures the proportion of correct predictions made by the evaluation model.

[0131] Recall: evaluates the proportion of correct predictions made by the model among all positive examples.

[0132] F1 score: An indicator that combines precision and recall, used to evaluate the balanced performance of the model.

[0133] Cross-validation: Divide the dataset into multiple subsets and perform training and validation alternately to evaluate the stability and reliability of the model.

[0134] In step S4, the division ratio is determined, and the data set is divided into a training set, a validation set, and a test set by random division, hierarchical division, or time division, and the rationality and balance of the division result are verified.

[0135] Random Split: Randomly divide the dataset into three parts, usually in the ratio of 70% training set, 15% validation set and 15% test set. This method is simple and easy to implement, but it may be affected by randomness.

[0136] Stratified partitioning: Based on the distribution of the target variable, the data is partitioned in each category to ensure that the proportion of each category in the training set, validation set, and test set is consistent. This method helps to maintain the balance of the data.

[0137] Time partitioning: For time series data, you can partition it in chronological order, such as using the first 70% of the data as the training set, the next 15% as the validation set, and the last 15% as the test set. This method helps evaluate the model's generalization ability over time.

[0138] In step S4, for the binary classification problem, the AUC value is used to evaluate the performance of the model at different thresholds.

[0139] Step S5: Model evaluation

[0140] Calculate the AUC value: Calculate the AUC value of the model to evaluate the classification performance of the model;

[0141] The AUC (Area Under the Curve) value, that is, the area under the receiver operating characteristic curve (ROC Curve), is an important indicator for evaluating the performance of a binary classification model. It comprehensively considers the classification performance of the model under different thresholds and can reflect the model's ability to distinguish between positive and negative samples.

[0142] Calculation method:

[0143] The AUC value is usually obtained by calculating the area under the ROC curve. The ROC curve is a curve drawn with the false positive rate (FPR) as the horizontal axis and the true positive rate (TPR) as the vertical axis. The AUC value ranges from 0 to 1. The larger the value, the better the classification performance of the model.

[0144] Application and interpretation:

[0145] When the AUC value is close to 1, it means that the model can distinguish positive and negative samples well and has excellent classification performance.

[0146] When the AUC value is close to 0.5, it means that the model has almost no discrimination ability, which is equivalent to random guessing.

[0147] The AUC value can also be used to compare the performance of different models and select the optimal model.

[0148] KS model evaluation: Calculate the KS statistic to evaluate the model's ability to distinguish at different thresholds;

[0149] KS (Kolmogorov-Smirnov) statistic is used to evaluate the model's ability to distinguish at different thresholds. It measures the maximum difference between the probability distribution of the model's predictions of positive samples and the probability distribution of the model's predictions of negative samples.

[0150] Calculation method:

[0151] The KS value is obtained by calculating the maximum value of the difference between TPR and FPR, that is, KS = max(TPR-FPR).

[0152] Application and interpretation:

[0153] The larger the KS value, the stronger the model's ability to distinguish is, and the more accurate the prediction of positive and negative samples is.

[0154] The KS value can also be used to screen the optimal threshold, that is, classification is performed at the threshold with the largest KS value to obtain the best classification effect.

[0155] In fields such as financial risk control, KS value is often used as an important reference indicator for model approval.

[0156] Feature PSI monitoring: monitor feature stability, calculate the difference in feature distribution between the training set and the validation set (or test set) to obtain the PSI value, and use the PSI value to evaluate changes in feature distribution. The larger the PSI value, the greater the change in feature distribution, which may affect model performance. When the PSI value exceeds the custom threshold, it is necessary to further check the cause of the feature change and notify the user to take appropriate measures to make adjustments. This allows for timely discovery of feature stability issues and avoids degradation of model performance.

[0157] Feature PSI monitoring is an important part of model monitoring and risk management. Methods such as chi-square test or KL divergence are usually used to measure distribution differences.

[0158] Model expert review: Invite experts to review the business logic, defects and interpretability of the model, and optimize and improve the model according to the experts' improvement suggestions to improve the rationality and accuracy of the model;

[0159] Review content:

[0160] Whether the business logic of the model is reasonable and meets business requirements.

[0161] How accurate is the model and whether there is overfitting or underfitting.

[0162] How interpretable is the model? Can the model’s predictions be clearly explained?

[0163] Continuous model optimization: Collect evaluation results and expert advice in real time, analyze the problems and deficiencies of the model, determine the optimization direction and implementation plan, iterate and optimize the model, and re-evaluate the model performance after iteration and optimization to verify the optimization effect. This will continuously improve the accuracy and stability of the model to adapt to the ever-changing data and business needs.

[0164] Optimization direction:

[0165] Adjust model parameters: According to the performance evaluation results, adjust the model's hyperparameters, such as learning rate, number of iterations, etc.

[0166] Add or delete features: Add new valid features or delete redundant features based on feature importance analysis and feature PSI monitoring results.

[0167] Improve the algorithm: Try using new machine learning algorithms or ensemble learning methods to improve the performance of the model.

[0168] The enterprise credit scoring model modeling system includes:

[0169] The data preprocessing module is responsible for obtaining data from custom data sources, normalizing the data, filtering out outliers, performing variable correlation tests, and standardizing the variables to give them a uniform dimension and distribution.

[0170] The sample processing module is responsible for preliminary analysis of sample data, processing unbalanced sample data, filling missing values, filtering out duplicate data and samples with missing values ​​exceeding a custom threshold;

[0171] The feature selection and derivation module is responsible for screening important features, sorting the feature importance based on XGBOOST, identifying the category features in the data set, implementing category feature encoding, expanding feature pairs or feature groups with nonlinear relationships, generating new combined features, analyzing feature correlation, and reducing the number of features to the specified dimension;

[0172] The model building module is responsible for dividing the training set, using the integrated learning algorithm to build the base model, finding the optimal model parameters, building the hierarchical model, determining the characteristics and weights of each level, and adjusting and optimizing the hierarchical model using expert experience; using the validation set to evaluate, optimize and adjust the model performance;

[0173] The model evaluation module is responsible for calculating the AUC value of the model and evaluating the classification performance of the model; calculating the KS statistic and evaluating the model's ability to distinguish under different thresholds; monitoring feature stability, evaluating changes in feature distribution through PSI values, inviting experts to review the model, collecting evaluation results and expert suggestions in real time, and continuously optimizing and improving the model based on the evaluation results and expert improvement suggestions.

[0174] The embodiment described above is only one specific implementation of the present invention. Common changes and substitutions made by those skilled in the art within the scope of the technical solution of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for building a corporate credit scoring model, characterized by: The following steps are involved: Step S1: Data preprocessing Get data: Get enterprise-related financial data, operating data, and industry data from custom data sources; Data normalization: clean the raw data, remove duplicate data, unify the data format, and perform preliminary processing on invalid data including missing values, null values ​​and outliers; Outlier filtering: Use statistical methods to identify and filter out outliers in the data; Variable correlation test: Use Pearson correlation coefficient or Spearman rank correlation coefficient to quantify the correlation between variables, and use visualization methods to intuitively display the correlation between variables; use correlation analysis to identify and remove highly correlated variables, and use recursive feature elimination (RFE) or model-based feature selection to automatically select custom important variables to reduce redundancy and improve the generalization ability of the model; Variable standardization: Customize the standardization method and standardization tool to standardize the variables so that they have a unified dimension and distribution to improve the training efficiency and accuracy of the model; Step S2: Sample processing Sample analysis: Conduct preliminary analysis on sample data, use charts to visually display the distribution of data, calculate and count the mean, standard deviation, median and quartile of the statistical data to quantify the distribution characteristics of the data; analyze the number and proportion of samples of different categories to determine whether the samples are balanced; Sample balancing: Use oversampling, undersampling or synthetic minority oversampling technology SMOTE to process unbalanced sample data; Missing value filling: fill missing values ​​using interpolation, mean filling or model prediction methods; Sample filtering: Identify and delete duplicate data records by comparing data IDs or timestamps, identify and delete abnormal samples using box plots, Z-score statistics, or isolation forest machine learning algorithms, and delete samples with missing values ​​exceeding a custom threshold. Step S3: Feature selection and derivation Feature filtering: Calculate the importance or relevance of features based on statistical tests and machine learning algorithms, and filter out the top M important features that exceed the threshold or are ranked according to the custom threshold or ranking results from the original feature set, and verify the impact of the filtered feature set on the model performance to ensure that no important information is lost, reduce noise and redundancy, and improve the generalization ability of the model; Sorting based on XGBOOST feature importance: Configure the parameters of the XGBOOST model, train the model using the XGBOOST algorithm, and obtain the importance scores of the features; sort the features according to the scores and select the top N important features; Category feature encoding: Identify the category features in the data set, encode the category features using the one-hot encoding method or the label encoding method, and generate numerical features; Feature combination: Identify feature pairs or feature groups with nonlinear relationships, perform Cartesian product operations on two or more features to generate new combined features, or generate combined features through polynomial expansion, and verify the impact of combined features on model performance; Feature correlation analysis: Calculate the correlation coefficient matrix between features, identify feature pairs whose correlation coefficients exceed a custom threshold, and choose to retain or delete redundant features to ensure that there is no high correlation between features; Feature dimensionality reduction uses principal component analysis (PCA) to perform linear transformation, project the data into a low-dimensional space, and retain the maximum variance information. Alternatively, linear discriminant analysis (LDA) is used to maximize the distance between classes and minimize the distance within classes to find the optimal projection direction. Alternatively, tree-based feature selection methods are used to sort the importance of features in random forests and select features with higher importance. Dimensionality reduction methods are applied to reduce the number of features to the specified dimension, and the impact of the reduced feature set on the model performance is verified to ensure that no important information is lost. Step S4: Model establishment Training set partitioning: custom partitioning of the data set into training set, validation set, and test set; Ensemble learning: Use random forest, gradient boosting tree GBDT or XGBOOST ensemble learning algorithms to build multiple base models, and combine the prediction results of each base model to obtain the final prediction result; Parameter optimization: Determine the parameter range to be optimized, use grid search, random search or Bayesian optimization method to find the optimal model parameters, and record the optimal parameter combination; AHP expert experience modeling: Use the analytic hierarchy process (AHP) to build a hierarchical model, determine the characteristics and weights of each level, use expert experience to adjust and optimize the hierarchical model, and train and verify the adjusted model to evaluate model performance; Verification and evaluation: Customize the accuracy, recall or F1 score as the evaluation indicator, use the validation set to evaluate the model performance, and record the evaluation results; optimize and adjust the model based on the evaluation results, and perform cross-validation to further verify the stability and reliability of the model; Step S5: Model evaluation Calculate the AUC value: Calculate the AUC value of the model to evaluate the classification performance of the model; KS model evaluation: Calculate the KS statistic to evaluate the model's ability to distinguish at different thresholds; Feature PSI monitoring: monitors feature stability, calculates the difference in feature distribution between the training set and the validation set or between the training set and the test set to obtain the PSI value, and uses the PSI value to evaluate changes in feature distribution. The larger the PSI value, the greater the change in feature distribution. When the PSI value exceeds the custom threshold, the user is notified to make adjustments. Model expert review: Invite experts to review the business logic, defects, and interpretability of the model, and optimize and improve the model based on the experts' improvement suggestions; Continuous model optimization: collect evaluation results and expert suggestions in real time, analyze the problems and deficiencies of the model, determine the optimization direction and implementation plan, iterate and optimize the model, and re-evaluate the model performance after iteration and optimization to verify the optimization effect.

2. The enterprise credit scoring modeling method according to claim 1, characterized in that: In step S1, a box plot, a Z-score statistical method or an interquartile range (IQR) is used to identify outliers in the data; a visualization method is used to intuitively display the distribution of the data so that users can identify outliers; and a data analysis software is used as a filtering tool to automatically filter out outliers.

3. The enterprise credit scoring modeling method according to claim 1, characterized in that: In the step S1, a Z-score standardization, a Min-Max standardization, a decimal calibration standardization or a logarithmic transformation method is selected by a user, and the Python NumPy library, the Pandas library or the scikit-learn library is used as a standardization tool to standardize the variables.

4. The enterprise credit scoring modeling method according to claim 1, characterized in that: In step S2, random oversampling and synthetic minority class oversampling techniques are used to make the ratio of positive and negative samples close by copying or generating minority class samples; Or use random undersampling and cluster undersampling to reduce the number of majority class samples by deleting or merging majority class samples, so that the ratio of positive and negative samples is close; Alternatively, oversampling and undersampling methods can be combined to achieve sample balancing.

5. The enterprise credit scoring modeling method according to claim 1, characterized in that: In step S2, a linear interpolation method or a polynomial interpolation method is used to calculate the missing value through an interpolation formula according to the value of the known data point; Or fill missing values ​​with the average value of the feature where the missing value is located; For skewed distribution data, fill the missing values ​​with the median of the feature where the missing value is located; For discrete features, the missing values ​​are filled with the mode of the feature where the missing value is located; Or use regression models or classification models to predict and fill missing values.

6. The enterprise credit scoring modeling method according to claim 1, characterized in that: In step S3, during statistical testing, a chi-square test is used for classification problems, and a t-test is used for regression problems, and features are screened by testing the statistical correlation between features and target variables; Random forest is used to customize the screening features by calculating the importance scores of the features; Lasso regression automatically selects important features through L1 regularization and compresses the unimportant feature coefficients to zero.

7. The enterprise credit scoring modeling method according to claim 1, characterized in that: In step S3, the Pearson correlation coefficient or the Spearman rank correlation coefficient is used to calculate the correlation coefficient between the features, and a visualization method is used to intuitively display the correlation between the features.

8. The enterprise credit scoring modeling method according to claim 1, characterized in that: In step S4, for the binary classification problem, the AUC value is used to evaluate the performance of the model at different thresholds.

9. An enterprise credit scoring model building system, characterized by: include: The data preprocessing module is responsible for obtaining data from custom data sources, normalizing the data, filtering out outliers, performing variable correlation tests, and standardizing the variables to give them a uniform dimension and distribution. The sample processing module is responsible for preliminary analysis of sample data, processing unbalanced sample data, filling missing values, filtering out duplicate data and samples with missing values ​​exceeding a custom threshold; The feature selection and derivation module is responsible for screening important features, sorting the feature importance based on XGBOOST, identifying the category features in the data set, implementing category feature encoding, expanding feature pairs or feature groups with nonlinear relationships, generating new combined features, analyzing feature correlation, and reducing the number of features to the specified dimension; The model building module is responsible for dividing the training set, using the integrated learning algorithm to build the base model, finding the optimal model parameters, building the hierarchical model, determining the characteristics and weights of each level, and adjusting and optimizing the hierarchical model using expert experience; using the validation set to evaluate, optimize and adjust the model performance; The model evaluation module is responsible for calculating the AUC value of the model to evaluate the classification performance of the model; calculating the KS statistic to evaluate the model's ability to distinguish at different thresholds; Monitor feature stability, evaluate changes in feature distribution through PSI values, invite experts to review the model, collect evaluation results and expert suggestions in real time, and continuously optimize and improve the model based on the evaluation results and expert improvement suggestions.

Images