A credit card default prediction method based on entropy features and adaptive hyperparameters

Abstract

The present application relates to a credit card default prediction method based on entropy features and adaptive hyperparameters, wherein the meta-network includes a hyperparameter generator and a base learner; the hyperparameter generator generates a first hyperparameter for updating the base learner; in any task, the state information of the task is extracted by a feature extraction function, and the entropy feature of the task is calculated by the probability of the category appearing in the task, and the entropy feature is used to weight each feature of each sample in the task; the second hyperparameter of the base learner is updated based on the first hyperparameter, and the first hyperparameter is dynamically adjusted according to the entropy feature in the task during the update; the entropy under the state information is calculated, and the first hyperparameter is updated according to the entropy value; after all tasks are executed, the total loss is calculated based on the weights of all features, the prediction loss in each task, and the regularization term to optimize the meta-network, and the optimized base learner is used to predict credit card default on the query set. The method improves the prediction accuracy of the meta-network and reduces the risk of overfitting.

Description

Technical Field

[0001] The present application relates to the technical field of credit card default prediction, and in particular to a credit card default prediction method based on entropy features and adaptive hyperparameters. Background Art

[0002] In the field of machine learning, sufficient data is the basis for efficient model training. In credit card default prediction, financial data usually has high-dimensional features, nonlinear relationships, and noisy data, which pose great challenges to model training and prediction. High-dimensional features make the data dimension too high, which makes the model easy to overfit; nonlinear relationships make it difficult for simple linear models to accurately capture the complex relationships between data; at the same time, there is a lot of noise in financial data, which further increases the difficulty of model training. Therefore, how to accurately identify potential default risks in such a complex data environment is an important issue currently faced; and credit card default prediction requires highly accurate models to identify potential default risks, but in actual applications, data samples are often scarce and unbalanced, which makes model training more difficult.

[0003] Small sample learning is designed to solve this problem by using limited data samples to train the model to achieve higher generalization ability. However, existing small sample learning methods still have many limitations, such as model overfitting and insufficient generalization ability. Summary of the invention

[0004] Based on this, it is necessary to provide a credit card default prediction method based on entropy features and adaptive hyperparameters, which includes:

[0005] S1: Divide the credit card default prediction dataset into a support set and a query set; define a meta-network, wherein the meta-network includes a hyperparameter generator and a base learner; the hyperparameter generator generates a first hyperparameter for updating the base learner;

[0006] S2: In any task, the state information of the task is extracted by a feature extraction function, and the entropy feature of the task is calculated by the probability of occurrence of the category in the task, and the entropy feature is used to weight each feature of each sample in the task;

[0007] S3: updating a second hyperparameter of the base learner based on the first hyperparameter, and dynamically adjusting the first hyperparameter according to the entropy feature in the task during the updating;

[0008] S4: Calculate the entropy of the state information, and update the first hyperparameter according to the entropy value;

[0009] S5: After all tasks are executed, the total loss is calculated based on the weights of all features, the prediction loss in each task, and the regularization term to optimize the meta-network. The optimized base learner performs credit card default prediction on the query set.

[0010] Beneficial effects: This method better measures and captures the uncertainty in credit card default prediction by introducing entropy features; in addition, a mechanism for adaptively adjusting the first hyperparameter based on entropy features is proposed. By dynamically adjusting the first hyperparameter, the meta-network can maintain stability and generalization ability when processing high-dimensional, nonlinear data. This method improves the prediction accuracy of the meta-network, reduces the risk of overfitting, and ensures that the meta-network can operate effectively under different tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0012] Figure 1 This is a flowchart of a credit card default prediction method based on entropy features and adaptive hyperparameters according to an embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0014] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0015] This embodiment provides a credit card default prediction method, specifically a meta-learning small sample credit card default prediction method based on entropy features and adaptive hyperparameters.

[0016] Meta-learning, as a method of "learning to learn", improves the adaptability of the model to the new environment by designing algorithms that can quickly adapt to new tasks. In small sample learning, meta-learning is particularly important because it can use previous learning experience to guide the rapid learning of new tasks, thereby effectively coping with the problem of data scarcity. Entropy is an important tool for measuring system uncertainty. In machine learning, entropy can be used to evaluate the information loss of the model, guide feature selection and model optimization. By utilizing entropy features, the inherent structure of the data can be more effectively captured, especially when the data sample is limited, which helps to improve the recognition ability and decision-making efficiency of the model. Hyperparameter settings are crucial to the performance of any machine learning model. Adaptive hyperparameter technology maximizes the effectiveness of the model by dynamically adjusting hyperparameters according to the performance of the model in different tasks, which is of great significance for solving the problem of insufficient generalization ability of the model in small sample learning.

[0017] like Figure 1 As shown, the method includes:

[0018] S1: Divide the credit card default prediction data set into a support set and a query set; define a meta network, wherein the meta network includes a hyperparameter generator and a base learner; the hyperparameter generator generates a first hyperparameter for updating the base learner.

[0019] Specifically, in each task,

[0020] ;

[0021] ;

[0022] in, Indicates i The support set in each task is used to update the parameters; Indicates i The query set in each task is used for model prediction and performance evaluation; Indicates i The first j samples; Indicates i The query set of the task j samples; Indicates i Task No. j The true labels of samples; Indicates i The query set of the task j The true labels of samples; X Indicates the total number of samples; Y Represents the total number of true labels.

[0023] The measurement formula for task division is:

[0024] ;

[0025] ;

[0026] in, Represents a diversity measure of tasks; n Indicates the total number of tasks; m Representation Task i The number of samples in ; Indicates i Task No. j Characteristics of the samples; Represents the average value of a feature across all samples.

[0027] In this embodiment, the characteristics of the sample include credit score, monthly income, credit card limit, most recent repayment amount, most recent repayment date, and historical repayment record.

[0028] Historical repayment records: Extract the customer's past repayment history data, such as monthly repayment amount, repayment date, etc. to calculate the following features: average repayment amount (average repayment amount in the past 12 months), repayment frequency (frequency of on-time repayment), number of overdue payments (number of overdue payments in the past 12 months).

[0029] Credit score: Directly use the customer’s credit score as an important feature, as the credit score is usually highly correlated with the customer’s default risk.

[0030] Monthly income: The customer's monthly income level, reflecting his / her repayment ability.

[0031] Credit card limit: The maximum credit card limit of a customer can reflect his or her credit risk tolerance.

[0032] In this embodiment, the hyperparameter generator generates a first hyperparameter suitable for the current task by analyzing the entropy characteristics and diversity metrics of each task, wherein the first hyperparameter includes a learning rate and a weight decay coefficient. After the base learner receives the first hyperparameter, it uses the support set to update the parameters, and performs model prediction and evaluation on the query set after optimization.

[0033] A fully connected layer in the meta-network is represented as:

[0034] ;

[0035] ;

[0036] in, represents a fully connected layer; W represents the weight matrix; brepresents the bias vector; represents the activation function; represents the first weight matrix; represents the second weight matrix; represents the first bias vector; represents the second bias vector; x Represents an algebraic expression; Represents the adjustment parameter, which is used to control the balance between the linear part and the nonlinear part. The nonlinear part helps to capture the complex patterns and nonlinear relationships of credit defaults, while the linear part can handle large inputs to avoid the gradient vanishing problem and improve the stability of the model.

[0037] S2: In any task, the state information of the task is extracted by a feature extraction function, and the entropy feature of the task is calculated by the probability of occurrence of the categories in the task, and the entropy feature is used to weight each feature of each sample in the task.

[0038] Specifically, the extraction of state information is expressed as:

[0039] ;

[0040] ;

[0041] in, Indicates t State information of each time step; represents the feature extraction function; Indicates i All samples of a task.

[0042] Entropy is an important indicator for measuring system uncertainty. In this embodiment, the entropy feature is used to evaluate the information loss of the model in different tasks and serves as a guide for feature selection and model optimization. The entropy feature calculation formula of the task is:

[0043] ;

[0044] ;

[0045] in, Representation Task i The entropy characteristics of C Indicates the total number of categories; Representation Task i Medium Category c Probability of occurrence; m Representation Task i The number of samples in ; Indicates j The category of the samples; represents the indicator function, when When , the value is 1, otherwise the value is 0;

[0046] In order to better utilize features in small sample prediction tasks, an entropy weighting mechanism is proposed, which weights each feature of each sample in the task by using the entropy feature:

[0047] Task-based i The entropy feature calculation task i The weights of all features in each sample are calculated as follows:

[0048] ;

[0049] in, Indicates i Task No. j Features in samples u The weight of Indicates i Task No. j The sample u Features U Indicates i Task No. j The total number of features of the samples; Representation characteristics The variance of .

[0050] S3: updating the second hyperparameter of the base learner based on the first hyperparameter, and dynamically adjusting the first hyperparameter according to the entropy feature in the task during the updating.

[0051] Specifically, the first hyperparameter is dynamically adjusted according to the entropy characteristics in the task, and the calculation formula is:

[0052] ;

[0053] ;

[0054] in, Indicates i Task No. t The learning rate for each time step; represents the minimum learning rate; represents the maximum learning rate; represents the first adjustment factor; Representation Task i The entropy characteristics of Represents a diversity measure of tasks; Indicates i Task No. t The weight decay coefficient for each time step; represents the minimum weight decay coefficient; represents the highest weight attenuation coefficient; Represents the second adjustment factor; the weight decay coefficient is used to adjust the learning rate.

[0055] Furthermore, the base learner is a standard neural network (including a network model suitable for binary classification tasks), and the update formula of the second hyperparameter is:

[0056] ;

[0057] ;

[0058] in, Indicates i Task No. t +2nd hyperparameter of 1 time step; Indicates i Task No. t The second hyperparameter of time steps; Indicates i Task No. t The learning rate for each time step; Indicates i The gradient of the loss function with respect to the second hyperparameter in the task; It is Input features of samples; Indicates i Task No. j The true labels of samples; Indicates i Task No. j The predicted labels of samples; m Representation Task i The number of samples in .

[0059] S4: Calculate the entropy of the state information, and update the first hyperparameter according to the entropy value.

[0060] Specifically, the entropy under the state information is calculated using the formula:

[0061] ;

[0062] in, Display status information Entropy under Indicates t State information of each time step; C Indicates the total number of categories; Display status information Output Category c The probability distribution of

[0063] The first hyperparameter is updated according to the entropy value under the state information, and the updating formula is:

[0064] ;

[0065] in, represents the learning rate at the tth time step; represents the initial learning rate; Represents the first adjustment factor.

[0066] S5: After all tasks are executed, the total loss is calculated based on the weights of all features, the prediction loss in each task, and the regularization term to optimize the meta-network. The optimized base learner performs credit card default prediction on the query set, and the results include default / non-default.

[0067] Specifically, meta-network optimization is achieved through loss functions and regularization terms, with the goal of improving the generalization ability of the model; in credit card default prediction, optimizing these functions can help the model more accurately predict whether a customer will default. The expression of the total loss is:

[0068] ;

[0069] ;

[0070] in, represents the total loss; n Indicates the total number of tasks; m Representation Task i The number of samples in ; U Indicates i Task No. j The total number of features of the samples; Indicates i Task No. j Features in samples u The weight of Indicates i Task No. j The prediction loss of samples; represents the regularization parameter; K represents the number of model parameters of the meta-network; Represents the first k model parameters; Indicates i Task No. j The true labels of samples; Indicates i Task No. j The predicted labels of samples;

[0071] Based on the total loss, a gradient descent algorithm is used to optimize the meta-network.

[0072] The credit card default prediction method based on entropy features and adaptive hyperparameters provided in this embodiment has the following beneficial effects:

[0073] This method introduces entropy features to better measure and capture the uncertainty in credit card default prediction. Entropy features can effectively identify key information in the data, especially when dealing with potential risk features in customer behavior patterns, showing its unique advantages. The application of this entropy feature is fundamentally different from the data quality assessment method used in the prior art. It not only focuses on the quality of the data, but also focuses on analyzing the inherent structure of the data, thereby improving the prediction accuracy of the model. In addition, the adaptive hyperparameter adjustment mechanism proposed in this method enables the model to maintain stability and generalization ability when processing high-dimensional and nonlinear data by dynamically adjusting the learning rate and weight decay coefficient. Especially when dealing with the problem of class imbalance in small sample data, this mechanism significantly improves the performance of the model and reduces the risk of overfitting. The design of this mechanism ensures that the model can operate effectively under different tasks, fully demonstrating the superiority of adaptive adjustment.

[0074] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0075] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A credit card default prediction method based on entropy features and adaptive hyperparameters, characterized in that: include: S1: Divide the credit card default prediction dataset into a support set and a query set; define a meta-network, wherein the meta-network includes a hyperparameter generator and a base learner; the hyperparameter generator generates a first hyperparameter for updating the base learner; S2: In any task, the state information of the task is extracted by a feature extraction function, and the entropy feature of the task is calculated by the probability of occurrence of the category in the task, and the entropy feature is used to weight each feature of each sample in the task; S3: updating a second hyperparameter of the base learner based on the first hyperparameter, and dynamically adjusting the first hyperparameter according to the entropy feature in the task during the updating; The first hyperparameters include a learning rate and a weight decay coefficient; The first hyperparameter is dynamically adjusted according to the entropy characteristics in the task, and the calculation formula is: ； ； in, Indicates i Task No. t The learning rate for each time step; represents the minimum learning rate; represents the maximum learning rate; represents the first adjustment factor; Indicates the task i The entropy characteristics of Represents a diversity measure of tasks; Indicates i Task No. t The weight decay coefficient for each time step; represents the minimum weight decay coefficient; represents the highest weight attenuation coefficient; represents a second adjustment factor; the weight decay coefficient is used to adjust the learning rate; S4: Calculate the entropy of the state information, and update the first hyperparameter according to the entropy value; S5: After all tasks are executed, the total loss is calculated based on the weights of all features, the prediction loss in each task, and the regularization term to optimize the meta-network. The optimized base learner performs credit card default prediction on the query set.

2. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 1, characterized in that: S1 also includes task division, and the measurement formula for task division is: ； ； in, Represents a diversity measure of tasks; n Indicates the total number of tasks; m Indicates the task i The number of samples in ; Indicates i Task No. j Characteristics of the samples; Represents the average value of a feature across all samples.

3. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 2, characterized in that: The characteristics of the sample include credit score, monthly income, credit card limit, most recent payment amount, most recent payment date, and historical repayment record.

4. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 1, characterized in that: The extraction of state information is expressed as: ； ； ； in, Indicates t State information of each time step; represents the feature extraction function; Indicates i All samples of a task; represents the activation function; represents the first weight matrix; represents the second weight matrix; represents the first bias vector; represents the second bias vector; x Represents an algebraic expression; Indicates the adjustment parameter.

5. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 2, characterized in that: The entropy characteristic calculation formula of the task is: ； ； in, Indicates the task i The entropy characteristics of C Indicates the total number of categories; Indicates the task i Medium Category c Probability of occurrence; m Indicates the task i The number of samples in ; Indicates j The category of the samples; represents the indicator function, when When , the value is 1, otherwise the value is 0; Using the entropy feature to weight each feature of each sample in the task includes: Task-based i The entropy feature calculation task i The weights of all features in each sample are calculated as follows: ； in, Indicates i Task No. j Features in samples u The weight of Indicates i Task No. j The sample u Features U Indicates i Task No. j The total number of features of the samples; Representation characteristics The variance of .

6. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 1, characterized in that: In S3, the update formula for the second hyperparameter is: ； ； in, Indicates i Task No. t +2nd hyperparameter of 1 time step; Indicates i Task No. t The second hyperparameter of time steps; Indicates i Task No. t The learning rate for each time step; Indicates i The gradient of the loss function with respect to the second hyperparameter in the task; It is Input features of samples; Indicates i Task No. j The true labels of samples; Indicates i Task No. j The predicted labels of samples; m Indicates the task i The number of samples in .

7. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 6, characterized in that: The entropy under the state information is calculated using the formula: ； in, Display status information Entropy under Indicates t State information of each time step; C Indicates the total number of categories; Display status information Output Category c The probability distribution of The first hyperparameter is updated according to the entropy value under the state information, and the updating formula is: ； in, represents the learning rate at the tth time step; represents the initial learning rate; Represents the first adjustment factor.

8. The credit card default prediction method based on entropy features and adaptive hyperparameters according to claim 5, characterized in that: The expression of total loss is: ； ； in, represents the total loss; n Indicates the total number of tasks; m Indicates the task i The number of samples in ; U Indicates i Task No. j The total number of features of the samples; Indicates i Task No. j Features in samples u The weight of Indicates i Task No. j The prediction loss of samples; represents the regularization parameter; K represents the number of model parameters of the meta-network; Represents the first k model parameters; Indicates i Task No. j The true labels of samples; Indicates i Task No. j The predicted labels of samples; Based on the total loss, a gradient descent algorithm is used to optimize the meta-network.

Images