Loan default prediction model training method and loan default prediction method
Through iterative training and nonlinear decreasing inertia weight optimization of hyperparameter combinations, the limitations of credit scoring models in large-scale data processing are solved, and the accuracy of loan default prediction and risk identification capabilities are improved.
Patent Information
- Application Number
- CN202510781480.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-12
- Publication Date
- 2025-09-19
AI Technical Summary
Existing credit scoring models have limitations in processing large-scale data and information depth, making it difficult to effectively mine potential risk information and having limited risk identification capabilities.
By determining multiple hyperparameter combinations of the initial prediction model for iterative training, the inertia weight is updated using nonlinear decreasing method to optimize the hyperparameter combination until the fitness value converges. The nonlinear decreasing inertia weight is used to improve the particle swarm algorithm to optimize the hyperparameters of the prediction model.
It improves the accuracy of loan default prediction models, enhances risk identification capabilities, and reduces loan risks.
Smart Images
Figure CN120672460A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of financial technology, artificial intelligence or other related fields, and in particular to a loan default prediction model training method and a loan default prediction method. Background Art
[0002] At present, the credit scoring model is usually adopted, which is based on the borrower's historical behavioral data, analyzes indicators such as credit status, debt-to-income ratio and capital flow, and calculates the credit score through scores and weights.
[0003] However, there are limitations in using credit scoring models to process large-scale data and information depth, and their ability to identify risks is relatively limited. Summary of the Invention
[0004] The present application provides a loan default prediction model training method and a loan default prediction method, which improves the accuracy of prediction, enhances risk identification ability, and reduces loan risk.
[0005] In a first aspect, the present application provides a loan default prediction model training method, comprising:
[0006] Acquiring sample data, wherein the sample data includes a plurality of loan default samples and a plurality of normal samples;
[0007] Determining multiple hyperparameter combinations corresponding to an initial prediction model, training the initial prediction model according to any one of the multiple hyperparameter combinations according to the hyperparameter combination, and determining a fitness value corresponding to the trained prediction model, the fitness value being used to indicate the performance of the trained prediction model;
[0008] Based on the multiple fitness values, a nonlinear decreasing updating process is performed on the inertia weight, and a hyperparameter combination is re-determined according to the updated inertia weight, a prediction model is trained according to the re-determined hyperparameter combination, and when the fitness value corresponding to the re-determined prediction model meets a preset condition, the re-determined hyperparameter combination is used as a target hyperparameter combination, and the preset condition is used to indicate the convergence of the fitness value;
[0009] The initial prediction model is trained based on the target hyperparameter combination and the training set data in the sample data.
[0010] In a second aspect, the present application provides a loan default prediction method, comprising:
[0011] Acquiring user data, wherein the user data includes at least one piece of loan default data;
[0012] The user data is input into a loan default prediction model to obtain a loan default prediction result, wherein the loan default prediction model is trained by the loan default prediction model training method provided above and / or in various possible implementations of the first aspect.
[0013] In a third aspect, the present application provides a loan default prediction model training device, comprising:
[0014] An acquisition module, configured to acquire sample data, wherein the sample data includes a plurality of loan default samples and a plurality of normal samples;
[0015] a processing module, configured to determine multiple hyperparameter combinations corresponding to an initial prediction model, train the initial prediction model according to any one of the multiple hyperparameter combinations according to the hyperparameter combination, and determine a fitness value corresponding to the trained prediction model, wherein the fitness value is used to indicate the performance of the trained prediction model;
[0016] The processing module is further configured to perform a nonlinear decreasing update process on the inertia weight based on the multiple fitness values, and to redetermine a hyperparameter combination based on the updated inertia weight, and to train the prediction model based on the redetermined hyperparameter combination; and when the fitness value corresponding to the redetermined prediction model satisfies a preset condition, use the redetermined hyperparameter combination as a target hyperparameter combination, wherein the preset condition is used to indicate that the fitness value has converged;
[0017] A training module is used to train the initial prediction model based on the target hyperparameter combination and the training set data in the sample data.
[0018] In a fourth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;
[0019] The memory stores computer-executable instructions;
[0020] The processor executes the computer-executable instructions stored in the memory to implement the above first aspect and / or various possible implementations of the first aspect.
[0021] In a fifth aspect, the present application provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the first aspect and / or various possible implementation methods of the first aspect as described above.
[0022] In a sixth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the above first aspect and / or various possible implementations of the first aspect.
[0023] The loan default prediction model training method and loan default prediction method provided in this application determine multiple hyperparameter combinations corresponding to an initial prediction model, iteratively train the initial prediction model based on the multiple hyperparameter groups and sample data, and then determine the fitness value corresponding to the iteratively trained prediction model; and based on the multiple fitness values, perform a linear decreasing update process on the inertia weight, and re-determine the hyperparameter combination based on the updated inertia weight until the fitness value converges and the target hyperparameter combination is determined; and the initial prediction model is trained using the target hyperparameter combination and training set data from the sample data. The particle swarm algorithm is improved by using nonlinear decreasing inertia weights to optimize the hyperparameters of the prediction model, thereby improving the accuracy of the prediction, enhancing the risk identification ability, and reducing the loan risk. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0025] Figure 1 A flowchart of a loan default prediction model training method provided in an embodiment of the present application Figure 1 ;
[0026] Figure 2 A flowchart of a loan default prediction model training method provided in an embodiment of the present application Figure 2 ;
[0027] Figure 3 A flowchart of a loan default prediction model training method provided in an embodiment of the present application Figure 3 ;
[0028] Figure 4 A schematic diagram of a process for generating a synthetic loan default sample provided in an embodiment of the present application Figure 1 ;
[0029] Figure 5 A schematic diagram of a process for generating a synthetic loan default sample provided in an embodiment of the present application Figure 2 ;
[0030] Figure 6 A flowchart of a loan default prediction method provided in an embodiment of the present application;
[0031] Figure 7 A schematic diagram of the structure of a loan default prediction model training device provided in an embodiment of the present application;
[0032] Figure 8 A schematic diagram of the structure of a loan default prediction device provided in an embodiment of the present application;
[0033] Figure 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.
[0034] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION
[0035] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. In the following description, when referring to the drawings, identical numerals in different figures represent identical or similar elements, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.
[0036] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0037] In addition, this application involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and the use of artificial intelligence technology for automated decision-making, and a technical solution for making decisions that have a significant impact on personal rights and interests based on the results of automated decision-making. The application provides users with corresponding operation entrances for users to choose to agree or reject the results of automated decision-making; if the user chooses to reject, the expert decision-making process will be entered.
[0038] It should be noted that the loan default prediction model training method and loan default prediction method provided in this application can be used in the fields of financial technology and artificial intelligence, and can also be used in any field other than the fields of financial technology and artificial intelligence. The loan default prediction model training method and loan default prediction method in this application.
[0039] "Multiple" refers to two or more, and other quantifiers are similar. "And / or" describes the relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.
[0040] It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described in this application as "exemplary" or "for example" should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.
[0041] With the rapid development of my country's internet technology, internet finance is also rapidly emerging. This has led to the rise of a growing number of consumer credit products. However, these conveniences also bring about credit problems, such as loan defaults. Borrower defaults increase the risks faced by financial institutions, making the study of financial risk management particularly important.
[0042] At present, the credit scoring model is usually adopted, which is based on the borrower's historical behavioral data, analyzes indicators such as credit status, debt-to-income ratio and capital flow, and calculates the credit score through scores and weights.
[0043] However, there are limitations in using credit scoring models to process large-scale data and information depth, making it difficult to mine potential risk information and having relatively limited risk identification capabilities.
[0044] In response to the above problems, the present application provides a loan default prediction model training method, which obtains a fitness value by iteratively training multiple hyperparameter combinations determined according to the initial prediction model, and then performs nonlinear decreasing update processing on the inertia weight according to the fitness value, and redetermines the hyperparameter combination according to the updated inertia weight. Until convergence is achieved according to the redetermined hyperparameter combination, the target hyperparameter combination is determined, and the model is trained based on the target hyperparameter combination and sample data. The loan default prediction model training method provided in the present application aims to solve the problems in the existing technology of using credit scoring models to process large-scale data and information depth, making it difficult to mine potential risk information and having relatively limited risk identification capabilities.
[0045] The following specific embodiments describe in detail the technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.
[0046] Figure 1 A flowchart of a loan default prediction model training method provided in an embodiment of the present application Figure 1 ,like Figure 1 As shown, the method includes:
[0047] S101, obtaining sample data;
[0048] The sample data includes multiple loan default samples and multiple normal samples;
[0049] Among them, loan default samples are used to indicate the credit data corresponding to users who have committed default; normal samples are used to indicate the credit data corresponding to users who have not committed default.
[0050] Sample data is obtained from historical user credit data, where the labels of loan default samples are different from those of normal samples, for use in subsequent prediction model training.
[0051] For example, the label of a loan default sample is 1, and the label of a normal sample is 0.
[0052] Each sample includes feature data and a predicted label. Feature data includes: loan amount, loan term, loan interest rate, installment amount, annual income, loan disbursement month, loan purpose category, region code, and total credit turnover balance; predicted labels include: 0 for a normal customer and 1 for a defaulting customer.
[0053] S102: determining multiple hyperparameter combinations corresponding to the initial prediction model, training the initial prediction model according to any one of the multiple hyperparameter combinations, and determining a fitness value corresponding to the trained prediction model;
[0054] Among them, the fitness value is used to indicate the performance of the trained prediction model; the initial prediction model is constructed according to the type of machine learning model; and the hyperparameters are used to indicate the number of years that control the model architecture, training process, and regularization strategy.
[0055] Determine multiple hyperparameters corresponding to the initial prediction model and multiple corresponding hyperparameter combinations. For any one of the multiple hyperparameter combinations, train the initial prediction model based on the hyperparameter combination and training set samples in the sample data, and determine a fitness value corresponding to the trained prediction model. The fitness value may be, for example, an F1 score.
[0056] The machine learning model can be, for example, a recurrent neural network (CNN), a long short-term memory network (LSTM), a convolutional neural network-long short-term memory network (CNN-LSTM), etc.
[0057] Hyperparameters can include, for example, the learning rate that affects the convergence speed of the model, the number of convolutional layers that affects the depth of the model, the number of units per layer that affects the capacity of the model, and the batch size that affects the stability of training.
[0058] Exemplary:
[0059] 1) Training prediction model based on CNN learning model:
[0060] a. Determine hyperparameters, including: learning rate, range [0.0001, 0.1]; batch size, range [16, 128]; number of convolutional layers: range [1, 5]; number of filters per layer, range [16, 128]; dropout rate, range [0.0, 0.5].
[0061] b. Determine the hyperparameter combination, for example: hyperparameter combination 1: [learning rate 0.01, batch size 32, number of convolutional layers 3, number of filters 64, dropout rate 0.2]; hyperparameter combination 2: [learning rate 0.001, batch size 64, number of convolutional layers 2, number of filters 32, dropout rate 0.3]; ...; hyperparameter combination 10: [learning rate 0.005, batch size 128, number of convolutional layers 5, number of filters 16, dropout rate 0.4].
[0062] 2) Training the prediction model based on the LSTM learning model:
[0063] a. Determine hyperparameters, including: learning rate (range [0.0001, 0.1]); batch size (range [16, 128]); number of LSTM layers (range [1, 3]); number of units per layer (range [32, 128]); and dropout rate (range [0.0, 0.5].
[0064] b. Determine the hyperparameter combinations, for example: Hyperparameter combination 1: [learning rate 0.01, batch size 32, number of LSTM layers 2, number of units per layer 64, dropout rate 0.2]; Hyperparameter combination 2: [learning rate 0.001, batch size 64, number of LSTM layers 1, number of units per layer 128, dropout rate 0.3]; ...; Hyperparameter combination 10: [learning rate 0.005, batch size 128, number of LSTM layers 3, number of units per layer 32, dropout rate 0.4].
[0065] 3) Training the prediction model based on the CNN-LSTM learning model:
[0066] a. Determine hyperparameters, including: learning rate (range [0.0001, 0.1]), batch size (range [16, 128]), number of LSTM layers (range [1, 3]), number of units per layer (range [32, 128]), number of convolutional layers (range [1, 3]), dropout rate (to prevent overfitting) (range [0.0, 0.5].
[0067] b. Determine the hyperparameter combination, for example: Hyperparameter combination 1: [learning rate 0.01, batch size 32, number of LSTM layers 2, number of units per layer 64, number of convolutional layers 2, dropout rate 0.2]; Hyperparameter combination 2: [learning rate 0.001, batch size 64, number of LSTM layers 1, number of units per layer 128, number of convolutional layers 1, dropout rate 0.3]; ...; Hyperparameter combination 10: [learning rate 0.005, batch size 128, number of LSTM layers 3, number of units per layer 32, number of convolutional layers 3, dropout rate 0.4].
[0068] S103: performing a nonlinear decreasing update process on the inertia weight based on the multiple fitness values, and re-determining a hyperparameter combination based on the updated inertia weight, training the prediction model based on the re-determined hyperparameter combination, and using the re-determined hyperparameter combination as the target hyperparameter combination when the fitness value corresponding to the re-determined prediction model meets a preset condition;
[0069] Among them, the preset condition is used to indicate the convergence of the fitness value; the target hyperparameter combination is used to indicate the hyperparameter combination corresponding to the prediction model of the optimal fitness value during the training process.
[0070] For each hyperparameter combination, the model is trained and scored, and a fitness value (e.g., F1 score) is calculated using cross-validation. Based on the fitness value, the inertia weight is updated nonlinearly and incrementally. Based on the updated inertia weight, the hyperparameter combination is re-determined, and the optimal position for each hyperparameter combination and the global optimal position are updated. For example, hyperparameter combination 8 has the highest fitness, so its corresponding hyperparameter combination is considered the optimal combination.
[0071] For example, when training a prediction model based on a CNN learning model, during the iteration process, the target hyperparameter combination determined is: [learning rate: 0.001, batch size: 32, number of convolutional layers: 3, number of filters per layer: 64, dropout rate: 0.5]; when training a prediction model based on a LSTM learning model, during the iteration process, the target hyperparameter combination determined is: [learning rate: 0.001, batch size: 64, number of LSTM layers: 1, number of units per layer: 128, dropout rate: 0.3]; when training a prediction model based on a CNN-LSTM learning model, during the iteration process, the target hyperparameter combination determined is: [learning rate: 0.001, batch size: 64, number of LSTM layers: 1, number of units per layer: 128, number of convolutional layers: 1, dropout rate: 0.3].
[0072] S104: Train the initial prediction model based on the target hyperparameter combination and the training set data in the sample data.
[0073] Based on the selected target hyperparameter combination, the initial prediction model is iteratively trained using the training set in the sample data.
[0074] In one possible implementation, training an initial prediction model based on a target hyperparameter combination and training set data in the sample data is described in detail, including:
[0075] Based on the target hyperparameter combination and the training set data in the sample data, the initial prediction model is trained; the number of iterations in the training process is determined, and it is judged whether the number of iterations meets the preset number; if so, the initial prediction model obtained by training is used as the target prediction model; if not, the hyperparameter combination is initialized and the target hyperparameter combination is re-determined.
[0076] By setting the number of iterations as the termination condition, we can avoid underfitting caused by insufficient training and overtraining. In combination with dynamic adjustment of hyperparameters, we can adapt the current optimal parameters in each round of iteration.
[0077] In one possible implementation, the initial prediction model includes a first prediction model determined based on a recurrent neural network; a second prediction model determined based on a long short-term memory network; and a third prediction model determined based on a synthetic neural network. The method further includes:
[0078] The validation set data in the sample data is respectively input into the trained first prediction model, second prediction model and third prediction model to obtain the evaluation indicators corresponding to each prediction model, including accuracy, precision, recall rate and F1 score; based on the accuracy, precision, recall rate and F1 score corresponding to each prediction model, the performance of each prediction model is determined, and the prediction model with the best performance is used as the loan default prediction model.
[0079] Among them, the evaluation indicators also include the area under the receiver operating characteristic curve (AUC).
[0080] The validation set data in the sample data is input into the trained first prediction model, second prediction model and third prediction model respectively to obtain the evaluation indicators corresponding to each prediction model, including accuracy, precision, recall rate, F1 score and AUC. A comparative analysis is performed based on the evaluation indicators of each model to determine the performance of each prediction model, and the prediction model with the best performance is used as the loan default prediction model.
[0081] Three hyperparameter-optimized loan default prediction models were obtained through nonlinear decreasing update processing. The evaluation indicators of each prediction model were compared and analyzed, and the optimal prediction model was selected, thereby improving the prediction accuracy of the loan default prediction model.
[0082] The present application provides a method for training a loan default prediction model. The method determines multiple hyperparameter combinations corresponding to an initial prediction model, iteratively trains the initial prediction model based on the multiple hyperparameter groups and sample data, and then determines the fitness value corresponding to the iteratively trained prediction model. The inertia weight is linearly and progressively updated based on the multiple fitness values, and the hyperparameter combination is re-determined based on the updated inertia weight until the fitness value converges, thereby determining the target hyperparameter combination. The initial prediction model is then trained using the target hyperparameter combination and training set data from the sample data. A particle swarm algorithm is improved using a nonlinear decreasing inertia weight to optimize the hyperparameters of the prediction model, improve prediction accuracy, and reduce loan risk.
[0083] The Particle Swarm Optimization (PSO) algorithm was inspired by the phenomenon of bird foraging. The PSO algorithm initializes a specified population and simulates the birds in the flock by designing massless particles. The algorithm then adjusts the position and velocity of each individual in the population in real time based on the flight experience of each particle and other particles in the search space, searching for the optimal solution through individual collaboration and information sharing.
[0084] Figure 2 A flowchart of a loan default prediction model training method provided in an embodiment of the present application Figure 2 , this embodiment Figure 1On the basis of the embodiment, a nonlinear decreasing update process is performed on the inertia weight based on multiple fitness values, and a hyperparameter combination is re-determined according to the updated inertia weight. The prediction model is trained according to the re-determined hyperparameter combination. When the fitness value corresponding to the re-determined prediction model meets the preset conditions, the re-determined hyperparameter combination is used as the target hyperparameter combination for detailed description. The method includes:
[0085] S201, for any one of the multiple fitness values, determining an inertia weight according to a maximum inertia weight, a minimum inertia weight, a current number of iterations, and a preset maximum number of iterations of a hyperparameter combination corresponding to the fitness value during training;
[0086] Each hyperparameter combination corresponds to a particle, and multiple hyperparameter combinations represent a population. During the optimization of the hyperparameter combination, the particles also correct their positions and velocities in the next iteration until all particles find the optimal position globally.
[0087] The inertia weight w keeps the particles inertial. A larger w can improve the algorithm's global search capability, while a smaller w is beneficial for improving the algorithm's local search capability. In the early stages of iteration, a stronger global search capability helps prevent the acquisition of a local optimal solution.
[0088] A nonlinear decreasing inertia weight (NLDIW) is constructed to optimize the POS algorithm.
[0089] The inertia weight is determined using the following formula:
[0090]
[0091] Among them, w is the inertia weight; w max is the maximum value of inertia weight; w min is the minimum inertia weight; k is the current number of iterations; G is the maximum number of iterations.
[0092] S202, updating the velocity of the particle corresponding to the hyperparameter combination according to the inertia weight, the historical local optimal position, and the historical global optimal position;
[0093] The particle velocity is updated using the following formula:
[0094]
[0095] in, and They represent the velocity and position of the i-th particle in the k-th iteration respectively;
[0096] is the velocity and position of the i-th particle in the k+1-th iteration; c1 and c2 are dimensionless learning factors that affect the optimal position by changing the velocity of individuals and populations; r1 and r2 are random numbers between 0 and 1; p i and p g are the individual's historical best position and the population's global best position respectively.
[0097] S203, updating the position of the particle according to the updated particle velocity and the current position of the particle;
[0098] Among them, the position of the particle is used to indicate the hyperparameter combination;
[0099] The particle position is updated using the following formula:
[0100]
[0101] in, is the position of the i-th particle in the k+1-th iteration.
[0102] S204, training the prediction model according to the re-determined hyperparameter combination;
[0103] S205. Determine the current global optimal hyperparameter combination based on the fitness value of each prediction model;
[0104] Step S204 and step S205 are similar to step S103 and are not described again here.
[0105] S206. Determine whether the fitness value corresponding to the current global optimal hyperparameter combination meets the preset conditions; if so, execute step S207; if not, execute step S208.
[0106] Determine whether the fitness value corresponding to the current global optimal hyperparameter combination converges, that is, the fitness value no longer changes significantly. If the fitness value corresponding to the current global optimal hyperparameter combination converges, the current global optimal hyperparameter combination is used as the target hyperparameter combination. If the fitness value corresponding to the current global optimal hyperparameter combination does not converge, the inertia weight is updated nonlinearly and gradually, and the hyperparameter combination is re-determined based on the updated inertia weight.
[0107] S207, using the current global optimal hyperparameter combination as the target hyperparameter combination;
[0108] S208: Perform nonlinear decreasing updating processing on the inertia weight, and re-determine the hyperparameter combination based on the updated inertia weight.
[0109] An embodiment of the present application provides a loan default prediction model training method, which determines the inertia weight through the maximum inertia weight, the minimum inertia weight, the current number of iterations, and the preset maximum number of iterations during the training process, and updates the speed of particles corresponding to the hyperparameter combination based on the inertia weight, the historical local optimal position, and the historical global optimal position; then, based on the updated particle speed and the current position of the particle, the position of the particle is updated, that is, the hyperparameter combination is updated to determine the optimal hyperparameter, and the particle swarm algorithm is improved using nonlinear decreasing inertia weight, thereby optimizing the hyperparameters of the prediction model, improving the accuracy of the prediction model, and reducing loan risk.
[0110] Figure 3 A flowchart of a loan default prediction model training method provided in an embodiment of the present application Figure 3 , this embodiment Figure 1 Based on the embodiment, obtaining sample data is described in detail. The method includes:
[0111] S301, obtaining sample data;
[0112] Step S301 is similar to step S101 and will not be described again here.
[0113] S302. For any one of the multiple loan default samples, determine a distance parameter between the loan default sample and other loan default samples in the multiple loan default samples, determine k neighboring samples based on the distance parameter, and determine m target neighboring samples from the k neighboring samples;
[0114] The distance parameter is used to indicate the similarity between two loan default samples; the neighboring samples are used to indicate samples that are close to the corresponding loan default samples.
[0115] In historical credit data, there is an imbalance between loan default samples and normal samples. Usually, the number of loan default samples is less than the number of normal samples. When training a prediction model, the imbalance between loan default samples and normal samples will cause the machine learning model to tend to predict the majority class, that is, it will tend to predict normal samples, thereby affecting the performance and accuracy of the model.
[0116] For multiple loan default samples x i , any loan default sample in (i=1,2,3,...,n) Determining loan default samples using distance measurement methods and other loan default samples x i The distance parameter between them, and based on the loan default sample and other loan default samples x iMultiple distance parameters between them, from multiple distance parameters, determine k distance parameters in order from small to large, and the loan default samples x corresponding to the k distance parameters i As k neighboring samples. It can be understood that the k neighboring samples determined are similar to the loan default samples The similarity is high. Then randomly determine m target neighboring samples x from the k neighboring samples ij , (j=1, 2, 3, ..., m), which increases the data diversity and makes the subsequent newly generated synthetic loan default samples more uniform.
[0117] Figure 4 A schematic diagram of a process for generating a synthetic loan default sample provided in an embodiment of the present application Figure 1 ;like Figure 4 As shown in the figure, the five-pointed stars represent loan default samples; the circles represent normal samples; and the squares represent synthetic loan default samples.
[0118] Exemplary: For a sample of loan defaults Find the loan default samples based on Euclidean distance calculation k loan default samples with similar characteristics, and then determine m target neighboring samples x ij ,like Figure 4 As shown, compared with the loan default sample There are connecting lines between the target neighboring samples x ij .
[0119] S303: For any one of the m target neighboring samples, perform random linear interpolation between the loan default sample and the target neighboring sample to generate a synthetic loan default sample;
[0120] Using the following formula, in the loan default sample Samples x adjacent to the target ij Random linear interpolation is performed between them to generate synthetic loan default samples P j :
[0121]
[0122] Among them, rand(0,1) is a random number in the interval (0,1) that satisfies the uniform distribution trend.
[0123] rand(0,1) is a randomly generated value that determines the synthetic loan default sample P j In the loan default sample Samples x adjacent to the target ij If rand(0,1) is close to 0, the synthetic loan default sample will be closer to the loan default sample If rand(0, 1) is close to 1, the synthetic loan default sample will be closer to the target neighboring sample x ij , ensuring that the generated synthetic loan default samples are diverse and avoiding overfitting.
[0124] Figure 5 A schematic diagram of a process for generating a synthetic loan default sample provided in an embodiment of the present application Figure 2 ,like Figure 5 As shown, the solid circle in the center is the loan default sample The other solid circles are target neighboring samples x i1 , x i2 , x i3 , x i4 , x i5 , x i6 , the value between the loan default sample and the target adjacent sample is the generated synthetic loan default sample P j .
[0125] S304. Update sample data based on the synthetic loan default sample.
[0126] The synthetic loan default sample is fused with the sample data to update the sample data.
[0127] In one possible implementation, updating the sample data based on the synthetic loan default sample is described in detail, including:
[0128] The synthetic loan default samples are merged with the loan default samples, and a ratio is determined based on the number of merged loan default samples and the number of normal samples; it is determined whether the ratio reaches a preset ratio; if not, the synthetic loan default samples are regenerated based on the merged loan default samples; if so, the merged loan default samples and multiple normal samples are used as new sample data.
[0129] By generating synthetic loan default samples, the imbalance between loan default samples and normal samples is improved. The generated loan default samples are diverse, avoiding overfitting, and improving the performance and accuracy of the prediction model.
[0130] An embodiment of the present application provides a loan default prediction model training method. This method determines k neighboring samples based on the distance parameters between a loan default sample and other loan default samples, randomly determines m target neighboring samples from the k neighboring samples, and then performs random linear interpolation between the m target neighboring samples and the loan default sample to generate a synthetic loan default sample. This method avoids the problem of the prediction model being biased towards predicting the majority class due to an imbalance in the proportion of loan default samples and normal samples in the sample data, thereby affecting the performance of the prediction model. This improves the performance and accuracy of the prediction model.
[0131] Figure 6 A flowchart of a loan default prediction method provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the method includes:
[0132] S601, obtaining user data;
[0133] The user data includes at least one loan default data; the loan default data may be, for example, the loan amount, loan term, loan interest rate, installment amount, annual income, loan issuance month, loan purpose category, region code, and total credit turnover balance.
[0134] S602: Input user data into a loan default prediction model to obtain a loan default prediction result.
[0135] The loan default prediction model is obtained by training through the above-mentioned model training embodiment.
[0136] This embodiment provides a loan default prediction method that obtains user data, such as loan amount, loan term, loan interest rate, installment amount, annual income, loan issuance month, loan purpose category, region code, and total credit turnover balance, and inputs the user data into a loan default prediction model to predict whether the user will default, thereby improving risk identification capabilities.
[0137] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all optional embodiments, and the actions and modules involved are not necessarily required by this application.
[0138] It should be further noted that, although the various steps in the flowchart are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps may be performed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily performed at the same time, but may be performed at different times. The execution order of these sub-steps or stages is not necessarily to be performed in sequence, but may be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.
[0139] Figure 7A schematic diagram of the structure of a loan default prediction model training device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the loan default prediction model training device 70 provided in this embodiment includes:
[0140] An acquisition module 701 is used to acquire sample data, where the sample data includes a plurality of loan default samples and a plurality of normal samples;
[0141] Processing module 702 is used to determine multiple hyperparameter combinations corresponding to the initial prediction model, train the initial prediction model according to any one of the multiple hyperparameter combinations, and determine a fitness value corresponding to the trained prediction model, where the fitness value is used to indicate the performance of the trained prediction model;
[0142] The processing module 702 is further configured to perform a nonlinear decreasing update process on the inertia weight based on the multiple fitness values, and re-determine a hyperparameter combination based on the updated inertia weight, train the prediction model based on the re-determined hyperparameter combination, and use the re-determined hyperparameter combination as a target hyperparameter combination when the fitness value corresponding to the re-determined prediction model meets a preset condition, wherein the preset condition is used to indicate that the fitness value has converged;
[0143] The training module 703 is used to train the initial prediction model based on the target hyperparameter combination and the training set data in the sample data.
[0144] In one possible implementation, the processing module 702 is further configured to determine, for any one of the multiple fitness values, an inertia weight according to a maximum inertia weight, a minimum inertia weight, a current number of iterations, and a preset maximum number of iterations of a hyperparameter combination corresponding to the fitness value during training;
[0145] Update the velocity of the particle corresponding to the hyperparameter combination according to the inertia weight, the historical local optimal position and the historical global optimal position;
[0146] The position of the particle is updated according to the updated particle velocity and the current position of the particle. The position of the particle is used to indicate the hyperparameter combination.
[0147] In a possible implementation, the processing module 702 is further configured to determine the current global optimal hyperparameter combination according to the fitness value of the prediction model corresponding to each hyperparameter combination;
[0148] Determine whether the fitness value corresponding to the current global optimal hyperparameter combination meets the preset conditions;
[0149] If so, the current global optimal hyperparameter combination is used as the target hyperparameter combination;
[0150] If not, the inertia weight is updated nonlinearly and gradually, and the hyperparameter combination is re-determined based on the updated inertia weight.
[0151] In a possible implementation, the acquisition module 701 is further configured to acquire sample data;
[0152] For any one of the multiple loan default samples, determine a distance parameter between the loan default sample and other loan default samples in the multiple loan default samples, determine k neighboring samples based on the distance parameter, and determine m target neighboring samples from the k neighboring samples;
[0153] For any target neighboring sample among the m target neighboring samples, random linear interpolation is performed between the loan default sample and the target neighboring sample to generate a synthetic loan default sample;
[0154] The sample data is updated based on the synthetic loan default sample.
[0155] In one possible implementation, the acquisition module 701 is further configured to merge the synthetic loan default samples with the loan default samples, and determine a ratio based on the number of merged loan default samples and the number of normal samples;
[0156] Determine whether the ratio reaches a preset ratio;
[0157] If not, then regenerate the synthetic loan default sample based on the fused loan default sample;
[0158] If so, the fused loan default samples and multiple normal samples are used as new sample data.
[0159] In one possible implementation, the training module 703 is further configured to train the initial prediction model based on the target hyperparameter combination and the training set data in the sample data;
[0160] Determine the number of iterations in the training process and judge whether the number of iterations meets the preset number;
[0161] If so, the initial prediction model obtained through training is used as the target prediction model;
[0162] If not, initialize the hyperparameter combination and re-determine the target hyperparameter combination.
[0163] In one possible implementation, the initial prediction model includes a first prediction model determined based on a recurrent neural network; a second prediction model determined based on a long short-term memory network; and a third prediction model determined based on a synthetic neural network. The training module 703 is further configured to input validation set data in the sample data into the trained first prediction model, the second prediction model, and the third prediction model, respectively, to obtain evaluation indicators corresponding to each prediction model, including accuracy, precision, recall, and F1 score.
[0164] Based on the accuracy, precision, recall rate, and F1 score of each prediction model, the performance of each prediction model is determined, and the prediction model with the best performance is used as the loan default prediction model.
[0165] This embodiment provides a loan default prediction model training device that can execute the method provided in the above-mentioned model training method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.
[0166] Figure 8 A schematic diagram of the structure of a loan default prediction device provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, the loan default prediction device 80 provided in this embodiment includes:
[0167] A data acquisition module 801 is configured to acquire user data, wherein the user data includes at least one piece of loan default data;
[0168] The prediction module 802 is used to input the user data into a loan default prediction model to obtain a loan default prediction result. The loan default prediction model is trained using the method provided in the above-mentioned model training method embodiment.
[0169] This embodiment provides a loan default prediction device that can execute the method provided in the above-mentioned loan default prediction method embodiment. Its implementation principle and technical effects are similar and will not be described in detail in this embodiment.
[0170] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 9 As shown, the electronic device 90 provided in this embodiment includes: at least one processor 901 and a memory 902. Optionally, the device 90 further includes a communication component 903. The processor 901, the memory 902, and the communication component 903 are connected via a bus 904.
[0171] During the specific implementation process, at least one processor 901 executes the computer-executable instructions stored in the memory 902, so that the at least one processor 901 performs the above method.
[0172] The specific implementation process of the processor 901 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.
[0173] The present application also provides a computer program product, including a computer program, which implements the above method when executed by a processor.
[0174] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above method is implemented.
[0175] It should be understood that the above-described device embodiments are merely illustrative, and the device of the present application may also be implemented in other ways. For example, the division of units / modules in the above-described embodiments is merely a logical functional division, and actual implementations may employ other division methods. For example, multiple units, modules, or components may be combined or integrated into another system, or some features may be omitted or not implemented.
[0176] In addition, unless otherwise specified, the functional units / modules in the various embodiments of the present application may be integrated into a single unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The aforementioned integrated units / modules may be implemented in the form of hardware or software program modules.
[0177] If the integrated unit / module is implemented in hardware, the hardware may be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor may be any appropriate hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC. Unless otherwise specified, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.
[0178] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes a number of instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes various media that can store program codes, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.
[0179] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a particular embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined in any way. To keep the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0180] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not disclosed herein. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present application are indicated by the following claims.
[0181] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.
Claims
1. A loan default prediction model training method, characterized in that: include: Acquiring sample data, wherein the sample data includes a plurality of loan default samples and a plurality of normal samples; Determining multiple hyperparameter combinations corresponding to an initial prediction model, training the initial prediction model according to any one of the multiple hyperparameter combinations according to the hyperparameter combination, and determining a fitness value corresponding to the trained prediction model, the fitness value being used to indicate the performance of the trained prediction model; Based on the multiple fitness values, a nonlinear decreasing updating process is performed on the inertia weight, and a hyperparameter combination is re-determined according to the updated inertia weight, a prediction model is trained according to the re-determined hyperparameter combination, and when the fitness value corresponding to the re-determined prediction model meets a preset condition, the re-determined hyperparameter combination is used as a target hyperparameter combination, and the preset condition is used to indicate the convergence of the fitness value; The initial prediction model is trained based on the target hyperparameter combination and the training set data in the sample data.
2. The method according to claim 1, characterized in that The nonlinear decreasing updating process of the inertia weight is performed based on multiple fitness values, and the hyperparameter combination is re-determined according to the updated inertia weight, including: For any one of the multiple fitness values, determine the inertia weight according to a maximum inertia weight, a minimum inertia weight, a current number of iterations, and a preset maximum number of iterations of a hyperparameter combination corresponding to the fitness value during training; updating the velocity of the particle corresponding to the hyperparameter combination according to the inertia weight, the historical local optimal position, and the historical global optimal position; The position of the particle is updated according to the updated particle velocity and the current position of the particle, and the position of the particle is used to indicate a hyperparameter combination.
3. The method according to claim 1, characterized in that When the fitness value corresponding to the re-determined prediction model meets a preset condition, using the re-determined hyperparameter combination as the target hyperparameter combination includes: Determine the current global optimal hyperparameter combination based on the fitness value of the prediction model corresponding to each hyperparameter combination; Determine whether the fitness value corresponding to the current global optimal hyperparameter combination meets the preset conditions; If so, the current global optimal hyperparameter combination is used as the target hyperparameter combination; If not, the inertia weight is updated nonlinearly and gradually, and the hyperparameter combination is re-determined based on the updated inertia weight.
4. The method according to claim 1, wherein The obtaining of sample data includes: Get sample data; For any one of the multiple loan default samples, determining a distance parameter between the loan default sample and other loan default samples in the multiple loan default samples, and determining k neighboring samples based on the distance parameter, and determining m target neighboring samples from the k neighboring samples; For any one of the m target neighboring samples, performing random linear interpolation between the loan default sample and the target neighboring sample to generate a synthetic loan default sample; The sample data is updated based on the synthetic loan default sample.
5. The method according to claim 4, characterized in that The updating of the sample data based on the synthetic loan default sample includes: fusing the synthetic loan default sample with the loan default sample, and determining a ratio based on the number of the fused loan default samples and the number of the normal samples; Determining whether the ratio reaches a preset ratio; If not, regenerating a synthetic loan default sample based on the fused loan default sample; If so, the merged loan default sample and the multiple normal samples are used as new sample data.
6. The method according to claim 1, characterized in that The training of the initial prediction model based on the target hyperparameter combination and the training set data in the sample data includes: Training the initial prediction model based on the target hyperparameter combination and the training set data in the sample data; Determine the number of iterations in the training process, and determine whether the number of iterations meets a preset number; If so, the initial prediction model obtained through training is used as the target prediction model; If not, initialize the hyperparameter combination and re-determine the target hyperparameter combination.
7. The method according to claim 1, characterized in that The initial prediction model includes a first prediction model determined based on a recurrent neural network; a second prediction model determined based on a long short-term memory network; and a third prediction model determined based on a synthetic neural network. The method further includes: Input the validation set data in the sample data into the trained first prediction model, the second prediction model, and the third prediction model respectively, and obtain the evaluation indicators corresponding to each prediction model, wherein the evaluation indicators include accuracy, precision, recall rate, and F1 score; Based on the accuracy, precision, recall, and F1 score corresponding to each prediction model, the performance corresponding to each prediction model is determined, and the prediction model with the best performance is used as the loan default prediction model.
8. A loan default prediction method, characterized in that: include: Acquiring user data, wherein the user data includes at least one piece of loan default data; The user data is input into a loan default prediction model to obtain a loan default prediction result, wherein the loan default prediction model is trained by the loan default prediction model training method according to any one of claims 1 to 7.
9. A loan default prediction model training device, characterized in that: include: An acquisition module, configured to acquire sample data, wherein the sample data includes a plurality of loan default samples and a plurality of normal samples; a processing module, configured to determine multiple hyperparameter combinations corresponding to an initial prediction model, train the initial prediction model according to any one of the multiple hyperparameter combinations according to the hyperparameter combination, and determine a fitness value corresponding to the trained prediction model, wherein the fitness value is used to indicate the performance of the trained prediction model; The processing module is further configured to perform a nonlinear decreasing update process on the inertia weight based on the multiple fitness values, and to redetermine a hyperparameter combination based on the updated inertia weight, and to train the prediction model based on the redetermined hyperparameter combination; and when the fitness value corresponding to the redetermined prediction model satisfies a preset condition, use the redetermined hyperparameter combination as a target hyperparameter combination, wherein the preset condition is used to indicate that the fitness value has converged; A training module is used to train the initial prediction model based on the target hyperparameter combination and the training set data in the sample data.
10. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.
11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.
12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.