Credit risk prediction method based on personalized federated continuous learning

Abstract

The present invention discloses a credit risk prediction method based on personalized federated continuous learning, which belongs to the field of risk prediction technology and includes the following steps: personalized federated learning, in which the server analyzes each layer of parameters of each client layer by layer through a matching algorithm to obtain the contribution of each local model parameter in the global model, and then optimizes the global parameters of the global model and sends it back to the client; based on the training data set of the current stage, the local model is incrementally updated, the credit risk prediction task is executed, and the prediction result is output. The present invention constructs a federated continuous learning framework, and uses local data to train the local model under the federated learning architecture, so that the local model has personalized adaptability. Through the continuous learning mechanism, the model can adapt to new data distribution and business needs in real time, can effectively capture the latest credit risk changes, reduce the risk of model obsolescence, and improve the practicality and robustness of the model.

Description

Technical Field

[0001] The present invention relates to the technical field of credit risk prediction, and in particular to a credit risk prediction method based on personalized federated continuous learning. Background Art

[0002] With the rapid development of FinTech and the acceleration of digital transformation, credit risk assessment has become a core component of risk management and decision-making for financial institutions. Traditional credit risk assessment methods primarily rely on historical data and static models, which have significant limitations when dealing with large-scale, dynamically changing data. For example, traditional methods often require centralized data processing and model training, which not only can lead to data leakage and privacy issues but also makes it difficult to effectively integrate heterogeneous data from different financial institutions or data sources. Furthermore, because data distribution and risk patterns change dynamically over time, traditional models struggle to adapt to new requirements and are unable to effectively capture the latest changes in credit risk. Furthermore, centralized models lack personalized adaptability, fail to fully leverage the characteristics of local data, and struggle to meet the specific needs of different financial institutions or regions.

[0003] In recent years, federated learning, a distributed machine learning framework, has emerged. It allows different clients to independently train models on local data and securely share model parameters, thereby optimizing the global model while protecting data privacy. However, traditional federated learning approaches still face challenges in credit risk prediction, including insufficient model adaptability and continuous learning capabilities. For example, the global model struggles to adapt to the local data distribution of different clients, resulting in poor performance on some clients. Furthermore, traditional federated learning approaches struggle to effectively accumulate knowledge and adapt to new phases of data when faced with dynamically changing data. Continuous learning techniques offer a new approach to addressing model updates in dynamic data environments. Continuous learning uses deep neural networks to accumulate knowledge from various phases, building a more stable model that not only helps achieve better results in future phases but also maintains good performance for previous phases. However, achieving global optimization within a federated learning framework while preserving the personalized characteristics of local models remains a key challenge in achieving efficient credit risk prediction. Summary of the Invention

[0004] The purpose of the present invention is to overcome the problems of the prior art and provide a credit risk prediction method based on personalized federated continuous learning.

[0005] The purpose of the present invention is to achieve this through the following technical solution: a credit risk prediction method based on personalized federated continuous learning, the method comprising the following steps:

[0006] Each client uploads all layer parameters of its local model to the server; the local model is a credit risk prediction model trained based on local loan sample data;

[0007] The server receives all layer parameters of the local model uploaded by each client, and analyzes all layer parameters of each local model layer by layer through a matching algorithm to obtain the contribution of each layer parameter of each local model to the global parameters of the corresponding layer of the global model. Then, the server optimizes the global parameters of each layer of the global model based on the contribution and sends the optimized global parameters to each client;

[0008] The client receives the optimized global parameters and applies them to the corresponding layers of the local model;

[0009] Incrementally update the local model based on the training dataset of the current stage;

[0010] Use the local model that has been incrementally updated to perform credit risk prediction tasks and output the prediction results.

[0011] In one example, applying the optimized global parameters to the corresponding layer of the local model includes:

[0012] Evaluate the importance of local model parameters, freeze the top P% of parameters in the local model, and use the optimized global parameters to replace the bottom 1-P% parameters in the local credit risk prediction model, where P is the parameter importance ratio threshold.

[0013] In one example, after sending the optimized global parameters to each client, the method further includes:

[0014] The final global parameters of the global model are weighted averaged according to the category distribution of the local lending sample data to update the global model parameters.

[0015] In one example, the local model uses elastic weight consolidation loss, or cross entropy loss and elastic weight consolidation loss to optimize model parameters.

[0016] In one example, the incremental updating of the local model based on the training dataset in the current stage includes:

[0017] During the update cycle, the newly added loan sample data is preprocessed to obtain the training data set for the current round. The local model obtained from the previous round of training is incrementally trained using the current round of training data set. During the incremental training process, the parameter knowledge extracted from the local model obtained from the previous training round is used to perform parameter optimization to obtain the local model corresponding to the current training round.

[0018] Utilize the parameter knowledge extracted from the local model obtained from the previous training round to perform parameter optimization, including:

[0019] The parameters of the local model in the current round of training task are compared with the parameter knowledge extracted from the local model obtained from the previous round of training task, and the parameter differences obtained from the comparison are incorporated into the loss function of the model parameter optimization of the current round of model training task, so that the parameters with larger weights in the local model obtained from the previous round of training task maintain larger weights in the loss function of the current round of training task.

[0020] In one example, after applying the optimized global parameters to the corresponding layers of the local model or after applying the optimized global parameters to the corresponding layers of the local model, the method further includes:

[0021] Determine whether the set local model update cycle has been reached. If so, determine whether the number of updates to the local model is an integer multiple of the constant S. If so, return to the step where each client uploads the parameters of any layer of its local model to the server until the optimized global parameters are applied to the corresponding layer of the local model. If not, perform an incremental update on the local model and execute the credit risk prediction task.

[0022] It should be further explained that the technical features corresponding to the above examples can be combined or replaced with each other to form a new technical solution.

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] 1. In one example, the present invention constructs a federated continuous learning framework, which not only realizes the continuous updating of the model in the time dimension, but also effectively integrates heterogeneous data from different financial institutions or data sources through a multi-client knowledge sharing mechanism in the spatial dimension, thereby realizing resource-efficient collective intelligence collaborative credit risk prediction. Compared with traditional centralized learning methods, the present invention fully utilizes the advantages of distributed data while protecting data privacy, namely: using local data to train local models under a federated learning architecture can fully utilize local characteristics, thereby making local models personalized and adaptable to meet the specific needs of different financial institutions or regions, significantly improving the generalization ability and prediction accuracy of the model. In addition, through the continuous learning mechanism, the model can adapt to new data distribution and business needs in real time, can effectively capture the latest changes in credit risk, reduce the risk of model obsolescence, and improve the practicality and robustness of the model.

[0025] Furthermore, the server uses a matching algorithm to analyze the parameters of any layer of each client layer by layer, and can identify hidden elements with similar characteristics, thereby more finely capturing the characteristics of different layers of each local model, and more accurately evaluating the contribution of each local model parameter to the global model parameter. This effectively solves the performance degradation problem caused by direct weight averaging in traditional federated learning, and ensures that the global model parameters can better reflect the common characteristics of client data.

[0026] 2. In one example, the importance of model parameters is evaluated, and the top P% of parameters ranked by importance are frozen based on the evaluation results. This protects these key parameters from interference in subsequent training stages, mitigates catastrophic forgetting, and ensures that the model does not destroy its performance on old tasks when learning new tasks. The last 1-P% of parameters are directly replaced with optimized global parameters, further optimizing local model parameters and improving the model's stability and adaptability, enabling the model to maintain good performance under different data distributions and stage requirements.

[0027] At the same time, freezing important parameters—that is, limiting the fluctuation range of important local model parameters—enables resource-efficient model updates, avoids frequent global updates and large data transfers, and significantly reduces computational and communication costs, making it particularly suitable for resource-constrained scenarios. Compared with traditional global model update methods, this method reduces unnecessary computational resource consumption and improves model update efficiency through local updates. Furthermore, by assessing parameter importance and freezing important parameters, the model update process is further optimized, ensuring model stability and effectiveness.

[0028] 3. In one example, regularization constraints were introduced during training: elastic weight consolidation loss and cross-entropy loss. By minimizing the cross-entropy loss, the model continuously adjusts its internal parameters, bringing predictions closer to the true labels. This optimizes the model's ability to identify normal samples while also better identifying high-risk samples. Elastic weight consolidation loss is used to prevent catastrophic forgetting, allowing the model to retain knowledge from previous stages while learning new ones.

[0029] 4. In one example, the final weights of the global model are weighted averaged based on the category distribution of local loan sample data. The weighted average parameters can more accurately reflect the category distribution of local data, thereby constructing a model that is more adapted to the global data distribution and improving the generalization ability of the global model on global data.

[0030] 5. In one example, during the incremental update process, the parameters of the local model in the current training phase are compared with the model parameter knowledge extracted in the previous training phase, and the parameter differences obtained from the comparison are incorporated into the loss function of the current model training phase. When learning a new phase, the model will be constrained by the regularization term, so that the parameters with larger weights in the previous training phase maintain higher weights in the current training phase, thereby further improving the accuracy of credit risk prediction.

[0031] 6. In one example, by determining whether the number of local model updates is an integer multiple of a constant S, a personalized federation step or a local model update step is executed, thereby controlling the timing of global model parameter aggregation and local model update frequency. This not only helps optimize the personalized training of the local model, but also improves communication efficiency and accelerates model convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The specific embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The same reference numerals are used in these drawings to represent the same or similar parts. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application.

[0033] Figure 1 A flow chart of a method provided as an example of the present invention;

[0034] Figure 2 A schematic diagram of a deep neural network provided for an example of the present invention;

[0035] Figure 3 A personalized protection schematic diagram provided for an example of the present invention;

[0036] Figure 4 A schematic diagram of continuous training of a local model provided for an example of the present invention. DETAILED DESCRIPTION

[0037] The technical solutions of the present invention are described clearly and completely below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. At the same time, the technical features involved in the different examples of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0038] In one example, a credit risk prediction method based on personalized federated continuous learning, such as Figure 1 As shown, the method includes the following steps:

[0039] S1: Each client uploads all layer parameters of its local model to the server.

[0040] The local model is a credit risk prediction model trained by each client (such as a bank client) based on its own local data. It can capture the unique patterns and regularities of local data, thereby providing diverse information for the global model. Credit risk prediction models can be deep neural networks, convolutional neural networks, recurrent neural networks, Transformer models, and other models. Local data refers to local loan sample data, which can be text data, image data, and other types of data. Furthermore, local loan sample data includes loan-related information, borrower basic information, and credit assessment data. Loan-related information includes loan amount, loan term, loan interest rate, loan status, and loan purpose. Borrower basic information includes annual income, debt-to-income ratio, homeownership, and years of work experience. Credit assessment data includes loan grade, credit score, and earliest credit record date. Repayment and history data includes loan disbursement date, last repayment date, and total amount paid. This local data serves as the training dataset for the local model, and the dataset also includes anomalous data, i.e., high-risk loan samples.

[0041] Optionally, after each client uploads all layer parameters of its local model to the server, the server then updates the parameters of the local model layer by layer, starting with the first layer, in conjunction with the subsequent personalized federated collaborative learning steps S2-S3. Model parameters include weights, biases, and other parameters, specifically the model weights in this example. Furthermore, a layer refers to the hierarchical structure of the local model, such as the hidden layers in a deep neural network.

[0042] Preferably, before step S1, the following steps are further included:

[0043] S0: Initialize the local model and the global model.

[0044] Specifically, model initialization includes defining the model structure, configuring learning parameters, etc. Among them, the global model is a credit risk prediction model obtained by aggregating the local model parameters of all clients. It can reflect the comprehensive characteristics of all client data and then learn a more comprehensive and universal feature representation. The local model and the global model can have the same network structure. In this example, the local model and the global model are preferably deep neural networks for performing credit risk prediction, that is, the local model and the global model are credit risk prediction models. The deep neural network includes an input layer, a hidden layer, and an output layer, wherein the input layer receives the original data (local data) and passes it to the network. The hidden layer extracts the high-level features of the local data layer by layer through the weighted summation of multiple layers of neurons and nonlinear activation function processing, and the output layer converts the features extracted by the hidden layer into the final prediction results according to the stage requirements.

[0045] Furthermore, the credit risk prediction model based on the deep neural network structure is constructed as follows:

[0046] Construct a deep neural network consisting of an input layer, several hidden layers and an output layer, such as Figure 2 As shown. The input loan sample undergoes a series of linear processing and activation processing through the weight matrix and bias vector of the deep neural network, and then is processed by logistic regression, and finally the final credit risk prediction result is output through the output layer. On the local client, the deep neural network first maps the input loan sample data to a low-dimensional feature space, and through the weighted summation and nonlinear activation function processing of multiple layers of neurons in the hidden layer, it extracts the characteristic patterns that can effectively characterize the credit risk layer by layer. These characteristic patterns reflect the differences between different borrowers and the potential laws related to credit risk. For example, the more historical overdue records a borrower has, the higher the risk proportion in the subsequent credit risk prediction. Finally, the model output layer converts the features extracted by the hidden layer into specific credit risk prediction results according to the stage requirements. The present invention mines the intrinsic correlation between the borrower's behavioral characteristics and credit risk through model loan samples, thereby realizing credit risk prediction.

[0047] By introducing a deep neural network as a local risk prediction model, we can fully leverage its powerful feature extraction capabilities to map complex credit risk data into a low-dimensional feature space. Through multi-layered nonlinear transformations, we extract characteristic patterns that effectively represent credit risk. This automated feature extraction approach not only avoids the subjectivity inherent in traditional methods that rely on manual experience to select features, but also captures subtle differences between borrowers and underlying patterns associated with credit risk, significantly improving the model's ability to identify credit risk, especially for high-risk samples.

[0048] Optionally, the local model and global model of the present invention are not only applicable to credit risk prediction, but also to various application scenarios such as financial abnormal transaction detection, financial fraud detection, and intelligent investment decision-making. Taking financial abnormal transaction detection as an example, the local model and global model are abnormal transaction detection models, and the network structure is any one of a deep neural network, a convolutional neural network, a recurrent neural network, and a Transformer model. In this case, the present invention provides an abnormal transaction detection method based on personalized federated continuous learning, which includes the following steps:

[0049] Each client uploads all layer parameters of its local model to the server; the local model is an abnormal transaction detection model trained based on local credit card sample data;

[0050] The server receives all layer parameters of the local model uploaded by each client, and analyzes all layer parameters of each local model layer by layer through a matching algorithm to obtain the contribution of each layer parameter of each local model to the global parameters of the corresponding layer of the global model. Then, the server optimizes the global parameters of the corresponding layer of the global model based on the contribution and sends the optimized global parameters to each client.

[0051] The client receives the optimized global parameters and applies them to the corresponding layers of the local model;

[0052] Incrementally update the local model based on the training dataset of the current stage;

[0053] The local model that has been incrementally updated is used to perform abnormal transaction detection tasks and output prediction results.

[0054] Specifically, the data types of the credit card sample data input into the local model are text, images, etc. The credit card sample data includes basic information of the cardholder and credit card transaction information. The basic information of the cardholder includes gender, education, marital status, and age. The credit card transaction information includes transaction time, transaction location, transaction amount, and transaction frequency. At this time, the input credit card sample undergoes a series of linear processing and activation processing through the weight matrix and bias vector of the deep neural network, and then is processed by logistic regression, and finally the final abnormal transaction detection result is output through the output layer. On the local client, the deep neural network first maps the input credit card sample data to a low-dimensional feature space, and through the weighted summation and nonlinear activation function processing of multiple layers of neurons in the hidden layer, it extracts feature patterns that can effectively characterize abnormal transactions layer by layer. Finally, the model output layer converts the features extracted by the hidden layer into specific abnormal transaction detection results according to the stage requirements.

[0055] It should be noted that the technical concepts of the model in the method of the present invention for executing credit risk prediction tasks, financial abnormal transaction detection tasks, financial fraud detection tasks, and intelligent investment decision-making tasks are the same. The only difference lies in the sample data input into the model and the prediction purpose.

[0056] S2: The server receives all the layer parameters of the local model uploaded by each client, and analyzes all the layer parameters of the local model layer by layer through the matching algorithm, and then identifies the hidden elements with similar characteristics, and obtains the contribution of each layer parameter of each local model to the global parameters of the corresponding layer of the global model (matching result), and then optimizes the global parameters of each layer of the global model layer by layer based on the contribution, instead of simply averaging them by element, and broadcasts the optimized global parameters to each client.

[0057] The matching algorithm may be a cosine similarity algorithm or other similarity measurement methods. In this case, the weighted aggregation of server hierarchical parameters (weights) specifically includes the following steps:

[0058] Assume we have K clients, each with a local model ,in , indicating the client number; Indicates the number of layers in the model.

[0059] The server first collects the local model from each client Layer weights These weights are then analyzed by a matching algorithm to identify hidden elements with similar characteristics. The matching algorithm produces a set of similarity weights that represent the contribution of each client weight to the global model.

[0060] Based on the matching results, the server calculates the first Global weight of the layer , the formula is as follows:

[0061] ;

[0062] in, Represents the client In the The contribution weight of the layer weight, where and This hierarchical parameter weighted aggregation method can ensure that the weights of the global model can better reflect the common characteristics of all client data, while avoiding the performance degradation that may be caused by directly averaging the weights.

[0063] Finally, the server will calculate the Global weight of the layer Distribute it back to all clients.

[0064] S3: The client receives the optimized global parameters and applies them to the corresponding layers of the local model.

[0065] S4: Incrementally update the local model based on the training dataset of the current stage.

[0066] Specifically, local model training employs a continuous learning strategy, with each round of model training serving as a training phase. For the first phase of the local model, the client preprocesses the existing credit sample data available for training to obtain a training credit sample set. Data preprocessing methods include, but are not limited to, addressing missing data, removing redundant features, constructing new features, and balancing data categories, all in preparation for training a high-quality model. The local model is then trained using backpropagation based on the training credit sample set, optimizing and adjusting network parameters to continuously improve the model's ability to identify credit risk and predict accuracy, ultimately obtaining the local model corresponding to the first phase.

[0067] Building on the existing local model, we introduce new phase training data to further train and optimize the local model to adapt to changes in abnormal patterns that may be contained in the new data. This approach allows the local model to continuously learn and update its knowledge base, thereby improving its ability to identify emerging abnormal lending behavior while retaining its memory of previous credit risk predictions, effectively integrating new knowledge without retraining the entire model.

[0068] S5: Use the local model that has completed the incremental update to perform the credit risk prediction task and output the prediction results.

[0069] For clients (such as banks), when faced with loan applications, they use preprocessed credit samples as input and use the latest local model updated in a dynamic manner to predict the loan samples to predict whether the loan will default.

[0070] The method of the present invention realizes personalized optimization of the model and continuous improvement of credit risk prediction capabilities by combining federated learning with continuous learning. First, by initializing the deep neural network model, basic support is provided for credit risk prediction; then, the client participates in federated learning by uploading the model weights layer by layer, and uses the optimized global parameters to update the parameters of the corresponding layer of the local model, ensuring that the model adapts to local data characteristics while optimizing globally. On this basis, a continuous learning strategy is adopted, and the model is incrementally updated locally based on the new stage data, dynamically adapting to data changes, and timely capturing new abnormal patterns, while retaining the memory of old data. It can be seen that the present invention innovatively constructs a federated continuous learning framework, which not only realizes continuous updating of the model in the time dimension, but also effectively integrates heterogeneous data from different financial institutions or data sources through a multi-client knowledge sharing mechanism in the spatial dimension, realizing resource-efficient collective intelligence collaborative credit risk prediction.

[0071] Compared with traditional centralized learning methods, the method of the present invention has stronger data privacy protection. Specifically, the present invention adopts a "data does not leave the local" federated learning mechanism to ensure the security and privacy of each client's data. By sharing model parameters instead of original data, efficient knowledge transfer and fusion are achieved, significantly enhancing data privacy protection. Compared with traditional data sharing methods, the present invention achieves efficient knowledge transfer and fusion by sharing model parameters without sharing original data, significantly enhancing data privacy protection and complying with increasingly stringent privacy regulations. This method not only protects data privacy, but also improves the generalization ability of the model.

[0072] While protecting data privacy, it fully utilizes the advantages of distributed data, namely: using local data to train local models under the federated learning architecture can fully utilize local characteristics, thereby making local models personalized and adaptable to meet the specific needs of different financial institutions or regions, significantly improving the model's generalization ability and prediction accuracy.

[0073] Furthermore, the dynamic adaptability of the model is improved through a continuous learning mechanism. Specifically, the present invention can update the model parameters in real time through a continuous learning mechanism, so that it can better adapt to new data distributions and business needs. This method significantly improves the dynamic adaptability of the model in a complex financial environment. The model can adapt to new data distributions and business needs in real time, can effectively capture the latest changes in credit risk, reduce the risk of model obsolescence, and improve the practicality and robustness of the model. Compared with traditional static models, the dynamic adaptability of the present invention significantly improves the performance of the model in practical applications, reduces the risk of model obsolescence, and improves the practicality and robustness of the model.

[0074] In addition, the federated continuous learning framework of the method of the present invention has good scalability and can flexibly respond to data and business needs of different scales. For example, it is not only suitable for credit risk prediction, but also suitable for various application scenarios such as financial fraud detection and intelligent investment decision-making, and has high practicality and economic value. Compared with existing solutions for single application scenarios, the present invention can be flexibly applied to a variety of financial business scenarios through a universal federated continuous learning framework, and has higher practicality and economic value. In addition, the framework design of the present invention is simple and flexible, and it is easy to expand to more clients and application scenarios, and has wide applicability and scalability.

[0075] In one example, the local model uses elastic weight consolidation loss, or cross entropy loss and elastic weight consolidation (EWC) loss to optimize model parameters. In this example, the loss function of the local model includes cross entropy loss and EWC loss. The cross entropy loss function is used to measure the difference between the model's predicted output and the true label, thereby evaluating the model's prediction performance. The cross entropy loss function of the cth client is The calculation expression is:

[0076] ;

[0077] Among them, 2 is the number of categories; is the true label (usually represented by one-hot encoding, with the target category corresponding to 1 and the rest being 0); is the probability distribution predicted by the model.

[0078] Elastic Weight Consolidation Loss measures the importance of parameters by introducing the Fisher Information Matrix and adding regularization terms in the optimization process to prevent drastic changes to important parameters, thereby preventing the model from forgetting the knowledge of old tasks when learning new tasks. EWC assigns greater weights to important parameters to ensure that these parameters remain stable in subsequent training, thereby achieving continuous accumulation and consolidation of knowledge in multi-task learning scenarios. Specifically, Elastic Weight Consolidation Loss The calculation expression is:

[0079] ;

[0080] in, Represents the weight coefficient of the regularization term; are the diagonal elements of the Fisher information matrix; is the current parameter; The optimal parameters obtained by training on the previous task. This example uses the elastic weight consolidation method to extract model parameter knowledge without preserving real credit samples from the previous stage. The elastic weight consolidation method uses the importance of model parameters as knowledge, and the information about the importance of model parameters is transferred between models. This knowledge is passed to the next stage, thereby limiting the range of changes in important parameters and guiding model training in the next stage.

[0081] During training, the total loss function of each local client can be expressed as the weighted sum of cross entropy loss and EWC loss, and the calculation expression is:

[0082] ;

[0083] in, Represents the total loss function.

[0084] In one example, after the client applies the optimized global parameters to the corresponding layer of the local model, the following steps are also performed:

[0085] Freeze the parameters of the corresponding layer (keeping the weights of the matched federated layer unchanged), and update the parameters of the corresponding subsequent layers of the local model based on the global parameters of the subsequent layers sent back. Preferably, the parameters of the subsequent layers of the model are updated layer by layer (progressively) until the parameters of all layers of the model are updated, thereby maintaining the adaptability of the local model to the local data.

[0086] In one example, after receiving the updated global weights issued by the server, the local client will compare these weights with the original weights before uploading, and selectively adjust and optimize the global weights based on the characteristics and needs of the local data, retaining the weight information that is highly relevant to the local data, while blurring or regularizing other less important weights to ensure that the model can adapt to the characteristics of local data while being globally optimized, thereby achieving personalized protection.

[0087] Specifically, if Figure 3 As shown in Figure 2, after applying the optimized global parameters to the corresponding layers of the local model, it includes:

[0088] Evaluate the importance of local model parameters, freeze the top P% of parameters in the local model, and use the optimized global parameters to replace the bottom 1-P% parameters in the local credit risk prediction model, where P is the parameter importance ratio threshold.

[0089] More specifically, after receiving the global weights, the client performs personalized processing on them, including the following sub-steps:

[0090] (1) Calculate the importance weight of each parameter :

[0091] ;

[0092] in, is the loss function, which measures the difference between the model's predicted value and the true value; Indicates parameter labels; is the total number of training steps, is the training step label; It is The client is in the local model Layer parameters; Indicates that when the parameter When a small change occurs, the loss function The rate of change of . During the training process, by calculating the partial derivatives And using optimization algorithms such as gradient descent, the model parameters can be updated to minimize the loss function .

[0093] (2) Organize the importance weights of all parameters into a matrix , used to guide subsequent parameter updates:

[0094] ;

[0095] in, is the total number of parameters.

[0096] (3) Freeze the top P% of parameters according to their importance weights and impose regularization constraints on the remaining 1-P% of parameters. The layer weights can be expressed as:

[0097] ;

[0098] ;

[0099] in, Indicates personalized treatment; Represents the global model Layer parameters; represents the learning rate; Represents the regularized loss function, and its calculation expression is:

[0100] ;

[0101] in, Represents the weight coefficient of the regularization term; the client updates After the layer weights are calculated, continue training subsequent layers on local data. , until the last layer .

[0102] After receiving the updated global weights, this example client performs personalized protection and regularization constraints to ensure the model's adaptability to local data while protecting the local model's key parameters (the top P% parameters) to avoid the negative impact of global updates on local model performance. Furthermore, by limiting the fluctuation range of important local model parameters, this achieves resource-efficient model updates, avoids frequent global updates and large amounts of data transfer, and significantly reduces computational and communication costs, making it particularly suitable for resource-constrained scenarios.

[0103] In one example, after sending the optimized global parameters to each client, the following steps are also included:

[0104] The final weight of the global model is weighted averaged according to the category distribution of the local lending sample data to update the global model parameters.

[0105] Specifically, steps S1-S3 are personalized federated collaboration steps. Uploading the parameters of any layer to the local model, the server calculating the global parameters of the corresponding layer and sending the global parameters back to the local model is defined as a round of communication. In each round of communication, the final global parameters (final weights) of the global model are weighted averaged according to the category distribution of the local loan sample data to adapt to the global data distribution. The calculation expression is:

[0106] ;

[0107] in, represents the training process; Represents the client local datasets;

[0108] After each round of communication, the client performs a weighted average of the global model based on the category distribution of local data to build a model that is more adapted to the global data distribution. Assume that the weight of each client is , then the final global parameters (final weights) of the global model processed by weighted averaging are for:

[0109] ;

[0110] in, It can be adjusted based on the amount of client data or other factors.

[0111] Optionally, the global model parameter update includes the following sub-steps:

[0112] (1) The client uploads category distribution information to the server;

[0113] (2) The server calculates the weight of each client based on the category distribution information provided by all clients. For example, if a client’s local data has a higher proportion of defaulting customers, the server may assign a higher weight to the model parameters of that client.

[0114] (3) The server uses the calculated weight ratio to perform a weighted average of the model parameters of all clients to generate new global model parameters.

[0115] Alternatively, the above global model parameter updating includes the following sub-steps:

[0116] (1) Each client calculates the weight ratio based on the category distribution of local data; for example, the client can assign weight ratios to local model parameters based on the proportion of defaulting customers in the local data.

[0117] (2) The client uploads the weighted model parameters to the server;

[0118] (3) The server aggregates the weighted parameters uploaded by all clients to generate new global model parameters.

[0119] This example uses the final global parameter update method described above to address the performance degradation caused by directly averaging weights in traditional federated learning, further improving the model's adaptability to local data and global optimization capabilities.

[0120] Optionally, the above examples can be combined. In this case, personalized federated learning includes the following steps:

[0121] (1) Initialize the global model and local model. In each round of federated learning, the server sends requests to all participating clients k to collect the weight information of each client's current layer. These weights are the model parameters trained on the client's local data and reflect the characteristics and patterns of the local data.

[0122] (2) After receiving the weights of all clients, the server analyzes these weights through a matching algorithm. The purpose of the matching algorithm is to identify the similarities between the weights of different clients and thus determine the contribution of each client weight to the global model. Based on the matching results, the server calculates the global weight and distributes it back to all clients. The global weight is a comprehensive representation of all client weights, designed to balance the contributions of different clients while optimizing the performance of the global model.

[0123] (3) After receiving the global weights, client k will make personalized adjustments to the global weights based on the characteristics of the local data. The purpose of this process is to protect the key parameters of the local model and avoid the negative impact of global updates on the performance of the local model. By analyzing the importance of local data, the client selectively retains or adjusts the global weights to ensure the adaptability of the model to the local data. After completing the personalized adjustment, the client gradually optimizes the parameters of the subsequent layers of the model in a layer-by-layer progressive manner until the parameters of all layers of the model are updated.

[0124] In one example, if Figure 4 As shown, the local model is incrementally updated based on the training dataset of the current stage, that is, the local model is continuously trained, including:

[0125] When the set model update cycle arrives, preprocess the newly added credit sample data available for training during the update period to obtain the training data set for the current model training phase. If it is the first training phase, preprocess the existing credit samples to build the local model corresponding to the first training round.

[0126] If it is not the first training stage, the current round of training data set is used to optimize the parameters of the local model obtained in the previous round of training. During incremental training, the parameter knowledge extracted from the local model obtained in the previous training stage is used to optimize the parameters, obtain the local model corresponding to the current training stage, and extract the parameter knowledge of the model.

[0127] In this example, incrementally updating the local model not only improves the model's adaptability to new data but also reduces the time and computational resource consumption required for model retraining. Furthermore, this invention addresses data imbalance by using data augmentation techniques or other methods to address this issue, significantly improving the model's ability to identify minority class samples and further enhancing its robustness and reliability.

[0128] Furthermore, parameter optimization is performed using the parameter knowledge extracted from the local model obtained in the previous training stage, specifically including: comparing the parameters of the model in the current training stage with the parameter knowledge extracted from the local model obtained in the previous stage, incorporating the difference between the two into the loss function of the model parameter optimization in the current stage, and making the parameters with larger weights in the credit risk prediction model obtained in the previous stage have larger weights in the loss function of the current stage. When minimizing the loss function of the current stage, the range of parameter changes is limited because the difference between the parameters of the new and old stages is taken into consideration in the loss function. The more important parameters in the previous stage have larger weights in the current loss function, and compared with general parameters, they have a greater impact on the loss function of the current stage. By minimizing the loss function, strict restrictions on the range of changes in different parameters are achieved, thereby retaining the knowledge learned in the previous stage.

[0129] Two-stage 、 For example, the total training data set of the two stages Contains training data sets corresponding to the two stages and Build phase After the local model is constructed, the EWC method is used to transfer the model knowledge to the stage . Weight information of model parameters As knowledge, Extracted by EWC and passed to the stage , restriction phase Parameter weight information The specific principle of constructing the loss function is as follows:

[0130] ① According to the Bayesian formula, use the parameter The prior probability of , marginal probability , conditional probability , the posterior probability can be calculated by the following formula :

[0131] .

[0132] ②Due to data set Completely divided into stages and stage , the posterior probability calculation formula is equivalent to:

[0133] ;

[0134] in, Representation stage The conditional probability of Representation stage The posterior probability of Representation stage The marginal probability of .

[0135] ③Since each subset affects the posterior probability, according to the Laplace approximation method, the loss function of EWC can be expressed as:

[0136] ;

[0137] in, represents the total loss; Representation stage losses; Based on the data set Calculate the Laplace approximation of the posterior probability; is the relative importance of the stages; Indicates the current model parameters The value of the parameter; Indicates that in stage The parameters learned The value of .

[0138] Stage 4 When building the model, the parameters are in the stage The loss in is part of the total loss, and the parameters are relative to the stage parameter The change of is another part of the total loss. The final selected parameters are to make To obtain the minimum parameters, ideally, the stage Have better performance; at the same stage The more important the parameter is, the smaller the change is compared to the current parameter.

[0139] Leveraging a continuous learning framework, knowledge contained in existing models is transferred to support the next stage of modeling. In credit risk prediction scenarios, to balance model performance and prediction accuracy, a regularization-based approach called elastic weight consolidation is employed to achieve continuous learning. This approach transfers knowledge learned in previous stages to the next stage by suppressing the fluctuations in important parameters, providing a solution for improving local model performance in complex financial environments.

[0140] In one example, after applying the optimized global parameters to the corresponding layers of the local model, freezing the corresponding layer parameters, and completing the parameter update of all layers of the local model, the following steps are also included:

[0141] Determine whether the set local model update cycle has been reached. If so, determine whether the number of updates to the local model is an integer multiple of the constant S. If so, execute steps S1-S3, i.e., perform personalized federated learning; if not, incrementally update the local model and perform the credit risk prediction task.

[0142] Here, S represents the condition that triggers global model aggregation after a certain number of local model updates. For example, if S is 5, it means that after every 5 local model updates, the client will upload the local model parameters to the server for global model aggregation.

[0143] Furthermore, the model cycle here can be a fixed time period (such as half a day or a day). Since the amount of loan sample data obtained in each fixed time period may vary greatly, for example, the amount of newly added loan sample data in a certain period is relatively small, while the amount of newly added loan sample data in another period is relatively large, in order to timely update the local model to obtain more accurate predictions, the time when the current loan sample data available for incremental training reaches a certain amount (threshold data amount) can be used as an update cycle.

[0144] This example sets the local model update cycle and determines whether the number of local model updates is an integer multiple of S. It then decides whether to perform personalized federated learning or incremental updates, ensuring the timeliness and efficiency of model updates. The dynamically updated local model is then used to output credit risk prediction results, providing decision support for financial institutions.

[0145] Combining the above examples, a preferred example of the method of the present invention is obtained, which includes the following steps:

[0146] S100: Initialize the local model and the global model;

[0147] S200: The client uploads all layer weights of the local model;

[0148] S300: The server aggregates the weights of each layer of the local model uploaded by the client based on a matching algorithm, and then optimizes the global weights of each layer of the global model;

[0149] S400: The client performs personalized federation on the global weight sent back by the server;

[0150] S500: Incrementally update the credit risk prediction model based on the current training dataset;

[0151] S600: Execute the credit risk prediction phase and output the prediction results;

[0152] S700: Determine whether the set local model update cycle has been reached. If so, determine whether the number of local model updates is an integer multiple of S. If so, execute steps S200-S400 for personalized federation. Otherwise, return to step S500 to start a new stage. After completing all training, execute step S600.

[0153] During local model training, the present invention initializes the local model and the global model. Each round of model training is then considered a training phase. For the first training phase, existing loan sample data is preprocessed to obtain a corresponding training dataset. Data preprocessing cleans, transforms, and prepares the raw data to make it suitable for model training. This includes addressing missing values, outliers, and duplicates, selecting the most relevant features, scaling and encoding features, and performing data conversion and dimensionality reduction, thereby improving model training efficiency and generalization. For subsequent training phases, the corresponding training datasets are obtained by preprocessing newly added credit sample data within the update period (the interval between the previous training round and the current training round). After each training round, the extracted model parameter knowledge is passed to the next phase. During the model training process, the next phase combines the parameter knowledge extracted from the credit risk prediction model obtained in the previous training phase to perform parameter optimization, guiding model updates and achieving continuous model learning. The model application incorporates a federated approach: at the beginning of each round, a determination is made as to whether the set local model update period has been reached. If the update cycle is reached, the system further determines whether the local model has been updated an integer multiple of S times. If it is an integer multiple of S times, then return to step S200 and re-perform personalized federated learning to further optimize the local model. In the process of the client uploading the model weights layer by layer for personalized federation, an innovative personalized federated learning mechanism is used to achieve a balance between global optimization and local personalization of the model. Specifically, the client first uploads the current layer weights of its own model to the server; after the server receives the weights of all clients, it analyzes the similarity of these weights through a matching algorithm and calculates the weight of the global model. Subsequently, the server issues the global weights back to each client. After receiving the global weights, the client makes personalized adjustments to the global weights based on the characteristics of the local data. Through this strategy of layer-by-layer matching aggregation and personalized federation, the system not only improves the adaptability of the model to local data, but also significantly improves communication efficiency and reduces communication rounds.

[0154] Based on the above method, the present invention not only significantly improves the accuracy, adaptability and flexibility of credit risk prediction, but also effectively solves the problems of data privacy protection, model performance degradation, low communication efficiency and data imbalance existing in the existing technology, providing a more efficient, reliable and practical solution for the field of credit risk prediction.

[0155] The above specific implementation methods are detailed descriptions of the present invention. It cannot be considered that the specific implementation methods of the present invention are limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, they can make several simple deductions and substitutions without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A credit risk prediction method based on personalized federated continuous learning, characterized by: The method comprises the following steps: Each client uploads all layer parameters of its local model to the server; the local model is a credit risk prediction model trained based on local loan sample data; the local loan sample data includes loan-related information, borrower basic information, and credit assessment data. The credit assessment data includes loan grade, credit score, earliest credit record time, and repayment and history records. The repayment and history records include loan disbursement date, last repayment date, and total amount paid. The server receives all layer parameters of the local model uploaded by each client, and analyzes all layer parameters of each local model layer by layer through a matching algorithm to obtain the contribution of each layer parameter of each local model to the global parameters of the corresponding layer of the global model. Then, the server optimizes the global parameters of each layer of the global model based on the contribution and sends the optimized global parameters to each client; The client receives the optimized global parameters and applies them to the corresponding layers of the local model; Incrementally update the local model based on the training dataset of the current stage; Use the local model that has been incrementally updated to perform credit risk prediction tasks and output the prediction results; Apply the optimized global parameters to the corresponding layers of the local model, including: Assess the importance of local model parameters, freeze the top P% of parameters in the local model, and use the optimized global parameters to replace the bottom 1-P% of parameters in the local credit risk prediction model, where P is the parameter importance ratio threshold; After receiving the global weights, the client performs personalized processing, including: (1) Calculate the importance weight of each parameter : ； is the loss function; Indicates parameter labels; is the total number of training steps, is the training step label; It is The client is in the local model Layer parameters; Indicates that when the parameter When a small change occurs, the loss function rate of change; (2) Organize the importance weights of all parameters into a matrix : ； is the total number of parameters; (3) Freeze the top P% of parameters according to their importance weights and impose regularization constraints on the remaining 1-P% of parameters. The layer weights are expressed as: ； ； Indicates personalized treatment; Represents the global model Layer parameters; represents the learning rate; represents the regularization loss function; After sending the optimized global parameters to each client or applying the optimized global parameters to the corresponding layer of the local model, it also includes: The final global parameters of the global model are weighted averaged according to the category distribution of the local loan sample data to update the global model parameters; the category distribution of the local loan sample data is the proportion of defaulting customers; In each round of communication, the server performs weighted averaging on the final global parameters of the global model based on the category distribution of local loan sample data. The calculation expression is: ； in, represents the training process; Represents the client local datasets; After each round of communication, the client performs a weighted average of the global model based on the category distribution of local data; let the weight of each client be , the final global parameters of the global model processed by weighted averaging for: ； in, It can be adjusted based on the amount of client data or other factors.

2. The credit risk prediction method based on personalized federated continuous learning according to claim 1 is characterized in that: The local model uses elastic weight consolidation loss, or cross entropy loss and elastic weight consolidation loss to optimize model parameters.

3. The credit risk prediction method based on personalized federated continuous learning according to claim 1 is characterized in that: The incremental update of the local model based on the training dataset of the current stage includes: During the update cycle, the newly added loan sample data is preprocessed to obtain the training data set for the current round. The local model obtained from the previous round of training is incrementally trained using the current round of training data set. During the incremental training process, the parameter knowledge extracted from the local model obtained from the previous training round is used to perform parameter optimization to obtain the local model corresponding to the current training round. Utilize the parameter knowledge extracted from the local model obtained from the previous training round to perform parameter optimization, including: The parameters of the local model in the current round of training task are compared with the parameter knowledge extracted from the local model obtained from the previous round of training task, and the parameter differences obtained from the comparison are incorporated into the loss function of the model parameter optimization of the current round of model training task, so that the parameters with larger weights in the local model obtained from the previous round of training task maintain larger weights in the loss function of the current round of training task.

4. The credit risk prediction method based on personalized federated continuous learning according to claim 1 is characterized in that: After applying the optimized global parameters to the corresponding layers of the local model, the method further includes: Determine whether the set local model update cycle has been reached. If so, determine whether the number of updates to the local model is an integer multiple of the constant S. If so, return to the step where each client uploads all parameters of its local model to the server until the optimized global parameters are applied to the corresponding layer of the local model. If not, perform incremental updates to the local model and execute the credit risk prediction task.

Images