Privacy protection credit risk assessment method and system based on federated learning

Abstract

The application provides a privacy protection credit risk evaluation method and system based on federated learning, and belongs to the technical field of data processing and privacy protection. In the method, each participant performs data preprocessing and feature engineering locally, and participates in horizontal federated training by taking a logistic regression as a base classifier; each participant performs Top-k sparsification on a gradient and generates a bitmap, uploads the sparse gradient to an aggregation server by using homomorphic encryption, and the server decrypts and updates the model after aggregation in a ciphertext domain; during the aggregation process, the weight is adjusted by performing distribution calibration, the learning rate is dynamically adjusted, the norm of the ciphertext gradient is verified, and the loss divergence is monitored; after the training converges, the model parameters are published after injecting differential privacy noise; and each participant deploys the model to predict new samples. The application realizes the collaborative optimization of privacy protection and model precision, and improves the communication efficiency and system robustness.

Description

Technical Field

[0001] This invention relates to the field of data processing and privacy protection technology, specifically to a privacy-protected credit risk assessment method and system based on federated learning. Background Technology

[0002] In distributed machine learning scenarios, multiple data holders aim to collaboratively train machine learning models without sharing the original data to improve model performance. Taking commercial bank credit risk assessment as an example, customer data from different banks is stored locally and cannot be physically centralized, resulting in a limited sample size for each institution and making it difficult to train a credit risk assessment model with good generalization ability. Horizontal federated learning, as a technical framework that allows multiple participants to collaboratively train models without leaving their local data locations, provides a feasible solution to these problems.

[0003] In the field of horizontal federated learning, participating clients and the central server need to frequently transfer gradient data to complete the aggregation and update of the global model. However, existing horizontal federated learning systems generally suffer from the following technical problems: First, there is a risk of information leakage during gradient transmission. In a horizontal federated learning framework, participants need to upload their locally computed gradients to a central server for aggregation. The gradients themselves carry statistical information from the participants' local training data. Existing research shows that attackers can reconstruct the features and even labels of the original training samples from the transmitted gradients through gradient inversion attacks. While existing technologies employ homomorphic encryption to encrypt gradient transmission, the encrypted object is the complete gradient vector. Once the ciphertext is cracked or an attacker obtains the plaintext gradient through other means, they can use the complete gradient information to launch an inversion attack. Current technologies generally do not destructively process the information structure of the gradient itself, resulting in a gradient transmission stage constituting an attack surface prone to information leakage.

[0004] Secondly, non-independent identically distributed (Non-IID) data leads to difficulties in global model convergence. In the actual deployment of horizontal federated learning, the local data distribution of each participant often differs significantly. Taking credit risk assessment as an example, the customer groups of different financial institutions naturally differ in terms of income level, geographical distribution, and credit behavior, which may cause significant discrepancies in the gradient directions calculated by each participant under the same global model parameters. Existing technologies generally use fixed weights to perform weighted averaging aggregation of the gradients from each party, and update the model with a fixed learning rate, without fully considering the impact of differences in the data distribution of each participant on the aggregation direction. When the data distribution of the participants is severely skewed, directly averaging the gradients of each party with fixed weights will cause the global model update direction to be dominated by the gradients of the participants with skewed distributions, deviating from the optimal solution, manifesting as loss oscillations, decreased convergence speed, or even non-convergence.

[0005] Third, the trained model parameters lack privacy protection. Current horizontal federated learning techniques primarily focus on data transmission security during training, lacking formal privacy protection mechanisms for the global model distribution phase after training. The trained model parameters themselves have a memory effect, potentially containing sensitive information from the training dataset. Attackers can use the distributed model to perform membership inference attacks to determine whether a specific sample participated in training, or to reconstruct the statistical distribution of the training data through model inversion attacks. These attacks can be carried out with the model parameters completely public, without needing to obtain any information transmitted during the training process. Summary of the Invention

[0006] This invention addresses the technical problems existing in the application of existing horizontal federated learning technology in credit risk assessment by providing a privacy-preserving credit risk assessment method and system based on federated learning.

[0007] The privacy-preserving credit risk assessment method based on federated learning provided by this invention is applied to a distributed system consisting of multiple commercial bank participating clients, an aggregation server, and two non-collusive auxiliary computing nodes. The original customer credit data is always stored locally at each commercial bank. The method specifically includes the following steps: Step 1: Local data preprocessing and feature engineering for each participant Each participating commercial bank performs feature engineering on the original customer credit data locally. The original repayment behavior field is labeled with binary values, and samples with overdue days exceeding a set threshold are labeled as defaulting, samples with no overdue records are labeled as normal, and samples with intermediate overdue days within a set range are removed from the training set. This formalizes credit risk assessment into a binary classification supervised learning task.

[0008] For continuous feature variables, a decision tree binning method based on the CART algorithm is used for discretization. The degree of Gini coefficient decrease at each candidate split point is recursively calculated, and the split point with the largest decrease in Gini coefficient is selected as the optimal split point. After splitting, the above process is recursively executed on the subset until the stopping condition is met. The binning results satisfy the sample size ratio constraint and monotonicity constraint to ensure that the feature transformation results have statistically stable ranking. The discretized variables after binning are converted into numerical features using evidence weight encoding, so that the encoded features naturally match the log-odds output of the logistic regression model on a numerical scale.

[0009] Feature derivation is performed based on business implications, generating derived features statistically according to different time windows. Initial feature screening is conducted on the original and derived features using missing rate analysis, concentration analysis, information value screening, and correlation analysis. Then, stepwise regression based on Bayesian information criteria is used for bidirectional feature dimensionality reduction, selecting core variables for model input from the original variables. Z-score standardization is applied to the selected features to eliminate dimensional differences between features and accelerate model convergence. During training, each participant maintains the mean and standard deviation of the features locally, updating local statistics using a sliding window moving average method and uploading them to the server for weighted aggregation based on sample size. At the end of training, the global sliding window moving average of the final set number of communication rounds is solidified as a globally standardized statistic.

[0010] Step 2: Construct a horizontal federated logistic regression training task and perform distributed collaborative training. Each participant uses a logistic regression model as the base classifier and collaboratively trains the global credit risk assessment model using a federated stochastic gradient descent algorithm. Each participant calculates the gradient of the current global model parameters with respect to its local samples; the gradient expression is obtained by differentiating the cross-entropy loss function with respect to the weight parameters.

[0011] Each participant performs Top-k sparsity processing on the gradient vector: sorting the gradient components in descending order of their absolute values, retaining the largest component (previously set to a predetermined proportion), and setting the remaining components to zero. Simultaneously, a sparse mode bitmap is generated, using a 0 / 1 bit sequence to mark the position index of the retained components. Gradient components set to zero are not discarded but accumulated in the local error buffer vector. In the next round of local gradient calculation, the new gradient is added to the error buffer vector before sparsification is performed, ensuring that small gradient information that was delayed in transmission still participates in the global update. Only the retained sparse gradient components are encrypted and uploaded; the sparse mode bitmap is transmitted in plaintext.

[0012] Each participant uses the Paillier homomorphic encryption public key distributed by the aggregation server to encrypt the gradient components retained after sparsification one by one. Each gradient component uses a different random number to ensure that the ciphertexts of different rounds of the same participant cannot be linked. After obtaining the ciphertext gradient, it is uploaded to the aggregation server along with the sparse mode bitmap.

[0013] After receiving the encrypted gradients and sparse pattern bitmaps uploaded by each participant, the aggregation server aligns the encrypted component positions according to each participant's bitmap, and aggregates the encrypted gradient components of each participant by exponentiation according to the integer weights at the corresponding positions to obtain the aggregated encrypted gradient. The aggregation server decrypts the aggregated encrypted gradient using its private key, divides the decryption result by the quantization factor to restore the plaintext aggregated gradient value, updates the global model parameters according to the federated stochastic gradient descent rule, and distributes the updated global parameters to each participant.

[0014] Before each round of aggregation, the aggregation server performs distribution calibration: it calculates the deviation metric between the proportion of default classes in each participant's local sample and the global proportion of default classes for all participants weighted by sample size. When the deviation metric exceeds a set multiple of the standard deviation of the global category distribution, the participant is deemed to have a significant distribution shift, and its aggregation weight is reduced according to a decay function with the deviation metric as the independent variable. Participants whose weights do not exceed the threshold retain their original sample size weights. After the weights of all participants are calculated, normalization is performed.

[0015] During training, the aggregation server dynamically adjusts the learning rate based on changes in the global gradient norm. The initial learning rate adaptively decays as the ratio of the L2 norm of the current round's aggregated gradient to the L2 norm of the first round's aggregated gradient decreases. The aggregation server performs L2 norm verification on the encrypted gradients uploaded by each participant: the encrypted gradient vector to be verified is split and sent to two non-colluding auxiliary computing nodes. The two auxiliary nodes collaboratively calculate the sum of squares of the components in the encrypted domain based on a secret sharing protocol. The aggregation server decrypts the sum of squares to obtain the gradient norm of each participant and calculates the median of the gradient norms of all participants in this round. If the gradient norm of a participant exceeds a set multiple of the median, the participant's gradient is determined to be abnormal, and its participation in this round of aggregation is rejected. An alarm is issued to the participant. Participants that are determined to be abnormal for a set number of consecutive rounds are removed from the current training task.

[0016] The aggregation server monitors the divergence of the local loss values ​​reported by each participant. The criteria for judgment are: a monotonically increasing loss value for a set number of consecutive communication rounds with a cumulative increase exceeding a set percentage, or a single round's increase exceeding a set percentage, or the current loss value exceeding a set multiple of its initial round's loss value. For participants judged to be diverging, their gradient participation in global aggregation is suspended. These participants continue local training at a set percentage of their original learning rate. They are rejoined in aggregation when their loss value decreases for a set number of consecutive rounds and the current loss value is lower than a set percentage of the loss value at the time of suspension. If the recovery conditions are not met within a set number of subsequent communication rounds, the participant is removed from the current training task.

[0017] During training, the aggregation server dynamically adjusts the ratio of communication frequency to local training rounds for each participant based on the rate of loss function decline: when the rate of loss decline is higher than the first threshold, the number of local training rounds is increased and the communication frequency is decreased; when the rate of loss decline is lower than the second threshold, the number of local training rounds is decreased and the communication frequency is increased; when the rate of loss decline is between the two thresholds, the current ratio is maintained.

[0018] The above training steps are iterated repeatedly until the change in the loss function is less than a preset threshold for a continuous set number of rounds or the preset maximum number of communication rounds is reached.

[0019] Step 3: Perform differential privacy post-processing on the trained global model. After training converges, the aggregation server injects Gaussian noise that satisfies differential privacy conditions into the global model parameters. The noise scale is determined based on the effective sensitivity, which is twice the gradient clipping threshold divided by the product of the number of participants and the average sample size. The more participants and the larger the data volume, the more diluted the influence of a single participant on the global model, and the smaller the required noise. The noisy global model parameters are then distributed to each participant as the final published model.

[0020] Step 4: Each participant deploys the predictive model and performs a credit risk assessment. Each participating party will deploy the global model parameters on the local prediction system. When new customer data is input, the input features are binned according to the binning rules fixed during the training phase, encoded and transformed according to the fixed evidence weight encoding rules, and then Z-score standardization is performed using fixed global standardized statistics. The input is then fed into the logistic regression model to calculate the default probability output value. The default probability is transformed into an integer credit score through a monotonic mapping function and output as the credit risk assessment result to the downstream risk control system for use.

[0021] The privacy-preserving credit risk assessment system based on federated learning provided by this invention includes: Multiple commercial bank participating client clients are deployed locally at each commercial bank. Each participating client includes: The feature engineering module is used to perform label binarization, decision tree binning, evidence weight encoding, feature derivation, feature selection and Z-score standardization on the original customer credit data locally. It updates the local feature statistics using a sliding window moving average method and uploads them to the server to participate in global statistics aggregation. At the end of training, the global standardized statistics are solidified.

[0022] The federated training module is used to participate in horizontal federated collaborative training with a logistic regression model as the base classifier. It calculates the gradient of the current global model parameters locally, performs Top-k sparsification on the gradient vector and generates a sparse pattern bitmap, encrypts the retained sparse gradient components with the homomorphic encryption public key distributed by the aggregation server to generate ciphertext gradients and uploads them to the aggregation server, receives the global model parameters issued by the aggregation server and updates the local model, calculates and reports the local loss value, and performs local recovery training when the loss diverges.

[0023] The prediction module is used to deploy the published global model parameters to the local prediction system, perform binning, evidence weight encoding and Z-score standardization on the new input samples, calculate the default probability of the input model, and perform score mapping output.

[0024] An aggregation server, deployed at the central node, includes: The key management module is used to generate Paillier homomorphic encryption key pairs, distribute the public key to each participating client, and retain the private key.

[0025] The ciphertext aggregation module is used to receive the ciphertext gradient and sparse mode bitmap uploaded by each participating client, align the positions of the ciphertext components according to the bitmap, and perform weighted aggregation in the ciphertext field to obtain the aggregated ciphertext gradient.

[0026] The decryption and update module is used to decrypt the aggregated ciphertext using the private key to restore the plaintext aggregated gradient, update the global model parameters according to the federated stochastic gradient descent rule, and distribute them to the clients of each participating party.

[0027] The distribution calibration module is used to calculate the deviation metric between the local category distribution and the global category distribution of each participant. When the deviation metric exceeds the set threshold, the aggregate weight of the participant is adjusted according to the decay function, and the weights of all participants are normalized.

[0028] The learning rate adjustment module is used to dynamically adjust the learning rate based on changes in the global gradient norm.

[0029] The gradient verification module is used to split the ciphertext gradient vector to be verified and send it to two auxiliary computing nodes. It receives the ciphertext domain sum of squares returned by the auxiliary computing nodes and decrypts it to obtain the gradient norm of each participant. Participants with abnormal gradient norms are rejected from participating in this round of aggregation.

[0030] The loss detection module is used to monitor the divergence of local loss values ​​reported by each participant, and to trigger a pause aggregation and recovery process for participants that are determined to be diverging.

[0031] The communication scheduling module is used to dynamically adjust the ratio of local training rounds to global communication frequency for each participant based on the rate of decrease of the loss function.

[0032] The differential privacy module is used to inject Gaussian noise that satisfies the differential privacy condition into the global model parameters after training convergence, generate the final published model, and distribute it to the clients of each participating party.

[0033] Two non-colluding auxiliary computing nodes are deployed in independent computing environments and communicate with the aggregation server. They are used to collaboratively calculate the sum of squares of gradient components in the ciphertext domain based on the secret sharing protocol and return it to the aggregation server when the aggregation server performs norm verification on the ciphertext gradients uploaded by each participant. Neither auxiliary computing node can know the plaintext gradient information on its own.

[0034] Each participating client communicates with the aggregation server via an encrypted channel, and the original customer credit data is always kept locally on each participating party's premises and is not transmitted externally.

[0035] Compared with the prior art, this application has the following beneficial effects: 1. Privacy Protection Mechanism Existing technologies employ a single-layer protection mechanism, transmitting encrypted gradients as complete gradient vectors. Once the ciphertext is cracked, attackers can obtain the complete gradient information for inversion attacks. This application constructs a three-layer privacy protection system: before training, Top-k sparsity is used to non-linearly truncate the gradient, retaining only the component with the largest amplitude, making the transmitted gradient inherently incomplete and violating the information integrity condition required for gradient inversion attacks; during training, Paillier homomorphic encryption is used, mathematically ensuring strict equivalence between ciphertext domain aggregation and plaintext domain aggregation; after training, differential privacy noise is injected into the global model parameters, providing formal privacy protection for the model deployment stage. These three layers correspond to three different attack surfaces: gradient transmission, server aggregation, and model deployment. Compared to the single-point protection of existing technologies, this application achieves end-to-end privacy protection.

[0036] 2. Data heterogeneity adaptability Existing technologies employ fixed sample size ratios and fixed learning rates, failing to consider convergence issues caused by non-IID data distributions of each participant. This application performs distribution calibration before aggregation, using a class distribution bias metric to identify significantly biased participants based on the 2σ criterion, and smoothly suppresses their aggregation weights according to a bounded decay function. Simultaneously, it adaptively adjusts the learning rate based on the relative change in the global gradient norm. These two aspects synergistically form a combined adjustment loop of "weight calibration + adaptive step size." Existing technologies lack technical guidance for combining distribution calibration with dynamic learning rate adjustment.

[0037] 3. Communication efficiency Existing technologies transmit the complete encrypted gradient vector, and the homomorphic encryption ciphertext expansion leads to high communication overhead. This application performs gradient sparsification before encryption, transmitting only the sparse components and bitmaps, reducing the ciphertext transmission volume by approximately 80%. Furthermore, it combines adaptive scheduling of communication rounds and local rounds, allowing communication resources to be concentrated on the critical training phase where loss decreases rapidly. Existing technologies do not disclose any of the aforementioned communication optimization methods.

[0038] 4. System robustness Existing technologies lack mechanisms for identifying and isolating distorted gradients uploaded by malicious participants. This application utilizes two non-colluding auxiliary computing nodes to perform gradient norm verification in the ciphertext domain, using multiples of the median to identify abnormal participants and reject their participation in aggregation. Simultaneously, it incorporates loss divergence detection and recovery mechanisms to ensure that anomalies from a single party do not affect the global model training.

[0039] The four technical features mentioned above are not simply superimposed, but rather form an organic whole around the core contradiction of "coordinated optimization of privacy protection and model accuracy," with inherent technical connections and mutual support among the features: Synergy between gradient sparsification and homomorphic encryption: Sparsification directly alleviates the ciphertext expansion problem of homomorphic encryption by reducing the number of encryption components, making Paillier encryption feasible in bandwidth-constrained scenarios; at the same time, the nonlinear truncation of sparsification provides additional security redundancy for homomorphic encryption, and the two mutually enhance each other in terms of communication efficiency and security.

[0040] The synergy between distribution calibration and dynamic learning rate: Distribution calibration suppresses the interference of gradients from offset participants on the aggregation direction, making the change in the global gradient norm more realistically reflect the model convergence process; the dynamic learning rate adjusts the step size based on the calibrated gradient norm, avoiding oscillations caused by residual distribution differences after calibration. These two elements constitute a closed-loop adjustment of "direction calibration + adaptive step size," neither of which can be omitted.

[0041] The vertical complementarity between differential privacy and sparsity / homogeneous encryption: Sparsity and homomorphic encryption protect gradient transfer and aggregation during training, while differential privacy protects the model deployment stage after training. Encryption during training cannot prevent model parameters from being used by member inference attacks, and differential privacy cannot protect the security of gradient transfer during training. These three mechanisms are vertically integrated in both the time and attack surface dimensions, forming a complete closed loop. Attached Figure Description

[0042] Figure 1 This is a flowchart of the method of the present invention; Figure 2 This is a schematic diagram of the gradient sparsity and homomorphic encryption transmission process. Figure 3 This is a schematic diagram of the distributed calibration process; Figure 4 This is a schematic diagram of the dynamic learning rate adjustment process; Figure 5 This is a schematic diagram of the ciphertext field gradient verification and anomaly detection process; Figure 6 This is a schematic diagram of the loss divergence detection and local recovery process; Figure 7 This is a schematic diagram of the differential privacy post-processing and model deployment process; Figure 8 This is a schematic diagram of the overall system architecture of the present invention. Detailed Implementation

[0043] Definitions: Horizontal federated learning refers to a federated learning scenario in which multiple participants have the same feature space but different sample spaces, and the data of each participant are horizontally segmented in the sample dimension.

[0044] Non-IID refers to non-independent and identically distributed data, meaning that the category and feature distributions of the local data of each participant are significantly different, and the independent and identically distributed hypothesis is not satisfied.

[0045] Paillier homomorphic encryption is an additive homomorphic encryption algorithm based on the compound residue class hard problem. It supports addition operations on ciphertext in the ciphertext field, and the decryption result is consistent with the result of performing the same addition operation on the corresponding plaintext.

[0046] Top-k sparsification refers to the operation of sorting a vector in descending order of the absolute value of its components, retaining only the top k largest components, and setting the remaining components to zero.

[0047] Evidence weight coding is a coding method that transforms discrete variables into numerical variables. Its value is the natural logarithm of the ratio of good customers to bad customers in the bin.

[0048] Information value is an indicator that measures the predictive power of an independent variable on a dependent variable. It is equal to the sum of the difference between the percentage of good customers and the percentage of bad customers in each sub-box multiplied by the corresponding evidence weight values.

[0049] The Gini coefficient is a measure of the purity of a dataset; the smaller the value, the higher the purity of the dataset.

[0050] The sparse mode bitmap is a data structure that records the indexes of the retained components in the gradient vector after sparsification in the form of a bit sequence. A bit value of 1 indicates that the component at that position is retained, and a bit value of 0 indicates that it is set to zero.

[0051] The error buffer vector is an accumulated vector used to store the gradient components that are set to zero in each round of sparsification. Its purpose is to allow small gradient information that is delayed in transmission to still participate in the global model update in subsequent rounds.

[0052] Distribution calibration refers to the process of adjusting the weights of each participant in gradient aggregation based on the degree of deviation between the local data distribution of each participant and the global data distribution, in order to reduce the interference of participants with severe distribution offset on the global model update direction.

[0053] Secret sharing refers to the technique of dividing secret information into several shares and distributing them to different participants. Only by collecting a sufficient number of shares can the secret information be recovered, and no single participant can obtain the complete secret information.

[0054] Differential privacy is a privacy protection framework that adds appropriate noise to query results, making it impossible for attackers to infer whether a specific sample is in the dataset by observing the output.

[0055] Example 1 like Figure 1As shown, this embodiment provides a privacy-preserving credit risk assessment method based on federated learning, which is applied to a distributed system consisting of multiple commercial bank participating clients, an aggregation server, and two non-collusive auxiliary computing nodes. The original customer credit data is always stored locally on each commercial bank's premises.

[0056] In this embodiment, the original customer credit data comes from personal housing loan and consumer loan business data in the core credit systems of various commercial banks, including basic customer information, loan contract information, repayment records, and credit inquiry records. The specific application scenario for credit risk assessment is the pre-loan approval stage of personal housing loans and consumer loans, and the assessment target is the natural person customer applying for the loan.

[0057] I. Local Data Preprocessing and Feature Engineering for Each Participant Each participating commercial bank acquires raw customer credit data locally, including customer repayment behavior fields and multiple dimensions of customer characteristic variables. Each participant performs label binarization: for the raw repayment behavior field, samples with overdue days greater than 30 days are labeled as 1, marking them as defaulting; samples with no overdue records are labeled as 0, marking them as normal; intermediate samples with overdue days between 1 and 30 days are removed from the training set, formalizing credit risk assessment as a binary classification supervised learning task. Each participant recalculates and maps local sample labels according to a uniformly agreed-upon overdue day threshold of 30 days. For participants with different historical label standards, the original labels are aligned to a unified threshold standard, ensuring that the label data input into the federated training by each participant is strictly consistent in statistical terms.

[0058] For continuous feature variables, a decision tree binning method based on the CART algorithm is used for discretization. The specific process is as follows: First, the median of adjacent element values ​​in the dataset is calculated as a candidate split point. For each candidate split point, the degree of decrease in the Gini coefficient when used as a split point is calculated. The Gini coefficient is calculated by subtracting the sum of the squares of the sample proportions of each category from 1. The split point with the largest decrease in the Gini coefficient is selected as the optimal split point, dividing the data into two. The above process is recursively executed on the split subsets until stopping conditions are met. The stopping conditions include that the sample size in each bin is not less than 5% of the total sample size, the number of bins is controlled between 2 and 6, and the binning results must meet the monotonicity requirement. The monotonicity requirement is determined by calculating the bad sample rate of each bin. If the bad sample rate of adjacent bins shows a non-monotonic fluctuation of first increasing and then decreasing or first decreasing and then increasing, adjacent bins that violate the monotonicity are merged until the bad sample rate of all bins increases or decreases monotonically with the variable value, ensuring that the feature transformation results have statistically stable ranking.

[0059] Discretized variables after binning are converted into numerical features using weighted evidence (WOE) encoding. The WEE is calculated as the natural logarithm of the ratio of good customers to bad customers in that bin. A higher WOE value indicates a higher proportion of good customers in that bin, while a lower WOE value indicates a higher proportion of bad customers. There is a strict mathematical connection between WEE encoding and the logistic regression model. The log odds of the binned samples and the WEE values ​​have a negative linear relationship, both operating on the log-odds scale. The encoded features naturally match the logit transformation of logistic regression on a numerical scale. The constant term is the prior log odds of the entire sample, which is automatically learned and absorbed by the intercept term of logistic regression during model training. The information value of each variable is calculated using WEE encoding. The information value is calculated by multiplying the difference between the proportion of good customers and the proportion of bad customers in each bin by the corresponding WEE value and then summing the results. This sum measures the variable's ability to distinguish the target variable. Generally, an information value less than 0.02 indicates that the variable has weak predictive power, between 0.02 and 0.1 is considered moderate, and greater than 0.1 indicates strong predictive power.

[0060] Based on business implications, features are derived using the number of first-time defaults as a foundation, and statistical analysis is conducted across five different time windows: the past 1 month, 3 months, 6 months, 9 months, and 12 months, generating derived features. A customer's historical delinquency behavior has different indicative significance for credit risk across different time windows. Short-window indicators reflect recent behavioral changes, while long-window indicators reflect long-term credit habits. The statistical analysis of the number of first-time defaults across different time windows reflects the timeliness differences in customer delinquency behavior. Short-window indicators (past 1 month and 3 months) capture recent trends in customer repayment behavior and are more sensitive to short-term default risk prediction; long-window indicators (past 9 months and 12 months) reflect customers' long-term credit habits and historical default frequency, providing a stronger ability to differentiate a customer's overall creditworthiness. The debt-to-income ratio directly measures the proportion of a customer's existing debt burden relative to their income level and is the most crucial debt repayment capacity indicator in credit risk assessment.

[0061] Feature selection is performed in two stages: initial feature screening and stepwise regression dimensionality reduction. Initial feature screening includes: missing rate analysis, calculating the missing percentage of each feature; features with a missing rate exceeding 80% are directly removed; features with a missing rate between 50% and 80% are retained based on correlation tests between the feature and the target variable, retaining only features with a chi-square test p-value less than 0.05; features with a missing rate between 30% and 50% are retained after median imputation; concentration analysis, checking if a single value accounts for more than 90% of a feature's total; features exceeding this are removed; information value screening, retaining features with an information value greater than 0.02; and correlation analysis, calculating the Pearson correlation coefficient between features. For highly correlated feature pairs with an absolute correlation coefficient exceeding 0.7, the feature with stronger predictive power is retained based on its information value. Stepwise regression dimensionality reduction uses a two-way selection method, with the Bayesian information criterion as the evaluation standard. The Bayesian information criterion is calculated as: negative twice the logarithm of the likelihood function plus the number of model parameters multiplied by the logarithm of the sample size. The two-way selection process is as follows: starting from an empty model, variables that are significant in the F test are gradually introduced, i.e., forward selection. The significance level of the introduced variables is set to 0.05. After each new variable is introduced, the variables already in the model are re-examined. If the significance of a certain variable decreases below the threshold due to the addition of the new variable, it is removed, i.e., backward elimination. The significance level of the removed variables is also set to 0.05. This process is repeated until there are no variables that need to be added or removed. The core variables for the model are then selected from the original variables.

[0062] Z-score standardization is performed on the selected features. The standardization method involves subtracting the mean of the feature from its original value and then dividing by its standard deviation to eliminate dimensional differences between features and accelerate model convergence. During training, each participant maintains the mean and standard deviation of its features locally and updates them using a moving average method with a sliding window. Specifically, after each communication round, the local statistics for that round are averaged with the statistics from the previous 10 communication rounds (within a window size W) to obtain smoothed local statistics. Each participant's smoothed statistics are encrypted using the same Paillier homomorphic encryption method as the gradient and then uploaded to the server. The server weights and aggregates the encrypted data by sample size and then decrypts it to obtain the global statistics. At the end of training, the system solidifies the global moving average of the last 10 communication rounds into the final global mean and standard deviation, which are distributed along with the model parameters.

[0063] II. Constructing a horizontal federated logistic regression training task and executing distributed collaborative training Each participant uses a logistic regression model as the base classifier and collaboratively trains the global credit risk assessment model using a federated stochastic gradient descent algorithm. The logistic regression model consists of a linear regression model combined with a sigmoid activation function. The linear regression model adds the weighted sum of the input features to the bias term. The sigmoid function is expressed as 1 divided by (1 plus e to the power of negative z), and its output value ranges from 0 to 1, representing the posterior probability that a sample belongs to the default class. The hypothesis function of the logistic regression model is the sum of the weighted sum of the input features and the bias term applied by the sigmoid function. The loss function uses cross-entropy loss, which is calculated as follows: for all samples, take the average of the sum of the negative label value multiplied by the logarithm of the predicted probability, plus the sum of the sum of the label value multiplied by the logarithm of the predicted probability. The gradient expression of the loss function with respect to the weight parameters is the predicted probability of all samples minus the label value, multiplied by the average of the corresponding feature values. Each participant independently calculates the above gradient locally using its own sample data.

[0064] The aggregation server employs the FedSGD algorithm as its aggregation strategy, performing a weighted average of the gradients uploaded by each participant. The weights are determined by the proportion of each participant's sample size to the total sample size after distribution calibration. The global objective function is defined as the weighted sum of the local objective functions of each participant, with the weights being the proportion of each participant's sample size to the total sample size. In the t-th round of communication, the server's global model weight update rule is: subtract the product of the learning rate and the aggregated gradient from the previous round's global model weights, where the aggregated gradient is the weighted sum of each participant's local gradients according to their calibrated aggregated weights.

[0065] like Figure 2 As shown, each participant performs Top-k gradient sparsity processing after completing local gradient calculations but before encryption. The specific steps are as follows: sort the components of the local gradient vector in descending order of absolute value; retain the top 20% of gradient components with the largest absolute value, set the remaining components to zero, and generate a sparse mode bitmap, marking the position index of the retained components with a 0 / 1 bit sequence; the gradient components set to zero are not discarded, but accumulated in the local error buffer vector. In the next round of local gradient calculation, the new gradient is added to the error buffer vector before sparsification is performed, ensuring that the small gradient information that is delayed in transmission still participates in the global update; only the retained sparse gradient components are encrypted and uploaded, and the sparse mode bitmap is transmitted in plaintext.

[0066] Each participant uses the Paillier homomorphic encryption public key distributed by the aggregation server to encrypt the gradient components retained after sparsification one by one. The key generation process of the Paillier algorithm is as follows: select two large prime numbers, calculate their product as the public key modulus, calculate the least common multiple of these two prime numbers minus one as the private key parameter, select a random number as the public key base, and calculate the other parameter of the private key through modulo operation. The encryption process is as follows: for the plaintext gradient component value, select a random number, calculate the plaintext power of the public key base, multiply by the random number raised to the power of the public key modulus, and then take the modulus of the square of the public key modulus. Each gradient component uses a different random number to ensure that ciphertexts from different rounds by the same participant cannot be linked. The decryption process is as follows: after performing a modulo exponentiation of the private key parameter on the ciphertext, transform it through a specific function, multiply by the other parameter of the private key, and then take the modulus of the public key modulus to obtain the plaintext gradient value. Floating-point gradient values ​​are first scaled by a common scaling factor of 10. 6 The data is quantized to an integer, decrypted, aggregated, and then divided by the factor to restore the original data. Each participant obtains the ciphertext gradient and uploads it along with the sparse pattern bitmap to the aggregation server.

[0067] After receiving the ciphertext gradients and sparse pattern bitmaps uploaded by each participant, the aggregation server aligns the ciphertext components according to the bitmaps of each participant and aggregates the ciphertext gradient components of each participant by exponentiation with integer weights at corresponding positions. Due to the mathematical properties of Paillier encryption, ciphertext multiplication corresponds to plaintext addition, and ciphertext exponentiation corresponds to plaintext scalar multiplication. The server exponentiations the ciphertexts of each participant according to their weights and then multiplies them together. The aggregated ciphertext corresponds to the weighted sum of the plaintext gradients of each participant. The random numbers independently selected by each participant during encryption are automatically combined into new valid random numbers after exponentiation aggregation and are completely eliminated by the Paillier decryption function during decryption. This does not affect the weighted gradient of the decrypted plaintext, ensuring that the results of ciphertext-domain weighted aggregation and plaintext-domain weighted aggregation are strictly consistent, i.e., lossless aggregation. To address the issue that the aggregation weights are decimals and cannot be directly used as ciphertext exponents, integer quantization is used, with a common quantization factor of 10. 6 Each weight is multiplied by the factor and rounded to obtain an integer weight. The quantization residual is maintained on the server side, and this residual is added to the participant with the largest weight, ensuring that the sum of all integer weights equals the common quantization factor (i.e., 10). 6 To maintain the unbiasedness of the weighted sum, the decryption result is uniformly divided by the quantization factor to obtain the true weighted average gradient. The aggregation server uses its private key to decrypt the aggregated ciphertext, divides the decryption result by the quantization factor to restore the plaintext aggregated gradient value, updates the global model parameters according to the FedSGD rules, and distributes the updated global parameters to all participants.

[0068] like Figure 3As shown, before each round of aggregation, the aggregation server performs distribution calibration. The distribution deviation of the k-th participant is measured using the category distribution bias, which is the absolute value of the difference between the proportion of bad customers in the participant's local sample and the global proportion of bad customers weighted by sample size for all participants. For bias at the feature distribution level, KL divergence is further used as a supplementary measure, measuring the difference between the binning frequency distribution of a participant's feature and the globally weighted binning frequency distribution. The proportion of bad customers and the binning frequency distribution of each participant are reported to the server in encrypted form during the first round of communication and are used only for aggregation weight calculation. The calibration trigger threshold is set to twice the standard deviation of the global category distribution. The standard deviation of the global category distribution is calculated by taking the square root of the average of the squares of the differences between the proportion of bad customers of each participant and the global mean, weighted by the sample size of each participant. When the bias of a participant exceeds this threshold, its distribution deviation is considered significant, triggering a weight adjustment; participants whose bias does not exceed the threshold retain their original sample size weights. For participants that trigger calibration, their aggregate weights are adjusted as follows: the participant's sample size proportion is divided by (1 plus the product of the calibration coefficient and the bias), with the calibration coefficient set to 10. The larger the bias, the stronger the weight decay. After the weights of all participants are calculated, normalization is performed so that the sum of the weights of all participants equals 1. This decay function ensures that the weights of participants with extreme biases are smoothly suppressed rather than simply removed, and their data can still contribute to the global model in a controlled manner.

[0069] During training, the aggregation server dynamically adjusts the learning rate based on changes in the global gradient norm. For example... Figure 4 As shown, the learning rate calculation rule for the t-th round of communication is: the initial learning rate divided by (1 plus the decay coefficient and the product of the ratio of the L2 norm of the current round's aggregated gradient to the L2 norm of the first round's aggregated gradient), where the initial learning rate is 0.01 and the decay coefficient is 0.5. In the early stages of training, the gradient norm is large, so the learning rate is adjusted accordingly to avoid overshooting in parameter updates; in the later stages of training, the gradient norm tends to decrease, so the learning rate automatically increases to maintain the convergence speed.

[0070] During training, the aggregation server performs a pruning operation on the gradients uploaded by the participants in each round. The pruning threshold is set to 1.0 to ensure that the L2 norm of a single gradient vector does not exceed this threshold, thus providing a basis for the effective sensitivity calculation of differential privacy in the future.

[0071] like Figure 5As shown, the aggregation server performs L2 norm verification on the encrypted gradients uploaded by each participant. The encrypted gradient vector to be verified is split and sent to two non-colluding auxiliary computing nodes. These two auxiliary nodes collaboratively calculate the sum of squares of the components in the encrypted domain based on a secret sharing protocol, i.e., the sum of the squares of each gradient component. Throughout this process, no single node can obtain the plaintext gradient. The aggregation server decrypts the scalar sum of squares to obtain the L2 norm value of each participant's gradient vector (i.e., the square root of the sum of squares of each component). The median of the gradient norms of all participants in this round is calculated. If a participant's norm exceeds three times the median, its gradient is deemed anomalous, and its gradient is rejected from participating in this round of aggregation, with an alarm issued to that participant. Participants deemed anomalous for three consecutive rounds are removed from the current training task. The verification only exposes the scalar statistic of the gradient norm, without revealing the values ​​of the gradient components.

[0072] like Figure 6 As shown, the aggregation server monitors the divergence of the local loss values ​​reported by each participant. The quantification criteria for divergence are as follows: a participant's local loss value is considered divergent if it meets any of the following conditions: monotonically increasing for three consecutive communication rounds with a cumulative increase exceeding 10%; a single round's increase exceeding 25%; or the current loss value exceeding 1.5 times its initial round's loss value. For participants deemed divergent, their gradient participation in global aggregation is paused, and a recovery process is executed: the participant continues training locally with 1 / 10 of its original learning rate. When the recovery condition of a two-round decrease in loss value and the current loss value being lower than 90% of the loss value at the time of pause is met, the participant is rejoined in the aggregation. If the recovery condition is not met within the next 10 communication rounds, the participant is removed from the current training task. The gradients previously contributed by this participant do not affect the completed global model updates, and its accumulated error buffer vector will no longer participate in subsequent training.

[0073] During training, the aggregation server dynamically adjusts the communication frequency and the ratio of local training rounds for each participant based on the rate of decrease of the loss function. The relative rate of decrease of loss in round t is defined as the absolute value of the difference between the loss value of the previous round and the loss value of the current round, divided by the loss value of the first round. The adaptive adjustment rules are as follows: When the relative rate of decrease of loss is greater than 1%, the loss decreases rapidly, indicating that there is still significant room for improvement in local training. The number of local training rounds in the next round is increased to twice that of the current round, but not exceeding the upper limit of 10, and the communication frequency is reduced accordingly. When the relative rate of decrease of loss is less than 0.1%, the loss decrease tends to stagnate, indicating that the local models of each participant are close to local optima and require more frequent global information synchronization. The number of local training rounds in the next round is reduced to half that of the current round, but not lower than the lower limit of 2, and the communication frequency is increased accordingly. When the relative rate of decrease of loss is between 0.1% and 1%, the current number of local rounds and the communication frequency remain unchanged. The initial number of local rounds is 5, the lower limit of local rounds is 2, and the upper limit is 10.

[0074] The hyperparameters for model training are set as follows: initial learning rate 0.01, adaptively adjusted using a dynamic learning rate strategy; batch size 128; initial local training epochs 5, dynamically adjusted between 2 and 10 using an adaptive scaling strategy; maximum communication epochs 50; gradient sparsity ratio 20%; gradient clipping threshold 1.0; distribution calibration coefficient 10; learning rate decay coefficient 0.5; convergence condition: the change in loss function is less than 10 for three consecutive communication epochs. -5 Training should be stopped early.

[0075] The above training steps are iterated repeatedly until the change in the loss function is less than 10 for three consecutive communication rounds. -5 Or reach the preset maximum number of communication rounds, 50 rounds.

[0076] III. Perform differential privacy post-processing on the trained global model. like Figure 7 As shown, after training convergence, the aggregation server performs differential privacy post-processing on the global model parameters before distribution. Gaussian noise satisfying the differential privacy condition is injected into the global model parameter vector. The scale of the noise is determined by the following formula: effective sensitivity multiplied by (2 multiplied by the square root of the privacy budget correlation factor) divided by the privacy budget, where the privacy budget is 1.0 and the relaxation parameter is 10. -5 The effective sensitivity value adaptively adjusts with the number of participants and the amount of data, equal to twice the gradient clipping threshold divided by (the product of the number of participants and the average sample size of each participant). The more participants and the larger the amount of data, the more diluted the influence of a single participant on the global model, and the smaller the required noise. The noisy global model parameters are then distributed to each participant as the final published model.

[0077] IV. Each participating party deploys predictive models and performs credit risk assessments. Each participating party deploys the published global model parameters on its local prediction system. When new customer data is input, the system executes the following data processing flow: First, the input features are binned according to the binning rules fixed during the training phase, and then encoded according to the fixed evidence weight encoding rules. Next, Z-score standardization is performed using the fixed global mean and standard deviation. Then, the default probability output value is calculated by inputting it into the logistic regression model, which is 1 divided by (1 plus e raised to the power of the negative logistic regression output value). In the model prediction output layer, a post-processing transformation is performed on the default probability, mapping the probability value to an integer score through a monotonic mapping function. The score is equal to a preset scale constant A minus a preset scale constant B multiplied by (the default probability divided by the natural logarithm of (1 minus the default probability)). Here, A and B are preset scale constants. Taking A=600 and B=60 as example scale constants, the output range after the score mapping is approximately 300 to 850 points. The lower the score, the higher the default risk. This credit score is output to the automated decision engine of the credit approval system. A score below 500 points triggers a manual review process, while a score above 500 points automatically approves the application. This monotonic mapping function belongs to the pure numerical post-processing stage of the model prediction output layer. It only performs monotonic numerical transformation on the output probability after the model inference is completed. It does not participate in the model training process, does not affect gradient calculation and parameter update, and the mapping result is only used as a risk quantification value output to the downstream risk control system for calling.

[0078] Example 2 like Figure 8 As shown, this embodiment provides a privacy-preserving credit risk assessment system based on federated learning. The system includes multiple commercial bank participating clients, an aggregation server, and two non-colluding auxiliary computing nodes.

[0079] Each participating client communicates with the aggregation server via an encrypted channel. The original customer credit data is always stored locally on each participating party's premises and is not transmitted externally. The aggregation server is deployed at the central node, while the two auxiliary computing nodes are deployed in independent computing environments.

[0080] Commercial bank participant client: Multiple commercial bank participating clients are deployed locally at each commercial bank. Each participating client includes a feature engineering module, a federated training module, and a prediction module.

[0081] The feature engineering module performs label binarization, decision tree binning, evidence weight encoding, feature derivation, feature selection, and Z-score standardization on the original customer credit data locally. Specifically, the feature engineering module performs label binarization on the original repayment behavior field, setting the label of samples with overdue days greater than 30 days to 1 (default class), and the label of samples with no overdue records to 0 (normal class). Samples with intermediate overdue days between 1 and 30 days are removed from the training set. For continuous feature variables, the feature engineering module uses a decision tree binning method based on the CART algorithm for discretization. It recursively calculates the Gini coefficient decrease at each candidate split point and selects the split point with the largest Gini coefficient decrease as the optimal split point. The binning results satisfy the following conditions: the sample size is not less than 5% of the total sample size, the number of bins is between 2 and 6, and the monotonicity constraint is met. The monotonicity constraint is achieved by merging adjacent bins that violate monotonicity. The feature engineering module then uses evidence weight encoding to convert the discretized variables after binning into numerical features. The feature engineering module generates derived features based on the number of first-overlap times, statistically analyzing data across five time windows: the past 1 month, 3 months, 6 months, 9 months, and 12 months. The module performs two phases of feature selection: initial screening and stepwise regression dimensionality reduction. Initial screening includes missing rate analysis, concentration analysis, information value screening, and correlation analysis. Features with a missing rate of 50% to 80% are retained only if their chi-square test p-value is less than 0.05. Stepwise regression dimensionality reduction uses a two-way selection method with Bayesian information criterion as the evaluation standard. The significance level for both variable introduction and removal is set to 0.05, selecting core variables for model inclusion from the original variables. The feature engineering module performs Z-score standardization on the selected features, updates local feature statistics using a sliding window moving average, encrypts them using Paillier homomorphic encryption, and uploads them to the server for global statistical aggregation. At the end of training, the global sliding window moving average of the last 10 communication rounds is solidified into a globally standardized statistic.

[0082] The federated training module is used to participate in lateral federated collaborative training with a logistic regression model as the base classifier. The federated training module locally calculates the gradient of the local samples with respect to the current global model parameters. The gradient expression is obtained by differentiating the cross-entropy loss function with respect to the weight parameters. The federated training module performs Top-k sparsity processing on the gradient vector, sorting the gradient components in descending order of absolute value, retaining the top 20% of the largest components and generating a sparse pattern bitmap. The remaining components are accumulated in the local error buffer vector for the next round of sparsity decision-making. The federated training module uses the Paillier homomorphic encryption public key distributed by the aggregation server to encrypt each of the retained sparse gradient components, using a different random number for each gradient component. The encrypted gradient, along with the sparse pattern bitmap, is then uploaded to the aggregation server. The federated training module receives the global model parameters from the aggregation server and updates the local model. The federated training module calculates and reports the local loss value. When the loss diverges, it performs local recovery training at 1 / 10 of the original learning rate. After two consecutive rounds of loss value decrease and the current loss value is lower than 90% of the loss value at the time of pause, it rejoins the aggregation.

[0083] The prediction module is used to deploy the published global model parameters to the local prediction system. When new customer data is input, the prediction module bins the input features according to the binning rules fixed during the training phase, performs encoding transformation according to the fixed evidence weight encoding rules, performs Z-score standardization using the fixed global mean and standard deviation, then inputs it into the logistic regression model to calculate the default probability output value, and performs a monotonic mapping transformation on the default probability to obtain an integer credit score. A score below 500 triggers manual review, while a score above 500 automatically passes approval and is output to the downstream risk control system for use.

[0084] Aggregator server: The aggregation server includes a key management module, a ciphertext aggregation module, a decryption update module, a distribution calibration module, a learning rate adjustment module, a gradient verification module, a loss detection module, a communication scheduling module, and a differential privacy module.

[0085] The key management module is used to generate Paillier homomorphic encryption key pairs. The module selects two large prime numbers, calculates their product as the modulus of the public key, calculates the least common multiple of these two prime numbers minus one as the private key parameter, selects a random number as the base of the public key, and calculates the other parameter of the private key through modulo operations. This generates the public and private keys. The public key is distributed to each participating client, while the private key is securely stored by the aggregation server.

[0086] The ciphertext aggregation module receives the ciphertext gradient and sparse pattern bitmap uploaded by each participating client. It aligns the ciphertext components according to each participant's bitmap and performs weighted aggregation in the ciphertext domain to obtain the aggregated ciphertext gradient. The module aggregates the ciphertext gradient components of each participant by exponentiation according to their corresponding integer weights. During aggregation, the random numbers independently selected by each participant during encryption are automatically combined into new valid random numbers after exponentiation and aggregation, and are completely eliminated during decryption, ensuring strict consistency between the ciphertext domain aggregation result and the plaintext domain aggregation result.

[0087] The decryption update module is used to decrypt the aggregated ciphertext using the private key to restore the plaintext aggregated gradient, and then divide the decryption result by the common quantization factor 10. 6 The model is restored to the true weighted average gradient, and the global model parameters are updated according to the FedSGD rules. The updated global parameters are then distributed to the clients of each participating party.

[0088] The distribution calibration module adjusts the aggregation weights of each participant before each round of aggregation. It calculates the absolute value of the deviation between the local bad customer proportion and the globally weighted bad customer proportion for each participant, with the calibration trigger threshold set to twice the standard deviation of the global category distribution. When a participant's deviation exceeds this threshold, the distribution calibration module adjusts its aggregation weight according to a decay function, i.e., the participant's sample size proportion divided by (1 plus the product of the calibration coefficient and the deviation). The calibration coefficient is 10. After all participant weights are calculated, normalization is performed to ensure the sum of all participant weights equals 1.

[0089] The learning rate adjustment module dynamically adjusts the learning rate based on changes in the global gradient norm. The module calculates the learning rate for the t-th round of communication according to a set rule: the initial learning rate is divided by (1 plus a decay coefficient multiplied by the ratio of the L2 norm of the current round's aggregated gradient to the L2 norm of the first round's aggregated gradient), where the initial learning rate is 0.01 and the decay coefficient is 0.5.

[0090] The gradient verification module performs L2 norm verification on the encrypted gradients uploaded by each participant. The module splits the encrypted gradient vector to be verified and sends it to two auxiliary computing nodes. It receives the sum of squares of the encrypted domains returned by the auxiliary computing nodes and decrypts it to obtain the L2 norm of each participant's gradient. It calculates the median of the gradient norms of all participants in this round. If a participant's gradient norm exceeds three times the median, it is rejected from participating in this round of aggregation. Participants deemed abnormal for three consecutive rounds are removed from the current training task.

[0091] The loss detection module monitors the divergence of local loss values ​​reported by each participant. The criteria for loss detection are as follows: a participant's local loss value is considered divergent if it monotonically increases for three consecutive communication rounds with a cumulative increase exceeding 10%, or if a single round's increase exceeds 25%, or if the current loss value exceeds 1.5 times its initial round's loss value. For participants deemed divergent, the loss detection module triggers a pause aggregation and resumption process.

[0092] The communication scheduling module dynamically adjusts the ratio of local training rounds to global communication frequency for each participant based on the loss function's decline rate. The module calculates the relative decline rate of the loss in round t, which is the absolute value of the difference between the loss value of the previous round and the current round, divided by the loss value of the first round. If this rate is greater than 1%, the number of local training rounds is increased to twice the current round, but not exceeding the upper limit of 10. If this rate is less than 0.1%, the number of local training rounds is decreased to half the current round, but not lower than the lower limit of 2. Otherwise, the current ratio is maintained, with an initial local round count of 5.

[0093] The differential privacy module injects Gaussian noise that satisfies the differential privacy condition into the global model parameters after training convergence. The differential privacy module determines the noise scale as the effective sensitivity multiplied by (2 multiplied by the square root of the privacy budget correlation factor) divided by the privacy budget, where the privacy budget is 1.0 and the relaxation parameter is 10. -5 The effective sensitivity is twice the gradient clipping threshold divided by (the product of the number of participants and the average sample size of each participant), Gaussian noise of this scale is applied to the global model parameter vector to generate the final release model and distribute it to each participant's client.

[0094] Non-colluding auxiliary computing nodes: Two non-colluding auxiliary computing nodes are deployed in independent computing environments and communicate with the aggregation server. Each auxiliary computing node is used to collaboratively calculate the sum of squares of gradient components in the ciphertext domain based on the secret sharing protocol and return it to the aggregation server when the aggregation server performs norm verification on the ciphertext gradients uploaded by each participant. Neither auxiliary computing node can know the plaintext gradient information on its own, and the two nodes do not collude to ensure the confidentiality of the ciphertext gradient values.

[0095] Example 3 This embodiment uses the actual credit risk assessment scenarios of two city commercial banks as examples to illustrate the specific implementation process of the technical solution of this application in a real business environment.

[0096] The original customer data of the two city commercial banks comes from their respective core credit systems, including basic information of personal housing loan applicants (age, education level, occupation type, annual income), loan application information (application amount, term, loan purpose), credit report information (historical credit records, credit card usage, number of credit applications), and their own historical repayment records.

[0097] Participant A is a commercial bank in a provincial capital city, possessing approximately 80,000 personal loan customer records. Its customer base primarily consists of local urban residents with an average income level in the upper-middle range of the province, and a historical average default rate of 2.3%. Participant B is a commercial bank in a prefecture-level city, possessing approximately 50,000 personal loan customer records. Its customer base covers surrounding counties and rural areas, with a relatively lower average income level, and a historical average default rate of 5.8%. Both banks share the same data feature dimensions, including 68 raw features such as age, income, occupation, education level, historical borrowing records, and credit card usage, but their sample spaces differ, representing a typical horizontal federated learning scenario. Due to data security regulations, the two banks cannot directly exchange raw customer data; therefore, they intend to jointly train a credit risk assessment model using the technical solution proposed in this application.

[0098] Data preprocessing and feature engineering were performed locally at both banks. In the original data from participant A, samples with overdue days greater than 30 days accounted for 2.3%, samples with no overdue records accounted for 94.5%, and samples in the intermediate state (overdue for 1 to 30 days) accounted for 3.2%. After label binarization and removal of intermediate state samples, approximately 77,000 samples remained. In participant B, samples with overdue days greater than 30 days accounted for 5.8%, samples with no overdue records accounted for 89.0%, and samples in the intermediate state accounted for 5.2%. After label binarization and removal of intermediate state samples, approximately 47,000 samples remained. Both banks recalculated and mapped the labels using a unified 30-day threshold to ensure consistent label definitions.

[0099] During feature engineering, participant A selected 7 core variables for model input from 68 original features, including the number of first-time delinquencies in the past 3 months, the number of first-time delinquencies in the past 6 months, the number of first-time delinquencies in the past 12 months, credit card usage rate, the number of loan applications in the past 3 months, income-to-debt ratio, and education level. Participant B also selected the same 7 variables for model input from the 68 original features, with consistent feature dimensions. After WOE encoding, each feature exhibited good monotonicity, and the information value was greater than 0.05. During Z-score standardization, each participant updated the local feature mean and standard deviation after each round of communication, using a sliding window moving average with a window size of 10, and finally solidified the global mean and standard deviation, ensuring that the feature transformation in the prediction stage was consistent with that in the training stage.

[0100] During the federated training phase, the aggregation server is deployed on a third-party neutral cloud platform, while the two auxiliary computing nodes are deployed on independent computing servers within each of the two banks. The two nodes do not communicate directly with each other, interacting only with the aggregation server. The aggregation server generates Paillier key pairs, with a 2048-bit public key, and distributes them to the clients of the two banks.

[0101] Participants A and B each use a logistic regression model as their base classifier locally, with a batch size of 128, an initial local epoch count of 5, and an initial learning rate of 0.01. In each training epoch, each participant calculates its local gradient and then performs Top-k sparsification at a rate of 20%, retaining only 20% of the maximum magnitude gradient components, reducing communication overhead by approximately 80%. Each participant encrypts the retained sparse gradient components using a Paillier public key before uploading them to the aggregation server. In the first round of communication, participant A encrypted approximately 15KB of gradient data, and participant B encrypted approximately 12KB of gradient data, including encrypted gradient values ​​and a sparse pattern bitmap. Compared to the approximately 75KB and 60KB of fully encrypted gradient data transmitted, respectively, the communication overhead is significantly reduced.

[0102] After receiving the ciphertext gradient, the aggregation server first performs a norm check on the ciphertext domain gradient using two auxiliary computing nodes. The auxiliary computing nodes calculate the sum of squares of the gradient vector components of each participant based on the secret sharing protocol. The norm values ​​decrypted by the aggregation server show that the gradient norm of participant A's row is 0.032, the gradient norm of participant B's row is 0.041, and the median of the gradient norms of all participants is 0.0365. Both are within three times the median, so the check passes.

[0103] The aggregation server performed distribution calibration. Participant A's local bad customer ratio was 2.3%, and Participant B's was 5.8%, with a globally weighted bad customer ratio of 3.83%. Participant A's bias was 1.53%, and Participant B's bias was 1.97%. The global category distribution standard deviation was approximately 1.78%. Neither bank's bias exceeded the threshold, and the weights maintained their original sample size proportions. Participant A's weight was 62.1%, and Participant B's weight was 37.9%.

[0104] The aggregation server dynamically adjusts the learning rate based on the global gradient norm. In the first round, the L2 norm of the aggregated gradient was 0.185, decreasing to 0.092 in the fifth round. The learning rate was adjusted from 0.01 to 0.01 / (1+0.5×0.092 / 0.185)≈0.008. Monitoring of the loss function's rate of decline during training showed an average decline rate of 0.8% from rounds 1 to 10, with approximately 30 rounds exceeding the 1% threshold. The communication scheduling module accordingly adjusted the number of local training rounds dynamically between 4 and 6. The entire training process involved 42 rounds of communication to achieve convergence, with the loss function change remaining less than 10 for three consecutive rounds. -5 .

[0105] After training convergence, the aggregation server performs differential privacy post-processing. The number of participants is 2, the average sample size is 62,000, and the effective sensitivity is approximately 1.61 × 10⁻⁶. -5 Privacy budget = 1.0, relaxation parameter = 10 -5 Noise scale ≈ 7.83 × 10 -5 The global model parameters are then subjected to Gaussian noise of this scale before being distributed to the two banks.

[0106] During the forecasting phase, participant A will deploy the model on its local risk control system. For new applicants, the system bins the seven input variables according to fixed binning rules, encodes them using a fixed WOE coding table, standardizes them using Z-scores with fixed means and standard deviations, inputs them into a logistic regression model to calculate the probability of default, and then converts them into integer scores through a scoring mapping, where A=600, B=60, and the score range is approximately between 300 and 850. The lower the score, the higher the default risk. The scoring results are output to the credit approval system as the basis for automatic approval decisions. Scores below 500 trigger manual review, while scores above 500 automatically approve the application.

[0107] In actual operation, the federated model achieved an AUC of 0.8358 and a KS of 0.5347 on the local test set of participant A bank, and an AUC of 0.8312 and a KS of 0.5281 on the local test set of participant B bank. The AUCs of the individual locally trained models of the two banks were 0.725 and 0.810, respectively, showing a significant performance improvement after joint training. Throughout the training process, the original customer data was always stored locally at both banks, and the aggregation server only processed encrypted gradient information, remaining unaware of any participant's plaintext gradients or original data, thus meeting data security regulatory requirements.

[0108] Example 4 This embodiment takes a joint risk control scenario involving four financial institutions as an example to illustrate the collaborative working process and performance of each module in the technical solution of this application when the number of participants increases.

[0109] The four participating parties include two branches of large state-owned banks and two regional joint-stock banks, with customer groups covering different regions and income levels, exhibiting typical Non-IID data distribution characteristics. The sample sizes for the four institutions are 120,000, 90,000, 50,000, and 40,000 records respectively, with historical default rates of 1.8%, 3.2%, 6.5%, and 7.2%, and each having 68 original variables as feature dimensions. The aggregation server is deployed on a financial industry cloud platform, and the two auxiliary computing nodes are deployed in the independent security domains of the two state-owned banks respectively.

[0110] During the feature engineering phase, each participant performed the work independently locally. Due to the significant differences in the customer groups of the four institutions, the binning boundaries of WOE encoding varied across the institutions. For example, for the annual income feature, the income of customers in large state-owned bank branches was concentrated in the range of 80,000 to 200,000 yuan, while that of customers in regional banks was concentrated in the range of 40,000 to 120,000 yuan, resulting in significantly different binning boundaries. However, WOE encoding mapped each bin to the same log-odds scale, and each feature, after being encoded locally, still maintained a natural match with the log-odds output of logistic regression, without affecting the mathematical consistency of subsequent federated training. Each of the four institutions selected 6 to 8 input variables from 68 original variables, and after taking the union, they were unified into 8 common input variables, including the number of first-time delinquencies in the past 3 months, the number of first-time delinquencies in the past 6 months, the number of first-time delinquencies in the past 12 months, credit card usage rate, the number of loan applications in the past 3 months, income-to-debt ratio, education level, and age.

[0111] During federated training, the aggregation server generates Paillier key pairs and distributes them to the four clients. Each participant performs Top-k sparsity reduction locally after calculating the gradient, with the sparsity ratio remaining at 20%. In each round of training, after receiving four ciphertext gradients and sparse pattern bitmaps, the aggregation server first verifies the norm of the ciphertext gradient through two auxiliary computing nodes. The first-round gradient norms for the four institutions were 0.028, 0.035, 0.062, and 0.071, respectively, with a median of 0.0485. The fourth institution's gradient norm of 0.071 did not exceed three times the median of 0.1455, and the verification passed.

[0112] The aggregation server performed distribution calibration. The proportions of bad customers for the four institutions were 1.8%, 3.2%, 6.5%, and 7.2%, respectively, with a globally weighted proportion of 4.0%. The biases D for each participant were 2.2%, 0.8%, 2.5%, and 3.2%, respectively, and the global class distribution standard deviation was 2.1%. The biases of all four institutions did not exceed the threshold, and the weights maintained the original sample size proportions. When scaling to more participants, the effective sensitivity decreases with the number of participants, and the noise scale required for differential privacy is correspondingly reduced, resulting in less loss of model accuracy.

[0113] During the 15th round of training, the loss detection module detected that the local loss value of the third regional bank had increased monotonically for three consecutive rounds, with a cumulative increase of 12%, exceeding the 10% threshold, thus determining that the loss of this participant was diverging. The aggregation server paused the participation of this participant's gradient in global aggregation. The participant continued local training with a learning rate of 0.001, which is 1 / 10 of the original learning rate of 0.01. After 4 rounds of local training, the loss value decreased for 2 consecutive rounds, dropping to 88% of the loss value at the time of the pause, meeting the recovery conditions and rejoining the aggregation.

[0114] The entire training process involved 38 rounds of communication to achieve convergence. Compared to a fixed communication frequency scheme (communication in every round, requiring approximately 50 rounds), this represents a 24% reduction in communication rounds and an approximately 80% reduction in total encrypted data transmission volume. After training, the global model's AUCs on the test sets of the four institutions were 0.842, 0.838, 0.829, and 0.826, respectively, all significantly outperforming the locally trained models of each institution. Upon model release, the differential privacy module calculated an effective sensitivity of approximately 6.67 × 10⁻⁶ based on the number of participants (4) and an average sample size of 75,000. -6 Noise scale ≈ 3.24 × 10 -5 It exhibits less noise and maintains better model accuracy compared to the scenarios of the two banks.

[0115] This embodiment verifies the scalability of the technical solution of this application in multi-party participation scenarios. The mechanisms such as distributed calibration, loss detection and recovery, and gradient verification work together in a multi-party environment, ensuring the stable convergence of the system and the model performance under Non-IID data distribution.

[0116] Example 5 This embodiment uses a scenario with extreme data distribution offset of the participants as an example to illustrate the effectiveness of the distribution calibration mechanism of this application in dealing with the extreme case of Non-IID.

[0117] Suppose there are two participating banks involved in joint modeling. Among them, the default rate of the customer group of participating bank C is as high as 12.5%, while the default rate of the customer group of participating bank D is only 0.8%. The data distribution of the two banks shows extreme bias.

[0118] During the feature engineering phase, both banks independently executed the same process, selecting consistent variables for model input from 68 original variables. In the first round of federated training, the aggregation server received the encrypted gradients and sparse pattern bitmaps uploaded by the two banks. First, the encrypted domain gradient norm was verified by two auxiliary computing nodes. The gradient norms of the two banks were 0.085 and 0.023, respectively, with a median of 0.054. Neither exceeded three times the median, and the verification passed.

[0119] The aggregation server performs distribution calibration. Participant C has a local bad customer ratio of 12.5%, participant D has a local bad customer ratio of 0.8%, and the globally weighted bad customer ratio is 5.3%. Participant C's bias is 7.2%, and participant D's bias is 4.5%. The global category distribution standard deviation is 5.69%. Participant C's bias of 7.2% is below 11.38%, but close to the threshold. In subsequent rounds, if the bias in C continues to widen, triggering calibration, its weight will be adjusted according to the decay function α_C = (5 / 13) / (1 + 10 × D_C). For example, when D_C reaches 15%, α_C = (5 / 13) / (1 + 1.5) = 0.3846 / 2.5 = 0.1538. After normalization, the weight decreases from the original 38.5% to approximately 22%, effectively suppressing the interference of extreme bias participants on the global model update direction.

[0120] During training, the aggregation server simultaneously monitored the local loss values ​​of both banks. Due to the extreme data distribution in bank C, its loss value decreased more slowly but did not trigger a divergence determination. The dynamic learning rate was adjusted based on the global gradient norm. In the first few rounds, the gradient norm was relatively large, so the learning rate was automatically reduced to avoid overshooting of parameter updates. After 48 rounds of communication, convergence was achieved. The global model achieved an AUC of 0.812 on the test set in bank C and 0.846 on the test set in bank D, which are significant improvements compared to the local training AUC of 0.685 in bank C and 0.792 in bank D.

[0121] This embodiment verifies the effectiveness of the distribution calibration mechanism of this application in extreme Non-IID scenarios. By combining bias measurement, a threshold of 2 times the standard deviation, and a bounded decay function, it effectively controls the excessive influence of extreme distribution participants on the global model update direction while preserving their data contributions.

[0122] The scope of protection of this invention is not limited to the specific embodiments described above. The above embodiments are only used to illustrate the technical solutions of this invention and are not intended to limit it. Any modifications, equivalent substitutions, improvements, etc., made by those skilled in the art to the above embodiments within the spirit and principles of this invention, as well as the application of the technical solutions of this invention to other horizontal federated learning classification tasks besides credit risk assessment, should be included within the scope of protection of this invention.

Claims

1. A privacy-preserving credit risk assessment method based on federated learning, characterized in that, This method is applied to a distributed system consisting of multiple participating clients, an aggregation server, and two non-collusive auxiliary computing nodes, where the original data is stored locally on each participating party's machine. Step 1: Each participant performs preprocessing and feature engineering on the raw data locally, including label binarization, feature binning, encoding conversion, feature filtering and standardization, and updates the local feature statistics using a sliding window moving average method, and uploads them to the server for global statistical aggregation. Step 2: Each participant uses a logistic regression model as the base classifier and collaboratively trains the global model using the federated stochastic gradient descent algorithm. After calculating the gradient locally, each participant performs Top-k sparsification on the gradient vector and generates a sparse pattern bitmap. The retained sparse gradient components are encrypted one by one using a homomorphic encryption public key. The resulting ciphertext gradient, along with the sparse pattern bitmap, is then uploaded to the aggregation server. The aggregation server aligns the ciphertext components according to the bitmap and performs weighted aggregation in the ciphertext domain. After decryption, the global model parameters are updated according to the federated stochastic gradient descent rules and distributed to each participant. During the aggregation process, the aggregation server performs distribution calibration to adjust the aggregation weights of each participant, performs norm verification on the ciphertext gradients uploaded by the participants to identify abnormal participants, and monitors the divergence of the local loss values ​​reported by each participant. In step two, the aggregation server performs distribution calibration, which includes: calculating the deviation metric between the proportion of default classes in each participant's local sample and the global proportion of default classes for all participants weighted by sample size; when the deviation metric exceeds a set multiple of the standard deviation of the global category distribution, the aggregation weight of that participant is reduced according to a decay function with the deviation metric as the independent variable; participants whose weights do not exceed the threshold retain their original sample size weights; and normalization is performed after the weights of all participants are calculated. The aggregation server dynamically adjusts the learning rate based on changes in the global gradient norm. The learning rate of the current round adapts to the ratio of the norm of the current aggregated gradient to the norm of the first round aggregated gradient, and works in conjunction with the distribution calibration to adjust the update step size of the global model. In step two, the aggregation server performs norm verification on the ciphertext gradients uploaded by each participant, including: splitting the ciphertext gradient vector to be verified and sending it to two non-colluding auxiliary computing nodes; the two auxiliary nodes collaboratively calculate the component square sum on the ciphertext domain based on the secret sharing protocol; the aggregation server decrypts the square sum to obtain the gradient norm of each participant; and calculates the median of the gradient norms of all participants in this round. If the gradient norm of a participant exceeds a set multiple of the median, the gradient of that participant is determined to be abnormal, and its participation in this round of aggregation is rejected. Step 3: After training converges, the aggregation server injects Gaussian noise that satisfies differential privacy conditions into the global model parameters, and distributes the noisy global model parameters as the final published model to each participant. Step 4: Each participating party deploys the published global model parameters on its local prediction system, performs the same feature transformations as in the training phase on the new input samples, inputs them into the model to calculate the default probability, and outputs the credit risk assessment results.

2. The privacy-preserving credit risk assessment method based on federated learning according to claim 1, characterized in that, In step two, each participant performs Top-k sparsification on the gradient vector, including: sorting the gradient components in descending order of their absolute values, retaining the largest component (previously set to a predetermined proportion), setting the remaining components to zero, and generating a sparse mode bitmap to mark the position index of the retained components; accumulating the zeroed gradient components into the local error buffer vector, and adding the new gradient to the error buffer vector before performing sparsification in the next round of local gradient calculation; only the retained sparse gradient components are encrypted and uploaded, and the sparse mode bitmap is transmitted in plaintext.

3. The privacy-preserving credit risk assessment method based on federated learning according to claim 1, characterized in that, In step two, the aggregation server monitors the divergence of the local loss values ​​reported by each participant. The judgment criteria are: monotonically increasing for a set number of consecutive communication rounds and the cumulative increase exceeding a set percentage, or the increase in a single round exceeding a set percentage, or the current loss value exceeding a set multiple of its first round loss value. For participants judged to be diverging, their gradient participation in global aggregation is suspended. The participant continues local training at the set percentage of the original learning rate. When the loss value decreases for a set number of consecutive rounds and the current loss value is lower than the set percentage of the loss value at the time of suspension, it rejoins the aggregation.

4. The privacy-preserving credit risk assessment method based on federated learning according to claim 1, characterized in that, In step two, the aggregation server dynamically adjusts the ratio of communication frequency to local training rounds for each participant based on the rate of loss function decline: when the rate of loss decline is higher than the first threshold, the number of local training rounds is increased and the communication frequency is decreased; when the rate of loss decline is lower than the second threshold, the number of local training rounds is decreased and the communication frequency is increased; when the rate of loss decline is between the two thresholds, the current ratio is maintained.

5. The privacy-preserving credit risk assessment method based on federated learning according to claim 1, characterized in that, In step two, each participant uses the Paillier homomorphic encryption public key distributed by the aggregation server to encrypt the preserved sparse gradient components one by one. Each gradient component uses a different random number, making the ciphertexts of the same participant in different rounds unlinkable. When the aggregation server performs weighted aggregation in the ciphertext domain, it exponentially aggregates the ciphertext gradient components of each participant according to the integer weights at the corresponding positions. The aggregated ciphertext corresponds to the weighted sum of the plaintext gradients of each participant with the same weights.

6. The privacy-preserving credit risk assessment method based on federated learning according to claim 1, characterized in that, In step three, the scale of the injected Gaussian noise is determined based on the effective sensitivity, which is equal to twice the gradient clipping threshold divided by the product of the number of participants and the average sample size of each participant. The more participants or the larger the sample size, the more diluted the influence of a single participant on the global model, and the smaller the required noise scale. The noise scale is also determined based on the privacy budget and relaxation parameters, specifically: the noise scale is equal to the effective sensitivity multiplied by the square root of twice the privacy budget correlation factor divided by the privacy budget.

7. A privacy-preserving credit risk assessment system based on federated learning, characterized in that: include: Multiple participant clients are deployed locally on each participant's local machine. Each participant client includes: The feature engineering module is used to perform label binarization, feature binning, encoding conversion, feature filtering and standardization on the raw data locally. It updates the local feature statistics using a sliding window moving average method and uploads them to the server to participate in global statistics aggregation. The federated training module is used to participate in horizontal federated collaborative training with the logistic regression model as the base classifier. It calculates gradients locally, performs Top-k sparsification on the gradient vector and generates a sparse pattern bitmap, encrypts the preserved sparse gradient components with a homomorphic encryption public key to generate ciphertext gradients and uploads them to the aggregation server, and receives global model parameters from the aggregation server and updates the local model. The prediction module is used to deploy the published global model parameters to the local prediction system, perform feature transformation on the new input samples, input the model to calculate the default probability and output a score; An aggregation server includes: The key management module is used to generate homomorphic encryption key pairs and distribute the public key to each participating client. The ciphertext aggregation module is used to receive the ciphertext gradient and sparse mode bitmap uploaded by each participating client, align the positions of the ciphertext components according to the bitmap, and perform weighted aggregation in the ciphertext domain to obtain the aggregated ciphertext gradient. The decryption and update module is used to decrypt the aggregated ciphertext using the private key to restore the plaintext aggregated gradient, update the global model parameters according to the federated stochastic gradient descent rule, and send them to the clients of each participating party. The distribution calibration module is used to calculate the deviation metric between the local category distribution and the global category distribution of each participant. When the deviation metric exceeds the set threshold, the aggregate weight of the participant is adjusted according to the decay function and normalization is performed on all weights. The learning rate adjustment module is used to dynamically adjust the learning rate based on changes in the global gradient norm. The gradient verification module is used to split the ciphertext gradient vector to be verified and send it to two auxiliary computing nodes. It receives the ciphertext domain squares returned by the auxiliary computing nodes and decrypts them to obtain the gradient norm of each participant. Participants with abnormal gradient norms are rejected from participating in this round of aggregation. The loss detection module is used to monitor the divergence of local loss values ​​reported by each participant, and to trigger the pause aggregation and recovery process for participants that are determined to be diverging. The differential privacy module is used to inject Gaussian noise that satisfies the differential privacy condition into the global model parameters after training convergence, generate the final release model, and distribute it to the clients of each participating party. Two non-colluding auxiliary computing nodes are deployed in independent computing environments and communicate with the aggregation server. They are used to collaboratively calculate the sum of squares of gradient components in the ciphertext domain based on a secret sharing protocol and return it to the aggregation server when the aggregation server performs norm verification.

Images