Risk assessment method and system based on multi-mechanism collaborative learning
Through the risk assessment method of collaborative learning by multi-institutions, a unified feature space is built and synthetic data is generated, and the data silos and privacy leakage problems in traditional credit assessment are solved, efficient and secure credit risk assessment is achieved, and model adaptability and training efficiency are improved.
Patent Information
- Application Number
- CN202510655820.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-21
- Publication Date
- 2025-08-12
AI Technical Summary
Traditional credit assessment methods are limited by data silos, making it difficult to achieve efficient, safe and efficient credit risk assessment among multiple institutions. In addition, traditional collaborative learning methods have problems such as privacy leakage, high communication costs and insufficient data heterogeneity adaptability.
Using a risk assessment method based on multi-institution collaborative learning, by designing an efficient knowledge distillation mechanism, cross-institutional knowledge fusion can be completed in just 1-2 rounds of iterations, using feature importance and correlation to build a unified feature space, generate synthetic data and filter through clustering, fuse shared data in stages, and optimize the risk assessment model.
Significantly reduce communication complexity, improve model generalization capabilities, ensure data privacy, effectively handle heterogeneous data environments, and improve model training efficiency and performance.
Smart Images
Figure CN120471707A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of risk assessment, and in particular to a risk assessment method and system based on multi-institutional collaborative learning. Background Art
[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.
[0003] Credit risk assessment is a crucial step in lending institutions' credit decisions. Traditional credit assessment methods typically rely on historical data accumulated within institutions and established rules. However, due to limitations in data size and characteristics, comprehensive and accurate assessments of users' creditworthiness are difficult. With the development of artificial intelligence (AI) technology, machine learning-based credit assessment models have been gradually applied to the financial sector, improving the accuracy and efficiency of assessments through large-scale data training.
[0004] The performance of credit assessment models is highly dependent on the quality and diversity of training data. Currently, data across financial institutions is often isolated (known as data silos) due to privacy concerns, regulatory requirements, and competitive factors, making it difficult for different institutions to directly share their private credit data. This data isolation limits the effectiveness of credit risk assessment model training, especially when data distribution varies significantly across institutions. Data from a single institution cannot support a highly generalizable model.
[0005] Collaborative learning technology has emerged as a potential solution to the data silo problem. Without directly sharing raw data, collaborative learning can leverage data from various institutions for joint modeling, improving the performance of credit assessment models. However, traditional collaborative learning methods still face three key challenges in achieving multi-institutional collaboration:
[0006] First, existing paradigms typically require participants to share intermediate information such as model parameters or feature representations. Although this is an improvement over direct data sharing, it still poses potential privacy risks.
[0007] Second, frequent parameter synchronization mechanisms lead to high communication costs, which may cause network transmission bottlenecks, especially in large-scale distributed scenarios.
[0008] Third, the data of each participating institution usually presents non-independent and identically distributed (Non-IID) characteristics. The homogeneous modeling strategy adopted by traditional methods is difficult to effectively adapt to the data heterogeneity of each node, resulting in limited personalized adaptation capabilities of the model. Summary of the Invention
[0009] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a risk assessment method and system based on multi-institutional collaborative learning. By designing an efficient knowledge distillation mechanism, cross-institutional knowledge fusion can be completed in only 1-2 rounds of iterations, significantly reducing the communication complexity to a linear level.
[0010] In order to achieve the above object, the present invention adopts the following technical solutions:
[0011] A first aspect of the present invention provides a risk assessment method based on multi-institutional collaborative learning, comprising the following steps:
[0012] All participants obtain their own user information and credit data and pre-process them. Based on the importance, relevance, and impact of features on the risk assessment task, they extract relevant features and send the generated feature summary to the coordinator. The coordinator identifies common features based on the received feature summary and returns them to all participants. Each participant then constructs a unified feature space based on the received common features.
[0013] Each participant generates synthetic data based on the constructed feature space and the original data collected by each participant. The generated data is filtered through clustering, data samples in dense areas are extracted, and outliers are retained.
[0014] Each participant will share the screened synthetic data with other participants. Based on the phased fusion mechanism, the shared data will be gradually introduced into the training process of the risk assessment model to optimize the risk assessment model of each participant.
[0015] Furthermore, preprocessing includes cleaning duplicate data and filling missing values in the local dataset; the local dataset at least includes user ID, age, annual income, housing situation, years of work, loan purpose, loan amount and historical default records.
[0016] Furthermore, the correlation between features and risk assessment and the importance of features are determined based on the mutual information value.
[0017] Furthermore, the generated feature summary is sent to the coordinator. Specifically, the feature summary includes the feature name and statistical information, and the statistical information at least includes the average value, variance and redundancy group of the feature. Each participant sends the generated feature summary to the coordinator.
[0018] Furthermore, the coordinator identifies common features based on the received feature summary and returns them to all participants, specifically:
[0019] All participants were provided with a summary of their characteristics;
[0020] Match and cluster the feature names and descriptions of each participant based on pre-defined standard dictionaries, domain knowledge, or natural language processing techniques to identify semantically similar or identical features;
[0021] The features are verified by combining data statistics, and the features that exist in all participants and have the same meaning are determined as common features. The names and units of these features are unified to form a standardized feature set, which is returned to all participants.
[0022] Furthermore, each participant generates synthetic data based on the constructed feature space and the original data collected by each participant, filters the generated data through clustering, extracts data samples from dense areas, and retains outliers; specifically: each participant uses the conditional table to generate an adversarial network, combines the constructed feature space and the original data collected by each participant, generates synthetic samples consistent with the statistical characteristics of the original data through the adversarial training mechanism, and filters the generated data through clustering.
[0023] Furthermore, the phased integration mechanism is as follows:
[0024] In the initial stage, only local data is used to train the risk assessment model to obtain baseline performance;
[0025] The synthetic data is divided into multiple batches and gradually introduced. The cumulative amount of data introduced gradually increases according to the set ratio.
[0026] The impact of the synthetic data set in each stage is determined by the changes in model performance at each stage. After all the synthetic data from all stages are introduced and the risk assessment model is verified and optimized, a trained risk assessment model is obtained.
[0027] A second aspect of the present invention provides a system for implementing the above method, comprising:
[0028] The feature construction module is configured as follows: all participants obtain their respective user information and credit data and pre-process them. Based on the feature importance, relevance, and the degree of impact of the features on the risk assessment task, they extract the corresponding features and send the generated feature summary to the coordinator. The coordinator identifies the common features based on the received feature summary and returns them to all participants. Each participant then constructs a unified feature space based on the received common features.
[0029] The feature screening module is configured as follows: each participant generates synthetic data based on the constructed feature space and the original data collected by each participant, filters the generated data through clustering, extracts data samples in dense areas, and retains outliers;
[0030] The risk assessment model optimization module is configured as follows: each participant shares the screened synthetic data with other participants, and based on a phased fusion mechanism, gradually introduces the shared data into the training process of the risk assessment model to optimize the risk assessment model of each participant.
[0031] A third aspect of the present invention provides a computer-readable storage medium.
[0032] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the risk assessment method based on multi-institutional collaborative learning.
[0033] A fourth aspect of the present invention provides a computer device.
[0034] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the risk assessment method based on multi-institutional collaborative learning are implemented.
[0035] Compared with the existing technology, one or more of the above technical solutions have the following beneficial effects:
[0036] 1. We propose a collaborative learning framework that enables each institution (participant) to train a generative model using its local data. This framework effectively captures the unique characteristics of the local dataset and meets the needs of personalized model training through knowledge extraction. After generating synthetic data, each institution evaluates and screens the data quality before securely sharing it with other participants. This multi-party collaboration mechanism improves the generalization capabilities of participating models while ensuring data privacy and significantly reducing communication overhead, providing an efficient solution for collaborative learning in heterogeneous data environments.
[0037] 2. We designed a feature selection algorithm that effectively eliminates low-relevance and redundant features in high-dimensional datasets. This algorithm not only addresses the heterogeneity of feature spaces across datasets but also significantly reduces the computational burden of model training. By ensuring feature importance and maintaining information integrity, the algorithm improves model efficiency and performance, providing a practical and effective solution for processing complex feature spaces.
[0038] 3. Combining a density-based spatial clustering algorithm with a conditional screening strategy, we precisely select representative samples within a cluster. This approach significantly reduces the amount of data required for transmission and alleviates the problem of uneven data distribution. While ensuring a more balanced and higher-quality dataset, it also improves data transmission efficiency, thereby optimizing model training results. BRIEF DESCRIPTION OF THE DRAWINGS
[0039] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.
[0040] Figure 1 is a schematic diagram of a multi-institution collaborative learning process provided by one or more embodiments of the present invention;
[0041] Figure 2 is a flow chart of step S1 during multi-institution collaborative learning provided by one or more embodiments of the present invention;
[0042] Figure 3 (a)- Figure 3 (d) Schematic diagram of the performance comparison of original data and synthetic data training in terms of accuracy, recall, F1 score and AUC;
[0043] Figure 4 (a)- Figure 4 (d) Schematic diagram showing the performance comparison of the model before and after synthetic data screening in terms of accuracy, recall, F1 score, and AUC;
[0044] Figure 5 This is a diagram showing the effects of applying the collaborative learning framework to different scenarios. DETAILED DESCRIPTION
[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.
[0046] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.
[0047] As introduced in the background technology, traditional collaborative learning methods still face three key challenges in achieving multi-institutional collaboration:
[0048] First, existing paradigms typically require participants to share intermediate information such as model parameters or feature representations. Although this is an improvement over direct data sharing, it still poses potential privacy risks.
[0049] Second, frequent parameter synchronization mechanisms lead to high communication costs, which may cause network transmission bottlenecks, especially in large-scale distributed scenarios.
[0050] Third, the data of each participating institution usually presents non-independent and identically distributed (Non-IID) characteristics. The homogeneous modeling strategy adopted by traditional methods is difficult to effectively adapt to the data heterogeneity of each node, resulting in limited personalized adaptation capabilities of the model.
[0051] The following examples provide a risk assessment method and system based on multi-institutional collaborative learning. By designing an efficient knowledge distillation mechanism, the knowledge of complex models is extracted and migrated to a smaller, more efficient model. Only 1-2 rounds of iterations are required to complete cross-institutional knowledge fusion, significantly reducing communication complexity to a linear level.
[0052] Example 1:
[0053] like Figure 1 As shown in FIG, the risk assessment method based on multi-institutional collaborative learning includes the following steps:
[0054] S1. Analyze the features of credit datasets from different sources, identify common features, and construct a unified feature space based on feature importance, relevance, and impact on the target task.
[0055] S2. Use the CTGAN model to learn local data and generate synthetic data. Then use the DBS CAN clustering algorithm to filter the generated data and extract data samples from dense areas while retaining outliers to ensure data validity and diversity.
[0056] S3. Share the screened synthetic dataset with other participating institutions and integrate it using a phased data fusion mechanism. Gradually introduce shared data into the model training process and dynamically adjust the data fusion strategy based on the performance of the validation set, retaining data that helps model optimization and eliminating data that may introduce noise.
[0057] like Figure 2 As shown, in step S1, a unified feature space is constructed, specifically:
[0058] 1) Data Preprocessing: Data preprocessing includes cleaning duplicate data in the local dataset and filling missing values. The local dataset primarily contains information such as user ID, age, annual income, housing status, years of work experience, loan purpose, loan amount, and historical default records. This information is collected with the user's permission, and the user is informed that it will only be used for the risk assessment process proposed in this solution.
[0059] 2) Feature selection: A feature selection method based on mutual information value, which aims to extract a set of candidate features that are strongly correlated with the target scene.
[0060] In this embodiment, the mutual information value (MI) is an indicator in information theory that measures the statistical dependence between two variables.
[0061] In this embodiment, the candidate feature set that is strongly related to the target scenario is, for example, the borrower's annual income (continuous variable), housing conditions (categorical variable), and career stability (ordered variable) in credit risk assessment.
[0062] The relevance of features to the target scenario is measured by calculating the mutual information value between the features and the target vector (the label in the training dataset, such as the binary variable of user default status in the credit scenario).
[0063] Mutual information I(X,Y) measures the correlation between variables X and Y by calculating the difference between the joint probability distribution p(x,y) and the marginal probability distributions p(x)p(y). A higher mutual information value indicates a stronger correlation between the two variables. Its calculation formula is as follows:
[0064]
[0065] Here, p(x,y) is the joint probability distribution of X and Y, that is, the probability of taking the values x and y simultaneously. p(x) is the marginal probability distribution of the random variable X, that is, the probability that X takes the value x. p(y) is the marginal probability distribution of the target vector Y, that is, the probability that Y takes the value y.
[0066] The importance of features is determined based on the mutual information value, and the redundancy between features is detected based on the Pearson correlation coefficient. The calculation formula of the Pearson correlation coefficient is as follows:
[0067]
[0068] Where Cov(X,Y) is the covariance of X and Y, σ X is the standard deviation of X, σ Y is the standard deviation of Y, is the mean of X, is the mean of Y. After selecting features through the above operations, we will get a set of features with high importance and low correlation.
[0069] 3) Information Aggregation: Each participant generates a feature summary based on their local feature set. The feature summary includes the feature name and statistical information (such as the feature's mean, variance, and redundancy groups). After generating the feature summary, each participant sends it to the coordinator.
[0070] 4) Processing information: After receiving the feature summaries of all participants, the coordinator identifies the common features and sends the results back to the participants. The identification process is as follows:
[0071] Collect feature summaries provided by all participants, including metadata such as feature name, description, unit, data type, etc.
[0072] Using predefined standard dictionaries, domain knowledge, or natural language processing techniques, the feature names and descriptions of each participant are matched and clustered to identify semantically similar or identical features.
[0073] The features are verified by combining data statistics (such as value range, distribution, etc.) to determine which features are common features that all parties have and have the same meaning. The names and units of these features are unified to form a standardized feature set, which is then sent back to each participant.
[0074] Common characteristics are generally user characteristics common to all credit institutions, such as age, income, housing conditions, credit history, etc.
[0075] 5) Construct a unified feature space: Each participant constructs a high-quality unified feature space locally based on the received coordination results (the common features identified previously, including feature name, unit, data type, and other information).
[0076] Step S2 learns the local data and generates and filters the synthetic data, including the following steps:
[0077] (1) Preprocess the local raw data based on the unified feature space obtained in step 1. Delete redundant features, unify common feature names, fill in missing values, and finally output a standardized training set that can be directly integrated with other client data.
[0078] (2) Use the preprocessed dataset from (1) to train a data generation model (CTGAN). During the training process, we extract some synthetic samples and train a lightweight local scoring model to evaluate the sample quality. Then, we dynamically adjust the learning rate, number of iterations, batch size and other hyperparameters until a stable and high-quality generator is obtained.
[0079] (3) Generate data using the generative model trained in (2). In this process, a balanced sampling strategy is adopted to increase the generation probability of rare default classes and improve the coverage of minority class samples.
[0080] (4) Perform DBSCAN clustering on all synthetic samples to identify the core points and outliers of each cluster. For large clusters, only the top 5% of the most representative points in the core area are retained; for small clusters, all core points are retained; all outliers are also retained. Output is a streamlined synthetic dataset that has been filtered for density and representativeness.
[0081] (5) Without leaking the original customer information, the streamlined synthetic dataset is distributed to other clients to achieve cross-institutional knowledge sharing and collaborative modeling.
[0082] The specific steps are as follows:
[0083] 1) Use a generative model to learn the distribution of the original data and generate synthetic credit data. The generative model used in this embodiment is the Conditional Tabular Generative Adversarial Network (CTGAN). Through an adversarial training mechanism, it learns the complex joint probability distribution in tabular data and uses pattern decoupling techniques (such as conditional vector embedding) to solve the problem of mixed modeling of discrete and continuous variables. Ultimately, it generates synthetic samples that are highly consistent with the statistical characteristics of the original data (such as class imbalance and nonlinear correlation between features).
[0084] Regarding the adversarial training mechanism, the generator and discriminator form dynamic feedback in alternating training: the generator first synthesizes simulated data, and the discriminator then compares the feature differences between the real data and the synthesized data and outputs an evaluation signal; the generator adjusts the network parameters through gradient backpropagation based on the discriminator's feedback signal and the preset loss function. After multiple iterative optimizations, the generator's data synthesis ability gradually approaches the real data distribution, and finally obtains a generation model that can stably output realistic synthetic samples.
[0085] In this embodiment, CTGAN uses a variational Gaussian mixture model (VGM) to model continuous variables as a mixture of multiple Gaussian sub-distributions, as continuous variables are often non-Gaussian and have multiple peaks (multimodal distribution). In contrast to image data where pixel values usually follow a Gaussian-like distribution, continuous variables in tabular data are usually non-Gaussian distributed. They tend to follow a multimodal distribution, in which the probability distribution has more than one mode, that is, they present different local maxima (or "peaks"). In order to solve the modeling problem of the multimodal distribution of continuous features in tabular data (such as fields such as income and age that may present multiple peaks or asymmetric distributions), CTGAN proposes a method that combines mode-specific normalization with a variational Gaussian mixture model (VGM). The core idea is to represent each continuous value as two parts by decomposing the intrinsic multimodal characteristics of the continuous features:
[0086] (1) One-hot vector: Indicates the “distribution mode” (i.e., a subdistribution in the Gaussian mixture model) to which the value belongs. For example, if a continuous feature (such as income) is decomposed into three Gaussian subdistributions (low, medium, and high income modes) by VGM, the one-hot vector of each sample will indicate the subdistribution to which it belongs (e.g., 1,0,0 corresponds to “low income mode”).
[0087] (2) Scalar: The value normalized based on the mean and standard deviation of the sub-distribution. Specifically, for a sample belonging to pattern k, its scalar value is:
[0088]
[0089] The generator first generates a unique hot vector and a scalar, and then combines the statistical parameters of the pattern k in the pre-trained VGM (mean μ k , standard deviation σ k ), and the final generated continuous value is obtained through the inverse normalization operation:
[0090] x 生成 =scalar×σ k +μ k
[0091] This decomposition effectively solves the multimodal distribution problem of continuous features and avoids the mode collapse problem caused by the unimodal assumption of traditional normalization (such as Min-Max or Z-Score).
[0092] For discrete variables, CTGAN introduces a conditional generator to address two major challenges introduced by categorical features. One is the sparsity of one-hot encoded vectors in real data. While the one-hot vectors of discrete features in real data are strictly unimodal (e.g., 0, 1, 0), the probability distribution output by the generator (e.g., 0.2, 0.7, 0.1) often lacks sparsity. This difference in sparsity makes it difficult for the discriminator to distinguish between real and synthetic data. To address this issue, CTGAN uses Gumbel-Softmax instead of standard Softmax. This approach uses a low-temperature coefficient (τ→0) to approximate the rigid distribution of the true one-hot encoding, avoiding the generation of intermediate results with ambiguous probabilities. Another challenge is the imbalanced class distribution. Certain low-frequency classes (e.g., rare disease types) are extremely underrepresented in the training data, making it difficult for the generator to fully learn their patterns. To address this issue, CTGAN introduces a conditional generator to handle class imbalance. The generator's input consists not only of a random noise vector z but also a concatenated conditional vector c, which specifies the class to be generated and the generator must generate samples that meet this condition.
[0093] In the field of risk assessment, most training datasets suffer from severe imbalanced distribution (the number of default samples is far less than the number of non-default samples). To adapt generative models to downstream risk assessment models, we employ a balanced sampling strategy to increase the probability of generating rare default classes and improve the coverage of minority class samples. Specifically, this strategy employs a dynamic conditional vector adjustment strategy, adaptively adjusting the default / non-default sampling ratio based on real-time generation performance. Secondly, to meet the downstream risk assessment model's requirements for training sample quantity and diversity (especially for minority default samples), we generate synthetic data 2–3 times the size of the original dataset. This improves model robustness, reduces the risk of overfitting, and enhances the ability to capture rare events (such as high-risk customers).
[0094] 2) Clustering is performed on the generated large batch of synthetic data using the DBSCAN clustering algorithm. For large clusters, only the top 5% of samples in the core area are retained; for small clusters, all samples are retained, and all outliers are retained to extract unique information.
[0095] Step S3: The phased fusion mechanism includes the following steps:
[0096] 1) Introducing synthetic data in stages: During training, synthetic datasets are gradually incorporated into the model in multiple stages. The goal of each stage is to combine synthetic data with real data and evaluate the impact of each stage on the local model based on changes in model performance. To ensure a reasonable division of stages, a dynamic stage division strategy based on performance stability is adopted, mainly following the following principles:
[0097] Initial stage (Baseline training): Use only local real data to train the model and obtain baseline performance (F1 score, etc.).
[0098] Gradual introduction of synthetic data: The synthetic data is divided into multiple batches, and the cumulative amount of data introduced is gradually increased according to a certain proportion (for example, 10%, 20%, and 30% of the total synthetic data set are gradually introduced).
[0099] 2) Model performance monitoring: After introducing synthetic data at each stage, the current model performance is evaluated using the validation set to determine whether the synthetic dataset has improved the generalization ability of the model or introduced interference information.
[0100] 3) Impact Judgment: The impact of the synthetic dataset at each stage is determined by the changes in model performance at each stage. The specific judgment process is as follows:
[0101] During the current training phase, if the model's F1 score stabilizes or improves (exceeds a certain threshold, such as 1%-2%) over multiple consecutive evaluation rounds (e.g., 3 rounds), the current data is considered to have made a positive contribution to the model, and synthetic data can be introduced into the next phase.
[0102] If the F1 score fluctuates greatly or decreases, it may indicate that the current data contains noise or interference. It is necessary to optimize the current stage data (for example, perform data cleaning and adjust weights) before deciding whether to continue introducing the next stage data.
[0103] 4) Gradually increase the amount of data: As the data is incorporated and evaluated at each stage, the amount of synthetic data will gradually increase. The amount of synthetic data at each stage can be gradually increased according to a predetermined ratio to ensure that each introduction of data can bring a positive impact, thereby optimizing the training process.
[0104] 5) Final Data Fusion: After gradually introducing synthetic data from all stages and undergoing multiple rounds of screening and evaluation, the resulting dataset fully validates and optimizes the impact of each synthetic dataset. This dynamic adjustment based on performance feedback allows for a steady introduction of synthetic data, avoiding the potential negative impact of adding a large amount all at once.
[0105] This proposal proposes a collaborative learning framework that enables each institution to train a generative model using its local data, effectively capturing the unique characteristics of its local dataset and meeting the needs of personalized model training. After generating synthetic data, each institution evaluates and screens the data quality before securely sharing it with other participants. This multi-party collaboration improves the generalization capabilities of participating models. Furthermore, by sharing filtered fake data instead of model parameters, this mechanism significantly reduces communication overhead while ensuring data privacy, providing an efficient solution for collaborative learning in heterogeneous data environments.
[0106] This solution designs an innovative feature selection algorithm that effectively eliminates low-relevance and redundant features in high-dimensional datasets. This algorithm not only addresses the heterogeneity of feature spaces across datasets but also significantly reduces the computational burden of model training. By ensuring feature importance and maintaining information integrity, the algorithm improves model efficiency and performance, providing a practical and effective solution for processing complex feature spaces.
[0107] This solution combines a density-based spatial clustering algorithm (DBSCAN) with a conditional filtering strategy to accurately select representative samples within a cluster. This approach significantly reduces the amount of data required for transmission and alleviates the problem of uneven data distribution. While ensuring a more balanced and higher-quality dataset, it also improves data transmission efficiency, thereby optimizing model training results.
[0108] The following is an introduction based on specific cases:
[0109] The Credit Risk Dataset, publicly available on the Kaggle platform, is used for demonstration. This dataset contains data on 32,581 loan users, each with 11 attributes. We randomly split the dataset into 10 parts, using both independent and identically distributed (IID) and non-IID scenarios, to simulate the datasets of 10 institutions participating in collaborative learning. We assume that these 10 institutions wish to use our framework and datasets from other institutions to enhance the generalization capabilities of their local credit models.
[0110] S1. All organizations perform feature analysis on their local datasets (including calculating mutual information and redundancy), then generate feature summaries. These summaries are then sent to a coordinator (a trusted organization). The coordinator identifies common features and constructs a unified feature space based on their importance, relevance, and impact on the target task. The coordinator then sends this unified feature space to other participants, forming the foundation for data sharing.
[0111] S2. Each institution trained the CTGAN model using its own local dataset and adjusted the CTGAN conditional vector to make the distribution of default and non-default samples more balanced. Finally, each institution used the model to generate synthetic data equivalent to 2 to 3 times the original data to fully cover the data characteristics. Comparative analysis shows that the prediction model trained with synthetic data performs slightly worse than the model trained with original data, indicating that the generated data can well preserve most of the information of the original data, such as Figure 3 Each institution then clustered the generated data using the DBSCAN algorithm and filtered the data within each cluster according to a pre-defined strategy, ultimately obtaining a streamlined dataset. Although small in size, this dataset still retained most of the key information.
[0112] Figure 3 and Figure 4 The AUC in it represents the area under the ROC curve, which is usually used to measure the overall performance of a two-class classification model.
[0113] Figure 4 The performance comparison of the model before and after the synthetic dataset is presented. The results show that the overall performance of the filtered dataset is comparable to that before filtering, and the recall rate is even improved.
[0114] S3. Each institution shares the screened dataset with other institutions. After receiving the synthetic datasets from other institutions, it first extracts some samples from these datasets and the local test set to construct a validation set. Subsequently, each institution merges the remaining synthetic datasets with the local training set in batches, and uses the validation set to evaluate the performance (F1 score) of the model trained with the merged dataset. If the performance of the merged model is better than the model trained using only the original data, the batch of data is formally incorporated; otherwise, the batch of data is discarded. After multiple rounds of screening and fusion, an optimized training dataset is finally formed, and the local credit model is trained based on it to achieve performance optimization. Figure 5 The performance comparison of this framework with other methods in two scenarios (iid and non-iid) is shown.
[0115] Figure 5The models are: a model trained using local data (MLP), a federated learning model (Fedavg), a federated learning method compatible with heterogeneous models, tasks and training processes (CoFed), the framework proposed in this invention but without a data fusion mechanism (KFDS), and the framework proposed in this invention (KFD SM).
[0116] Example 2:
[0117] A system for implementing the above method includes:
[0118] The feature construction module is configured as follows: all participants obtain their respective user information and credit data and pre-process them. Based on the feature importance, relevance, and the degree of impact of the features on the risk assessment task, they extract the corresponding features and send the generated feature summary to the coordinator. The coordinator identifies the common features based on the received feature summary and returns them to all participants. Each participant then constructs a unified feature space based on the received common features.
[0119] The feature screening module is configured as follows: each participant generates synthetic data based on the constructed feature space and the original data collected by each participant, filters the generated data through clustering, extracts data samples in dense areas, and retains outliers;
[0120] The risk assessment model optimization module is configured as follows: each participant shares the screened synthetic data with other participants, and based on a phased fusion mechanism, gradually introduces the shared data into the training process of the risk assessment model to optimize the risk assessment model of each participant.
[0121] Example 3:
[0122] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the risk assessment method based on multi-institutional collaborative learning in the above-mentioned embodiment 1 are implemented.
[0123] Example 4:
[0124] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the risk assessment method based on multi-institutional collaborative learning in the first embodiment are implemented.
[0125] The steps or modules involved in Examples 2 to 4 above correspond to those in Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media that includes one or more instruction sets; it should also be understood to include any medium that can store, encode, or carry an instruction set for execution by a processor and cause the processor to perform any method of the present invention.
[0126] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.
Claims
1. A risk assessment method based on multi-institutional collaborative learning, characterized by: The following steps are involved: All participants obtain their own user information and credit data and pre-process them. Based on the importance, relevance, and impact of features on the risk assessment task, they extract relevant features and send the generated feature summary to the coordinator. The coordinator identifies common features based on the received feature summary and returns them to all participants. Each participant then constructs a unified feature space based on the received common features. Each participant generates synthetic data based on the constructed feature space and the original data collected by each participant. The generated data is filtered through clustering, data samples in dense areas are extracted, and outliers are retained. Each participant will share the screened synthetic data with other participants. Based on the phased fusion mechanism, the shared data will be gradually introduced into the training process of the risk assessment model to optimize the risk assessment model of each participant.
2. The risk assessment method based on multi-institutional collaborative learning according to claim 1, characterized in that: Preprocessing includes cleaning duplicate data and filling missing values in the local dataset; the local dataset at least includes user ID, age, annual income, housing situation, years of work, loan purpose, loan amount and historical default records.
3. The risk assessment method based on multi-institutional collaborative learning according to claim 1, characterized in that: The correlation between features and risk assessment, as well as feature importance, were determined based on the mutual information value.
4. The risk assessment method based on multi-institutional collaborative learning according to claim 1, characterized in that: The generated feature summary is sent to the coordinator. Specifically, the feature summary includes the feature name and statistical information. The statistical information at least includes the mean value, variance, and redundancy group of the feature. Each participant sends the generated feature summary to the coordinator.
5. The risk assessment method based on multi-institutional collaborative learning according to claim 1, characterized in that: The coordinator identifies the common features based on the received feature summary and returns them to all participants. Specifically: All participants were provided with a summary of their characteristics; Match and cluster the feature names and descriptions of each participant based on pre-defined standard dictionaries, domain knowledge, or natural language processing techniques to identify semantically similar or identical features; The features are verified by combining data statistics, and the features that exist in all participants and have the same meaning are determined as common features. The names and units of these features are unified to form a standardized feature set, which is returned to all participants.
6. The risk assessment method based on multi-institutional collaborative learning according to claim 1, characterized in that: Each participant generates synthetic data based on the constructed feature space and the original data they collected, filters the generated data through clustering, extracts data samples from dense areas, and retains outliers; specifically: each participant uses the conditional table to generate an adversarial network, combines the constructed feature space and the original data they collected, generates synthetic samples consistent with the statistical characteristics of the original data through the adversarial training mechanism, and filters the generated data through clustering.
7. The risk assessment method based on multi-institutional collaborative learning as claimed in claim 1, characterized in that: The phased integration mechanism is as follows: In the initial stage, only local data is used to train the risk assessment model to obtain baseline performance; The synthetic data is divided into multiple batches and gradually introduced. The cumulative amount of data introduced gradually increases according to the set ratio. The impact of the synthetic data set in each stage is determined by the changes in model performance at each stage. After all the synthetic data from all stages are introduced and the risk assessment model is verified and optimized, a trained risk assessment model is obtained.
8. The risk assessment method system based on multi-institutional collaborative learning is characterized by: include: The feature construction module is configured as follows: all participants obtain their respective user information and credit data and pre-process them. Based on the feature importance, relevance, and the degree of impact of the features on the risk assessment task, they extract the corresponding features and send the generated feature summary to the coordinator. The coordinator identifies the common features based on the received feature summary and returns them to all participants. Each participant then constructs a unified feature space based on the received common features. The feature screening module is configured as follows: each participant generates synthetic data based on the constructed feature space and the original data collected by each participant, filters the generated data through clustering, extracts data samples in dense areas, and retains outliers; The risk assessment model optimization module is configured as follows: each participant shares the screened synthetic data with other participants, and based on a phased fusion mechanism, gradually introduces the shared data into the training process of the risk assessment model to optimize the risk assessment model of each participant.
9. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the risk assessment method based on multi-institutional collaborative learning as described in any one of claims 1 to 7.
10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the risk assessment method based on multi-institutional collaborative learning as claimed in any one of claims 1 to 7 are implemented.