Risk control model-based data risk identification method and system
By enhancing features through generative adversarial networks and business knowledge graphs, and combining dynamic model selection and reinforcement learning, a complete technical closed loop for risk control models is constructed. This solves the shortcomings of traditional risk control methods in identifying new risk patterns and adapting to rapidly changing environments, and achieves efficient and accurate risk identification and self-optimization.
Patent Information
- Application Number
- CN202511664331.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-13
- Publication Date
- 2026-02-17
AI Technical Summary
Traditional risk control methods struggle to identify novel or rare risk patterns when faced with massive, multi-source, and rapidly changing data. They have limited feature differentiation and the models cannot adjust themselves in real time, resulting in low accuracy and efficiency in identification. Furthermore, the various stages are fragmented and fail to form an integrated closed loop.
Generative adversarial networks are used to generate synthetic samples to construct a balanced training dataset. This dataset is then combined with business knowledge graphs for feature processing and semantic enhancement. A dynamic model is selected and optimized online through reinforcement learning, forming a complete technical loop.
It enhances the ability to identify risk patterns in scarce samples, improves the accuracy and recall of risk identification, and ensures the high robustness and business adaptability of the risk control process in dynamic environments.
Smart Images
Figure CN121544366A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data security technology, specifically a data risk identification method and system based on a risk control model. Background Technology
[0002] In high-risk business sectors such as finance and e-commerce, traditional risk control methods primarily rely on manually formulated static rules and single-point models for risk decision-making. These methods are increasingly revealing their limitations when faced with massive, multi-source, and rapidly changing data.
[0003] First, for novel or rare risk patterns (i.e., "long-tail risks"), the scarcity of historical samples makes it difficult for models to learn and identify effectively, leading to missed warnings. Second, traditional feature engineering relies heavily on expert experience and statistical methods, making it difficult to deeply mine the complex semantic relationships behind the data, resulting in limited feature discriminative power. Third, once deployed, models often become fixed and cannot quickly self-adjust based on real-time online feedback and dynamically changing risk patterns, causing model performance to degrade as business grows. Furthermore, the various stages from data preparation and model training to deployment and maintenance are often fragmented, failing to form an integrated, automated closed loop, which severely challenges the efficiency, accuracy, and adaptability of the entire risk control system.
[0004] Therefore, there is an urgent need for a more reliable data security management technology. Summary of the Invention
[0005] The purpose of this invention is to provide a data risk identification method and system based on a risk control model to solve the technical problems mentioned in the background.
[0006] To achieve the above objectives, the present invention discloses the following technical solutions: In a first aspect, this invention discloses a data risk identification method based on a risk control model, the method comprising: Data preparation phase: Identify risk targets and collect data from multiple sources. Clean and preprocess the collected data to improve data quality. In particular, for risk scenarios with scarce samples, a generative adversarial network is used to generate synthetic samples that conform to the risk characteristics of the risk scenario, and mix them with real samples to construct a balanced training dataset for model training. Feature processing stage: Construct and filter feature variables associated with the risk objective from the preprocessed data, and transform the feature variables to obtain the final feature set; Model building and validation phase: Using samples from the balanced training dataset and their corresponding final feature sets, a machine learning model is selected based on the risk objective for training and hyperparameter tuning, and the performance and stability of the model are evaluated using cross-time validation or cross-validation. Model deployment and iteration phase: Deploy the validated model to the production environment and continuously monitor the model performance indicators and feature variable distribution; and construct a reward function based on reinforcement learning agents with the goal of comprehensively improving business indicators, and dynamically adjust the model's decision threshold or feature variable weights based on real-time feedback of business and model data to perform online adaptive iterative optimization of the model.
[0007] In one optional implementation, the cleaning and preprocessing of the collected data includes: Missing values can be filled using mean imputation or model prediction imputation. Use the IQR rule or Z-score method to identify and remove outliers; One-hot encoding or label encoding is performed on categorical feature variables; The preprocessed data is correlated and verified with a pre-built business knowledge graph. The business relationships in the business knowledge graph are used to identify and remove isolated abnormal data points that do not conform to business logic, so as to obtain high-quality cleaned data.
[0008] In one optional implementation, the steps for constructing the business knowledge graph include: Extract the core entities related to risk control business and their business relationships from the multi-source data; Each business relationship is assigned a weight, with the initial weight of any business relationship set based on the relationship type and frequency of occurrence, and dynamically adjusted during operation based on the frequency of occurrence of that business relationship in historical risk cases.
[0009] In one optional implementation, the feature processing stage includes filtering and transforming feature variables to obtain the final feature set, including: Calculate the IV values of the initially constructed feature variables, and select feature variables with IV values greater than the first preset IV threshold; Calculate the Pearson correlation coefficient among the initially constructed feature variables and remove redundant feature variables whose correlation coefficients are higher than the preset correlation coefficient threshold; For feature variables whose IV values fall between a second preset IV threshold and a first preset IV threshold, a semantic enhancement process is initiated. This process includes: mapping the feature variable to a corresponding entity in the business knowledge graph; calculating the association path strength between the entity and known high-risk entities; and if the association path strength of a feature variable is higher than a preset path threshold, then using the formula: The IV value of the feature variable is compensated. If the corrected IV value is greater than the first preset IV threshold, the feature variable is retained. The method for calculating the strength of the associated path is as follows: in the business knowledge graph, the weighted sum of all paths between two entities is calculated. The strength of a single path is the product of the weights of all business associations on the path, multiplied by the attenuation coefficient of the path length. The selected and retained feature variables are normalized, standardized, or logarithmically transformed to form the final feature set.
[0010] In one alternative implementation, the method for selecting a machine learning model based on the risk objective includes: When the risk target is credit risk, a logistic regression or gradient boosting tree model is selected, and the graph embedding features extracted from the business knowledge graph are used as supplementary inputs to form the model input together with the final feature set, so as to enhance the model's ability to identify associated risks. When the risk target is fraud risk, a model resource pool consisting of decision tree model, isolated forest model and graph neural network is configured.
[0011] In one alternative implementation, the method further includes constructing a model switching strategy based on real-time risk event density, the model switching strategy comprising: Real-time statistics on the number of risk events per unit of time; When the number of risk events does not exceed the preset density threshold, inference is performed using the decision tree model or the isolated forest model. When the number of risk events exceeds the preset density threshold, the inference is switched to the graph neural network model.
[0012] In one optional implementation, the model switching strategy includes: When the logistic regression model is selected, the risk probability is output through the Sigmoid function, and the weights of the feature variables are provided to explain the decision-making basis. At the same time, the entities corresponding to the top K feature variables that contribute the most to the risk decision and their association paths with known risk patterns in the business knowledge graph are extracted from the business knowledge graph as causal link information output to assist decision-making. When the isolated forest model is selected, the probability of the feature variable being selected as a split point when constructing each tree of the isolated forest is dynamically adjusted based on the frequency of the feature variable as the main distinguishing feature in historically confirmed fraud samples, and the higher the frequency, the higher the probability of being selected.
[0013] In one alternative implementation, the performance evaluation model includes: For models selected for credit risk or fraud risk, AUC, precision, and recall are used as evaluation metrics. For the scorecard model built based on the logistic regression model, the KS value is used as the evaluation index.
[0014] In one optional implementation, the monitoring model performance metric includes monitoring accuracy drift, and the feature variable distribution includes feature variable distribution drift; the reinforcement learning agent's online adaptive iterative optimization process for the model includes: When the monitoring accuracy drift is detected to cause performance degradation or the feature variable distribution drift exceeds the preset iteration trigger threshold, the reinforcement learning agent triggers the incremental learning or parameter adjustment process of the model; and generates an audit log by combining the triggering reason of each model iteration optimization, the adjusted parameters, the performance comparison data before and after the adjustment, and the location of the related feature variable drift root cause in the business knowledge graph, and records it to the blockchain evidence storage system.
[0015] Secondly, this application discloses a system for applying the data risk identification method based on the risk control model described above, the system comprising: The data preparation module is configured to: identify risk targets and collect multi-source data, clean and preprocess the collected data to improve data quality; among them, for risk scenarios with scarce samples, a generative adversarial network is used to generate synthetic samples that conform to the risk characteristics of the risk scenario, and mix them with real samples to construct a balanced training dataset for model training. The feature processing module is configured to: construct and filter feature variables associated with the risk objective from the preprocessed data, and transform the feature variables to obtain the final feature set; The model building module is configured to: use samples in the balanced training dataset and their corresponding final feature sets to select a machine learning model for training and hyperparameter tuning according to the risk objective, and evaluate the performance and stability of the model using cross-time validation or cross-validation. The model iteration module is configured to: deploy validated models to the production environment and continuously monitor model performance metrics and feature variable distribution; and construct a reward function based on reinforcement learning agents with the goal of comprehensively improving business metrics, and dynamically adjust the model's decision threshold or feature variable weights based on real-time feedback of business and model data.
[0016] Beneficial Effects: The data risk identification method and system based on risk control models of this invention, by constructing a complete technical closed loop from data augmentation, semantic feature engineering, dynamic model selection to online adaptive iteration, realizes the intelligentization and automation of the entire lifecycle of the risk control model. It can improve the ability to identify risk patterns of scarce samples and enhance the business semantics of features using knowledge graphs, thereby comprehensively improving the accuracy and recall of risk identification. At the same time, it can intelligently schedule the optimal model according to the real-time risk situation and realize the online self-optimization of the model based on reinforcement learning and continuous monitoring mechanisms, ensuring the high robustness and business adaptability of the risk control process in a dynamically changing environment, and ultimately achieving a synergistic improvement in the accuracy and efficiency of risk control. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 A flowchart illustrating the data risk identification method based on a risk control model provided in this application embodiment; Figure 2 The structural block diagram of the data risk identification system based on the risk control model provided in the embodiments of this application is shown. Detailed Implementation
[0019] To facilitate understanding of the technical solutions provided in the embodiments of this application, the background technology involved in the embodiments of this application will be described below.
[0020] In the fields of modern fintech and e-commerce, risk control is a core element in ensuring business security. Traditional risk control systems, primarily built on rule engines and statistical models, suffer from several significant technical bottlenecks: 1. Sample scarcity problem: In the field of risk control, the ratio of positive to negative samples is often severely imbalanced. Taking credit card fraud detection as an example, normal transactions usually account for more than 99%, while fraudulent transactions account for less than 1%. This sample imbalance makes it difficult for machine learning models to effectively learn the feature patterns of the minority class. Especially for new fraud methods, the recognition accuracy of traditional models drops significantly due to the lack of historical samples.
[0021] 2. Limitations of Feature Engineering: Traditional feature engineering mainly relies on expert experience to manually construct features, lacking the ability to mine deep semantic relationships between data. For example, in anti-money laundering scenarios, a single transaction amount feature is insufficient to effectively identify money laundering behavior, while the relationships within a transaction network often contain richer risk information.
[0022] 3. Model Stagnation and Performance Degradation: Existing risk control models often maintain fixed parameters after deployment, making them unable to adapt to rapidly changing business environments. When new fraud patterns emerge or business rules are adjusted, model performance will significantly degrade, requiring manual intervention for retraining. This results in long response times and fails to meet real-time risk control requirements.
[0023] Therefore, a more reliable data security management technology is urgently needed. Based on this, this implementation proposes a data risk identification method and system based on a risk control model. By constructing a complete technical closed loop from data augmentation, semantic feature engineering, dynamic model selection to online adaptive iteration, it effectively solves the aforementioned technical challenges.
[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present application. Secondly, in this document, the term "comprising" is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.
[0025] This embodiment provides a data risk identification method based on a risk control model in its first aspect, such as Figure 1 As shown, the model includes the data preparation stage, feature processing stage, model building and validation stage, and model deployment and iteration stage, each of which is described in detail below.
[0026] I. Data Preparation Stage This phase includes: identifying risk targets and collecting multi-source data; cleaning and preprocessing the collected data to improve data quality; and, for risk scenarios with scarce samples, using generative adversarial networks to generate synthetic samples that conform to the risk characteristics of the risk scenario, and mixing them with real samples to construct a balanced training dataset for model training.
[0027] In one implementation example, the risk objective could be in a credit scenario, such as whether a user will default on payments within the next 3 months (label: 1 = default, 0 = normal), or in an e-commerce scenario, such as whether a transaction constitutes fraudulent activity (label: 1 = fraud, 0 = normal). Multi-source data includes content data and external data. For different risk objectives, internal data in a credit scenario could be "behavioral characteristics (such as login frequency in the past 6 months)" or "credit characteristics (such as repayment records)," while external data could be such as central bank credit reporting. Internal data in an e-commerce scenario could be "anti-fraud relationship characteristics (such as device fingerprints)" or "transaction characteristics (such as transaction amount fluctuations)," while external data could be such as logistics address verification.
[0028] Specifically, the cleaning and preprocessing of the collected data includes: Missing values can be filled using mean imputation or model prediction imputation. Use the IQR rule or Z-score method to identify and remove outliers; One-hot encoding or label encoding is performed on categorical feature variables; The preprocessed data is correlated and verified with a pre-built business knowledge graph. The business relationships in the business knowledge graph are used to identify and remove isolated abnormal data points that do not conform to business logic, so as to obtain high-quality cleaned data.
[0029] In one implementation example, for handling missing values, numerical features such as "monthly income" are filled with the mean value. For example, in a credit dataset, the average monthly income is 8500 yuan, and missing values are filled. For predictive numerical features such as "number of overdue payments in the past 3 months," XGBoost prediction is used to fill in the missing values, and a predictive model is built using relevant features such as "number of credit inquiries" and "historical repayment records" (correlation coefficients of 0.6-0.8).
[0030] Secondly, for outlier handling, the IQR rule is used to process single transaction amounts. Referring to the e-commerce scenario where "daily small transactions are the main focus," the quartiles are calculated as follows: Q1 = 300 yuan (25th quartile, corresponding to "regular consumption of 100-500 yuan") and Q3 = 3000 yuan (75th quartile, corresponding to "large but reasonable consumption"). The interquartile range formula is: Substituting the data yields Yuan, remove those less than (-3750 yuan, meaningless) or greater Abnormal transactions (e.g., a single non-daily expense of 12,000 yuan) are considered abnormal, thus eliminating extreme values. The frequency of login location changes is processed using the Z-score method, referencing industry data showing that "ordinary users log in from different locations an average of 1-3 times per month" to calculate the average. Times / month, standard deviation The Z-score formula is as follows: (Time / Month) Remove (i.e., the number of logins is greater than) Abnormal data (e.g., a user logs in from a different location 8 times per month, rounded down to 6 times per month). (This is used to determine the threshold for abnormal behavior), thereby achieving the determination of abnormal behavior thresholds.
[0031] Furthermore, for categorical feature coding, one-hot coding is used for unordered variables such as "occupation," for example, "teacher" is coded as [1,0,0,0] (corresponding to four categories: "teacher, white-collar worker, student, and freelancer," covering 80% of credit users' occupational types); for ordered variables such as "credit rating" (A>B>C>D), label coding is used, with A=4, B=3, C=2, and D=1.
[0032] In addition, for the verification of business knowledge graph associations, for example, if a user's "account location is Shanghai, but the login IPs for the past month have all been Chengdu and there are no login records from other locations", and the association rule of "user-account-login region" in the business knowledge graph is "90% of the login IPs of Shanghai accounts are from the Yangtze River Delta region" (based on the statistics of 100,000 historical data), this data is determined to be an "isolated outlier" and removed.
[0033] Specifically, the steps for constructing a business knowledge graph include: Extract core entities related to risk control business and their business relationships from multi-source data. Core entities may include users (U), accounts (A), devices (D), merchants (M), regions (L), etc., and business relationships may include holding (UA), ownership (DU), transaction (AM), location (AL), etc. Each business relationship is assigned a weight, with the initial weight of any business relationship set based on the relationship type and frequency of occurrence, and dynamically adjusted during operation based on the frequency of occurrence of that business relationship in historical risk cases.
[0034] In one implementation example, for the initial weights, "Transaction (AM)" is directly associated with fraud risk, so its initial weight is set to 0.8 to ensure a high weight for core risk features; "Location (AL)" is indirectly associated with risk, so its initial weight is set to 0.3 to ensure a low weight for non-core features. Based on this, during dynamic adjustment, if "Transaction (AM)" appears 15 times in historical fraud cases (out of a total of 120 cases), the weight adjustment formula is used: Substituting the data yields This ensures that high-risk relationships are given greater weight.
[0035] Furthermore, for risk scenarios with scarce samples (such as the scarcity of samples in the "new type of delinquency" in the credit scenario (50 real positive samples, 5000 negative samples, a positive-to-negative ratio of 1:100), the DCGAN model (the mainstream sample generation algorithm in the industry) is adopted. Inputting real positive sample features ("credit inquiries in the past 3 months > 5 times + monthly repayment amount < 30% of income"), 250 synthetic samples are generated. Secondly, the following sample quality assessment process is used to ensure sample quality: (1) Define parameters: : Synthetic sample quality score; Feature distribution similarity weights (prioritizing features that closely resemble reality); Business logic and tag consistency weight; Feature distribution similarity (KL divergence calculation, value 0-1, the smaller the KL, the better). The closer to 1); Business logic consistency (rule base validation, value 0-1); Label consistency (the matching rate between the predicted labels and the generated labels of the pre-trained model, with a value of 0-1).
[0036] (2) According to the formula Calculate the quality score of the synthetic sample, with a value between 0 and 1.
[0037] (3) When the quality score of the synthetic sample is not lower than 0.8, it is considered qualified, that is, the synthetic sample can be used and mixed with the real sample to obtain a balanced training dataset, thus avoiding the underfitting problem of the model biased to the prediction of negative samples.
[0038] II. Feature Processing Stage This stage includes: constructing and filtering feature variables associated with risk objectives from the preprocessed data, and transforming the feature variables to obtain the final feature set.
[0039] Specifically, the selection and transformation of feature variables to obtain the final feature set includes: (1) Calculate the IV (Information Value) of the initially constructed feature variables, and select feature variables with IV values greater than the first preset IV threshold (e.g., 0.3).
[0040] (2) Calculate the Pearson correlation coefficient among the initially constructed feature variables and remove redundant feature variables whose correlation coefficient is higher than the preset correlation coefficient threshold (e.g., 0.9).
[0041] For feature variables whose IV values fall between the second preset IV threshold (e.g., 0.15) and the first preset IV threshold, a semantic enhancement process is initiated. The semantic enhancement process includes: The feature variable is mapped to the corresponding entity in the business knowledge graph. The association path strength between the entity and the known high-risk entity is calculated. If the association path strength of a feature variable is higher than the preset path threshold, the IV value of the feature variable is compensated. If the corrected IV value is greater than the first preset IV threshold, the feature variable is retained.
[0042] The strength of a single path is calculated by multiplying the weights of all business relationships along the path by a decay coefficient based on the path length. In other words, the strength of a single path is... , The weight of the i-th association on the path (e.g., 0.7 for belonging, 0.825 for transaction); This represents the total number of relationships along the path. The path length (i.e., n); This is the attenuation coefficient (value 0.9, the longer the path, the weaker the risk correlation).
[0043] The method for calculating the strength of a related path is as follows: In the business knowledge graph, calculate the weighted sum of all paths between two entities. The formula for the strength of the related path, i.e., the total path strength, is as follows: k is the total number of paths.
[0044] IV value formula for characteristic variables: Corrected IV value formula: That is, the corrected IV value = original IV value × (1 + compensation coefficient × associated path strength), where the compensation coefficient... The value range is from 0.1 to 0.5.
[0045] (3) Normalize, standardize or logarithmically transform all the selected and retained feature variables to form the final feature set.
[0046] III. Model Building and Validation Phase This phase includes: using samples from the balanced training dataset and their corresponding final feature sets, selecting a machine learning model for training and hyperparameter tuning based on the risk objective (e.g., for gradient boosting tree models, adjusting parameters such as learning rate, tree depth, and subsample ratio; for graph neural networks, adjusting parameters such as the number of convolutional layers, number of attention heads, and dropout rate), and evaluating the performance and stability of the model using cross-time validation or cross-validation.
[0047] Specifically, methods for selecting machine learning models based on risk objectives include: When the risk target is credit risk, logistic regression or gradient boosting tree model is selected, and graph embedding features extracted from the business knowledge graph are used as supplementary inputs to form the model input together with the final feature set, so as to enhance the model's ability to identify associated risks. When the risk target is fraud risk, configure a model resource pool consisting of decision tree model, isolated forest model and graph neural network.
[0048] In one implementation example, for credit risk (credit scenario), a logistic regression model is selected (as stated in the risk control document "Credit Risk Preferred Explainable Model"), supplemented with Node2Vec graph embedding features (128-dimensional vectors covering "user-account-merchant" relationships) to improve the model's AUC. For fraud risk (e-commerce scenario), a model resource pool is constructed, including: a decision tree model (CART algorithm) for rapid reasoning of simple fraud patterns, identifying features such as "unfamiliar device + login from a different location + single transaction amount > 5000 yuan" (high-risk rules in e-commerce scenarios, based on historical fraud case statistics); an isolated forest model for identifying new types of unlabeled fraud, handling features such as "newly registered accounts initiate > 8 transfers within 1 hour" (typical characteristics of new types of fraud in the industry); and a graph neural network (GNN) for identifying group fraud, mining features such as "more than 5 accounts sharing the same IP + all transferring to the same merchant" (group fraud association patterns).
[0049] Furthermore, the method also includes constructing a model switching strategy based on real-time risk event density, the model switching strategy including: Real-time statistics on the number of risk events per unit of time; When the number of risk events does not exceed a preset density threshold (e.g., 10 times / hour), the decision tree model or the isolated forest model is selected for inference. When the number of risk events exceeds a preset density threshold, the system switches to inference using a graph neural network model to achieve an adaptive balance between detection efficiency and inference depth.
[0050] The preset density threshold is the boundary between low and high fraud risk. For low density (e.g., 5 times / hour), choose decision tree (inference latency < 40ms, meeting the real-time transaction requirements of e-commerce). For high density (e.g., 18 times / hour), switch to GNN (recall rate increases from 85% to 93%, achieving high-risk density requires associative inference).
[0051] Specifically, model switching strategies include: When selecting a logistic regression model, the risk probability is output through the Sigmoid function, and the weights of the feature variables are provided to explain the decision-making basis (e.g., the weight of "number of overdue payments in the past 3 months" = 0.78 (for every additional overdue payment, the probability of default increases by 16%). Simultaneously, from the business knowledge graph, the entities corresponding to the top K feature variables that contribute the most to risk decisions and their association paths with known risk patterns in the business knowledge graph are extracted as causal link information to assist in decision-making (e.g., the top 3 association paths are extracted (e.g., "number of overdue payments - U3 - A5 - overdue record R4"), as the basis for risk control review). When the isolated forest model is selected, the probability of a feature variable being selected as a split point when constructing each tree of the isolated forest is dynamically adjusted based on the frequency with which that feature variable is the main distinguishing feature in historically confirmed fraud samples. The higher the frequency, the higher the probability of being selected, in order to improve the detection efficiency of known fraud patterns.
[0052] IV. Model Deployment and Iteration Phase This phase includes: deploying the validated model (deployment methods include offline batch processing deployment or online real-time inference deployment; offline batch processing deployment: processing historical data through scheduled tasks to generate risk reports; online real-time inference deployment: receiving real-time requests through a RESTful API service and returning risk assessment results within 100ms) to the production environment, and continuously monitoring model performance metrics and feature variable distribution; and constructing a reward function based on reinforcement learning agents with the goal of comprehensively improving business metrics, and dynamically adjusting the model's decision threshold or feature variable weights based on real-time feedback of business and model data to perform online adaptive iterative optimization of the model.
[0053] Specifically, evaluating the model's performance includes: For models selected for credit risk or fraud risk (i.e., binary classification tasks), AUC, precision, and recall are used as evaluation metrics. For models selected for credit scenarios, evaluation metrics such as AUC > 0.85 are considered excellent, precision = 0.85 (low false rejection rate), and recall = 0.87 (low false negative rate) are used. For models selected for e-commerce scenarios, evaluation metrics such as test set AUC = 0.91, precision = 0.82, and recall = 0.90 are used. For scorecard models built based on logistic regression, the KS value is used as the evaluation index. For example, when building a credit score of 300-850 points based on logistic regression (such as the FICO scoring system), the KS value of the test set is 0.48 (KS > 0.4 is an excellent scorecard), and the score difference between high-risk and low-risk groups is > 200 points (strong discrimination).
[0054] Furthermore, the performance metrics of the monitoring model include monitoring accuracy drift, and the feature variable distribution includes feature variable distribution drift.
[0055] Furthermore, the process of online adaptive iterative optimization of the model by the reinforcement learning agent includes: When monitoring accuracy drift leads to performance degradation or feature variable distribution drift exceeds the preset iteration trigger threshold, the reinforcement learning agent triggers the incremental learning or parameter adjustment process of the model. The triggering reason for each model iteration optimization, the adjusted parameters, the performance comparison data before and after the adjustment, and the location of the root cause of the related feature variable drift in the business knowledge graph are jointly generated into an audit log and recorded in the blockchain evidence storage system to form a traceable and tamper-proof closed loop for model lifecycle management.
[0056] In one optional implementation, the drift contribution of the feature variable is determined by the following formula. : in, The contribution of the i-th feature to the total drift (values range from 0 to 1, summing to 1, with a value not less than 0.3 set as the main drift feature). The drift level of the i-th feature is represented by the PSI (Population Stability Index), which ranges from 0 to 1, with a PSI > 0.25 indicating significant drift. Let be the risk discrimination index (i.e., the aforementioned IV value) for the i-th feature. This represents the total number of feature variables. This quantifies the contribution of a single feature, enabling precise identification of the feature variables causing drift and avoiding indiscriminate retraining.
[0057] In one implementation example, the monitoring data for the five core features of the credit model are shown in Table 1: Table 1 Based on the data in Table 1, C was calculated. 月收入 =0.36860.144≈0.391≥0.3, C 征信查询次数 =0.36860.1428≈0.387≥0.3, then “monthly income” and “number of credit inquiries” are determined to be the main drift features, and only these two features need to be reprocessed.
[0058] Furthermore, the reward function used in this embodiment is as follows: in, The total reward value (ranging from -2 to 2, with a positive value indicating that the iteration is valid); This is a weight for model performance (e.g., a value of 0.3). As a risk loss weight (e.g., a value of 0.4), As a weight for business revenue (e.g., a value of 0.3); , which represents the average change in model performance; This represents the rate of change in risk loss (a negative value indicates a decrease). This represents the rate of change in business revenue (positive indicates an increase). The difference between the model AUC value after iteration and the model AUC value before iteration. This is the difference between the model recall rate after iteration and the model recall rate before iteration. This avoids the bias of a single metric (such as "increased recall but a surge in loss").
[0059] This embodiment provides a second aspect as follows: Figure 2 The data risk identification system based on the risk control model shown is applied to the data risk identification method based on the risk control model described above. The system includes: The data preparation module is configured to: identify risk targets and collect multi-source data, clean and preprocess the collected data to improve data quality; among them, for risk scenarios with scarce samples, a generative adversarial network is used to generate synthetic samples that conform to the risk characteristics of the risk scenario, and mix them with real samples to construct a balanced training dataset for model training. The feature processing module is configured to: construct and filter feature variables associated with risk objectives from the preprocessed data, and transform the feature variables to obtain the final feature set; The model building module is configured to: use samples from the balanced training dataset and their corresponding final feature sets to select a machine learning model for training and hyperparameter tuning based on the risk objective, and evaluate the performance and stability of the model using cross-time validation or cross-validation. The model iteration module is configured to: deploy validated models to the production environment and continuously monitor model performance metrics and feature variable distribution; and construct a reward function based on reinforcement learning agents with the goal of comprehensively improving business metrics, and dynamically adjust the model's decision threshold or feature variable weights based on real-time feedback of business and model data.
[0060] The modules are interconnected through a system bus and work together to achieve automated and intelligent management of the risk control process.
[0061] It should be noted that the data risk identification system based on the risk control model in this embodiment corresponds to the aforementioned data risk identification method based on the risk control model. Therefore, the parts of the data risk identification system based on the risk control model in this embodiment that are not specifically described (including but not limited to specific technical means and technical effects) can be referred to the relevant descriptions in the aforementioned data risk identification method based on the risk control model, and will not be repeated here.
[0062] This embodiment also provides a computer device, including a memory and a processor. The memory stores information including program instructions, and the processor controls the execution of the program instructions. When the program instructions are loaded and executed by the processor, they implement the aforementioned data risk identification method based on a risk control model. Specifically, the computer device is configured with: a CPU with no fewer than 8 cores, a memory capacity of no less than 32GB, and a GPU accelerator card to support deep learning model training and inference. The operating system uses a Linux distribution and has the necessary deep learning framework and database management system installed.
[0063] In the embodiments provided by this invention, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any suitable combination thereof. For hardware implementation, the processor can be implemented in one or more of the following: application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to implement the functions described herein, or combinations thereof. For software implementation, some or all of the processes of the embodiments can be performed by a computer program instructing the associated hardware. During implementation, the program can be stored in a computer-readable storage medium or transmitted as one or more instructions or code on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a computer. Computer-readable storage media can include, but are not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer.
[0064] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for data risk identification based on a risk control model, characterized in that, The method comprises: a data preparation stage: determining a risk target and collecting multi-source data, and cleaning and preprocessing the collected data to improve data quality; wherein for a risk scenario with scarce samples, a generative adversarial network is used to generate synthetic samples of risk features conforming to the risk scenario, and the synthetic samples are mixed with real samples to construct a balanced training dataset for model training; a feature processing stage: constructing and screening feature variables associated with the risk target from the preprocessed data, and converting the feature variables to obtain a final feature set; a model construction and verification stage: using the samples in the balanced training dataset and their corresponding final feature set, selecting a machine learning model according to the risk target for training and hyperparameter tuning, and evaluating the performance and stability of the model using cross-time verification or cross-validation; a model deployment and iteration stage: deploying the verified model to a production environment, continuously monitoring model performance indicators and feature variable distributions, and based on a reinforcement learning agent, constructing a reward function with the goal of comprehensive improvement of business indicators, dynamically adjusting the decision threshold or feature variable weight of the model based on real-time feedback of business and model data, to perform online adaptive iterative optimization of the model.
2. The method of claim 1, wherein, The cleaning and preprocessing of the collected data comprises: using mean filling or model prediction filling to handle missing values; using IQR rule or Z-score method to identify and remove outliers; one-hot encoding or label encoding for categorical feature variables; associating and verifying the preprocessed data with a preconstructed business knowledge graph, identifying and removing isolated abnormal data points that do not conform to business logic using the business association relationships in the business knowledge graph, to obtain high-quality cleaned data.
3. The method of claim 2, wherein, The construction steps of the business knowledge graph comprise: extracting core entities related to risk control business and their business association relationships from the multi-source data; assigning a weight to each business association relationship, wherein the initial weight of any business association relationship is set based on the relationship type and frequency of occurrence, and is dynamically adjusted according to the frequency of occurrence of the business association relationship in historical risk cases during operation.
4. The method of claim 3, wherein, In the feature processing stage, the feature variables are screened and converted to obtain the final feature set, comprising: calculating the IV value of the initially constructed feature variables, and selecting feature variables with IV values greater than a first preset IV threshold; calculating the Pearson correlation coefficient between the initially constructed feature variables, and removing redundant feature variables with correlation coefficients higher than a preset correlation coefficient threshold; For the feature variable with the IV value between the second preset IV threshold and the first preset IV threshold, a semantic enhancement process is started, and the semantic enhancement process comprises: mapping the feature variable to a corresponding entity in the business knowledge graph, calculating the association path strength between the entity and a known high-risk entity, and if the association path strength of a feature variable is higher than a preset path threshold, then the IV value of the feature variable is compensated by the formula: If the modified IV value is greater than the first preset IV threshold, the feature variable is retained; wherein the calculation method of the association path strength is: in the business knowledge graph, the weighted sum of all paths between two entities is calculated; the strength of a single path is the product of the weights of all business association relationships on the path, multiplied by the attenuation coefficient of the path length; normalizing, standardizing or logarithmically converting all the screened and retained feature variables to form the final feature set. 5.The method of claim 1, wherein, The way of selecting a machine learning model according to the risk target comprises: when the risk target is credit risk, selecting a logistic regression or gradient boosting tree model, and using graph embedding features extracted from the business knowledge graph as supplementary input, together with the final feature set, to constitute model input, to enhance the model's ability to identify associated risks; when the risk target is fraud risk, configuring a model resource pool consisting of decision tree models, isolation forest models and graph neural networks.
6. The method of claim 5, wherein, The method further comprises constructing a model switching strategy based on a real-time risk event density, the model switching strategy comprising: counting the number of risk events per unit time in real time; when the number of risk events does not exceed a preset density threshold, selecting inference by the decision tree model or the isolated forest model; when the number of risk events exceeds the preset density threshold, switching to inference by the graph neural network model.
7. The method of claim 5, wherein the risk control model is a machine learning model. The model switching strategy comprises: when the logistic regression model is selected, outputting a risk probability by a Sigmoid function and providing feature variable weights to explain the basis for decision-making; meanwhile, extracting, from the business knowledge graph, entities corresponding to the top K feature variables with the highest contribution to risk decision-making and their associated paths with known risk patterns in the business knowledge graph as causal link information for auxiliary decision-making output; when the isolated forest model is selected, the probability of a feature variable being selected as a split point is dynamically adjusted according to the frequency of the feature variable as a primary distinguishing feature in historical confirmed fraud samples during construction of each tree of the isolated forest, and the higher the frequency, the higher the probability of being selected. 8.The method of claim 1, wherein, The performance of the evaluation model comprises: for the model selected for credit risk or fraud risk, AUC, precision and recall are used as evaluation indicators; for a scoring card model constructed based on a logistic regression model, a KS value is used as an evaluation indicator.
9. The method of claim 1, wherein, The monitoring of model performance indicators comprises monitoring accuracy drift, and the feature variable distribution includes feature variable distribution drift; The process of online adaptive iterative optimization of the reinforcement learning agent on the model comprises: when the monitored monitoring accuracy drift leads to performance degradation or the feature variable distribution drift exceeds a preset iteration trigger threshold, the reinforcement learning agent triggers an incremental learning or parameter adjustment process of the model; and the trigger cause of each model iteration optimization, the adjusted parameters, the performance comparison data before and after adjustment, and the positioning of the associated feature variable drift source in the business knowledge graph are collectively generated as an audit log and recorded to a blockchain storage system.
10. A system for applying the data risk identification method based on a risk control model according to any one of claims 1-9, characterized in that, The system comprises: a data preparation module configured to determine a risk target and collect multi-source data, and clean and preprocess the collected data to improve data quality; wherein for a risk scenario with scarce samples, a generative adversarial network is used to generate synthetic samples of risk features conforming to the risk scenario, and the synthetic samples are mixed with real samples to construct a balanced training dataset for model training; a feature processing module configured to construct and select feature variables associated with the risk target from the preprocessed data, and convert the feature variables to obtain a final feature set; a model construction module configured to use samples in the balanced training dataset and their corresponding final feature set to train and optimize hyperparameters of a machine learning model selected according to the risk target, and evaluate the performance and stability of the model using cross-time validation or cross-validation methods; The model iteration module is configured to: deploy the verified model to a production environment and continuously monitor model performance indicators and feature variable distributions; and based on a reinforcement learning agent, construct a reward function with the goal of comprehensive improvement of business indicators, and dynamically adjust the decision threshold or feature variable weight of the model based on real-time feedback of business and model data.