An intelligent financial risk control system based on ABTest technology

Abstract

This invention discloses an intelligent financial risk control system based on ABTest technology, belonging to the field of financial risk control technology. It includes a process configuration module, a strategy management module, an ABTest management module, a decision execution engine, a verification and analysis module, a machine learning model integration module, and a real-time monitoring and early warning module. The system allows for the configuration of risk control decision-making processes in a visual manner and supports the flexible integration of various strategy tools and machine learning models. Utilizing ABTest and a champion-challenger (Champion Challenger) mechanism, it conducts parallel testing and scientific evaluation of online strategies and challenger strategies. Based on statistical indicators and model performance indicators, it automatically analyzes and recommends the optimal strategy, achieving intelligent switching and iteration of strategies. This invention features high flexibility, scalability, and intelligence, improving the accuracy, efficiency, and system stability of risk control decisions. It is suitable for the risk management and business optimization needs of financial institutions in rapidly changing business environments.

Description

Technical Field

[0001] This invention relates to the field of financial risk control technology, specifically to a financial risk intelligent risk control system based on ABTest technology. Background Technology

[0002] As the digitalization and internetization of financial services accelerate, financial institutions face increasingly complex and frequently changing risks. Traditional risk control models rely mainly on manual review, static rules, and limited historical data analysis, which results in problems such as long approval cycles, high labor costs, lagging rule updates, and difficulty in dealing with new types of fraud and credit risks.

[0003] Although some institutions have introduced automated risk control systems based on rule engines in recent years, their strategy iterations still largely rely on offline assessments and manual decisions. New strategies lack sufficient validation in real business scenarios before going live, which can easily lead to strategies failing to meet expectations or triggering new risks.

[0004] Furthermore, existing risk control systems typically lack flexible strategy comparison and evaluation mechanisms. When multiple risk control strategies need to be validated, they are often deployed sequentially or piloted on a small scale. This not only results in long validation cycles but also makes it difficult to objectively and fairly evaluate the merits of strategies due to factors such as changes in the business environment and sample bias. In addition, strategy switching mostly relies on manual judgment and operation, which is slow to respond and cannot adapt to rapidly changing business needs.

[0005] Therefore, there is an urgent need for a financial risk intelligent risk control system that can integrate flexible configuration, scientific verification, intelligent decision-making and real-time monitoring to improve risk control efficiency, accuracy and adaptability, and support financial institutions in achieving digital transformation and continuous optimization of risk management in a rapidly changing market environment. Summary of the Invention

[0006] The purpose of this invention is to provide a financial risk intelligent risk control system based on ABTest technology, which integrates technologies such as big data, machine learning, artificial intelligence, decision support, software engineering, and risk management, providing financial institutions with an efficient, intelligent, and reliable risk control decision support platform to solve the problems mentioned in the background.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a financial risk intelligent risk control system based on ABTest technology, comprising:

[0008] The process configuration module receives user input and configures the financial risk control decision process in a visual manner. The financial risk control decision process includes several decision nodes, which are connected by condition lines.

[0009] The strategy management module includes online risk control strategies and challenger risk control strategies;

[0010] The ABTest management module is used to configure ABTest verification tasks for the same financial risk control decision process. The ABTest verification tasks include: associating the challenger risk control strategy and the online risk control strategy with the same decision process node, setting traffic allocation rules for the risk control strategies respectively, and conducting parallel testing with the corresponding real traffic.

[0011] The decision execution engine responds to risk control decision requests and, according to the traffic allocation rules in the ABTest verification task, dynamically routes the risk control decision requests to the corresponding online risk control strategy or challenger risk control strategy for execution, and collects the execution result data of each solution.

[0012] The verification and analysis module is used to analyze and compare the execution result data of each collected scheme according to preset verification rules, and automatically switch to the optimal scheme based on the comparison results.

[0013] Preferably, the decision execution engine specifically includes, according to the traffic allocation rules, allocating risk control decision requests with the same characteristics or belonging to the same time period to the online risk control strategy scheme and the at least one challenger risk control strategy scheme for parallel execution.

[0014] Preferably, the verification rules include at least one of the following:

[0015] Rules based on statistical indicators: compare the overall pass rate, rejection rate, or bad debt rate of each plan within a preset time period;

[0016] The rule based on model performance metrics is to compare the stability metrics or predictive performance metrics of the machine learning models associated with each scheme. The stability metrics include the population stability index (PSI), and the predictive performance metrics include the area under the receiver operating characteristic (AUC) or the Kolmogorov-Smirnov KS value.

[0017] Preferably, the process configuration module specifically includes generating and storing structured process configuration data, which consists of decision nodes and connection relationships configured by the user through visual operations.

[0018] The decision node type includes at least one of start node, end node, rule node, rule set node, scorecard node and model node; the condition line is a directed line, and each directed line is associated with a flow condition, which includes pass condition, rejection condition or abnormal condition.

[0019] Preferably, the system also includes a machine learning model integration module, which receives and deploys externally trained machine learning model files and encapsulates the deployed models as model nodes for the visualization decision flow configuration module to orchestrate into the decision process.

[0020] As a preferred method, the method of automatically switching the optimal strategy based on the comparison results includes:

[0021] Based on the comparison results, when it is determined that the overall performance of the challenger risk control strategy is better than the current online risk control strategy, and its advantage exceeds the preset winning threshold, the strategy switching instruction is automatically generated.

[0022] The strategy switching instruction is sent to the strategy management module and the decision execution engine to update the winning challenger risk control strategy to a new online risk control strategy, which will then take effect in subsequent decision requests.

[0023] Preferably, the machine learning model integration module integrates a model stability evaluation unit, which is used to calculate at least one of the PSI value, KS value or AUC value of the deployed model in real time based on historical data, so that the real-time monitoring and warning module can monitor it.

[0024] Preferably, the system also includes a real-time monitoring and warning module, wherein the monitoring objects of the real-time monitoring and warning module include: the number of calls to the decision process, the pass rate, the rejection rate, and the average execution time;

[0025] The business metrics performance of each scheme in the ABTest strategy verification module;

[0026] The warning trigger conditions are set based on the number of anomalies or the anomaly rate counted within the sliding time window.

[0027] Preferably, the strategy management module further includes a strategy editing and management unit, which provides unified strategy editing and management for the online risk control strategy and the challenger risk control strategy;

[0028] The implementation logic of the risk control strategy scheme is defined by calling and combining at least one strategy tool in the pre-set strategy tool library, which includes: basic rules, rule sets, scoring cards, decision tables, decision trees, and decision matrices.

[0029] Preferably, the predefined traffic allocation rules include at least one of the following configuration methods:

[0030] a) Proportional allocation: Set a fixed percentage of traffic for the online risk control strategy and each challenger's risk control strategy;

[0031] b) Quantity allocation: Set a number of requests to be processed within a specified time window for the online risk control strategy and each challenger's risk control strategy;

[0032] c) Time period allocation: Set specific effective time periods for each challenger's risk control strategy.

[0033] In summary, the beneficial effects of this invention are:

[0034] High flexibility and scalability: The data model adopts a modular design, which can easily adapt to different business scenarios and decision-making needs. It supports the upload of machine learning model scripts in multiple formats, including PMML, Python and R language files, so that users can choose according to their own technology stack.

[0035] Powerful decision support capabilities: It provides a rich and flexible set of strategy tools, such as rules, rule sets, rule strategies, formula editor, decision tables, decision trees, decision matrices and scorecards, to meet the decision-making needs of complex business scenarios. The introduction of machine learning models brings more possibilities to business decision-making and improves the level of decision intelligence.

[0036] Efficient visual configuration: Utilizing SVG technology to visualize decision flow configuration, users can easily complete process configuration through drag-and-drop actions, lowering the configuration threshold, improving work efficiency, and providing an intuitive and easy-to-use visual configuration interface, making it convenient for users to set monitoring configurations and monitoring strategies.

[0037] Precise business iteration and validation: Through A / B testing and crowning techniques, enterprises can efficiently evaluate the merits of new ideas and solutions, ensuring accurate implementation in real-world business scenarios. These technologies provide strong support for enterprises to quickly adapt to market changes and continuously optimize products and services.

[0038] Comprehensive monitoring and alerting services: Provides comprehensive and flexible monitoring and alerting services to ensure stable operation at all levels and for all objects. When a monitored object reaches or exceeds a preset number of anomalies or anomaly rate threshold, the system will automatically trigger an alert mechanism, improving the system's response speed and accuracy.

[0039] Easy to understand and maintain: The data model is clearly designed and easy to understand, facilitating subsequent system maintenance and upgrades. Visual configuration and monitoring alert services provide a clear overview of the system status, allowing users to promptly identify and address potential problems. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the overall process framework of a financial risk intelligent risk control system based on ABTest technology according to the present invention;

[0042] Figure 2 This is a schematic diagram illustrating the configuration start and end nodes of the decision-making process in an intelligent financial risk control system based on ABTest technology according to the present invention.

[0043] Figure 3 This is a schematic diagram of the scoring card node configuration for creating a decision-making process in a financial risk intelligent risk control system based on ABTest technology according to the present invention.

[0044] Figure 4 This is a schematic diagram illustrating the configuration of rule nodes for creating a decision-making process in an intelligent financial risk control system based on ABTest technology according to the present invention.

[0045] Figure 5 This is a schematic diagram of the rule set node configuration for creating a decision-making process in a financial risk intelligent risk control system based on ABTest technology according to the present invention.

[0046] Figure 6 This is a schematic diagram of the ABTest scenario configuration in a financial risk intelligent risk control system based on ABTest technology according to the present invention;

[0047] Figure 7 This is a schematic diagram of the crowning scenario configuration in a financial risk intelligent risk control system based on ABTest technology according to the present invention. Detailed Implementation

[0048] The present invention will now be described in further detail with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. These drawings are simplified schematic diagrams, which are only used to illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0049] To facilitate understanding of the present invention, a more complete description of the invention will be given below with reference to the accompanying drawings, which illustrate several embodiments of the invention. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided so that the disclosure of the invention will be more thorough and complete.

[0050] All features disclosed in this specification, or all steps in all disclosed methods or processes, may be combined in any way, except for mutually exclusive features and / or steps.

[0051] Any feature disclosed in this specification (including any appended claims, abstract, and drawings) may be replaced by other equivalent or similar features for a similar purpose, unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is merely one example of a series of equivalent or similar features.

[0052] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; they can refer to the internal communication of at least two elements or the interaction relationship of at least two elements, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0053] Please see Figures 1-7 This invention provides an embodiment of a financial risk intelligent risk control system based on A / B testing technology. This system integrates technologies from big data, machine learning, artificial intelligence, decision support, software engineering, and risk management, providing financial institutions with an efficient, intelligent, and reliable risk control decision support platform. The system includes: a process configuration module, a strategy management module, an A / B testing management module, a verification analysis module, a machine learning model integration module, and a real-time monitoring and warning module.

[0054] The process configuration module receives user input and configures the financial risk control decision-making process in a visual manner. The process includes several decision nodes connected by conditional lines, enabling flexible adaptation to the complex and ever-changing process requirements of various financial decision-making scenarios. Through its modular design, users can easily build, adjust, and optimize the decision-making process according to business needs, ensuring the efficiency and accuracy of the decision path.

[0055] Specifically, the financial risk control decision-making process consists of a series of carefully designed node modules, including but not limited to start node, end node, rule node, rule set node, scorecard node, rule strategy node, formula editor node, model node, decision tree node, decision table node, decision matrix node, list management node, and decision node.

[0056] Relationships between nodes: To visually represent the complex relationships between these nodes, we introduce directed line nodes. These are subdivided according to function into directed lines for passing conditions, directed lines for rejection, and directed lines for exceptions. These lines tightly connect upstream and downstream nodes, forming a logically rigorous and clearly defined decision-making network.

[0057] Scorecard Node: Integrates complex logical judgment conditions, enabling accurate evaluation and output of scorecard results. (Reference) Figure 3 Scorecard node configuration.

[0058] Rule / Strategy Node: By combining multiple logical judgment conditions, it enables flexible rule configuration and efficient strategy execution. (See reference...) Figure 4 and Figure 5 ,in Figure 5 Configure the rule set node.

[0059] Formula Editor Node: Supports custom formula editing and has a built-in complex logical judgment mechanism to meet diverse calculation needs.

[0060] Model nodes encapsulate the complex logical judgments of model results, improving the level of intelligent decision-making.

[0061] List Management Node: Provides list option management and executes corresponding logic based on whether a list is hit or not, enhancing the accuracy of decision-making.

[0062] Decision node: It integrates functions such as decision return value, custom return value, previous node value, and list writing to ensure the comprehensiveness and flexibility of decision results.

[0063] It should be noted that when configuring the financial risk control decision-making process, we cleverly introduced SVG (Scalable Vector Graphics) technology. This technology, based on the precise description of images, achieves miniaturization of file size and lossless image scaling, laying a solid foundation for the visualization configuration of the decision flow.

[0064] The core advantages of SVG technology:

[0065] High-fidelity scaling: SVG images maintain their original clarity and detail when enlarged or reduced, avoiding the distortion problems that may occur in traditional image formats during scaling.

[0066] Lightweight files: Thanks to its description-based storage method, SVG files are relatively small in size, which effectively improves loading speed and system responsiveness.

[0067] Visualization practices for decision flow configuration:

[0068] Icon differentiation: We designed unique icons for different model nodes. These icons are not only intuitive and easy to identify, but also enhance the readability and aesthetics of the decision flow configuration interface.

[0069] Drag-and-drop configuration: Leveraging the visualization capabilities of SVG technology, we have implemented a visual interface for decision flow configuration. Users can easily complete configurations ranging from simple to complex processes through simple drag-and-drop actions, greatly reducing the configuration threshold and improving work efficiency. See details... Figure 2 You can create decision-making processes on the canvas by dragging and dropping, and configure start and end nodes.

[0070] Data modeling and persistence:

[0071] Datafication of Visualization Results: To transform SVG visualizations into storable and manageable data, we conducted detailed data modeling. This step ensured that user-drawn decision flowcharts were accurately recorded in a structured format.

[0072] Database persistence: The modeled data is persisted to the database, which means that the decision flowcharts drawn by users can be preserved in a real and complete manner, making it easy to view, modify and reuse them later.

[0073] By introducing SVG technology and implementing a visual configuration strategy, our decision flow configuration function not only achieves an efficient and flexible configuration experience, but also ensures high-fidelity storage and convenient management of decision flowcharts, providing strong support for the intelligent upgrade of decision service systems.

[0074] The strategy management module and decision-making services play a crucial role in fields such as financial risk control, relying on diverse strategy tools for accurate and efficient decision-making. Therefore, this product has meticulously crafted a comprehensive and flexible set of decision-making tools to meet the diverse needs of business stakeholders.

[0075] This decision-making toolset encompasses a variety of core tools, including but not limited to:

[0076] 1. Basic Rules: Rules receive data metrics as input parameters and configure conditions according to the user-selected rule category. After appropriate logical judgment, the rule outputs a "pass" or "reject" result. Furthermore, rules also support if / else rule types, which can be used directly as a single node in the decision flow, or as an important element in components such as rule sets, rule strategy management, formula editors, decision tables, decision trees, and decision matrices.

[0077] 2. Rule Sets: A rule set is a decision-making method composed of multiple rules, used to determine rule fulfillment. As a key node in the decision-making process, rule sets can be used independently by the decision flow or as part of decision tables, decision trees, and decision matrices. The output of a rule set depends on the execution results of its internal rules, providing strong support for complex decisions. By combining multiple basic rules, more complex decision logic can be formed, improving the flexibility of decision services.

[0078] 3. Rule-based Strategies: Rule-based strategies enable a more refined decision-making approach by configuring rule weights. They accept rules and data metrics as input parameters and calculate scores for conditions met based on configured logic. Rule-based strategy management provides more flexible and accurate evaluation tools for business decisions, allowing users to customize specific decision-making strategies according to business scenarios and meet personalized decision-making needs.

[0079] 4. Formula Editor: The formula editor is an application based on input parameters of variables, rules, formulas, and constants. By configuring conditional logic, the formula editor can output corresponding numerical results. This tool provides powerful computational capabilities and flexibility for business decision-making, supports user-defined calculation formulas, and provides strong mathematical support for decision-making services.

[0080] 5. Decision Table: The decision table uses a tabular format for risk decision-making. It outputs decisions by invoking components such as data metrics, rules, rule sets, and scorecards. As a crucial node in the decision-making process, the decision table provides an intuitive and user-friendly interface with robust functional support for business decisions. It visually displays the decision logic in tabular form, facilitating user understanding and operation.

[0081] 6. Decision Tree: Decision tree management uses a tree-structured diagram to display decision problems. Compared to decision tables, decision trees offer a different representation, similarly using components such as data metrics, rules, rule sets, and scorecards to deliver decision output. Decision trees provide a more intuitive and easier-to-understand decision path for business decisions, displaying the decision path in a tree structure and supporting the rapid location and optimization of key decision nodes.

[0082] 7. Decision Matrix: The decision matrix uses a two-dimensional matrix approach to configure the decision-making process, providing another flexible and powerful tool for business decision-making. By configuring different conditions and output results, the decision matrix can meet the decision-making needs of complex business scenarios. By comparing different decision-making schemes in matrix form, it helps users make the optimal choice.

[0083] 8. Scorecard: Taking data metrics as input parameters, score situations that meet certain conditions based on configured conditional logic. As a single node, the scorecard can be directly used in the decision flow or serve as an important component of decision tables, decision trees, and decision matrices. Scorecards provide a quantitative evaluation tool for business decisions, helping to improve decision-making efficiency and accuracy. They quantify decision-making objects according to preset scoring criteria, providing intuitive decision-making basis.

[0084] These decision-making tools not only possess unique advantages individually, but can also be used in combination to form more complex and sophisticated decision-making logic. By flexibly combining these tools, this product can comprehensively meet the diverse needs of business stakeholders in decision-making services, ensuring the accuracy and efficiency of decision-making results.

[0085] ABTest management module, see reference Figure 6 This system introduces an advanced A / B testing validation scheme, providing financial institutions with efficient and accurate data-driven decision support. Specifically, the A / B testing scheme randomly allocates real traffic to different experimental and control groups within the same time period to effectively validate new businesses or scenarios. Different risk control strategies or business logics are applied to the experimental and control groups, and the actual effectiveness of the new business or scenario is evaluated by collecting and analyzing the performance of the two groups' data. It supports configuring multiple test schemes and allows for flexible settings of traffic ratios, traffic volumes, and validation times according to actual needs. This flexible and diverse control method enables enterprises to accurately evaluate the performance of different schemes in a real-world environment, thereby quickly determining which scheme is superior.

[0086] It is worth mentioning that, for reference Figure 7The A / B testing solution is enhanced with the Crown Challenge technology, improving testing efficiency. The Crown Challenge technology further upgrades the solution verification process, allowing enterprises to configure multiple challenger solutions in addition to their existing online solutions and allocate a portion of real traffic for parallel testing. After a period of real traffic comparison, if a challenger solution performs well, it is judged as a successful challenger and automatically replaced with the new online solution. This process not only ensures the fairness and objectivity of solution verification but also significantly accelerates the pace of business iteration. This means that enterprises can leverage A / B testing's traffic splitting and control capabilities during the Crown Challenge testing process to conduct a more detailed and comprehensive evaluation of multiple challenger solutions. This technological integration not only improves verification efficiency but also provides enterprises with more diversified decision-making basis.

[0087] ABTest and crown testing technologies, as important tools for business iteration and verification, provide strong support for enterprises to quickly adapt to market changes and continuously optimize products and services due to their accuracy and efficiency.

[0088] Based on the ABTest verification scheme integrating the Challenger Technology, the strategy management module uses the strategy editing and management unit to edit and generate online risk control strategies and the challenger risk control strategies. The challenger risk control strategies and the online risk control strategies are associated with the same decision process node, and traffic allocation rules are set for each risk control strategy. Through the decision execution engine, risk control decision requests with the same characteristics or belonging to the same time period are allocated to the online risk control strategy scheme and the at least one challenger risk control strategy scheme for parallel execution. The corresponding real traffic is used for parallel testing, and the test result data of each scheme is collected.

[0089] The predefined traffic allocation rules include at least one of the following configuration methods:

[0090] a) Proportional allocation: Set a fixed percentage of traffic for the online risk control strategy and each challenger's risk control strategy;

[0091] b) Quantity allocation: Set a number of requests to be processed within a specified time window for the online risk control strategy and each challenger's risk control strategy;

[0092] c) Time period allocation: Set specific effective time periods for each challenger's risk control strategy.

[0093] The verification and analysis module is used to analyze and compare the execution result data of each collected scheme according to preset verification rules, and automatically switch to the optimal scheme based on the comparison results.

[0094] The verification rules mentioned above include at least one of the following:

[0095] Based on statistical indicators, the rule is to compare the overall pass rate, rejection rate, or bad debt rate of each scheme within a preset time period.

[0096] The rule based on model performance metrics is to compare the stability metrics or predictive performance metrics of the machine learning models associated with each scheme. The stability metrics include the population stability index (PSI), and the predictive performance metrics include the area under the receiver operating characteristic curve (AUC) or the Kolmogorov-Smirnov KS value.

[0097] Specifically, the PSI metric measures the stability of the model's predicted distribution over different time periods, helping users identify potential model biases in a timely manner. The ROC curve visually reflects the model's classification performance by displaying the True Positive Rate and False Positive Rate at different thresholds. The KS curve further highlights the model's ability to distinguish between positive and negative samples, providing users with a more in-depth perspective on model evaluation. All these stability metrics are calculated based on feedback from real data, ensuring the accuracy and practicality of the results. Furthermore, the product's user interface is simple and clear; users can easily obtain model stability reports with just a few simple steps, greatly improving ease of use.

[0098] Based on the comparison results, when it is determined that the overall performance of the challenger risk control strategy is better than the current online risk control strategy, and its advantage exceeds the preset winning threshold, the strategy switching instruction is automatically generated.

[0099] The strategy switching instruction is sent to the strategy management module and the decision execution engine to update the winning challenger risk control strategy to a new online risk control strategy, which will then take effect in subsequent decision requests.

[0100] It is worth mentioning that, in this embodiment, a machine learning model integration module is also included, which is used to receive and deploy externally trained machine learning model files, and encapsulate the deployed model as the model node, so that the visualization decision flow configuration module can orchestrate it into the decision process.

[0101] It supports machine learning models such as linear regression and logistic regression formulas. These models can be configured via a visual interface, including modeling elements such as coefficients, variables, constants, and output value types and ranges. During the modeling process, the system also provides evaluation metrics such as PSI, KS, and AUC values ​​to help users optimize model performance. Linear and logistic regression formula templates offer users a concise and clear modeling approach, facilitating rapid implementation of business decision-making needs.

[0102] Linear formula template:

[0103] Y = Constant + Coefficient B1 * Variable 1 + Coefficient B2 * Variable 2

[0104] Logistic Regression Formula Template:

[0105] Ln(odds) = constant A + coefficient B1 * variable 1 + coefficient B2 * variable 2

[0106] Model result = constant C + constant D / Ln2 * Ln(odds).

[0107] The system offers a rich and flexible suite of strategy tools to meet the complex decision-making needs of various business scenarios. From rules, rule sets, and rule strategies to tools such as formula editors, decision tables, decision trees, decision matrices, and scorecards, all provide users with powerful decision support capabilities. Furthermore, the introduction of machine learning models brings more possibilities to business decision-making, helping enterprises achieve intelligent decision-making and efficient operations.

[0108] The system provides an efficient and convenient way to easily integrate offline-trained machine learning models into the system, achieving a deep fusion of decision services and model technology. Specifically, users can upload pre-trained model files, model scripts, and sample data to the system to create and manage this trained model information.

[0109] The system fully supports uploading model scripts in various formats, including but not limited to PMML files, Python files, and R language files. This diverse support allows users to choose the most suitable model script format for uploading based on their preferences and technology stack. Furthermore, the system provides a user-friendly upload interface and workflow, ensuring users can easily complete the model file import process.

[0110] Through this mechanism, machine learning models can be seamlessly integrated into financial risk control decision-making service systems, enabling direct connection between model predictions and decision-making processes. This not only simplifies the complexity of model deployment and application but also significantly improves the intelligence level and response speed of the decision-making system. Users can quickly enjoy the business optimization and efficiency improvements brought by machine learning technology without having to perform cumbersome model integration and configuration work.

[0111] The system seamlessly integrates decision-making services with machine learning technology by providing comprehensive and convenient functions for uploading and managing machine learning models. This key technological feature not only demonstrates the system's high flexibility and scalability but also brings users a more intelligent and efficient decision support experience.

[0112] It is also worth mentioning that, in this embodiment, a real-time monitoring and warning module is included. The monitoring objects of the real-time monitoring and warning module include: the number of calls to the decision process, the pass rate, the rejection rate, and the average execution time.

[0113] The business metrics performance of each scheme in the ABTest strategy verification module;

[0114] The warning trigger conditions are set based on the number of anomalies or the anomaly rate counted within the sliding time window.

[0115] In our financial risk control decision-making service system, we provide users with comprehensive and flexible monitoring and alerting services to ensure the stable operation of all levels and objects. Through an intuitive and user-friendly visual configuration interface, users can easily set up monitoring configurations to achieve comprehensive monitoring of business system calls, decision flow objects, rule objects, and other dimensions.

[0116] In terms of monitoring strategy, we adopted a time-window-based approach to setting thresholds for the number of anomalies or the anomaly rate. When the monitored object reaches or exceeds the preset threshold for the number of anomalies or the anomaly rate within the set time window, the system will automatically trigger an alarm mechanism, sending alarm information to the user via SMS and email. This mechanism not only improves the system's response speed and accuracy but also helps users to promptly identify and address potential risks and problems.

[0117] Our financial risk control decision-making service system provides users with efficient and reliable decision support by offering comprehensive, multi-layered monitoring and alerting services, an intuitive and easy-to-use configuration interface, and flexible monitoring strategies. This key technology not only demonstrates the system's powerful risk management capabilities but also reflects our deep attention to user experience and needs.

[0118] In summary, the advantages of this invention can be summarized as follows:

[0119] High flexibility and scalability:

[0120] The data model adopts a modular design, which can easily adapt to different business scenarios and decision-making needs.

[0121] It supports uploading machine learning model scripts in multiple formats, including PMML, Python, and R language files, allowing users to choose according to their own technology stack.

[0122] Strong decision support capabilities:

[0123] It provides a rich and flexible set of strategy tools, such as rules, rule sets, rule strategies, formula editors, decision tables, decision trees, decision matrices, and scorecards, to meet the decision-making needs of complex business scenarios.

[0124] The introduction of machine learning models has brought more possibilities to business decision-making and improved the level of intelligence in decision-making.

[0125] Efficient visual configuration:

[0126] By leveraging SVG technology to visualize decision flow configuration, users can easily complete process configuration through drag-and-drop actions, lowering the configuration threshold and improving work efficiency.

[0127] It provides an intuitive and easy-to-use visual configuration interface, making it convenient for users to set monitoring configurations and monitoring policies.

[0128] Precise business iteration and validation:

[0129] Through A / B testing and crowning techniques, enterprises can efficiently evaluate the merits of new ideas and solutions, ensuring accurate implementation in real-world business scenarios. These technologies provide strong support for enterprises to quickly adapt to market changes and continuously optimize their products and services.

[0130] Comprehensive monitoring and alerting services:

[0131] It provides comprehensive and flexible monitoring and alarm services to ensure stable operation at all levels and for all objects. When a monitored object reaches or exceeds a preset number of anomalies or anomaly rate threshold, the system will automatically trigger an alarm mechanism, improving the system's response speed and accuracy.

[0132] Easy to understand and maintain:

[0133] The data model is clearly designed and easy to understand, facilitating subsequent system maintenance and upgrades. Visual configuration and monitoring / alarm services provide a clear overview of the system status, enabling users to promptly identify and address potential problems.

[0134] In practice, the existing online risk control strategy of the consumer finance platform (referred to as "Strategy A") is used, which is mainly based on rule sets and scoring cards for approval. In order to improve the approval rate and control bad debts, the risk control team developed a new strategy that integrates machine learning models (referred to as "Strategy B"), and its effectiveness needs to be verified in real traffic.

[0135] Step 1: Configure the decision-making process in the process configuration module.

[0136] Risk control personnel can configure an approval decision-making process by dragging and dropping nodes through a visual interface:

[0137] Start Node → Rule Set Node (Anti-fraud Rules) → Model Node (Credit Scoring Model) → Decision Node (Output Approval Results) → End Node.

[0138] In the decision flow, the "model node" is currently associated with the traditional scorecard used by strategy A.

[0139] Step 2: Prepare the challenger strategy in the strategy management module

[0140] Create "Strategy B" in the strategy editor and combine the following tools:

[0141] Rule set: Anti-fraud rules (consistent with strategy A);

[0142] Model node: The newly launched XGBoost credit assessment model (deployed by uploading PMML files through the machine learning model integration module);

[0143] Decision table: Different approval limits are set according to the model output;

[0144] Step 3: Configure the verification task in the ABTest management module.

[0145] Create an ABTest task "Loan Strategy Comparison and Verification":

[0146] Control group: Online strategy A, allocating 70% of traffic;

[0147] Experimental group: Challenger strategy B, allocating 30% of the traffic;

[0148] Traffic allocation method: Randomly allocated based on user ID hash value;

[0149] Verification period: 14 days;

[0150] Step 4: Verification rule settings

[0151] Set evaluation metrics in the validation analysis module:

[0152] Business metrics: Approval rate, bad debt rate (30-day overdue rate);

[0153] Model metrics: AUC (to assess the model's discriminative power), PSI (to assess the model's stability);

[0154] Winning threshold: Strategy B's pass rate increases by ≥2% and its bad debt rate is not higher than that of Strategy A;

[0155] 3. System Execution Process

[0156] Real-time routing for decision execution engine:

[0157] When user Zhang San applies for a loan, the system assigns him to strategy group B (30% of traffic) based on his user ID hash value.

[0158] The engine calls strategy B to execute: first, it executes the anti-fraud rules, then it calls the XGBoost model to score, and finally it outputs "Pass, credit limit 8000 yuan" through the decision table.

[0159] Simultaneously, the feature data, model score, and approval results of the request are collected, labeled, and stored.

[0160] Real-time monitoring and early warning:

[0161] The monitoring panel displays:

[0162] Strategy A: Pass rate 41%, bad debt rate 1.05%, average response time 85ms;

[0163] Strategy B: Pass rate 44%, bad debt rate 0.98%, average response time 92ms;

[0164] Model stability: Strategy B has a PSI of 0.03 (<0.1, stable);

[0165] The system calculates metrics hourly and no alerts are triggered.

[0166] 4. Validation Analysis and Automatic Switching

[0167] Validation and analysis module cycle evaluation:

[0168] After the 14-day test, the system automatically compiled the results:

[0169] Strategy A: Approval rate 41.2%, bad debt rate 1.07%, AUC = 0.78

[0170] Strategy B: Pass rate 44.5% (+3.3%), bad debt rate 0.99% (-0.08%), AUC=0.82

[0171] The comparison results meet the preset winning threshold.

[0172] Automatically generate switching instructions: The system determines that strategy B has better overall performance and automatically generates a strategy switching instruction. The instruction is sent to the strategy management module and the decision execution engine: strategy B is updated to the new online strategy, traffic allocation is adjusted to 100% flow to strategy B, and strategy A is archived as a historical version.

[0173] 5. Monitoring after switching

[0174] After Strategy B was fully implemented, the real-time monitoring module continuously tracked the results: the pass rate remained stable at around 44%, the bad debt rate remained below 1%, the model PSI value remained below 0.1 without drift, and the system automatically generated a performance report and pushed it to the risk control team.

[0175] Through this system, the risk control team has shortened the strategy iteration cycle from the previous 1-2 months to 2 weeks, while ensuring full verification and stability before the new strategy goes live.

[0176] The above description is merely a specific embodiment of the invention, but the scope of protection of the invention is not limited thereto. Any variations or substitutions conceived without inventive effort should be included within the scope of protection of the invention. Therefore, the scope of protection of the invention should be determined by the scope defined in the claims.

Claims

1. A financial risk intelligent risk control system based on A / B testing technology, characterized in that: include: The process configuration module receives user input and configures the financial risk control decision process in a visual manner. The financial risk control decision process includes several decision nodes, which are connected by condition lines. The strategy management module includes online risk control strategies and challenger risk control strategies; The ABTest management module is used to configure ABTest verification tasks for the same financial risk control decision process. The ABTest verification tasks include: associating the challenger risk control strategy and the online risk control strategy with the same decision process node, setting traffic allocation rules for the risk control strategies respectively, and conducting parallel testing with the corresponding real traffic. The decision execution engine responds to risk control decision requests and, according to the traffic allocation rules in the ABTest verification task, dynamically routes the risk control decision requests to the corresponding online risk control strategy or challenger risk control strategy for execution, and collects the execution result data of each solution. The verification and analysis module is used to analyze and compare the execution result data of each collected scheme according to preset verification rules, and automatically switch to the optimal scheme based on the comparison results.

2. The intelligent financial risk control system based on ABTest technology according to claim 1, characterized in that: The decision execution engine specifically includes, according to the traffic allocation rules, allocating risk control decision requests with the same characteristics or belonging to the same time period to the online risk control strategy scheme and the at least one challenger risk control strategy scheme for parallel execution.

3. The intelligent financial risk control system based on ABTest technology according to claim 2, characterized in that: The verification rules include at least one of the following: Rules based on statistical indicators: compare the overall pass rate, rejection rate, or bad debt rate of each plan within a preset time period; The rule based on model performance metrics is to compare the stability metrics or predictive performance metrics of the machine learning models associated with each scheme. The stability metrics include the population stability index (PSI), and the predictive performance metrics include the area under the receiver operating characteristic (AUC) or the Kolmogorov-Smirnov KS value.

4. The intelligent financial risk control system based on ABTest technology according to claim 1, characterized in that: The process configuration module specifically includes generating and storing structured process configuration data, which consists of decision nodes and connection relationships configured by the user through visual operations. The decision node type includes at least one of start node, end node, rule node, rule set node, scorecard node and model node; the condition line is a directed line, and each directed line is associated with a flow condition, which includes pass condition, rejection condition or abnormal condition.

5. The intelligent financial risk control system based on ABTest technology according to claim 1, characterized in that: It also includes a machine learning model integration module, which is used to receive and deploy externally trained machine learning model files, and encapsulate the deployed models as model nodes for the visualization decision flow configuration module to orchestrate into the decision process.

6. The intelligent financial risk control system based on ABTest technology according to claim 1, characterized in that: Methods for automatically switching to the optimal strategy based on comparison results include: Based on the comparison results, when it is determined that the overall performance of the challenger risk control strategy is better than the current online risk control strategy, and its advantage exceeds the preset winning threshold, the strategy switching instruction is automatically generated. The strategy switching instruction is sent to the strategy management module and the decision execution engine to update the winning challenger risk control strategy to a new online risk control strategy, which will then take effect in subsequent decision requests.

7. The intelligent financial risk control system based on ABTest technology according to claim 5, characterized in that: The machine learning model integration module integrates a model stability evaluation unit, which is used to calculate at least one of the PSI value, KS value or AUC value of the deployed model in real time based on historical data, so that the real-time monitoring and warning module can monitor it.

8. The intelligent financial risk control system based on ABTest technology according to claim 1, characterized in that: It also includes a real-time monitoring and warning module, whose monitoring objects include: the number of calls to the decision process, the pass rate, the rejection rate, and the average execution time; The business metrics performance of each scheme in the ABTest strategy verification module; The warning trigger conditions are set based on the number of anomalies or the anomaly rate counted within the sliding time window.

9. The intelligent financial risk control system based on ABTest technology according to claim 1, characterized in that: The strategy management module also includes a strategy editing and management unit, which provides unified strategy editing and management for the online risk control strategy and the challenger risk control strategy; The implementation logic of the risk control strategy scheme is defined by calling and combining at least one strategy tool in the pre-set strategy tool library, which includes: basic rules, rule sets, scoring cards, decision tables, decision trees, and decision matrices.

10. A financial risk intelligent risk control system based on ABTest technology according to claim 1, characterized in that: The traffic allocation rules include at least one of the following: allocating traffic according to a fixed ratio, allocating traffic according to a fixed quantity, or allocating traffic according to a time window.

Images