Bank digital intelligence model enabling platform and digital intelligence system

Abstract

The invention relates to the technical field of digital intelligence, in particular to a bank digital intelligence model enabling platform and a digital intelligence system. According to the application, firstly, performance reference standards of various models are established through the reference threshold acquisition module, then, historical data are deeply analyzed based on the model stability coefficient acquisition module, and stability characteristics of the models in different scenes are quantified; and then the stability coefficient and the reference threshold value are combined through the monitoring threshold value updating module to generate the dynamic threshold value, and finally the self-adaptive updating of the monitoring standard is realized through the threshold value adjusting module, so that the monitoring strategy can be automatically adjusted according to the historical performance characteristics of the model and the current service scene, and the accuracy and the practicability of model monitoring are remarkably improved.

Description

Technical Field

[0001] The present application relates to the technical field of digitalization, and in particular, to a bank digital intelligence model enabling platform and digital intelligence system. Background Art

[0002] With the rapid development of financial technology, the digital transformation of the banking industry has continued to deepen, and the application scope of artificial intelligence models in banking business has been expanding. These models play an important role in many fields such as credit assessment, risk control, and marketing recommendations, and are of great significance to improving the operating efficiency and service quality of banks.

[0003] The existing bank model monitoring system mainly adopts a fixed threshold monitoring method, which monitors the model operation status in real time by setting static performance indicator thresholds. When the model performance indicator exceeds the preset threshold, the system automatically triggers the early warning mechanism to remind relevant personnel to intervene and handle it.

[0004] However, due to the complexity and diversity of banking business scenarios, the performance of different models in different business scenarios varies significantly. The fixed threshold monitoring method cannot adapt to the dynamic changes in model performance, and is prone to false positives or false negatives, reducing the accuracy and efficiency of monitoring. This situation needs to be further improved. Summary of the invention

[0005] In order to solve the problem that the existing fixed threshold monitoring method cannot adapt to the dynamic changes of model performance and is prone to false positives or false negatives, this application provides a bank digital intelligence model empowerment platform and digital intelligence system, which adopts the following technical solutions: In a first aspect, the present application provides a bank digital intelligence model enabling platform, the bank digital intelligence model enabling platform includes a model monitoring device, and the model monitoring device includes: A benchmark threshold acquisition module is used to obtain the performance benchmark threshold of each model; A model stability coefficient acquisition module is used to acquire the historical operation data of each model, and acquire the model stability coefficient of each model according to the historical operation data; A monitoring threshold updating module, used for obtaining an updated monitoring threshold according to the model stability coefficient and the performance benchmark threshold; The threshold adjustment module is used to update the performance benchmark threshold with the updated monitoring threshold, and to monitor the operation of the bank's digital intelligence model empowerment platform according to the updated monitoring threshold.

[0006] By adopting the above technical solution, in order to solve the problem that the fixed monitoring threshold of the bank's intelligent model becomes invalid due to the dynamic changes in the business environment in actual operation, for example, in the credit card anti-fraud model, if a fixed 95% accuracy rate is used as the monitoring threshold, during special periods such as holidays, due to significant changes in transaction behavior, even if the model performs normally, it may trigger a false alarm. When the means of fraud evolve rapidly, the threshold may be too loose and the model performance degradation may not be discovered in time. This application first establishes performance benchmark standards for various models through a benchmark threshold acquisition module, and then conducts in-depth analysis of historical data based on the model stability coefficient acquisition module to quantify the stability characteristics of the model in different scenarios. Then, the monitoring threshold update module combines the stability coefficient with the benchmark threshold to generate a dynamic threshold. Finally, the threshold adjustment module realizes the adaptive update of the monitoring standard, which can automatically adjust the monitoring strategy according to the historical performance characteristics of the model and the current business scenario, significantly improving the accuracy and practicality of model monitoring.

[0007] Optionally, the model stability coefficient acquisition module includes: The model scenario acquisition submodule is used to obtain the model business category of each model; A performance stability coefficient acquisition submodule is used to obtain a model performance stability coefficient based on the historical operation data; The model stability coefficient calculation submodule is used to calculate the model stability coefficient according to the model business category and the performance stability coefficient.

[0008] By adopting the above technical solution, since the requirements of bank intelligent models for stability under different business categories are significantly different, simply using a single indicator to evaluate the stability of the model cannot accurately reflect the importance and stability requirements of the model. For example, risk assessment models (such as credit scoring and anti-fraud) have extremely high requirements for stability, and fluctuations may lead to serious risk problems; customer analysis models (such as customer profiling and churn warning) have moderate requirements for stability and can accept moderate fluctuations; marketing recommendation models have relatively low requirements for stability and have a certain tolerance. This application first identifies the business category to which the model belongs (such as risk assessment, customer analysis, marketing recommendation, operational decision-making, etc.) through the model scenario acquisition submodule, and then uses the performance stability coefficient acquisition submodule to calculate the basic performance stability index of the model based on historical operation data. Finally, the model stability coefficient calculation submodule performs differentiated weighting of performance stability according to the importance of different business categories to obtain a model stability evaluation result that is more in line with the actual business. It can establish differentiated stability evaluation standards based on the business category characteristics of the model to improve the accuracy and reliability of the evaluation.

[0009] Optionally, the performance stability coefficient acquisition submodule includes: An operation data acquisition unit, used to acquire model performance operation data from the historical operation data; The performance stability coefficient calculation unit is used to calculate the model performance stability coefficient according to the performance benchmark threshold and the performance operation data.

[0010] By adopting the above technical solution, since the historical operation data of the bank's intelligent model contains a large amount of non-performance related information, directly using the original data to calculate the performance stability coefficient will introduce noise and affect the accuracy of the evaluation. This application first uses the operation data acquisition unit to filter and extract the operation records of the model's core performance indicators from the historical data, and then uses the performance stability coefficient calculation unit to compare and analyze these performance operation data with the preset performance benchmark threshold, calculate the degree of fluctuation of the performance indicator and the degree of deviation from the benchmark, and thus obtain a quantitative indicator that reflects the true stability level of the model, effectively improving the accuracy and reliability of the stability evaluation.

[0011] Optionally, the monitoring threshold updating module includes: The performance change data acquisition submodule is used to acquire the model operation data within a preset time period and analyze and obtain the performance change data; A scenario impact coefficient acquisition submodule, used to calculate a business scenario impact coefficient based on the performance change data; The monitoring threshold calculation submodule is used to calculate the updated monitoring threshold according to the model stability coefficient, the business scenario impact coefficient and the performance benchmark threshold.

[0012] By adopting the above technical solution, since the performance changes of the bank model are dynamically affected by multiple factors, relying solely on the static model stability coefficient cannot accurately reflect the actual impact of the current business environment on the model performance; this application first analyzes the model operation data within a preset time period through the performance change data acquisition sub-module to obtain the change trend of the model performance; then, the scenario impact coefficient acquisition sub-module is used to quantify the impact of the current business scenario according to the performance change characteristics; finally, the model stability coefficient, business scenario impact coefficient and performance benchmark threshold are integrated and calculated through the monitoring threshold calculation sub-module to generate a dynamic monitoring threshold that is more in line with the current business environment, thereby improving the ability of the monitoring standard to dynamically adjust with the business environment.

[0013] Optionally, the monitoring threshold calculation submodule includes: A model scenario judgment unit, used to determine the business scenario type of the model, wherein the business scenario type includes daily transaction scenarios, peak business scenarios, marketing activity scenarios, and system maintenance scenarios; An initial threshold calculation unit, used to adjust the performance benchmark threshold according to the model stability coefficient to obtain an initial monitoring threshold; The final threshold updating unit is used to adjust the initial monitoring threshold according to the business scenario impact coefficient to obtain an updated monitoring threshold.

[0014] By adopting the above technical solution, the present application first clarifies the specific business scenario type of the current model through the model scenario judgment unit, and then performs the first-level adjustment of the performance benchmark threshold based on the model stability coefficient through the initial threshold calculation unit to obtain the initial monitoring threshold that reflects the intrinsic stability of the model; finally, the business scenario influence coefficient is introduced through the final threshold update unit for the second-level adjustment to obtain the final monitoring threshold that considers both model stability and scenario influence; through the hierarchical threshold calculation process, the interference caused by the direct fusion of multiple factors is avoided, so that the role of each influencing factor can be more accurately reflected.

[0015] Optionally, the threshold adjustment module further includes: A fluctuation coefficient acquisition unit is used to obtain the model operation fluctuation coefficient; A threshold updating unit, configured to update the performance benchmark threshold according to the model operation fluctuation coefficient; A monitoring unit, configured to monitor the model operation status according to the updated monitoring threshold; An early warning judgment unit is used to compare the model operation status with the updated monitoring threshold to determine whether an early warning is triggered; The warning sending unit is used to send warning information when the warning condition is triggered.

[0016] By adopting the above technical solution, the present application first obtains the short-term operation fluctuation characteristics of the unit calculation model through the fluctuation coefficient, and then the threshold updating unit dynamically adjusts the performance benchmark threshold according to the fluctuation coefficient. Then, the monitoring unit is used to continuously track the operation status of the model, and the operation status is compared with the updated monitoring threshold in multiple dimensions through the early warning judgment unit. Finally, the early warning sending unit sends out graded early warning information in time when an abnormality is found, thereby improving the ability of abnormality identification and early warning.

[0017] Optionally, the fluctuation coefficient obtaining unit includes: A time window division subunit is used to divide the model operation data according to a preset time window; The performance index calculation subunit is used to calculate the mean and variance of the model performance index in each time window; The fluctuation coefficient calculation subunit is used to calculate the model operation fluctuation coefficient according to the mean and variance of the performance indicator.

[0018] By adopting the above technical solution, the present application first divides the model operation data into multiple time windows according to business characteristics through the time window division subunit, and then the performance indicator calculation subunit calculates the performance mean and variance in each window respectively. Finally, the fluctuation coefficient calculation subunit comprehensively considers the statistical characteristics of each time window to obtain a more accurate model operation fluctuation coefficient.

[0019] Optionally, the models monitored by the bank digital intelligence model enabling platform include: Risk assessment models, used for credit scoring, anti-fraud and risk warning; Customer analysis models, used for customer profiling, churn warning, and value assessment; Marketing recommendation models, used for product recommendations and marketing response prediction; Operational decision-making models are used for pricing, quota management, and resource allocation.

[0020] By adopting the above technical solutions, this application divides bank models into four categories: risk assessment, customer analysis, marketing recommendation and operational decision-making. A differentiated monitoring indicator system is set up according to the characteristics of each type of model, and a differentiated monitoring system that adapts to the characteristics of different models is established, thereby achieving accurate allocation of monitoring resources and highlighting the focus of risk management.

[0021] In a second aspect, the present application provides a banking digitalization system, including: The above-mentioned banking digital intelligence model empowerment platform; Business systems, used to provide basic business data and transaction processing; Data warehouse platform, used for unified management and processing of various types of data; Application platform, used to host application development and deployment.

[0022] In summary, the present application includes at least one of the following beneficial technical effects: This application first establishes performance benchmark standards for various models through a benchmark threshold acquisition module, then conducts in-depth analysis of historical data based on a model stability coefficient acquisition module, quantifies the stability characteristics of the model in different scenarios, and then combines the stability coefficient with the benchmark threshold through a monitoring threshold update module to generate a dynamic threshold. Finally, the threshold adjustment module implements adaptive updates of monitoring standards, which can automatically adjust monitoring strategies based on the historical performance characteristics of the model and the current business scenario, significantly improving the accuracy and practicality of model monitoring. Since the requirements for stability of bank intelligent models under different business categories are significantly different, simply using a single indicator to evaluate the stability of the model cannot accurately reflect the importance and stability requirements of the model. For example, risk assessment models (such as credit scoring and anti-fraud) have extremely high requirements for stability, and fluctuations may lead to serious risk problems; customer analysis models (such as customer profiling and churn warning) have moderate requirements for stability and can accept moderate fluctuations; marketing recommendation models have relatively low requirements for stability and have a certain tolerance. This application first identifies the business category to which the model belongs (such as risk assessment, customer analysis, marketing recommendation, operational decision-making, etc.) through the model scenario acquisition submodule, and then uses the performance stability coefficient acquisition submodule to calculate the basic performance stability index of the model based on historical operation data. Finally, the model stability coefficient calculation submodule performs differentiated weighting of performance stability according to the importance of different business categories to obtain a model stability evaluation result that is more in line with the actual business. It can establish differentiated stability evaluation standards based on the business category characteristics of the model to improve the accuracy and reliability of the evaluation. Since the historical operating data of the bank's intelligent model contains a large amount of non-performance related information, directly using the original data to calculate the performance stability coefficient will introduce noise and affect the accuracy of the evaluation. This application first uses the operating data acquisition unit to filter and extract the operating records of the model's core performance indicators from the historical data, and then uses the performance stability coefficient calculation unit to compare and analyze these performance operating data with the preset performance benchmark threshold, calculate the degree of fluctuation of the performance indicators and the degree of deviation from the benchmark, and thus derive quantitative indicators that reflect the model's true stability level, effectively improving the accuracy and reliability of the stability evaluation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a structural block diagram of a model monitoring device in an embodiment of the present application; Figure 2 It is a structural block diagram of a model stability coefficient acquisition module in an embodiment of the present application; Figure 3 It is a structural block diagram of the performance stability coefficient acquisition submodule in an embodiment of the present application; Figure 4 It is a structural block diagram of a monitoring threshold updating module in an embodiment of the present application; Figure 5 It is a structural block diagram of the monitoring threshold calculation submodule in an embodiment of the present application; Figure 6 is a structural block diagram of a threshold adjustment module in an embodiment of the present application; Figure 7 is a structural block diagram of a fluctuation coefficient acquisition unit in an embodiment of the present application; Figure 8 It is a structural block diagram of the bank digitalization system according to an embodiment of the present application. DETAILED DESCRIPTION

[0024] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to be used as limitations to the present application. As used in the specification and appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to also include plural expressions, unless there is a clear indication to the contrary in the context. It should also be understood that the term "and / or" used in the present application refers to any or all possible combinations comprising one or more listed items.

[0025] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0026] The embodiments of the present application are further described in detail below in conjunction with the drawings in the specification.

[0027] In the first aspect, the present application provides a bank digital intelligence model enabling platform, the bank digital intelligence model enabling platform includes a model monitoring device, referring to Figure 1 , the model monitoring device comprises: The benchmark threshold acquisition module is used to obtain the performance benchmark threshold of each model.

[0028] Among them, the benchmark threshold acquisition module is mainly responsible for setting the initial performance benchmark threshold for different types of models.

[0029] In this embodiment, the models monitored by the bank's digital intelligence model empowerment platform include risk assessment models, customer analysis models, marketing recommendation models, and operational decision models; risk assessment models are used for credit scoring, anti-fraud, and risk warning; customer analysis models are used for customer profiling, churn warning, and value assessment; marketing recommendation models are used for product recommendations and marketing response prediction; operational decision models are used for pricing, quota management, and resource allocation. For risk assessment models, the performance benchmark thresholds mainly include the benchmark values ​​of model accuracy, precision, and recall; for customer analysis models, they mainly include the benchmark values ​​of prediction accuracy and feature importance distribution; for marketing recommendation models, they mainly include the benchmark values ​​of click-through rate, conversion rate, and real-time response time; for operational decision models, they mainly include the benchmark values ​​of decision accuracy and resource utilization.

[0030] Specifically, the process of obtaining the benchmark threshold includes two aspects: setting expert experience values ​​and statistical analysis of historical data. Industry experts set initial benchmark values ​​for different types of models based on business experience. The accuracy benchmark value of risk assessment models is not less than 95%, and the conversion rate benchmark value of marketing recommendation models is not less than 3%. At the same time, combined with the historical operation data of the model, the distribution characteristics of various performance indicators are calculated through statistical analysis methods, and the 90% quantile with the best historical performance is taken as the benchmark value, thus forming a performance benchmark system.

[0031] The model stability coefficient acquisition module is used to obtain the historical operation data of each model, and obtain the model stability coefficient of each model based on the historical operation data.

[0032] In this embodiment, the model stability coefficient acquisition module evaluates the stability level of the model performance by analyzing the historical operation data of the model, reflecting the fluctuation degree and deviation trend of the performance indicators of the model during the historical operation process. The stability evaluation of different types of models has different focuses. The risk assessment model pays more attention to the stability of the accuracy rate, while the marketing recommendation model pays more attention to the stability of the conversion effect.

[0033] Specifically, the calculation process of the stability coefficient mainly includes two steps: data collection and indicator calculation. First, collect the operation data of the model in the past period of time, including the time series data of various performance indicators; then, by calculating the coefficient of variation and trend characteristics of these indicators, comprehensively evaluate the stability level of the model performance. For example, the accuracy of an anti-fraud model has an average level of 96% in the past three months, and the standard deviation is 0.5%, so its stability coefficient can be calculated to be 0.995. It is understandable that the accuracy acquisition methods of different types of models have their own characteristics. For customer analysis models, the accuracy is mainly obtained by comparing the actual behavior of customers with the predicted results. For example, the customer churn warning model will track whether customers predicted to have a high risk of churn actually churn in the next 3-6 months, and the customer value assessment model will compare the actual transaction volume, asset size and other indicators of customers over a period of time with the predicted level. The accuracy of marketing recommendation models is verified by tracking user response behaviors such as clicks and purchases after recommendations, and different conversion windows such as 7 days and 15 days need to be set for statistics. Operational decision-making models are evaluated based on the actual effects of the decision results. For example, pricing models are verified by customer acceptance and actual transaction conditions, credit limit management models are confirmed by customer credit performance and default conditions, and resource allocation models are measured by the matching degree between actual business volume and expected configuration. Data needs to be collected continuously over a sufficiently long period of time, automatically recorded by the daily business system or manually sorted regularly, and finally summarized to form the accuracy time series data of the model. For example, in the anti-fraud model, the sources of accuracy data include real-time feedback (such as transaction interception, user complaints, merchant feedback, manual review) and delayed feedback (such as customer post-reporting, bank reconciliation findings, public security notifications, and inter-bank warnings). Since fraud marking has a lag, the sliding window method is usually used, such as calculating the accuracy 20 days ago, to ensure that sufficient and complete marking data is obtained. When dealing with special situations, confirmed transactions are temporarily not included in the statistics, an observation period is set for suspicious transactions, some frauds are identified according to the actual amount, and repeated complaints are merged and processed, etc.

[0034] The monitoring threshold updating module is used to obtain an updated monitoring threshold according to a model stability coefficient and a performance benchmark threshold.

[0035] In this embodiment, the monitoring threshold is essentially a "warning line" in the model monitoring system, which is used to ensure that the model always operates within a controllable range. The monitoring threshold update module is responsible for dynamically adjusting the performance benchmark threshold according to the model stability coefficient, generating a monitoring threshold that is more in line with the actual operation of the model, taking into account the historical stability performance of the model, and being able to more accurately reflect the normal fluctuation range of the model, avoiding overly strict or overly loose monitoring standards.

[0036] Specifically, the update process of the monitoring threshold adopts a weighted adjustment method. By using the model stability coefficient as a weight factor, the performance benchmark threshold is adjusted accordingly. The more stable the model, the smaller the adjustment of the monitoring threshold; for models with lower stability, the monitoring threshold will be relaxed. For example, for an anti-fraud model with a stability coefficient of 0.995, if its accuracy benchmark threshold is 95%, the updated monitoring threshold may be adjusted to 94.5% to accommodate normal performance fluctuations.

[0037] The threshold adjustment module is used to update the performance benchmark threshold with the updated monitoring threshold, and to monitor the operation of the bank's digital intelligence model empowerment platform based on the updated monitoring threshold.

[0038] In this embodiment, the threshold adjustment module is responsible for applying the updated monitoring threshold to the model operation monitoring process, and performing continuous monitoring and early warning based on the actual operation of the model. By comparing the model operation indicators with the monitoring threshold in real time, model performance anomalies can be discovered in a timely manner to ensure the stability and reliability of the model operation.

[0039] Specifically, the threshold adjustment module adopts a multi-level warning mechanism. When the model performance indicator approaches the monitoring threshold (such as reaching 90% of the threshold), the system issues a yellow warning; when the indicator exceeds the monitoring threshold, the system issues a red warning and automatically notifies the relevant person in charge to conduct investigation and processing.

[0040] Furthermore, the monitoring device of the bank's digital model empowerment platform adopts a layered architecture design, and the modules communicate and exchange data through standardized interfaces. The benchmark threshold acquisition module is located at the bottom layer, providing basic parameter support for the entire monitoring system; the model stability coefficient acquisition module and the monitoring threshold update module are located in the middle layer, responsible for data processing and threshold calculation; the threshold adjustment module is located at the top layer, responsible for monitoring execution and early warning management. The data flow process between the modules is as follows: the benchmark threshold acquisition module passes the initial threshold to the monitoring threshold update module, and the calculation result of the model stability coefficient acquisition module is also input into the monitoring threshold update module. The updated monitoring threshold is then passed to the threshold adjustment module for real-time monitoring.

[0041] In one embodiment, referring to Figure 2 , the model stability coefficient acquisition module includes: The model scenario acquisition submodule is used to obtain the model business category of each model.

[0042] In this embodiment, the model scenario acquisition submodule is mainly responsible for identifying and classifying the specific business category to which the model belongs. The business categories of the bank model mainly include risk assessment (such as credit scoring, anti-fraud warning), customer analysis (such as customer profiling, churn warning), marketing recommendation (such as product recommendation, marketing response) and operational decision-making (such as pricing strategy, credit limit management). Different categories of models have significant differences in stability requirements; the module determines the business category of the model by analyzing the functional positioning, application scenarios and business rules of the model.

[0043] Specifically, the business category identification process includes two steps: model image analysis and functional feature extraction. First, the category is judged based on the basic information of the model, such as the model's input features, output results, and application areas; at the same time, the business rules and data features of the model are combined to perform refined classification.

[0044] The performance stability coefficient acquisition submodule is used to obtain the model performance stability coefficient based on historical operation data.

[0045] In this embodiment, the performance stability coefficient acquisition submodule is responsible for evaluating the stability of the model performance by analyzing the historical operation data of the model; the module comprehensively considers the time series characteristics of the model performance indicators, including fluctuation amplitude, trend changes and periodic characteristics, so as to obtain quantitative indicators reflecting the stability level of the model.

[0046] Specifically, the performance stability coefficient is calculated using a weighted evaluation method. First, each performance indicator is standardized and its volatility and deviation are calculated; then different indicator weights are assigned according to the model business category. For example, for risk assessment models, the accuracy and stability indicators have higher weights, while for marketing recommendation models, the real-time indicators have relatively higher weights; finally, the overall performance stability coefficient is calculated comprehensively.

[0047] The model stability coefficient calculation submodule is used to calculate the model stability coefficient according to the model business category and performance stability coefficient.

[0048] In this embodiment, the model stability coefficient calculation submodule is responsible for combining the business category characteristics of the model and the performance stability evaluation results to calculate the final model stability coefficient; the module establishes differentiated stability evaluation standards for different business categories and adjusts the performance stability coefficient according to the importance of the business category.

[0049] Specifically, the model stability coefficient is calculated using the category weight method. Benchmark weight coefficients are set for different business categories, such as 1.2 for risk assessment models, 1.0 for customer analysis models, and 0.8 for marketing recommendations models, to adjust the performance stability coefficient. For example, the performance stability coefficient of a risk assessment model is 0.95. Considering the high requirements for stability in this category, the weight coefficient is 1.2, and the final model stability coefficient is 1.14, reflecting the stricter stability requirements for core business models.

[0050] In one embodiment, referring to Figure 3 , the performance stability coefficient acquisition submodule includes: The operation data acquisition unit is used to acquire model performance operation data from historical operation data.

[0051] In this embodiment, the operation data acquisition unit is responsible for extracting and filtering effective performance operation data from the model historical operation database; by setting the time window, sampling frequency and data quality standard for data acquisition, it ensures that the acquired operation data is representative and reliable. The performance operation data includes the core performance indicators of the model, system resource usage and data processing efficiency.

[0052] Specifically, the original operation data is first extracted according to the preset time span, then the data is quality checked, outliers and missing values ​​are removed, and finally the data is classified and sorted according to business importance. For example, for a credit scoring model, the system collects the scoring accuracy, feature distribution and response time at 24 time points every day. After data cleaning and outlier processing, the valid samples are retained as input for stability evaluation.

[0053] The performance stability coefficient calculation unit is used to calculate the model performance stability coefficient according to the performance benchmark threshold and performance operation data.

[0054] In this embodiment, the performance stability coefficient calculation unit is responsible for calculating the quantitative index reflecting the stability level of the model based on the performance benchmark threshold and historical operation data; by comparing the degree of deviation between the actual operation index and the benchmark threshold, the fluctuation characteristics and change trends of the model performance are evaluated, thereby obtaining the stability evaluation result.

[0055] Specifically, we first calculate the deviation rate and volatility of each performance indicator relative to the benchmark threshold, then analyze the time series characteristics of these indicators, such as trend, periodicity and mutation, and finally obtain the overall performance stability coefficient through comprehensive calculation. For example, the conversion rate benchmark threshold of a product recommendation model is 5%, and the actual operation data shows that the average daily conversion rate in the past three months fluctuates between 4.5% and 5.5%. By calculating statistics such as standard deviation and coefficient of variation, we can deduce that the performance stability coefficient of the model is 0.92.

[0056] In one embodiment, referring to Figure 4 , the monitoring threshold update module includes: The performance change data acquisition submodule is used to obtain the model operation data within a preset time period and analyze it to obtain the performance change data.

[0057] In this embodiment, the performance change data acquisition submodule is responsible for collecting and analyzing the model's operating performance data within a preset time window; by setting an appropriate observation period (such as nearly 1 month) and data sampling frequency, the performance indicator data of the model at different time points and under different business conditions are collected.

[0058] Specifically, the system records the performance indicators of the model at preset time intervals (such as every hour or every day), and annotates the corresponding business background information, such as business volume and user characteristics. For example, the fraud identification accuracy of an anti-fraud model may be significantly different on weekdays and weekends. The system will record this periodic change pattern and regard it as an important part of the performance change characteristics.

[0059] The scenario impact coefficient acquisition submodule is used to calculate the business scenario impact coefficient based on the performance change data.

[0060] In this embodiment, the scenario impact coefficient acquisition submodule is responsible for evaluating the impact of different business scenarios on model performance; by analyzing the correlation between performance change data and business scenarios, the scenario factors that significantly affect model performance are identified and their impact is quantified.

[0061] Specifically, the collected performance data is first classified according to business characteristics, such as distinguishing between regular transaction scenarios and promotional activity scenarios; then the performance differences under various scenarios are analyzed, and the impact of the scenarios on performance is calculated. For example, the prediction accuracy of a customer churn warning model during the year-end settlement period is 5% lower than usual. The system will quantify this seasonal impact factor as a scenario impact coefficient of 0.95.

[0062] The monitoring threshold calculation submodule is used to calculate the updated monitoring threshold based on the model stability coefficient, the business scenario impact coefficient and the performance benchmark threshold.

[0063] In this embodiment, the monitoring threshold calculation submodule is responsible for comprehensively considering the model stability coefficient, business scenario impact coefficient and benchmark threshold to calculate a more targeted monitoring threshold; by weighted combination of various influencing factors, a dynamic monitoring standard that can reflect the actual operating status of the model is obtained.

[0064] Specifically, by using the model stability coefficient and scenario impact coefficient as adjustment factors, the performance benchmark threshold is dynamically adjusted to obtain a monitoring threshold that is more in line with the actual situation. For example, the conversion rate benchmark threshold of a marketing response prediction model is 3%, the model stability coefficient is 0.98, and the impact coefficient in the e-commerce promotion scenario is 1.2, then the monitoring threshold in this scenario can be calculated to be 3.528%.

[0065] In one embodiment, referring to Figure 5 , the monitoring threshold calculation submodule includes: The model scenario judgment unit is used to determine the business scenario type of the model.

[0066] Among them, business scenario types include: daily transaction scenarios (the operating status of the model during regular business hours), peak business scenarios (such as month-end settlement, market opening peak, etc.), marketing activity scenarios (such as holiday promotions, special marketing activities), and system maintenance scenarios (such as data updates, system upgrades, and other special periods).

[0067] In this embodiment, the model scenario judgment unit is responsible for accurately identifying and classifying the business scenarios of the current model operation; by analyzing the business environment characteristics when the model is running, including changes in transaction volume, business model characteristics and system status, intelligent judgment of the business scenarios is achieved.

[0068] Specifically, the system first collects business environment indicators at the current point in time, such as transaction frequency, transaction scale, user distribution, etc., and then performs multi-dimensional matching with the preset scenario feature library. For example, when it is detected that the transaction volume has increased by 30% compared with the previous average and is concentrated in a specific user group, it is determined to be a marketing activity scenario; when it is observed that it is the last three days of the month and the proportion of corporate business has increased significantly, it is determined to be a peak business scenario.

[0069] The initial threshold calculation unit is used to adjust the performance benchmark threshold according to the model stability coefficient to obtain the initial monitoring threshold.

[0070] In this embodiment, the initial threshold calculation unit is responsible for combining the model stability coefficient with the performance benchmark threshold and performing dynamic adjustment by setting the grading standard.

[0071] Specifically, differentiated adjustment strategies are adopted according to different intervals of the model stability coefficient: when the model stability coefficient is ≥0.9, it indicates that the model performance is highly stable, and the initial monitoring threshold is increased by 10%; when 0.7≤model stability coefficient<0.9, the performance benchmark threshold is kept unchanged; when the model stability coefficient is <0.7, considering the poor model stability, the initial monitoring threshold is lowered by 10% to strengthen monitoring.

[0072] The final threshold updating unit is used to adjust the initial monitoring threshold according to the business scenario impact coefficient to obtain an updated monitoring threshold.

[0073] In this embodiment, the final threshold updating unit establishes a multi-level dynamic adjustment mechanism for different business scenarios.

[0074] For daily trading scenarios, the basic impact coefficient is set to 1.0. The system makes fine adjustments based on the daily trading volume volatility (δ): when δ≤5%, the basic impact coefficient remains unchanged; when 5%<δ≤10%, the impact coefficient is adjusted to 1.0+0.02×δ; when δ>10%, the impact coefficient is adjusted to 1.0+0.03×δ. This adjustment mechanism can reflect the impact of daily trading fluctuations on model performance. In peak business scenarios, the system sets the basic impact coefficient to 1.2, and adjusts the impact coefficient based on the peak trading volume multiple (M), increasing it by 0.1×log(M); at the same time, considering the impact of the peak duration (T hours), increase it by 0.05×(T / 24). To prevent excessive relaxation of monitoring standards, the maximum impact coefficient is set to 1.5. For marketing activity scenarios, the system uses 1.15 as the basic impact coefficient, and increases it by 0.05×log(S) according to the activity scale (S), where S represents the expected transaction increment; and increases it by 0.02×C according to the activity complexity (C), where C is the number of business lines involved. For system maintenance scenarios, the basic impact coefficient is set to 1.3, and is adjusted comprehensively according to the maintenance type (K) and the impact range (R). The maintenance type is divided into three categories: data update (K=0.1), version upgrade (K=0.15), and architecture adjustment (K=0.2). The impact range is also divided into three levels: local impact (R=0.1), partial impact (R=0.15), and global impact (R=0.2). The final impact coefficient is calculated by 1.3+K+R. In the calculation process of the final monitoring threshold, the system first obtains the impact coefficient α of the corresponding scenario, and then introduces the time decay factor β=e^(-t / T), where t is the scenario duration and T is the characteristic time constant. Finally, the final monitoring threshold is calculated by the formula “updated monitoring threshold = initial monitoring threshold × [1 + (α-1) × β]”, which not only takes into account the influence of scene characteristics, but also reflects the dynamic change characteristics of the influence through the time attenuation factor.

[0075] In one embodiment, referring to Figure 6 , the threshold adjustment module also includes: The fluctuation coefficient acquisition unit is used to obtain the model operation fluctuation coefficient.

[0076] In this embodiment, the fluctuation coefficient acquisition unit is responsible for calculating and acquiring the fluctuation degree index of the model operation; by analyzing the changes in the performance indicators of the model on different time scales, including short-term fluctuations, periodic fluctuations and long-term trends, the stability level of the model operation is quantified.

[0077] Specifically, we first collect performance indicator data at different time granularities, and then measure the degree of fluctuation by calculating the standard deviation and coefficient of variation. For example, the accuracy of a certain transaction anti-fraud model shows obvious tidal characteristics during the day, with a fluctuation coefficient of 0.05 in the morning and 0.08 in the afternoon. The system will adjust the monitoring strategy based on this fluctuation characteristic.

[0078] The threshold updating unit is used to update the performance benchmark threshold according to the model operation fluctuation coefficient.

[0079] In this embodiment, the threshold updating unit is responsible for combining the model operation fluctuation coefficient with the performance benchmark threshold to calculate the updated threshold; by establishing a corresponding relationship between fluctuation characteristics and threshold adjustment, it ensures that the monitoring threshold can adapt to the fluctuation characteristics of model performance.

[0080] Specifically, the system relaxes or tightens the benchmark threshold according to the size of the fluctuation coefficient to meet the monitoring needs under different fluctuation levels. For example, the accuracy benchmark threshold of a customer churn warning model is 85%. When the fluctuation coefficient is 0.1, the monitoring threshold can be adjusted to 76.5%, thereby avoiding frequent alarms caused by normal fluctuations.

[0081] The monitoring unit is used to monitor the model operation status according to the updated monitoring threshold.

[0082] In this embodiment, the monitoring unit is responsible for real-time monitoring of the model operation status based on the updated threshold value; by continuously collecting and analyzing the operation data of the model, it is determined whether the model performance is within the normal range.

[0083] Specifically, the system monitors model performance according to different time windows and sets different monitoring frequencies based on business importance. For example, a credit scoring model performs a performance check every 5 minutes during daily trading hours, and may be extended to every 15 minutes during off-peak hours.

[0084] The early warning judgment unit is used to compare the model operation status with the updated monitoring threshold to determine whether an early warning is triggered.

[0085] In this embodiment, the early warning judgment unit is responsible for comparing the real-time model operation status with the updated monitoring threshold value; by setting a series of early warning rules and judgment conditions, abnormal situations that require attention are identified.

[0086] The warning sending unit is used to send warning information when the warning condition is triggered.

[0087] Among them, the warning sending unit is responsible for transmitting the warning information to the relevant responsible persons when the warning conditions are triggered; by configuring different levels of warning methods and notification objects, it ensures that the warning information can be delivered in a timely and accurate manner.

[0088] In one embodiment, referring to Figure 7 , the fluctuation coefficient acquisition unit includes: The time window division subunit is used to divide the model operation data according to the preset time window.

[0089] In this embodiment, the time window division subunit divides the continuous operation data into multiple ordered time segments by setting a reasonable time window size and sliding step size.

[0090] Specifically, the system simultaneously maintains multiple time windows of different sizes, such as 5 minutes, 1 hour, and 1 day, and updates data through window sliding. For example, the operating data of an online trading risk control model can be set as a 15-minute basic window, sliding every 5 minutes, while maintaining a 1-hour medium window and a 6-hour large window, forming a hierarchical time series data structure.

[0091] The performance indicator calculation subunit is used to calculate the mean and variance of the model performance indicator in each time window.

[0092] In this embodiment, the performance indicator calculation subunit is responsible for performing statistical analysis on the performance data in each time window.

[0093] Specifically, the calculation of performance indicators adopts the sliding statistical method. For each time window, the system first performs data preprocessing, removes obviously abnormal data points, and then calculates the statistics within the window. For example, a user behavior prediction model calculates the average and standard deviation of the accuracy in each hour window. If the accuracy of a certain time point is found to be abnormally deviated (more than 3 standard deviations), it will be removed when calculating the statistics to ensure the reliability of the calculation results.

[0094] The volatility coefficient calculation subunit is used to calculate the model operation volatility coefficient based on the mean and variance of the performance indicators.

[0095] In this embodiment, the fluctuation coefficient calculation subunit is responsible for quantifying the fluctuation degree of the model operation based on the statistical characteristics of the performance indicators; by comprehensively considering the changes in the mean and variance, an evaluation model for the fluctuation degree is established to obtain the fluctuation coefficient reflecting the stability of the model.

[0096] Specifically, the final coefficient of fluctuation is obtained by taking the ratio of standard deviation to mean as the basis and making appropriate adjustments based on business characteristics. For example, the average accuracy of a credit scoring model in a 4-hour window is 92%, and the standard deviation is 2.76%. Its coefficient of variation is 0.03, and considering the business importance weight of 1.2, the final coefficient of fluctuation is 0.036. The system will also analyze the fluctuation trend of model performance by comparing the coefficients of fluctuation in different time windows.

[0097] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0098] On the second aspect, the present application provides a banking digital intelligence system. The banking digital intelligence system of the present application is described below in combination with the above-mentioned banking digital intelligence model empowerment platform.

[0099] Reference Figure 8 , a bank digital intelligence system, including the aforementioned bank digital intelligence model empowerment platform, business system, data warehouse platform and application platform; Business system, used to provide basic business data and transaction processing.

[0100] In this embodiment, the business system, as a carrier platform for the core business of the bank, is responsible for processing daily transaction business and providing basic data support; the system adopts a distributed architecture design, and realizes the flexible combination and expansion of various business functions in a modular way, ensuring that the system has high availability and scalability. The system supports parallel processing of multiple business scenarios, and can automatically adjust system resource configuration according to changes in business volume to ensure the timeliness and stability of business processing.

[0101] Specifically, the system defines the business processing flow in a configurable manner and supports flexible adjustment of business rules and processing logic. For example, in the transfer business processing, the system will automatically select the corresponding processing flow and risk control rules based on factors such as transaction amount, user level, transaction time, etc. For large transfers, the system will trigger multiple review mechanisms and require higher levels of authorization and approval; while for small and high-frequency payment businesses, fast channel processing is used to improve transaction efficiency.

[0102] Data warehouse platform, used for unified management and processing of various types of data.

[0103] In this embodiment, the data warehouse platform adopts a multi-level data architecture to achieve unified management of structured and unstructured data; the platform ensures data consistency and availability by establishing unified data standards and processing specifications. It meets data requirements in different scenarios by combining real-time data collection and batch data processing, and ensures data accuracy and integrity through a data quality management system.

[0104] Specifically, the platform has designed a complete data processing process, including data collection, cleaning, conversion and loading. For example, when processing credit card transaction data, the system first cleans the original transaction data to remove duplicate and abnormal data; then performs format conversion and standardization according to unified data specifications; finally, the processed data is classified and stored according to the predetermined subject domain.

[0105] Application platform, used to host application development and deployment.

[0106] In this embodiment, the application platform provides a unified application development and operating environment, supports agile development and rapid deployment of various types of applications; the platform uses containerization technology to achieve standardized packaging and unified management of applications, and provides complete application lifecycle management capabilities.

[0107] Specifically, the platform provides developers with standardized development tools and component libraries to support rapid application construction and testing. For example, when developing personal marketing applications, developers can directly call the user portrait components and marketing rule engines provided by the platform to quickly implement precision marketing functions; during the application deployment phase, the platform automatically completes environment configuration and resource allocation, and provides features such as grayscale release to ensure smooth application launch. The platform also provides real-time monitoring and operation and maintenance tools to help operation and maintenance personnel discover and solve problems in a timely manner.

[0108] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A bank digital intelligence model empowerment platform, characterized by: The bank digital intelligence model empowerment platform includes a model monitoring device, and the model monitoring device includes: A benchmark threshold acquisition module is used to obtain the performance benchmark threshold of each model; A model stability coefficient acquisition module is used to acquire the historical operation data of each model, and acquire the model stability coefficient of each model according to the historical operation data; A monitoring threshold updating module, used for obtaining an updated monitoring threshold according to the model stability coefficient and the performance benchmark threshold; The threshold adjustment module is used to update the performance benchmark threshold with the updated monitoring threshold, and to monitor the operation of the bank's digital intelligence model empowerment platform according to the updated monitoring threshold.

2. The bank digital intelligence model enabling platform according to claim 1 is characterized in that: The model stability coefficient acquisition module includes: The model scenario acquisition submodule is used to obtain the model business category of each model; A performance stability coefficient acquisition submodule is used to obtain a model performance stability coefficient based on the historical operation data; The model stability coefficient calculation submodule is used to calculate the model stability coefficient according to the model business category and the performance stability coefficient.

3. The bank digital intelligence model enabling platform according to claim 2 is characterized in that: The performance stability coefficient acquisition submodule includes: An operation data acquisition unit, used to acquire model performance operation data from the historical operation data; The performance stability coefficient calculation unit is used to calculate the model performance stability coefficient according to the performance benchmark threshold and the performance operation data.

4. The bank digital intelligence model enabling platform according to claim 1 is characterized in that: The monitoring threshold updating module comprises: The performance change data acquisition submodule is used to acquire the model operation data within a preset time period and analyze and obtain the performance change data; A scenario impact coefficient acquisition submodule, used to calculate a business scenario impact coefficient based on the performance change data; The monitoring threshold calculation submodule is used to calculate the updated monitoring threshold according to the model stability coefficient, the business scenario impact coefficient and the performance benchmark threshold.

5. The bank digital intelligence model enabling platform according to claim 4 is characterized in that: The monitoring threshold calculation submodule includes: A model scenario judgment unit, used to determine the business scenario type of the model, wherein the business scenario type includes daily transaction scenarios, peak business scenarios, marketing activity scenarios, and system maintenance scenarios; An initial threshold calculation unit, used to adjust the performance benchmark threshold according to the model stability coefficient to obtain an initial monitoring threshold; The final threshold updating unit is used to adjust the initial monitoring threshold according to the business scenario impact coefficient to obtain an updated monitoring threshold.

6. The bank digital intelligence model empowerment platform according to claim 1 is characterized in that: The threshold adjustment module also includes: A fluctuation coefficient acquisition unit is used to obtain the model operation fluctuation coefficient; A threshold updating unit, configured to update the performance benchmark threshold according to the model operation fluctuation coefficient; A monitoring unit, configured to monitor the model operation status according to the updated monitoring threshold; An early warning judgment unit is used to compare the model operation status with the updated monitoring threshold to determine whether an early warning is triggered; The warning sending unit is used to send warning information when the warning condition is triggered.

7. The bank digital intelligence model enabling platform according to claim 6 is characterized in that: The fluctuation coefficient obtaining unit comprises: A time window division subunit is used to divide the model operation data according to a preset time window; The performance index calculation subunit is used to calculate the mean and variance of the model performance index in each time window; The fluctuation coefficient calculation subunit is used to calculate the model operation fluctuation coefficient according to the mean and variance of the performance indicator.

8. The bank digital intelligence model enabling platform according to claim 1 is characterized in that: The models monitored by the bank’s digital intelligence model empowerment platform include: Risk assessment models, used for credit scoring, anti-fraud and risk warning; Customer analysis models, used for customer profiling, churn warning, and value assessment; Marketing recommendation models, used for product recommendations and marketing response prediction; Operational decision-making models are used for pricing, quota management, and resource allocation.

9. A digital banking system, characterized in that: include: The bank digital intelligence model enabling platform as described in any one of claims 1 to 8; Business systems, used to provide basic business data and transaction processing; Data warehouse platform, used for unified management and processing of various types of data; Application platform, used to host application development and deployment.

Images