Financial risk control model management method and related equipment

Abstract

The invention discloses a financial risk control model management method and device, a server and a computer readable storage medium. According to the method, starting from the whole model, the weight distribution condition of each feature and each sub-box in the model is researched, so that the role played by the feature in the model can be determined, the feature processing mode of the model is understood, and the interpretability of the model is improved; besides, the contribution value of each feature to the prediction of the single sample is determined, the contribution value reflects the direct influence of the feature on the risk prediction of the sample, and the contribution value of each feature to the prediction of the single sample is calculated, so that the transparency and interpretability of the model can be improved, and powerful support can be provided in multiple aspects of model debugging, optimization, business decision and the like. According to the method, the feature level and the binning level are subjected to refined monitoring, and the possible problems of the model can be pre-judged in time before the model shows the problems, so that the recognition reliability and stability of the model in actual business are improved.

Description

Technical Field

[0001] The present disclosure generally relates to the financial field, and specifically to a management method, device, server and computer-readable storage medium for a financial risk control model. Background Art

[0002] With the widespread application of big data and artificial intelligence, models based on machine learning have become an important tool for financial risk prediction and decision-making. However, although traditional black box models (such as deep neural networks and integrated models) have strong predictive capabilities, they lack transparency and it is difficult to understand how the models make decisions. When applied to the field of financial risk control, model verification and compliance review are required, which undoubtedly brings severe challenges. In addition, the current monitoring of models is mostly focused on overall performance indicators, lacking refined management, and it is difficult to discover and correct problems in certain details in a timely manner, which may lead to a decline in the overall performance of the model and affect the reliability of financial decision-making. Summary of the invention

[0003] In view of the above-mentioned defects or deficiencies in the prior art, it is desired to provide a management method, device, server and computer-readable storage medium for a financial risk control model, which can improve the interpretability of the model and realize refined monitoring and management of the model.

[0004] In a first aspect, an embodiment of the present application provides a method for managing a financial risk control model, including:

[0005] Obtaining an input sample of a financial risk control model and determining all features in the input sample;

[0006] Determine the weight distribution of each of the features and each of their sub-boxes in the financial risk control model;

[0007] Calculate the contribution value of each feature to sample prediction;

[0008] Model monitoring is performed according to the weight distribution and the contribution value.

[0009] In one embodiment, determining the weight distribution of each of the features and each of their sub-boxes in the financial risk control model includes:

[0010] Determine the weight corresponding to each of the features;

[0011] Calculate the mean and variance of each weight as the distribution of feature weights;

[0012] The weighted sum of each weight and the binning interval length is calculated as the influence intensity of the binning weight on the corresponding binning length.

[0013] In one embodiment, calculating the contribution value of each feature to sample prediction includes:

[0014] Determine the weight value of each feature in each bin and the positive and negative weight;

[0015] The weight value is used as the contribution value, and the positive or negative value of the weight is used as the positive or negative value of the contribution.

[0016] In one embodiment, it also includes:

[0017] The sum of the contribution values ​​of the top N digits in the positive and negative contributions is calculated separately.

[0018] In one embodiment, it also includes:

[0019] Count the distribution ratios of the features in each bin;

[0020] Calculate the matching degree between the bin weight and data distribution;

[0021] The binning stability is monitored according to the distribution ratio and the matching degree.

[0022] In one embodiment, it also includes:

[0023] Calculate the weight change range and sample change of each adjacent bin;

[0024] The reliability of binning prediction is monitored based on the weight change amplitude and the sample change amount.

[0025] In one embodiment, it also includes:

[0026] Determining the structure of the decision tree in the financial risk control model;

[0027] Convert each node in the decision tree into a linear neuron according to the structure;

[0028] Combining the outputs of the nodes to construct a segmented MLP model;

[0029] Get the vectors output by all the nodes as the evaluation probability.

[0030] In a second aspect, an embodiment of the present application provides a management device for a financial risk control model, including:

[0031] A feature acquisition unit, used to acquire an input sample of a financial risk control model and determine all features in the input sample;

[0032] A feature bin weight determination unit, used to determine the weight distribution of each feature and each bin thereof in the financial risk control model;

[0033] A prediction contribution determination unit, used to calculate the contribution value of each of the features to the sample prediction;

[0034] A model monitoring unit is used to perform model monitoring according to the weight distribution and the contribution value.

[0035] In a third aspect, an embodiment of the present application provides a server, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the embodiment of the present application when executing the program.

[0036] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the embodiment of the present application.

[0037] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0039] Figure 1 A schematic diagram showing a flow chart of a method for managing a financial risk control model provided in an embodiment of the present application is shown;

[0040] Figure 2 A schematic diagram of the model structure of the evaluation system provided in an embodiment of the present application is shown;

[0041] Figure 3 A structural block diagram of a management device for a financial risk control model provided in an embodiment of the present application is shown;

[0042] Figure 4 A schematic diagram of the structure of a computer system suitable for implementing a server of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] The present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant invention, rather than to limit the invention. It is also necessary to explain that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0044] It should be noted that, in the absence of conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments. Although the embodiments of the present application provide the method operation instruction steps shown in the following embodiments or drawings, more or fewer operation instruction steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided by the embodiments of the present application. The method may be executed in the order of the methods shown in the embodiments or drawings or in parallel during the actual processing process or when the device is executed.

[0045] Please refer to Figure 1 , Figure 1 The flowchart of the management method of the financial risk control model provided by an embodiment of the present application is shown. Figure 1 As shown, the method includes:

[0046] S101. Obtain input samples of the financial risk control model and determine all features in the input samples;

[0047] Among them, the financial risk control model refers to the target financial risk control model to be managed, which is used to conduct risk assessment on input samples and output predicted risk values. There is no limitation on the specific model structure and type.

[0048] S102, determining the weight distribution of each feature and each sub-box in the financial risk control model;

[0049] Among them, binning refers to several intervals of feature division, which can refer to the generation method of related technologies.

[0050] This step starts from the model as a whole, studies the weight distribution of each feature and its various bins in the model, and analyzes the role of the feature in the model. Through these analyses, we can understand the feature processing method of the model, thereby helping to improve the model design, interpretability and stability.

[0051] The specific steps for implementing the analysis of the weight distribution are not limited. In this embodiment, a specific analysis indicator is proposed, which can be specifically performed according to the following steps:

[0052] Step S21, determining the weight corresponding to each feature;

[0053] Step S22: Calculate the mean and variance of each weight as the distribution of feature weights;

[0054] The mean and variance of weights can be used to measure the overall level and volatility of feature weights, reflecting the distribution of weights among all features and helping to identify the stability of key features in the model. A calculation formula for the mean and variance of weights is as follows:

[0055]

[0056] Among them, w i represents the weight of the i-th feature, μ w represents the mean of the weights, represents the variance of the weight.

[0057] Step S23: Calculate the weighted sum of each weight and the binning interval length as the influence intensity of the binning weight on the corresponding binning length.

[0058] The weighted sum of the weight and the bin length can be used to evaluate the influence of each bin weight on its bin length, revealing the sensitivity of the model to different data intervals. A calculation formula for the weighted sum of the weight and bin length is as follows:

[0059]

[0060] Among them, w i represents the weight of the i-th feature, l i represents the length of the i-th bin, S w It represents the weighted sum of the weight and the binning interval length, which can be used to analyze the predictive impact of the model in combination with the binning interval length.

[0061] S103, calculating the contribution value of each feature to sample prediction;

[0062] When a specific sample is given, analyze how the input value of each feature falls into the corresponding bin, and understand the impact of its corresponding weight on the prediction of the sample. This part focuses on the interpretability of the feature, that is, the contribution of each feature to the prediction of a single sample. The contribution value reflects the direct impact of the feature on the risk prediction of the sample.

[0063] The specific method for calculating the contribution value is not limited in this embodiment, and a specific implementation step is proposed here, as follows:

[0064] S31, determining the weight value of each feature in each bin and the positive and negative weight;

[0065] The contribution of each feature in the sample is directly related to the weight of the corresponding bin. The contribution value reflects the direct impact of the feature on the risk prediction of the sample. Specifically, a calculation formula for the feature contribution value is as follows:

[0066] Contribution(X i )=w i,j

[0067] Among them, X i represents the i-th eigenvalue, w i,jRepresents feature X i The weight of the j-th bin.

[0068] S32. The weight value is used as the contribution value, and the positive or negative value of the weight is used as the positive or negative value of the contribution.

[0069] By analyzing the positive and negative values ​​of the weights, we can evaluate whether each feature increases or decreases the risk of the sample. The comparison of positive and negative contributions can reveal the directionality of the feature in risk prediction. Specifically, a formula for calculating positive and negative contributions is as follows:

[0070]

[0071] Among them, X i represents the i-th eigenvalue, w i,j Represents feature X i The weight of the j-th bin.

[0072] The above steps deeply analyze how the input of each feature in a single sample affects the risk prediction result through the weight of its corresponding bin, and determine the specific directionality of each feature to risk prediction through positive and negative contribution analysis.

[0073] Furthermore, in addition to the above steps, the sum of the contribution values ​​of the first N bits of the positive and negative contributions may be counted respectively.

[0074] For the top N (e.g. 20) features in terms of feature contribution, calculate the sum of their positive and negative contributions respectively, and understand the main risk direction contribution of the feature in a single sample, which helps to explain the source of risk prediction for a single sample. A calculation formula for separately counting positive and negative contribution values ​​is as follows:

[0075]

[0076] Among them, X i represents the i-th eigenvalue.

[0077] S104. Monitor the model according to the weight distribution and contribution value.

[0078] After obtaining the weight distribution and contribution value, these indicators are used to evaluate and monitor the performance of the model. Specifically, how to adjust and optimize the model or input data is not limited in this embodiment and can be set according to actual application requirements, which will not be repeated here.

[0079] It should be noted that although the operations of the method of the present invention are described in a particular order in the drawings, this does not require or imply that the operations must be performed in this particular order or that all illustrated operations must be performed to achieve desired results.

[0080] Based on the above introduction, the method provided in this embodiment starts from the overall model and studies the weight distribution of each feature and its various bins in the model, so as to determine the role of the feature in the model, understand the feature processing method of the model, improve the interpretability of the model, and ensure the long-term stability of the model; in addition, determine the contribution value of each feature to the prediction of a single sample, which reflects the direct impact of the feature on the risk prediction of the sample. Calculating the contribution value of each feature to the prediction of a single sample can not only improve the transparency and interpretability of the model, but also provide strong support in model debugging, optimization, business decision-making and other aspects. This method can timely predict the problems that may occur in the model before the model shows problems by fine-tuning the feature level and the bin level, thereby improving the recognition reliability and stability of the model in actual business.

[0081] In the actual application of the model, the data distribution may change over time, and the existing models are often unable to adapt to these changes in a timely manner, resulting in unstable performance of the model in actual applications. Especially in the financial field, this problem of insufficient stability is particularly serious, which may lead to misjudgment and cause the gradual accumulation of risks, seriously affecting the accuracy of financial decisions. Moreover, in the monitoring of the model, when the sample size is small or the distribution changes drastically, the model's prediction results may produce significant deviations, affecting the reliability of financial decisions.

[0082] To solve this problem, a monitoring method for model stability is proposed in this embodiment, comprising the following steps:

[0083] Step S105, counting the distribution ratio of each feature in each bin;

[0084] In data distribution, the stability of bins is determined by determining the distribution ratio of each feature in each bin, such as whether the distribution of feature values ​​in each bin is stable and whether these distributions correspond to consistent prediction results. Specifically, a calculation formula for the distribution ratio is as follows:

[0085] Stability(X i )=∑PSI(E i ,X i )

[0086] Among them, X i represents the i-th eigenvalue, E i Represents the expected value corresponding to the i-th feature.

[0087] Step S106, calculating the matching degree between the bin weight and the data distribution;

[0088] Calculate the degree of match between the weight of the feature bins and the data distribution. If the weight of a bin is too high or too low and the number of samples is large, it may affect the prediction stability of the model. Specifically, one way to calculate the match is as follows:

[0089]

[0090] Among them, X i represents the i-th eigenvalue, E i represents the expected value corresponding to the i-th feature, w i,j Represents feature X i The weight of the jth bin. PSI (Population Stability Index) is a statistical method used to measure the difference between two data distributions.

[0091] Step S107: monitor the stability of binning according to the distribution ratio and matching degree.

[0092] Through in-depth analysis of the synergy between feature weights and data distribution, we can ensure that the model can maintain the consistency and reliability of predictions when the data distribution changes.

[0093] Furthermore, in addition to the above steps, the weight change range and sample change amount of each adjacent bin can be calculated; and the bin prediction reliability can be monitored based on the weight change range and sample change amount.

[0094] Inspect the changes in weights and sample numbers of adjacent bins, and evaluate prediction bias through changes in bin weights and sample numbers. If a bin has a large weight but a small number of samples, the prediction result of the bin may not be reliable, which helps to discover potential model bias and instability, ensure the robustness and consistency of the model under changes in data distribution, and thus improve the performance of the model in a dynamic data environment.

[0095] Specifically, a method for calculating the change range of bin weights is as follows:

[0096]

[0097] This formula can be used to measure the weight changes of adjacent bins. A large variability indicates that the bin weights of the feature vary greatly over the interval. Where B is the total number of bins, X i represents the i-th eigenvalue, w i,j Represents feature X i The weight of the jth bin, w i,j+1 Represents feature X i The weight of the j+1th bin.

[0098] Specifically, a method for calculating the sample change is as follows:

[0099]

[0100] Among them, B is the total number of bins, X i represents the ith eigenvalue, and j represents the number of bins. By measuring the change in the number of samples in adjacent bins, if the weight variability is large but the sample number variability is small, it means that the model prediction is not stable enough.

[0101] The method provided in this embodiment can help discover potential model bias and instability by analyzing the changes in weights and sample numbers of adjacent bins, thereby improving the performance of the model in a dynamic data environment.

[0102] In one embodiment, the financial risk control model can be modified to significantly improve the applicability of the model in the risk control field.

[0103] Specifically, the steps include:

[0104] Step S11, determining the structure of the decision tree in the financial risk control model;

[0105] Step S12: convert each node in the decision tree into a linear neuron according to the structure;

[0106] Step S13, combining the outputs of each node to construct a segmented MLP model;

[0107] Step S14: Obtain the vectors output by all nodes as evaluation probabilities.

[0108] The concept of treating the decision tree structure equivalently as a segmented multi-layer perceptron (MLP), such as Figure 2 The figure shows a conceptual diagram of the equivalent of segmented MLP in decision making provided by this embodiment. This method breaks through the boundary between traditional decision trees and neural network models. By treating each node of the decision tree as a linear neuron and combining the segmented processing mechanism of the tree structure, the neural network expression of the decision tree model is realized. This equivalent processing not only retains the interpretability and visualization advantages of the decision tree model, but also introduces the flexibility of neural networks and the powerful expressiveness of deep learning, providing more sophisticated and dynamic risk prediction capabilities for complex financial risk control scenarios.

[0109] This embodiment proposes a new model evaluation and monitoring framework based on the decision tree segmented neuron hypothesis. By introducing the characteristics of decision trees to transform neural networks, the applicability of the model in the risk control field can be significantly improved; by designing a more concise, sparse and easy-to-interpret network structure, it can not only meet the high requirements of risk control for model transparency and interpretability, but also explain the risk prediction process of the model for specific samples, and improve computing efficiency and reduce the complexity of training and maintenance. This modified network can better adapt to the dual needs of efficiency and accuracy in the field of financial risk control.

[0110] This embodiment provides a management device for a financial risk control model. Please refer to Figure 3 , Figure 3 The figure shows a structural block diagram of a management device for a financial risk control model, which mainly includes:

[0111] The feature acquisition unit 101 is used to acquire an input sample of the financial risk control model and determine all features in the input sample;

[0112] A feature bin weight determination unit 102, used to determine the weight distribution of each feature and each bin in the financial risk control model;

[0113] A prediction contribution determination unit 103, used to calculate the contribution value of each feature to the sample prediction;

[0114] The model monitoring unit 104 is used to perform model monitoring according to the weight distribution and the contribution value.

[0115] It should be understood that the units described in the above device are similar to those in the reference Figure 1 The steps in the method described above correspond to each other. Therefore, the operations and features described above for the method are also applicable to the device and the units contained therein, and will not be repeated here. The device can be pre-implemented in the browser or other security application of the server, or loaded into the browser or its security application of the server by downloading or the like. The corresponding units in the device can cooperate with the units in the server to implement the solution of the embodiment of the present application.

[0116] For the several units mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units described above can be embodied in one unit. On the contrary, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0117] It should be noted that for details not disclosed in the management device of the financial risk control model in the embodiment of the present application, please refer to the details disclosed in the above embodiments of the present application, which will not be repeated here.

[0118] Reference below Figure 4 , Figure 4 A schematic diagram of the structure of a computer system suitable for implementing a server of an embodiment of the present application is shown.

[0119] like Figure 4 As shown, the computer system includes a central processing unit (CPU) 401, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 402 or the program loaded from the storage part 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation instructions of the system are also stored. The CPU 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0120] The following components are connected to the I / O interface 405: an input section 406 including a keyboard, a mouse, etc.; an output section 407 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 408 including a hard disk, etc.; and a communication section 409 including a network interface card such as a LAN card, a modem, etc. The communication section 409 performs communication processing via a network such as the Internet. A drive 410 is also connected to the I / O interface 405 as needed. A removable medium 411, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 410 as needed, so that a computer program read therefrom is installed into the storage section 408 as needed.

[0121] It should be noted that the computer-readable medium shown in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium such as a computer-readable storage medium that can send, propagate or transmit a program for use by or in conjunction with an instruction execution system, device or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0122] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation instruction of the system and method according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the aforementioned module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs a specified function or operation instruction, or can be implemented by a combination of dedicated hardware and computer instructions.

[0123] The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not, in some cases, constitute limitations on the units or modules themselves.

[0124] As another aspect, the present application further provides a computer-readable storage medium, which may be included in the server described in the above embodiment, or may exist independently without being assembled into the server. The above computer-readable storage medium stores one or more programs, and when the above programs are used by one or more processors to execute the management method of the financial risk control model described in the present application.

[0125] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the aforementioned disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in this application (but not limited to) by each other to form a technical solution.

Claims

1. A management method for a financial risk control model, characterized in that: include: Obtaining an input sample of a financial risk control model and determining all features in the input sample; Determine the weight distribution of each of the features and each of their sub-boxes in the financial risk control model; Calculate the contribution value of each feature to sample prediction; Model monitoring is performed according to the weight distribution and the contribution value.

2. The method according to claim 1, characterized in that Determining the weight distribution of each of the features and each of their sub-boxes in the financial risk control model includes: Determine the weight corresponding to each of the features; Calculate the mean and variance of each weight as the distribution of feature weights; The weighted sum of each weight and the binning interval length is calculated as the influence intensity of the binning weight on the corresponding binning length.

3. The method according to claim 1, characterized in that Calculate the contribution value of each feature to sample prediction, including: Determine the weight value of each feature in each bin and the positive and negative weight; The weight value is used as the contribution value, and the positive or negative value of the weight is used as the positive or negative value of the contribution.

4. The method according to claim 3, characterized in that Also includes: The sum of the contribution values ​​of the top N digits in the positive and negative contributions is calculated separately.

5. The method according to claim 1, characterized in that Also includes: Count the distribution ratios of the features in each bin; Calculate the matching degree between the bin weight and data distribution; The binning stability is monitored according to the distribution ratio and the matching degree.

6. The method according to claim 5, characterized in that Also includes: Calculate the weight change range and sample change of each adjacent bin; The reliability of binning prediction is monitored based on the weight change amplitude and the sample change amount.

7. The method according to any one of claims 1 to 6, characterized in that: Also includes: Determining the structure of the decision tree in the financial risk control model; Convert each node in the decision tree into a linear neuron according to the structure; Combining the outputs of the nodes to construct a segmented MLP model; Get the vectors output by all the nodes as the evaluation probability.

8. A management device for a financial risk control model, characterized in that: include: A feature acquisition unit, used to acquire an input sample of a financial risk control model and determine all features in the input sample; A feature bin weight determination unit, used to determine the weight distribution of each feature and each bin thereof in the financial risk control model; A prediction contribution determination unit, used to calculate the contribution value of each of the features to the sample prediction; A model monitoring unit is used to perform model monitoring according to the weight distribution and the contribution value.

9. A server comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images