Monitoring method and device for credit scoring model

Abstract

The present invention discloses a monitoring method and device for a credit scoring model, wherein the method comprises: obtaining training data, and inputting the training data into a current credit scoring model to output predicted credit information, wherein the training data comprises: actual credit information; performing a model performance test on the current credit scoring model according to the predicted credit information and the actual credit information; in response to the model performance test result being model failure, replacing the current credit scoring model with a trained spare model as a new current credit scoring model, wherein the spare model is iteratively trained based on a predetermined basic model and an updateable data sample during the working process of the current credit scoring model. Through the present invention, it is possible to be ready to replace an online failed model at any time, thereby ensuring the normal operation of the business.

Description

Technical Field

[0001] The present invention relates to the field of information processing, and in particular to a monitoring method and device for a credit scoring model. Background Art

[0002] The consumer finance credit scoring model refers to the process of assessing the credit risk of customers by integrating relevant customer data (including basic personal information data, relationship data, and credit data) and using machine learning technology to establish a corresponding credit scoring model during the application approval or early warning collection process of a consumer finance company's customers. Consumer finance business has the characteristics of a large number of applications, rapid changes in customer characteristics, and fast response requirements, so comprehensive and effective automatic assessment of consumer finance customers is a very important part of this business.

[0003] Due to the rapid changes in the characteristics of consumer finance customer groups, models are prone to failure and cannot effectively evaluate consumer finance customers. Traditional modeling methods have a long cycle, and rebuilding the model after discovering that the model has failed often results in a long window period of model failure. In other words, the online model has failed, but the new model has not yet been built. This situation will seriously affect the business progress. Summary of the invention

[0004] In view of this, the present invention provides a method and device for monitoring a credit scoring model to solve at least one of the above-mentioned problems.

[0005] According to a first aspect of the present invention, a method for monitoring a credit scoring model is provided, the method comprising:

[0006] Acquiring training data, and inputting the training data into a current credit scoring model to output predicted credit information, wherein the training data includes: actual credit information;

[0007] Performing a model performance test on the current credit scoring model based on the predicted credit information and the actual credit information;

[0008] In response to the model performance test result being model failure, the current credit scoring model is replaced by a trained backup model as a new current credit scoring model. The backup model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model.

[0009] According to a second aspect of the present invention, there is provided a monitoring device for a credit scoring model, the device comprising:

[0010] A data acquisition unit, used to acquire training data, wherein the training data includes: actual credit information;

[0011] A data input unit, used to input the training data into the current credit scoring model to output predicted credit information;

[0012] A performance testing unit, used to perform a model performance test on the current credit scoring model based on the predicted credit information and the actual credit information;

[0013] A replacement unit is used to replace the current credit scoring model with a trained backup model as a new current credit scoring model in response to a model performance test result indicating model failure, wherein the backup model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model.

[0014] According to a third aspect of the present invention, there is provided an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0015] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0016] It can be seen from the above technical solution that by training the backup model while the current credit scoring model is working, and when the performance of the current credit scoring model is tested to be invalid based on the predicted credit information and actual credit information of the current credit scoring model, the trained backup model will replace the current credit scoring model as the new current credit scoring model, thereby overcoming the defect in the prior art that the online model fails and an effective new model cannot be used quickly. Through this technical solution, the online invalid model can be prepared to be replaced at any time, thereby ensuring the normal operation of the business. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is a flow chart of a monitoring method of a credit scoring model according to an embodiment of the present invention;

[0019] Figure 2 is an example flow chart of a credit scoring model monitoring method according to an embodiment of the present invention;

[0020] Figure 3 is a logical schematic diagram of a model feature library according to an embodiment of the present invention;

[0021] Figure 4 is a flowchart of automatically training a basic model according to an embodiment of the present invention;

[0022] Figure 5 is a flowchart of automatic iteration of a model according to an embodiment of the present invention;

[0023] Figure 6 is a structural block diagram of a credit scoring model monitoring device according to an embodiment of the present invention;

[0024] Figure 7 is a detailed structural block diagram of a credit scoring model monitoring device according to an embodiment of the present invention;

[0025] Figure 8 It is a schematic block diagram of the system structure of the electronic device 600 according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] The current credit scoring model update strategy is mainly to actively monitor online model indicators. When the model indicators exceed the preset threshold or the features used by the model fail, a new credit scoring model will be constructed to update the current credit scoring model. This method can essentially be summarized as monitoring the model indicators or the features used by the model, promptly discovering model failures, and then notifying the system to update. This update strategy can promptly discover model failures, but it cannot guarantee that a new effective model can be quickly used to replace the model after it fails. Based on this, an embodiment of the present invention provides a monitoring solution for a credit scoring model, which can quickly update the model according to changes in customer data. Once the online model fails, the failed model can be quickly replaced with a valid new model, which can overcome the defect of the online model failure in the prior art and the inability to quickly use a valid new model. The following is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

[0028] Figure 1 is a flow chart of a method for monitoring a credit scoring model according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0029] Step 101, obtaining training data, and inputting the training data into the current credit scoring model to output predicted credit information, wherein the training data includes: actual credit information;

[0030] Step 102, performing a model performance test on the current credit scoring model based on the predicted credit information and the actual credit information;

[0031] Step 103, in response to the model performance test result being model failure, the current credit scoring model is replaced by a trained backup model as a new current credit scoring model, wherein the backup model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model.

[0032] By training a backup model while the current credit scoring model is working, and when the performance of the current credit scoring model is tested to be invalid based on the predicted credit information and actual credit information of the current credit scoring model, the trained backup model replaces the current credit scoring model as the new current credit scoring model, thereby overcoming the defect in the prior art that the online model fails and an effective new model cannot be used quickly. Through the embodiments of the present invention, the online invalid model can be prepared to be replaced at any time, thereby ensuring the normal operation of the business.

[0033] In actual operation, the backup model can be trained in the following manner: select a corresponding basic model and a pre-set optimal feature data set according to the type of the current credit scoring model; train the basic model based on the selected optimal feature data set and based on the XGB (Extreme Gradient Boosting) algorithm and a predetermined objective function to obtain the backup model.

[0034] In one embodiment, based on different application scenarios, multiple basic models and optimal feature data sets corresponding to each basic model may be pre-set. The type of the current credit scoring model here may be a specific application scenario.

[0035] During the training process, the data samples for training the backup model are updateable. Specifically, new feature data is selected from the candidate feature data according to the feature screening algorithm; the new feature data is then input into the basic model for training to determine whether the performance of the basic model is improved; in response to the judgment result that the performance of the basic model is improved, the new feature data is added to the optimal feature data set, otherwise, the new feature data is removed.

[0036] In one embodiment, for the spare feature data, a feature quality assessment operation may be performed on the candidate feature data to sort the feature data in the candidate feature data; and then new feature data may be selected from the sorted candidate feature data according to the feature screening algorithm. The feature screening algorithm here may select feature data in sequence according to the sorting.

[0037] The above-mentioned feature quality assessment operation can be to calculate the association rate (or missing rate) of each data feature at the subject level, IV value (an indicator used to measure the predictive ability of the feature) and other feature indicators for feature analysis and viewing. For example, in a certain application scenario, there are 100,000 customers. According to the association of these customers with a certain feature data, 70% can be associated (that is, the feature data corresponding to each customer can be found), then 70% is the association rate of the feature data.

[0038] In response to the fact that the optional feature data in the above-mentioned alternative feature data and the optimal feature data set have been iterated or the time for iterative training of the basic model has reached a predetermined time, it means that the basic model has been trained. At this time, the basic model can be output as a trained standby model, and the corresponding optimal feature data set can be output.

[0039] During the training process of the backup model, that is, when the current credit scoring model is still valid, the model performance comparison operation can be periodically performed between the current credit scoring model and the backup model in the training process to improve the prediction performance of the backup model.

[0040] In order to facilitate modelers and business personnel to analyze and make decisions, the current credit scoring model can also be visualized based on the Shapley value theory (SHAPLEY value, i.e., Shapley value-game theory, which represents the average marginal contribution of a feature in all feature sequences), that is, to perform correlation evaluation operations on multiple feature data in the training data and the credit information predicted by the model.

[0041] Figure 2 is an example flow chart of a credit scoring model monitoring method according to an embodiment of the present invention. Figure 2 As shown in the figure, the process includes the following five parts:

[0042] (1) Construct an extensible hierarchical subject feature library: By integrating multi-dimensional data, a feature library with different themes that is easy to expand, with vertical stratification and horizontal pooling can be formed. The feature data can be updated regularly for the training of the basic model.

[0043] (2) Automatic training of basic models: Using the XGB algorithm, combined with the self-learning and self-iterative characteristics of the algorithm, the objective function is introduced to mechanize and engineer the experience, realize the full automation of the training process, and reduce a large number of tedious manual operations in model training. Among them, the objective function is combined with the forward and backward thinking to select important features from the massive features into the model and eliminate unimportant feature variables.

[0044] (3) Monitor online model performance: By visually analyzing the model, the model process and model results are displayed in a rich and easy-to-understand visual manner, solving the problem of poor interpretability of most machine learning scorecards. Methods such as SHAPLEY value are introduced to increase the interpretability of the model for modelers and business personnel to analyze and make decisions. At the same time, monitor various indicators of the model, such as KS (Kolmogorov-Smirnov, used to evaluate the risk differentiation ability of the model, the indicator measures the difference between the cumulative distribution of good and bad samples), AUC (Area Under Curve, the area under the ROC curve and the coordinate axis), PSI (Population Stability Index, population stability index), etc., and set the model failure conditions through the thresholds of each indicator.

[0045] (4) Automatic model iteration: Based on the continuously updated feature library data, the new model is iteratively trained. The system supports the comparison between the effects of historical models (i.e., basic models in different training processes) and online models. The system can configure the update cycle, variable exclusion rules, indicator thresholds, etc.

[0046] (5) Rapid model deployment: Based on the pre-set model failure conditions, determine whether the online model has failed. If so, switch the online model file and feature data used to the model file and feature data prepared by the iterative program, realize the automatic conversion from development to operation state during the modeling process, and realize rapid model deployment; otherwise, continue to use the online model and continuously update the backup model.

[0047] In order to better understand the embodiments of the present invention, the above five parts are described in detail below.

[0048] (1) Building an extensible hierarchical topic feature library

[0049] The model feature library is a collection of feature variable sources for automated iterative training, including: basic feature sets and derived feature sets. The model feature library has a multi-level subject structure that can meet the application of multiple model feature sets (corresponding to different application scenarios). On the basis of building a top-down unified planning feature library, an elastic expansion mechanism for model automated iterative feature dimensions is established, that is, as available data is continuously discovered, the model variable dimensions are gradually enriched to optimize the model effect.

[0050] Figure 3 is a logical diagram of the model feature library, such as Figure 3 As shown, the model feature library includes: basic data layer, feature processing layer, feature pooling layer, feature collection layer and model application layer.

[0051] Among them, the basic data layer is the basic data table, and the feature processing layer periodically processes the valuable data and derived feature data in the basic data to the feature pool layer to form a feature pool divided by topic. When modeling, the data of the corresponding topic is taken out according to the topic involved in the basic model to form a feature set ready for modeling, and finally applied to model construction and training.

[0052] Among them, personal credit characteristics can be credit industry code, credit education code, number of lending institutions, etc.; consumer finance refers to a modern financial service method that provides consumer loans to consumers of all classes.

[0053] Consumer finance feature sets A, B and C belong to different consumer finance scenarios respectively. For example, consumer finance feature set A is derived from data in the personal credit feature pool + APP data feature pool, and consumer finance feature set B is derived from data in the personal credit feature pool + association relationship feature pool. In other words, the feature pool data ranges included in consumer finance feature sets A, B and C are different.

[0054] At the model application layer, corresponding models are established for each consumer finance feature set A, B and C.

[0055] In addition, if the newly added feature data belongs to an existing type of data, it is added to the corresponding feature pool, otherwise a new subject feature pool is added.

[0056] In one embodiment, a feature quality assessment operation may also be performed, and details may refer to the description in the above embodiment.

[0057] Through the above model feature library, it is possible to support the selection of required training data by topic when training the backup model.

[0058] (2) Automatically train the basic model

[0059] In the embodiment of the present invention, the XGB algorithm is mainly used, combined with the self-learning and self-iterative characteristics of the algorithm, an objective function (such as maximizing KS or maximizing AUC index) is introduced to mechanize and engineer the experience, realize the full automation of the training process, and reduce a large number of tedious manual operations in model training. Among them, the objective function is combined with the forward and backward ideas to select important features from the massive features into the model, and unimportant feature variables are eliminated, and finally a model that meets the preset index threshold is obtained.

[0060] Figure 4 It is a flowchart for automatically training the basic model, such as Figure 4As shown in the figure, the optimization objective function (i.e., the objective function used to optimize the result) refers to the indicator target that needs to be achieved in the training process, such as maximizing KS or maximizing AUC, etc.; determining the base model based on the whitelist means using the relatively high-quality features obtained from the feature quality assessment results in the previous step to train the basic model; then calculating the indicators of the basic model as the initial indicator values, specifically, calculating the KS or AUC value of the basic model, and the calculated value is determined according to the objective function; next, selecting new features to be added according to the feature screening algorithm, the feature screening algorithm can use methods such as sorting by quality assessment results (for example, IV value) and then adding one by one, or adding two by two, or cross-adding, etc., input the selected new features into the model to train the model, and check whether the model indicators are improved. If so, the new features are added to the corresponding feature pool. The process is a forward search for newly added features, followed by a backward search for deleted features (when the model indicators are reduced, the features are deleted).

[0061] In one embodiment, when two new features are added at one time, if the model index is improved (or reduced), the two new features are added (or deleted) together.

[0062] Finally, the model training is terminated based on whether all feature iterations have been completed or the iteration time limit has been reached, and the model results and corresponding features are output.

[0063] (3) Monitoring online model performance

[0064] The poor interpretability of most machine learning scorecards can be solved by visually analyzing the model results. For example, for a customer, the corresponding default probability is calculated using the model, but due to the uninterpretability of the XGB algorithm, it is impossible to evaluate the role of each feature in this result. Based on this, the SHAPLEY value is introduced. The calculation of the SHAPLEY value can be used to evaluate the role of each feature, thereby increasing the interpretability of the model for modelers and business personnel to analyze and make decisions. At the same time, monitor various indicators of the model, such as KS, AUC, PSI, etc., set model failure conditions, such as certain indicators exceeding the threshold twice in a row, etc. When the model fails, it can be replaced with a backup model in time.

[0065] (4) Automatic model iteration

[0066] Figure 5 It is a flowchart of the model automatic iteration, such as Figure 5As shown, for models that need to be iterated, the latest data is used to regularly complete the self-iteration of the model. According to the continuously updated feature library data, the basic model is trained in a rolling manner, which can support the comparison between the historical model effect and the online model effect. The update (iteration) cycle (for example, 1 month), variable exclusion rules (for example, excluding features that are not required for some application scenarios), indicator thresholds (for example, exiting the iteration when the model KS reaches 40% or the AUC reaches 80%), etc. can be configured.

[0067] (5) Rapid model deployment

[0068] Continue to see Figure 5 According to the set model failure conditions, for example, when KS is less than 40%, it can be judged that the online model has failed. Then, the automatic conversion from development to operation state is realized in the modeling process, that is, the backup model replaces the failed online model to realize rapid model deployment, thereby ensuring the normal operation of the business.

[0069] The embodiment of the present invention first builds an extensible hierarchical subject feature library, establishes a basic model, deploys the basic model online and monitors it, and configures and starts an automatic model iteration process. The process automatically builds the latest valid model according to data changes and is ready to replace the online invalid model at any time, thereby ensuring the normal operation of the business.

[0070] Based on similar inventive concepts, an embodiment of the present invention further provides a monitoring device for a credit scoring model, which can be used to implement the process in the above method embodiment.

[0071] Figure 6 is a structural diagram of the credit scoring model monitoring device, such as Figure 6 As shown, the credit scoring model monitoring device includes: a data acquisition unit 61, a data input unit 62, a performance testing unit 63 and a replacement unit 64, wherein:

[0072] The data acquisition unit 61 is used to acquire training data, wherein the training data includes: actual credit information;

[0073] A data input unit 62, used to input the training data into the current credit scoring model to output predicted credit information;

[0074] A performance testing unit 63, configured to perform a model performance test on the current credit scoring model according to the predicted credit information and the actual credit information;

[0075] A replacement unit 64 is used to replace the current credit scoring model with a trained backup model as a new current credit scoring model in response to a model performance test result indicating model failure, wherein the backup model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model.

[0076] By training the backup model while the current credit scoring model is working, and when the performance testing unit 63 tests the performance of the current credit scoring model as invalid based on the predicted credit information and actual credit information of the current credit scoring model, the replacement unit 64 replaces the current credit scoring model with the trained backup model as the new current credit scoring model, thereby overcoming the defect in the prior art that the online model fails and the effective new model cannot be used quickly. Through the embodiments of the present invention, the online invalid model can be prepared to be replaced at any time, thereby ensuring the normal operation of the business.

[0077] In actual operation, Figure 7 As shown, the monitoring device further includes: a model training unit 65, which is used to train the backup model.

[0078] Specifically, the model training unit 65 includes: a selection module and a model training module, wherein: the selection module is used to select a corresponding basic model and a pre-set optimal feature data set according to the type of the current credit scoring model; the model training module is used to train the basic model according to the selected optimal feature data set and based on the XGB algorithm and a predetermined objective function to obtain the backup model.

[0079] In one embodiment, Figure 7 As shown, the monitoring device further includes: a new feature data selection unit 66, a judgment unit 67 and a feature data updating unit 68, wherein:

[0080] The new feature data selection unit 66 is used to select new feature data from the candidate feature data according to the feature screening algorithm.

[0081] In one embodiment, the new feature data selection unit includes: a feature evaluation module and a new feature data selection module, wherein: the feature evaluation module is used to perform feature quality evaluation operations on the candidate feature data to sort the feature data in the candidate feature data; the new feature data selection module is used to select new feature data from the sorted candidate feature data according to the feature screening algorithm.

[0082] The judging unit 67 is used to input the new feature data into the basic model for training to judge whether the performance of the basic model is improved.

[0083] The feature data updating unit 68 is configured to add the new feature data to the optimal feature data set in response to the judgment result that the performance of the basic model is improved, or remove the new feature data otherwise.

[0084] In one embodiment, the above-mentioned model training module is also used to: in response to the optional feature data in the alternative feature data and the optimal feature data set having been iteratively completed or the time for iterative training of the basic model has reached a predetermined time, output the basic model as a trained alternative model and output the corresponding optimal feature data set.

[0085] Continue to see Figure 7 , the above-mentioned monitoring device also includes:

[0086] The model performance comparison unit 69 is used to perform a model performance comparison operation on the current credit scoring model and the backup model in the training process in response to the model performance test result indicating that the model is valid.

[0087] The feature association evaluation unit 610 is used to perform an association evaluation operation on the multiple feature data in the training data and the predicted credit information based on the Shapley value theory.

[0088] The specific execution process of the above-mentioned units and modules can be found in the description of the above-mentioned method embodiment, which will not be repeated here.

[0089] In actual operation, the above-mentioned units and modules may be provided in combination or individually, and the present invention is not limited thereto.

[0090] This embodiment also provides an electronic device, which may be a desktop computer, a tablet computer, a mobile terminal, etc., but this embodiment is not limited thereto. In this embodiment, the electronic device may be implemented with reference to the above method embodiment and the embodiment of the monitoring device for the credit scoring model, the contents of which are incorporated herein, and the repeated parts are not repeated.

[0091] Figure 8 FIG. 6 is a schematic block diagram of a system structure of an electronic device 600 according to an embodiment of the present invention. Figure 8 As shown, the electronic device 600 may include a central processor 100 and a memory 140; the memory 140 is coupled to the central processor 100. It should be noted that this figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0092] In one embodiment, the monitoring function of the credit scoring model may be integrated into the central processor 100. The central processor 100 may be configured to perform the following control:

[0093] Acquiring training data, and inputting the training data into a current credit scoring model to output predicted credit information, wherein the training data includes: actual credit information;

[0094] Performing a model performance test on the current credit scoring model based on the predicted credit information and the actual credit information;

[0095] In response to the model performance test result being model failure, the current credit scoring model is replaced by a trained backup model as a new current credit scoring model. The backup model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model.

[0096] From the above description, it can be seen that the electronic device provided in the embodiment of the present application trains a backup model while the current credit scoring model is working, and when the performance of the current credit scoring model is tested to be invalid based on the predicted credit information and actual credit information of the current credit scoring model, the trained backup model replaces the current credit scoring model as the new current credit scoring model, thereby overcoming the defect in the prior art that the online model fails and the effective new model cannot be used. Through the embodiment of the present invention, the online failed model can be prepared to be replaced at any time to ensure the normal operation of the business.

[0097] In another embodiment, the monitoring device of the credit scoring model may be configured separately from the central processor 100. For example, the monitoring device of the credit scoring model may be configured as a chip connected to the central processor 100, and the monitoring function of the credit scoring model may be implemented through the control of the central processor.

[0098] like Figure 8 As shown, the electronic device 600 may further include: a communication module 110, an input unit 120, an audio processing unit 130, a display 160, and a power supply 170. It is worth noting that the electronic device 600 does not necessarily have to include Figure 8 In addition, the electronic device 600 may also include Figure 8 For components not shown, reference may be made to the prior art.

[0099] like Figure 8 As shown, the central processor 100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The central processor 100 receives inputs and controls the operations of various components of the electronic device 600.

[0100] The memory 140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 100 may execute the program stored in the memory 140 to implement information storage or processing.

[0101] The input unit 120 provides input to the CPU 100. The input unit 120 is, for example, a key or a touch input device. The power supply 170 is used to provide power to the electronic device 600. The display 160 is used to display display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0102] The memory 140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 140 may also be some other type of device. The memory 140 includes a buffer memory 141 (sometimes referred to as a buffer). The memory 140 may include an application / function storage unit 142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 600 through the central processor 100.

[0103] The memory 140 may also include a data storage unit 143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 144 of the memory 140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0104] The communication module 110 is a transmitter / receiver 110 that transmits and receives signals via an antenna 111. The communication module (transmitter / receiver) 110 is coupled to the central processor 100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0105] Based on different communication technologies, multiple communication modules 110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module. The communication module (transmitter / receiver) 110 is also coupled to a speaker 131 and a microphone 132 via an audio processor 130 to provide an audio output via the speaker 131 and receive an audio input from the microphone 132, thereby realizing a common telecommunication function. The audio processor 130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 130 is also coupled to the central processor 100, so that the sound can be recorded on the local machine through the microphone 132, and the sound stored on the local machine can be played through the speaker 131.

[0106] An embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the monitoring method of the credit scoring model described above are implemented.

[0107] In summary, the embodiments of the present invention provide a solution for constructing a consumer finance credit scoring model that can be automatically iterated. It utilizes the self-learning and self-iterative characteristics of machine learning, especially decision tree ensemble learning and deep learning, introduces an objective function, mechanizes and engineers the experience, and realizes full automation of the iteration process, thereby reducing a large amount of tedious manual operations in model iteration. At the same time, it attaches importance to visual analysis, and provides a rich and easy-to-understand visual display of the model process and model results, so as to solve the problem that most machine learning scorecards have poor interpretability. It introduces SHAPLEY values ​​to increase the interpretability of the model for modelers and business personnel to analyze and make decisions. In addition, it realizes the automatic conversion from development to operation state in the modeling process, realizes the rapid deployment of the model, and thus ensures the normal operation of the business.

[0108] The preferred embodiments of the present invention are described above with reference to the accompanying drawings. Many features and advantages of these embodiments are clear from this detailed description, and therefore the claims are intended to cover all such features and advantages of these embodiments that fall within their true spirit and scope. In addition, since many modifications and changes are easily conceivable to those skilled in the art, it is not intended to limit the embodiments of the present invention to the precise structure and operation illustrated and described, but all suitable modifications and equivalents falling within its scope may be covered.

[0109] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0110] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0111] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0112] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0113] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for monitoring a credit scoring model, characterized in that: The method comprises: Acquire training data, and input the training data into the current credit scoring model to output predicted credit information, wherein the training data includes: actual credit information; Performing a model performance test on the current credit scoring model based on the predicted credit information and the actual credit information; In response to the model performance test result indicating that the model is invalid, replacing the current credit scoring model with a trained standby model as a new current credit scoring model, wherein the standby model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model; It also includes: when the model performance test result is that the model is valid, periodically comparing the model performance of the current credit scoring model with the backup model in the training process; The backup model is trained as follows: Selecting a corresponding basic model and a preset optimal feature data set according to the type of the current credit scoring model; According to the selected optimal feature data set, the basic model is trained based on the XGB algorithm and a predetermined objective function to obtain the backup model; Also includes: Selecting new feature data from candidate feature data according to a feature screening algorithm; Inputting the new feature data into the basic model for training to determine whether the performance of the basic model is improved; In response to the judgment result that the performance of the basic model is improved, adding the new feature data to the optimal feature data set, otherwise, removing the new feature data; Also includes: Monitor online model performance: Visually analyze the basic model, visualize the model process and model results, and introduce SHAPLEY values ​​to increase the interpretability of the model. Monitor various indicators of the model and set model failure conditions through thresholds of various indicators.

2. The method according to claim 1, characterized in that Selecting new feature data from candidate feature data according to the feature screening algorithm includes: Performing a feature quality assessment operation on the candidate feature data to sort each feature data in the candidate feature data; New feature data is selected from the sorted candidate feature data according to the feature screening algorithm.

3. The method according to claim 1, characterized in that The method further comprises: In response to the optional feature data in the candidate feature data and the optimal feature data set having been iteratively completed or the time for iterative training of the basic model has reached a predetermined time, the basic model is output as a trained standby model, and the corresponding optimal feature data set is output.

4. The method according to claim 1, characterized in that: The method further comprises: In response to the model performance test result being that the model is valid, a model performance comparison operation is performed on the current credit scoring model and the backup model in the training process.

5. The method according to claim 1, characterized in that: The method further comprises: Based on the Shapley value theory, an association evaluation operation is performed on the multiple feature data in the training data and the predicted credit information.

6. A monitoring device for a credit scoring model, characterized in that: The device is used to implement the method according to any one of claims 1 to 5, and the device comprises: A data acquisition unit, used to acquire training data, wherein the training data includes: actual credit information; A data input unit, used to input the training data into the current credit scoring model to output predicted credit information; A performance testing unit, used to perform a model performance test on the current credit scoring model based on the predicted credit information and the actual credit information; A replacement unit is used to replace the current credit scoring model with a trained backup model as a new current credit scoring model in response to a model performance test result indicating model failure, wherein the backup model is iteratively trained based on a predetermined basic model and updateable data samples during the operation of the current credit scoring model.

7. The device according to claim 6, characterized in that The device further comprises: a model training unit, configured to train the backup model. The model training unit comprises: A selection module, used to select a corresponding basic model and a preset optimal feature data set according to the type of the current credit scoring model; The model training module is used to train the basic model according to the selected optimal feature data set, based on the XGB algorithm and a predetermined objective function, to obtain the backup model.

8. The device according to claim 7, characterized in that The device also includes: A new feature data selection unit, used for selecting new feature data from candidate feature data according to a feature screening algorithm; A judging unit, used for inputting the new feature data into the basic model for training to judge whether the performance of the basic model is improved; The feature data updating unit is used to add the new feature data to the optimal feature data set in response to the judgment result that the performance of the basic model is improved, otherwise, remove the new feature data.

9. The device according to claim 8, characterized in that The new feature data selection unit comprises: A feature evaluation module, used for performing a feature quality evaluation operation on the candidate feature data to sort each feature data in the candidate feature data; The new feature data selection module is used to select new feature data from the sorted candidate feature data according to the feature screening algorithm.

10. The device according to claim 8, characterized in that The model training module is also used to: In response to the optional feature data in the candidate feature data and the optimal feature data set having been iteratively completed or the time for iterative training of the basic model has reached a predetermined time, the basic model is output as a trained standby model, and the corresponding optimal feature data set is output.

11. The device according to claim 6, characterized in that The device also includes: The model performance comparison unit is used to perform a model performance comparison operation on the current credit scoring model and the backup model in the training process in response to the model performance test result indicating that the model is valid.

12. The device according to claim 6, characterized in that The device also includes: A feature association evaluation unit is used to perform an association evaluation operation on multiple feature data in the training data and the predicted credit information based on the Shapley value theory.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 5 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images