Automatic iteration method and device of risk control model and electronic equipment

Abstract

The invention provides an automatic iteration method of a risk control model, and relates to the technical field of model iteration, in particular to the technical field of automatic iteration methods and devices of mutual risk control models and electronic equipment. According to the specific implementation scheme, a timed iteration task is triggered according to a preset period, and a preset hyper-parameter corresponding to the timed iteration task of the current round and latest risk business data are obtained; and based on the hyper-parameters, taking the latest risk business data as a training sample, and training a risk control model of a current version to obtain an iterated risk control model. According to the method provided by the invention, the automatic iteration of the model can be realized based on the preset period and the hyper-parameter, and the timeliness and accuracy of the model are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of model iteration technology, and in particular to the field of automatic iteration methods, devices and electronic equipment for mutual risk control models. Background Art

[0002] With the rapid development of technologies such as the Internet, big data, and artificial intelligence, the platform is facing more and more business risk data in the business process. For example, there are a variety of business risk data in the transaction business of the financial platform. Effectively evaluating and controlling these business risk data is of great significance to the business security of the platform. Existing risk assessment methods usually use static risk control models to evaluate risk business data. In this way, the iteration of the risk control model is uncertain. After the model indicators drop to a certain level, they are retrained and updated, or retraining is triggered artificially. This method can quantify the risk to a certain extent, but the uncertainty of the iteration timing leads to the instability of the model timeliness, and due to the fixedness of the model parameters, it is difficult to adapt to the changing business scenarios, so the accuracy of the risk control assessment cannot be guaranteed. Summary of the invention

[0003] The present disclosure provides a method, device and electronic device for automatically iterating a risk control model to solve at least one of the above-mentioned technical problems.

[0004] According to one aspect of the present disclosure, a method for automatically iterating a risk control model is provided, wherein the method comprises:

[0005] Triggering a scheduled iterative task according to a preset period, and obtaining preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round;

[0006] Based on the hyperparameters, the latest risk business data is used as a training sample to train the current version of the risk control model to obtain an iterated risk control model.

[0007] In some examples, before triggering the scheduled iterative task according to a preset period and obtaining the preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round, the method includes:

[0008] Setting the period of the scheduled iterative task, wherein the period includes: a scheduling period, a time period, and a task effective date;

[0009] The scheduling period indicates the interval between two adjacent rounds of the scheduled iterative tasks; the time period indicates the training duration of a single scheduled iterative task; the task effective date indicates the effective triggering time of the scheduled iterative task;

[0010] The triggering of the scheduled iterative task according to the preset period includes:

[0011] Determining whether the current date satisfies the task effective date;

[0012] If it is determined that the condition is met, determining whether the interval between the current time and the previous round of scheduled iterative tasks meets the scheduling period;

[0013] When it is determined that the conditions are met, the scheduled iterative task is triggered according to the preset training duration.

[0014] In some examples, after the current version of the risk control model is trained based on the hyperparameters and the latest risk business data is used as a training sample to obtain an iterated risk control model, the method further includes:

[0015] The iterated risk control model is evaluated based on preset evaluation conditions, and when it is determined that the risk control model meets the evaluation conditions, the risk control model is put into online application.

[0016] In some examples, the preset evaluation condition includes: at least one of a confidence threshold evaluation condition and an indicator curve threshold evaluation condition;

[0017] In the case where the preset evaluation condition is the confidence threshold evaluation condition, the evaluating the iterated risk control model based on the preset evaluation condition includes:

[0018] Determining the confidence of the risk control model after iteration, and determining whether the confidence meets the confidence threshold evaluation condition;

[0019] and / or,

[0020] In a case where the preset evaluation condition is the indicator curve threshold evaluation condition, the evaluating the iterated risk control model based on the preset evaluation condition includes:

[0021] Determine an indicator curve of the risk control model after iteration, and determine whether the indicator curve meets the indicator curve threshold evaluation condition.

[0022] In some examples, after training the current version of the risk control model based on the hyperparameters and taking the latest risk business data as training samples to obtain an iterative risk control model, the iterative risk control model is evaluated based on preset evaluation conditions. When it is determined that the risk control model meets the evaluation conditions, before the risk control model is put online for application, the method further includes:

[0023] According to the preset file parameters, the iterated risk control model is output as a risk control model file and saved to the model management center;

[0024] The preset file parameters include: the file name of the model file, the label of the model file, the description information of the model file, and the version number of the model file;

[0025] The step of evaluating the iterated risk control model based on a preset evaluation condition, and applying the risk control model online when it is determined that the risk control model meets the evaluation condition, includes:

[0026] The iterated risk control model is evaluated at the model management center, and when it is determined that the risk control model meets the evaluation conditions, the risk control model is pushed to the online business system for going online.

[0027] In some examples, before setting the period of the scheduled iterative task, the method further includes:

[0028] Create an initial risk control model and set initial hyperparameters;

[0029] The step of creating an initial risk control model and setting initial hyperparameters includes:

[0030] Determine multiple target operator modules from a preset operator module list;

[0031] Arranging connection relationships between the multiple target operator modules;

[0032] Setting operator hyperparameters of at least some of the target operator modules among the multiple target operator modules, and obtaining the initial hyperparameters based on the operator hyperparameters of at least some of the target operator modules;

[0033] Among them, the first round of scheduled iterative tasks trains the initial risk control model based on the initial hyperparameters; the non-first round of scheduled iterative tasks trains the iterated risk control model based on the latest version of the hyperparameters and the previous round of scheduled iterative tasks.

[0034] In some examples, based on the hyperparameters, the latest risk business data is used as a training sample to train the current version of the risk control model to obtain an iterated risk control model, including:

[0035] The training stop condition parameters are obtained from the hyperparameters, and the latest risk business data is used as a training sample to train the current version of the risk control model. When the risk control model in training meets the training stop condition parameters, the training is stopped to obtain the iterated risk control model.

[0036] In some examples, before triggering the scheduled iterative task according to a preset period and obtaining the preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round, the method further includes:

[0037] According to a preset data collection cycle, original data of risk business is collected, and the original data of risk business is preprocessed to obtain the risk business data, and the risk business data is processed into a feature wide table form.

[0038] According to one aspect of the present disclosure, there is provided an automatic iteration device for a risk control model, wherein the device comprises:

[0039] A timing trigger module is used to trigger the timing iteration task according to a preset period, and obtain the preset hyperparameters and the latest risk business data corresponding to the timing iteration task of the current round;

[0040] The iteration module is used to train the current version of the risk control model based on the hyperparameters and take the latest risk business data as training samples to obtain an iterated risk control model.

[0041] In some examples, the apparatus further comprises:

[0042] A cycle setting module, used to set the cycle of the scheduled iterative task, wherein the cycle includes: a scheduling cycle, a time cycle, and a task effective date;

[0043] The scheduling period indicates the interval between two adjacent rounds of the scheduled iterative tasks; the time period indicates the training duration of a single scheduled iterative task; the task effective date indicates the effective triggering time of the scheduled iterative task;

[0044] The timing trigger module is used, when the timing iteration task is triggered according to the preset period, to:

[0045] Determining whether the current date satisfies the task effective date;

[0046] If it is determined that the condition is met, determining whether the interval between the current time and the previous round of scheduled iterative tasks meets the scheduling period;

[0047] When it is determined that the conditions are met, the scheduled iterative task is triggered according to the preset training duration.

[0048] In some examples, the apparatus further comprises:

[0049] The evaluation module is used to evaluate the iterated risk control model based on preset evaluation conditions, and put the risk control model online for application when it is determined that the risk control model meets the evaluation conditions.

[0050] In some examples, the preset evaluation condition includes: at least one of a confidence threshold evaluation condition and an indicator curve threshold evaluation condition;

[0051] In the case where the preset evaluation condition is the confidence threshold evaluation condition, the evaluation module is used to:

[0052] Determining the confidence of the risk control model after iteration, and determining whether the confidence meets the confidence threshold evaluation condition;

[0053] and / or,

[0054] In the case where the preset evaluation condition is the indicator curve threshold evaluation condition, the evaluation module is used to:

[0055] Determine an indicator curve of the risk control model after iteration, and determine whether the indicator curve meets the indicator curve threshold evaluation condition.

[0056] In some examples, the apparatus further comprises:

[0057] A file output module, used to output the iterated risk control model as a risk control model file according to preset file parameters, and save it to the model management center;

[0058] The preset file parameters include: the file name of the model file, the label of the model file, the description information of the model file, and the version number of the model file;

[0059] The evaluation module is specifically used for:

[0060] The iterated risk control model is evaluated at the model management center, and when it is determined that the risk control model meets the evaluation conditions, the risk control model is pushed to the online business system for going online.

[0061] In some examples, the apparatus further comprises:

[0062] Model creation module, used to create the initial risk control model and set the initial hyperparameters;

[0063] The model creation module is specifically used for:

[0064] Determine multiple target operator modules from a preset operator module list;

[0065] Arranging connection relationships between the multiple target operator modules;

[0066] Setting operator hyperparameters of at least some of the target operator modules among the multiple target operator modules, and obtaining the initial hyperparameters based on the operator hyperparameters of at least some of the target operator modules;

[0067] Among them, the first round of scheduled iterative tasks trains the initial risk control model based on the initial hyperparameters; the non-first round of scheduled iterative tasks trains the iterated risk control model based on the latest version of the hyperparameters and the previous round of scheduled iterative tasks.

[0068] In some examples, the iteration module is specifically used to:

[0069] The training stop condition parameters are obtained from the hyperparameters, and the latest risk business data is used as a training sample to train the current version of the risk control model. When the risk control model in training meets the training stop condition parameters, the training is stopped to obtain the iterated risk control model.

[0070] In some examples, the apparatus further comprises:

[0071] The data collection module is used to collect the original data of risk business according to the preset data collection cycle, pre-process the original data of risk business, obtain the risk business data, and process the risk business data into a feature wide table form.

[0072] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0073] at least one processor; and

[0074] a memory communicatively connected to the at least one processor; wherein,

[0075] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above method.

[0076] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the above method.

[0077] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program implements the above method when executed by a processor.

[0078] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0080] Figure 1 It is a flowchart of an automatic iteration method of a risk control model provided by the first embodiment of the present disclosure;

[0081] Figure 2 It is a schematic diagram of an interface of a process of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0082] Figure 3 This is a schematic diagram of an interface of an operator module list of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0083] Figure 4 It is a schematic diagram of an interface of a visual creation method of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0084] Figure 5 It is a schematic diagram of an interface for setting hyperparameters of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0085] Figure 6 This is a schematic diagram of an interface for periodic settings of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0086] Figure 7 It is a schematic diagram of an interface of evaluation conditions of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0087] Figure 8 It is a schematic diagram of an interface for field settings of an automatic iteration method of a risk control model provided in the second embodiment of the present disclosure;

[0088] Fig. 9 It is a structural schematic diagram of an automatic iteration method device of a risk control model provided in the third embodiment of the present disclosure;

[0089] Fig.10 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0090] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0091] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0092] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0093] The terminology used herein is for describing particular embodiments only and is not intended to limit the present disclosure.As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0094] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0095] The automatic iteration method of the risk control model according to the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory. Alternatively, the automatic iteration method of the risk control model provided by the present disclosure can be executed by a server.

[0096] In the first embodiment disclosed, see Figure 1 , Figure 1 A flow chart of an automatic iteration method of a risk control model provided by the first embodiment of the present disclosure is shown. The method includes:

[0097] S101. Trigger a scheduled iterative task according to a preset period, and obtain preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round.

[0098] S102. Based on the hyperparameters, the latest risk business data is used as a training sample to train the current version of the risk control model to obtain an iterated risk control model.

[0099] Among them, risk business data refers to various types of data generated in the course of business operations, such as identity authentication data, business operation data, etc., which are not limited here; the risk control model is a model used to assess the risk level of risk business data, which can be a deep learning model or a mathematical model, etc., which are not limited here.

[0100] The purpose of the scheduled iterative task is to trigger the retraining of the risk control model so as to iterate the risk control model according to the training results.

[0101] In the present disclosure, a cycle is preset for the scheduled iterative task. When the current time meets the preset cycle, the scheduled iterative task is automatically triggered, and the preset hyperparameters corresponding to the current round of scheduled iterative tasks and the latest risk business data are obtained to train the risk control model. In this way, iterative training can be automatically started at a scheduled time, and the latest risk business data and preset hyperparameters can be introduced for training, thereby improving the timeliness of the risk control model, and the risk control model can be adaptively trained based on the adjustable hyperparameters, thereby improving the adaptability and accuracy of the risk control model.

[0102] In some examples, after S102, the method further includes:

[0103] S103. Evaluate the iterated risk control model based on preset evaluation conditions, and if it is determined that the risk control model meets the evaluation conditions, put the risk control model online for application.

[0104] After the training is completed and the iterative risk control model is obtained, in order to further ensure the accuracy of automatic iteration, the iterative risk control model can be evaluated based on the evaluation conditions, that is, the accuracy and stability of the risk control model can be determined. After the evaluation passes, the iterative risk control model can be put online for application. If the evaluation fails, it is regarded as an iteration failure, and the iterative risk control model will not be put online for application. In this way, the iterative risk control model can be applied after determining that the predictive ability of the iterative risk control model is compliant, thereby ensuring the safety and accuracy of automatic iteration.

[0105] Among them, the preset evaluation conditions are conditions for evaluating the predictive ability of the risk control model, for example, may include: at least one of the confidence threshold evaluation conditions and the indicator curve threshold evaluation conditions, which are not limited here.

[0106] In disclosing the second embodiment, see Figure 2 , Figure 2 A flow chart of an automatic iteration method of a risk control model provided by the second embodiment of the present disclosure is shown. The method includes:

[0107] S201. Collect original risk business data according to a preset data collection cycle, pre-process the original risk business data to obtain risk business data, and process the risk business data into a feature wide table format.

[0108] The present disclosure provides data management capabilities to collect original risk business data (i.e., unprocessed risk business data) according to a preset collection cycle, wherein the collection cycle can be set according to the data warehousing time T (i.e., the time when the data is added to the database), and there is no restriction here.

[0109] The preprocessing may include various types of data processing operations, such as data collection, data labeling, data cleaning, data processing, data conversion, data screening, etc., and may support the processing of risk business data in the form of images, text, tables, etc.

[0110] In some examples, data may also be processed through an external platform, such as a Structured Query Language (SQL) platform, and the distributed computing capabilities of the SQL data analysis engine may be used to analyze and process large amounts of data.

[0111] In the present disclosure, by connecting with the underlying data platform, new risk business data is acquired in real time and preprocessed to form training samples. During iterative training, the latest risk business data is used for training, thereby ensuring the timeliness of the model.

[0112] In some examples, in S201, the original data of risk business is preprocessed to obtain risk business data, and the risk business data is processed into a feature wide table format, including:

[0113] Sub-step 1: Construct risk characteristics based on the original data of risk business.

[0114] For example, in the actual scenario based on the characteristics of risky business, for example, in the financial scenario, risk characteristics are constructed from the dimensions of pre-loan, mid-loan, post-loan and combination. Pre-loan dimension: This dimension includes the borrower's credit information, credit record and other characteristics and evaluations, asset-liability profit characteristics, loan amount evaluation characteristics, etc. Mid-loan dimension: This dimension design includes policy characteristics, compliance approval process and risk factor characteristics, contract content characteristics, etc. Post-loan dimension: This dimension design includes each loan data monitoring characteristics, maturity characteristics and overdue processing characteristics.

[0115] Sub-step 2: Screen the original data of risky business based on risk characteristics to obtain risky business data.

[0116] Samples are designed based on existing customer tags, iteration time, etc., and a data screening mechanism is introduced simultaneously. A filtering mechanism is set up from the feature dimension to clean the original data of risky businesses, so as to obtain the filtered risky business data and the unfiltered original risky business data, and save both of them in the data warehouse.

[0117] Sub-step three: Process the risk business data into a feature wide table format.

[0118] Among them, the data warehouse storage design can perform offline calculation of feature values ​​at a preset period and then update them to obtain a more comprehensive set of candidate feature values. Finally, a feature wide table is synthesized by correlating the risk business data in different feature libraries. In iterative training, the feature wide table is used as the input of the risk control model.

[0119] S202: Create an initial risk control model and set initial hyperparameters.

[0120] Among them, S201 has multiple creation methods, for example: at least one of a coding creation method and a visualization creation method.

[0121] The coding creation method supports multiple types, such as interactive notebook (jupyter notebook) coding creation method, notebook (jupyterlab) coding creation method, and Red Spider (spyder) coding creation method. Taking the interactive notebook coding creation method as an example, Notebook modeling has a built-in fully managed interactive programming environment to achieve data processing and code debugging. It can be designed to support development frameworks such as PyTorch, scikit-learn, and extreme gradient boosting tree (XGBoost). In some examples, the container (Docker) technology is used to encapsulate the required machine learning framework and required resources into the Notebook instance for automatic generation. Business personnel do not need to pay attention to the installation and configuration of the basic environment, and can directly use the Notebook development environment to create risk control models. The Notebook code details are automatically completed. In terms of resources, instances can be stopped and restarted to save resources. The trained resources can mount data sets and storage, and the risk control model can be published to the model management center through the command line to improve the efficiency and effect of model training.

[0122] See also Figure 3-Figure 5 , Figure 3 A schematic diagram showing an exemplary interface of an operator module list; Figure 4 An interface schematic diagram showing an exemplary visualization creation method; Figure 5 An exemplary interface diagram of hyperparameter settings is shown. Taking the visualization creation method as an example, S201 includes:

[0123] Sub-step 1: Determine multiple target operator modules from a preset operator module list.

[0124] Sub-step 2: Arrange the connection relationship between multiple target operator modules.

[0125] Sub-step three: setting operator hyperparameters of at least some of the target operator modules among the multiple target operator modules, and obtaining initial hyperparameters based on the operator hyperparameters of at least some of the target operator modules.

[0126] First, multiple operator modules can be set according to the form of the formed risk control model. For example, see Figure 3 , if the risk control model is based on XGBoost, multiple module class library objects are set. One operator module corresponds to one module. The module contains the display form, logic and dependency, and the relevant SQL scripts to be executed, etc. Set the running process for each operator module, that is, through the input and configuration parameters, the results are obtained through the operator logic and written to the output end. For example, in the figure, the operator module list (i.e., "algorithm component") includes multiple operator modules, such as logistic regression binary classification operator module, random forest binary classification operator module, etc.

[0127] Among them, you can drag the selected target operator module from the preset operator module list to the visual arrangement window (such as Figure 4 As shown in the figure, the connection relationship between multiple target operator modules is then arranged through wiring to form a directed acyclic graph that constitutes the pipeline of the entire risk control model. The entire pipeline supports replication to facilitate the construction of models of the same type. In some examples, when running the pipeline, each operator is executed separately in the container in turn, and the data of each operator is connected through files and stored on a shared file platform.

[0128] Among them, hyperparameters can include various types of hyperparameters, for example, hyperparameters include: learning rate, number of training times (number of iterations), etc. Hyperparameters include operator hyperparameters of each operator module, see Figure 5 Taking the data set operator module as an example, the operator hyperparameters that can be adjusted by this operator module include: maximum tree depth, number of iterations, minimum leaf node sample weight and so on; taking the logistic regression binary classification operator module as an example, the operator hyperparameters that can be adjusted by this operator module include: alpha value range, maximum number of iterations, target convergence threshold, etc., which are not limited here. According to the functions and operation logic of different operator modules, operator hyperparameters can be configured for them. During iteration, when an operator module is operated, the input and output of the operator module are processed and operated according to its corresponding operator hyperparameters.

[0129] For some examples, see Figure 8 , Figure 8 A schematic diagram of an exemplary field setting interface is shown. Hyperparameters may also include field settings, and feature columns of the input feature wide table may be selected for batch operation, and label columns, positive sample label values, etc. may also be selected.

[0130] Among them, the first round of scheduled iterative tasks trains the initial risk control model based on the initial hyperparameters; the non-first round of scheduled iterative tasks trains the iterated risk control model based on the latest version of the hyperparameters and the previous round of scheduled iterative tasks.

[0131] It should be noted that in each step of the entire method, the hyperparameters (including each operator hyperparameter) can be modified as needed, and the next round of iterative training uses the modified hyperparameters. If the hyperparameters are not modified, the original hyperparameters are used for training in each round of iteration. In this way, the hyperparameters can be set flexibly and modified according to different business needs and business scenarios, thereby ensuring the adaptability of the iterative risk control model.

[0132] There is no restriction on the order of S201 and S202, and the execution order can be interchanged.

[0133] S203: Set the period of the scheduled iterative task.

[0134] See also Figure 6 , Figure 6 An exemplary interface diagram of period setting is shown. In some examples, the period includes: a scheduling period, a time period, and a task effective date (ie, the "effective date" in the figure).

[0135] The scheduling period indicates the interval between two adjacent rounds of scheduled iterative tasks; the time period indicates the training duration of a single scheduled iterative task; and the task effective date indicates the effective triggering time of the scheduled iterative task.

[0136] In addition, the scheduling modes of scheduled iterative tasks include single-round running and cyclic running, which can be flexibly adjusted.

[0137] S204: Trigger the scheduled iterative task according to a preset period, and obtain the preset hyperparameters and the latest risk business data corresponding to the current round of scheduled iterative tasks.

[0138] Among them, triggering the scheduled iteration task according to the preset period in S204 specifically includes:

[0139] Sub-step 1: Determine whether the current date meets the task effective date.

[0140] Sub-step 2: If it is determined that the condition is met, determine whether the interval between the current time and the previous round of scheduled iteration tasks meets the scheduling period.

[0141] Sub-step three: When it is determined that the conditions are met, trigger the scheduled iteration task according to the preset training duration.

[0142] If any of the above periodic conditions is not met, the scheduled iteration task will not be triggered.

[0143] Based on the scheduling cycle, time cycle, and task effective date set in S203, the triggering conditions may include: triggering within the effective time (i.e., verification of sub-step one), triggering when the scheduling cycle arrives (i.e., verification of sub-step two), and the time of an iterative training can be controlled according to the training duration. Through the above mechanism, the training time, duration, and frequency of triggering training can be flexibly adjusted, and different cycles can be set according to business scenarios to ensure the adaptability of automatic iteration.

[0144] S205. Based on the hyperparameters, the latest risk business data is used as a training sample to train the current version of the risk control model to obtain an iterated risk control model.

[0145] Based on the above, the hyperparameters include at least a plurality of operator hyperparameters, and the hyperparameters can control the operation effect of the operator. In some examples, the hyperparameters may also include training stop condition parameters.

[0146] In some examples, S205 specifically includes:

[0147] The training stop condition parameters are obtained from the hyperparameters, and the latest risk business data is used as training samples to train the current version of the risk control model. When the risk control model in training meets the training stop condition parameters, the training is stopped to obtain the iterated risk control model.

[0148] For example, the training stop condition parameters are: number of iterations, learning rate threshold, etc.; when the risk control model meets the training stop condition parameters, the training is stopped.

[0149] S206. According to preset file parameters, the iterated risk control model is output as a risk control model file and saved to the model management center.

[0150] The preset file parameters include: the file name of the model file, the label of the model file, the description information of the model file, and the version number of the model file.

[0151] Continue to see Figure 6 , Figure 6 The lower part shows an exemplary file parameter setting interface (i.e., the part below "Automatic Output Module Settings"). Among them, "Model Name" is the file name of the model file, and "Model Label" is the label of the model file. The description information of the model file and the version number of the model file are not shown in the figure.

[0152] The model management center is used to uniformly manage risk control models of all modeling methods and record the process of their creation and training. Third-party models can also be imported and managed uniformly. In this disclosure, models (including risk control models) can be classified, saved and managed based on labels and description information, and version management can be performed based on the process and result information such as data, parameters, environment, and model used in model training to achieve horizontal comparison of various indicators and accelerate the model tuning process. The model management center supports the import of existing model files with specified file structures. In this way, it is easy to manage and evaluate each model and compare the training process.

[0153] S207. Evaluate the iterated risk control model in the model management center. If it is determined that the risk control model meets the evaluation conditions, push the risk control model to the online business system for launch.

[0154] Among them, the preset evaluation conditions are conditions for evaluating the predictive ability of the risk control model, for example, may include: at least one of the confidence threshold evaluation conditions and the indicator curve threshold evaluation conditions, which are not limited here.

[0155] Based on this, when the preset evaluation condition is a confidence threshold evaluation condition, S103 includes:

[0156] Determine the confidence level of the risk control model after iteration, and determine whether the confidence level meets the confidence threshold assessment conditions.

[0157] Among them, the confidence threshold evaluation condition is a preset confidence threshold, and the risk control model is evaluated based on whether the confidence of the risk control model reaches the confidence threshold.

[0158] and / or,

[0159] When the preset evaluation condition is an indicator curve threshold evaluation condition, S103 includes:

[0160] Determine the indicator curve of the iterated risk control model and determine whether the indicator curve meets the indicator curve threshold evaluation conditions.

[0161] The indicator curve threshold may include various curve features of the indicator curve, such as the area under the curve, the curve vertex value, etc., which are not limited here.

[0162] See also Figure 7 , Figure 7 A schematic diagram of an exemplary evaluation condition interface is shown. The indicator curve may include multiple types, such as: precision-recall (PR) curve, generalization capability (ROC) curve, discrimination evaluation index (KS) curve, gain curve, etc. Taking the ROC curve as an example, the indicator curve threshold may be, for example, the area under the ROC curve.

[0163] After the training is completed and the iterative risk control model is obtained, in order to further ensure the accuracy of automatic iteration, the iterative risk control model can be evaluated based on the evaluation conditions, that is, the accuracy and stability of the risk control model can be determined. After the evaluation passes, the iterative risk control model can be put online for application. If the evaluation fails, it is regarded as an iteration failure, and the iterative risk control model will not be put online for application. In this way, the iterative risk control model can be applied after determining that the predictive ability of the iterative risk control model is compliant, thereby ensuring the safety and accuracy of automatic iteration.

[0164] It should be noted that S207 is an implementation method of S103. In some examples, S206 can be omitted, that is, there may be no model management center, and the iterated risk control model can be evaluated directly on the training platform.

[0165] In the third embodiment disclosed, based on Figure 1 The same principle, Fig. 9 An automatic iteration device 90 of a risk control model provided by the third embodiment of the present disclosure is shown, wherein the device comprises:

[0166] The timing trigger module 901 is used to trigger the timing iteration task according to the preset period, and obtain the preset hyperparameters and the latest risk business data corresponding to the current round of the timing iteration task;

[0167] The iteration module 902 is used to train the current version of the risk control model based on the hyperparameters and take the latest risk business data as training samples to obtain an iterated risk control model.

[0168] In some examples, the apparatus further includes:

[0169] The cycle setting module is used to set the cycle of the scheduled iterative task, where the cycle includes: scheduling cycle, time cycle, and task effective date;

[0170] The scheduling period indicates the interval between two adjacent rounds of scheduled iterative tasks; the time period indicates the training duration of a single scheduled iterative task; the task effective date indicates the effective triggering time of the scheduled iterative task;

[0171] The timing trigger module is used to trigger the timing iteration task according to the preset period:

[0172] Determine whether the current date meets the task effective date;

[0173] If it is determined that the conditions are met, determine whether the interval between the current time and the previous round of scheduled iteration tasks meets the scheduling period;

[0174] When it is determined that the conditions are met, the scheduled iterative task is triggered according to the preset training duration.

[0175] In some examples, the apparatus further includes:

[0176] The evaluation module is used to evaluate the iterated risk control model based on preset evaluation conditions, and put the risk control model online for application if it is determined that the risk control model meets the evaluation conditions.

[0177] In some examples, the preset evaluation condition includes: at least one of a confidence threshold evaluation condition and an indicator curve threshold evaluation condition;

[0178] When the preset evaluation condition is a confidence threshold evaluation condition, the evaluation module is used to:

[0179] Determine the confidence level of the risk control model after iteration, and determine whether the confidence level meets the confidence threshold assessment conditions;

[0180] and / or,

[0181] When the preset evaluation condition is the indicator curve threshold evaluation condition, the evaluation module is used to:

[0182] Determine the indicator curve of the iterated risk control model and determine whether the indicator curve meets the indicator curve threshold evaluation conditions.

[0183] In some examples, the apparatus further includes:

[0184] The file output module is used to output the iterated risk control model as a risk control model file according to the preset file parameters and save it to the model management center;

[0185] The preset file parameters include: the file name of the model file, the label of the model file, the description information of the model file, and the version number of the model file;

[0186] The evaluation modules are specifically used to:

[0187] The iterated risk control model is evaluated in the model management center. When it is determined that the risk control model meets the evaluation conditions, the risk control model is pushed to the online business system for launch.

[0188] In some examples, the apparatus further includes:

[0189] Model creation module, used to create the initial risk control model and set the initial hyperparameters;

[0190] The model creation module is specifically used for:

[0191] Determine multiple target operator modules from a preset operator module list;

[0192] Arrange the connection relationship between multiple target operator modules;

[0193] Setting operator hyperparameters of at least some of the target operator modules among the multiple target operator modules, and obtaining initial hyperparameters based on the operator hyperparameters of at least some of the target operator modules;

[0194] Among them, the first round of scheduled iterative tasks trains the initial risk control model based on the initial hyperparameters; the non-first round of scheduled iterative tasks trains the iterated risk control model based on the latest version of the hyperparameters and the previous round of scheduled iterative tasks.

[0195] In some examples, the iteration module is specifically used to:

[0196] The training stop condition parameters are obtained from the hyperparameters, and the latest risk business data is used as training samples to train the current version of the risk control model. When the risk control model in training meets the training stop condition parameters, the training is stopped to obtain the iterated risk control model.

[0197] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0198] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0199] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0200] like Fig.10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0201] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0202] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the automatic iteration method of the risk control model. For example, in some embodiments, the automatic iteration method of the risk control model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the automatic iteration method of the risk control model described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured as an automatic iteration method of the risk control model in any other appropriate manner (eg, by means of firmware).

[0203] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0204] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0205] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0206] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0207] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0208] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0209] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0210] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. An automatic iteration method for a risk control model, wherein: The method comprises: Triggering a scheduled iterative task according to a preset period, and obtaining preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round; Based on the hyperparameters, the latest risk business data is used as a training sample to train the current version of the risk control model to obtain an iterated risk control model.

2. The method according to claim 1, wherein: Before triggering the scheduled iterative task according to the preset period and obtaining the preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round, the method includes: Setting the period of the scheduled iterative task, wherein the period includes: a scheduling period, a time period, and a task effective date; The scheduling period indicates the interval between two adjacent rounds of the scheduled iterative tasks; the time period indicates the training duration of a single scheduled iterative task; the task effective date indicates the effective triggering time of the scheduled iterative task; The triggering of the scheduled iterative task according to the preset period includes: Determining whether the current date satisfies the task effective date; If it is determined that the condition is met, determining whether the interval between the current time and the previous round of scheduled iterative tasks meets the scheduling period; When it is determined that the conditions are met, the scheduled iterative task is triggered according to the preset training duration.

3. The method according to claim 1 or 2, wherein: After the current version of the risk control model is trained based on the hyperparameters and the latest risk business data is used as a training sample to obtain an iterated risk control model, the method further includes: The iterated risk control model is evaluated based on preset evaluation conditions, and when it is determined that the risk control model meets the evaluation conditions, the risk control model is put into online application.

4. According to the method of claim 3, the preset evaluation conditions include: At least one of a confidence threshold evaluation condition and an indicator curve threshold evaluation condition; In the case where the preset evaluation condition is the confidence threshold evaluation condition, the evaluating the iterated risk control model based on the preset evaluation condition includes: Determining the confidence of the risk control model after iteration, and determining whether the confidence meets the confidence threshold evaluation condition; and / or, In a case where the preset evaluation condition is the indicator curve threshold evaluation condition, the evaluating the iterated risk control model based on the preset evaluation condition includes: Determine an indicator curve of the risk control model after iteration, and determine whether the indicator curve meets the indicator curve threshold evaluation condition.

5. According to the method of any one of claims 3 or 4, the method further comprises: based on the hyperparameters, taking the latest risk business data as training samples, training the current version of the risk control model, and obtaining the iterative risk control model; then, evaluating the iterative risk control model based on preset evaluation conditions; and when it is determined that the risk control model meets the evaluation conditions, before the risk control model is put online for application, the method further comprises: According to the preset file parameters, the iterated risk control model is output as a risk control model file and saved to the model management center; The preset file parameters include: the file name of the model file, the label of the model file, the description information of the model file, and the version number of the model file; The step of evaluating the iterated risk control model based on a preset evaluation condition, and applying the risk control model online when it is determined that the risk control model meets the evaluation condition, includes: The iterated risk control model is evaluated at the model management center, and when it is determined that the risk control model meets the evaluation conditions, the risk control model is pushed to the online business system for going online.

6. The method according to any one of claims 2 to 5, wherein: Before setting the period of the scheduled iterative task, the method further includes: Create an initial risk control model and set initial hyperparameters; The step of creating an initial risk control model and setting initial hyperparameters includes: Determine multiple target operator modules from a preset operator module list; Arranging connection relationships between the multiple target operator modules; Setting operator hyperparameters of at least some of the target operator modules among the multiple target operator modules, and obtaining the initial hyperparameters based on the operator hyperparameters of at least some of the target operator modules; Among them, the first round of scheduled iterative tasks trains the initial risk control model based on the initial hyperparameters; the non-first round of scheduled iterative tasks trains the iterated risk control model based on the latest version of the hyperparameters and the previous round of scheduled iterative tasks.

7. The method according to any one of claims 1 to 6, wherein: Based on the hyperparameters, the latest risk business data is used as a training sample to train the current version of the risk control model to obtain an iterative risk control model, including: The training stop condition parameters are obtained from the hyperparameters, and the latest risk business data is used as a training sample to train the current version of the risk control model. When the risk control model in training meets the training stop condition parameters, the training is stopped to obtain the iterated risk control model.

8. The method according to any one of claims 1 to 7, wherein: Before triggering the scheduled iterative task according to the preset period and obtaining the preset hyperparameters and the latest risk business data corresponding to the scheduled iterative task of the current round, the method further includes: According to a preset data collection cycle, original data of risk business is collected, and the original data of risk business is preprocessed to obtain the risk business data, and the risk business data is processed into a feature wide table form.

9. An automatic iteration device for a risk control model, wherein: The device comprises: A timing trigger module is used to trigger the timing iteration task according to a preset period, and obtain the preset hyperparameters and the latest risk business data corresponding to the timing iteration task of the current round; The iteration module is used to train the current version of the risk control model based on the hyperparameters and take the latest risk business data as training samples to obtain an iterated risk control model.

10. The device according to claim 9, wherein: The device also includes: A cycle setting module, used to set the cycle of the scheduled iterative task, wherein the cycle includes: a scheduling cycle, a time cycle, and a task effective date; The scheduling period indicates the interval between two adjacent rounds of the scheduled iterative tasks; the time period indicates the training duration of a single scheduled iterative task; the task effective date indicates the effective triggering time of the scheduled iterative task; The timing trigger module is used, when the timing iteration task is triggered according to the preset period, to: Determining whether the current date satisfies the task effective date; If it is determined that the condition is met, determining whether the interval between the current time and the previous round of scheduled iterative tasks meets the scheduling period; When it is determined that the conditions are met, the scheduled iterative task is triggered according to the preset training duration.

11. The device according to claim 9 or 10, wherein: The device also includes: The evaluation module is used to evaluate the iterated risk control model based on preset evaluation conditions, and put the risk control model online for application when it is determined that the risk control model meets the evaluation conditions.

12. The device according to claim 11, wherein The preset evaluation condition includes: at least one of a confidence threshold evaluation condition and an indicator curve threshold evaluation condition; In the case where the preset evaluation condition is the confidence threshold evaluation condition, the evaluation module is used to: Determining the confidence of the risk control model after iteration, and determining whether the confidence meets the confidence threshold evaluation condition; and / or, In the case where the preset evaluation condition is the indicator curve threshold evaluation condition, the evaluation module is used to: Determine an indicator curve of the risk control model after iteration, and determine whether the indicator curve meets the indicator curve threshold evaluation condition.

13. The device according to claim 11 or 12, wherein: The device also includes: A file output module, used to output the iterated risk control model as a risk control model file according to preset file parameters, and save it to the model management center; The preset file parameters include: the file name of the model file, the label of the model file, the description information of the model file, and the version number of the model file; The evaluation module is specifically used for: The iterated risk control model is evaluated at the model management center, and when it is determined that the risk control model meets the evaluation conditions, the risk control model is pushed to the online business system for going online.

14. The device according to any one of claims 10 to 13, wherein: The device also includes: Model creation module, used to create the initial risk control model and set the initial hyperparameters; The model creation module is specifically used for: Determine multiple target operator modules from a preset operator module list; Arranging connection relationships between the multiple target operator modules; Setting operator hyperparameters of at least some of the target operator modules among the multiple target operator modules, and obtaining the initial hyperparameters based on the operator hyperparameters of at least some of the target operator modules; Among them, the first round of scheduled iterative tasks trains the initial risk control model based on the initial hyperparameters; the non-first round of scheduled iterative tasks trains the iterated risk control model based on the latest version of the hyperparameters and the previous round of scheduled iterative tasks.

15. The device according to any one of claims 9 to 14, wherein: The iteration module is specifically used for: The training stop condition parameters are obtained from the hyperparameters, and the latest risk business data is used as a training sample to train the current version of the risk control model. When the risk control model in training meets the training stop condition parameters, the training is stopped to obtain the iterated risk control model.

16. The device according to any one of claims 9 to 15, wherein: The device also includes: The data collection module is used to collect the original data of risk business according to the preset data collection cycle, pre-process the original data of risk business, obtain the risk business data, and process the risk business data into a feature wide table form.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images