Financial risk control model updating method and device, equipment and storage medium

Abstract

The invention discloses a financial risk control model updating method and device, equipment and a storage medium, and relates to the field of automatic modeling, and the method comprises the steps: detecting KL divergence and model accuracy before and after data change when financial risk control model training data change, and determining an analysis result according to a KL divergence change value and a model accuracy change value. If the analysis result shows that data drift exists, a target case set is selected from a financial risk control modeling case library according to the KL divergence change value and the model accuracy change value, then a target modeling scheme set is determined by means of a preset intelligent agent system, and the consistency of the target modeling scheme set is judged. And if the modeling schemes in the target modeling scheme set meet a preset consistency condition, voting the modeling schemes meeting the preset consistency condition, and finally selecting the target modeling scheme according to a preset scheme detection rule and a voting result to update the financial risk control model. Therefore, the updating timeliness of the financial risk control model can be improved.

Description

Technical Field

[0001] The present invention relates to the field of automated modeling, and in particular to a financial risk control model updating method, device, equipment and storage medium. Background Art

[0002] In today's financial technology field, financial risk control models play an extremely critical role. With the continuous expansion and innovation of financial business, financial data is changing rapidly. However, there are many defects in the existing financial risk control model update methods. On the one hand, its timeliness is seriously insufficient, and it is difficult to conduct real-time and reasonable analysis of rapidly changing data, resulting in the model's inability to adapt to new data characteristics and risk trends in a timely manner. On the other hand, over-reliance on expert intervention to rebuild the model will consume a lot of manpower, making the update efficiency of the financial risk control model low, and it is also easy to introduce errors due to human factors.

[0003] In summary, how to improve the timeliness of updating financial risk control models to accelerate the updating of financial risk control models is a technical problem that needs to be solved urgently. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a financial risk control model update method, device, equipment and storage medium, which can improve the timeliness of financial risk control model update to accelerate the update of financial risk control model. The specific scheme is as follows:

[0005] In a first aspect, the present application provides a method for updating a financial risk control model, comprising:

[0006] When the training data of the financial risk control model changes, the KL divergence and model accuracy before and after the data change are calculated respectively, and the analysis results are determined based on the obtained KL divergence change value and model accuracy change value;

[0007] When the analysis result indicates that data drift has occurred, a target case set is determined in the financial risk control modeling case library based on the KL divergence change value and the model accuracy change value;

[0008] Determine a target modeling solution set based on a preset intelligent agent system and the target case set, and judge the consistency of the target modeling solution set based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the modeling solution function call name in the target modeling solution set;

[0009] If the judgment result indicates that the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency condition, voting is performed on the modeling schemes that meet the preset consistency condition based on the preset intelligent agent system;

[0010] Based on the preset scheme detection rules and the obtained voting results, a target modeling scheme is determined among the modeling schemes that meet the preset consistency conditions, and the financial risk control model is updated using the target modeling scheme.

[0011] Optionally, when the training data of the financial risk control model changes, the KL divergence and the model accuracy before and after the data change are detected respectively, and the analysis result is determined based on the obtained KL divergence change value and the model accuracy change value, including:

[0012] Acquire the first training data feature before the training data changes, and determine the first target training data feature that meets the preset importance condition based on the financial risk control model;

[0013] Calculating based on the first target training data feature and a preset KL divergence calculation formula to obtain a first KL divergence before the data changes;

[0014] Acquire a second training data feature after the training data is changed, and determine a second target training data feature that meets the preset importance condition based on the financial risk control model;

[0015] Calculating based on the second target training data feature and the preset KL divergence calculation formula to obtain a second KL divergence after the data is changed;

[0016] Performing a difference calculation using the first KL divergence and the second KL divergence to obtain a KL divergence change value;

[0017] Determine the first model accuracy before the training data is changed and the second model accuracy after the training data is changed based on the number of correctly predicted positive samples, the number of incorrectly predicted positive samples, the number of correctly predicted negative samples, and the number of incorrectly predicted negative samples corresponding to the financial risk control model; wherein the positive samples are samples that match the target category, and the negative samples are samples that do not match the target category;

[0018] Performing a difference calculation using the first model accuracy and the second model accuracy to obtain a model accuracy change value;

[0019] An analysis result is determined based on the KL divergence change value and the model accuracy change value.

[0020] Optionally, determining a target case set in a financial risk control modeling case library based on the KL divergence change value and the model accuracy change value includes:

[0021] Determine a current drift situation based on the KL divergence change value and the model accuracy change value;

[0022] Calculate based on the current drift situation and the drift situation of each case in the financial risk control modeling case library to obtain the L2 distance; each case includes a drift situation, a tool list, and a solution example. The drift situation includes the KL divergence change value and the model accuracy change value corresponding to the case. The tool list is used to record the modeling tools corresponding to the case, and the solution example is the modeling solution corresponding to the case.

[0023] Determine whether the L2 distance meets a preset distance condition to obtain a distance determination result;

[0024] If the distance judgment result shows that the condition is satisfied, the case satisfying the preset distance condition is determined as the target case, and the target case is saved in the target case set.

[0025] Optionally, determining a target modeling solution set based on a preset intelligent agent system and the target case set includes:

[0026] Obtain target drift conditions, target solution instances, and a target tool list in the target case set;

[0027] Analyzing the target drift situation and the target solution instance based on a preset intelligent agent system to obtain an analysis result;

[0028] The target modeling tools in the target tool list are combined based on the analysis results and preset tool combination rules to determine a target modeling solution set based on the obtained combination results.

[0029] Optionally, judging the consistency of the target modeling solution set based on the preset intelligent agent system includes:

[0030] Based on the preset intelligent agent system, the function call name of each modeling scheme in the target modeling scheme set is judged for consistency to obtain a judgment result;

[0031] If the judgment result shows that the function call names of the modeling schemes that meet the preset consistency condition in the target modeling scheme set are all inconsistent, a corresponding scheme revision operation is triggered;

[0032] If the judgment result indicates that there is a modeling scheme that meets the preset consistency condition and has the same function call name in the target modeling scheme set, a voting operation for the modeling scheme that meets the preset consistency condition is triggered.

[0033] Optionally, determining a target modeling scheme from the modeling schemes that meet the preset consistency condition based on the preset scheme detection rule and the obtained voting result, and updating the financial risk control model using the target modeling scheme includes:

[0034] Determine a modeling scheme that meets a preset number of votes based on the obtained voting results;

[0035] Detecting whether the definition of the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset clarity condition to obtain a first detection result;

[0036] Detecting whether the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset executable condition to obtain a second detection result;

[0037] Detecting whether the calling order of the modeling functions corresponding to the modeling scheme that meets the preset number of votes condition meets the preset order calling condition, so as to obtain a third detection result;

[0038] If the first detection result, the second detection result, and the third detection result all indicate that the corresponding conditions are met, the modeling scheme that meets the preset number of votes condition is determined as the target modeling scheme;

[0039] The financial risk control model is updated using the target modeling solution, and the target modeling solution is saved in the financial risk control modeling case library.

[0040] Optionally, the financial risk control model updating method further includes:

[0041] If there is a test result among the first test result, the second test result and the third test result indicating that the corresponding condition is not met, a scheme revision operation corresponding to the modeling scheme that meets the preset number of votes condition is triggered.

[0042] In a second aspect, the present application provides a financial risk control model updating device, comprising:

[0043] A result determination module is used to calculate the KL divergence and model accuracy before and after the data change when the training data of the financial risk control model changes, and determine the analysis result based on the obtained KL divergence change value and model accuracy change value;

[0044] A case set determination module, configured to determine a target case set in a financial risk control modeling case library based on the KL divergence change value and the model accuracy change value when the analysis result indicates that data drift has occurred;

[0045] A consistency judgment module is used to determine a target modeling solution set based on a preset intelligent agent system and the target case set, and to judge the consistency of the target modeling solution set based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the modeling solution function call name in the target modeling solution set;

[0046] A scheme voting module, configured to vote on the modeling schemes that meet the preset consistency conditions based on the preset intelligent agent system if the judgment result indicates that the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency conditions;

[0047] A model updating module is used to determine a target modeling scheme from the modeling schemes that meet the preset consistency conditions based on preset scheme detection rules and the obtained voting results, and to update the financial risk control model using the target modeling scheme.

[0048] In a third aspect, the present application provides an electronic device, including:

[0049] Memory, used to store computer programs;

[0050] A processor is used to execute the computer program to implement the aforementioned financial risk control model updating method.

[0051] In a fourth aspect, the present application provides a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the aforementioned financial risk control model updating method is implemented.

[0052] In the present application, when the training data of the financial risk control model changes, the KL divergence and model accuracy before and after the data change are calculated respectively, and the analysis result is determined based on the obtained KL divergence change value and model accuracy change value; when the analysis result shows that data drift has occurred, the target case set is determined in the financial risk control modeling case library based on the KL divergence change value and the model accuracy change value; the target modeling scheme set is determined based on the preset intelligent agent system and the target case set, and the consistency of the target modeling scheme set is judged based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the function call name of the modeling scheme in the target modeling scheme set; if the judgment result shows that the consistency of the modeling scheme in the target modeling scheme set meets the preset consistency condition, the modeling scheme that meets the preset consistency condition is voted based on the preset intelligent agent system; the target modeling scheme is determined from the modeling schemes that meet the preset consistency condition based on the preset scheme detection rule and the obtained voting result, and the financial risk control model is updated using the target modeling scheme. As can be seen from the above, when the training data of the financial risk control model changes, this application detects the KL divergence and model accuracy before and after the data change, and determines the analysis results according to the KL divergence change value and the model accuracy change value. If the analysis results show that there is data drift, the target case set is selected from the financial risk control modeling case library according to the KL divergence change value and the model accuracy change value, and then the target modeling solution set is determined with the help of the preset intelligent agent system, and the consistency of the target modeling solution set is judged. If the modeling scheme in the target modeling solution set meets the preset consistency condition, the modeling scheme that meets the preset consistency condition is voted, and finally the target modeling scheme is selected according to the preset solution detection rules and the voting results to update the financial risk control model. In this way, the present application can improve the timeliness of the update of the financial risk control model, accelerate the update of the financial risk control model, and thus improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0054] Figure 1 A flow chart of a financial risk control model updating method disclosed in this application;

[0055] Figure 2 A specific schematic diagram of updating a financial risk control model disclosed in this application;

[0056] Figure 3 This is a schematic diagram of the structure of a financial risk control model updating device disclosed in this application;

[0057] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] There are many defects in the existing financial risk control model update method. On the one hand, its timeliness is seriously insufficient, and it is difficult to conduct real-time and reasonable analysis of rapidly changing data, resulting in the model being unable to adapt to new data characteristics and risk trends in a timely manner. On the other hand, over-reliance on expert intervention to rebuild the model will consume a lot of manpower, making the update efficiency of the financial risk control model low, and it is also easy to introduce errors due to human factors. To this end, the present application provides a financial risk control model update method, device, equipment and storage medium, which can improve the timeliness of the financial risk control model update.

[0060] See also Figure 1 As shown, an embodiment of the present invention discloses a financial risk control model updating method, including:

[0061] Step S11: When the training data of the financial risk control model changes, the KL divergence and model accuracy before and after the data change are calculated respectively, and the analysis results are determined based on the obtained KL divergence change value and model accuracy change value.

[0062] In this embodiment, when the training data of the financial risk control model changes, the training data and the financial risk control model can be subjected to black and white data drift detection. It should be noted that black data refers to fraudulent data. In actual financial activities, there are various fraudulent behaviors that intend to defraud financial institutions of funds and illegally obtain profits. The data records generated by these fraudulent behaviors constitute black data. On the contrary, white data is non-fraudulent data. This type of data reflects normal, legal and compliant financial transactions and business operations.

[0063] In a specific implementation, for the situation before the data changes, the first training data feature before the training data changes is obtained, and the first target training data feature that meets the preset importance condition is determined based on the financial risk control model. The training data features include but are not limited to the frequency of financial transactions, the size and distribution of transaction amounts, etc. After determining the first target training data feature, a calculation is performed based on the first target training data feature and a preset KL divergence (i.e., Kullback-Leibler, an asymmetric measurement method created by Solomon Kullback and Richard A. Leibler) calculation formula to obtain the first KL divergence before the data changes;

[0064] Similarly, for the situation after the data changes, the second training data feature after the training data changes is obtained, and the second target training data feature that meets the preset importance condition is determined based on the financial risk control model; after determining the second target training data feature, calculation is performed based on the second target training data feature and the preset KL divergence calculation formula to obtain the second KL divergence after the data changes; after obtaining the first KL divergence and the second KL divergence, the first KL divergence and the second KL divergence are further used to perform difference calculation to obtain the KL divergence change value.

[0065] In addition, this embodiment also needs to calculate the change value of the model accuracy (i.e., ACC, Accuracy). Specifically, based on the number of correctly predicted positive samples, the number of incorrectly predicted positive samples, the number of correctly predicted negative samples, and the number of incorrectly predicted negative samples corresponding to the financial risk control model, the first model accuracy before the training data changes and the second model accuracy after the training data changes are determined respectively. That is, by calculating the first model accuracy before the training data changes and the second model accuracy after the changes, the impact of data changes on model performance can be intuitively understood. Among them, positive samples are samples that match the target category, and negative samples are samples that do not match the target category.

[0066] Furthermore, in this embodiment, the difference between the first model accuracy and the second model accuracy is calculated to obtain a model accuracy change value, and finally an analysis result is determined based on the calculated KL divergence change value and the model accuracy change value. This analysis result will serve as an important reference for subsequent judgment on whether the model needs to be adjusted and what update strategy to adopt.

[0067] Step S12: When the analysis result indicates that data drift has occurred, a target case set is determined in the financial risk control modeling case library based on the KL divergence change value and the model accuracy change value.

[0068] After obtaining the above analysis results, it is determined whether the analysis results indicate that data drift has occurred. It is understandable that once data drift is detected, it means that the training data on which the financial risk control model relies has changed, which may affect the financial risk control model's ability to identify fraudulent behavior. Therefore, if the analysis results indicate that data drift has occurred, the current drift situation is determined based on the KL divergence change value and the model accuracy change value.

[0069] In this embodiment, the calculation is performed based on the current drift situation and the drift situation of each case in the financial risk control modeling case library to obtain the L2 (i.e., Euclidean Distance, a geometric distance measurement method) distance; wherein, the financial risk control modeling case library contains multiple cases, so the financial risk control modeling case library can provide certain solutions for the subsequent update of the financial risk control model, and it is not difficult to introduce that new effective modeling cases will also be continuously recorded in the financial risk control modeling case library. In addition, each case contains a drift situation, a tool list, and a solution example. The drift situation contains the KL divergence change value and the model accuracy change value corresponding to the case. The tool list is used to record the modeling tool corresponding to the case, and the solution example is the modeling solution corresponding to the case.

[0070] Further, in this embodiment, it is determined whether the L2 distance satisfies the preset distance condition to obtain a distance judgment result. If the distance judgment result indicates that it is satisfied, the case that satisfies the preset distance condition is determined as the target case, and the target case is saved in the target case set. It should be noted that the target case refers to a case in the financial risk control modeling case library that is similar to the current data drift situation. In other words, the target case is a similar case screened out based on the L2 distance. When the calculated L2 distance meets the preset distance condition, the corresponding case is determined as the target case. In other words, the L2 distance is a tool for screening target cases, and the target case is the result of L2 distance screening.

[0071] Step S13, determining a target modeling solution set based on a preset intelligent agent system and the target case set, and judging the consistency of the target modeling solution set based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the modeling solution function call name in the target modeling solution set.

[0072] In this embodiment, the target drift situation, target solution instance and target tool list in the target case set are obtained. Specifically, the target drift situation presents the change characteristics of the data in the form of specific KL divergence change values ​​and model accuracy change values; the target solution instance records in detail the key information such as the modeling process and parameters used in the past to deal with similar data drift, which is an important reference for building new solutions; and the target tool list records the various modeling tools used in these cases.

[0073] Afterwards, the target drift situation and the target solution instance are parsed based on the preset intelligent agent system to obtain the parsing result. After obtaining the parsing result, the target modeling tools in the target tool list are combined based on the parsing result and the preset tool combination rule to determine the target modeling solution set based on the obtained combination result. It should be further emphasized that each modeling solution in the target modeling solution set contains multiple tool calling methods, which are expressed in JSON format. Among them, the tool as a whole can be divided into feature automation amplification, feature automation filtering and feature modeling tuning tools. Specifically, the feature automation amplification tool performs feature amplification on different types of data in the data through defined rules to achieve the process of feature engineering after simulating experts based on knowledge analysis of raw data, such as logical judgment; feature automation filtering refines and filters data features to achieve the role of simulating experts filtering redundant features, such as corr (correlation) filtering; feature modeling tuning tools simulate the process of experts analyzing and tuning model parameters, such as model training, model tuning and model compression.

[0074] In addition, based on the preset intelligent agent system, the consistency of the function call name of each modeling scheme in the target modeling scheme set is judged to obtain a judgment result.

[0075] In a specific implementation, if the judgment result shows that the function call names of the modeling solutions that meet the preset consistency conditions in the target modeling solution set are all inconsistent, this means that the solutions lack commonality in the core logic of tool use and may have great uncertainty or irrationality, thus triggering the corresponding solution revision operation. That is, at this time, expert intervention is required to conduct corresponding situation analysis and solution revision.

[0076] In another specific implementation, if the judgment result indicates that there is a modeling scheme that meets the preset consistency condition and has the same function call name in the target modeling scheme set, a voting operation for the modeling scheme that meets the preset consistency condition is triggered.

[0077] Step S14: If the judgment result indicates that the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency condition, voting is performed on the modeling schemes that meet the preset consistency condition based on the preset intelligent agent system.

[0078] In this embodiment, when the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency condition, the modeling schemes that meet the preset consistency condition are voted through the preset intelligent agent system. It should be noted that the number of votes cast has a preset upper limit value and a preset lower limit value, and after the voting is completed, the preset intelligent agent system will return the voting result.

[0079] Step S15: determining a target modeling scheme from the modeling schemes that meet the preset consistency conditions based on the preset scheme detection rules and the obtained voting results, and updating the financial risk control model using the target modeling scheme.

[0080] In this embodiment, a modeling scheme that meets the preset number of votes condition is determined based on the obtained voting results. Specifically, it is detected whether the definition of the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset clarity condition to obtain a first detection result. Among them, the preset clarity condition may include checking whether the definition of the modeling function complies with the JSON specification and checking whether the definition of the modeling function contains all necessary fields. Next, it is detected whether the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset executable condition to obtain a second detection result. Among them, the preset executable condition may include checking whether the modeling function name is in the preset modeling function list, checking whether the modeling function provides all required parameters, and checking whether all parameter formats match the definition of the modeling function. Finally, it is detected whether the calling order of the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset order calling condition to obtain a third detection result. Among them, the preset order calling condition may include checking whether the calling order of the modeling function complies with the modeling process of the financial risk control model.

[0081] In a specific implementation, if the first test result, the second test result, and the third test result all indicate that the corresponding conditions are met, the modeling scheme that meets the preset number of votes condition is determined as the target modeling scheme. Then, the financial risk control model is updated using the target modeling scheme, and the target modeling scheme is saved in the financial risk control modeling case library. The operation of saving the target modeling scheme in the financial risk control modeling case library can provide reference cases for future model updates and optimizations, and enrich the resource reserves of the case library.

[0082] In another specific implementation, if there is a test result indicating that the corresponding conditions are not met among the first test result, the second test result, and the third test result, a scheme revision operation corresponding to the modeling scheme that meets the preset number of votes condition is triggered. That is, at this time, the preset intelligent agent system will prompt an error and prompt that expert intervention is required. After the expert intervenes and modifies the corresponding scheme, the corresponding new scheme is executed. The new scheme must be tested again by the preset scheme detection rules until all conditions are met. It can be determined as the target modeling scheme and applied to the update of the financial risk control model, and the new scheme is recorded in the financial risk control modeling case library.

[0083] As can be seen from the above, when the training data of the financial risk control model changes, the present application detects the KL divergence and model accuracy before and after the data change, and determines the analysis result according to the KL divergence change value and the model accuracy change value. If the analysis result shows that there is data drift, the target case set is selected from the financial risk control modeling case library according to the KL divergence change value and the model accuracy change value, and then the target modeling solution set is determined with the help of the preset intelligent agent system, and the consistency of the target modeling solution set is judged. If the modeling scheme in the target modeling solution set meets the preset consistency condition, the modeling scheme that meets the preset consistency condition is voted, and finally the target modeling solution is selected according to the preset solution detection rules and the voting results to update the financial risk control model. In this way, the present application can improve the timeliness of the update of the financial risk control model, accelerate the update of the financial risk control model, and thus improve the user experience.

[0084] Combine the following Figure 2 The schematic diagram shown specifically illustrates the technical solution of the embodiment of the present application.

[0085] Specifically, given the original data features and financial risk control models , where x1, x2, …, x z Indicates that there are z data features. Next, determine the training data features M (i.e., the first target training data features) that meet the top (i.e., the preset importance condition) k1 before the data change, where k1 is the number of training data that meet the preset importance condition, and , and then calculate according to M and the preset KL divergence calculation formula to obtain the KL divergence before the data change (that is, the first KL divergence). The preset KL divergence calculation formula is as follows:

[0086] ;

[0087] In the formula, P(x) represents the target distribution, Q(x) represents the distribution to be matched, and rank(x) represents the feature importance ranking of feature x. Represents the weight parameter.

[0088] Next, determine the top k1 features of the data feature importance after the data changes , and similarly Substitute it into the preset KL divergence calculation formula, and then calculate based on the KL divergence before the above data change and the KL divergence change formula to get the KL divergence change. The KL divergence change formula is as follows:

[0089] ;

[0090] In the formula, , and D KL represents the change value of KL divergence, Indicates the KL divergence value after the data changes.

[0091] At the same time, the accuracy of the model before the data change is obtained through the accuracy calculation formula, where the accuracy calculation formula is as follows:

[0092] ;

[0093] In the formula, TP is the number of correctly predicted positive samples, FN is the number of incorrectly predicted positive samples, FP is the number of samples predicted to be positive but actually negative, and TN is the number of correctly identified negative samples.

[0094] Afterwards, the model accuracy change value is calculated based on the model accuracy before the data change and the model accuracy after the data change, wherein the calculated model accuracy change value D ACC The formula is as follows:

[0095] ;

[0096] In the formula, Indicates the accuracy of the model after the data changes.

[0097] When financial anti-fraud data drifts, according to the data drift index Retrieve all similar cases (i.e. target cases) in the financial risk control modeling case library and save all similar cases to the similar case set (i.e., the target case set), where Indicates that there are m similar cases in the similar case set. In addition, the formula of the retrieval process is as follows:

[0098] ;

[0099] It should be emphasized that each case in the financial risk control modeling case library i Contains drift conditions, tool lists and solution examples, so c i It can be expressed as ,in, represents the drift in the case, Represents the corresponding tool list, which represents a specific solution example (also known as a solution example). By calculating the L2 distance between the current drift situation and the drift situation of each case in the case set, similar cases are selected in the financial risk control modeling case library and a similar case set is formed. For example, according to the L2 distance, the topk2 cases with the smallest distance are selected to form a similar case set. . Where k2 represents the number of similar cases, and the calculation formula of L2 distance is as follows:

[0100] ;

[0101] In the formula, is the divergence change for each case, The change in model accuracy for each case.

[0102] Subsequently, a set of similar financial anti-fraud modeling cases is input into the intelligent agent (i.e., the preset intelligent agent system) to generate n modeling schemes (i.e., the target modeling scheme set). The formula for generating n modeling schemes is as follows:

[0103] ;

[0104] Where LLM (Large Language Model) is the large language model in the intelligent agent. There are n modeling schemes.

[0105] Specifically, the intelligent agent will analyze all drift situations and specific solution examples based on the input similar financial anti-fraud modeling case set, and then sort all tool lists according to the analysis results. The tools in the above code are combined to generate corresponding n modeling schemes.

[0106] Furthermore, the JSON format of a similar financial anti-fraud modeling case can be as follows:

[0107] {

[0108] "Vector Drift Indicator": "...",

[0109] "Tool List": [{

[0110] Function name: Logical judgment expansion,

[0111] Description: Perform logical judgment and amplification on special data.

[0112] parameter:{

[0113] Type: Dictionary,

[0114] Parameter content: {

[0115] Logical judgment group: {

[0116] Type: Dictionary,

[0117] Description: data type and corresponding logical judgment rule group}},

[0118] Required parameters: [Logical judgment group]}

[0119] },

[0120] {

[0121] Function name: model training,

[0122] Description: Train financial anti-fraud models,

[0123] parameter:{

[0124] Type: Dictionary,

[0125] Parameter content: {training set: {type: DataFrame,

[0126] Description: Model training data},

[0127] Validation set: {type: DataFrame,

[0128] Description: Model training data},

[0129] Random number seed: {type: int,

[0130] Description: Random number seed for the model}},

[0131] Required parameters: [training set, validation set]}

[0132] }]

[0133] "Solution Example":[

[0134] Logical judgment expansion ({double:["special_number","special_event_num"]}),

[0135] Model training (training set, validation set, random number seed)]

[0136] }

[0137] For generating n modeling scenarios The JSON format can be as follows:

[0138] {

[0139] "Scheme 1": [

[0140] Logical judgment expansion ({double:["special_number"]}),

[0141] Model training (training set, validation set, random number seed)]

[0142] "Scheme 2": [...],

[0143] ...,

[0144] "Scheme n": [...]

[0145] }

[0146] Regarding the generation of n modeling scenarios, the input and output of the intelligent agent can be as follows:

[0147] enter:

[0148] Output:

[0149] Afterwards, the intelligent agent will perform function call consistency judgment on the n generated solutions. The intelligent agent will compare whether the function call names in the two solutions are exactly the same. If at least one set of solution function call names are exactly the same, it means that the consistency is passed; if all n modeling solutions are inconsistent, it means that the intelligent agent has not found a suitable solution, which means that the agent cannot generate a new modeling solution based on similar cases in the financial anti-fraud modeling case library. The system will prompt an error, and experts are required to intervene to analyze the situation and revise the solution. If the error prompt is set to err, then the input and output of the intelligent agent for function call consistency judgment on n solutions can be as follows:

[0150] enter:

[0151] Output:

[0152]

[0153] When the consistency judgment of the intelligent agent's solution is passed, the group of solutions will be voted on. The intelligent agent will vote on the solutions and select the solution with the most votes (that is, the modeling solution that meets the preset consistency conditions) as the final financial risk control modeling solution.

[0154] Specifically, input n solutions to the intelligent agent , the intelligent agent votes for each solution. Each solution can cast [0,10] votes, with a maximum of 10 votes and a minimum of 0 votes. Finally, the solution with the most votes is selected. The formula for selecting the solution with the most votes is as follows:

[0155] ;

[0156] In the formula, It represents voting and returns the solution with the most votes, and Ea represents the final automated financial anti-fraud modeling solution.

[0157] In addition, for selecting the solution with the most votes, the input and output of the intelligent agent can be as follows:

[0158] enter: ;

[0159] Output: ;

[0160] In addition, in order to ensure the feasibility of the financial anti-fraud modeling solution, rule constraints (that is, preset solution detection rules) are added for automatic detection and expert intervention, and rule detection is performed on all tool call functions in the solution with the most votes. The solution is evaluated from two key aspects: the clarity of the financial anti-fraud modeling function definition and the executableness of the financial anti-fraud modeling function call.

[0161] When the rule constraint detection is compliant, the corresponding financial anti-fraud modeling function is directly called to execute the qualified solution (that is, the target modeling solution) of the rule constraint detection, so as to achieve the effect of automated modeling, and the qualified solution of the rule constraint detection is recorded in the financial risk control modeling case library. On the contrary, when the rule constraint detects that the solution is not compliant, the system will prompt an error and prompt that expert intervention is required. After the expert intervenes to modify the corresponding solution, the corresponding solution is executed and the new solution is recorded in the financial risk control modeling case library.

[0162] Accordingly, see Figure 3 As shown, the embodiment of the present application provides a financial risk control model updating device, including:

[0163] A result determination module 11 is used to calculate the KL divergence and model accuracy before and after the data change when the training data of the financial risk control model changes, and determine the analysis result based on the obtained KL divergence change value and model accuracy change value;

[0164] A case set determination module 12, configured to determine a target case set in a financial risk control modeling case library based on the KL divergence change value and the model accuracy change value when the analysis result indicates that data drift has occurred;

[0165] The consistency judgment module 13 is used to determine the target modeling solution set based on the preset intelligent agent system and the target case set, and judge the consistency of the target modeling solution set based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the modeling solution function call name in the target modeling solution set;

[0166] A scheme voting module 14 is used to vote on the modeling schemes that meet the preset consistency conditions based on the preset intelligent agent system if the judgment result shows that the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency conditions;

[0167] The model updating module 15 is used to determine a target modeling scheme from the modeling schemes that meet the preset consistency conditions based on preset scheme detection rules and the obtained voting results, and to update the financial risk control model using the target modeling scheme.

[0168] As can be seen from the above, when the training data of the financial risk control model changes, this application detects the KL divergence and model accuracy before and after the data change, and determines the analysis results according to the KL divergence change value and the model accuracy change value. If the analysis results show that there is data drift, the target case set is selected from the financial risk control modeling case library according to the KL divergence change value and the model accuracy change value, and then the target modeling solution set is determined with the help of the preset intelligent agent system, and the consistency of the target modeling solution set is judged. If the modeling scheme in the target modeling solution set meets the preset consistency condition, the modeling scheme that meets the preset consistency condition is voted, and finally the target modeling scheme is selected according to the preset solution detection rules and the voting results to update the financial risk control model. In this way, the present application can improve the timeliness of the update of the financial risk control model, accelerate the update of the financial risk control model, and thus improve the user experience.

[0169] In a specific implementation, the result determination module 11 specifically includes:

[0170] A first feature determination unit, configured to obtain a first training data feature before the training data is changed, and determine a first target training data feature that meets a preset importance condition based on a financial risk control model;

[0171] a first divergence determination unit, configured to perform calculation based on the first target training data feature and a preset KL divergence calculation formula to obtain a first KL divergence before the data changes;

[0172] A second feature determination unit, configured to obtain a second training data feature after the training data is changed, and determine a second target training data feature that meets the preset importance condition based on the financial risk control model;

[0173] A second divergence determination unit, configured to perform calculation based on the second target training data feature and the preset KL divergence calculation formula to obtain a second KL divergence after the data is changed;

[0174] a divergence change determining unit, configured to perform a difference calculation using the first KL divergence and the second KL divergence to obtain a KL divergence change value;

[0175] An accuracy determination unit, used to determine the first model accuracy before the training data is changed and the second model accuracy after the training data is changed, respectively, based on the number of correctly predicted positive samples, the number of incorrectly predicted positive samples, the number of correctly predicted negative samples, and the number of incorrectly predicted negative samples corresponding to the financial risk control model; wherein the positive samples are samples that match the target category, and the negative samples are samples that do not match the target category;

[0176] an accuracy change determining unit, configured to perform a difference calculation using the first model accuracy and the second model accuracy to obtain a model accuracy change value;

[0177] A result determination unit is used to determine an analysis result based on the KL divergence change value and the model accuracy change value.

[0178] In a specific implementation, the case set determination module 12 specifically includes:

[0179] a drift determination unit, configured to determine a current drift situation based on the KL divergence change value and the model accuracy change value;

[0180] A distance calculation unit, used to calculate based on the current drift situation and the drift situation of each case in the financial risk control modeling case library to obtain the L2 distance; wherein each case includes a drift situation, a tool list and a solution example, the drift situation includes the KL divergence change value and the model accuracy change value corresponding to the case, the tool list is used to record the modeling tool corresponding to the case, and the solution example is the modeling solution corresponding to the case;

[0181] A distance judgment unit, used to judge whether the L2 distance meets a preset distance condition to obtain a distance judgment result;

[0182] The case determination unit is used to determine the case that meets the preset distance condition as the target case if the distance judgment result shows that it is met, and save the target case into the target case set.

[0183] In a specific implementation, the consistency judgment module 13 specifically includes:

[0184] An information acquisition unit, used to acquire target drift conditions, target solution instances, and a target tool list in the target case set;

[0185] An information parsing unit, used for parsing the target drift situation and the target solution instance based on a preset intelligent agent system to obtain a parsing result;

[0186] A solution set determination unit is used to combine the target modeling tools in the target tool list based on the analysis result and the preset tool combination rule, so as to determine the target modeling solution set based on the obtained combination result.

[0187] In a specific implementation, the consistency judgment module 13 specifically includes:

[0188] A consistency judgment unit, used for performing consistency judgment on the function call name of each modeling scheme in the target modeling scheme set based on the preset intelligent agent system to obtain a judgment result;

[0189] A first scheme revision unit, configured to trigger a corresponding scheme revision operation if the judgment result indicates that the function call names of the modeling schemes satisfying the preset consistency condition in the target modeling scheme set are all inconsistent;

[0190] A scheme voting unit is used to trigger a voting operation on the modeling scheme that meets the preset consistency condition if the judgment result shows that there is a modeling scheme that meets the preset consistency condition and has the same function call name in the target modeling scheme set.

[0191] In a specific implementation, the model updating module 15 specifically includes:

[0192] A first scheme determination unit, configured to determine a modeling scheme that meets a preset number of votes based on the obtained voting results;

[0193] A first detection unit, used to detect whether the definition of the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset clarity condition, so as to obtain a first detection result;

[0194] A second detection unit, used to detect whether the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset executable condition, so as to obtain a second detection result;

[0195] A third detection unit, used to detect whether the calling order of the modeling functions corresponding to the modeling scheme that meets the preset number of votes condition meets the preset order calling condition, so as to obtain a third detection result;

[0196] A second scheme determination unit, configured to determine the modeling scheme that meets the preset number of votes condition as a target modeling scheme if the first detection result, the second detection result, and the third detection result all indicate that corresponding conditions are met;

[0197] A solution saving unit is used to update the financial risk control model using the target modeling solution and save the target modeling solution in the financial risk control modeling case library.

[0198] In a specific implementation, the model updating module 15 specifically includes:

[0199] The second scheme revision unit is used to trigger a scheme revision operation corresponding to the modeling scheme that meets the preset number of votes condition if there is a test result among the first test result, the second test result and the third test result that indicates that the corresponding condition is not met.

[0200] Furthermore, the present application also discloses an electronic device. Figure 4 It is a structural diagram of an electronic device 20 according to an exemplary embodiment, and the content in the figure cannot be regarded as any limitation on the scope of use of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. Among them, the memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to implement the relevant steps in the financial risk control model update method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0201] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0202] In addition, the memory 22 as a carrier for resource storage may be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon may include an operating system 221, a computer program 222, etc., and the storage method may be temporary storage or permanent storage.

[0203] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the financial risk control model update method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0204] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the financial risk control model updating method disclosed above is implemented. The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, and will not be repeated here.

[0205] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0206] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0207] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0208] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0209] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A financial risk control model updating method, characterized in that: include: When the training data of the financial risk control model changes, the KL divergence and model accuracy before and after the data change are calculated respectively, and the analysis results are determined based on the obtained KL divergence change value and model accuracy change value; When the analysis result indicates that data drift has occurred, determining a target case set in the financial risk control modeling case library based on the KL divergence change value and the model accuracy change value; Determine a target modeling solution set based on a preset intelligent agent system and the target case set, and judge the consistency of the target modeling solution set based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the modeling solution function call name in the target modeling solution set; If the judgment result indicates that the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency condition, voting is performed on the modeling schemes that meet the preset consistency condition based on the preset intelligent agent system; Based on the preset scheme detection rules and the obtained voting results, a target modeling scheme is determined among the modeling schemes that meet the preset consistency conditions, and the financial risk control model is updated using the target modeling scheme.

2. The financial risk control model updating method according to claim 1, characterized in that: When the training data of the financial risk control model changes, the KL divergence and model accuracy before and after the data change are detected respectively, and the analysis results are determined based on the obtained KL divergence change value and model accuracy change value, including: Acquire the first training data feature before the training data changes, and determine the first target training data feature that meets the preset importance condition based on the financial risk control model; Calculating based on the first target training data feature and a preset KL divergence calculation formula to obtain a first KL divergence before the data changes; Acquire a second training data feature after the training data is changed, and determine a second target training data feature that meets the preset importance condition based on the financial risk control model; Calculating based on the second target training data feature and the preset KL divergence calculation formula to obtain a second KL divergence after the data is changed; Performing a difference calculation using the first KL divergence and the second KL divergence to obtain a KL divergence change value; Determine the first model accuracy before the training data is changed and the second model accuracy after the training data is changed based on the number of correctly predicted positive samples, the number of incorrectly predicted positive samples, the number of correctly predicted negative samples, and the number of incorrectly predicted negative samples corresponding to the financial risk control model; wherein the positive samples are samples that match the target category, and the negative samples are samples that do not match the target category; Performing a difference calculation using the first model accuracy and the second model accuracy to obtain a model accuracy change value; An analysis result is determined based on the KL divergence change value and the model accuracy change value.

3. The financial risk control model updating method according to claim 1, characterized in that: The determining of a target case set in a financial risk control modeling case library based on the KL divergence change value and the model accuracy change value includes: Determine a current drift situation based on the KL divergence change value and the model accuracy change value; Calculate based on the current drift situation and the drift situation of each case in the financial risk control modeling case library to obtain the L2 distance; each case includes a drift situation, a tool list, and a solution example. The drift situation includes the KL divergence change value and the model accuracy change value corresponding to the case. The tool list is used to record the modeling tools corresponding to the case, and the solution example is the modeling solution corresponding to the case. Determine whether the L2 distance meets a preset distance condition to obtain a distance determination result; If the distance judgment result shows that the condition is satisfied, the case satisfying the preset distance condition is determined as the target case, and the target case is saved in the target case set.

4. The financial risk control model updating method according to claim 3, characterized in that: The determining of the target modeling solution set based on the preset intelligent agent system and the target case set includes: Obtain target drift conditions, target solution instances, and a target tool list in the target case set; Analyzing the target drift situation and the target solution instance based on a preset intelligent agent system to obtain an analysis result; The target modeling tools in the target tool list are combined based on the analysis results and preset tool combination rules to determine a target modeling solution set based on the obtained combination results.

5. The financial risk control model updating method according to claim 1, characterized in that: The judging of the consistency of the target modeling scheme set based on the preset intelligent agent system includes: Based on the preset intelligent agent system, the function call name of each modeling scheme in the target modeling scheme set is judged for consistency to obtain a judgment result; If the judgment result shows that the function call names of the modeling schemes that meet the preset consistency condition in the target modeling scheme set are all inconsistent, a corresponding scheme revision operation is triggered; If the judgment result indicates that there is a modeling scheme that meets the preset consistency condition and has the same function call name in the target modeling scheme set, a voting operation for the modeling scheme that meets the preset consistency condition is triggered.

6. The financial risk control model updating method according to any one of claims 1 to 5, characterized in that: The method of determining a target modeling scheme from the modeling schemes that meet the preset consistency conditions based on the preset scheme detection rules and the obtained voting results, and updating the financial risk control model using the target modeling scheme, includes: Determine a modeling scheme that meets a preset number of votes based on the obtained voting results; Detecting whether the definition of the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset clarity condition to obtain a first detection result; Detecting whether the modeling function corresponding to the modeling scheme that meets the preset number of votes condition meets the preset executable condition to obtain a second detection result; Detecting whether the calling order of the modeling functions corresponding to the modeling scheme that meets the preset number of votes condition meets the preset order calling condition, so as to obtain a third detection result; If the first detection result, the second detection result, and the third detection result all indicate that corresponding conditions are met, the modeling scheme that meets the preset number of votes condition is determined as the target modeling scheme; The financial risk control model is updated using the target modeling solution, and the target modeling solution is saved in the financial risk control modeling case library.

7. The financial risk control model updating method according to claim 6, characterized in that: Also includes: If there is a test result among the first test result, the second test result and the third test result indicating that the corresponding condition is not met, a scheme revision operation corresponding to the modeling scheme that meets the preset number of votes condition is triggered.

8. A financial risk control model updating device, characterized in that: include: A result determination module is used to calculate the KL divergence and model accuracy before and after the data change when the training data of the financial risk control model changes, and determine the analysis result based on the obtained KL divergence change value and model accuracy change value; A case set determination module, configured to determine a target case set in a financial risk control modeling case library based on the KL divergence change value and the model accuracy change value when the analysis result indicates that data drift has occurred; A consistency judgment module is used to determine a target modeling solution set based on a preset intelligent agent system and the target case set, and to judge the consistency of the target modeling solution set based on the preset intelligent agent system to obtain a judgment result; wherein the preset intelligent agent system is a system including a preset large language model and a preset logical architecture, and the consistency is the consistency of the modeling solution function call name in the target modeling solution set; A scheme voting module, configured to vote on the modeling schemes that meet the preset consistency conditions based on the preset intelligent agent system if the judgment result indicates that the consistency of the modeling schemes in the target modeling scheme set meets the preset consistency conditions; A model updating module is used to determine a target modeling scheme from the modeling schemes that meet the preset consistency conditions based on preset scheme detection rules and the obtained voting results, and to update the financial risk control model using the target modeling scheme.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, used to execute the computer program to implement the financial risk control model updating method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the financial risk control model updating method as described in any one of claims 1 to 7 is implemented.

Images