Evaluation index determination method and device, storage medium and electronic device

A technology for evaluating indicators and determining methods, which is applied in the field of communication and can solve problems such as the accuracy reduction of the strategy analysis module

Pending Publication Date: 2021-04-02
光大科技有限公司
0 Cites 0 Cited by

AI-Extracted Technical Summary

Problems solved by technology

[0004] Embodiments of the present invention provide a method and device for determining evaluation indicators, a storage medium, and an electronic device, so as to at least solve the problems cau...
View more

Method used

As shown in Figure 5, further by the application information of the user in the application information form and the corresponding customer information recorded in the system, compare, and combine the loan information in the system to comprehensively analyze, ensure the stability of the monitoring scoring model .
By above-mentioned device, obtain the target data structure corresponding to the target object in the policy analysis module, wherein, the target data structure includes at least one of the following: the information of the target object, the application information of the target object, the collateral information of the target object . For the decision data information of the target object, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object; input the target data structure into the policy monitoring model to obtain the target data structure A target evaluation report, wherein the policy monitoring model is trained by machine learning using multiple sets of data, each set of data in the multiple sets of data includes: a target data structure, and a target evaluation report corresponding to the target data structure; Judging whether the target evaluation index in the target evaluation report reaches the expected index, to determine whether the decision-making meets the preset conditions, adopting the above-mentioned technical solution, solves the problems in the related technology, due to the economic environment, the customer group Various internal and external factors such as the change of data source and data source acquisition lead to problems such as the reduction of the accuracy of the strategy analysis module. The effectiveness of the strategy analysis module is analyzed through the strategy monitoring model to determine whether the decision made by the strategy analysis module is biased. Then adjust the analysis strategy of the decision analysis module, and improve the accuracy of the strategy analysis module in the case of changes in external conditions.
By above-mentioned steps, obtain the target data structure corresponding to the target object in the strategy analysis module, wherein, the target data structure includes at least one of the following: the information of the target object, the application information of the target object, the collateral information of the target object . For the decision data information of the target object, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object; input the target data structure into the policy monitoring model to obtain the target data structure A target evaluation report, wherein the policy monitoring model is trained by machine learning using multiple sets of data, each set of data in the multiple sets of data includes: a target data structure, and a target evaluation report corresponding to the target data structure; Judging whether the target evaluation index in the target evaluation report reaches the expected index, to determine whether the decision-making meets the preset conditions, adopting the above-mentioned technical solution, solves the problems in the related technology, due to the economic environment, the customer group Various internal and external factors such as the change of data source and data source acquisition lead to problems such as the reduction of the accuracy of the strategy analysis module. The effectiveness of the strategy analysis module is analyzed through the strategy monitoring model to determine whether the decision made by the strategy analysis module is biased. Then adjust the analysis strategy of the decision analysis module, and improve the accuracy of the strategy analysis module in the case of changes in external conditions.
Optionally, according to the characteristics of the consumer finance business, by determining the input variables, output variables and log data that the decision engine produces in the process of making risk decisions for the decision engine in the consumer finance risk control decision-making process, design the corresponding decision The data structure facilitates the storage and processing of data for follow-up risk strategy monitoring, and also lays the foundation for changes in user portraits, user asset quality analysis, and user behavior analysis during lending.
That is to say, because there may be a plurality of target evaluation indicators in the target data structure, but different target objects have different requirements for providing resources to it, a plurality of expectations corresponding to a plurality of target evaluation indicators corresponding to the acquisition target data structure After determining the threshold range of the indicators, further judgments are made on multiple target evaluation indicators. After confirming that all target evaluation indicators meet the threshold range of the expected indicators, it is confirmed that the decisions generate...
View more

Abstract

The invention provides an evaluation index determination method and device, a storage medium and an electronic device, and the method comprises the steps of obtaining a target data structure corresponding to a target object in a strategy analysis module, wherein the target data structure comprises at least one of information of a target object, application information of the target object and pledge information of the target object, and for decision data information of the target object, a decision generated by the strategy analysis module is used for indicating whether resources are providedfor the target object or not; inputting the target data structure into a strategy monitoring model to obtain a target evaluation report of the target data structure, the strategy monitoring model being trained by using multiple groups of data through machine learning, and each group of data in the multiple groups of data including the target data structure and the target evaluation report corresponding to the target data structure; and judging whether a target evaluation index in the target evaluation report reaches an expected index or not to determine whether the decision meets a preset condition or not.

Application Domain

FinanceResources +2

Technology Topic

Information strategiesData information +5

Image

  • Evaluation index determination method and device, storage medium and electronic device
  • Evaluation index determination method and device, storage medium and electronic device
  • Evaluation index determination method and device, storage medium and electronic device

Examples

  • Experimental program(1)

Example Embodiment

[0028] Hereinafter, the present invention will be described in detail with reference to the drawings and examples. It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other.
[0029] It should be noted that the terms "first" and "second" in the description and claims of the present invention and the above drawings are used to distinguish similar objects, but not necessarily used to describe a specific sequence or sequence.
[0030] The method embodiment provided in Embodiment 1 of the present application may be executed in a computer terminal or a similar computing device. Take running on a computer terminal as an example, figure 1It is a block diagram of the hardware structure of a computer terminal according to a method for determining an evaluation index in an embodiment of the present invention. like figure 1 As shown, the computer terminal may include one or more (only one is shown in the figure) processors 102 (the processors 102 may include but not limited to processing devices such as microprocessor MCU or programmable logic device FPGA, etc.), for storing Data storage 104, and transmission means 106 for communication functions. Those of ordinary skill in the art can understand that, figure 1 The shown structure is only for illustration, and it does not limit the structure of the above-mentioned electronic device. For example, a computer terminal may also include a figure 1 more or fewer components than shown in, or with figure 1 Different configurations are shown.
[0031] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for determining the evaluation index in the embodiment of the present invention, and the processor 102 runs the computer program stored in the memory 104, thereby Executing various functional applications and data processing is to realize the above-mentioned method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory that is remotely located relative to the processor 102, and these remote memories may be connected to a computer terminal through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.
[0032] The transmission device 106 is used to receive or transmit data via a network. The specific example of the above-mentioned network may include a wireless network provided by the communication provider of the computer terminal. In one example, the transmission device 106 includes a network interface controller (NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.
[0033] An embodiment of the present invention provides a method for determining an evaluation index, which is applied to the above-mentioned computer terminal, figure 2 is a flowchart of a method for determining an evaluation index according to an embodiment of the present invention, such as figure 2 As shown, the process includes the following steps:
[0034] Step S202, obtaining the target data structure corresponding to the target object, wherein the target data structure includes at least one of the following: target object information, target object application information, target object collateral information, decision result information for the target object ;
[0035] Step S204, input the target data structure into a policy analysis model to obtain target feature values ​​of the target data structure, wherein the policy analysis model is trained by using multiple sets of data through machine learning, and the multiple Each set of data in the set of data includes: a target data structure, and a target feature value corresponding to the target data structure;
[0036] Step S206, determine whether to provide resources for the target object according to the target feature value.
[0037] Through the above steps, the target data structure corresponding to the target object in the strategy analysis module is obtained, wherein the target data structure includes at least one of the following: information of the target object, application information of the target object, collateral information of the target object, The decision data information of the object, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object; the target data structure is input into the policy monitoring model to obtain the target evaluation report of the target data structure , wherein, the policy monitoring model is trained by using multiple sets of data through machine learning, each set of data in the multiple sets of data includes: a target data structure, and a target evaluation report corresponding to the target data structure; judging the Whether the target evaluation index in the target evaluation report reaches the expected index is to determine whether the decision-making meets the preset conditions. The above-mentioned technical solution is adopted to solve the problems in the related technology due to changes in the economic environment, customer groups and Various internal and external factors such as data source acquisition lead to problems such as reduced accuracy of the strategy analysis module. The effectiveness of the strategy analysis module is analyzed through the strategy monitoring model to determine whether the decision made by the strategy analysis module deviates from expectations, and then Adjust the analysis strategy of the decision analysis module to improve the accuracy of the strategy analysis module in the case of changes in external conditions.
[0038] Optionally, acquiring the target data structure corresponding to the target object in the policy analysis module includes: acquiring result information of the target object in the policy analysis module after executing a target event, wherein the target event is used to indicate that the target object successfully acquires resources record; acquire the target data structure corresponding to the target object from the result information.
[0039] In order to ensure that the obtained target data structure of the target object is more in line with the actual financial situation, when obtaining the target data structure, the result information of the target event that the target object successfully acquires resources can also be obtained, thereby ensuring the accuracy of the target data structure.
[0040] For example, to determine the user's settlement status and other information after the loan is successful, according to the user's repayment behavior, you can evaluate the user's historical overdue and current overdue situation. At this time, you can use the application number in the application information form and the application in the loan information form. The number is associated to realize the linkage analysis of decision-making data and credit repayment performance data, and then realize real-time monitoring, the number of application orders, the number of successful credit granting orders, the pass rate of credit granting, anti-fraud rejection, etc., the average credit line, and the hit of each strategy Combined with credit performance data, it conducts flexible and diversified analysis and tracking of credit repayment performance data to monitor the stability and accuracy of the scoring model.
[0041] Optionally, the policy monitoring model includes: a plurality of evaluation indicators, and the target data structure is input into the policy monitoring model to obtain a target evaluation report of the target data structure, including: according to the input target data Determine the evaluation strategy corresponding to the target data structure, and determine the target evaluation index corresponding to the evaluation strategy from a variety of target evaluation indicators; analyze the target data structure through the target evaluation index corresponding to the evaluation strategy; A target evaluation report of the target data structure is determined according to the analysis result.
[0042] In short, in order to ensure the pertinence of the policy monitoring model, the evaluation strategy of the target data structure can be confirmed by judging the data information in the target data structure in advance, and then the target evaluation index corresponding to the evaluation strategy can be determined from various target evaluation indicators Analyze the data information, and determine the target evaluation report of the target data structure according to the analysis results.
[0043] Optionally, the evaluation index includes at least one of the following: scorecard feature IV value, scorecard stability contribution index value, scorecard K-S statistical value, and scorecard Gini coefficient value.
[0044] Optionally, judging whether the target evaluation index in the target evaluation report reaches the expected index, so as to determine whether the decision meets the preset condition, includes: when there are multiple target evaluation indicators in the target evaluation report Acquiring threshold ranges of expected indicators corresponding to multiple target evaluation indicators; when multiple target evaluation indicators meet the threshold ranges of multiple expected indicators, determining a policy analysis module corresponding to the target evaluation report Meet the preset conditions, the decision generated by the strategy analysis module meets the requirements; when multiple target evaluation indicators do not meet the threshold range of the expected indicators, it is determined that the strategy analysis module corresponding to the target evaluation report does not meet the preset requirements condition, the decision generated by the policy analysis module does not meet the requirements, and the policy analysis module needs to be reset.
[0045] That is to say, since there may be multiple target evaluation indicators in the target data structure, but different target objects have different requirements for providing resources, when obtaining the thresholds of multiple expected indicators corresponding to multiple target evaluation indicators corresponding to the target data structure After determining the range of target evaluation indicators, make further judgments on multiple target evaluation indicators. After confirming that all target evaluation indicators meet the threshold range of the expected indicators, confirm that the decisions generated by the strategy analysis module meet the requirements and can be implemented smoothly without adjustment. When there is a target evaluation index that does not meet the threshold range of the expected index, it means that the decision generated by the policy analysis module does not meet the requirements. According to the actual situation, the policy analysis module needs to be reset. Analyze the effectiveness of the strategy analysis module through the strategy monitoring model, judge whether the decision made by the strategy analysis module deviates from expectations, and then adjust the analysis strategy of the decision analysis module to improve the accuracy of the strategy analysis module in the case of changes in external conditions sex.
[0046] Optionally, after judging whether the target evaluation index in the target evaluation report reaches the expected index to determine whether the strategy analysis module meets the preset condition, the method further includes: after confirming that the strategy analysis module meets the preset condition In the case of conditions, execute the decision generated by the strategy analysis module, and save the decision data information in the target evaluation report corresponding to the strategy analysis module.
[0047] In short, after determining that the strategy analysis module meets the preset conditions, that is, the accuracy of the strategy analysis module meets the application threshold, and the decision-making process does not deviate from expectations, further, according to the judgment result of the strategy analysis module's strategy , to determine whether to provide resources to the target object or not to provide resources to the target object, determine the decision-making data information corresponding to the target object according to the judgment result, and refer to it when providing resources to the target object to speed up the time interval of resource release, and also The multiple successful records of the target object can be saved, and it can be processed with priority when it is executed again, and the record of the number of times the policy analysis module is used with validity and the target evaluation report corresponding to the policy analysis module are stored in the database in one-to-one correspondence .
[0048] In order to better understand the determination process of the above evaluation index, the following description will be made in conjunction with optional embodiments, but it is not intended to limit the technical solutions of the embodiments of the present invention.
[0049] An optional embodiment of the present invention provides a consumer finance-oriented risk control strategy monitoring system, including: a strategy monitoring data structure module 32 and a strategy model monitoring and analysis module 34 . Among them, the policy monitoring data structure module 32 includes the information data (equivalent to the target data structure in the embodiment of the present invention) that the user (equivalent to the target object in the embodiment of the present invention) needs to collect when applying for consumer finance; The analysis module 34 is used to analyze the user information data stored in the policy monitoring data structure module 32 through the monitoring scoring model (equivalent to the policy monitoring model in the embodiment of the present invention), so as to realize the monitoring of the credit risk control strategy.
[0050] It should be noted that the detailed data structure in the policy monitoring data structure module 32 is as follows Figure 4 As shown, the application information stored in the policy monitoring data structure module 32 includes user information, collateral information, and decision result information; the decision result information includes scorecard return information and trigger rule information, and the scorecard return information is based on the corresponding characteristic variable information Corresponding conversion can be carried out; in addition, since the decision result will be affected by the derived variable, the decision result information will also include the derived variable information. The user information also includes user credit information, contact information, and fraud information, wherein the user credit information includes user loan information, user credit card information, and user inquiry information.
[0051] Optionally, according to the characteristics of the consumer finance business, design the data of the corresponding decision data by determining the input variables and output variables of the decision engine in the consumer finance risk control decision-making process and the log data generated by the decision engine in the process of making risk decisions The structure facilitates the storage and processing of data for follow-up risk strategy monitoring, and also lays the foundation for user profile changes, user asset quality analysis, and user behavior analysis during lending.
[0052]For example, in the process of risk decision-making, one application corresponds to one user, and one user corresponds to multiple credit records. Query records or handle multiple loan products internally. The borrower has multiple contacts. When the decision engine outputs, the loan application may hit multiple rules and similar one-to-many situations. At this time, it is necessary to establish a one-to-many The table-level association relationship of the risk decision-making supports the input, output, and derived variable data generated during the decision-making process. When the user makes the final loan through risk decision-making and the user signs the contract, the consumer financial service institution generates a repayment plan for each contract, including the repayment date, principal repayment, interest repayment, penalty interest repayment and synchronous repayment every day In the payment details, the actual repayment date, actual principal repayment, actual repayment of interest, actual repayment of penalty interest and reduction or exemption information, and determine the settlement status and other information. According to the user's repayment behavior, the user's historical overdue and current overdue situation can be evaluated. At this time, you can associate the application number in the application information form with the application number in the loan information form to realize the linkage analysis of decision data and credit repayment performance data, and then realize real-time monitoring, such as the number of application orders, the number of successful credit grants, the credit approval rate, and anti-fraud Rejection and other pass rates, average credit line, and the pass rate of the number of hits, passes, and rejections for each strategy, combined with credit performance data, conduct flexible and diversified analysis and track credit repayment performance data, and monitor the scoring model stability.
[0053] like Figure 5 As shown, the user's application information in the application information table is further compared with the corresponding customer information recorded in the system, and combined with the loan information in the system for comprehensive analysis to ensure the stability of the monitoring scoring model.
[0054] Optionally, use the system to monitor in real time the daily incoming shipments, approvals, approvals, disbursements, approval rate, approved amount, average approved pieces, disbursed amount, and disbursed pieces. The specific table structure is shown in the table 1 shows:
[0055] Table 1
[0056]
[0057] Optionally, the analysis of rule violations included in the decision result information can use the rule as a unit to count the number of people triggered by a single rule, and the statistics of the final review results corresponding to a single rule. The specific table structure can be shown in Table 2:
[0058] Table 2
[0059]
[0060] It should be noted that when the approval pass rate suddenly drops, business personnel can use the above monitoring report to analyze what causes the pass rate to drop, whether it is caused by the rejection rule, and which rule caused the pass rate to drop. Through the strategy analysis logic, the The proportion of rejected data distribution of each rule is compared and analyzed, and the rule with higher proportion is found, and then the variable corresponding to the rule is analyzed (equivalent to the target characteristic value in the embodiment of the present invention), and finally the risk control strategy is formulated.
[0061] Optionally, the monitoring and scoring model corresponds to the scorecard characteristic quantity IV (information value, information value referred to as IV) value analysis report, the characteristic quantity is the variable corresponding to each scorecard, and the goal of the scoring model is to distinguish good customers from bad customers and bad customers It is a non-target customer identified by the company, for example, the overdue number of historical overdue exceeds the set number of times. The scorecard feature quantity IV value analysis report is further represented by WOE (Weight Of Evidence, weight of evidence, referred to as WOE) coding. When calculating WOE coding, it is necessary to group according to the different variables. After grouping, for the i-th group , the calculation formula of WOE is as follows: py i is the proportion of defaulting customers in this group to all defaulting customers in the sample, pn i is the proportion of non-defaulting customers in this group to all non-defaulting customers in the sample. It can also be understood as the difference between the ratio of defaulted customers and non-defaulted customers in the current group and this ratio in all samples. The larger the WOE, the greater the difference, and the greater the possibility of default in the sample in the group. Further, the formula for determining the IV value according to the value of WOE is as follows: Then calculate the IV value of the entire variable, the formula is as follows: It is the addition of all IV values. In addition, because the IV value takes into account the influence of the number of variables in each group, it can comprehensively reflect the influence of each group level.
[0062] Optionally, the scorecard feature quantity IV value analysis report may be as shown in Table 3.
[0063] table 3
[0064]
[0065] Optionally, the monitoring scoring model corresponds to a score card as a stability analysis report, where PS (PopulationStability Index, PSI population stability index, referred to as PSI) reflects the distribution of verification samples in each score segment and the stability of the distribution of modeling samples . In modeling, we often use it to screen feature variables and evaluate model stability. Stability is referenced, so there need to be two distributions - the actual distribution (actual) and the expected distribution (expected). Among them, the training sample (In the Sample, INS) is usually used as the expected distribution during modeling, and the verification sample is usually used as the actual distribution. The verification samples generally include Out of Sample (OOS for short) and Out of Time (OOT for short) samples. The calculation formula of PSI is as follows: Among them: A represents the actual reality, E represents the expected expectation, the smaller the PSI value, the smaller the difference between the two distributions, and the more stable it is. The range of PSI values ​​and recommendations are shown in Table 4 below:
[0066] Table 4
[0067]
[0068] Optionally, the monitoring scoring model corresponds to the scoring card K-S statistical analysis report, and KS (Kolmogorov-smirnov, KS for short) is often used to evaluate the model discrimination. The greater the degree of discrimination, the stronger the ranking ability of the model. The definition between good accounts (good_rate) and bad accounts (bad_rate) is often vague and continuous, depending on actual business needs. KS indicators tend to start from The probability perspective measures the difference between positive and negative sample distributions. Generally speaking, the larger the KS, the better the distinction between positive and negative samples, expressed by the following formula:
[0069] KS=max{|cum(bad_rate)-cum(good_rate)|}
[0070] Optionally, the scorecard K-S statistical analysis report can be as shown in Table 5.
[0071] table 5
[0072] scoring model scoring interval number of good accounts bad debts unsure number of accounts Accumulate the number of accounts Cumulative number of bad accounts Cumulative Good Account Percentage Cumulative Bad Account Percentage solve coefficient K-S
[0073] Optionally, the monitoring and scoring model corresponds to the statistical analysis report of the scoring Kagini coefficient, GINI coefficient: it is also used to evaluate the risk discrimination ability of the model. The GINI statistic measures the area between the cumulative distribution of the number of bad accounts on the number of good accounts and the random distribution curve. The greater the difference between the distribution of good accounts and the distribution of bad accounts, the higher the GINI index, indicating that the risk discrimination ability of the model is stronger .
[0074] The calculation steps of the GINI coefficient are as follows: Calculate the number of good and bad accounts in each scoring interval. Calculate the ratio of cumulative good accounts to total good accounts (cumulative good%) and cumulative bad accounts to total bad accounts (cumulative bad%) for each scoring interval. According to the proportion of accumulated good accounts and the proportion of accumulated bad accounts, it can be obtained as follows: Image 6 ADC curve shown. Calculate the area of ​​the shaded part, and the percentage of the shaded area to the area of ​​the right triangle ABC is the GINI coefficient.
[0075] Optionally, the scorecard K-S statistical analysis report can be shown in Table 6.
[0076] Table 6
[0077] scoring model scoring interval number of good accounts bad debts unsure number of accounts Accumulate the number of accounts Cumulative number of bad accounts Cumulative Good Account Percentage Cumulative Bad Account Percentage solve coefficient Gini Coefficient
[0078] In an optional embodiment of the present invention, the intelligent risk control strategy requires continuous optimization, adjustment, iteration and update, while the complete consumer finance risk control strategy and model system can dynamically monitor the accuracy of the online and running strategies on user credit risk assessment in real time Through the analysis of operation strategy monitoring, data-driven business can be achieved, the risk control strategy can be continuously improved, and the purpose of promoting the differentiation of risk control management and the humanization of credit business can be achieved.
[0079] To sum up, from the perspective of risk business, the optional embodiment of the present invention uses unstructured log data in the risk process to establish multi-level log data such as application information, applicant information, credit information, decision results, scoring return results, and trigger rule information. The parent-child relationship table stores the input and output variables of the decision-making engine and the intermediate data generated during the decision-making process, and monitors the number of application orders, the number of successful credit granting orders, the pass rate of credit granting, the pass rate of anti-fraud rejection, and the average credit limit for risk control situation, and the number of hits, passes, and rejections of each strategy, as well as the combination of credit performance data, flexible and diversified analysis and credit repayment performance data tracking, monitoring the stability and effectiveness of the scoring model provides a Technical data and technical support, and provides approval monitoring, rejection reason analysis, score grade overdue distribution report, scorecard feature quantity IV value analysis report, scorecard stability analysis report, scorecard K-S statistics for risk control strategies and models Analysis and scoring Kagini coefficient statistical analysis reports provide optimization basis for strategy optimization and scoring models. Through the analysis of operation strategy model monitoring, data-driven business can be achieved, risk control strategies are constantly improved, and risk control management differentiation and credit are promoted. The purpose of business humanization makes it closer to the consumer finance business scenario. At the same time, in the process of actual risk strategy monitoring, it can achieve the purpose of real-time monitoring strategies and models, which can be directly implemented, and can automatically generate monitoring reports. At the same time, through Store all the data of the decision engine and realize the linkage analysis with the repayment performance data, which makes the analysis of customer portrait more complete and the risk assessment of credit customers more accurate.
[0080] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the essence of the technical solution of the present invention or the part that contributes to the prior art can be embodied in the form of software products, and the computer software products are stored in a storage medium (such as ROM/RAM, disk, CD) contains several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) execute the method of each embodiment of the present invention.
[0081] In this embodiment, a device for determining an evaluation index is also provided, and the device is used to implement the above embodiments and preferred implementation modes, and those that have been described will not be repeated here. As used below, the term "module" may be a combination of software and/or hardware that realizes a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementations in hardware, or a combination of software and hardware are also possible and contemplated.
[0082] Figure 7 is a structural block diagram of a device for determining an evaluation index according to an embodiment of the present invention, such as Figure 7 As shown, the device includes:
[0083] (1) The acquisition module 72 is used to acquire the target data structure corresponding to the target object in the policy analysis module, wherein the target data structure includes at least one of the following: information of the target object, application information of the target object, deposit of the target object Product information, decision data information for the target object, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object;
[0084] (2) An evaluation module 74, configured to input the target data structure into a strategy monitoring model to obtain a target evaluation report of the target data structure, wherein the strategy monitoring model is trained by machine learning using multiple sets of data Each set of data in the multiple sets of data includes: a target data structure, and a target evaluation report corresponding to the target data structure;
[0085] (3) A determining module 76, configured to judge whether the target evaluation index in the target evaluation report reaches the expected index, so as to determine whether the decision meets the preset condition.
[0086] Through the above-mentioned device, the target data structure corresponding to the target object in the policy analysis module is obtained, wherein the target data structure includes at least one of the following: information of the target object, application information of the target object, collateral information of the target object, The decision data information of the object, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object; the target data structure is input into the policy monitoring model to obtain the target evaluation report of the target data structure , wherein, the policy monitoring model is trained by using multiple sets of data through machine learning, each set of data in the multiple sets of data includes: a target data structure, and a target evaluation report corresponding to the target data structure; judging the Whether the target evaluation index in the target evaluation report reaches the expected index is to determine whether the decision-making meets the preset conditions. The above-mentioned technical solution is adopted to solve the problems in the related technology due to changes in the economic environment, customer groups and Various internal and external factors such as data source acquisition lead to problems such as reduced accuracy of the strategy analysis module. The effectiveness of the strategy analysis module is analyzed through the strategy monitoring model to determine whether the decision made by the strategy analysis module deviates from expectations, and then Adjust the analysis strategy of the decision analysis module to improve the accuracy of the strategy analysis module in the case of changes in external conditions.
[0087] Optionally, the acquisition module is further used for: the acquisition module is also used for: acquisition result information of the target object in the policy analysis module after executing the target event, wherein the target event is used to indicate that the target object has successfully acquired A resource record; obtaining the target data structure corresponding to the target object from the result information.
[0088] In order to ensure that the obtained target data structure of the target object is more in line with the actual financial situation, when obtaining the target data structure, the result information of the target event that the target object successfully acquires resources can also be obtained, thereby ensuring the accuracy of the target data structure.
[0089] For example, to determine the user's settlement status and other information after the loan is successful, according to the user's repayment behavior, you can evaluate the user's historical overdue and current overdue situation. At this time, you can use the application number in the application information form and the application in the loan information form. The number is associated to realize the linkage analysis of decision-making data and credit repayment performance data, and then realize real-time monitoring, the number of application orders, the number of successful credit granting orders, the pass rate of credit granting, anti-fraud rejection, etc., the average credit line, and the hit of each strategy Combined with credit performance data, it conducts flexible and diversified analysis and tracking of credit repayment performance data to monitor the stability and accuracy of the scoring model.
[0090] Optionally, the above-mentioned evaluation module further includes: a plurality of evaluation indicators, and is also used to determine the evaluation strategy corresponding to the target data structure according to the input target data structure, and determine the evaluation strategy from various target evaluation indicators Corresponding target evaluation index; analyze the target data structure through the target evaluation index corresponding to the evaluation strategy; determine the target evaluation report of the target data structure according to the analysis result.
[0091] In short, in order to ensure the pertinence of the policy monitoring model, the evaluation strategy of the target data structure can be confirmed by judging the data information in the target data structure in advance, and then the target evaluation index corresponding to the evaluation strategy can be determined from various target evaluation indicators Analyze the data information, and determine the target evaluation report of the target data structure according to the analysis results.
[0092] Optionally, the above-mentioned evaluation module includes at least one of the following: scorecard feature IV value, scorecard contribution index value for stability, scorecard K-S statistical value, and scorecard Gini coefficient value.
[0093] Optionally, the above-mentioned determination module is further configured to obtain the threshold range of the expected index corresponding to the multiple target evaluation indicators when there are multiple target evaluation indicators in the target evaluation report; When the target evaluation index satisfies the threshold range of multiple expected indicators, it is determined that the strategy analysis module corresponding to the target evaluation report meets the preset conditions, and the decision generated by the strategy analysis module meets the requirements; when multiple targets When the evaluation index does not meet the threshold range of the expected index, it is determined that the strategy analysis module corresponding to the target evaluation report does not meet the preset conditions, the decision generated by the strategy analysis module does not meet the requirements, and the strategy analysis module needs to be reset .
[0094] That is to say, since there may be multiple target evaluation indicators in the target data structure, but different target objects have different requirements for providing resources, when obtaining the thresholds of multiple expected indicators corresponding to multiple target evaluation indicators corresponding to the target data structure After determining the range of target evaluation indicators, make further judgments on multiple target evaluation indicators. After confirming that all target evaluation indicators meet the threshold range of the expected indicators, confirm that the decisions generated by the strategy analysis module meet the requirements and can be implemented smoothly without adjustment. When there is a target evaluation index that does not meet the threshold range of the expected index, it means that the decision generated by the policy analysis module does not meet the requirements. According to the actual situation, the policy analysis module needs to be reset. Analyze the effectiveness of the strategy analysis module through the strategy monitoring model, judge whether the decision made by the strategy analysis module deviates from expectations, and then adjust the analysis strategy of the decision analysis module to improve the accuracy of the strategy analysis module in the case of changes in external conditions sex.
[0095] Optionally, the above device further includes: a saving module, configured to execute the decision generated by the strategy analysis module when it is confirmed that the strategy analysis module meets the preset conditions, and save the decision data information in the strategy analysis module. In the target evaluation report corresponding to the module.
[0096] In short, after determining that the strategy analysis module meets the preset conditions, that is, the accuracy of the strategy analysis module meets the application threshold, and the decision-making process does not deviate from expectations, further, according to the judgment result of the strategy analysis module's strategy , to determine whether to provide resources to the target object or not to provide resources to the target object, determine the decision-making data information corresponding to the target object according to the judgment result, and refer to it when providing resources to the target object to speed up the time interval of resource release, and also The multiple successful records of the target object can be saved, and it can be processed with priority when it is executed again, and the record of the number of times the policy analysis module is used with validity and the target evaluation report corresponding to the policy analysis module are stored in the database in one-to-one correspondence .
[0097] It should be noted that the above-mentioned modules can be realized by software or hardware. For the latter, it can be realized by the following methods, but not limited to this: the above-mentioned modules are all located in the same processor; or, the above-mentioned modules can be combined in any combination The forms of are located in different processors.
[0098] An embodiment of the present invention also provides a storage medium, the storage medium includes a stored program, wherein the above-mentioned program executes any one of the above-mentioned methods when running.
[0099] Optionally, in this embodiment, the above-mentioned storage medium may be configured to store program codes for performing the following steps:
[0100] S1. Obtain the target data structure corresponding to the target object in the policy analysis module, wherein the target data structure includes at least one of the following: information about the target object, application information for the target object, collateral information for the target object, Decision data information, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object;
[0101] S2. Input the target data structure into a policy monitoring model to obtain a target evaluation report of the target data structure, wherein the policy monitoring model is trained by machine learning using multiple sets of data, and the multiple sets of Each set of data in the data includes: the target data structure, and the target evaluation report corresponding to the target data structure;
[0102] S3, judging whether the target evaluation index in the target evaluation report reaches the expected index, so as to determine whether the decision meets the preset condition.
[0103] An embodiment of the present invention also provides a storage medium, the storage medium includes a stored program, wherein the above-mentioned program executes any one of the above-mentioned methods when running.
[0104] An embodiment of the present invention also provides an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.
[0105] Optionally, the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.
[0106] Optionally, in this embodiment, the above-mentioned processor may be configured to execute the following steps through a computer program:
[0107] S1. Obtain the target data structure corresponding to the target object in the policy analysis module, wherein the target data structure includes at least one of the following: information about the target object, application information for the target object, collateral information for the target object, Decision data information, the decision generated by the policy analysis module is used to indicate whether to provide resources for the target object;
[0108] S2. Input the target data structure into a policy monitoring model to obtain a target evaluation report of the target data structure, wherein the policy monitoring model is trained by machine learning using multiple sets of data, and the multiple sets of Each set of data in the data includes: the target data structure, and the target evaluation report corresponding to the target data structure;
[0109] S3, judging whether the target evaluation index in the target evaluation report reaches the expected index, so as to determine whether the decision meets the preset condition.
[0110] Optionally, in this embodiment, the above-mentioned storage medium may include but not limited to: U disk, read-only memory (Read-Only Memory, ROM for short), random access memory (Random Access Memory, RAM for short), Various media that can store program codes such as removable hard disks, magnetic disks, or optical disks.
[0111] Optionally, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and optional implementation manners, and details are not repeated in this embodiment.
[0112] Obviously, those skilled in the art should understand that each module or each step of the above-mentioned present invention can be realized by a general-purpose computing device, and they can be concentrated on a single computing device, or distributed in a network formed by multiple computing devices Alternatively, they may be implemented in program code executable by a computing device so that they may be stored in a storage device to be executed by a computing device, and in some cases in an order different from that shown here The steps shown or described are carried out, or they are separately fabricated into individual integrated circuit modules, or multiple modules or steps among them are fabricated into a single integrated circuit module for implementation. As such, the present invention is not limited to any specific combination of hardware and software.
[0113] The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

PUM

no PUM

Description & Claims & Application Information

We can also present the details of the Description, Claims and Application information to help users get a comprehensive understanding of the technical details of the patent, such as background art, summary of invention, brief description of drawings, description of embodiments, and other original content. On the other hand, users can also determine the specific scope of protection of the technology through the list of claims; as well as understand the changes in the life cycle of the technology with the presentation of the patent timeline. Login to view more.
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products