A method, device, and storage medium for predicting resource transfer results

Abstract

Embodiments of the present application disclose a method, apparatus, and storage medium for predicting resource transfer results. Embodiments of the present application can obtain resource transfer objects labeled with credit tags and annotation reference information of the credit tags, where the credit tags indicate the credit types of the resource transfer objects; determine classification rules corresponding to the credit tags based on the annotation reference information; determine an association level associated with the credit tags and the resource transfer results based on the prior information of the resource transfer and the classification rules of the credit tags; calculate a credibility parameter representing whether the association between the credit tags and the resource transfer results is credible based on the sample resource transfer objects labeled with the resource transfer results and the classification rules of the credit tags; perform availability verification on the credit tags based on the association level, the credibility parameter, and a preset parameter threshold; and when the availability verification passes, use the credit tags to predict the resource transfer results. Thereby, the accuracy of the prediction results of the resource transfer results can be improved.

Description

Technical Field

[0001] This application relates to the technical field of information processing, and particularly to a method, device, and storage medium for predicting resource transfer results. Background Art

[0002] In recent years, in order to describe a user's profile, information tags can be generated based on the user's information, and these information tags can be used in scenarios such as predicting user behavior and making item recommendations to users. Among them, credit tags can be used to predict the result of a user's resource transfer and control the resource transfer according to the prediction result to avoid risks. For example, credit tags can be used to review a user's loan project. These tags are developed by developers based on a large number of samples and relevant theoretical knowledge of resource transfer. To determine whether these tags can effectively predict the result of resource transfer, it is necessary to verify these tags before applying them. Currently, it is usually necessary to manually review and verify the tags, and the review criteria for manual review are chaotic and have a large degree of uncertainty. Therefore, the current method cannot accurately verify whether the credit tags are effective, thus cannot accurately predict the result of resource transfer, and ultimately cannot effectively avoid the risk of resource transfer. Summary of the Invention

[0003] In view of this, the embodiments of this application provide a method, device, and storage medium for predicting resource transfer results, which can improve the accuracy of predicting resource transfer results.

[0004] On the one hand, the embodiments of this application provide a method for predicting resource transfer results, including:

[0005] Obtain a resource transfer object labeled with a credit tag and the annotation reference information of the credit tag, where the credit tag indicates the credit type of the resource transfer object;

[0006] Determine the classification rule corresponding to the credit tag;

[0007] Based on the prior information of the resource transfer and the classification rule of the credit tag, determine the association level associated with the credit tag and the resource transfer result;

[0008] Based on the sample resource transfer object labeled with the resource transfer result and the classification rule of the credit tag, calculate a credibility parameter representing whether the association between the credit tag and the resource transfer result is credible;

[0009] Based on the association level, the credibility parameter, and a preset parameter threshold, perform availability verification on credit tags with different classification rules;

[0010] When the availability verification passes, use the credit tag labeled on the resource transfer object to predict the resource transfer result.

[0011] On the one hand, an embodiment of the present application provides a resource transfer result prediction device, including:

[0012] An acquisition unit, configured to acquire a resource transfer object labeled with a credit label and annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object;

[0013] A determination unit, configured to determine a classification rule corresponding to the credit label;

[0014] An association evaluation unit, configured to determine an association level associated with the credit label and the resource transfer result based on the prior information of the resource transfer and the classification rule of the credit label;

[0015] A parameter calculation unit, configured to calculate a credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible based on a sample resource transfer object labeled with a resource transfer result and the classification rule of the credit label;

[0016] An availability verification unit, configured to perform availability verification on credit labels with different classification rules based on the association level, the credibility parameter, and a preset parameter threshold;

[0017] A prediction unit, configured to, when the availability verification passes, use the credit label labeled on the resource transfer object to predict the resource transfer result.

[0018] In one embodiment, the parameter calculation unit may specifically include:

[0019] A division subunit, configured to divide the credit label into at least two level labels according to the classification rule and a preset label division node;

[0020] A calculation subunit, configured to determine a credibility parameter corresponding to the level label according to the sample resource transfer object and the level label;

[0021] A determination subunit, configured to use the credibility parameter corresponding to the level label as the credibility parameter of the credit label.

[0022] In one embodiment, the resource transfer result prediction device further includes an annotation verification unit, configured to perform annotation verification based on the annotation reference information of the credit label.

[0023] In one embodiment, the annotation verification unit may specifically include:

[0024] A basic information verification subunit, configured to perform basic information verification according to basic setting information in the annotation reference information and a preset first condition;

[0025] A sample verification subunit, configured to perform sample verification according to the modeling sample statistical parameters and a preset second condition in the annotation reference information when the basic information verification is passed.

[0026] A backtracking verification subunit, configured to perform backtracking verification according to the backtracking result and a preset third condition in the annotation reference information when the sample verification is passed.

[0027] A determination subunit, configured to determine that the credit label passes the initial verification when the basic information verification, the sample verification, and the backtracking verification are all passed.

[0028] In one embodiment, the resource transfer result prediction device further includes a stability verification unit, configured to perform stability verification on the credit labels of different classification rules according to the annotation reference information.

[0029] In one embodiment, the stability verification unit may specifically include:

[0030] A stability parameter calculation unit, configured to obtain the stability parameter of the credit label according to the modeling sample and the classification rule in the annotation reference information.

[0031] A stability verification subunit, configured to perform stability verification on the credit label according to the stability parameter and a preset stable parameter threshold.

[0032] On the one hand, an embodiment of the present application further provides a terminal, including a memory storing multiple instructions; the processor loads the instructions from the memory to execute the steps in any one of the resource transfer result prediction methods provided by the embodiments of the present invention.

[0033] On the other hand, a storage medium provided by an embodiment of the present application stores multiple instructions, and the instructions are used to store a computer program, and when the computer program runs on a computer, the computer is enabled to execute the resource transfer result prediction method provided by any embodiment of the present application.

[0034] Embodiments of the present application can obtain a resource transfer object labeled with a credit label and annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object; determine a classification rule corresponding to the credit label based on the annotation reference information; determine an association level associated with the resource transfer result based on the prior information of the resource transfer and the classification rule of the credit label; calculate a credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible based on a sample resource transfer object labeled with the resource transfer result and the classification rule of the credit label; perform availability verification on credit labels with different classification rules based on the association level, the credibility parameter, and a preset parameter threshold; when the availability verification passes, use the credit label labeled on the resource transfer object to predict the resource transfer result. In this solution, different methods are required for availability verification according to the classification rules of the credit label, and when performing availability verification, in addition to qualitatively determining the association between the credit label and the resource transfer result according to the prior information, the availability verification also quantitatively describes this association according to the credibility parameter, so that the effectiveness of the credit label can be verified more accurately and quickly. Using the credit label verified in this way to predict resource transfer can improve the accuracy of the prediction result of the resource transfer result. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention, and those skilled in the art can obtain other drawings without creative efforts based on these drawings.

[0036] Figure 1 FIG. is a schematic diagram of an application scenario of the resource transfer result prediction method provided by the embodiments of the present application.

[0037] Figure 2a FIG. is a flowchart of the resource transfer result prediction method provided by the embodiments of the present application.

[0038] Figure 2b FIG. is another flowchart of the resource transfer result prediction method provided by the embodiments of the present application. Figure 2c FIG. is a third flowchart of the resource transfer result prediction method provided by the embodiments of the present application Figure 3a FIG. is a first structural diagram of the resource transfer result prediction method provided by the embodiments of the present application.

[0039] Figure 3b FIG. is a second structural diagram of the resource transfer result prediction method provided by the embodiments of the present application.

[0040] Figure 4 It is a schematic diagram of the terminal provided by an embodiment of the present application.

[0041] Figure 5a It is a schematic diagram of a verification idea provided by an embodiment of the present application.

[0042] Figure 5b It is a schematic diagram of the annotation verification process provided by an embodiment of the present application.

[0043] Figure 5c It is a schematic diagram of the application level provided by an embodiment of the present application.

[0044] Figure 5d It is a schematic diagram of a stability verification process provided by an embodiment of the present application. Detailed implementation manners

[0045] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0046] An embodiment of the present invention provides a resource transfer result prediction method, device, and storage medium.

[0047] An embodiment of the present invention provides a resource transfer result prediction system, including the resource transfer result prediction device provided by any one of the embodiments of the present invention. The resource transfer result prediction device can be specifically integrated in a terminal, and the terminal can include: a mobile phone, a tablet computer, a notebook computer, or a personal computer (PC, Personal Computer), etc.

[0048] In addition, the resource transfer result prediction system may further include other devices, such as a server, etc.

[0049] For example, referring to Figure 1 , the resource transfer result prediction system includes a terminal and a server, and the terminal is linked to the server through a network. Among them, the network includes network entities such as routers and gateways. In another embodiment, the resource transfer result prediction system may further include other terminals, and the terminal can be connected to other terminals through a network.

[0050] Among them, the terminal can obtain a resource transfer object labeled with a credit label, and annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object.

[0051] Among them, the terminal can directly retrieve the resource transfer object labeled with a credit label and the annotation reference information from local storage or a removable storage device (such as a USB flash drive). In another embodiment, the terminal can also retrieve the resource transfer object labeled with a credit label and the annotation reference information from a server or other terminals through a network link.

[0052] The terminal can also obtain the resource transfer object labeled with a credit label and the annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object; determine the classification rule corresponding to the information label based on the annotation reference information; determine the association level associated with the resource transfer result based on the prior information of the resource transfer and the classification rule; calculate a credibility parameter representing whether the label is associated with the resource transfer result based on the sample resource transfer object and the classification rule for annotating the resource transfer result; verify the usability of the label based on the association level, the credibility parameter, and the annotation parameter information. When the verification passes, use the credit label annotated on the resource transfer object to predict the resource transfer result.

[0053] In one embodiment, the terminal can also send the above-predicted resource transfer result (hereinafter referred to as the prediction result for short) to the server. The server can save the prediction result or forward the prediction result to other terminals. For example, the prediction result can be forwarded to the terminal of the recommendation system so that the recommendation system can provide a processing recommendation for the resource transfer of the above object to the user based on the prediction result. For example, when the above object applies for a loan, the recommendation system can generate an approval recommendation for the loan application of the above object based on the prediction result and feedback it to the staff approving the loan, thereby avoiding the default risk of the loan project and ensuring profitability.

[0054] The above Figure 1 example is only an example of a system architecture for implementing the embodiments of the present invention. The embodiments of the present invention are not limited to the Figure 1 system structure shown above. Based on this system architecture, various embodiments of the present invention are proposed.

[0055] The following will be described in detail respectively. It should be noted that the serial numbers of the following embodiments do not limit the preferred order of the embodiments.

[0056] This embodiment will be described from the perspective of a resource transfer result prediction device, which can be specifically integrated in a terminal, and the terminal can be a mobile phone, a tablet computer, a notebook computer, or a personal computer (PC, Personal Computer), etc.

[0057] Embodiment 1

[0058] As Figure 2aAs shown, a method for predicting resource transfer results is provided. This method can be executed by a processor of a terminal. The specific process of this data method is as follows:

[0059] 101. Obtain a resource transfer object labeled with a credit label and the annotation reference information of the credit label.

[0060] Among them, resource transfer refers to the transfer of resources such as goods and funds, and specifically can include business forms such as loans, transactions, and savings. In the embodiments of the present application, the resource transfer business form will be taken as an example of a loan for detailed description. The specific process of predicting resource transfer for other business forms is also the same.

[0061] Among them, the resource transfer object (hereinafter can be simply referred to as the object) refers to the participating entity in the resource transfer, including banks, enterprises, individuals, etc. In the embodiments of the present application, the object will be taken as an example of the party applying for a loan for detailed description. The specific process of predicting other resource transfer objects is also the same.

[0062] Among them, the credit label indicates the credit type of the resource transfer object. The credit type can be divided into credit anomaly and credit normal according to the conclusion. Of course, the credit type can also be represented by credit anomaly parameters. The following will be described in combination with the form of the credit label, and will not be elaborated here. Among them, the credit label is an information label that can be used to predict the resource transfer result, and the information label is a label used to describe object information and is a concise expression form of information.

[0063] In one embodiment, since the purpose of predicting the loan execution result is mainly to prevent loan defaults, in order to unify the influence direction of the same credit label on the prediction result, it can be considered that the credit label is a label used to represent the credit anomaly situation. The prediction result of the loan is true when the loan defaults, and false when the loan prediction result is loan performance.

[0064] In one embodiment, the resource transfer object labeled with a credit label is provided by the label developer. In another embodiment, the label developer can also provide an algorithm model for annotating the credit label, and the terminal uses this algorithm model to annotate the above object.

[0065] Among them, the annotation reference information is information provided by the developer related to the annotation of the credit label. The specific information included in the annotation reference information can be seen in the following embodiments and will not be elaborated here.

[0066] In one embodiment, the steps for the label developer to label the above-mentioned credit label for the above-mentioned object may specifically include: The label developer selects, according to the business requirements of resource transfer and experience and common sense, a credit label that can be used to predict the result of resource transfer from the information labels, and then obtains a modeling sample from the data source. The modeling sample includes a training sample and a validation sample. Among them, the training sample includes multiple objects without labeled labels, and the validation sample includes multiple objects labeled with true credit labels. Then, use the training sample in the modeling sample to train an algorithm model, and use this algorithm model to label the resource transfer object to obtain a resource transfer object labeled with a credit label. Among them, common algorithm models include forms such as logistic regression, decision tree, and machine learning.

[0067] In another embodiment, the label developer can also train an algorithm model that can correctly label ordinary information labels. After the labeling is completed, then select a credit label that can be used to predict the result of resource transfer from the information labels.

[0068] In addition, it is also possible that the label developer does not train an algorithm model, but directly determines the credit label according to simple rules and performs labeling.

[0069] 102. Determine the classification rules corresponding to the credit label.

[0070] Among them, according to different classification rules, resource transfer objects can be divided into multiple different credit types, and each credit type obtained by classification is labeled with a credit label.

[0071] In this application, the above classification rules are mainly divided into two types. One is to classify according to one credit indicator, and the credit label can be expressed as a conclusion about this credit indicator or a characteristic parameter of the credit indicator; the other is to perform comprehensive classification according to multiple credit indicators, and this credit label can be expressed as a resource transfer result or a prediction probability of the resource transfer result. When classifying according to one credit label, the classification results can include two types. One is that the classification results include two credit labels, and the other is that the classification results include more than two credit labels. The credit label can be expressed as different forms of the credit indicator. In summary, the classification rules can actually be divided into three types. Correspondingly, the obtained credit labels correspond to three different classification rules.

[0072] According to the classification rules corresponding to the credit label, the credit label can have different forms of expression. For example, the credit indicator can be expressed as "the situation of participating in gambling". When classifying only according to this one credit indicator, if the classification result is two credit labels, the credit label can be expressed as "participating in gambling" and "not participating in gambling" two types. If the classification result is more than three credit labels, the credit label can be expressed as "mild gambling", "severe gambling" and "not participating in gambling" three types.

[0073] It should be noted that credit labels can be presented not only in the form of abnormal credit conclusions such as "mild gambling", "participation in gambling", and "loan default as a result of resource transfer", but also in the form of abnormal credit parameters. For example, "the probability of mild gambling is 10%", "the conclusion score of loan default as a result of resource transfer is xx", "the number of gambling times is 10", etc.

[0074] According to the above description, the classification rules can include three types. The first type is to divide according to one credit indicator to obtain two credit labels, which is called the first classification rule; the second type is to divide according to one credit indicator to obtain three or more credit labels, which is called the second classification rule; the third type is to divide according to multiple credit indicators to obtain the resource transfer result or its predicted probability, which is called the third classification rule.

[0075] In the embodiments of the present application, the classification rules are provided by label developers and are included in the annotation reference information. The terminal can determine the classification rules corresponding to the credit labels from the annotation reference information. Of course, the annotation reference information may not include the classification rules, and the terminal can determine the classification rules corresponding to the credit labels according to the manifestation form of the credit labels. For example, if the credit label is presented as "mild gambling", its corresponding classification rule can be determined as the second classification rule; if the credit label is presented as "predicted result is default" or the probability of default, its corresponding classification rule can be determined as the third classification rule.

[0076] It should be noted that the credit labels in each classification rule can have different manifestation forms. According to the classification rules and the manifestation forms of the credit labels, the credit labels can be divided into the following types:

[0077] The binary classification type, that is, the labels of yes and no, directly defines whether the object's credit is abnormal. For example, whether to participate in gambling, whether to belong to the blacklist, etc.

[0078] The multi-classification type, where the credit labels are finite or infinite, is a hierarchical description of the abnormal credit situation. For example, mild gambling, severe gambling, etc.

[0079] The single-model classification type refers to the labels developed using a single model. Here, "single" means that the credit indicator is single, that is, the model is only used to predict whether a single credit indicator of the object is abnormal. For example, the gambling model predicts the probability of a user gambling. After the credit labels of the single-model classification type are divided, they are actually multi-classification labels. Because of the difference in the manifestation form from the credit labels of the multi-classification type, they are distinguished.

[0080] The classification type of the comprehensive model, different from the classification type of the single model, lies in that the modeling samples of the comprehensive model classification type are generally loan default samples, and the credit label is expressed as the probability of the user's comprehensive default. The comprehensive model calculates the credit score of the object based on multiple credit indicators, and determines the loan execution result or the corresponding probability of the result according to the credit score as the credit label.

[0081] The classification type of credit anomaly parameters does not directly define whether a user has credit anomalies, but focuses on the parameter description of a certain credit indicator, such as the monthly number of gambling transactions and the monthly gambling amount. Credit anomaly parameters are generally processed by converting them into binary or multi-classification types according to the division nodes. However, when the label developer cannot give the division nodes, or after converting them into binary or multi-classification types, verification and prediction cannot be carried out, the feature parameters can be directly output.

[0082] Among them, the binary classification type belongs to the first classification rule, the comprehensive model classification type belongs to the third classification rule, and the remaining types belong to the second classification rule.

[0083] 103. Determine the association level associated with the resource transfer result between the credit label based on the prior information and classification rules of the resource transfer.

[0084] Among them, the prior information refers to the information summarized from the business form of resource transfer, based on the experience and common sense of the evaluators, with the predicted resource transfer result of the overall population of resource transfer objects as the evaluation criterion.

[0085] Among them, the association level is used to represent the degree of association between the credit label and the resource transfer result.

[0086] In one embodiment, the association level may include the following levels:

[0087] A. High association: High correlation with the resource transfer result; B. Relatively high association: Relatively high correlation with the resource transfer result; C. Medium association: General correlation with the resource transfer result; D. Low association: Low correlation with the resource transfer result; E. No association: No correlation with the resource transfer result.

[0088] In one embodiment, to determine the association level according to prior information and classification rules, the following principles can be followed: 1. If the classification rule is the first classification rule, the association level is determined according to the credit nature of the credit label and the business form of resource transfer. For example, when the business form of the resource transfer result is a loan, taking the credit label of whether to participate in gambling as an example, participating in gambling is a generally recognized credit-abnormal behavior. Therefore, it can be considered that the association level between participating in gambling and loan default is a high association; 2. If the classification rule is the second classification rule, first, according to the business nature of the credit label, determine its upper limit of the association level, and then calibrate an appropriate qualitative level according to the value. Taking the "gambling level" as an example, first, the credit nature of gambling is relatively bad. Referring to the association level of the first classification rule, its upper limit of the association level is a high association. Then, the lower the gambling level, the lower the association level. Suppose gambling is divided into 3 levels in total, and the greater the level, the worse the credit. Then, level 3 can be determined as a high association, level 2 as a relatively high association, and level 1 as a medium association; 3. If the classification rule is the third classification rule, since the label describes the resource transfer result, there is no need to evaluate the association level between the credit label and the resource transfer result, and they must be relevant.

[0089] In one embodiment, the second classification rule also includes a special form, namely the credit anomaly parameter classification type, and its credit label is manifested as the characteristic parameter of the credit index. For example, the credit label can be manifested as "the number of gambling times". Since the characteristic parameter (such as "the number of gambling times") is more inaccurate in describing the resource transfer object and indicates more possibilities of the credit index conclusion (such as "mild gambling") compared with the credit index conclusion, it is more difficult to determine its association level. When obtaining prior information, in order to reduce the excessive dependence on business experience, data analysis can be supplemented. Suppose that users with more than 10 transactions account for 1%, users with 5-10 transactions account for 3%, and users with less than 5 transactions account for 96%. Then, it can be set that more than 10 transactions are determined as a high association, 5-10 transactions as a relatively high association, and less than 5 transactions as a medium association.

[0090] Based on the prior information of the resource transfer and the classification rules, determine the association level associated between the credit label and the resource transfer result, which can qualitatively describe the correlation between the credit label and the resource transfer result. According to the prior information and classification rules, the correlation between the credit label and the resource transfer result can be determined, and the degree of correlation, that is, the association level, can be further determined. This step is essentially to qualitatively verify the availability of the credit label of different classification rules according to the prior information, the business form of the resource transfer, and the credit meaning of the credit label.

[0091] 104. Calculate a credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible based on the sample resource transfer object with the labeled resource transfer result and the classification rules of the credit label.

[0092] Among them, the credibility parameter is a parameter used to characterize whether the association between the credit label and the resource transfer result is credible, and may include forms such as coverage statistical parameters, prediction gain parameters, prediction coincidence rates, etc. Using samples to calculate the credibility parameter to quantitatively verify the availability of the credit label can more accurately verify the availability of the credit label.

[0093] In one embodiment, the step of "calculating a credibility parameter characterizing whether the label is associated with the resource transfer result based on the sample resource transfer object annotated with the resource transfer result and the classification rule of the credit label" may include the following steps:

[0094] Dividing the credit label into at least two level labels according to the classification rule and the preset label division node;

[0095] Determining the credibility parameter corresponding to the level label according to the sample resource transfer object and the level label;

[0096] Taking the credibility parameter corresponding to the level label as the credibility parameter of the credit label.

[0097] Among them, the sample resource transfer object is an object annotated with the true resource transfer result. In the loan scenario, it refers to the loan object with the known loan execution result stored in the bank or credit guarantee company, and the relevant information of these loan objects is also known. Among them, the loan execution result can be manifested as loan default and loan performance.

[0098] Among them, the preset label division node is the basis for re-dividing the credit label into level labels. For different classification rules, the values and meanings of the label division nodes are different. For specific details, please refer to the following embodiments and will not be elaborated here.

[0099] Among them, the level label is the label obtained after re-division, and the manifestation form of the level label is the credit label of the first classification rule.

[0100] In one embodiment, dividing the credit label into at least two level labels can be carried out according to the following criteria:

[0101] When the classification rule is the first classification rule, taking the corresponding credit label as two level labels;

[0102] When the classification rule is the second classification rule, dividing the multi-class credit label into at least three level labels according to the preset label division node, and the manifestation form of the level label is the label of the first classification rule;

[0103] When the classification rule is the third classification rule, obtain the label parameters corresponding to the credit label from the annotation reference information;

[0104] According to the label parameters and the preset label division nodes, divide the credit label into at least two level labels, and the manifestation form of the level label is the label of the first classification rule.

[0105] Among them, the label parameter is used to represent the possibility corresponding to the predicted resource transfer result. The label parameter can be in the form of a score or a probability. The larger the score, the more likely the corresponding resource transfer result is considered to occur, and the score can be mapped to the corresponding probability.

[0106] The following will combine the label division nodes and take "gambling" as an example to specifically illustrate the division process of the level labels: If the classification rule is the first classification rule, the credit labels "participate in gambling" and "do not participate in gambling" are used as the level labels; If the classification rule is the second classification rule and the credit labels are "mild gambling", "severe gambling" and "do not participate in gambling", "severe gambling" and "mild gambling" can both be divided into the level label of "participate in gambling", and "do not participate in gambling" is divided into the level label of "do not participate in gambling", thus obtaining 3 level labels; If the classification rule is the third classification rule and the credit labels are "loan default" and "loan performance", they can be divided according to the scores corresponding to the credit labels into multiple level labels.

[0107] Among them, the label division nodes can be determined in combination with the previous association levels. For example, when the association level is high association, the label division nodes can be set at a position with a relatively low degree of credit abnormality; when the association level is low availability, the label division nodes can be set at a position with a relatively high degree of credit abnormality.

[0108] In an embodiment, the determining the credible parameter corresponding to the level label according to the sample resource transfer object and the level label specifically may include the following steps:

[0109] Based on the level label, annotate the sample resource transfer object to obtain an object annotated with the level label as the label coverage object;

[0110] Based on the true result annotated in the sample resource transfer object and the predicted result of the label coverage object, determine the credible parameter of the level label of different classification rules for the predicted result;

[0111] Take the credible parameter of the level label and the statistical parameter of the label coverage object as the credible parameter corresponding to the level label.

[0112] Among them, the credible parameters include the prediction gain parameter, the prediction coincidence rate, the default rate of the approved population, etc. The prediction gain parameter refers to the accuracy improvement parameter of the prediction result using this credit label compared to the prediction result without using this credit label (which can include the improvement amount or the improvement rate). The prediction coincidence rate refers to the coincidence ratio of the resource transfer result obtained by predicting with this credit label and the resource transfer result obtained by predicting without using this credit label. The default rate of the approved population refers to the ratio of the people who actually default (do not repay the loan as agreed) among the population allowed to take loans.

[0113] Among them, the default rate of the approved population is to ensure that the loan project will not result in losses. In one embodiment, the default rate of the approved population can be calculated based on the income / loss of the loan project. The principle is that when the default rate of the approved population is within a certain threshold, the loan project can make a profit.

[0114] In one embodiment, the default rate of the approved population can be specifically calculated in the following manner:

[0115] Let the default rate be Pf, the credit default rate be Pe, the utilization rate of default amount be Uf, the utilization rate of credit default amount be Uc, the utilization rate of the credit of normal users be Un, the interest rate be R, and the non-borrowing ratio be Q. Then the default loss Lg = Pf × Uf, the credit loss Lc = Pc × U, and the income G = (1 - Pf - Pc - Q) × Un × R. Then the income / loss = G / (Lf + Lc). According to the target to be adjusted, for example, to control the default rate Pf within a reasonable range, the corresponding values can be substituted into the formula at this time. If the product is to make a profit, the result should be greater than 1, and Pf that meets the conditions can be obtained.

[0116] Among them, the statistical parameter of the label coverage object (which can be called the coverage statistical parameter) is a parameter used to represent the coverage degree of this credit label on the sample resource transfer object, and can be expressed in forms such as the coverage rate and the coverage quantity. Among them, the meaning of coverage is that the object is marked with this credit label, that is, this object can be described by this credit label.

[0117] 105. Based on the association level, the credible parameters, and the preset parameter threshold, perform availability verification on the credit labels of different classification rules.

[0118] Among them, the availability verification refers to verifying whether the credit label can effectively predict the resource transfer result.

[0119] The credible parameters and preset parameter thresholds selected for credit labels under different classification rules may vary. In one embodiment, when the classification rules are the first classification rule and the second classification rule, verification is mainly performed based on coverage statistical parameters, prediction gain parameters, and correlation levels, and the prediction coincidence rate and prediction gain parameters are combined to assist in verifying some unclear situations. When the classification rule is the third classification rule, verification is mainly performed through coverage statistical parameters and population default rates, and is combined with prediction gain parameters, rejection rates, pass rates, and ks (Kolmogorov-Smirnov) parameters for verification. Among them, the rejection rate refers to the proportion of people whose loans are rejected, the pass rate refers to the proportion of people who are allowed to take loans, and the ks parameter is an indicator used to evaluate the risk discrimination ability of the model, which measures the difference between the cumulative distributions of good and bad samples. Among them, good samples refer to samples with actual performance, and bad samples refer to samples with actual defaults.

[0120] Among them, each credible parameter corresponds to one or two preset parameter thresholds. By comparing the preset parameter thresholds with the credible parameters, the usability of the credit label is verified.

[0121] In one embodiment, after the step of "verifying the usability of credit labels for different classification rules based on the prior information of the resource transfer and the sample resource transfer objects labeled with true results", the following steps are further included:

[0122] When the usability verification fails, the credit label is updated according to the credible parameter, the correlation level, and the preset parameter threshold.

[0123] Among them, the update can be performed by the label developer or by the terminal in this system. In one embodiment, the label developer can update the parameters in the algorithm model and train the algorithm model according to the credible parameter, the correlation level, and the preset parameter threshold, so as to continuously optimize the algorithm model, and use the optimized algorithm model to label the credit label to achieve the update of the credit label.

[0124] 106. When the usability verification passes, the credit label labeled on the resource transfer object is used to predict the resource transfer result.

[0125] In one embodiment, after the credit label passes the availability verification, the level label with the optimal performance of the verification parameter can be selected for prediction according to the above availability verification result and the risk control severity. For example, when comparing the labels of the third classification rule, on the premise that the cumulative prediction gain parameter ≥ 1, if the risk control is strict, the level label with the largest coverage rate is selected; if it is loose, the level label with the largest KS is selected. If there is no available level label, the direct use of this label is temporarily abandoned and re-optimized; for the credit label of the second classification rule, if the risk control requirement is strict, the threshold with the largest coverage rate is selected; if the risk control requirement is loose, the threshold with the largest prediction gain parameter is selected. If there is no level label that meets the threshold requirement, the direct use of this label is temporarily abandoned, and it is added to the label combination system to combine other credit labels for evaluation and application.

[0126] In one embodiment, when the availability verification fails, the credit label can be updated according to the credibility parameter, the relevance level, and the preset parameter threshold. Among them, the update of the credit label can be executed by this terminal or by the label developer. For the specific update process, refer to the above embodiment and will not be elaborated here.

[0127] As can be seen from the above, the embodiment of the present application can obtain the resource transfer object labeled with the credit label and the annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object; determine the classification rule corresponding to the credit label based on the annotation reference information; determine the association level associated with the resource transfer result between the credit label and the resource transfer result based on the prior information of the resource transfer and the classification rule of the credit label; calculate the credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible based on the sample resource transfer object labeled with the resource transfer result and the classification rule of the credit label; perform availability verification on the credit labels of different classification rules based on the association level, the credibility parameter, and the preset parameter threshold; when the availability verification passes, use the credit label labeled on the resource transfer object to predict the resource transfer result. In this solution, different methods are required for availability verification according to the classification rule of the credit label, and when performing availability verification, in addition to qualitatively determining the association between the credit label and the resource transfer result according to the prior information, the availability verification also quantitatively describes this association according to the credibility parameter, so that the effectiveness of the credit label can be verified more accurately and quickly. Using the credit label verified in this way to predict the resource transfer can improve the accuracy of the prediction result of the resource transfer result.

[0128] Embodiment 2

[0129] In addition to the above steps, the method described in this embodiment may further include step 107 and step 108, and the overall process can be referred to Figure 2b, Step 107 and Step 108 will be described in detail below.

[0130] 107. Perform annotation verification based on the annotation reference information of the credit label.

[0131] Step 107 is set before Step 105, and Step 105 is only executed after the annotation verification passes.

[0132] In one embodiment, the annotation reference information may include three types, namely basic setting information, modeling sample statistical parameters, and backtracking results. Refer to Figure 5b , Step 107 may specifically include the following steps:

[0133] Verify the basic information according to the basic setting information in the annotation reference information and a preset first condition;

[0134] When the basic information verification passes, perform sample verification according to the modeling sample statistical parameters in the annotation reference information and a preset second condition;

[0135] When the sample verification passes, perform backtracking verification according to the backtracking results in the annotation reference information and a preset third condition;

[0136] When the basic information verification, sample verification, and backtracking verification all pass, determine that the credit label passes the initial verification.

[0137] Among them, the basic setting information refers to the setting information for the label developer to develop the credit label, which may include the business meaning, data source, and design concept of the credit label. The preset first condition is determined in advance according to the business form and business scenario of the resource transfer business (taking loans as an example, the business scenario includes information such as loan amount, repayment method, and number of loans).

[0138] Among them, the business meaning is used to understand the association between the credit label and the resource transfer business. The business meaning is the most basic condition for determining whether the label is available. The starting point of the label developer may not coincide with the goal of this resource transfer business. For example, there is a credit label of "lottery enthusiasts", and the business meaning of this credit label can include: information such as the types of lottery tickets purchased by lottery enthusiasts, the purchase frequency, the purchase amount, and the identity of the purchaser. For instance, the types of lottery tickets can include sports lottery, welfare lottery, etc. If the business meaning of the credit label provided by the developer is not clear, or the business meaning does not match the form and scenario of the resource transfer business, there is no need to conduct an availability verification, and it can be directly returned to the developer for optimization. For example, for the "lottery enthusiasts" label, if it describes people with a low purchase frequency, a small amount, and a worry-free life, from the perspective of the risk of default in the loan business, they do not belong to the high-risk group, so it does not match the loan business. Verify whether the business meaning of the credit label meets the preset first condition, that is, according to the annotation reference information and the form and scenario of the resource transfer business, verify whether the business meaning of the credit label is clear and whether it is relevant to the resource transfer business.

[0139] Among them, the data source refers to the source of the modeling samples of the label developer, and it is necessary to verify whether the data source is credible. Specifically, if the modeling samples are from outside the bank or credit institution, then it is also necessary to verify whether the source of the modeling samples is credible; if the modeling samples are from inside the bank or credit institution, the data source is considered credible, and at this time, the focus is on verifying whether the design idea of the credit label is reasonable. Specifically, the design idea of the credit label can be verified from the following aspects: whether the label mining method uses simple rules or complex algorithm models, whether the rule formulation of label mining has reliable data support, whether the algorithm model of label mining is reasonable, whether the division method of the training samples and test sets of the samples is credible; whether the information features used to determine the label are credible. Verify whether the data source and design idea of the credit label meet the preset first condition, that is, verify whether the data source and design idea are reasonable and credible according to the above ideas.

[0140] When the modeling samples are representative, sample verification is carried out according to the statistical parameters of the modeling samples and the preset threshold. Among them, the modeling samples are representative, that is, the label distribution of the modeling samples can represent the actual distribution of the credit labels of the resource transfer objects.

[0141] Among them, the statistical parameters of the modeling samples refer to the parameters verified based on the verification samples of the modeling samples. Specifically, they may include indicators such as accuracy rate, recall rate, AUC (Area Under Curve), and KS. Among them, the label developer must provide the accuracy rate and the recall rate. For some credit labels, the AUC and KS may not be provided. Different classification rules have different requirements for AUC and KS. The accuracy rate, recall rate, AUC (Area Under Curve), KS and other parameters provided by the label developer can be verified respectively according to the preset thresholds corresponding to each parameter to determine whether they meet the preset second condition.

[0142] Among them, backtracking means that the label developer checks the credit labels re - labeled for the resource transfer object at regular intervals. The fields required to be checked during the backtracking process include the resource transfer object identifier (such as the loan object id), and the corresponding credit label of the object. The credit label can be in the form of a conclusion or in the form of parameters. The backtracking results include: the event interval of backtracking, and the actual field types to be checked (which can be called backtracking fields).

[0143] In one embodiment, when the annotation verification fails, it is necessary to update and optimize the credit label and re - perform the verification. The steps of update and optimization can refer to the above - mentioned embodiment and will not be elaborated here. The steps when the annotation verification fails also include:

[0144] When the basic information verification fails, update the credit label according to the basic setting information and the preset first condition, and repeat the steps of the basic information verification according to the basic setting information of the updated credit label and the preset first condition;

[0145] When the sample verification fails, if the application level of the credit label allows downgrading, re - divide the credit label to obtain a downgraded label, and repeat the steps of the sample verification according to the statistical parameters of the modeling samples of the downgraded label and the preset first condition;

[0146] If the application level of the credit label does not allow downgrading, update the credit label according to the statistical parameters of the modeling samples and the preset second condition, and repeat the steps of the sample verification according to the statistical parameters of the modeling samples of the updated credit label and the preset first condition;

[0147] When the backtracking verification fails, update the credit label according to the backtracking results and the preset third condition, and repeat the steps of the backtracking verification according to the backtracking results of the updated credit label and the preset third condition.

[0148] From an application perspective, credit tags with different classification rules have different levels of intuitiveness and convenience when applied to resource transfer results. Different credit tags have different application levels. The higher the application level, the easier it is to intuitively predict resource transfer results. Credit tags with different classification rules and different forms of expression also have different application levels. For example, "resulting in default" is more intuitive than "participating in gambling", and "participating in gambling" is more intuitive than "minor gambling", so the application level is also lower. For the specific division of the application level, please refer to Figure 5c , and the application levels corresponding to various types of credit tags form a pyramid structure. The more business-intuitive it is from bottom to top, the higher the application level. For example, the number of gambling transactions belongs to the credit anomaly parameter classification and does not have the conditions for direct application. If it is reclassified into multi-classification or binary-classification type tags, it can intuitively reflect the gambling level or whether there is gambling. Credit tags with higher application levels can be more intuitively used for predicting resource transfer results.

[0149] When the application level decreases from large to small, it is called downgrading. Downgraded tags refer to tags with a lower application level than the reclassified credit tags. For example, when tag developers cannot determine the boundary between yes and no, cannot provide binary-classification type credit tags, or cannot provide a reasonable prediction method, the credit tag can be downgraded to multi-classification or credit anomaly parameters with a lower application level for re-verification. For the reference of credit tag downgrading Figure 5c .

[0150] 108. According to the annotation reference information, perform stability verification on credit tags with different classification rules.

[0151] Step 108 is set before step 106, and step 106 is only executed after the annotation verification passes.

[0152] Among them, stability verification refers to verifying whether the credit tag can effectively predict resource transfer results in the long term.

[0153] In one embodiment, the developer will provide stability parameters, which are included in the annotation reference information. The terminal can obtain the stability parameters from the annotation reference information, and the stability parameters are parameters used to verify the stability of the credit tag.

[0154] In one embodiment, the annotation reference information does not include stability parameters. The terminal can calculate the stability parameters by itself according to the modeling samples. The stability verification can specifically include the following steps:

[0155] Obtain the stability parameters of the credit tag according to the modeling samples in the annotation reference information and the classification rules;

[0156] Verify the stability of the credit label according to the stability parameter and the preset stable parameter threshold.

[0157] The stability parameters corresponding to credit labels under different classification rules are different, and the specific principles are as follows:

[0158] When the classification rule is the first classification rule, determine the reference time period according to the time type of the credit label;

[0159] Summarize the labels marked for the modeling samples within different reference time periods to obtain a summary result;

[0160] Obtain the stability parameter of the credit label according to the summary result.

[0161] When the classification rule is not the first classification rule, summarize the labels marked for the training samples and validation samples in the modeling samples to obtain a summary result;

[0162] Obtain the stability parameter of the credit label according to the summary result.

[0163] The principle of stability verification is: if the historical samples (i.e., modeling samples) are similar to the future samples (resource transfer objects that need to be predicted), the credit label is considered stable.

[0164] According to the above steps, elaborate on the process of the stability parameter and the meanings of the terms involved:

[0165] A. According to the retrospective frequency of the credit label, the credit label has different time types, and the time intervals for label retrospective of different time types are different. For example, the daily label that is retrospectively calculated daily and the monthly label that is retrospectively calculated monthly. The stability parameter can include volatility and migration rate. Among them, the volatility index is the change range of the label coverage rate over time. For example, if the coverage rate of label A changes by 4% within two adjacent time intervals, the volatility of label A is 4%. The migration rate is the degree of difference in the set of objects (population) covered by the label over time. For example, 20% of the population labeled with label A is not marked in the adjacent time interval, then the migration rate of label A is 20%. When the classification rule is the first classification rule, if the time type of the credit label is a daily label, calculate the volatility and migration rate within adjacent N days as the stability parameter; if the time type of the credit label is a monthly label, look at the volatility and migration rate of adjacent N months as the stability parameter.

[0166] B. When the classification rule is not the first classification rule, PS I (Population Stability Index) can be used as the stability parameter. PS I can reflect the consistency between the distribution of each segment of the validation sample and the distribution of the modeling sample. In actual operation, two distributions are required: the actual distribution and the expected distribution. During the label development process, the training sample is usually used as the expected distribution, and the validation sample is used as the actual distribution.

[0167] Among them, the calculation formula of PS I is as follows:

[0168]

[0169] Among them, A i refers to the proportion of credit labels in different bins in the validation sample; E i refers to the proportion of credit labels in the corresponding bins in the training sample.

[0170] Among them, the binning method can be equal frequency, equal distance or other methods. Different binning methods will result in slightly different calculation results; the binning method can also refer to the method of dividing the grade labels mentioned above, which will not be elaborated here.

[0171] In addition, the stability parameter threshold is preset and related to the business meaning of the credit label itself. For the labels of the first classification rule, for example, the gambling label has a large fluctuation in coverage, while the blacklist label basically has no fluctuation. In this embodiment, an empirical value of 20% can be set as the stability parameter threshold, and it is considered unstable if the volatility and migration rate exceed 20%. For the credit labels of the second and third classification rules, if psi ≤ 0.1, it is considered relatively stable and no update is required. If 0.1 < psi ≤ 0.25, other stability-related indicators need to be checked. If psi > 0.25, the credit indicator needs to be updated.

[0172] Embodiment III

[0173] According to the method described in the previous embodiments, the following will take the specific integration of the resource transfer result prediction device in the terminal as an example for further detailed description.

[0174] Refer to Figure 2c and Figure 5a , taking the business form of resource transfer as a loan as an example for detailed description. Since the purpose of predicting the loan execution result is mainly to prevent loan defaults, in order to unify the influence direction of the prediction result of the same credit label, it can be considered that the credit label is a label used to represent credit anomalies. The prediction result of the loan is true when the loan defaults, and false when the loan prediction result is loan performance.

[0175] The specific process of the resource transfer result prediction method according to the embodiment of the present invention is as follows:

[0176] 201. The terminal obtains a resource transfer object labeled with a credit label and the annotation reference information of the credit label.

[0177] In this embodiment, the terminal can receive the above information from the terminal of the label developer through the network. For specific reference, see Embodiment 1, which will not be elaborated here.

[0178] 202. The terminal performs annotation verification based on the annotation reference information.

[0179] In this embodiment, for different types of credit labels, the principles of the information required by the label developer for annotation verification can be referred to in the following table.

[0180]

[0181]

[0182]

[0183] Among them, "Y" means that the label developer needs to provide; "N" means that the label developer does not need to provide; "H" means nice to have, and it is preferably provided by the label developer, but it is not necessary. For the specific verification process, see Embodiment 2, which will not be elaborated here.

[0184] 203. The terminal performs availability verification on the credit label based on prior information, sample resource transfer objects, and classification rules.

[0185] The terminal first qualitatively evaluates the credit label according to the prior information and classification rules to determine the associated level corresponding to the credit label. For the specific steps, see the above embodiments and will not be elaborated here. Then, availability verification is performed according to the associated level, credibility parameter, and preset parameter threshold. Among them, the sample resource transfer object can be abbreviated as a sample in the following description.

[0186] In one embodiment, the following verification matrix can be used for availability verification:

[0187]

[0188]

[0189] For the credit label of the first classification rule, the embodiment of the present application also provides a program of a label verification tool. Only by inputting the above credibility parameter, the result can be output. The code is as follows:

[0190] import analysis_template as attable_name = 'xxx_table_name'label_list = ['label 1', 'label_2']

[0191] at.analysis_binary_feature(table_name + '.csv', target_col = 'target', label_list = label_list, result_path = 'D: / Doc / tmp / ' + table_name + '.csv')

[0192] In one embodiment, the output result of the above label verification tool is shown in the following table:

[0193]

[0194] Among them, T in the above table represents that the actual loan execution result is true, F represents that the actual loan execution result is false, P represents that the credit label is credit anomaly, and N represents that the credit label is credit normal. Correspondingly, Tp means that the credit label of an object is credit anomaly and the actual loan execution result is loan default. fp means that the credit label of an object is credit anomaly and the actual loan execution result is loan performance. Tn means that the credit label of an object is credit normal and the actual loan execution result is loan default. fn means that the credit label of an object is credit normal and the actual loan execution result is loan performance.

[0195] Among them, because the purpose of predicting the loan execution result is mainly to prevent loan default, in order to unify the influence direction of the credit label on the prediction result, it can be considered that the credit label is a label used to represent credit anomaly situations, the prediction result of the loan is true when the loan default occurs, and false when the loan performance occurs.

[0196] For the credit label of the second classification rule, the method of presetting and gradually screening can be adopted for usability verification, which specifically includes the following steps:

[0197] 1. Determine the upper limit of the coverage rate of the label of credit anomaly, generally 1-5%.

[0198] 2. Determine the division node. According to the qualitative evaluation of reference multi-classification, single model classification and anomaly eigenvalue labels, obtain the label parameter range corresponding to each association level, and then determine the division node between credit normal and credit anomaly according to the label parameter range and the upper limit of the coverage rate.

[0199] 3. Convert the credit labels of multi-classification, single-model classification, and abnormal eigenvalue types into binary classification type level labels. That is, "divide into two" the label parameters according to the division nodes in step 2, and convert them into N binary classification type level labels. And calculate the credibility parameters corresponding to each level label.

[0200] 4. According to the above availability verification matrix and credibility parameters, screen the available level labels. And from multiple available results, select the optimal result according to the severity of the risk control requirements and the data performance. (If the risk control requirements are strict, select the threshold with the largest coverage rate; if the risk control requirements are loose, select the threshold with the largest lift (gain)). If there is no available value, temporarily abandon the direct use of this credit label and add it to the label combination system.

[0201] For the credit labels of the second classification rule, the embodiments of the present application also provide a program for another label verification tool. Only by inputting the corresponding credibility parameters, the result can be output. The code is as follows:

[0202] import analysis_template as at

[0203] table_name='xxx_table_name'

[0204] at.analysis_seg_feature(table_name+'.csv',target_col='target',feature_col='label',resu lt_path='D: / Doc / tmp / '+table_name+'.csv')

[0205] In an embodiment, the output results of the above label verification tool are shown in the following table:

[0206]

[0207] Among them, the coverage rate corresponding to label 1 refers to the ratio of the samples that can be described by label 1 to the total samples, and the accuracy rate corresponding to label 1 is calculated according to the actual default situation of the samples, and is the accuracy rate of the loan execution results predicted by label 1.

[0208] In addition, for the credit labels of the third classification rule (i.e., comprehensive model classification), the credit labels of the third classification rule are loan results, and the results are obtained by the model classifying according to the credit scores. There is also a set of standard processes for reference, which can specifically include the following steps:

[0209] 1. Determine the upper limit of the coverage rate of the credit anomaly label, generally 1-10%. According to the upper limit of the coverage rate anomaly, determine the division node between normal credit and credit anomaly.

[0210] 2. Derive the upper limit of the population default rate based on the ratio of the income to the loss of the loan project.

[0211] 3. According to the division node between normal credit and credit anomaly, divide the credit scores of the comprehensive model into multiple grade labels, and calculate the credibility parameters of each grade label respectively, such as the rejection rate, the passing rate, KS, the cumulative gain, the population default rate of the passed population, etc.

[0212] 4. Initially screen out the candidate credit score range. That is, according to the requirements of the upper limit of the coverage rate and the upper limit of the population default rate of the passed population, and the above verification matrix, initially screen out the credit score range that meets the requirements.

[0213] 5. Select the optimal credit score range, that is, select the optimal result according to the severity of the risk control requirements and the credibility parameters. (On the premise that the cumulative gain ≥ 1 is satisfied, if the risk control requirements are strict, select the threshold with the largest coverage rate; if the risk control requirements are loose, select the threshold with the largest KS). If there is no available credit score, temporarily abandon the direct use of this label and return it to the label developer for re-optimization.

[0214] In addition, for the credit label of the third classification rule, the embodiment of the present application also provides a program of another label verification tool. Only by inputting the above credibility parameters, the result can be output. The code is as follows:

[0215] import analysis_template as at

[0216] table_name = 'xxx_table_name'

[0217] at.analysis_continue_feature(table_name + '.csv', target_col='target', feature_col='label', bin_cnt=200, result_path='D: / Doc / tmp / ' + table_name + '.csv')

[0218] In an embodiment, the output result of the above label verification tool is shown in the following table:

[0219] Among them, the credit label of the third classification rule is expressed as the probability value of the prediction result.

[0220]

[0221] Among them, the rejection rate refers to the ratio of the number of people who are not allowed to obtain a loan to the total number of samples. The passing rate refers to the ratio of the group of people who are allowed to obtain a loan to the total number of samples. The default rate of the passing group refers to the ratio of the people who actually default (do not repay the loan as agreed) among the group of people who are allowed to obtain a loan.

[0222] 204. The terminal verifies the stability of the credit label according to the labeled reference information.

[0223] In the embodiment of the present application, for the credit label of the first classification rule, the volatility and migration rate can be calculated as stability parameters. For the credit label of the second classification rule, psi can be calculated as a stability parameter. The stability verification is performed according to the stability parameter and the preset stability parameter threshold. The process of stability verification can refer to Figure 5d .

[0224] 205. The terminal uses the credit label labeled on the resource transfer object to predict the resource transfer result.

[0225] After the credit label passes the availability verification, according to the above availability verification result and the risk control severity, the level label with the best performance of the verification parameter can be selected for prediction. For example, when comparing the labels of the third classification rule, on the premise that the cumulative prediction gain parameter ≥ 1 is satisfied, if the risk control is strict, the level label with the largest coverage rate is selected; if it is loose, the level label with the largest KS is selected. If there is no available level label, the direct use of this label is temporarily abandoned and re-optimized; for the credit label of the second classification rule, if the risk control is strict, the threshold with the largest coverage rate is selected; if it is loose, the threshold with the largest prediction gain parameter is selected. If there is no level label, the direct use of this label is temporarily abandoned and it is added to the label combination system.

[0226] As can be seen from the above, the embodiment of the present invention provides a set of standardized and systematic verification processes, and the verification process is quantitatively verified based on parameters, which is clearer and more accurate than manual verification.

[0227] To better implement the above method, the embodiment of the present application further provides a resource transfer result prediction device. The resource transfer result prediction device can be specifically integrated in an electronic device, and the electronic device can be a terminal, a server, a personal computer, etc. For example, in this embodiment, the method of the embodiment of the present invention will be described in detail by taking the resource transfer result prediction device integrated in the terminal as an example.

[0228] For example, as Figure 3a shown, the resource transfer result prediction device may include an acquisition unit 301, a determination unit 302, an association evaluation unit 303, a parameter calculation unit 304, an availability verification unit 305, and a prediction unit 306 as follows:

[0229] (1) Acquisition unit 301, configured to acquire a resource transfer object labeled with a credit label and annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object.

[0230] (2) Determination unit 302, configured to determine a classification rule corresponding to the credit label.

[0231] (3) Association evaluation unit 303, configured to determine an association level associated between the credit label and a resource transfer result based on prior information of the resource transfer and the classification rule of the credit label.

[0232] (4) Parameter calculation unit 304, configured to calculate a credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible based on a sample resource transfer object labeled with a resource transfer result and the classification rule of the credit label.

[0233] In an embodiment, the parameter calculation unit may specifically include a division subunit, a calculation subunit, and a determination subunit, as follows:

[0234] The division subunit is configured to divide the credit label into at least two level labels according to the classification rule and a preset label division node.

[0235] The calculation subunit is configured to determine a credibility parameter corresponding to the level label according to the sample resource transfer object and the level label.

[0236] The determination subunit is configured to use the credibility parameter corresponding to the level label as the credibility parameter of the credit label.

[0237] In an embodiment, the division subunit may specifically be configured to:

[0238] When the classification rule is a first classification rule, use the corresponding credit label as two level labels.

[0239] When the classification rule is a second classification rule, divide the multi-class credit label into at least three level labels according to a preset label division node, and the manifestation form of the level label is a label of the first classification rule.

[0240] When the classification rule is a third classification rule, obtain a label parameter corresponding to the credit label from the annotation reference information; divide the credit label into at least two level labels according to the label parameter, and the manifestation form of the level label is a label of the first classification rule.

[0241] In an embodiment, the calculation subunit may specifically be configured to:

[0242] Label the sample resource transfer object based on the level label to obtain an object labeled with a level label as the label-covered object;

[0243] Based on the true result labeled in the sample resource transfer object and the prediction result of the label-covered object, determine the credibility parameter of the prediction result for the level labels of different classification rules;

[0244] Use the credibility parameter of the level label and the statistical parameter of the label-covered object as the credibility parameter corresponding to the level label.

[0245] (5) The availability verification unit 305 is configured to perform availability verification on the credit labels of different classification rules based on the associated level, the credibility parameter, and a preset parameter threshold.

[0246] In one embodiment, the availability verification unit can also be used for:

[0247] When the availability verification fails, update the credit label according to the credibility parameter, the correlation level, and the preset parameter threshold.

[0248] (6) The prediction unit is configured to, when the availability verification passes, use the credit label labeled on the resource transfer object to predict the resource transfer result.

[0249] Reference Figure 3b , Optionally, the resource transfer result prediction device further includes a labeling verification unit 307 configured to perform labeling verification based on the labeling reference information of the credit label.

[0250] In one embodiment, the labeling verification unit 307 may specifically include a basic information verification subunit, a sample verification subunit, a backtracking verification subunit, and a determination subunit, as follows:

[0251] The basic information verification subunit is configured to perform basic information verification according to the basic setting information in the labeling reference information and a preset first condition;

[0252] The sample verification subunit is configured to, when the basic information verification passes, perform sample verification according to the modeling sample statistical parameters in the labeling reference information and a preset second condition;

[0253] The backtracking verification subunit is configured to, when the sample verification passes, perform backtracking verification according to the backtracking result in the labeling reference information and a preset third condition;

[0254] The determination subunit is configured to determine that the credit label passes the initial verification when the basic information verification, the sample verification, and the backtracking verification all pass.

[0255] In one embodiment, the annotation verification unit can also be used for:

[0256] When the basic information verification fails, update the credit label according to the basic setting information and the preset first condition, and repeat the basic information verification step according to the basic setting information of the updated credit label and the preset first condition;

[0257] When the sample verification fails, if the application level of the credit label allows downgrading, reclassify the credit label to obtain a downgraded label, and repeat the sample verification step according to the statistical parameters of the modeling samples of the downgraded label and the preset first condition; if the application level of the credit label does not allow downgrading, update the credit label according to the statistical parameters of the modeling samples and the preset second condition, and repeat the sample verification step according to the statistical parameters of the modeling samples of the updated credit label and the preset first condition;

[0258] When the backtracking verification fails, update the credit label according to the backtracking result and the preset third condition, and repeat the backtracking verification step according to the backtracking result of the updated credit label and the preset third condition.

[0259] Reference Figure 3b , optionally, the resource transfer result prediction device may further include a stability verification unit 308, configured to perform stability verification on the credit labels of different classification rules according to the annotation reference information.

[0260] In one embodiment, the stability verification unit 308 may specifically include a stability parameter calculation unit and a stability verification subunit, as follows:

[0261] The stability parameter calculation unit is configured to obtain the stability parameter of the credit label according to the modeling samples in the annotation reference information and the classification rule;

[0262] The stability verification subunit is configured to perform stability verification on the credit label according to the stability parameter and a preset stable parameter threshold.

[0263] In specific implementation, each of the above units may be implemented as an independent entity, or may be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of each of the above units, reference may be made to the foregoing method embodiments, which will not be elaborated herein.

[0264] Since this solution needs to adopt different methods for availability verification according to the classification rules of credit tags, and when performing availability verification, in addition to qualitatively determining the association between the credit tag and the resource transfer result based on prior information, the availability verification also quantitatively describes this association according to credible parameters, so that the validity of the credit tag can be verified more accurately and quickly. Using the credit tags verified in this way to predict resource transfer can improve the accuracy of the prediction result of the resource transfer result.

[0265] The embodiments of this application also provide a terminal, which can be a mobile phone, a tablet computer, a microprocessing box, a drone, an image acquisition device, etc. As Figure 4 shown, it shows a schematic structural diagram of the terminal involved in the embodiments of this application. Specifically:

[0266] The terminal may include a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, an input module 404, a communication module 405 and other components. Those skilled in the art can understand that Figure 4 the terminal structure shown does not constitute a limitation on the terminal, and may include more or fewer components than shown, or combine certain components, or arrange different components. Among them:

[0267] The processor 401 is the control center of the terminal, connecting various parts of the entire terminal through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, it executes various functions of the terminal and processes data, thereby performing overall detection of the terminal. In some embodiments, the processor 401 may include one or more processing cores; in some embodiments, the processor 401 may integrate an application processor and a modulation and demodulation processor, where the application processor mainly processes the operating system, user interface, application programs, etc., and the modulation and demodulation processor mainly processes wireless communication. It can be understood that the above modulation and demodulation processor may not be integrated into the processor 401.

[0268] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and resource transfer result predictions by running the software programs and modules stored in the memory 402. The memory 402 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the terminal. In addition, the memory 402 can include a high-speed random access memory and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices. Correspondingly, the memory 402 can also include a memory controller to provide the processor 401 with access to the memory 402.

[0269] The terminal further includes a power supply 403 for powering each component. In some embodiments, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to implement functions such as management of charging, discharging, and power consumption management through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0270] The terminal may further include an input module 404, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0271] The terminal may further include a communication module 405. In some embodiments, the communication module 405 can include a wireless sub-module. The terminal can perform short-distance wireless transmission through the wireless sub-module of the communication module 405, so as to provide wireless broadband Internet access. For example, the communication module 405 can be used to help users obtain credit labels, send and receive emails, browse web pages, and access streaming media, etc.

[0272] Although not shown, the terminal may further include a display unit, etc., which will not be elaborated here. Specifically, in this embodiment, the processor 401 in the terminal will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402 to implement various functions as follows:

[0273] Obtain a resource transfer object labeled with a credit label and the annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object;

[0274] Determine the classification rule corresponding to the credit label;

[0275] Determine the association level associated between the credit label and the resource transfer result based on the prior information of the resource transfer and the classification rules of the credit label;

[0276] Based on the sample resource transfer objects annotating the resource transfer results and the classification rules of the credit label, calculate the credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible;

[0277] Based on the association level, the credibility parameter, and a preset parameter threshold, perform availability verification on the credit labels of different classification rules;

[0278] When the availability verification passes, adopt the credit label annotated on the resource transfer object to predict the resource transfer result.

[0279] As can be seen from the above, in this solution, different methods are required for availability verification according to the classification rules of the credit label, and when performing the availability verification, in addition to qualitatively determining the association between the credit label and the resource transfer result based on the prior information, the association is also quantitatively described according to the credibility parameter, so that the effectiveness of the credit label can be verified more accurately and quickly. Using the credit label verified in this way to predict the resource transfer can improve the accuracy of the prediction result of the resource transfer result.

[0280] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by instructions controlling related hardware. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0281] For this reason, an embodiment of the present application provides a storage medium, which stores multiple instructions that can be loaded by a processor to execute the steps in any one of the resource transfer result prediction methods provided by the embodiments of the present application. For example, the instructions can execute the following steps:

[0282] Obtain a resource transfer object annotated with a credit label and the annotation reference information of the credit label, where the credit label indicates the credit type of the resource transfer object;

[0283] Determine the classification rules corresponding to the credit label;

[0284] Based on the prior information of the resource transfer and the classification rules of the credit label, determine the association level associated between the credit label and the resource transfer result;

[0285] Calculate a credibility parameter that characterizes whether the credit label associated with the resource transfer result is credible based on the classification rules of the sample resource transfer object and the credit label for the labeled resource transfer result;

[0286] Based on the association level, the credibility parameter, and a preset parameter threshold, perform an availability verification on the credit labels of different classification rules;

[0287] When the availability verification passes, use the credit label marked on the resource transfer object to predict the resource transfer result.

[0288] Among them, the storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disc, etc.

[0289] Since the instructions stored in the storage medium can execute the steps in any of the resource transfer result prediction methods provided in the embodiments of the present application, the beneficial effects achievable by any of the resource transfer result prediction methods provided in the embodiments of the present application can be achieved. For details, refer to the previous embodiments and will not be elaborated here.

[0290] The above has introduced in detail a resource transfer result prediction method, device, server, and storage medium provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for predicting resource transfer results, characterized in that, it includes: Obtain resource transfer objects labeled with credit tags and the annotation reference information of the credit tags, where the credit tags indicate the credit types of the resource transfer objects; Determine the classification rules corresponding to the credit tags; Based on the prior information of the resource transfer and the classification rules of the credit tags, determine the association level associated with the credit tags and the resource transfer results; Based on the sample resource transfer objects labeled with resource transfer results and the classification rules of the credit tags, calculate the credibility parameters characterizing whether the association between the credit tags and the resource transfer results is credible, including: dividing the credit tags into at least two level tags according to the classification rules and preset label division nodes; annotating the sample resource transfer objects based on the level tags to obtain objects labeled with level tags as label-covered objects; determining the credibility parameters of the level tags of different classification rules for the prediction results based on the true results labeled in the sample resource transfer objects and the prediction results of the label-covered objects; using the credibility parameters of the level tags and the statistical parameters of the label-covered objects as the credibility parameters corresponding to the level tags; using the credibility parameters corresponding to the level tags as the credibility parameters of the credit tags; Based on the association level, the credibility parameters, and the preset parameter threshold, verify the availability of the credit tags of different classification rules; When the availability verification passes, use the credit tags labeled on the resource transfer objects to predict the resource transfer results.

2. The method for predicting resource transfer results according to claim 1, characterized in that, before determining the association level associated with the label and the resource transfer results based on the prior information of the resource transfer and the classification rules of the credit tags, it further includes: Performing annotation verification based on the annotation reference information of the credit tags; The determining the association level associated with the label and the resource transfer results based on the prior information of the resource transfer and the classification rules of the credit tags includes: When the annotation verification passes, determining the association level associated with the label and the resource transfer results based on the prior information of the resource transfer and the classification rules of the credit tags.

3. The method for predicting resource transfer results according to claim 1, characterized in that, before using the credit tags labeled on the resource transfer objects to predict the resource transfer results, it further includes: Performing stability verification on the credit tags of different classification rules according to the annotation reference information; The using the credit tags labeled on the resource transfer objects to predict the resource transfer results includes: When the stability verification passes, using the credit tags labeled on the resource transfer objects to predict the resource transfer results.

4. The method for predicting resource transfer results according to claim 1, characterized in that, the dividing the credit tags into at least two level tags according to the classification rules and preset label division nodes includes: When the classification rule is the first classification rule, the corresponding credit label is used as two grade labels.

5. The resource transfer result prediction method according to claim 1, characterized in that the step of dividing the credit label into at least two grade labels according to the classification rule and the preset label includes: When the classification rule is the second classification rule, nodes are divided according to the preset label, and the multi-class credit label is divided into at least three grade labels, and the manifestation form of the grade label is the label of the first classification rule.

6. The resource transfer result prediction method according to claim 1, characterized in that the step of dividing the credit label into at least two grade labels according to the classification rule and the preset label includes: When the classification rule is the third classification rule, the label parameter corresponding to the credit label is obtained from the annotation reference information; According to the label parameter, the credit label is divided into at least two grade labels, and the manifestation form of the grade label is the label of the first classification rule.

7. The resource transfer result prediction method according to claim 1, characterized in that after verifying the availability of the credit labels of different classification rules based on the prior information of the resource transfer and the sample resource transfer objects marked with the true results, it further includes: When the availability verification fails, the credit label is updated according to the credibility parameter, the relevance level and the preset parameter threshold.

8. The resource transfer result prediction method according to claim 2, characterized in that the annotation verification based on the annotation reference information of the credit label includes: Performing basic information verification according to the basic setting information and the preset first condition in the annotation reference information; When the basic information verification passes, sample verification is performed according to the modeling sample statistical parameters and the preset second condition in the annotation reference information; When the sample verification passes, retrospective verification is performed according to the retrospective result and the preset third condition in the annotation reference information; When the basic information verification, the sample verification and the retrospective verification all pass, it is determined that the credit label passes the initial verification.

9. The resource transfer result prediction method according to claim 3, characterized in that the stability verification of the credit labels of different classification rules according to the annotation reference information includes: Obtaining the stability parameter of the credit label according to the modeling sample in the annotation reference information and the classification rule; Performing stability verification on the credit label according to the stability parameter and the preset stability parameter threshold.

10. The resource transfer result prediction method according to claim 9, characterized in that the step of obtaining the stability parameter of the credit label according to the modeling sample in the annotation reference information and the classification rule includes: When the classification rule is the first classification rule, the reference time period is determined according to the time type of the credit label; Summarize the credit labels marked by the modeling samples in different reference time periods to obtain a summary result; According to the summary result, the stability parameter of the credit label is obtained.

11. The resource transfer result prediction method according to claim 9, characterized in that obtaining the stability parameter of the credit label according to the modeling sample and the classification rule in the annotation reference information includes: when the classification rule is not the first classification rule, summarizing the credit labels annotated in the training samples and validation samples in the modeling sample to obtain a summary result; obtaining the stability parameter of the credit label according to the summary result.

12. The resource transfer result prediction method according to claim 9, characterized in that after performing stability verification on the credit labels of different classification rules according to the annotation reference information, it further includes: when the stability verification fails, updating the credit label according to the stability parameter and the stable parameter threshold.

13. A resource transfer result prediction device, characterized in that it includes: an acquisition unit, configured to acquire a resource transfer object annotated with a credit label and the annotation reference information of the credit label, wherein the credit label indicates the credit type of the resource transfer object; a determination unit, configured to determine the classification rule corresponding to the credit label; a correlation evaluation unit, configured to determine the correlation level associated with the resource transfer result based on the prior information of the resource transfer and the classification rule of the credit label; a parameter calculation unit, configured to calculate a credibility parameter characterizing whether the association between the credit label and the resource transfer result is credible based on a sample resource transfer object annotated with a resource transfer result and the classification rule of the credit label, including: dividing the credit label into at least two level labels according to the classification rule and a preset label division node; annotating the sample resource transfer object based on the level label to obtain an object annotated with a level label as a label coverage object; determining the credibility parameter of the level label of different classification rules for the prediction result based on the true result annotated in the sample resource transfer object and the prediction result of the label coverage object; using the credibility parameter of the level label and the statistical parameter of the label coverage object as the credibility parameter corresponding to the level label; using the credibility parameter corresponding to the level label as the credibility parameter of the credit label; an availability verification unit, configured to perform availability verification on the credit labels of different classification rules based on the correlation level, the credibility parameter, and a preset parameter threshold; a prediction unit, configured to, when the availability verification passes, predict the resource transfer result using the credit label annotated on the resource transfer object.

14. A terminal, characterized in that it includes a memory storing multiple instructions; a processor loads the instructions from the memory to execute the steps in the resource transfer result prediction method according to any one of claims 1-12.

15. A storage medium, characterized in that it stores multiple instructions thereon, and the instructions are for a computer program stored thereon. When the computer program runs on a computer, it causes the computer to execute the resource transfer result prediction method according to any one of claims 1-12.

Images