Sample evaluation method, device, storage medium and electronic device

Abstract

The present invention discloses a sample evaluation method, device, storage medium and electronic device. Among them, the method includes: determining a first task label and a second task label of a training sample, where the first task label is used to predict a passing probability, and the second task label is used to predict a compliance probability; performing feature screening on the training sample to obtain a first feature and a second feature; training an initial multi-task model through the first task label, the second task label, the first feature and the second feature to obtain a target multi-task model; and evaluating a target sample through the target multi-task model. The present invention solves the technical problem that the related art cannot accurately identify high-quality users in rejected samples.

Description

Technical Field

[0001] The present invention relates to the field of financial risk assessment, and in particular to a sample assessment method, device, storage medium and electronic device. Background Art

[0002] As the cost of acquiring users continues to rise, financial institutions are paying more and more attention to the management of existing customers. If they can identify lower-risk users among rejected users and win them back, they can reduce customer acquisition costs and increase overall profits.

[0003] Currently, there are three methods for identifying low-risk users among rejected users: the downward approach, the co-occurrence method, and the rejection inference method. The rejection inference method includes expansion, reweighting, bundling, iterative reclassification, and a two-stage approach. The downward approach incurs a certain amount of bad debt loss, resulting in high costs. The co-occurrence method is difficult to implement, and none of the above rejection inference methods can accurately identify high-quality users among the rejected samples. Summary of the Invention

[0004] The embodiments of the present invention provide a sample evaluation method, device, storage medium, and electronic device to at least solve the technical problem that related technologies cannot accurately identify high-quality users in rejected samples.

[0005] According to one embodiment of the present invention, a sample evaluation method is provided, comprising:

[0006] Determine the first task label and the second task label of the training sample, wherein the first task label is used to predict the passing probability and the second task label is used to predict the keeping probability; perform feature screening on the training sample to obtain the first feature and the second feature; train the initial multi-task model using the first task label, the second task label, the first feature and the second feature to obtain the target multi-task model; and evaluate the target sample using the target multi-task model.

[0007] Optionally, feature screening is performed on the training samples to obtain the first feature and the second feature, including: according to the first algorithm, feature screening is performed on the training samples through the first model to obtain the first feature, wherein the first algorithm is used to select features with high contribution, and the first feature includes samples with post-loan performance and rejected samples; according to the first algorithm, feature screening is performed on the training samples through the second model to obtain the second feature, wherein the second feature includes samples with post-loan performance.

[0008] Optionally, it also includes: optimizing the initial multi-task model according to the first loss function and the second loss function to obtain the target multi-task model, wherein the first loss function is the loss function of the first model, and the second loss function is the loss function of the second model.

[0009] Optionally, optimizing the initial multi-task model according to the first loss function and the second loss function to obtain the target multi-task model includes: performing uncertainty weighting by taking the variances of the first loss function and the second loss function as weights to obtain the target loss function; optimizing the initial multi-task model by using the target loss function to obtain the target multi-task model.

[0010] Optionally, the target multi-task model is a Progressive Layered Extraction (PLE) model.

[0011] Optionally, evaluating the target sample by using the target multi-task model includes: inputting the target sample into the target multi-task model to obtain a first probability and a second probability, where the first probability is the passing probability of the target sample, and the second probability is the probability of keeping the contract of the target sample; evaluating the target sample according to the first probability and the second probability.

[0012] Optionally, in response to obtaining the qualification information of the training sample, it further includes: inputting the target sample into the target multi-task model to obtain a third probability, where the third probability is the qualification passing probability of the target sample; evaluating the target sample according to the first probability, the second probability, and the third probability.

[0013] According to one embodiment of the present invention, there is also provided a sample evaluation device, including:

[0014] A determination module, which is used to determine the first task label and the second task label of the training sample, where the first task label is used to predict the passing probability, and the second task label is used to predict the probability of keeping the contract; a screening module, which is used to perform feature screening on the training sample to obtain a first feature and a second feature; a training module, which is used to train the initial multi-task model by using the first task label, the second task label, the first feature, and the second feature to obtain the target multi-task model; an evaluation module, which is used to evaluate the target sample by using the target multi-task model.

[0015] Optionally, the screening module is further used to perform feature screening on the training sample by using the first model according to the first algorithm to obtain the first feature, where the first algorithm is used to select features with high contribution degrees, and the first feature includes samples with post-loan performance and rejected samples; perform feature screening on the training sample by using the second model according to the first algorithm to obtain the second feature, where the second feature includes samples with post-loan performance.

[0016] Optionally, the training module is further used to optimize the initial multi-task model according to the first loss function and the second loss function to obtain the target multi-task model, where the first loss function is the loss function of the first model, and the second loss function is the loss function of the second model.

[0017] Optionally, the training module is further configured to perform uncertainty weighting on the variances of the first loss function and the second loss function as weights to obtain an objective loss function; and optimize the initial multi-task model by using the objective loss function to obtain an objective multi-task model.

[0018] Optionally, the objective multi-task model is a Progressive Layered Extraction (PLE) model.

[0019] Optionally, the evaluation module is further configured to input the target sample into the objective multi-task model to obtain a first probability and a second probability, where the first probability is the passing probability of the target sample, and the second probability is the compliance probability of the target sample; and evaluate the target sample according to the first probability and the second probability.

[0020] Optionally, in response to obtaining the qualification information of the training sample, the evaluation module is further configured to input the target sample into the objective multi-task model to obtain a third probability, where the third probability is the qualification passing probability of the target sample; and evaluate the target sample according to the first probability, the second probability, and the third probability.

[0021] According to one embodiment of the present invention, there is also provided a computer-readable storage medium storing a computer program, where the computer program is configured to execute the sample evaluation method in any one of the above when running on a computer or a processor.

[0022] According to one embodiment of the present invention, there is also provided an electronic device including a memory and a processor, where the memory stores a computer program, and the processor is configured to run the computer program to execute the sample evaluation method in any one of the above.

[0023] In the embodiment of the present invention, by determining the first task label and the second task label of the training sample, where the first task label is used to predict the passing probability, and the second task label is used to predict the compliance probability; performing feature screening on the training sample to obtain a first feature and a second feature; training an initial multi-task model by using the first task label, the second task label, the first feature, and the second feature to obtain an objective multi-task model; and evaluating the target sample by using the objective multi-task model. By adopting the above method, considering that there is a certain correlation between predicting the passing probability and predicting the compliance probability, the above two tasks are modeled using a multi-task learning framework, increasing the sample size for modeling, thereby avoiding the problem of modeling deviation. At the same time, it is not necessary to manually correct the "good person probability" and no noise is introduced, and high-quality users in the rejected samples can be accurately identified, improving the overall revenue, and further solving the technical problem that the related technology cannot accurately identify high-quality users in the rejected samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings described herein are used to provide a further understanding of the present invention, and constitute a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the drawings:

[0025] Figure 1 is a flowchart of a sample evaluation method according to one embodiment of the present invention;

[0026] Figure 2 is a schematic diagram of the term of a credit product according to one embodiment of the present invention;

[0027] Figure 3 is a structural diagram of a multi-task network model according to one embodiment of the present invention;

[0028] Figure 4 is a structural diagram of a multi-task network model according to one embodiment of the present invention;

[0029] Figure 5 is a structural block diagram of a sample evaluation device according to one embodiment of the present invention. Detailed Embodiments

[0030] For ease of understanding, some explanations of concepts related to the embodiments of the present invention are exemplarily given for reference.

[0031] As follows:

[0032] Known Good Bad (KGB) model: Using the lending samples to build a model, defining Y by the degree of overdue (1 = bad, 0 = good) to predict the overdue probability P (bad) , that is, the "probability of a bad person".

[0033] Accept Reject (AR) model: Using the full samples (rejected samples + lending samples) to build a model, defining Y by "whether to lend" (1 = accept, 0 = reject) to predict the transaction probability P (accept) , that is, the passing probability.

[0034] Known Good Bad of Accepted and Assumed Rejected (AGB) model: Using the lending samples and the inferred rejected samples to build a model, defining Y by the degree of overdue (1 = bad, 0 = good) to predict the overdue probability P (bad) , that is, the "probability of a bad person". , that is, the "probability of a bad person".

[0035] Downward exploration method: Manually mark a part of the rejected users and continuously observe their post-loan performance.

[0036] Co - occurrence representation method: Using the post - loan performance data or credit data of other financial institutions to label the rejected users and identify users with lower risks.

[0037] The augmentation method is to score the rejected samples by constructing a KGB model, then divide the threshold (cutoff) to label the rejected samples as good or bad, and construct an AGB model based on it.

[0038] The reweighting method is to score all samples with the KGB model and then bin them. Calculate the ratio of the total number of samples in each bin to the number of loan - issued samples, so as to adjust the weights of the loan - issued samples in the corresponding bins. Then reconstruct the KGB model by introducing sample weights.

[0039] The parcelling method is similar to the augmentation method. That is, use the KGB model to label the rejected samples, and then add them to the labeled rejected samples to construct an AGB model. The difference is that when the parcelling method labels the rejected samples, the loan - issued samples and the rejected samples are grouped according to the same boundary. Under the condition of ensuring the same binning and making the ratio of the good - bad ratio of the rejected samples to the good - bad ratio of the loan - issued samples a fixed value (empirical value), randomly label the rejected samples in each bin.

[0040] The iterative reclassification method is multiple iterations of the augmentation method. Build an AGB model through the augmentation method to score the rejected samples, re - label the rejected samples according to the obtained "bad person probability" (P (bad) ), and then build and train the AGB model until the set parameters converge.

[0041] The two - stage method takes into account the loan - granting decision in the credit business process. Add loan - granting prediction in the modeling process, and separately construct an AR model and a KGB model to score all samples. By observing the distribution differences of the average "good person probability" P (accept) of the loan - issued samples and the rejected samples at different passing probabilities (P (good) ), manually correct the P (good) of some rejected samples based on business experience, and then use the parcelling method to construct an AGB model.

[0042] Currently, using the downward - probing method requires bearing certain bad - debt losses, and using the co - occurrence representation method requires obtaining the post - loan performance data of other financial institutions. Due to regulatory issues with post - loan performance data, it is usually not shared externally. Even if the post - loan performance data is obtained, due to differences between different credit businesses (such as different interest rates, terms, label definitions, etc.), the effect of identifying users will be poor.

[0043] The basic ideas of the expansion method, reweighting method, packing method, and iterative reclassification method are all to build an AGB model by establishing a KGB model and labeling the rejected samples, or to build a model by combining the labeling of rejected samples to change the weights of the lending samples. These methods are implemented on the premise that the KGB model is effective for rejected samples. However, only using lending samples for modeling and then performing rejection inference will lead to sample bias, and the KGB model cannot accurately evaluate rejected samples. Therefore, the effect of using such methods for rejection inference is also poor.

[0044] The two-stage method divides the modeling process into two stages: loan approval prediction and good / bad prediction according to the credit business logic. However, the introduction of loan approval prediction is to assist in correcting the prediction bias of the KGB model for rejected samples. The ultimate idea is still similar to that of the expansion method, reweighting method, packing method, and iterative reclassification method. Moreover, artificially correcting the "good person probability" of rejected samples will introduce noise. In addition, the two-stage method models the two tasks of loan approval prediction and good / bad prediction separately, without considering the correlation between them, resulting in poor rejection inference effect and inability to accurately identify high-quality users among rejected samples.

[0045] To enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0046] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0047] According to one embodiment of the present invention, an embodiment of a sample evaluation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0048] This method embodiment can be executed in an electronic device, a similar control device or system including a memory and a processor. Taking the electronic device as an example, the electronic device can include one or more processors and a memory for storing data. Optionally, the above-mentioned electronic device can also include a communication device for communication functions and a display device. Those of ordinary skill in the art can understand that the above structural description is only illustrative and does not limit the structure of the above-mentioned electronic device. For example, the electronic device can also include more or fewer components than the above structural description, or have a different configuration from the above structural description.

[0049] The processor can include one or more processing units. For example: the processor can include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microcontroller unit (MCU), a field-programmable gate array (FPGA), a neural-network processing unit (NPU), a tensor processing unit (TPU), a processing device such as an artificial intelligent (AI) type processor, etc. Among them, different processing units can be independent components or integrated in one or more processors. In some instances, the electronic device can also include one or more processors.

[0050] The memory can be used to store computer programs, such as the computer program corresponding to the sample evaluation method in the embodiments of the present invention. The processor realizes the above sample evaluation method by running the computer program stored in the memory. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely disposed relative to the processor, and these remote memories can be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0051] The communication device is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of a mobile terminal. In one instance, the communication device includes a network interface controller (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one instance, the communication device can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0052] The display device can be, for example, a touch-screen liquid crystal display (LCD) and a touch display (also referred to as a "touch screen" or "touch display screen"). The liquid crystal display enables a user to interact with the user interface of the mobile terminal. In some embodiments, the above mobile terminal has a graphical user interface (GUI), and the user can perform human-computer interaction with the GUI through finger contacts and / or gestures on the touch-sensitive surface. The human-computer interaction function herein optionally includes the following interactions: creating web pages, drawing, word processing, creating electronic documents, games, video conferencing, instant messaging, sending and receiving emails, call interfaces, playing digital videos, playing digital music, and / or web browsing, etc. The executable instructions for performing the above human-computer interaction functions are configured / stored in a computer program product or a readable storage medium executable by one or more processors.

[0053] In this embodiment, a sample evaluation method running on an electronic device is provided. Figure 1 It is a flowchart of the sample evaluation method according to one embodiment of the present invention, as Figure 1 shown, and the process includes the following steps:

[0054] Step S101, determine the first task label and the second task label of the training sample.

[0055] Among them, the first task label is used to predict the passing probability, which can be understood as predicting whether a user applying for credit passes the loan application, that is, predicting whether to grant a loan. The second task label is used to predict the probability of keeping the contract, which can be understood as, after a financial institution grants a loan to a user who has passed the loan application, predicting whether the user applying for credit can repay the loan on time, that is, predicting the repayment ability.

[0056] It can be seen that the embodiment of the present invention includes two tasks. The first task is loan granting prediction, corresponding to the first task label y1. The first task label y1 can be defined as "whether to grant a loan". y1 = 1 means to conduct a transaction, that is, to grant a loan, and y1 = 0 means to reject the transaction, that is, not to grant a loan. The second task is repayment ability prediction, corresponding to the second task label y2. Since the repayment ability is related to the income of the user applying for credit, the second task label y2 can be defined as "income". y2 = 1 means weak repayment ability, that is, unable to repay the loan on time, and y2 = 0 means normal repayment ability, that is, able to repay the loan on time.

[0057] Since it is difficult to collect the income information of users applying for credit, the repayment ability can be defined by the post-loan performance of the users to predict the income level of the users. The definition of the second task label y2 can refer to Figure 2 , Figure 2 FIG.

[0058] is a schematic diagram of the term of a credit product according to an embodiment of the present invention. Taking the credit product with 12 installments of repayment as an example, the total time length of 12 installments is T4. Among them, T1 is the time from the 1st to the 3rd period of the credit product, T2 is the time from the 1st to the 6th period of the credit product, and T3 is the time from the 6th to the 12th period of the credit product.

[0059] When selecting modeling samples, users who can repay the loan normally within the T2 time period can be selected as training samples, that is, users who can repay the loan on time from the 1st period to the 6th period are selected as training samples, so as to determine the income level of users by predicting the overdue probability of users within the T3 time period. y2 = 0 indicates normal repayment ability, and it is a user with normal repayment. It can be understood that this user has no overdue behavior at all or only has minor overdue, such as only overdue for 1 to 3 days. y2 = 1 indicates that the user can repay the loan normally within the T2 time period, but has had relatively serious overdue within the T3 time period, such as overdue for 15 days or more than 30 days.

[0060] Step S102: Screen the features of the training samples to obtain the first feature and the second feature.

[0061] By screening the features of the training samples, the first feature and the second feature suitable for model training are screened out. Among them, the first feature is used to train the first model, and the first feature includes samples with post-loan performance and rejected samples, that is, the full amount of samples. The second feature is used to train the second model, and the second feature includes samples with post-loan performance.

[0062] Step S103: Train the initial multi-task model with the first task label, the second task label, the first feature and the second feature to obtain the target multi-task model.

[0063] Among them, the initial multi-task model is a multi-task network model. By training the initial multi-task model with the first task label, the second task label, the first feature and the second feature, the target multi-task model can be obtained. This target multi-task model can perform the first task and the second task simultaneously, that is, the target multi-task model is a multi-task network model and can predict the approval probability and the compliance probability at the same time.

[0064] Step S104: Evaluate the target samples through the target multi-task model.

[0065] The target samples can be, for example, rejected samples. Input the rejected samples into the target multi-task model. By evaluating the rejected samples through the target multi-task model, the approval probability and the compliance probability of the rejected samples can be obtained, so as to accurately identify high-quality users among the rejected samples.

[0066] Through the above steps, by determining the first task label and the second task label of the training samples, where the first task label is used to predict the passing probability and the second task label is used to predict the probability of keeping the contract; performing feature screening on the training samples to obtain the first feature and the second feature; training the initial multi-task model through the first task label, the second task label, the first feature and the second feature to obtain the target multi-task model; and evaluating the target samples through the target multi-task model. By adopting the above method, considering that there is a certain correlation between predicting the passing probability and predicting the probability of keeping the contract, the above two tasks are modeled using a multi-task learning framework, increasing the sample size for modeling, thus avoiding the problem of modeling deviation. At the same time, there is no need for manual correction of the "good person probability", no noise is introduced, high-quality users in the rejected samples can be accurately identified, the overall revenue is improved, and thus the technical problem that the related technology cannot accurately identify high-quality users in the rejected samples is solved.

[0067] Optionally, in step S102, performing feature screening on the training samples to obtain the first feature and the second feature may include the following implementation steps:

[0068] Step S102a: According to the first algorithm, use the first model to perform feature screening on the training samples to obtain the first feature.

[0069] Step S102b: According to the first algorithm, use the second model to perform feature screening on the training samples to obtain the second feature.

[0070] Among them, the first algorithm is used to select features with high contribution degrees. The first feature includes samples with post-loan performance and rejected samples, and the second feature includes samples with post-loan performance.

[0071] The first algorithm can be, for example, the eXtreme Gradient Boosting (XGBoost) algorithm. The first algorithm can also be other algorithms used for model training and feature screening, which are not limited in the present invention. According to the XGBoost algorithm, perform feature screening on the training samples through the first model and the second model, select features with high contribution degrees, and obtain the first feature for training the first model and the second feature for training the second model.

[0072] Optionally, the process includes the following steps:

[0073] Step S105: Optimize the initial multi-task model according to the first loss function and the second loss function to obtain the target multi-task model.

[0074] The first loss function is the loss function of the first model, and the second loss function is the loss function of the second model. The first loss function L1 = y1logp1 + (1-y1)log(1-p1), where y1 is the first task label and p1 is the predicted probability of approval, that is, the predicted probability of loan approval. The second loss function L2 = y2logp2 + (1-y2)log(1-p2), where y2 is the second task label and p2 is the predicted probability of compliance, that is, the predicted probability of on-time repayment.

[0075] Optionally, in step S105, optimizing the initial multi-task model according to the first loss function and the second loss function to obtain the target multi-task model may include the following execution steps:

[0076] Step S105a: The variance of the first loss function and the variance of the second loss function are weighted as the uncertainty of the weight to obtain the target loss function.

[0077] Step S105b: Optimize the initial multi-task model using the target loss function to obtain a target multi-task model.

[0078] For classification tasks, the loss function is cross entropy. In order to enable each task to share information, multi-task learning (MTL) generally performs a weighted summation on the loss functions of each task, that is, directly performs a weighted summation on the first loss function and the second loss function in the embodiment of the present invention. However, in the embodiment of the present invention, since the first task and the second task use different sample sizes when modeling, the first task is modeled on the full amount of samples (samples with post-loan performance and rejected samples), while the second task is only modeled on samples with post-loan performance. The difference in sample size may result in a certain task dominating after the target loss function is determined by weighted summation, and other tasks cannot be fully optimized. It may also lead to an imbalance in the ratio of positive and negative samples, making training difficult. Therefore, the embodiment of the present invention adopts uncertainty weighting (uncertainty weighting) with the variance of the loss function of different tasks as the weight to achieve dynamic adjustment of weights and level the convergence rate of the two tasks.

[0079] The obtained target loss function can be expressed as Among them, σ i Denotes the variance of the loss function. The initial multi-task model is optimized based on the target loss function to obtain the target multi-task model, which can more accurately identify high-quality customers in the rejected samples.

[0080] Optionally, the target multi-task model is a progressive layered extraction (PLE) model.

[0081] Generally, after being screened by the risk control system, the default rate of the rejected samples is surely higher than that of the loan samples. However, for the rejected users, "rejection" is a passive action, while "overdue" is the users' active behavior and requires a relatively long manifestation time. Therefore, the correlation between the two tasks of loan prediction and repayment ability prediction in the embodiments of the present invention is weak, but they are not two completely independent tasks.

[0082] Therefore, in the embodiments of the present invention, the target multi-task model can be a Progressive Layered Extraction (PLE) model. The shared layer is divided into multiple experts, and a threshold (gate) is set so that different tasks can use the shared layer in a diversified manner. As Figure 3 shown, Figure 3 is the structural diagram of the multi-task network model according to one embodiment of the present invention. The sample features are input into the shared layer, and the shared layer processes the features common to the two tasks. Then, the expert layer processes the two tasks respectively to obtain the results of the two tasks and output them. That is, the target multi-task model in the embodiments of the present invention can simultaneously output the passing probability and the compliance probability of the sample, so as to more accurately identify high-quality users among the rejected samples.

[0083] Optionally, in step S104, evaluating the target sample through the target multi-task model may include the following execution steps:

[0084] Step S104a: Input the target sample into the target multi-task model to obtain the first probability and the second probability.

[0085] Step S104b: Evaluate the target sample according to the first probability and the second probability.

[0086] Among them, the first probability is the passing probability of the target sample, and the second probability is the compliance probability of the target sample.

[0087] The target sample can be a rejected sample. By inputting the rejected sample into the target multi-task model, the target multi-task model can output the passing probability and the compliance probability of the rejected sample. Evaluating the rejected sample according to the passing probability and the compliance probability of the rejected sample can more accurately identify high-quality users among the rejected samples.

[0088] Optionally, in response to obtaining the qualification information of the training sample, this process further includes the following steps:

[0089] Step S106: Input the target sample into the target multi-task model to obtain the third probability.

[0090] Step S107: Evaluate the target sample according to the first probability, the second probability, and the third probability.

[0091] Among them, the third probability is the qualification passing probability of the target sample.

[0092] The above repayment ability prediction task is to label the repayment ability of users through their repayment behaviors when data related to "income" of users cannot be collected. However, the ultimate goal is to measure the qualifications of users. Therefore, when data related to "income" of users or other data that can be used to measure the qualifications of users (such as education level, user stratification, etc.) can be obtained, a third task can also be added for multi-task learning. The third task is user qualification prediction, and the label y3 of the third task can be directly defined through the education level or income of the user. The specific definition method is not limited in the embodiments of the present invention.

[0093] After adding the third task, the target multi-task model can be as Figure 4 shown. By inputting the target sample into the Figure 4 multi-task network model shown, the results of three tasks can be obtained, that is, the passing probability, the compliance probability, and the qualification passing probability of the target sample can be obtained. Evaluating the target sample according to the passing probability, the compliance probability, and the qualification passing probability of the target sample can further improve the accuracy of identifying high-quality customers.

[0094] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that makes contributions to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0095] In this embodiment, a sample evaluation device is also provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0096] Figure 5 is a structural block diagram of a sample evaluation device according to an embodiment of the present invention, as Figure 5As shown, taking the sample evaluation device 500 as an example, the device includes: a determination module 501 for determining a first task label and a second task label of a training sample, where the first task label is used to predict the passing probability, and the second task label is used to predict the compliance probability; a screening module 502 for performing feature screening on the training sample to obtain a first feature and a second feature; a training module 503 for training an initial multi-task model through the first task label, the second task label, the first feature, and the second feature to obtain a target multi-task model; and an evaluation module 504 for evaluating a target sample through the target multi-task model.

[0097] Optionally, the screening module 502 is further configured to perform feature screening on the training sample through a first model according to a first algorithm to obtain a first feature, where the first algorithm is used to select features with high contribution degrees, and the first feature includes samples with post-loan performance and rejected samples; and perform feature screening on the training sample through a second model according to the first algorithm to obtain a second feature, where the second feature includes samples with post-loan performance.

[0098] Optionally, the training module 503 is further configured to optimize the initial multi-task model according to a first loss function and a second loss function to obtain a target multi-task model, where the first loss function is the loss function of the first model, and the second loss function is the loss function of the second model.

[0099] Optionally, the training module 503 is further configured to use the variances of the first loss function and the second loss function as uncertainty weighting of weights to obtain a target loss function; and optimize the initial multi-task model using the target loss function to obtain a target multi-task model.

[0100] Optionally, the target multi-task model is a Progressive Layered Extraction (PLE) model.

[0101] Optionally, the evaluation module 504 is further configured to input the target sample into the target multi-task model to obtain a first probability and a second probability, where the first probability is the passing probability of the target sample, and the second probability is the compliance probability of the target sample; and evaluate the target sample according to the first probability and the second probability.

[0102] Optionally, in response to obtaining the qualification information of the training sample, the evaluation module 504 is further configured to input the target sample into the target multi-task model to obtain a third probability, where the third probability is the qualification passing probability of the target sample; and evaluate the target sample according to the first probability, the second probability, and the third probability.

[0103] It should be noted that the above-mentioned various modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above-mentioned modules are all located in the same processor; or, the above-mentioned various modules are respectively located in different processors in any combination form.

[0104] An embodiment of the present invention also provides a computer-readable storage medium, in which a computer program is stored. Wherein, the computer program is configured to execute the steps in any one of the above method embodiments when running on a computer or a processor.

[0105] Optionally, in this embodiment, the above computer-readable storage medium may be configured to store a computer program for executing the following steps:

[0106] Step S1: Determine the first task label and the second task label of the training sample;

[0107] Step S2: Perform feature screening on the training sample to obtain the first feature and the second feature;

[0108] Step S3: Train the initial multi-task model through the first task label, the second task label, the first feature and the second feature to obtain the target multi-task model;

[0109] Step S4: Evaluate the target sample through the target multi-task model.

[0110] Optionally, in this embodiment, the above computer-readable storage medium may include but not be limited to: various media such as USB flash drives, read-only memories (ROM), random access memories (RAM), mobile hard disks, magnetic disks or optical discs that can store computer programs.

[0111] An embodiment of the present invention also provides an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0112] Optionally, in this embodiment, the processor in the above electronic device may be configured to run a computer program to execute the following steps:

[0113] Step S1: Determine the first task label and the second task label of the training sample;

[0114] Step S2: Perform feature screening on the training sample to obtain the first feature and the second feature;

[0115] Step S3: Train the initial multi-task model using the first task tag, the second task tag, the first feature, and the second feature to obtain the target multi-task model;

[0116] Step S4: Evaluate the target sample using the target multi-task model.

[0117] Optionally, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation manners, and details thereof will not be elaborated herein.

[0118] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0119] In the above embodiments of the present invention, the descriptions of the various embodiments have their own emphases. For parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0120] In several embodiments provided in the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units can be a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0121] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0122] In addition, in each embodiment of the present invention, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0123] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0124] The foregoing are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A sample evaluation method, characterized in that, Including: Determine a first task label and a second task label for a training sample, where the first task label is used to predict a passing probability, and the second task label is used to predict a compliance probability; Perform feature screening on the training sample to obtain a first feature and a second feature, where the first feature includes samples with post-loan performance and rejected samples, and the second feature includes samples with post-loan performance; Train an initial multi-task model through the first task label, the second task label, the first feature, and the second feature to obtain a target multi-task model; Evaluate a target sample through the target multi-task model; Wherein, the method further includes: Determine a first loss function using the first task label, and determine a second loss function using the second task label, where the first loss function is the loss function of a first model, the second loss function is the loss function of a second model, the first model is trained using the first feature, and the second model is trained using the second feature; use the variances of the first loss function and the second loss function as uncertainty weighting of weights to obtain a target loss function; optimize the initial multi-task model using the target loss function to obtain the target multi-task model.

2. The method according to claim 1, wherein The performing feature screening on the training sample to obtain a first feature and a second feature includes: According to a first algorithm, perform feature screening on the training sample through the first model to obtain the first feature, where the first algorithm is used to select features with high contribution degrees; According to the first algorithm, perform feature screening on the training sample through the second model to obtain the second feature.

3. The method according to any one of claims 1-2, characterized in that, The target multi-task model is a Progressive Layered Extraction (PLE) model.

4. The method according to any one of claims 1-2, characterized in that The evaluating the target sample through the target multi-task model includes: Input the target sample into the target multi-task model to obtain a first probability and a second probability, where the first probability is the passing probability of the target sample, and the second probability is the compliance probability of the target sample; Evaluate the target sample according to the first probability and the second probability.

5. The method according to claim 4, wherein In response to obtaining the qualification information of the training sample, it further includes: Input the target sample into the target multi-task model to obtain a third probability, where the third probability is the qualification passing probability of the target sample; Evaluate the target sample according to the first probability, the second probability, and the third probability.

6. A sample evaluation device, characterized in that, Including: A determination module, where the determination module is used to determine a first task label and a second task label for a training sample, where the first task label is used to predict a passing probability, and the second task label is used to predict a compliance probability; A screening module, where the screening module is used to perform feature screening on the training sample to obtain a first feature and a second feature, where the first feature includes samples with post-loan performance and rejected samples, and the second feature includes samples with post-loan performance; A training module, which is used to train an initial multi-task model through the first task label, the second task label, the first feature, and the second feature to obtain a target multi-task model; An evaluation module, which is used to evaluate a target sample through the target multi-task model; Wherein, the determination module is further configured to: determine a first loss function by using the first task label, and determine a second loss function by using the second task label, wherein the first loss function is the loss function of a first model, the second loss function is the loss function of a second model, the first model is trained by using the first feature, and the second model is trained by using the second feature; The training module is further configured to: use the variances of the first loss function and the second loss function as uncertainty weighting of weights to obtain a target loss function; optimize the initial multi-task model by using the target loss function to obtain the target multi-task model.

7. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program is configured to execute the sample evaluation method described in any one of claims 1 to 5 above when running on a computer or a processor.

8. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the sample evaluation method described in any one of claims 1 to 5 above.

Images