User risk assessment method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a user risk assessment method and device, equipment and a storage medium. The method comprises the following steps: determining a target model set for user risk assessment from a model library; determining at least one risk assessment result corresponding to the at least one machine learning model based on the user data of the target user by using the at least one machine learning model in the target model set; in response to determining that the prediction of the risk confidence of the target user indicated by the risk assessment result has different confidence intervals, calibrating the at least one risk assessment result by using a logistic regression-based model calibration mechanism; and based on a predetermined target model weight, performing weighted fusion on the at least one calibrated risk assessment result to obtain a risk assessment result for the target user. Therefore, user risk assessment is performed by integrating a plurality of machine learning models, and a calibration mechanism is introduced, so that a more accurate risk assessment result is generated.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of information technology, and in particular, to a method, an apparatus, a device, and a computer-readable storage medium for user risk assessment. Background Art

[0002] In the financial industry, accurately assessing user risk is an important part of loan decisions, credit limit determination, and other financial services. User risk scores are often used to measure a user's credit status to support different financial application scenarios, such as cash loans, credit card issuance, and personal loans.

[0003] In related technologies, risk scores of users are usually generated by using algorithms such as logistic regression and decision trees, relying on user data such as historical credit data, repayment behavior, and personal financial information. However, the scoring results obtained directly through the model often have limitations and are difficult to fully reflect the risk characteristics of users. In addition, the existing scoring system is not sufficiently related to actual business scenarios, which limits the effectiveness of the scoring results in business applications.

[0004] Therefore, a more comprehensive and accurate solution for user risk assessment is needed to improve the accuracy and business applicability of the assessment. Summary of the invention

[0005] In the first aspect of the present disclosure, a method for user risk assessment is provided. The method includes: determining a target model set for user risk assessment from a model library, the target model set including at least one machine learning model; using at least one machine learning model in the target model set, based on the user attribute data and user behavior data of the target user, determining at least one risk assessment result corresponding to the at least one machine learning model, each risk assessment result indicating the corresponding machine learning model's prediction of the risk confidence of the target user; in response to determining that the prediction of the risk confidence of the target user indicated by at least one risk assessment result corresponding to at least one machine learning model has different confidence intervals, calibrating at least one risk assessment result using a model calibration mechanism based on logistic regression; performing weighted fusion on at least one calibrated risk assessment result based on a predetermined target model weight to obtain a risk assessment result for the target user; and presenting a resource allocation strategy for the target user based on the risk assessment result for the target user.

[0006] In the second aspect of the present disclosure, a device for user risk assessment is provided. The device includes: a model determination module, configured to determine a target model set for user risk assessment from a model library, the target model set including at least one machine learning model; a risk assessment result determination module, configured to use at least one machine learning model in the target model set, based on the user attribute data and user behavior data of the target user, to determine at least one risk assessment result corresponding to the at least one machine learning model, each risk assessment result indicating the corresponding machine learning model's prediction of the risk confidence of the target user; a calibration module, configured to calibrate at least one risk assessment result using a model calibration mechanism based on logistic regression in response to determining that the prediction of the risk confidence of the target user indicated by at least one risk assessment result corresponding to at least one machine learning model has different confidence intervals; a fusion module, configured to perform weighted fusion on at least one calibrated risk assessment result based on a predetermined target model weight to obtain a risk assessment result for the target user; and a policy module, configured to present a resource allocation policy for the target user based on the risk assessment result for the target user.

[0007] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0009] In a fifth aspect of the present disclosure, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer executable instructions, which when executed by a device cause the device to perform the method according to the first aspect of the present disclosure.

[0010] It should be understood that the content described in this content section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0012] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0013] Figure 2 A flowchart showing a process of user risk assessment according to some embodiments of the present disclosure;

[0014] Figure 3A A schematic diagram showing a process of determining a target model set according to some embodiments of the present disclosure;

[0015] Figure 3B A schematic diagram showing a process of determining a target model set from a first model set according to some embodiments of the present disclosure;

[0016] Figure 3C A schematic diagram showing the model evaluation effect of some embodiments of the present disclosure;

[0017] Figure 3D A schematic diagram showing a comparison of effects after model calibration in some embodiments of the present disclosure;

[0018] Figure 3E A schematic diagram showing the evaluation effect before calibration of some embodiments of the present disclosure;

[0019] Figure 3F A schematic diagram showing the evaluation effect after calibration of some embodiments of the present disclosure;

[0020] Figure 3G A schematic diagram showing credit scoring standards in business scenarios of some embodiments of the present disclosure;

[0021] Figure 3H A schematic diagram showing a model score calibration system of some embodiments of the present disclosure is shown;

[0022] Fig. 3I A schematic diagram showing the effects of the risk assessment method of some embodiments of the present disclosure;

[0023] Figure 4 A block diagram showing an apparatus for user risk assessment according to some embodiments of the present disclosure; and

[0024] Figure 5 A block diagram of an electronic device capable of implementing one or more embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0025] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0026] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0027] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0028] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0029] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0030] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0031] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0032] As briefly mentioned above, in the financial industry, accurately assessing user risk is an important part of loan decisions, credit limit determination, and other financial services. User risk scores are often used to measure a user's credit status to support different financial application scenarios, such as cash loans, credit card issuance, and personal loans.

[0033] In the related technologies, user risk assessment mainly relies on the following technologies: Data-driven scoring: Risk assessment models are usually based on historical credit data, repayment behavior, personal financial information, etc. to generate a score such as delinquency rate to measure the user's credit risk.

[0034] The delinquency rate refers to the proportion of users who fail to fulfill their repayment obligations on time within a certain period of time. It is usually expressed as a percentage and is calculated by dividing the overdue loan amount or number of borrowers by the total loan amount or number of borrowers, and then multiplying by 100. The delinquency rate is used to assess the current repayment situation of borrowers and help institutions determine whether borrowers may have repayment risks. A high delinquency rate may indicate that the user's repayment ability is insufficient or that there are problems with their financial situation.

[0035] Single model assessment: Many institutions use a single scoring model, such as logistic regression, decision tree, etc., to assess users’ credit risk.

[0036] The single model approach may be easy to implement and understand, but it may be insufficient in accuracy and comprehensiveness, and may not be able to fully capture the risk characteristics of users, especially in different business scenarios, where the focus and evaluation criteria of each model may differ. In addition, due to data security and privacy issues, different organizations or enterprises cannot share data features with each other, which further limits the effectiveness of risk control models. Therefore, model fusion may be an effective solution to make up for data deficiencies. By integrating models from different sources or focusing on different business scenarios, and combining multiple data features, the comprehensiveness and accuracy of the model can be improved.

[0037] However, in the related technologies, the method of fusing multiple models may still have the following shortcomings:

[0038] Inconsistent model standards: Different models often have different output score ranges, such as model A's score range is 0-100, and model B's score range is 300-850. If you simply use linear weighting (such as 0.5 score A + 0.5 score B) to merge the scores, the final score will deviate from the actual risk situation, thus affecting the accuracy of the assessment.

[0039] Limitations of tree models: In related technologies, most use tree models (such as decision trees, random forests, etc.) for user risk assessment. Boosting tree is a boosting method that uses classification trees or regression trees as basic classifiers. It actually uses an additive model (i.e., a linear combination of basis functions) and a forward step-by-step algorithm. For classification problems, the base learner can be directly set as a classification tree; for regression problems, the input space can be divided into non-intersecting regions by using the generation algorithm of the regression tree, and the output constant is determined in each region to represent the regression tree model. Although the tree model has fast calculation speed and high accuracy, its predicted posterior probability is usually not accurate enough, especially when the number of bad users (overdue risk users) is small. This is because the construction of the tree model makes it difficult to achieve a situation where the prediction probability is close to 0 or 1.

[0040] Problems caused by downsampling: In the field of risk control, bad users (users with overdue risk) usually account for a small proportion. If the downsampling method is used to balance the data set, the model’s prediction probability for bad users will be inaccurate, which will in turn affect the overdue risk assessment in actual business.

[0041] Therefore, in order to correct the deviation of model prediction and make the final risk assessment results closer to the actual business situation, a more accurate and efficient solution for user risk assessment is needed, which can make full use of the advantages of different models, generate more accurate risk assessment results, and calibrate the risk assessment results, so as to unify the scoring range and scale of different models.

[0042] In an embodiment of the present disclosure, a scheme for user risk assessment is provided. According to an embodiment of the present disclosure, a target model set for user risk assessment is determined from a model library, the target model set including at least one machine learning model; at least one risk assessment result corresponding to at least one machine learning model is determined based on user attribute data and user behavior data of the target user using at least one machine learning model in the target model set, each risk assessment result indicates that the target user corresponding to the corresponding machine learning model has a confidence level of risk; in response to determining that at least one risk assessment result corresponding to at least one machine learning model has different confidence standards, at least one risk assessment result is calibrated using a model calibration mechanism based on logistic regression; based on a predetermined target model weight, weighted fusion is performed on at least one calibrated risk assessment result to obtain a risk assessment result for the target user; and based on the risk assessment result for the target user, a resource allocation strategy for the target user is presented.

[0043] Therefore, by using the machine learning model in the target model set, at least one risk assessment result is determined, thereby performing user risk assessment in various business scenarios. In addition, by introducing a model calibration mechanism, the risk assessment results are calibrated, thereby unifying the range and scale of the assessment results of different models, reducing the error in the score conversion.

[0044] Figure 1 A schematic diagram of an example environment 100 in which an embodiment of the present disclosure can be implemented is shown. In the environment 100, the electronic device 110 determines a target model set for user risk assessment from a model library. The target model set includes at least one machine learning model 130-1, 130-2...130-N (collectively or individually referred to as a machine learning model 130). The electronic device 110 can use at least one machine learning model 130 in the target model set to determine at least one risk assessment result 140-1, 140-2...140-N (collectively or individually referred to as a risk assessment result 140) corresponding to the at least one machine learning model 130 based on the user data 120 of the target user. In some embodiments, the electronic device 110 can use a model calibration mechanism based on logistic regression to calibrate the risk assessment result 140 and perform weighted fusion on the at least one calibrated risk assessment result 140 based on a predetermined target model weight to obtain a target risk assessment result 150 for the target user.

[0045] In the environment 100, the electronic device 110 can be any type of device with computing capabilities, including a terminal device or a server device. The terminal device can be any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a gaming device, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The server device can include, for example, a computing system / server, such as a mainframe, an edge computing node, a computing device in a cloud environment, and the like.

[0046] It should be understood that the structure and functionality of environment 100 are described for exemplary purposes only and does not imply any limitation on the scope of the present disclosure.

[0047] Figure 2 1 is a flowchart of a process 200 for user risk assessment according to some embodiments of the present disclosure. The process 200 may be implemented in the environment 100, for example, the process 200 may be implemented at the electronic device 110. In the following, for ease of discussion, reference will be made to Figure 1 The process 200 is described below.

[0048] In block 210 , the electronic device 110 determines a target model set for user risk assessment from a model library.

[0049] In some embodiments, there may be multiple pre-trained machine learning models 130 for different business scenarios and / or user data in the model library. Different models may show different effects on different data sets and different features. The electronic device 110 may determine a target model set from the model library before performing a risk assessment for a target user. The target model set includes at least one machine learning model 130.

[0050] Figure 3A FIG. 3 is a schematic diagram of a process 300A for determining a target model set according to some embodiments of the present disclosure. The process 300A may be implemented at the electronic device 110 .

[0051] In block 311 , the electronic device 110 determines a test data set based on historical user data.

[0052] In some embodiments, historical user data includes historical user attribute data and historical user behavior data. Historical user attribute data can help the model identify the basic risk characteristics of users. Different combinations of user attributes may mean different risk levels. The historical user behavior data model can understand the user's payment habits, loan repayment status, borrowing frequency, etc., so as to better predict the user's future default risk.

[0053] In some embodiments, the electronic device 110 may further sample a portion of the user data 120 as a test data set.

[0054] In block 312 , the electronic device 110 obtains a first model set for user risk assessment from a model library.

[0055] In some embodiments, the electronic device 110 obtains a first model set for user risk assessment from a plurality of pre-trained machine learning models for different business scenarios and / or user data in a model library.

[0056] In box 313, the electronic device 110 determines the category of each machine learning model in the first model set based on a predetermined category label.

[0057] In some embodiments, the categories of each machine learning model include at least a first category for risk prediction before the user's first behavior and a second category for risk prediction after the user's first behavior. For example, the first behavior may be a loan behavior, and the period before the user's first behavior refers to the pre-loan stage, and the period after the first behavior refers to the mid-loan stage.

[0058] As an example, the first category of machine learning models used for risk prediction in the pre-loan stage refers to machine learning models used to assess the credit risk of users before they apply for loans. This type of model is usually based on preliminary data related to the loan, such as the user's historical credit record, income level, debt status, etc., and focuses on the borrower's historical behavior and current financial status.

[0059] As another example, the second category of machine learning models used for risk prediction during the loan process refers to machine learning models used to monitor and assess risks based on the user's repayment behavior after the loan has been issued. This type of model focuses on the user's performance during the repayment period, including their ability to repay on time, whether there are delays or defaults, etc. The second category of machine learning models can be analyzed based on real-time data such as the user's repayment history data, the user's financial changes during the loan period, and consumption behavior, to help financial institutions take timely measures when risk signals are discovered, such as adjusting repayment plans, collection or other management strategies.

[0060] In other embodiments, the categories of each machine learning model in the first model set may also include more, fewer, or different other categories.

[0061] In box 314, the electronic device 110 determines a target model set from the first model set based on the risk assessment result of the test data set and / or the category of each machine learning model.

[0062] In some embodiments, different machine learning models may show different effects on different data sets and different features. Therefore, the electronic device 110 can pre-test the effects of each machine learning model in the first model set based on a predetermined test data set. The target model set ultimately used for user risk assessment is determined based on the risk assessment results for the test data set and based on the categories of each machine learning model.

[0063] In the disclosed embodiment, by determining the category of each machine learning model and conducting preliminary testing based on a test data set, a target model set is determined to ensure that the selected machine learning model is suitable for the current risk assessment task, avoid introducing poorly performing models, and dynamically adapt to changes in user data and optimize the effect of model evaluation, thereby providing more accurate risk assessment in practical applications.

[0064] Figure 3B A schematic diagram of a process 300B for determining a target model set from a first model set according to some embodiments of the present disclosure is shown. The process 300B may be implemented at the electronic device 110 .

[0065] In box 321, the electronic device 110 inputs the test data set into each machine learning model in the first model set to determine the output results of each machine learning model.

[0066] In some embodiments, the electronic device 110 may sample a portion of samples from the user data as a test data set, input them into each machine learning model in a predetermined first model set, and obtain the output results of each machine learning model (i.e., risk assessment results).

[0067] In box 322, the electronic device 110 determines, based on the output results of each machine learning model, whether the output result indicates that there is a machine learning model in the first model set that does not output a risk assessment result.

[0068] In some embodiments, the electronic device 110 first checks whether each machine learning model in the first model set can effectively assess the risk of the user, that is, checks whether these models "hit" the user data. This method can quickly determine the most effective scoring source, avoid unnecessary calculations and model fusion steps, and thus improve scoring efficiency.

[0069] In box 323, the electronic device 110 determines a target model set based on at least one machine learning model in the first model set that outputs a risk assessment result.

[0070] In some embodiments, when the electronic device 110 determines that at least one machine learning model does not output a risk assessment result for the user (i.e., a miss), and at least one machine learning model successfully outputs an assessment result (i.e., a hit), the electronic device 110 can directly determine at least one machine learning model that outputs a risk assessment result for the user as a machine learning model in the target model set.

[0071] In some embodiments, if the electronic device 110 determines that there is no machine learning model in the first model set that does not output a risk assessment result, that is, all models are hits, the electronic device 110 can determine the target model set based on the categories of each machine learning model in the first model set.

[0072] In box 324, the electronic device 110 determines whether there is at least one machine learning model of the first category and at least one machine learning model of the second category in the first model set.

[0073] If it is determined that there is at least one first-category machine learning model and at least one second-category machine learning model in the first model set, box 325 is executed. In box 325, the electronic device 110 determines a target model set based on at least one second-category machine learning model in the first model set.

[0074] In some embodiments, when each machine learning model in the first model set can output a risk assessment result for the user, the electronic device 110 can preferentially select the second category of loan models as the target model set. The reason is that the data of the second category of machine learning models is more real-time and directly reflects the user's lending behavior, and thus can more accurately predict the user's overdue risk.

[0075] In the disclosed embodiment, by giving priority to the evaluation results of the second category of machine learning models, the actual credit risk of the user can be more accurately reflected. This is because the second category of machine learning models is based on more dynamic and real-time behavioral data, which can better evaluate the risk situation of the user.

[0076] In some embodiments, the machine learning models in the first model set may also be: machine learning models of the first category trained with different algorithms or different feature sets; machine learning models of the second category trained with different algorithms or different feature sets; or other types of machine learning models based on different data sources or evaluation targets. At this time, the electronic device 110 may execute block 326.

[0077] In box 326, the electronic device 110 can determine the target machine learning model based on the evaluation effects of each machine learning model in the first model set.

[0078] In some embodiments, the evaluation effect of the machine learning model may refer to the results of measuring and comparing the performance of different models through a series of indicators. These indicators may include but are not limited to the prediction accuracy, recall rate, F1 value (F1Score), area under the receiver operating characteristic curve (Receiver Operating Characteristic Curve, ROC curve), etc. of the model.

[0079] As an example, when the electronic device 110 determines that the machine learning model 130-1 is significantly better than the machine learning model 130-2 in terms of evaluation effect, and the fusion model generated by fusing the machine learning model 130-1 with the machine learning model 130-2 is not as good as the machine learning model 130-1 alone, this indicates that the machine learning model 130-2 has generated a negative gain for the machine learning model 130-1, that is, the fused model does not perform as well as the single model machine learning model 130-1 in terms of prediction accuracy or other key indicators. In this case, the electronic device 110 can choose to directly determine the machine learning model 130-1 as the machine learning model in the target model set.

[0080] As another example, Figure 3CA schematic diagram of the model evaluation effect 300C of some embodiments of the present disclosure is shown. The first model set includes a machine learning model 130-1 constructed based on the data of financial platform A and the LightGBM algorithm of the Gradient Boosting Decision Tree (GBDT) and a machine learning model 130-2 constructed based on the data of bank B and the LightGBM algorithm. And the receiver operating characteristic curve (Receiver Operating Characteristic Curve, ROC curve) is used to quantify the effect of the model's user risk assessment.

[0081] In some embodiments, the performance of the model under different thresholds is shown by the ROC curve. The horizontal axis is the false positive rate (1-Specificity), and the vertical axis is the true positive rate (Sensitivity). The closer the curve is to the upper left corner, the better the evaluation effect of the model. The area under the ROC curve (Area Under the Curve, AUC) represents the distinguishing ability of the model. The larger the AUC, the stronger the distinguishing ability of the model.

[0082] like Figure 3C As shown, the ROC curve of the machine learning model 130-1 is 331, and its corresponding AUC is 0.614. The ROC curve of the machine learning model 130-2 is 332, and its corresponding AUC is 0.616, which means that the evaluation effects of the two are basically equivalent. At this time, the electronic device 110 can determine all the machine learning models of the evaluation effect as the machine learning models in the target model set. For example, the electronic device 110 can obtain a fusion model based on all the features of the machine learning model 130-1 and the machine learning model 130-2, and its ROC curve is 333, and the corresponding AUC is 0.669, which is significantly higher than the individual machine learning model 130-1 or the machine learning model 130-2. This shows that the fusion model is superior to the model of a single data source in terms of distinguishing ability because it integrates more data and contains more information.

[0083] Continue to refer Figure 2 In box 220, the electronic device 110 uses at least one machine learning model 130 in the target model set to determine at least one risk assessment result corresponding to the at least one machine learning model based on the target user attribute data and the user behavior data.

[0084] In some embodiments, the electronic device 110 determines the relevant data of the target user to be evaluated, such as the user's historical behavior data, credit record, information provided when applying for a loan, and post-loan repayment behavior. These user data will be input as input data to each machine learning model in the determined target model set. Each machine learning model in the target model set will execute the corresponding algorithm based on the input user data to calculate the risk assessment result of the corresponding user.

[0085] In some embodiments, the risk assessment result indicates the corresponding machine learning model's prediction of the target user's risk confidence, representing the possibility of the user's credit risk or overdue risk. For example, the risk assessment result output by the model can be a quantitative value, representing the model's comprehensive assessment of the user's risk. Generally, the higher the risk score, the better the user's credit or repayment ability; the lower the risk score, the higher the user's default risk. The risk assessment results output by different machine learning models may have different ranges, such as a score range from 300 to 850.

[0086] In some embodiments, since the risk assessment results can be generated by different machine learning models, such as decision tree models, random forest models, etc. based on machine learning. Each model can be trained based on different data sources and use different methods and features to calculate the user's risk score. Therefore, the range of risk assessment results output by each machine learning model may be different (for example, the score range of one model may be 0 to 100, and the range of another model is 300 to 850), and directly fusing these assessment results will produce errors. In this case, the electronic device 110 can calibrate these risk assessment results.

[0087] Figure 3D FIG. 300D is a schematic diagram showing the effect comparison of the model calibration of some embodiments of the present disclosure. Figure 3D As shown, the horizontal axis represents the probability score predicted by the model (from 0 to 1). The vertical axis represents the proportion of actual positive samples (fraction of positives), that is, the proportion of events that actually occurred under a given predicted probability. Among them, the performance comparison after probability calibration includes: Logistic regression model 341: the dotted line closest to the ideal calibration. Since logistic regression itself is a generalized linear model, the probability value it outputs is naturally calibrated, and the predicted probability is closest to the actual probability, so it performs most accurately in the calibration curve.

[0088] Naive Bayes model 342: shows significant bias, especially in the medium probability range (0.4-0.6). The Naive Bayes model assumes that the features are independent of each other, but this is often not the case, which may lead to bias in the predicted probabilities.

[0089] Linear Support Vector Machine (Linear SVC) model 343: There are also some deviations in performance. Although the support vector machine can provide good classification boundaries, the scores it outputs in the original state are not of the same interval standard, and after calibration, it may still not be as accurate as logistic regression.

[0090] Random Forest Model 344: Prediction bias is large, especially in low and high probability areas. Random Forest is an ensemble model based on multiple decision trees, which itself has biases in probability estimation, especially when the class distribution is uneven.

[0091] In some embodiments, based on the performance of different machine learning models after calibration, the prediction probability of the logistic regression model 341 is the most accurate. Therefore, the electronic device 110 can use the model calibration mechanism based on logistic regression to calibrate the risk assessment results output by the model.

[0092] Continue to refer Figure 2 In box 230, in response to determining that at least one risk assessment result corresponding to at least one machine learning model indicates that the prediction of the risk confidence level of the target user has different confidence intervals, the electronic device 110 calibrates at least one risk assessment result using a model calibration mechanism based on logistic regression.

[0093] In some embodiments, the electronic device 110 determines that at least one risk assessment result corresponding to at least one machine learning model indicates that the prediction of the risk confidence level of the target user has different confidence intervals, which may mean that the risk assessment results obtained by different machine learning models have different scoring scales (for example, the scoring range of one model is 0 to 100, and the other is 300 to 850, which means that they use different scoring criteria to measure the user's risk.), thereby resulting in different confidence intervals for the prediction of the risk confidence level of the target user.

[0094] In some embodiments, the electronic device 110 can apply a model calibration mechanism based on logistic regression (e.g., PlattScaling method) to perform probability calibration based on the risk assessment results output by each machine learning model (e.g., tree model) in the target model set. The model calibration method based on logistic regression is a post-processing method that converts the original output of the tree model into a probability value that is closer to the actual situation by fitting a simple logistic regression model.

[0095] In some embodiments, the electronic device 110 inputs the at least one risk assessment result into a trained logistic regression model. Using the logistic regression model, the electronic device 110 converts the at least one risk assessment result into at least one risk assessment result corresponding to the same interval standard.

[0096] As an example, a logistic regression model can be trained as follows:

[0097] First, the electronic device 110 determines a training data set, which includes known true labels (e.g., whether the user is overdue or defaulted) and the risk assessment results before calibration output by the model. These risk assessment results can be risk scores in continuous numerical types, which may have different interval scales.

[0098] Then, the risk assessment results before calibration output by the model are used as the input features of the logistic regression model, and the known true labels are used as the targets of the logistic regression model. The logistic regression model is trained to fit the relationship between the output assessment results and the true probability. By continuously optimizing the parameters of the logistic regression model, the output of the logistic regression model is made closest to the actual probability, and finally a trained logistic regression model is obtained. For example, cross-validation or other evaluation methods are usually used to select the best parameter combination.

[0099] In some embodiments, the electronic device 110 uses a trained logistic regression model to map the risk assessment results output by each machine learning model in the target model set to a probability range of [0,1]. After calibration, the assessment results of each model are uniformly converted into a probability value. These probability values ​​are easier to compare and merge because they are all on the same scale.

[0100] As an example, Figure 3E A schematic diagram showing an evaluation effect 300E before calibration according to some embodiments of the present disclosure is shown. Figure 3E The model calibration mechanism based on logistic regression of the electronic device 110 shows the reliability curve before calibration of the risk assessment results output by the machine learning model 130-1 and the machine learning model 130-2. Among them, curves 351 and 352 are the reliability curves of the machine learning model 130-1 and the machine learning model 130-2 before calibration. It can be seen that before calibration, it deviates from the black dotted line representing the ideal situation, which indicates that there is a large error between the risk assessment results obtained by the model evaluation and the actual situation.

[0101] Figure 3F A schematic diagram showing a calibrated evaluation effect 300F according to some embodiments of the present disclosure is shown. Figure 3FThe electronic device 110 shows a reliability curve after calibrating the risk assessment results output by the machine learning model 130-1 and the machine learning model 130-2 based on the model calibration mechanism of logistic regression. Curves 353 and 354 are the reliability curves of the two models after calibration (after calibration) by the model calibration mechanism based on logistic regression. It can be seen that they are significantly closer to the ideal curve, which means that the risk assessment results output by the calibrated machine learning model are more consistent with the actual situation. This improvement is particularly important for machine learning models used in the field of financial risk control, and can significantly improve the effect and accuracy of user risk assessment in different business scenarios.

[0102] In some embodiments, for multi-classification problems in actual business scenarios, that is, the model not only needs to distinguish between two categories of whether the user defaults, but also needs to make predictions between multiple categories. The electronic device 110 can also use a model calibration mechanism based on isotonic regression to calibrate at least one risk assessment result. Isotonic Regression is a non-parametric probability calibration method suitable for regression analysis of data with monotonicity constraints. When the risk assessment results output by the model need to meet monotonicity, Isotonic Regression can be used to replace the calibration method of logistic regression. It is applicable to binary and multi-classification problems and does not depend on any specific probability model.

[0103] In the disclosed embodiment, the electronic device 110 unifies the output standards of different models to the same scale by using a trained logistic regression model, making it easier to compare and integrate the outputs of different models, and can reduce the overall error caused by the range differences and errors in the evaluation results, thereby improving the prediction accuracy of the model.

[0104] Continue to refer Figure 2 In box 240, the electronic device 110 performs weighted fusion on at least one calibrated risk assessment result based on a predetermined target model weight to obtain a risk assessment result for the target user.

[0105] In some embodiments, after calibration, each machine learning model in the target model set will output an independent risk assessment result (or risk score) for the target user. These assessment results reflect the assessment of the user's credit risk by each model based on its own characteristics and algorithms. However, in the field of risk control, downsampling is usually used to solve the problem of class imbalance. The output of the model may lead to inaccurate assessment of minority classes (such as defaulting users) because downsampling reduces the model's learning of the majority class, thereby affecting the stability and accuracy of the overall model.

[0106] In this case, the electronic device 110 can balance the advantages and disadvantages of different models and improve the accuracy of the output results by integrating the risk assessment results output by multiple machine learning models (rather than relying solely on a single machine learning model).

[0107] In some embodiments, in the process of fusing the output scores of multiple machine learning models, in order to optimize the overall prediction effect, different weights need to be assigned to each model. The weight reflects the contribution of each model to the final risk assessment result. By performing weighted fusion on the risk assessment results output by each machine learning model and calibrated based on the weight of each machine learning model in the target model set indicated by the target model weight, the prediction effect of the fusion model is optimized.

[0108] In some embodiments, the electronic device 110 performs weighted fusion on the calibrated risk assessment results output by each machine learning model based on the weights of each machine learning model in the target model set indicated by the target model weight, and determines the risk assessment result for the target user.

[0109] In the disclosed embodiment, by performing weighted fusion on the calibrated risk assessment results output by each machine learning model based on the target model weight, errors that may be caused by overfitting of a single model can be avoided, and the generalization ability in different data sets and business scenarios can be improved, thereby improving the accuracy of user risk assessment.

[0110] In some embodiments, the electronic device 110 performs weighted fusion on at least one risk assessment result output by each machine learning model and calibrated based on each model weight in a predetermined model weight combination.

[0111] In some embodiments, the electronic device 110 may obtain some model weight combinations predetermined based on empirical values. Through different model weight combinations, weighted fusion is performed on at least one risk assessment result output by each machine learning model and calibrated, and the fusion result is tested and verified. For example, by using a permutation and combination method, the electronic device 110 can systematically test various possible weight configurations. This method ensures that all potential optimal weight combinations are taken into account.

[0112] In some embodiments, the electronic device 110 determines the target model weight from a predetermined model weight combination based on the results of weighted fusion corresponding to the various model weights.

[0113] In some embodiments, the electronic device 110 can evaluate the effect of the weighted fusion result corresponding to each weight combination. This usually includes using specific indicators (such as AUC, accuracy, recall, etc.) to evaluate the performance of the model on the validation data set, with the goal of finding the target model weight that can maximize these indicators.

[0114] In the disclosed embodiment, by testing the predetermined model weight combination and determining the target model weight, the evaluation effect of the machine learning model can be optimized so that it can accurately reflect the user's risk situation and adapt to different business scenarios. This method can fully utilize the advantages of each model while reducing the deviation and error that may exist in a single model.

[0115] In some embodiments, after performing weighted fusion on at least one calibrated risk assessment result to obtain a risk assessment result for a target user, the electronic device 110 may also re-calibrate the risk assessment result based on a credit scoring standard in a business scenario to determine a calibrated risk assessment result for the target user.

[0116] In some embodiments, different business scenarios (such as cash loans, credit cards, etc.) have different risk management standards and user group characteristics. For example, in the cash loan business, users in different score segments have specific risk distribution characteristics, which are summarized based on long-term historical data and industry experience. Based on the credit scoring standards in the business scenario, the purpose of recalibrating the risk assessment results for the target user is to make the output results of the model (i.e., risk score) consistent with the empirical risk level (such as bad debt rate) in these actual business scenarios.

[0117] As an example, Figure 3G A schematic diagram of a credit scoring standard 300G in a business scenario of some embodiments of the present disclosure is shown. Figure 3G As shown, curve 361 is the relationship between the risk assessment result of the user output by the model and the user's default probability. Curve 362 represents the empirical bad debt rate of different user score segments in the actual business scenario A under the personal credit score FICO (Fair Isaac Corporation) score system.

[0118] FICO score is a commonly used credit scoring model. It is based on statistical analysis methods and provides a comprehensive credit score by modeling and analyzing the credit data of individuals or institutions. Curve 362 reflects the risk situation of each score segment based on long-term accumulated historical data and business experience, and is the standard that the expected model can achieve. Table 363 shows the distribution density and corresponding bad debt rate of different segments of user scores in actual business scenario A. For example, users with a score segment of 300-499 have an experience bad debt rate of 87%, while users with a score segment of 800-850 have a bad debt rate of only 1%.

[0119] The electronic device 110 can map the risk assessment results output by the model to the FICO score range. For example, the risk assessment results output by the model for a user risk score of 0-100 can be proportionally mapped to FICO scores of 300-850. The electronic device 110 can then adjust the model score by analyzing the difference between the user default probability corresponding to the risk score output by the model and the empirical bad debt rate corresponding to the FICO score. For those score segments that overestimate the risk, the electronic device 110 can appropriately lower the risk assessment results (risk scores); for those score segments that underestimate the risk, the electronic device 110 can appropriately increase the risk assessment results (risk scores), with the goal of making the risk assessment results output by the model as close to the actual business scenario as possible.

[0120] In some embodiments. Based on the credit scoring standard in the business scenario, after recalibrating the target risk assessment result, the electronic device 110 can also adjust the structure, feature selection, parameter setting, etc. of the machine learning model in the target model set based on the recalibration result. For example, if it is found that the calibration deviation of a certain score segment is large, it may indicate that the model's feature capture capability in this score segment is insufficient. At this time, you can consider introducing new features or adjusting the model's training method. And after each calibration and model adjustment, the electronic device 110 can apply the new model to actual business and continuously monitor its performance. After the accumulation of new data, similar calibration and optimization are performed again to gradually improve the precision and accuracy of the model's risk assessment.

[0121] In the disclosed embodiment, the risk assessment results for the target users are recalibrated based on the credit scoring standards in the business scenario to ensure that the output results of the model are consistent with the actual business needs. This process not only makes the output of the model closer to the actual risk level, but also serves as an important basis for iterative optimization of the model, improving the precision and accuracy of the risk assessment of the model.

[0122] Return to reference Figure 2 In block 250 , the electronic device 110 may present a resource allocation strategy for the target user based on the risk assessment result for the target user.

[0123] In some embodiments, the electronic device 110 may not only simply output the risk assessment results for the target user, but may also dynamically generate a resource allocation strategy for the target user based on the risk assessment results of the target user in combination with the business scenario and user characteristics, such as lending strategy, credit limit adjustment, insurance product pricing, interest rate setting, etc. This strategy can help financial institutions implement personalized resource allocation plans for users according to different risk levels, ensuring risk control while maximizing business benefits.

[0124] The method for user risk assessment in the embodiment of the present disclosure is described below with examples. The model score calibration system is used as the electronic device 110 to execute the method for user risk assessment in the present disclosure, model 1 and model 2 are used as machine learning models for user risk assessment in the target model set, and FICO business calibration is used as the credit scoring standard in the business scenario.

[0125] As an example, Figure 3H FIG. 3 is a schematic diagram of a model score calibration system 300H according to some embodiments of the present disclosure. Figure 3H As shown, the model score calibration system performs model effect logical inference by executing box 371, and determines model 1 and model 2 for user risk assessment in the target model set from the model library. A risk score is calculated based on model 1 and model 2 respectively. Next, the model score calibration system executes box 372 to calibrate the risk scores output by model 1 and model 2 by using the Platt Scaling method. This method can map the risk score output by the model to a probability score within the same interval label ([0,1]). The model score calibration system executes box 373 to fuse the calibrated probability scores of each model through a weight combination method. The model score calibration system executes box 374 to further calibrate the fused probability score based on the FICO scoring system. Finally, the model score calibration system executes box 375 to convert the secondary calibrated probability score into the risk score of the target customer and output it.

[0126] As another example, Fig. 3I A schematic diagram of the effect 300I of the risk assessment method of some embodiments of the present disclosure is shown, and the performance of the original model is specifically compared with the performance of the model after the Platt Scaling calibration and the credit scoring standard calibration in the business scenario.

[0127] like Fig. 3IAs shown, curve 381 represents the result of simply adding machine learning model 1 and machine learning model 2 after multiplying each by a weight of 0.5 without the calibration mechanism of logistic regression. Although the AUC value (0.6198) of curve 381 is relatively high, the risk score output by it may not be consistent with the actual business situation due to the lack of calibration. This means that in actual applications, although it seems to be more effective, it may cause deviations in the actual risk assessment.

[0128] Curve 382 and curve 383 represent the results of machine learning model 1 and machine learning model 2 after the calibration mechanism of logistic regression, using weights of 0.5, 0.5 and 0.7, 0.3, respectively, and the AUC values ​​are 0.6197 and 0.6189, respectively. After Platt Scaling calibration, the probability output of the model is more consistent with the actual risk probability distribution.

[0129] Curve 384 shows the results of machine learning model 1 and machine learning model 2 after the calibration mechanism of logistic regression and the credit scoring standard in the business scenario. Its AUC value is the highest (0.6270), which means that it performs best in risk assessment for target users. Since it combines the calibration of scoring meaning in actual business, this method not only improves the accuracy of prediction, but also ensures that the score output by the model is consistent with the actual business risk situation, making it more usable in practice.

[0130] By comparison, we can find that using appropriate calibration methods, such as Platt Scaling and calibration based on credit scoring standards in business scenarios, can achieve a better balance between model effects and actual business needs. This makes the final output risk score not only have better differentiation ability, but also more in line with business expectations.

[0131] Figure 4 A block diagram of an apparatus 400 for user risk assessment according to some embodiments of the present disclosure is shown. The apparatus 400 may be implemented as or include the electronic device 110.

[0132] The device 400 includes a model determination module 410, which is configured to determine a target model set for user risk assessment from a model library, the target model set including at least one machine learning model; a risk assessment result determination module 420, which is configured to use at least one machine learning model in the target model set to determine at least one risk assessment result corresponding to the at least one machine learning model based on user attribute data and user behavior data of the target user, each risk assessment result indicating a prediction of the risk confidence of the target user by the corresponding machine learning model; a calibration module 430, which is configured to calibrate at least one risk assessment result using a model calibration mechanism based on logistic regression in response to determining that at least one risk assessment result corresponding to at least one machine learning model indicates a prediction of the risk confidence of the target user with different confidence intervals; a fusion module 440, which is configured to perform weighted fusion on at least one calibrated risk assessment result based on a predetermined target model weight to obtain a risk assessment result for the target user; and a policy module 450, which is configured to present a resource allocation policy for the target user based on the risk assessment result for the target user.

[0133] In some embodiments, the model determination module 410 is further configured to determine a test data set based on historical user data, the historical user data including historical user attribute data and historical user behavior data; obtain a first model set for user risk assessment from a model library; determine the category of each machine learning model in the first model set based on predetermined category labels, the category of each machine learning model including at least a first category for performing risk prediction before the user's first behavior and a second category for performing risk prediction after the user's first behavior; and determine a target model set from the first model set based on the risk assessment results for the test data set and / or the category of each machine learning model.

[0134] In some embodiments, the model determination module 410 is further configured to input a test data set into each machine learning model in the first model set to determine the output results of each machine learning model; if the output result indicates that there is a machine learning model in the first model set that does not output a risk assessment result, then based on at least one machine learning model in the first model set that outputs a risk assessment result, determine the target model set; and if the output result indicates that there is no machine learning model in the first model set that does not output a risk assessment result, then based on the categories of each machine learning model in the first model set, determine the target model set.

[0135] In some embodiments, the model determination module 410 is further configured to determine the target model set based on at least one machine learning model of the second category in the first model set if it is determined that the first model set includes at least one machine learning model of the first category and at least one machine learning model of the second category.

[0136] In some embodiments, the calibration module 430 is further configured to input at least one risk assessment result into a trained logistic regression model; and utilize the logistic regression model to convert at least one risk assessment result into at least one risk assessment result corresponding to the same interval standard.

[0137] In some embodiments, the fusion module 440 is further configured to perform weighted fusion on the calibrated risk assessment results output by each machine learning model based on the weights of each machine learning model in the target model set indicated by the target model weight, so as to determine the risk assessment result for the target user.

[0138] In some embodiments, the fusion module 440 is further configured to perform weighted fusion on at least one risk assessment result output by each machine learning model and calibrated based on each model weight in a predetermined model weight combination; and determine the target model weight from the predetermined model weight combination based on the result of the weighted fusion corresponding to each model weight.

[0139] In some embodiments, the apparatus 400 includes a statement second calibration module 450, which is configured to calibrate the risk assessment result for the target user based on the credit scoring standard in the business scenario, and determine the calibrated risk assessment result for the target user.

[0140] The modules included in the device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the units in the device 400 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0141] Figure 5 1 shows a block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 5The electronic device 500 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

[0142] like Figure 5 As shown, the electronic device 500 is in the form of a general electronic device. The components of the electronic device 500 may include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. The processing unit 510 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 520. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 500.

[0143] The electronic device 500 typically includes a plurality of computer storage media. Such media may be any available media accessible to the electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 520 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 500.

[0144] The electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 5 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 520 may include a computer program product 525 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0145] The communication unit 540 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 500 can be implemented in a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 500 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0146] The input device 550 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 560 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 500 may also communicate with one or more external devices (not shown) through the communication unit 540 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 500, or communicate with any device that allows the electronic device 500 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0147] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which one or more computer instructions are stored, wherein the one or more computer instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0148] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0149] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0150] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0151] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0152] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the marketplace, or to enable other persons of ordinary skill in the art to understand the implementations disclosed herein.

Claims

1. A method for user risk assessment, comprising: Determining a target model set for user risk assessment from a model library, wherein the target model set includes at least one machine learning model; Using at least one machine learning model in the target model set, based on user attribute data and user behavior data of the target user, determine at least one risk assessment result corresponding to the at least one machine learning model, each risk assessment result indicating a prediction of the risk confidence of the target user by the corresponding machine learning model; In response to determining that the prediction of the risk confidence of the target user indicated by the at least one risk assessment result corresponding to the at least one machine learning model has different confidence intervals, calibrating the at least one risk assessment result using a model calibration mechanism based on logistic regression; Based on a predetermined target model weight, performing weighted fusion on at least one calibrated risk assessment result to obtain a risk assessment result for the target user; as well as Based on the risk assessment result for the target user, a resource allocation strategy for the target user is presented.

2. The method according to claim 1, wherein determining a target model set for user risk assessment from a model library comprises: Determine a test data set based on historical user data, wherein the historical user data includes historical user attribute data and historical user behavior data; Acquire a first model set for user risk assessment from a model library; Based on a predetermined category label, determine the category of each machine learning model in the first model set, wherein the category of each machine learning model includes at least a first category for performing risk prediction before a first behavior of a user and a second category for performing risk prediction after the first behavior of the user; as well as Based on the risk assessment result of the test data set and / or the category of each machine learning model, the target model set is determined from the first model set.

3. The method according to claim 2, wherein based on the risk assessment result for the test data set and / or the category of each machine learning model, determining the target model set from the first model set comprises: Inputting the test data set into each machine learning model in the first model set to determine an output result of each machine learning model; If the output result indicates that there is a machine learning model in the first model set that does not output a risk assessment result, determining the target model set based on at least one machine learning model in the first model set that outputs a risk assessment result; as well as If the output result indicates that there is no machine learning model in the first model set that does not output a risk assessment result, the target model set is determined based on the categories of each machine learning model in the first model set.

4. The method according to claim 3, wherein determining the target model set based on the category of each machine learning model in the first model set comprises: If it is determined that the first model set includes at least one machine learning model of the first category and at least one machine learning model of the second category, the target model set is determined based on at least one machine learning model of the second category in the first model set.

5. The method according to claim 1, wherein calibrating the at least one risk assessment result using a model calibration mechanism based on logistic regression comprises: inputting the at least one risk assessment result into a trained logistic regression model; as well as The at least one risk assessment result is converted into at least one risk assessment result corresponding to the same interval standard using the logistic regression model.

6. The method according to claim 1, wherein performing weighted fusion on at least one calibrated risk assessment result based on a predetermined target model weight to obtain a risk assessment result for the target user comprises: Based on the weights of each machine learning model in the target model set indicated by the target model weight, weighted fusion is performed on the calibrated risk assessment results output by each machine learning model to determine the risk assessment result for the target user.

7. The method of claim 1 , further comprising determining the target model weight by: Based on each model weight in a predetermined model weight combination, performing weighted fusion on at least one risk assessment result output by each machine learning model and calibrated; and Based on the results of weighted fusion corresponding to the various model weights, the target model weight is determined from the predetermined model weight combination.

8. The method according to claim 1, further comprising: Based on the credit scoring standard in the business scenario, the risk assessment result for the target user is calibrated to determine the calibrated risk assessment result for the target user.

9. A device for user risk assessment, comprising: A model determination module is configured to determine a target model set for user risk assessment from a model library, wherein the target model set includes at least one machine learning model; a risk assessment result determination module, configured to use at least one machine learning model in the target model set to determine at least one risk assessment result corresponding to the at least one machine learning model based on user attribute data and user behavior data of the target user, each risk assessment result indicating a prediction of the risk confidence of the target user by the corresponding machine learning model; a calibration module configured to, in response to determining that the prediction of the risk confidence of the target user indicated by the at least one risk assessment result corresponding to the at least one machine learning model has different confidence intervals, calibrate the at least one risk assessment result using a model calibration mechanism based on logistic regression; a fusion module configured to perform weighted fusion on at least one calibrated risk assessment result based on a predetermined target model weight to obtain a risk assessment result for the target user; as well as The policy module is configured to present a resource allocation policy for the target user based on a risk assessment result for the target user.

10. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit, the instructions, when executed by the at least one processing unit, causing the electronic device to perform the method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 8.

12. A computer program product tangibly stored in a computer storage medium and comprising computer executable instructions which, when executed by a device, cause the device to perform the method according to any one of claims 1 to 8.

Images