Model training method and model training device

Abstract

The invention provides a model training method and a model training device, and relates to the field of computers, in particular to the field of machine learning. Acquiring historical data of risk assessment, wherein the historical data comprises risk influence information and a risk score; according to the risk score of the historical data, determining a time change function of the risk calibration coefficient for determining the risk calibration coefficient of the historical time; correcting the risk score of the historical data of each historical time according to the risk assessment target of the reference time and the risk calibration coefficient of each historical time based on the consistent correction target of the risk assessment targets of different times; and training a machine learning model by using the corrected historical data, taking the risk influence information as an input and the corrected risk score as an output reference to obtain a risk assessment model. According to the method, the historical data of risk assessment are corrected, the corrected historical data are utilized to carry out risk assessment model modeling, and the potential information value of the historical data is fully mined.

Description

Technical Field

[0001] This disclosure relates to the field of computer science, particularly to the field of machine learning, and especially to a model training method and a model training apparatus. Background Technology

[0002] With the rapid development of big data economy and artificial intelligence technology, big data risk control has been widely applied and is thriving in various financial companies. Big data credit assessment models believe that all user data is related to user credit. By mining valuable data from massive amounts of data and analyzing user behavior patterns, it is possible to reasonably avoid business risks, reduce company losses, and increase profits.

[0003] Current big data-driven intelligent risk control technologies store a large amount of historical risk assessment data in their databases. For example, different business lines may access different user credit-related data to conduct risk assessments on users at different times or from different dimensions, and all of this historical risk assessment data will be stored. Summary of the Invention

[0004] Research has revealed that historical data for risk assessment suffers from issues such as time decay and data drift. Time decay refers to the diminishing importance of data over time, considering the influence of time on data weighting. Data drift refers to changes in the statistical properties of input data over time, causing alterations in the relationship between the features used in the model and the target. Therefore, historical risk assessment data is generally not used for risk assessment model building, resulting in the underutilization of its potential informational value.

[0005] This disclosure discloses an embodiment that corrects historical risk assessment data and uses the corrected historical data to build a risk assessment model, fully exploring the potential information value of the historical data. It can be applied to scenarios such as big data risk control, expanding the range of training data for modeling, providing diversified and multi-scenario training data, and improving the model's performance.

[0006] This disclosure proposes a model training method in some embodiments, including: acquiring historical risk assessment data, including risk impact information and risk scores; determining a time variation function of risk calibration coefficients based on the risk scores of the historical data, used to determine the risk calibration coefficients for historical time periods; correcting the risk scores of historical data for each historical time period based on a consistent correction target for risk assessment objectives at different times, according to the risk assessment objective of a reference time period and the risk calibration coefficients for each historical time period; and training a machine learning model using the corrected historical data, with risk impact information as input and the corrected risk scores as output, to obtain a risk assessment model.

[0007] In some embodiments, determining the time variation function of the risk calibration coefficient includes: scoring the risk of historical data for each historical time, sorting and grouping them; and using the grouped data and risk calibration coefficients for each historical time to fit the risk assessment target for each historical time to determine the time variation function of the risk calibration coefficient.

[0008] In some embodiments, determining the time variation function of the risk calibration coefficient includes: using grouped data for each historical time period and varying the risk calibration coefficient for each historical time period to fit the risk assessment target for each historical time period, thereby obtaining the risk calibration coefficient for each historical time period corresponding to the fitting result; and fitting the risk calibration coefficient for each historical time period corresponding to the fitting result to determine the time variation function of the risk calibration coefficient.

[0009] In some embodiments, where the risk assessment target and the grouped data have a linear fit, the risk calibration coefficient includes a linear risk calibration coefficient and a constant risk calibration coefficient. Determining the time variation function of the risk calibration coefficient includes: fitting the linear risk calibration coefficients for each historical time corresponding to the fitting result to determine the time variation function of the linear risk calibration coefficients; and fitting the constant risk calibration coefficients for each historical time corresponding to the fitting result to determine the time variation function of the constant risk calibration coefficients.

[0010] In some embodiments, correcting the risk score of historical data for each historical time includes: based on a corrected target that is consistent with the risk assessment target of the reference time and the risk assessment target of the historical time, subtracting the constant term risk calibration coefficient of each historical time from the risk assessment target of the reference time, and dividing by the linear term risk calibration coefficient of each historical time to obtain the corrected risk score of the historical data for each historical time.

[0011] In some embodiments, the time variation function of the risk calibration coefficient includes an exponential function or a logarithmic function.

[0012] In some embodiments, the risk assessment target includes the bad debt rate; or the reference time is determined based on the latest historical time in historical data.

[0013] In some embodiments, the model training method further includes one or more of the following: filtering historical data based on the distribution of data labels; or filtering historical data based on whether the historical time of the historical data matches the modeling time window for training the machine learning model.

[0014] In some embodiments, the model training method further includes: filtering historical data based on one or more of the following: coverage, discrimination, stability, and group monotonicity corresponding to the risk score, wherein the group monotonicity of the risk score feature is determined based on the changing trend of the risk assessment target of each group.

[0015] In some embodiments, the model training method further includes: monitoring one or more of the coverage, discrimination, and stability of the output results of the risk assessment model; and performing a correction step again if one or more of the coverage, discrimination, and stability do not meet the requirements.

[0016] In some embodiments, historical data for risk assessment includes: historical data for user risk assessment includes one or more of the following risk impact information and user risk score: user occupation information, financial information, credit information, and lending information; or historical data for enterprise risk assessment includes one or more of the following risk impact information and enterprise risk score: financial information, operational information, credit information, and lending information.

[0017] In some embodiments, the machine learning model includes: a logistic regression model, an extreme gradient boosting model, and a random forest model.

[0018] Some embodiments of this disclosure provide a model training apparatus, including: a memory; and a processor coupled to the memory, the processor being configured to execute a model training method based on instructions stored in the memory.

[0019] This disclosure provides some embodiments of a model training apparatus, including a module for performing a model training method.

[0020] Some embodiments of this disclosure propose a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of a model training method.

[0021] Some embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the steps of a model training method. Attached Figure Description

[0022] The accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. This disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings.

[0023] Obviously, the accompanying drawings described below are merely some embodiments of this disclosure. Those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0024] Figure 1A schematic flowchart illustrating a model training method according to some embodiments of this disclosure is shown.

[0025] Figure 2 The diagram illustrates a model training method under multiple data sources according to some embodiments of this disclosure.

[0026] Figure 3 This diagram illustrates the acquisition of historical data for risk assessment in some embodiments of this disclosure.

[0027] Figure 4 A schematic diagram illustrating the correction of the time decay correction function in some embodiments of this disclosure is shown.

[0028] Figure 5 A schematic diagram of the structure of a model training apparatus according to some embodiments of the present disclosure is shown.

[0029] Figure 6 A schematic diagram of the structure of a model training apparatus according to some embodiments of the present disclosure is shown. Detailed Implementation

[0030] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0031] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0032] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0033] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0034] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0035] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0036] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0037] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0038] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0039] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0040] Furthermore, to avoid obscuring this disclosure with unnecessary detail, only processing steps and / or apparatus structures closely related to at least the solutions according to this disclosure are shown in the accompanying drawings, while other details not closely related to this disclosure are omitted. It should also be noted that similar reference numerals and letters in the drawings indicate similar items, and therefore once an item is defined in one drawing, it need not be discussed again in subsequent drawings.

[0041] To address the common problem of ineffective utilization of historical data in risk control scenarios such as finance, where historical data gradually becomes inactive and loses its sensitivity to current risks over time, this disclosure proposes a historical data modeling scheme based on a time decay correction function. This scheme fully explores and utilizes the potential information value of historical data, improving its effective utilization rate. Based on the unchanged relationship between features and true labels, the scheme analyzes the fitting function between risk calibration coefficients of different historical durations and related features. It fits the time decay correction function of historical duration and features with risk trend changes, calibrating and correcting feature value deviations caused by risk shifts. This more accurately identifies the ability of historical data features to capture current risks and improves the stability of historical data features over time. This stability refers to the stability of P(Y|X), that is, the stability of the probability P(Y|X) that the model predicts a specific output Y based on a specific input X.

[0042] Figure 1 A schematic flowchart illustrating a model training method according to some embodiments of this disclosure is shown.

[0043] Figure 2 The diagram illustrates a model training method under multiple data sources according to some embodiments of this disclosure.

[0044] like Figure 1 andFigure 2 As shown, the model training method includes steps 110-150, wherein the data filtering step and the model monitoring step can be selected to be executed or not executed according to business needs.

[0045] In step 110, historical data for risk assessment is obtained, including risk impact information and risk scores.

[0046] Depending on the specific risk assessment scenario, the corresponding historical data may vary. For example, in user risk assessment, the risk score refers to the user's risk rating, and the relevant risk impact information includes, for instance, user occupational information, financial information, credit information, and loan information—all information related to user risk. Similarly, in enterprise risk assessment, the risk score refers to the enterprise's risk rating, and the relevant risk impact information includes, for instance, financial information, operational information, credit information, and loan information—all information related to enterprise risk.

[0047] In the training of the machine learning model in this embodiment, risk impact information is used as input to the machine learning model being trained. The machine learning model predicts and outputs a risk score based on the input information. The risk score in the historical data is a feature that needs to be corrected. The corrected risk score is used as a reference (or label) for the output of the machine learning model. The parameters of the machine learning model can be updated based on the difference between the predicted value of the risk score and the reference value of the risk score (i.e., the corrected risk score) so as to train the machine learning model to obtain a risk assessment model.

[0048] Historical data can come from risk assessment-related data retrieved by business lines and stored in the database; therefore, it is also called cached data or historical cached data.

[0049] like Figure 3As shown, to establish a risk assessment model (referred to as "modeling"), historical data from various data sources is obtained through data source interfaces to determine the modeling time window and data label distribution. Taking user risk assessment as an example, the data label distribution might include information such as the proportion of users classified as good or bad, where risk scores determine whether a user is good or bad. Then, the historical data can be parsed and cleaned. Based on whether the historical time of the historical data matches the modeling time window used for training the machine learning model, historical data is filtered, removing data whose historical time does not meet the modeling time window requirements. Similarly, based on the data label distribution required for modeling, historical data is filtered, removing data whose label distribution does not meet the modeling requirements. This initial filtering of the acquired historical data ensures that basic modeling requirements are met and avoids affecting the modeling effect. If the historical data sources include not only internal data sources but also external data sources such as third parties, it is also necessary to match the feature primary keys of the historical data from external data sources with those from internal data sources to unify the feature identification information of each data source.

[0050] In step 120, historical data of the feature domain is filtered to improve the consistency of the performance of each data source in the feature domain, especially when there are many historical data sources, thereby improving the modeling effect.

[0051] Historical data is filtered based on the performance of the "risk score" feature across selection metrics. The filtering is performed according to one or more of the following: coverage, discrimination, stability, and grouping monotonicity of the "risk score" feature. The grouping monotonicity of the risk score feature is determined based on the changing trend of the risk assessment target for each group. Grouping is also called binning, and the risk assessment target is, for example, the bad debt rate. Therefore, the binning monotonicity can be determined based on the bad debt rate of each bin. For example, historical data with low coverage, low discrimination (e.g., low KS value), poor stability (e.g., high PSI value), or no monotonicity in the risk assessment target of the group (e.g., no monotonicity in the binning bad debt rate) are removed. This improves the accuracy and stability of the trained model.

[0052] The feature coverage = N1 / N, where N1 is the number of non-empty samples in feature i and N is the total number of samples. Specifically, in this embodiment, the coverage of the "risk score" feature refers to the ratio of the amount of customer data from the intersection of external data calls to the amount of customer data from internal data calls.

[0053] The stability of a feature can be characterized, for example, by the Population Stability Index (PSI). The PSI measures the stability of a feature / model between the actual and expected distributions. The smaller the PSI value, the smaller the difference between the two distributions, and the more stable the feature / model. Among them, A i and E i These represent the actual proportion and expected proportion of the feature in the i-th segment of its value, respectively.

[0054] The discriminative power of features can be characterized, for example, by the KS (Kolmogorov-Smirnov) metric. The KS value measures the model's ability to distinguish between positive and negative samples. The KS value is the maximum absolute value of the difference between the cumulative percentage of bad samples (TPR) and the cumulative percentage of good samples (FPR), i.e., KS = max(|TPR - FPR|). Here, TPR (True Positive Rate) represents the classifier's ability to correctly identify positive examples, calculated as: TPR = TP / (TP + FN), where TP represents true positives and FN represents false negatives. FPR (False Positive Rate) represents the classifier's ability to incorrectly predict negative examples as positives, calculated as: FPR = FP / (FP + TN), where FP represents false positives and TN represents true negatives.

[0055] The bad debt ratio refers to the proportion of samples judged as bad (bad individuals) out of the total sample. The formula for calculating the bad debt ratio is Bad Ratio = Bad / (Bad + Good), where Bad represents bad samples (bad individuals) and Good represents good samples (good individuals). To observe the monotonicity of binning, historical data showing a monotonic relationship between the bad sample ratio and binning is selected based on the "risk score" feature.

[0056] In step 130, the fitted time decay correction function is used to correct the feature value of the historical data "risk score" to take into account both the timeliness and stability of the historical data "risk score" feature.

[0057] Based on the risk scores from historical data, a time-varying function for the risk calibration coefficient is determined to identify the risk calibration coefficient for each historical time period. A consistent correction target, based on the risk assessment objectives across different time periods, is used to correct the risk scores of historical data for each historical time period, according to the risk assessment objectives of the reference time and the risk calibration coefficient for each historical time period. Risk assessment objectives include, but are not limited to, the bad debt rate.

[0058] The process of determining the time-varying function of the risk calibration coefficient includes: assigning risk scores to historical data for each historical time period, sorting and grouping them; and using the grouped data and risk calibration coefficients for each historical time period to fit the risk assessment target for that historical time period, thus determining the time-varying function of the risk calibration coefficient. Therefore, the time-varying function of the risk calibration coefficient is determined through function fitting.

[0059] The determination of the time variation function of the risk calibration coefficient includes: using grouped data for each historical time period and varying the risk calibration coefficient for each historical time period to fit the risk assessment target for each historical time period, obtaining the risk calibration coefficient for each historical time period corresponding to the fitting result; fitting the risk calibration coefficient for each historical time period corresponding to the fitting result to determine the time variation function of the risk calibration coefficient. The time variation function of the risk calibration coefficient includes, but is not limited to, exponential functions or logarithmic functions.

[0060] Where the risk assessment target and the grouped data exhibit a linear fit, the risk calibration coefficients include linear risk calibration coefficients and constant risk calibration coefficients. Determining the time variation function of the risk calibration coefficients includes: fitting the linear risk calibration coefficients for each historical time corresponding to the fitting result to determine the time variation function of the linear risk calibration coefficients; and fitting the constant risk calibration coefficients for each historical time corresponding to the fitting result to determine the time variation function of the constant risk calibration coefficients.

[0061] The process of correcting the risk score of historical data for each historical time includes: based on the corrected target that is consistent with the risk assessment target of the reference time, subtracting the constant term risk calibration coefficient of each historical time from the risk assessment target of the reference time, and dividing by the linear term risk calibration coefficient of each historical time to obtain the corrected risk score of the historical data for each historical time.

[0062] The following describes a method for correcting the "risk score" feature value of historical data, using the bad debt rate as an example of a risk assessment objective.

[0063] First, considering the timeliness of historical data, the reference time is determined based on the latest historical time in the historical data, ensuring that the features reflect the user's recent risks and guaranteeing the highest level of timeliness. For example, by analyzing the decay changes of feature KS values ​​at different time points, the validity period of cached features is determined. The validity period may vary for different data domains. The most recent feature among all historical time point features within the validity period is selected to ensure the highest level of timeliness.

[0064] Next, based on the duration *t* from the reference time, the historical data is divided into time windows. For example, *t0* represents the reference time, *t1* represents 0-30 days from the reference time, *t2* represents 30-60 days from the reference time, *t3* represents 60-90 days from the reference time, and so on, up to *t*. n For example, t m ={t0,t1,t2,t3,...,t n},n <n ε ε is the threshold for duration t (which may vary depending on the data source). ε is set based on the difference in KS value decay and the actual business background requirements. For example, the validity period cannot exceed 720 days. If the data interval is too long, the risk capture will be less timely and the risk identification capability will be weaker.

[0065] Next, the risk scores for historical data at each historical time point are sorted and then grouped (or binned), as expressed by the formula:

[0066]

[0067] in, This represents the bin score of the risk score at each historical time t. bin These bins are score0, score1, score2, score3, ..., score n .

[0068] Next, construct each historical time t m Risk score binning data bin With each historical time t m The relationship between the bad debt rate (BR) under different risk rating bins is f. BR (score bin ,t m The function is used to fit the risk change trend at each t, ​​in order to find the risk calibration coefficient a that best fits the function. n and b n .

[0069]

[0070] coefficient a n b n set:

[0071] a n ={a0,a1,a2,…,a n},b n ={b0,b1,b2,…,b n}

[0072] Where, fBR (score bin ,t m (f) is a trend function of the risk calibration coefficient as the historical data "risk score" feature changes with different time intervals t from the reference. The fitted f is analyzed experimentally. BR (score bin ,t m The function is a linear function, therefore the function coefficients a under different t values ​​can be obtained. n b n set.

[0073] Experiments revealed that in practical applications, due to potential differences in data from different data sources, the f values ​​from different data sources may vary. BR (score bin ,t m The function may also exist in different forms, such as exponential functions and logarithmic functions. Correspondingly, the mathematical function form of the risk calibration coefficient under different values ​​of t may also differ, such as exponential functions and logarithmic functions. This embodiment fits a... n b n The risk calibration coefficient a obtained from the set n b n Taking the logarithmic function as an example, the formula is as follows:

[0074]

[0075] Where t represents time, and α1β1α2β2 are constants obtained from the fitting.

[0076] Finally, a time decay correction function related to risk trend changes is fitted to perform time decay correction. Based on the goal of consistent bad debt rates at different times, the bad debt rate corresponding to the risk score at different times (t) is made consistent with the bad debt rate corresponding to the initial risk score at t=0. This ensures that the relationship P(Y|X) between the "risk score" feature and the risk assessment target "bad debt rate" does not shift, where X represents the risk score and Y represents the bad debt rate. This further corrects the "risk score" feature at different historical time points. The formula is expressed as follows:

[0077] BR0 = a0score0 + b0 = a n score′ n +b n ,

[0078] The corrected risk score is obtained from the above formula. n :

[0079]

[0080] The time decay correction function obtained by this scheme has favorable mathematical properties and is easy to interpret and analyze in risk control scenarios. For example, experimental analysis found that the trend of bad debt rate risk change under different feature bins for different call record durations shows that the difference between the Lift values ​​at the beginning and end of the high and low score segments becomes smaller as the call time interval increases. The Lift value refers to the multiple by which the probability of hitting bad debt risk under bins after sorting each feature is increased based on the overall probability of hitting bad debt risk.

[0081] Figure 4 A schematic diagram illustrating the time decay correction function is shown. For example... Figure 4 As shown, the historical data cache feature is "risk score" x n After correction by the time decay correction function f(), the corrected "risk score" f(x) is obtained. n ).

[0082] In step 140, the machine learning model is trained using the corrected historical data to obtain a risk assessment model.

[0083] Using corrected historical data, risk impact information is used as input to the model, and the corrected risk score is used as the reference for the model's output. The machine learning model is then trained to obtain a risk assessment model. During training, the parameters of the machine learning model are iteratively updated based on the difference between the predicted risk score output by the machine learning model based on the input risk impact information and the reference risk score (i.e., the corrected risk score), continuously narrowing this gap. Specifically, the corrected historical dataset can be divided into training and testing sets for training and evaluation of the machine learning model. The machine learning model that has been trained and passed the tests is used as the risk assessment model.

[0084] The machine learning models and risk assessment models mentioned include, but are not limited to, models suitable for risk control such as logistic regression, extreme gradient boosting (XGBoost), and random forest. Specifically, the XGBoost model continuously adds trees to fit the residuals from the previous step, constantly splitting features to grow a tree. Each time a tree is added, it learns a new function to fit the residuals from the previous prediction.

[0085] In step 150, the effectiveness of the risk assessment model is monitored.

[0086] Set various monitoring metrics for the model, such as coverage, stability (e.g., PSI value), effectiveness (e.g., discrimination KS value), bad debt rate, mean, maximum, minimum, and null values ​​of model variables, etc. Set decay thresholds for the monitoring metrics, and then monitor the coverage, discrimination, stability, and other metrics of the risk assessment model's output results. If the monitoring metrics of the model features fall below the preset standard principle as time decays, that is, if one or more of the monitoring metrics such as coverage, discrimination, and stability fail to meet the requirements, the correction step is executed again to slow down the speed of feature drift and the frequency of model iteration updates, thereby reducing unnecessary time and effort expenditure.

[0087] This disclosure discloses an embodiment that corrects historical risk assessment data and uses this corrected historical data to build a risk assessment model, fully exploring the potential information value of the historical data. In practical application, the timeliness and stability of historical data are fully considered and balanced. It can be applied to scenarios such as big data risk control, expanding the range of training data for modeling, providing diversified and multi-scenario training data, and improving model performance. Regarding model monitoring, the model is managed with fine detail, and the decay of model features is observed in a timely manner. Reasonable decay thresholds for monitoring indicators are set, and when the decay falls below a preset standard, a time decay correction function is used to further correct the features, slowing down the rate of feature drift.

[0088] Figure 5 A schematic diagram of the structure of a model training apparatus according to some embodiments of the present disclosure is shown.

[0089] like Figure 5 As shown, the model training device 500 of this embodiment includes modules 510-550. Modules 520 and 550 can be configured as needed.

[0090] The data acquisition module 510 is configured to acquire historical data for risk assessment, including risk impact information and risk scores.

[0091] The data acquisition module 510 or the filtering module 520 is configured to filter historical data based on the distribution of data labels; or to filter historical data based on whether the historical time of the historical data matches the modeling time window of the machine learning model training.

[0092] The filtering module 520 is configured to filter historical data based on one or more of the following: coverage, discrimination, stability, and group monotonicity corresponding to the risk score. The group monotonicity of the risk score feature is determined based on the changing trend of the risk assessment objectives of each group.

[0093] The correction module 530 is configured to determine the time variation function of the risk calibration coefficient based on the risk score of historical data, and to determine the risk calibration coefficient of historical time; based on the consistent correction target of risk assessment objectives at different times, the risk score of historical data for each historical time is corrected according to the risk assessment objective of the reference time and the risk calibration coefficient of each historical time.

[0094] The correction module 530 is configured to score the risk of historical data for each historical time, sort and group them; and use the grouped data and risk calibration coefficients for each historical time to fit the risk assessment target for each historical time to determine the time variation function of the risk calibration coefficients.

[0095] The correction module 530 is configured to use grouped data for each historical time period and vary the risk calibration coefficient for each historical time period to fit the risk assessment target for each historical time period, thereby obtaining the risk calibration coefficient for each historical time period corresponding to the fitting result; and to fit the risk calibration coefficient for each historical time period corresponding to the fitting result to determine the time variation function of the risk calibration coefficient.

[0096] The correction module 530 is configured such that, when the risk assessment target and the grouped data have a linear fitting relationship, the risk calibration coefficient includes a linear risk calibration coefficient and a constant risk calibration coefficient. The time variation function of the risk calibration coefficient is determined by: fitting the linear risk calibration coefficient for each historical time corresponding to the fitting result to determine the time variation function of the linear risk calibration coefficient; and fitting the constant risk calibration coefficient for each historical time corresponding to the fitting result to determine the time variation function of the constant risk calibration coefficient.

[0097] The correction module 530 is configured to correct the risk assessment target based on the reference time and the risk assessment target of the historical time. It subtracts the constant term risk calibration coefficient of each historical time from the risk assessment target of the reference time and divides it by the linear term risk calibration coefficient of each historical time to obtain the corrected risk score of the historical data of each historical time.

[0098] Training module 540 is configured to use corrected historical data, with risk impact information as input and corrected risk scores as output, to train a machine learning model to obtain a risk assessment model.

[0099] The monitoring module 550 is configured to monitor one or more of the coverage, discrimination, and stability of the output results of the risk assessment model; if one or more of the coverage, discrimination, and stability do not meet the requirements, the correction steps are executed again.

[0100] Figure 6 A schematic diagram of the structure of a model training apparatus according to some embodiments of the present disclosure is shown.

[0101] like Figure 6 As shown, the model training apparatus 600 of this embodiment includes a memory 610 and a processor 620 coupled to the memory 610. The processor 620 is configured to execute the model training method in each embodiment based on instructions stored in the memory 610.

[0102] The model training device 600 may also include an input / output interface 630, a network interface 640, a storage interface 650, etc. These interfaces 630, 640, 650, as well as the memory 610 and the processor 620, can be connected, for example, via a bus 660.

[0103] The memory 610 may include, for example, system memory, fixed non-volatile storage media, etc. The system memory may store, for example, the operating system, application programs, boot loader, and other programs.

[0104] The processor 620 can be implemented using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates, or transistors, or other discrete hardware components.

[0105] The input / output interface 630 provides a connection interface for input / output devices such as monitors, mice, keyboards, and touchscreens. The network interface 640 provides a connection interface for various networked devices. The storage interface 650 provides a connection interface for external storage devices such as SD cards and USB flash drives. The bus 660 can use any bus architecture from a variety of bus structures. For example, bus architectures include, but are not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, and Peripheral Component Interconnect (PCI) bus.

[0106] It should be noted that the technical solutions in this disclosure, including the collection, updating, analysis, processing, use, transmission, and storage of user personal information, all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security, network security, and national security.

[0107] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more (non-transitory) computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, cloud storage, etc.) containing computer program code. A computer program product should be understood as a software product that primarily implements its solution through a computer program.

[0108] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0109] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0110] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0111] The above description is only a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A model training method, comprising: Obtain historical data for risk assessment, including risk impact information and risk scores; Based on the risk score from historical data, a time variation function of the risk calibration coefficient is determined, which is used to determine the risk calibration coefficient for historical time. Based on the consistent risk assessment objectives across different time periods, the risk scores of historical data for each historical time period are corrected according to the risk assessment objectives of the reference time and the risk calibration coefficients for each historical time period. Using corrected historical data, with risk impact information as input and corrected risk scores as output, a machine learning model is trained to obtain a risk assessment model.

2. The model training method according to claim 1, wherein, The time variation function for determining the risk calibration coefficient includes: Risk scores are assigned to historical data for each historical time period, and then the data is sorted and grouped. By using grouped data and risk calibration coefficients for each historical time period, a risk assessment target for each historical time period is fitted to determine the time variation function of the risk calibration coefficients.

3. The model training method according to claim 2, wherein, The time variation function for determining the risk calibration coefficient includes: By using the grouped data of each historical time and varying the risk calibration coefficient of each historical time, the risk assessment target of each historical time is fitted, and the risk calibration coefficient of each historical time corresponding to the fitting result is obtained. The risk calibration coefficients for each historical time corresponding to the fitting results are fitted to determine the time variation function of the risk calibration coefficients.

4. The model training method according to claim 3, wherein, When the risk assessment objective and the grouped data show a linear fit, the risk calibration coefficients include a linear term risk calibration coefficient and a constant term risk calibration coefficient. The time variation function for determining the risk calibration coefficient includes: Fit the linear risk calibration coefficients for each historical time corresponding to the fitting results to determine the time variation function of the linear risk calibration coefficients; The constant term risk calibration coefficients for each historical time corresponding to the fitting results are fitted to determine the time variation function of the constant term risk calibration coefficients.

5. The model training method according to claim 4, wherein, The risk score for historical data at each historical time point is revised by including: Based on the corrected target that aligns the risk assessment target of the reference time with the risk assessment target of the historical time, the risk assessment target of the reference time is subtracted from the constant term risk calibration coefficient of each historical time, and then divided by the linear term risk calibration coefficient of each historical time to obtain the corrected risk score of the historical data for each historical time.

6. The model training method according to any one of claims 1-5, wherein, The time-varying function of the risk calibration coefficient includes either an exponential function or a logarithmic function.

7. The model training method according to any one of claims 1-5, wherein, The risk assessment targets include the bad debt rate; or The reference time is determined based on the latest historical time from the historical data.

8. The model training method according to any one of claims 1-5, further comprising one or more of the following: Filter historical data based on data label distribution; or Historical data is filtered based on whether the historical time of the historical data matches the modeling time window used for training the machine learning model.

9. The model training method according to any one of claims 1-5, further comprising: Historical data is filtered based on one or more of the following criteria corresponding to the risk score: coverage, discrimination, stability, and group monotonicity. The grouping monotonicity of the risk scoring characteristics is determined based on the changing trend of the risk assessment objectives of each group.

10. The model training method according to any one of claims 1-5, further comprising: Monitor one or more of the following in the output results of the risk assessment model: coverage, discrimination, and stability; If one or more of the requirements for coverage, discrimination, and stability are not met, the correction steps are performed again.

11. The model training method according to any one of claims 1-5, wherein, Historical data for risk assessment includes: Historical data for user risk assessment includes: one or more of the following risk impact information and user risk score: user occupational information, financial information, credit information, and loan information; or Historical data for enterprise risk assessment includes one or more of the following: financial information, operational information, credit information, and lending information, as well as enterprise risk scores.

12. The model training method according to any one of claims 1-5, wherein, The machine learning models include: logistic regression model, extreme gradient boosting model, and random forest model.

13. A model training device, comprising: Memory; And a processor coupled to the memory, the processor being configured to execute the model training method of any one of claims 1-12 based on instructions stored in the memory.

14. A model training device, comprising: A module for performing the model training method according to any one of claims 1-12.

15. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the model training method according to any one of claims 1-12.

16. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the model training method according to any one of claims 1-12.

Images