Parameter verification method and device of default probability model, storage medium and electronic equipment

Abstract

The embodiment of the invention provides a parameter verification method and device for a default probability model, a storage medium and electronic equipment, and the method comprises the steps: inputting target object data into a target default probability model, and obtaining a predicted default probability; performing binomial test operation according to the predicted default probability, determining a target object interval, and verifying the target default probability model according to the target object interval to obtain an accuracy evaluation result, the accuracy evaluation result being used for indicating the accuracy degree of the target default probability model; performing confidence interval unilateral inspection operation on the predicted default probability to obtain a predicted default probability threshold, verifying the model parameters according to the predicted default probability threshold to obtain a judicial evaluation result, the judicial evaluation result being used for indicating the judicial degree of the target default probability model; and determining a parameter verification result of the default probability model according to the accuracy evaluation result and the judicial evaluation result. Through the method, the technical problem that the verification result of the model parameters is not accurate enough is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computers, and more specifically, to a method, device, storage medium, and electronic device for verifying parameters of a default probability model. Background Art

[0002] At present, the relevant technology mainly realizes the verification and optimization adjustment of the model parameters of the default probability model through a large amount of object data. Specifically, multiple influencing factor parameters in the object data are extracted to analyze the digital characteristics of each object in the object data, and then the model is optimized based on the digital characteristics of each object. Due to the large amount of data, the verification process of the model parameters is incomplete, which in turn causes the technical problem that the verification results of the model parameters are not accurate enough.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present application provide a method, device, storage medium and electronic device for verifying parameters of a default probability model, so as to at least solve the technical problem that the verification results of the model parameters are not accurate enough.

[0005] According to one aspect of the present application, a method for verifying parameters of a default probability model is provided, comprising: inputting target object data into a target default probability model to obtain a predicted default probability, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents the default risk of each object in the plurality of objects predicted by the target default probability model; performing a binomial test operation according to the predicted default probability to determine a target object interval, and verifying the target default probability model according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, and the binomial test operation is used to determine the target object in which default actually occurs among the plurality of objects. The method comprises the following steps: determining the number of target objects, and determining the accuracy evaluation result based on whether the number of target objects is within the target object interval, wherein the accuracy evaluation result is used to indicate the accuracy of the target default probability model; performing a confidence interval unilateral test operation on the predicted default probability to obtain a predicted default probability threshold, and verifying the model parameters according to the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval unilateral test operation is used to compare the predicted default probability threshold with the actual default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model; and determining the parameter verification result of the default probability model according to the accuracy evaluation result and the prudence evaluation result.

[0006] According to another aspect of the present application, a device for optimizing a default probability model is provided, comprising: a prediction module, configured to input target object data into a target default probability model to obtain a predicted default probability, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents the default risk of each object in the plurality of objects predicted by the target default probability model; a determination module, configured to perform a binomial test operation according to the predicted default probability, determine a target object interval, and verify the target default probability model according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, and the binomial test operation is used to determine the target object interval in which the actual default occurs among the plurality of objects. The method comprises the following steps: a first step is to determine the number of target objects, and based on whether the number of target objects is within the target object interval, determine the accuracy evaluation result, wherein the accuracy evaluation result is used to indicate the accuracy of the target default probability model; an execution module is used to perform a confidence interval unilateral test operation on the predicted default probability to obtain a predicted default probability threshold, and verify the model parameters according to the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval unilateral test operation is used to compare the predicted default probability threshold with the actual default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model; and an optimization module is used to determine a parameter verification result of the default probability model according to the accuracy evaluation result and the prudence evaluation result.

[0007] Optionally, the device is used to perform a binomial test operation based on the predicted default probability to determine a target object interval in the following manner: setting a first confidence level and determining the total number of objects of the multiple objects; when the product of the total number of objects and the predicted default probability is less than a preset threshold, determining the minimum and maximum values ​​of the target object interval based on a cumulative distribution function and the first confidence level, wherein the value of the independent variable of the cumulative distribution function is less than or equal to the total number of objects; determining the target object interval based on the minimum and maximum values ​​of the target object interval.

[0008] Optionally, the device is used to perform a binomial test operation based on the predicted default probability to determine a target object interval in the following manner: setting a second confidence level and determining the total number of objects of the multiple objects; when the product of the total number of objects and the predicted default probability is greater than a preset threshold, determining the minimum value and the maximum value of the target object interval based on a test statistic of a normal distribution and the second confidence level, wherein the test statistic of the normal distribution is related to the second confidence level; determining the target object interval based on the minimum value and the maximum value of the target object interval.

[0009] Optionally, the device is used to perform a confidence interval one-sided test operation on the predicted default probability to obtain a predicted default probability threshold in the following manner: setting a third confidence level and a target error, wherein the target error is used to determine the predicted default probability threshold; determining an intermediate value based on a test statistic of a normal distribution and the target error, wherein the test statistic of the normal distribution is related to the third confidence level; and summing the intermediate value with the predicted default probability to obtain the predicted default probability threshold.

[0010] Optionally, the device is used to determine a parameter verification result of the default probability model based on the accuracy evaluation result and the prudence evaluation result in the following manner: when the number of target objects is within the target object interval and the predicted default probability is less than or equal to the predicted default probability threshold, determine that both the accuracy evaluation result and the prudence evaluation result meet the requirements; when the number of target objects is not within the target object interval or the predicted default probability is greater than the predicted default probability threshold, adjust the parameters of the target default probability model according to the parameter verification result.

[0011] Optionally, the device is also used to: train the target default probability model using reserved samples at a first moment; and adjust the parameters of the target default probability model according to the target object data and the parameter verification result at a second moment, wherein the second moment is later than the first moment.

[0012] Optionally, the device is also used to: determine a set of default definition parameters based on the multiple objects, wherein the set of default definition parameters is used to determine whether a default has occurred among the multiple objects; when the set of default definition parameters meets preset conditions, use the target object data to train an initial default probability model to obtain the target default probability model; when the target default definition parameters do not meet the preset conditions, adjust the target default definition parameters, wherein the set of default definition parameters includes the target default definition parameters.

[0013] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0014] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0015] In an embodiment of the present application, a predicted default probability is obtained by inputting target object data into a target default probability model, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents the default risk of each object in the plurality of objects predicted by the target default probability model; a binomial test operation is performed according to the predicted default probability to determine the target object interval, and the target default probability model is verified according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, the binomial test operation is used to determine the number of target objects that actually default among the plurality of objects, and the accuracy is determined based on whether the number of target objects is within the target object interval. The accuracy evaluation result is used to indicate the accuracy of the target default probability model; a confidence interval unilateral test operation is performed on the predicted default probability to obtain a predicted default probability threshold, and the model parameters are verified based on the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval unilateral test operation is used to compare the predicted default probability threshold with the actual default probability determined based on the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model; the parameter verification result of the default probability model is determined based on the accuracy evaluation result and the prudence evaluation result to implement a standardized parameter verification method for the default probability model, thereby solving the technical problem that the verification result of the model parameters is not accurate enough. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a hardware structure block diagram of a mobile terminal of a parameter verification method of a default probability model according to an embodiment of the present application;

[0017] Figure 2 is a flow chart of a parameter verification method of a default probability model according to an embodiment of the present application;

[0018] Figure 3 It is a flow chart of a parameter verification method of a default probability model according to an embodiment of the present application;

[0019] Figure 4 is a flow chart of another parameter verification method of a default probability model according to an embodiment of the present application;

[0020] Figure 5 It is a structural block diagram of an optimization device for a default probability model according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0022] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0023] First, some nouns or terms that appear in the description of the embodiments of the present application are subject to the following interpretation:

[0024] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for verifying parameters of a default probability model according to an embodiment of the present application. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and a terminal device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0025] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the parameter verification method of the default probability model in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0026] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0027] Optionally, in this embodiment, the parameter verification method of the above-mentioned default probability model can also be implemented by a server, or implemented by a terminal device and a server together.

[0028] The above is only an example and is not specifically limited in this embodiment.

[0029] Optionally, as an optional implementation, Figure 2 is a flow chart of a parameter verification method of a default probability model according to an embodiment of the present application, such as Figure 2 As shown, the parameter verification method of the above default probability model includes:

[0030] S202, inputting target object data into a target default probability model to obtain a predicted default probability, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents the default risk of each object in the plurality of objects predicted by the target default probability model;

[0031] Optionally, in this embodiment, the above-mentioned target object data can be used as an input of a target default probability model to obtain a predicted default probability corresponding to the target object data. The target default probability model can include but is not limited to being deployed in the cloud. Specifically, taking a bank application scenario as an example, the target object data can correspond to account-related data that can be queried in the bank system, including but not limited to account information (such as age, gender, income, etc.), loan information (such as loan amount, repayment period, etc.), etc. The above-mentioned target default probability model can include but is not limited to a statistically based model, or a machine learning-based model, etc. The target default probability model can be used to predict the probability of default of an individual or enterprise to help financial institutions conduct risk management and decision-making, etc.

[0032] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data calculation, storage, processing and sharing.

[0033] Cloud computing refers to the delivery and use model of IT infrastructure, which means obtaining required resources through the network in an on-demand and easily scalable manner; in a broad sense, cloud computing refers to the delivery and use model of services, which means obtaining required services through the network in an on-demand and easily scalable manner. This service can be related to IT and software, the Internet, or other services. Cloud computing is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing.

[0034] With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the demand for search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Different from the previous parallel distributed computing, the emergence of cloud computing will promote revolutionary changes in the entire Internet model and enterprise management model from a conceptual perspective.

[0035] Exemplarily, the target object data may include multiple groups of data, which can be understood as each object corresponding to a group of data. For example, the target object data in the banking system is still used as an example, wherein the target object data may be data of multiple customers participating in the default probability rating. For example, the data corresponding to customers with a credit rating of A in the banking system is used as the target object data, and then the target object data is input into the target default probability model for processing, and the model output is the predicted default probability of each customer with a credit rating of A, or the data corresponding to customers with credit ratings of A and B in the banking system is used as the target object data, and then the target object data is input into the target default probability model for processing, and the model output is the predicted default probability of each customer with a credit rating of A, and the predicted default probability of each customer with a credit rating of B, and the predicted default probability of customer A with a credit rating of A and the predicted default probability of customer B may be the same or different, and the present application does not impose any limitation on this.

[0036] S204, performing a binomial test operation according to the predicted default probability, determining a target object interval, and verifying a target default probability model according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, the binomial test operation is used to determine the number of target objects that actually default among multiple objects, and determining an accuracy evaluation result based on whether the number of target objects is within the target object interval, and the accuracy evaluation result is used to indicate the accuracy of the target default probability model;

[0037] Optionally, in an embodiment of the present application, the target object interval is used to define an interval of the number of objects corresponding to the predicted probability of default, and the target object interval can be determined by the binomial test operation. Specifically, the binomial test operation can be used to compare whether there is a significant difference between two sets of data. The binomial test operation can be implemented based on the binomial distribution, wherein the binomial distribution is a probability distribution method that describes the probability distribution of the number of successes in a series of independently repeated Bernoulli trials. In the binomial distribution, each trial has only two possible results (success or failure), and the probabilities of these two results remain unchanged. The probability mass function of the binomial distribution can be used to calculate the probability of the number of successes given the number of trials and the probability of success.

[0038] Specifically, the target object interval is composed of the minimum value and the maximum value, and the target object interval may be determined by, but is not limited to, a binomial test operation. The data in a bank system is used as target object data for example, including but not limited to the following steps:

[0039] S1, obtaining target object data corresponding to X bank customers with a credit rating of A, where X is an integer greater than or equal to 0;

[0040] S2, set the confidence level α and the null hypothesis as "the expected default probability of X bank customers with credit rating A is equal to the actual default probability of these X bank customers", where in statistics, the confidence level usually indicates the degree of confidence in a parameter estimate or hypothesis test result, for example, the confidence level is set to 0.05;

[0041] S3, input the target object data into the target default probability model and output the predicted default probability;

[0042] S4, calculating the minimum value and the maximum value according to the predicted default probability, and obtaining the target object interval, the target object interval is expressed as [minimum value, maximum value];

[0043] S5, if the number of customers who actually default among these X bank customers is within the target range, it means accepting the null hypothesis and believing that the expected probability of default is accurate; otherwise, rejecting the null hypothesis means that the expected probability of default is inaccurate;

[0044] S6, optimize the target default probability model when the null hypothesis is rejected.

[0045] Specifically, it may include but is not limited to using the cumulative distribution function in the binomial distribution, or calculating the minimum value and the maximum value based on the normal distribution, wherein, when the product of X and the predicted probability of default is less than a preset threshold, the minimum value and the maximum value may be determined using the cumulative distribution function in the binomial distribution, and the present application does not impose any limitation on the preset threshold. For example, the preset threshold is set to 10, and the cumulative distribution function may be expressed as CDF(K), wherein K is a value less than or equal to X:

[0046] First, traverse K from 0 to X to obtain the minimum value of K that satisfies CDF(K)>=1-α / 2 as the above minimum value, where:

[0047] Next, K is still traversed from 0 to X to obtain the maximum value of K that satisfies CDF(K)<=α / 2 as the above maximum value.

[0048] For another example, when the product of X and the predicted default probability is greater than or equal to a preset threshold, the minimum value and the maximum value may be determined based on the normal distribution, and the specific implementation steps may include but are not limited to:

[0049] First, assume that N K is the number of objects of the above multiple objects, PD K To predict the default probability, determine the average μ when the binomial distribution tends to normal distribution from the above multiple objects. K With standard deviation σ K :μ K =N K ×PD K , Next, the test statistic Z of the normal distribution corresponding to the confidence interval may be determined by looking up the standard normal distribution table, but is not limited to: α / 2，Upper and Z α / 2，Lower , and then, based on Z α / 2，Upper , Z α / 2，Lower , μ K , σ K The minimum quantity and the maximum quantity are calculated respectively, wherein the minimum quantity can be determined by the following formula:

[0050] in, represents the minimum quantity; the maximum quantity can be determined by the following formula: in, Represents the maximum number. After transposition, we get:

[0051]

[0052] Further, the above accuracy evaluation result represents the accuracy of the target default probability model, the degree of difference between the predicted default probability output by the target default probability model and the actual default probability. For example, according to the target object data, the number of objects that actually default among the above multiple objects is 10, and the target object interval is expressed as [20, 30]. The target object interval represents the number interval of objects that default among the multiple objects predicted by the target default probability model corresponding to the predicted default probability. In combination with the example, the number of objects that default among the multiple objects predicted by the target default probability model is at least 20 and at most 30, that is, the above target object number is 10, the above minimum value is 20, and the above maximum value is 30. Then, since the number of objects that actually default is not within the target object interval, at this time, the accuracy evaluation result can be set to 0. If the number of objects that actually default is within the target object interval, the accuracy evaluation result can be set to 1. The present application does not impose any limitation on the accuracy evaluation result, and can include but is not limited to being expressed in the form of text, symbols, etc., or using images, text, video, voice, etc. to display the accuracy evaluation result.

[0053] It should be noted that the accuracy evaluation result can represent the accuracy of the predicted default probability. For example, an accuracy evaluation result value of 1 indicates that the predicted default probability is highly accurate, and the target default probability model can be used to predict the default situations of the above-mentioned multiple customers. Conversely, an accuracy evaluation result value of 0 indicates that the predicted default probability is less accurate, and the target default probability model cannot be used to predict the default situations of the above-mentioned multiple customers, and further optimization of the target default probability model is required.

[0054] S206, performing a confidence interval unilateral test operation on the predicted default probability to obtain a predicted default probability threshold, and verifying the model parameters according to the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval unilateral test operation is used to compare the predicted default probability threshold with the actual default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model;

[0055] Optionally, in an embodiment of the present application, the above-mentioned confidence interval unilateral test operation can be understood as, when performing a hypothesis test, only focusing on the rejection area in one direction, that is, only focusing on the situation greater than or less than a specific value, which can be used to determine whether the data is greater than or less than a specific value. When performing a confidence interval unilateral test operation, it is necessary to first determine the confidence level, for example, determine the confidence level to 0.05, etc., and then determine the upper or lower limit of the confidence interval according to the data. Then, the upper or lower limit of the confidence interval can be determined as the above-mentioned predicted default probability threshold, and the above-mentioned prudence evaluation result is obtained by comparing the predicted default probability threshold with the actual default probability, wherein the actual default probability can be understood as the probability of actual default of multiple objects corresponding to the above-mentioned target object data.

[0056] Specifically, the above confidence interval one-sided test operation can be implemented including but not limited to the following methods:

[0057] S1, set the confidence level;

[0058] S2, the interval of predicted default probability is determined as [a, b] through the target default probability model, for example:

[0059] Assuming that the above multiple objects and the target default probability conform to the normal distribution, a preset error can be preset and the confidence interval [a, b] can be constructed using the following formula:

[0060] a = output parameter of default probability model - Z* preset error;

[0061] b = output parameter of default probability model + Z*preset error;

[0062] Where Z represents the test statistic of the normal distribution, and statistical tables or computer software can be used to find the Z value corresponding to the confidence level;

[0063] S3, taking b as the predicted default probability threshold, assuming the actual default probability is c, if c<=b, the prudence evaluation result is set to 1, indicating that the prudence of the target default probability model is high;

[0064] S4, when c>b, the prudence evaluation result is set to 0, indicating that the prudence level of the target default probability model is low and the parameters of the default probability model need to be adjusted.

[0065] Alternatively, a can also be used as the predicted default probability threshold. When the true default probability c satisfies c >= a, the value of the prudence evaluation result is set to 1, indicating a higher level of prudence of the target default probability model. Conversely, when c < b, the value of the prudence evaluation result is set to 0, indicating a lower level of prudence of the target default probability model, so as to achieve the purpose of increasing the scalability of determining the predicted default probability threshold and improving the accuracy of the prudence evaluation result.

[0066] Exemplarily, the above-mentioned prudence evaluation result represents the prudence level of the target default probability model and can be used to determine whether the predicted default probability output by the target default probability model is credible. For example, based on the target object data, the true default probabilities of the above-mentioned multiple objects are 0.68, and the confidence interval obtained through the one-sided confidence interval test operation is [0.75, 0.87]. At this time, the predicted default probability threshold can be determined as, but not limited to, the upper limit value or the lower limit value of the confidence interval. For example, the predicted default probability threshold can be determined as the upper limit value of the confidence interval, that is, the value of the predicted default probability threshold is 0.87.

[0067] Furthermore, if the true default probability value is less than or equal to the predicted default probability threshold, the prudence evaluation result can be set to 1, indicating that the predicted default probability is prudent, that is, the prudence level of the target default probability model is higher. If the true default probability value is greater than the predicted default probability threshold, the prudence evaluation result can be set to 0, indicating that the predicted default probability is not prudent, that is, the prudence level of the target default probability model is lower. This application does not make any limitations on the prudence evaluation result, which can be represented in forms including but not limited to text, symbols, etc., or the prudence evaluation result can be displayed in ways such as images, texts, videos, voices, etc.

[0068] It should be noted that the prudence evaluation result can represent the prudence level of the predicted default probability. For example, when the value of the prudence evaluation result is 1, it indicates a higher level of prudence of the predicted default probability. The target default probability model can be used to predict the default situations of the above-mentioned multiple customers. Conversely, when the value of the prudence evaluation result is 0, it indicates a lower level of prudence of the predicted default probability, and the target default probability model cannot be used to predict the default situations of the above-mentioned multiple customers. It is necessary to further optimize the model of the target default probability model.

[0069] S208. Determine the parameter verification result of the default probability model according to the accuracy evaluation result and the prudence evaluation result.

[0070] Exemplarily, assuming that the accuracy evaluation result is 1 and the prudence evaluation result is 0, it means that the accuracy of the target default probability model is high and the prudence is low, and the model parameters of the target default probability model need to be adjusted and the training optimization is required; or, the accuracy evaluation result is 0 and the prudence evaluation result is 1, indicating that the accuracy of the target default probability model is low and the prudence is high, and the model parameters of the target default probability model need to be adjusted and the training optimization is required; or, the accuracy evaluation result is 0 and the prudence evaluation result is 0, indicating that both the accuracy and prudence of the target default probability model are low, and the target default probability model needs to be adjusted and the training optimization is required. The model parameters of the target default probability model are adjusted and trained for optimization, where the model parameters of the target default probability model include but are not limited to architecture parameters for adjusting model input, threshold parameters for adjusting model output, weight parameters for weighing the impact of different features, regularization parameters for adjusting model complexity and generalization ability, sampling parameters for processing unbalanced data, and optimization parameters for adjusting model learning rate, etc.; when the accuracy evaluation result is 1 and the prudence evaluation result is 1, it means that the accuracy and prudence of the target default probability model are both high, and the target default probability model can be used to predict the default risks of the above-mentioned multiple objects, and the credibility of the prediction results is high.

[0071] In an exemplary embodiment, Figure 3 1 is a flow chart of a parameter verification method of a default probability model according to an embodiment of the present application. The present application can be applied to application scenarios in which a banking system performs customer default risk prediction, such as Figure 3 As shown, it may include but is not limited to firstly performing qualitative verification on the target default probability model, and then respectively performing binomial test operation and confidence interval one-sided test operation to determine whether the target default probability model needs to be optimized. Specifically, Figure 4 is a flow chart of another parameter verification method of a default probability model according to an embodiment of the present application, such as Figure 4 As shown:

[0072] S402, obtaining target object data, and inputting the target object data into a target default probability model to obtain a predicted default probability;

[0073] S404, performing a binomial test operation on the predicted default probability to obtain a target object interval;

[0074] S406, performing a confidence interval one-sided test operation on the predicted default probability to obtain a predicted default probability threshold;

[0075] S408, determining the number of target objects that actually default and the actual default probability corresponding to the multiple objects according to the target object data;

[0076] S410, determining whether the number of target objects is within the target object interval. If the number of target objects is within the target object interval, determining that the accuracy evaluation result indicates that the target default probability model has a high degree of accuracy, and executing step S416; if the number of target objects is not within the target object interval, executing step S414;

[0077] S412, determining whether the actual default probability is less than or equal to the predicted default probability threshold. If the actual default probability is less than or equal to the predicted default probability threshold, determining that the prudence evaluation result indicates that the target default probability model has a high degree of prudence, and executing step S416; if the actual default probability is greater than the predicted default probability threshold, executing step S414;

[0078] S414, adjusting and training optimization of model parameters of the target default probability model, and executing S416 when the actual default probability is less than or equal to the predicted default probability threshold and the number of target objects is within the target object interval.

[0079] S416, end.

[0080] Through the embodiment of the present application, the target object data is input into the target default probability model to obtain the predicted default probability, wherein the target object data represents the object data corresponding to the multiple objects, and the predicted default probability represents the default risk of each object in the multiple objects predicted by the target default probability model; a binomial test operation is performed according to the predicted default probability to determine the target object interval, and the target default probability model is verified according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, the binomial test operation is used to determine the number of target objects that actually default among the multiple objects, and the accuracy evaluation result is determined based on whether the number of target objects is in the target object interval, and the accuracy evaluation result is used to indicate the accuracy of the target default probability model; a confidence interval unilateral test operation is performed on the predicted default probability. The predicted default probability threshold is obtained, and the model parameters are verified according to the predicted default probability threshold to obtain the prudence evaluation result, wherein the confidence interval one-sided test operation is used to compare the predicted default probability threshold with the actual default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model; the parameter verification result of the default probability model is determined according to the accuracy evaluation result and the prudence evaluation result, so as to determine the accuracy of prediction using the target default probability model through the accuracy evaluation result, and also determine the prudence of prediction using the target default probability model through the prudence evaluation result, thereby achieving the technical effect of optimizing the target default probability model based on the accuracy evaluation result and the prudence evaluation result to improve the optimization efficiency, and further solving the technical problem that the verification result of the model parameters is not accurate enough.

[0081] As an optional scheme, the above-mentioned binomial test operation is performed according to the above-mentioned predicted default probability to determine the target object interval, including: setting a first confidence level, and determining the total number of objects of the above-mentioned multiple objects; when the product of the above-mentioned total number of objects and the above-mentioned predicted default probability is less than a preset threshold value, determining the minimum value and the maximum value of the above-mentioned target object interval based on the cumulative distribution function and the above-mentioned first confidence level, wherein the value of the independent variable of the above-mentioned cumulative distribution function is less than or equal to the above-mentioned total number of objects; and determining the above-mentioned target object interval according to the minimum value and the maximum value of the above-mentioned target object interval.

[0082] Optionally, in an embodiment of the present application, the above-mentioned first confidence level and the above-mentioned preset threshold value can be flexibly set artificially, and the present application does not impose any limitation. For example, the first confidence level is set to 0.05, and the preset threshold value is set to 10. In addition, by comparing the numerical relationship between the total number of the above-mentioned multiple objects and the product of the predicted default probability and the preset threshold value, it can be determined whether the cumulative distribution function in the binomial distribution can be used to determine the target object interval.

[0083] Specifically, assuming that the preset threshold is 10, then when the product of the total number of objects and the predicted default probability is less than or equal to 10, the cumulative distribution function can be used to determine the target object interval. For example, the total number of objects is 100, and the preset default probability is 0.05, then the product of the total number of objects and the predicted default probability is less than 10. At this time, the above minimum value and maximum value can be determined by the cumulative distribution function, and then the target object interval is obtained, which is expressed as [minimum value, maximum value].

[0084] Through the embodiments of the present application, a first confidence level is set, and the total number of objects of multiple objects is determined; when the product of the total number of objects and the predicted default probability is less than a preset threshold, the minimum value and the maximum value of the target object interval are determined based on the cumulative distribution function and the first confidence level, wherein the value of the independent variable of the cumulative distribution function is less than or equal to the total number of objects; the target object interval is determined according to the minimum value and the maximum value of the target object interval to implement a binomial test operation, and the target object interval is quickly determined, so as to facilitate the subsequent generation of accuracy evaluation results through the target object interval, and further optimize the target default probability model, thereby achieving the technical effect of improving the optimization efficiency.

[0085] As an optional scheme, the above-mentioned binomial test operation is performed according to the above-mentioned predicted default probability to determine the target object interval, including: setting a second confidence level, and determining the total number of objects of the above-mentioned multiple objects; when the product of the above-mentioned total number of objects and the above-mentioned predicted default probability is greater than a preset threshold, determining the minimum value and the maximum value of the above-mentioned target object interval based on the test statistic of the normal distribution and the above-mentioned second confidence level, wherein the above-mentioned test statistic of the normal distribution is related to the above-mentioned second confidence level; determining the above-mentioned target object interval according to the minimum value and the maximum value of the above-mentioned target object interval.

[0086] Optionally, in an embodiment of the present application, the second confidence level and the preset threshold can be flexibly set manually, and the present application does not impose any limitation thereon. For example, the first confidence level is set to 0.05, and the preset threshold is set to 10. If the product of the total number of objects and the predicted probability of default is greater than the preset threshold, then if the binomial distribution is used to calculate the target object interval, the calculation speed will be slowed down due to the excessive amount of data and the large amount of computing resources occupied. Therefore, the minimum and maximum values ​​can be determined based on the normal distribution to determine the target object interval.

[0087] It should be noted that the test statistic of the normal distribution mentioned above may include but is not limited to being determined by statistical tables or computer software. The test statistic of the normal distribution is related to the second confidence level mentioned above. It can be understood that the test statistic of the normal distribution is used to evaluate whether the target object data follows a normal distribution.

[0088] As an optional scheme, the above-mentioned one-sided test operation of the confidence interval is performed on the above-mentioned predicted default probability to obtain the predicted default probability threshold, including: setting a third confidence level and a target error, wherein the above-mentioned target error is used to determine the above-mentioned predicted default probability threshold; determining an intermediate value according to a test statistic of a normal distribution and the above-mentioned target error, wherein the test statistic of the above-mentioned normal distribution is related to the above-mentioned third confidence level; summing the above-mentioned intermediate value with the above-mentioned predicted default probability to obtain the above-mentioned predicted default probability threshold.

[0089] Exemplarily, the third confidence level and the target error can be flexibly set manually, and the present application does not impose any limitation on this. For example, the third confidence level is set to 0.05, the target error is set to 0.04, and the intermediate value can be obtained by the following formula:

[0090] Middle value = Z * preset error, where Z represents the test statistic of the normal distribution corresponding to the third confidence level, which can be determined including but not limited to by statistical tables or computer software. Then, the sum of the middle value and the predicted default probability is used as the above-mentioned predicted default probability threshold. For example, Z is 4, the preset error is 0.05, the predicted default probability is 0.02, and the corresponding predicted default probability threshold is 0.22.

[0091] Through the embodiments of the present application, a third confidence level and a target error are set, wherein the target error is used to determine a predicted default probability threshold; an intermediate value is determined based on a test statistic of a normal distribution and the target error, wherein the test statistic of the normal distribution is related to the third confidence level; the intermediate value is summed with the predicted default probability to obtain the predicted default probability threshold, so as to achieve the purpose of determining the prudence evaluation result by comparing the predicted default probability threshold with the actual default probability, thereby improving the efficiency of testing the prudence of the target default probability model.

[0092] As an optional scheme, the above-mentioned determination of the parameter verification result of the above-mentioned default probability model based on the above-mentioned accuracy evaluation result and the above-mentioned prudence evaluation result includes: when the above-mentioned number of target objects is within the above-mentioned target object interval and the above-mentioned predicted default probability is less than or equal to the above-mentioned predicted default probability threshold, determining that the above-mentioned accuracy evaluation result and the above-mentioned prudence evaluation result both meet the requirements; when the above-mentioned number of target objects is not within the above-mentioned target object interval or the above-mentioned predicted default probability is greater than the above-mentioned predicted default probability threshold, adjusting the parameters of the above-mentioned target default probability model according to the above-mentioned parameter verification results.

[0093] In an exemplary embodiment, if the number of target objects is within the target object interval, it indicates that the accuracy of the target default probability model is verified, and the predicted default probability predicted by the target default probability model has a high degree of accuracy; if the number of target objects is not within the target object interval, it indicates that the predicted default probability predicted by the target default probability model has a low degree of accuracy, and the target default probability model needs to be optimized and adjusted.

[0094] In another exemplary embodiment, if the predicted default probability is less than or equal to the predicted default probability threshold, it indicates that the prudence of the target default probability model is verified, and the prudence of the predicted default probability predicted by the target default probability model is relatively high; if the predicted default probability is greater than the predicted default probability threshold, it indicates that the accuracy of the predicted default probability predicted by the target default probability model is relatively low, and the target default probability model needs to be optimized and adjusted.

[0095] Furthermore, when the predicted default probability is greater than the predicted default probability threshold, or the number of target objects is less than the minimum value in the target object interval, or the number of target objects is greater than the maximum value in the target object interval, the target default probability model will be optimized and adjusted.

[0096] Through the embodiments of the present application, when the number of target objects is within the target object interval and the predicted default probability is less than or equal to the predicted default probability threshold, it is determined that both the accuracy evaluation result and the prudence evaluation result meet the requirements; when the number of target objects is not within the target object interval or the predicted default probability is greater than the predicted default probability threshold, the parameters of the target default probability model are adjusted according to the parameter verification results, so as to reduce the risk of overfitting of the target default probability model during the training process by optimizing the target default probability model, thereby improving the generalization ability of the target default probability model in practical applications.

[0097] As an optional solution, the method further includes: using reserved samples to train the target default probability model at a first moment; and adjusting the parameters of the target default probability model according to the parameter verification result at a second moment, wherein the second moment is later than the first moment.

[0098] Optionally, in an embodiment of the present application, the reserved samples corresponding to the above-mentioned first moment may include but are not limited to data with the same source as the data contained in the above-mentioned target object data, and data with a different source from the data contained in the above-mentioned target object data. In other words, the data corresponding to multiple objects at the first moment can be used as the above-mentioned reserved samples, and the data corresponding to multiple objects at the second moment can be used as the above-mentioned target object data.

[0099] Exemplarily, the reserved samples can be used to train the target default probability model at the first moment. Specifically, taking a banking application scenario as an example, the reserved samples can correspond to account-related data that can be queried in the banking system, including but not limited to account personal information (such as age, gender, income, etc.), loan information (such as loan amount, repayment period, etc.), etc. After the model training is completed, the target object data is used to optimize the target default probability model at the second moment, so that the credit scoring results are more reliable, which helps financial institutions make more informed decisions in risk management, credit approval, and other aspects.

[0100] Through the embodiments of the present application, a target default probability model is trained using reserved samples at a first moment; and the target default probability model is optimized according to the target object data at a second moment, wherein the target object data represents the default history data determined between the first moment and the second moment, and the second moment is later than the first moment. In other words, the target default probability model is trained using reserved samples, and the target default probability model is optimized using the target object data, ensuring the quality and representativeness of the reserved samples and the target object data, so that the target default probability model can learn accurate default characteristics and improve the accuracy of default probability prediction.

[0101] As an optional scheme, the above method also includes: determining a set of default definition parameters based on the above multiple objects, wherein the above set of default definition parameters is used to determine whether a default has occurred in the above multiple objects; when the above set of default definition parameters meets the preset conditions, using the above target object data to train the initial default probability model to obtain the above target default probability model; when the target default definition parameters do not meet the above preset conditions, adjusting the above target default definition parameters, wherein the above set of default definition parameters includes the above target default definition parameters.

[0102] It should be noted that the above-mentioned set of default definition parameters and the above-mentioned preset conditions can be set flexibly, and this application does not impose any limitation. Specifically, taking the banking system as an example, the above-mentioned set of default definition parameters can be used to indicate the conditional parameters for an object to be deemed as in default, for example, the object has not repaid for 2 months after the due date. In combination with the example, the above-mentioned preset conditions are related to the corresponding banking regulatory requirements, that is, the regulatory agency requires that the above-mentioned set of default definition parameters must comply with risk management principles and the accuracy of risk identification. For example, it is judged through expert experience whether the default definition parameters are clear and easy to understand, whether the default definition parameters comply with the risk management principles of the regulatory agency, and whether the bank has established corresponding internal controls and processes to ensure the effective implementation of the default definition parameters.

[0103] In an exemplary embodiment, the target default definition parameter may be any one of the above-mentioned set of default definition parameters. If the target default definition parameter satisfies the above-mentioned preset conditions, relevant data may be collected based on the target default definition parameter. For example, a bank may collect the borrower's loan record, including loan amount, interest rate, repayment period, overdue situation, etc., and then, based on the default definition represented by the target default definition parameter, target object data related to the default risk is selected from the collected data, such as the borrower's credit score, income level, debt ratio, loan amount, etc., and then the target default probability model is optimized through the target object data. Conversely, if the target default definition parameter does not meet the above-mentioned preset conditions, the target default definition parameter may be adjusted. For example, before the target default definition parameter is adjusted, it indicates that if the object fails to repay for 2 months after the due date, the object is considered to be in default. After the target default definition parameter is adjusted, it indicates that if the object fails to repay for 1 month after the due date, the object is considered to be in default.

[0104] Through the embodiment of the present application, a set of default definition parameters is determined based on multiple objects, wherein a set of default definition parameters is used to determine whether multiple objects have defaulted; when a set of default definition parameters meets the preset conditions, the target object data is used to train the initial default probability model to obtain the target default probability model; when the target default definition parameters do not meet the preset conditions, the target default definition parameters are adjusted, wherein a set of default definition parameters includes the target default definition parameters, which realizes effective identification and prediction of the default risks of multiple objects, and provides strong support for enterprises to reduce potential losses. At the same time, the default definition parameters can be adjusted according to actual business needs to improve the prediction accuracy of the target default probability model.

[0105] The above is only an example, and this application does not make any specific limitation.

[0106] Obviously, the embodiments described above are only part of the embodiments of the present application, rather than all the embodiments.

[0107] The present application is described in detail below with reference to specific embodiments:

[0108] Exemplarily, the parameter verification method of the default probability model proposed in the present application can realize a complete model parameter verification method and system to verify the target default probability model parameters, ensure the accuracy and prudence of the predicted default probability, and enable the predicted default probability to play a role in practical applications. The embodiments of the present application can be widely used in various commercial institutions and financial institutions, such as loan companies, consumer finance companies, etc. Secondly, the parameter verification method of the default probability model proposed in the present application has clear processing logic, can quickly guide the construction of a verification framework for the target default probability model, and has a good verification effect.

[0109] Specifically, the embodiments of the present application may include but are not limited to:

[0110] S1. Verification data preparation, including but not limited to in-sample and out-of-sample verification. The in-sample verification data can be understood as the verification samples reserved in advance during the development of the target default probability model. The out-of-sample verification data can be understood as collecting the latest default data as the verification sample, calculating the actual default probability based on the latest default data, and verifying the expected default probability.

[0111] S2. The verification process is divided into qualitative verification and quantitative verification. Qualitative verification mainly relies on expert judgment and evaluation methods to verify whether the default definition meets regulatory requirements and whether the default definition implemented by the institution is effectively implemented. If the implementation of the default definition does not meet regulatory requirements, it will affect the subsequent quantitative verification, which may include but is not limited to using a list to confirm whether each default definition meets regulatory requirements; quantitative verification mainly uses statistical methods to verify whether the target default probability model is accurate and prudent. Quantitative verification indicators include accuracy (binomial test) and prudence (confidence interval test).

[0112] S2-1, accuracy verification. For the target default probability model, whether the expected default probability output by the target default probability model is accurate is an important indicator to measure the target default probability model, indicating whether the default probability predicted by the model can accurately predict the actual default probability. Among them, the binomial test assumes that there are K credit ratings, and the default probability of each credit rating is PD k , in different credit ratings, there are N k The number of rated customers is D, among which the number of rated customers who actually defaulted in a credit rating is D k If the credit group of this level meets the binomial distribution B(N K ,PD K ), then at the confidence level (1-α), there are two critical values and The probability that the number of actual defaulting customers in this level is between the critical values ​​is P[d K,Lα / 2 <=D k <=d K,Uα / 2 ]>=1-α. When the amount of data calculation is large, the critical value under the binomial distribution is difficult to calculate because the power of the calculation is too high. The binomial distribution tends to normal distribution when the sample number n is large. Therefore, if the product of the number of customers participating in the rating and the predicted default probability is greater than the above preset threshold, the normal distribution can be used instead of the binomial distribution to calculate the target object interval. For example, the preset threshold is 10. Under the assumption of normal distribution, the target object interval is determined. First, find the average μ when the binomial distribution tends to normal distribution in each credit rating. KWith standard deviation σ K :

[0113] μ K =N K ×PD K

[0114]

[0115] Under the standard normal distribution, the corresponding After the transposition process, we get In other words, when the significance level α is determined, the Z value (the test statistic of the normal distribution mentioned above) corresponding to the significance level is queried through statistical tables or computer software, wherein the calculation steps of the binomial test may include but are not limited to:

[0116] S2-1-1, calculate the number of defaulting and non-defaulting customers and predict the probability of default;

[0117] S2-1-2, calculate the mean μ when the binomial distribution tends to the normal distribution K With standard deviation σ K ;

[0118] S2-1-3, use the formula of standard normal distribution to calculate the target object interval, the target object interval is expressed as

[0119] S2-1-4, binomial test can be done with D K Interval estimation is used to determine the relationship between the actual number of default rating customers at each level and the critical value. or If the null hypothesis corresponding to the credit rating is rejected, then the null hypothesis corresponding to the credit rating is accepted.

[0120] S2-2. For the target default probability model, whether the expected default probability output by the target default probability model is prudent is an important indicator for measuring the target default probability model. That is, the default probability predicted by the model cannot be too low and needs to be estimated prudently. The test of prudence can include but is not limited to the use of the confidence interval unilateral test method. For example, a 95% confidence interval is constructed through the predicted default probability of the target default probability model. If the actual default rate falls within the upper limit of the confidence interval, it means that the prudence verification requirement is met.

[0121] It can be understood that qualitative verification mainly qualitatively judges whether the key definition of default meets the requirements, while the accuracy verification dimension of quantitative verification and the prudence verification dimension comprehensively evaluate whether the predicted default probability is accurate and prudent. The effectiveness of the expected default probability can be comprehensively determined by combining the qualitative and quantitative verification results.

[0122] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk), and includes a number of instructions for enabling a front-end device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of each embodiment of the present application. The information collected in the above embodiments is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. The above embodiments are based on big data analysis of personal information, and / or use artificial intelligence technology to make automated decisions to make decisions that have a significant impact on personal rights and interests. The technical solution will provide users with corresponding operation entrances for users to choose to agree or refuse the automated decision-making results; if the user chooses to refuse, the expert decision-making process will be entered.

[0123] According to another aspect of the embodiments of the present application, a system access device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0124] Figure 5 is a structural block diagram of an optimization device for a default probability model according to an embodiment of the present application, such as Figure 5 As shown, the device comprises:

[0125] Prediction module 502, used to input target object data into a target default probability model to obtain a predicted default probability, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents the default risk of each object in the plurality of objects predicted by the target default probability model;

[0126] A determination module 504 is used to perform a binomial test operation according to the predicted default probability, determine a target object interval, and verify the target default probability model according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, the binomial test operation is used to determine the number of target objects that actually default among the multiple objects, and determine the accuracy evaluation result based on whether the number of target objects is within the target object interval, and the accuracy evaluation result is used to indicate the accuracy of the target default probability model;

[0127] An execution module 506 is used to perform a confidence interval one-sided test operation on the predicted default probability to obtain a predicted default probability threshold, and verify the model parameters according to the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval one-sided test operation is used to compare the predicted default probability threshold with a true default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model;

[0128] The optimization module 508 is used to determine the parameter verification result of the default probability model according to the accuracy evaluation result and the prudence evaluation result.

[0129] As an optional scheme, the above-mentioned device is used to perform a binomial test operation based on the above-mentioned predicted default probability to determine the target object interval in the following manner: set a first confidence level, and determine the total number of objects of the above-mentioned multiple objects; when the product of the above-mentioned total number of objects and the above-mentioned predicted default probability is less than a preset threshold value, determine the minimum value and the maximum value of the above-mentioned target object interval based on the cumulative distribution function and the above-mentioned first confidence level, wherein the value of the independent variable of the above-mentioned cumulative distribution function is less than or equal to the above-mentioned total number of objects; determine the above-mentioned target object interval according to the minimum value and the maximum value of the above-mentioned target object interval.

[0130] As an optional scheme, the above-mentioned device is used to perform a binomial test operation according to the above-mentioned predicted default probability in the following manner to determine the target object interval: set a second confidence level, and determine the total number of objects of the above-mentioned multiple objects; when the product of the above-mentioned total number of objects and the above-mentioned predicted default probability is greater than a preset threshold value, determine the minimum value and the maximum value of the above-mentioned target object interval based on the test statistic of the normal distribution and the above-mentioned second confidence level, wherein the above-mentioned test statistic of the normal distribution is related to the above-mentioned second confidence level; determine the above-mentioned target object interval according to the minimum value and the maximum value of the above-mentioned target object interval.

[0131] As an optional scheme, the above-mentioned device is used to perform a confidence interval one-sided test operation on the above-mentioned predicted default probability in the following manner to obtain the predicted default probability threshold value: setting a third confidence level and a target error, wherein the above-mentioned target error is used to determine the above-mentioned predicted default probability threshold value; determining an intermediate value based on a test statistic of a normal distribution and the above-mentioned target error, wherein the test statistic of the above-mentioned normal distribution is related to the above-mentioned third confidence level; summing the above-mentioned intermediate value with the above-mentioned predicted default probability to obtain the above-mentioned predicted default probability threshold value.

[0132] As an optional scheme, the above-mentioned device is used to determine the parameter verification result of the above-mentioned default probability model according to the above-mentioned accuracy evaluation result and the above-mentioned prudence evaluation result in the following manner: when the above-mentioned number of target objects is within the above-mentioned target object interval and the above-mentioned predicted default probability is less than or equal to the above-mentioned predicted default probability threshold, determine that the above-mentioned accuracy evaluation result and the above-mentioned prudence evaluation result both meet the requirements; when the above-mentioned number of target objects is not within the above-mentioned target object interval or the above-mentioned predicted default probability is greater than the above-mentioned predicted default probability threshold, adjust the parameters of the above-mentioned target default probability model according to the above-mentioned parameter verification results.

[0133] As an optional solution, the above-mentioned device is also used to: use reserved samples to train the above-mentioned target default probability model at a first moment; adjust the parameters of the above-mentioned target default probability model according to the above-mentioned parameter verification results based on the above-mentioned target object data at a second moment, wherein the above-mentioned second moment is later than the above-mentioned first moment.

[0134] As an optional scheme, the above-mentioned device is also used to: determine a set of default definition parameters based on the above-mentioned multiple objects, wherein the above-mentioned set of default definition parameters is used to determine whether a default has occurred in the above-mentioned multiple objects; when the above-mentioned set of default definition parameters meets the preset conditions, use the above-mentioned target object data to train the initial default probability model to obtain the above-mentioned target default probability model; when the target default definition parameters do not meet the above-mentioned preset conditions, adjust the above-mentioned target default definition parameters, wherein the above-mentioned set of default definition parameters includes the above-mentioned target default definition parameters.

[0135] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0136] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0137] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0138] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0139] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0140] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0141] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0142] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0143] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A parameter verification method for a default probability model, characterized in that: include: Inputting target object data into a target default probability model to obtain a predicted default probability, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents the default risk of each object in the plurality of objects predicted by the target default probability model; A binomial test operation is performed according to the predicted default probability to determine a target object interval, and the target default probability model is verified according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, the binomial test operation is used to determine the number of target objects that actually default among the multiple objects, and the accuracy evaluation result is determined based on whether the number of target objects is within the target object interval, and the accuracy evaluation result is used to indicate the accuracy of the target default probability model; Performing a confidence interval one-sided test operation on the predicted default probability to obtain a predicted default probability threshold, and verifying the model parameters according to the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval one-sided test operation is used to compare the predicted default probability threshold with the actual default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model; The parameter verification result of the default probability model is determined according to the accuracy evaluation result and the prudence evaluation result.

2. The method according to claim 1, characterized in that: The performing of a binomial test operation according to the predicted default probability to determine the target object interval includes: setting a first confidence level and determining a total number of objects of the plurality of objects; When the product of the total number of objects and the predicted probability of default is less than a preset threshold, determining the minimum value and the maximum value of the target object interval based on the cumulative distribution function and the first confidence level, wherein the value of the independent variable of the cumulative distribution function is less than or equal to the total number of objects; The target object interval is determined according to a minimum value and a maximum value of the target object interval.

3. The method according to claim 1, characterized in that The performing of a binomial test operation according to the predicted default probability to determine the target object interval includes: setting a second confidence level and determining a total number of objects of the plurality of objects; When the product of the total number of objects and the predicted default probability is greater than a preset threshold, determining the minimum value and the maximum value of the target object interval based on a test statistic of a normal distribution and the second confidence level, wherein the test statistic of the normal distribution is related to the second confidence level; The target object interval is determined according to a minimum value and a maximum value of the target object interval.

4. The method according to claim 1, characterized in that: The performing a confidence interval one-sided test operation on the predicted default probability to obtain a predicted default probability threshold value includes: Setting a third confidence level and a target error, wherein the target error is used to determine the predicted default probability threshold; determining an intermediate value according to a test statistic of a normal distribution and the target error, wherein the test statistic of the normal distribution is related to the third confidence level; The predicted default probability is obtained by summing the intermediate value with the predicted default probability. Probability threshold.

5. The method according to claim 1, characterized in that The step of determining the parameter verification result of the default probability model according to the accuracy evaluation result and the prudence evaluation result includes: When the number of target objects is within the target object interval and the predicted default probability is less than or equal to the predicted default probability threshold, determining that both the accuracy evaluation result and the prudence evaluation result meet the requirements; When the number of target objects is not within the target object interval or the predicted default probability is greater than the predicted default probability threshold, the parameters of the target default probability model are adjusted according to the parameter verification result.

6. The method according to claim 1, characterized in that The method further comprises: Using the reserved samples to train the target default probability model at a first moment; At a second moment, the parameters of the target default probability model are adjusted according to the parameter verification result, wherein the second moment is later than the first moment.

7. The method according to claim 6, characterized in that The method further comprises: Determining a set of default definition parameters based on the plurality of objects, wherein the set of default definition parameters is used to determine whether a default has occurred in the plurality of objects; When the set of default definition parameters meets the preset conditions, the initial default probability model is trained using the target object data to obtain the target default probability model; In the case that the target default definition parameter does not meet the preset condition, the target default definition parameter is adjusted, wherein the set of default definition parameters includes the target default definition parameter.

8. An optimization device for a default probability model, characterized in that: include: A prediction module, configured to input target object data into a target default probability model to obtain a predicted default probability, wherein the target object data represents object data corresponding to a plurality of objects, and the predicted default probability represents a default risk of each object in the plurality of objects predicted by the target default probability model; a determination module, configured to perform a binomial test operation according to the predicted default probability, determine a target object interval, and verify the target default probability model according to the target object interval to obtain an accuracy evaluation result, wherein the minimum value and the maximum value of the target object interval are determined by the predicted default probability, the binomial test operation is used to determine the number of target objects that actually default among the multiple objects, and determine the accuracy evaluation result based on whether the number of target objects is within the target object interval, and the accuracy evaluation result is used to indicate the accuracy of the target default probability model; an execution module, configured to perform a confidence interval unilateral test operation on the predicted default probability to obtain a predicted default probability threshold, and verify the model parameters according to the predicted default probability threshold to obtain a prudence evaluation result, wherein the confidence interval unilateral test operation is used to compare the predicted default probability threshold with a true default probability determined according to the target object data, and the prudence evaluation result is used to indicate the prudence of the target default probability model; An optimization module is used to determine the parameter verification result of the default probability model according to the accuracy evaluation result and the prudence evaluation result.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images