Credit risk identification model training method and credit risk identification method and device

Abstract

The embodiment of the invention provides a credit risk identification model training method, a credit risk identification method and a credit risk identification device, and the method carries out the training of a credit risk identification model through training sample data, and specifically carries out the training of a main deep network in the credit risk identification model. An adversarial sample generation network is introduced on the basis of a main deep network, and the adversarial sample generation network performs training adjustment on the output precision of the main deep network based on adversarial disturbance feature vectors generated by delay of the main deep network in combination with each feature vector in training sample data. And until the main deep network model can accurately output the risk overdue result corresponding to each piece of training sample data. Therefore, the disturbance resisting capability of the main neural network can be improved, so that the performance of the whole credit risk identification model facing data distribution transfer or noise disturbance is improved, and the stability and reliability of the target credit risk identification model in practical application are enhanced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a credit risk identification model training method, a credit risk identification method and a device. Background Art

[0002] With the development of artificial intelligence technology, neural network models are gradually applied to various industries. In the financial industry, neural network models are often constructed to predict users' potential credit risks in order to reduce losses caused by high-risk credit users. Among them, credit risk refers to the risk probability of overdue payments after users apply for credit products. The neural network model commonly used in the industry for credit risk prediction is referred to as a credit risk identification model. Commonly used credit risk identification model construction methods include: constructing a credit risk identification model based on traditional statistical methods, constructing a credit risk identification model based on machine learning algorithms, and constructing a credit risk identification model based on deep learning algorithms.

[0003] The credit risk identification models constructed with current technology often have low recognition accuracy when faced with data distribution shifts or noise disturbances. Summary of the invention

[0004] In view of this, embodiments of the present application provide a credit risk identification model training method, a credit risk identification method and a device to improve.

[0005] In a first aspect, an embodiment of the present application provides a credit risk identification model training method, wherein the method comprises:

[0006] Acquire training sample data, wherein the training sample data is feature vector data extracted based on historical credit record data;

[0007] Input each feature vector in the training sample data into the initial credit risk identification model, and the credit risk identification model calculates the risk prediction result corresponding to each piece of training sample data based on the input feature vector; wherein the initial credit risk identification model includes: a main deep network and an adversarial sample generation network; the loss function in the adversarial sample generation network is an adversarial loss function, the adversarial sample generation network is used to generate an adversarial perturbation feature vector based on the delayed data feature generated by the main deep network delay, and the main deep network is used to output the risk prediction result corresponding to each piece of training sample data based on the adversarial perturbation feature vector and each feature vector in the training sample data;

[0008] According to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data, the model parameters of the credit risk identification model are adjusted until the target data difference is less than a preset data difference threshold;

[0009] The credit risk identification model corresponding to the target data difference being less than a preset data difference threshold is determined as the target credit risk identification model.

[0010] In a second aspect, an embodiment of the present application provides a credit risk identification method, wherein the method comprises:

[0011] Acquire user characteristics of the user to be identified, and input the user characteristics into a target credit risk identification model, so that the target credit risk identification model determines the overdue probability of the user to be identified in each credit cycle based on the user characteristics;

[0012] Wherein, the target credit risk identification model is a network model pre-trained by using the credit risk identification model training method as described in the first aspect.

[0013] In a third aspect, an embodiment of the present application provides a credit risk identification model training device, the device comprising:

[0014] An input module, used to obtain training sample data, wherein the training sample data is feature vector data extracted based on historical credit record data;

[0015] A training module, used to input each feature vector in the training sample data into an initial credit risk identification model, and the credit risk identification model calculates the risk prediction result corresponding to each piece of training sample data based on the input feature vector; wherein the initial credit risk identification model includes: a main deep network and an adversarial sample generation network; the loss function in the adversarial sample generation network is an adversarial loss function, the adversarial sample generation network is used to generate an adversarial perturbation feature vector based on the data features generated by the main deep network with a delay, and the main deep network is used to output the risk prediction result corresponding to each piece of training sample data based on the adversarial perturbation feature vector and each feature vector in the training sample data;

[0016] A parameter adjustment module, used to adjust the model parameters of the credit risk identification model according to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data, until the target data difference is less than a preset data difference threshold;

[0017] The determination module is used to determine the credit risk identification model corresponding to the target data difference being less than a preset data difference threshold as the target credit risk identification model.

[0018] In a fourth aspect, an embodiment of the present application provides a credit risk identification device, wherein the device includes:

[0019] An acquisition module, used to acquire user characteristics of the user to be identified, and input the user characteristics into a target credit risk identification model;

[0020] An identification module, used for the target credit risk identification model to determine the overdue probability of the user to be identified in each credit cycle based on the user characteristics;

[0021] Wherein, the target credit risk identification model is a network model pre-trained by using the credit risk identification model training method as described in the first aspect.

[0022] In a fifth aspect, an embodiment of the present application provides an electronic device, wherein the electronic device includes:

[0023] Processor; and

[0024] Memory for storing programs,

[0025] Wherein, the program includes instructions, and when the instructions are executed by the processor, the processor executes the credit risk identification model training method described in the first aspect and the credit risk identification method described in the second aspect.

[0026] In a sixth aspect, an embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to enable a computer to execute the credit risk identification model training method described in the first aspect and the credit risk identification method described in the second aspect.

[0027] Beneficial effects of this application:

[0028] The embodiment of the present application provides a credit risk identification model training method, a credit risk identification method and a device, wherein the method trains the credit risk identification model with the help of training sample data, and specifically trains the main deep network in the credit risk identification model, by introducing an adversarial sample generation network based on the main deep network, and the adversarial perturbation feature vector generated by the adversarial sample generation network based on the main deep network delay, and the output accuracy of the main deep network is trained and adjusted in combination with each feature vector in the training sample data, until the main deep network model can accurately output the risk overdue results corresponding to each training sample data. In this way, the ability of the main neural network to resist disturbances can be improved, thereby improving the performance of the entire credit risk identification model in the face of data distribution transfer or noise disturbance, and thereby enhancing the stability and reliability of the target credit risk identification model in practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Further details, features and advantages of the present application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:

[0030] Figure 1 A flow chart of a credit risk identification model training method provided in an embodiment of the present application is shown;

[0031] Figure 2 A schematic diagram of a model architecture of a credit risk identification model provided in an embodiment of the present application is shown;

[0032] Figure 3 A schematic diagram of a process of a credit risk identification method provided by an embodiment of the present application is shown;

[0033] Figure 4 A structural schematic diagram of a credit risk identification model training device provided in an embodiment of the present application is shown;

[0034] Figure 5 A schematic diagram of the structure of a credit risk identification device provided in an embodiment of the present application is shown;

[0035] Figure 6 A structural block diagram of an exemplary electronic device that can be used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0037] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0038] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0039] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0040] As described in the background technology, existing methods for constructing credit risk identification models include: based on traditional statistics, based on machine learning algorithms, and based on deep learning algorithms. Although the credit risk identification models constructed by these methods can improve the accuracy and robustness of the credit risk identification models to a certain extent, robustness means that the credit identification model can still maintain stable operation of the model and give stable output results when facing changes in internal structure or external environment, which can be simply understood as the anti-interference ability of the credit risk identification model. However, when facing data distribution shifts or noise disturbances, the credit risk identification models constructed by these methods still have many deficiencies. Specifically, there are the following deficiencies:

[0041] 1) The shortcomings of the credit risk identification model based on traditional statistical methods:

[0042] Credit risk identification models constructed using traditional statistical methods include: Bayesian credit scoring model, FICO (Fair Isaac Credit Organization, credit scoring system) scoring model. Such credit risk identification models rely on static characteristics such as the borrower's historical credit record, income level, debt ratio, etc. Although they have certain stability and interpretability, once the data changes dynamically, the output risk identification results are often not as expected.

[0043] 2) Shortcomings of the credit risk identification model based on machine learning algorithms:

[0044] The application of machine learning algorithms in the field of credit risk identification usually includes: decision trees, random forests, support vector machines, etc. By training a large amount of historical data, the complex nonlinear relationships existing in the historical data are captured to improve the accuracy of risk prediction. However, the credit risk identification model constructed by the machine learning algorithm usually relies on the assumption that the distribution of training data and test data is consistent. When faced with a data distribution shift, the performance of the model will drop significantly.

[0045] 3) Shortcomings of the credit risk identification model based on deep learning algorithms:

[0046] Credit risk identification models built based on deep learning algorithms include: deep neural network models and convolutional neural network models. Among them, through end-to-end learning, deep learning models can automatically extract data features from massive amounts of data, and then determine the correlation between data features and credit risk overdue based on the extracted features, thereby improving the accuracy of credit risk assessment. However, the credit risk identification model built based on the deep learning algorithm is less sensitive to data distribution shifts, and adversarial attacks will reduce the basic performance of the model.

[0047] In order to improve the anti-interference ability of the above-mentioned models, adversarial training and data enhancement techniques are usually used in the prior art to strengthen the training of the models. Specifically, adversarial training refers to the introduction of adversarial samples in the model training process, thereby improving the model's ability to cope with unknown data disturbances. Data enhancement technology refers to the process of expanding the diversity of data in the training data set during model training to reduce the overfitting problem of the model. Although these methods have improved the anti-interference ability and pan-processing ability of the model to a certain extent, the adversarial disturbances generated by the generated adversarial samples are often not enough to cope with the real large-scale data transfer and noise faced by the model in practical applications.

[0048] In summary, although the existing credit risk identification models perform reliably in conventional market environments, they still have obvious limitations when dealing with rapidly changing economic environments and processing large-scale real-time data. In view of this, the present application provides a credit risk identification model training method, a credit risk identification method, an apparatus and an electronic device. Among them, in the first aspect, the present application provides a credit risk identification model training method, and the credit risk identification model training method can be applied to any electronic device with model training capabilities. In the second aspect, the present application provides a credit risk identification method, and the credit risk identification method can be applied to any electronic device with credit risk identification capabilities, wherein the electronic devices include but are not limited to: personal mobile terminals, computer devices or servers, etc.

[0049] In some embodiments, the credit risk identification model training method can be as follows: Figure 1 As shown, the following steps are included:

[0050] S11, obtaining training sample data, wherein the training sample data is feature vector data extracted based on historical credit record data;

[0051] S12, input each feature vector in the training sample data into the initial credit risk identification model, and the credit risk identification model calculates the risk prediction result corresponding to each piece of training sample data based on the input feature vector; wherein the initial credit risk identification model includes: a main deep network and an adversarial sample generation network; the loss function in the adversarial sample generation network is an adversarial loss function, the adversarial sample generation network is used to generate an adversarial perturbation feature vector based on the data features generated by the main deep network with a delay, and the main deep network is used to output the risk prediction result corresponding to each piece of training sample data based on the adversarial perturbation feature vector and each feature vector in the training sample data;

[0052] S13, adjusting the model parameters of the credit risk identification model according to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data, until the target data difference is less than a preset data difference threshold;

[0053] S14. Determine the credit risk identification model corresponding to the target data difference being less than a preset data difference threshold as the target credit risk identification model.

[0054] The embodiment of the present application is selected to add an anti-interference network on the basis of the main deep network, and the anti-interference network generates an adversarial disturbance feature vector based on the data features generated by the main deep network with a delay, and then the main neural network is trained with the help of the adversarial disturbance feature vector and each feature vector in the original training sample data, until the data difference between the risk prediction result output by the main neural network and the actual risk overdue result corresponding to each training sample data is less than the preset data difference threshold. In the embodiment of the present application, the anti-interference network is introduced to generate the adversarial disturbance feature vector to enhance the ability of the main neural network to resist disturbance, thereby enhancing the performance of the entire credit risk identification model in the face of data distribution shift and noise disturbance, thereby enhancing the stability and reliability of the target credit risk identification model in practical applications.

[0055] The following will explain the above steps S11 to S14 with reference to specific examples:

[0056] In an embodiment of the present application, the training sample data may be original credit record data, such as basic information of credit users, credit records of credit users, loan status and repayment status of credit users, etc. As a preferred implementation, in order to improve training efficiency, the training sample data may be a feature vector after preliminary data processing based on the original credit record data. Exemplarily, the original credit record data is preprocessed and standardized to generate a feature vector X, which is the input of the model during subsequent model training. Preprocessing includes cleaning, word segmentation, stop word filtering, entity recognition, part-of-speech tagging, vectorization and other processing. Standardization processing specifically includes: normalization processing, centering processing, scaling, etc., to reduce data errors caused by differences in the dimension and range of feature vectors.

[0057] As an implementation method, preprocessing can be performed through feature selection and dimensionality reduction techniques (such as principal component analysis PCA or linear discriminant analysis LDA) to eliminate collinearity and reduce noise. Specifically, by reducing the dimension of input features and screening out the most representative features, the computational burden of model training can be reduced and the prediction performance of the model when facing unknown data can be improved.

[0058] As an implementation method, the feature vector obtained by preliminary data processing based on the original credit record can be stored in a specified storage path, and then when executing step S11, the training sample data can be obtained by accessing the storage path. As another implementation method, the training sample data can be obtained by scheduling the historical credit records of the credit application. The specific method of obtaining the training sample data can be flexibly selected according to actual needs, and this application does not strictly limit it.

[0059] In the embodiment of the present application, after executing step S11, the acquired training sample data is input into the initial credit risk identification model, and the credit risk identification model performs data processing based on the input training sample data, and outputs the corresponding credit risk prediction result. In the embodiment of the present application, the name of the credit risk identification model is different in different training stages of the model, but the network structure is the same, and only the parameters inside the model are different in different training stages, wherein the credit risk identification model before model training is the initial credit risk identification model, and the credit risk identification model that can accurately predict risks after the training is completed is the target credit risk identification model, wherein the target credit risk identification model is finally applied to the credit risk identification method of the second aspect to predict the credit risk of the user to be identified.

[0060] Regardless of which stage the credit risk identification model is in, it has two networks: the main deep network and the adversarial sample generation network. Specifically, the core of the main deep network is used to output the corresponding credit risk prediction results based on the input feature vector, and the adversarial sample generation network is used to provide interference training sample data for the main deep network training. Simply put, the adversarial sample generation network is used to generate a series of interference information to interfere with the training of the main deep network, challenge the training of the main deep network, improve the robustness of the main deep network, and enhance the credit risk identification ability of the trained main deep network when dealing with poor data, thereby improving the training accuracy of the main deep network.

[0061] Among them, as an implementation method, the credit risk prediction result specifically refers to whether the user will default in different credit cycles and the specific overdue probability, such as the overdue probability of user A in the case of 12 credit installments, where the overdue probability P = 0 means that user A will not default, and the overdue probability P is greater than 0 and less than or equal to 1, indicating that user A has an overdue, and the greater the overdue probability, the greater the risk that user A will not repay the credit loan.

[0062] In an embodiment of the present application, the credit risk identification model specifically includes: a main deep network and an adversarial sample generation network. Among them, the main deep network also includes: a first fully connected layer, a first coding layer, and an output label probability layer. The adversarial sample generation network also includes: a second fully connected layer, a second coding layer, and an adversarial loss layer. Among them, the output of the fully connected layer is the input of the first coding layer, the output of the first coding layer is the input of the output label probability layer, the output of the second fully connected layer is the input of the second coding layer, and the output of the second coding layer is the input of the adversarial loss layer. In other words, the connection relationship between the main deep network and the adversarial sample generation network is:

[0063] 1) The output of the first fully connected layer is the input of the second fully connected layer. The first fully connected layer of the main deep network can output to the second fully connected layer in real time, or it can output to the second fully connected layer according to a preset delay period, as shown in Figure 2 As shown, the first fully connected layer generates delayed data features with delay and inputs them into the second fully connected layer.

[0064] 2) The adversarial training sample data obtained by the output of the second coding layer is combined with the original training sample data to form new training sample data, and then the new training sample data is output to the first coding layer.

[0065] In some embodiments, the first fully connected layer outputs delayed data features to the second fully connected layer, so that the second fully connected layer and the second encoding layer generate an adversarial perturbation feature vector based on the input delayed data features.

[0066] In the embodiment of the present application, the first fully connected layer is used to map the input feature vector X to a high-dimensional data space and extract preliminary features present in the input feature vector X. Specifically, the first fully connected layer implements feature extraction through linear transformation and activation function (such as ReLU activation function), and outputs the extracted feature vector to the first coding layer as input of the first coding layer.

[0067] The first coding layer is used to perform high-level feature extraction on the feature vector output by the first fully connected layer and generate an intermediate representation corresponding to the training sample data. Specifically, the first coding layer further compresses and refines feature information through a multi-layer neural network structure.

[0068] The output label probability layer is used to output the predicted label probability distribution according to the intermediate representation output by the first coding layer, and the predicted label probability distribution is the risk prediction result in step S12. Specifically, the output label probability layer generates the probability distribution result of whether the credit loan is in default or overdue by using the Sigmod activation function, and the probability distribution result is the risk prediction result.

[0069] In the embodiment of the present application, the primary feature generated by the delay of the main deep network is called the delayed data feature, that is, the delayed data feature is: the first fully connected layer outputs the feature vector according to the preset delay period based on the input target training sample feature vector. Exemplarily, if the target training sample is the feature vector X1 corresponding to Zhang San's historical credit record data, the first fully connected layer maps the feature vector X1 to a high-dimensional space, extracts the preliminary feature X1', and then outputs X1' to the second fully connected layer every 12 hours according to the preset delay period, such as 12 hours.

[0070] The second fully connected layer is used to receive the delayed data feature and generate an adversarial disturbance feature vector. Specifically, the second fully connected layer maps the delayed data feature to a new data feature space through a linear transformation and an activation function to obtain a second delayed data feature, and the second delayed data feature is the input of the second coding layer. Similarly, taking the above-mentioned delayed data feature as X1' as an example, after the second fully connected layer receives X1', it maps X1' to a new data feature space through a linear transformation and an activation function to obtain a second delayed data feature x1, and outputs the second delayed data feature x1 to the second coding layer. Then, after the second coding layer receives the second delayed data feature x1, it generates an adversarial disturbance feature vector by maximizing the loss. Specifically, the second coding layer generates an adversarial disturbance feature vector x1' through a multi-layer neural network structure, and generates an adversarial disturbance feature vector by maximizing the loss in the process. In principle, the adversarial perturbation feature vector x1' is as dissimilar as possible to the feature vector X1 corresponding to the historical credit record data of Zhang San, the target training sample. This can confuse the main deep network and make it unable to classify correctly.

[0071] In some embodiments, the adversarial sample generation network further includes: an adversarial loss layer. The adversarial loss layer adjusts the adversarial perturbation feature vector x1' output by the second coding layer so that the difference between the output x1' and the original feature vector X1 gradually increases, so as to use x1' to challenge the main deep network and improve the robustness of the main deep network. Based on this, the adversarial loss layer is used to:

[0072] Receiving the adversarial perturbation feature vector output by the second coding layer according to the target training sample feature vector, and calculating the loss of the adversarial perturbation feature vector according to a preset loss function;

[0073] According to the loss of the adversarial perturbation feature vector, the network parameters of the adversarial sample generation network are adjusted until the adversarial feature vector generated by the adversarial sample generation network makes the preset loss function reach a maximum value.

[0074] Exemplarily, the adversarial perturbation feature vector x1' output by the second coding layer according to the target training sample feature vector X1 is received, and the difference between the adversarial perturbation feature vector x1' and the target training sample feature vector X1 is calculated according to the preset loss function. Compared with the loss function of the main deep network model, which expects the difference between the risk prediction result output by the main deep network and the actual risk overdue result to be as small as possible, the loss function of the adversarial sample generation network expects the difference between the adversarial perturbation feature vector x1' and the target training sample feature vector X1 to be as large as possible.

[0075] Based on this, in some embodiments, the preset loss function is a maximum entropy loss function, wherein the purpose of the maximum entropy loss function is to find a probability distribution under given constraints so as to maximize the entropy of the probability distribution. Here, entropy is the uncertainty or degree of confusion of a random variable. The larger the entropy, the greater the difference between the adversarial perturbation feature vector generated by the adversarial sample generation network and the target training sample feature vector, thereby better avoiding overfitting of the data.

[0076] Specifically, the maximum entropy loss function satisfies the following formula:

[0077]

[0078] in, It refers to the optimal solution selected by the i-th sample at the k-th iteration, x represents the sample, Y i Represents the label value corresponding to the i-th sample x; is the first cross entropy loss function, h(θ; x) is the information entropy loss function, is the Wasserstein distance function (i.e., Wassertein distance function), It means to find the element in set X that maximizes the objective function.

[0079] In the embodiment of the present application, in the first cross entropy loss function, θ is the network parameter of the adversarial sample generation network. The first cross entropy loss function is used to measure the effect of the network parameter of the adversarial sample generation network on the sample (x, Y i ), the goal is to make the adversarial perturbation feature vector output by the adversarial sample generation network as dissimilar as possible to the true label result.

[0080] In the embodiment of the present application, β is the first weight parameter, specifically the weight parameter of the information entropy loss function. Among them, the role of the information entropy loss function is to increase the prediction uncertainty of the adversarial perturbation feature vector generated by the adversarial sample generation network, thereby making the prediction result of the main deep network for the adversarial perturbation feature vector more uncertain. In layman's terms, it can be understood as generating adversarial perturbation training sample data that makes the main deep network model very uncertain about how to classify, so that the main deep network is "indecisive" when facing these adversarial perturbation training sample data.

[0081] In the embodiment of the present application, γ is the second weight parameter, specifically the weight parameter of the Wasserstein distance function. Among them, the role of the Wasserstein distance function is to ensure that the generated adversarial perturbation feature vector is as close as possible to the distribution of the original sample, so that the adversarial perturbation feature vector looks like normal training sample data. In layman's terms, it can be understood as ensuring that the adversarial perturbation training sample data looks as much like the original normal training sample data as possible to avoid the model identifying them as adversarial perturbation training sample data.

[0082] The adversarial perturbation feature vector generated by the adversarial sample generation network constructed based on the above maximum entropy loss function has the following advantages: it can make it difficult for the main deep network to classify, and it is similar to the original sample, thereby effectively improving the robustness of the main deep network. For example, in some practical application scenarios, credit users with a monthly income of more than 5,000 are considered high-quality users, but in fact, users with a monthly income of 4,999 can also be included in high-quality users. Using the embodiment of the present application, based on the sample with a monthly income of 5,000, adversarial perturbation samples such as 4,995, 4,996, 4,997, 4,998, 4,999, 5,000, 5,001, 5,002, etc. are generated. Since the adversarial perturbation feature vector and the target training sample feature vector have the same real data label, that is, 4,995, 4,996, 4,997, 4,998, 4,999, 5,000, 5,001, 5,002 have the same real data label as the value of 5,000. Therefore, the adversarial sample generation network trained by the embodiment of the present application can continuously provide adversarial samples around 5000 to the main deep network, and train the main deep network. The trained main deep network can accurately predict credit risks when the input monthly income is 4995, 4996, 4997, 4998, 4999, 5000, 5001, and 5002, which helps to improve the robustness of the main deep network.

[0083] On this basis, the training method of the credit risk identification model in the embodiment of the present application can be specifically divided into:

[0084] Execute step S11 to prepare data, obtain training sample data, specifically obtain historical data collected in the credit risk control business. Then, perform data cleaning and preprocessing to generate a standardized feature vector X.

[0085] Then, step S12 and step S13 are executed, and the feature vector X obtained after preprocessing is input into the main deep network, and the main deep network is trained to optimize the network parameters of the main deep network, so that the target data difference between the corresponding risk prediction result calculated based on the input feature vector X and the actual risk overdue result of each feature vector X (specifically represented by the real data label) is less than the preset data difference threshold. In this process, in some embodiments, the network parameters of the main deep network can be adjusted in the following way:

[0086] A second cross entropy loss function is constructed according to the risk prediction result and the actual risk overdue result, and the parameters of the main deep network are adjusted according to the function value of the second cross entropy loss function until the second cross entropy loss function converges.

[0087] As an implementation method, the parameters of the main deep network can be iteratively adjusted, and the network parameters of the main deep network can be gradually adjusted using a back propagation algorithm until the function value of the cross entropy loss function reaches a minimum value.

[0088] In the embodiment of the present application, the training of the credit risk identification model can be divided into three stages: the first stage: the initial training stage, during which the main deep network is trained separately; the second stage: the adversarial training stage, specifically the adversarial sample generation network is trained; the third stage: iterative optimization, specifically, during the training of the main deep network, the adversarial perturbation feature vector output by the adversarial sample generation network is introduced for adversarial interference training.

[0089] Among them, in the initial training stage, the feature vector output by the first fully connected layer will not be output to the second fully connected layer, which is equivalent to isolating the adversarial sample generation network. The first fully connected layer, the first encoding layer, and the output probability label layer of the main deep network perform data processing, and adjust the network parameters of the main deep network based on the cross entropy loss function.

[0090] For the second stage, based on the initial training stage, an adversarial sample generation network is introduced to conduct adversarial forest patrol. Specifically, through the delayed training method, different training rounds are divided according to the preset delay period, and the adversarial sample generation network is trained once after a certain period of time to avoid the overfitting problem between the adversarial sample generation network and the main deep network.

[0091] The second stage is implemented through the following steps:

[0092] Step 1: Get the intermediate representation from the first fully connected layer of the main deep network as the input of the adversarial sample generation network.

[0093] Step 2: The adversarial sample generation network generates an adversarial perturbation feature vector through the second fully connected layer and the second encoding layer.

[0094] Step 3: Calculate the adversarial loss (maximum entropy loss) and optimize the adversarial network parameters to make the generated adversarial sample features more "difficult" and increase the prediction uncertainty of the main deep network.

[0095] Step 4: Merge the adversarial sample with the original sample to form a new training set, which further helps to train the main deep network in the third stage. Merging the samples can be adding the adversarial perturbation feature vector to the vector set of the feature vector X corresponding to the original sample, and using the vector set to continue training the main deep network.

[0096] For the third stage, in some embodiments, it can be implemented as follows:

[0097] The main deep network and the adversarial sample generation network are trained alternately until the target data difference is less than a preset data difference threshold.

[0098] During adversarial training, the main deep network and the adversarial sample generation network update parameters alternately. In each iteration, the adversarial sample generation network generates adversarial perturbation feature vector samples, and the main deep network is trained on the expanded training set. Through repeated iterative optimization, the adversarial sample generation network continuously generates more challenging adversarial perturbation feature vector samples, and the main deep network gradually improves its coping ability, thereby achieving the purpose of improving the robustness of the model.

[0099] Specifically, the main deep network and the adversarial sample generation network are optimized alternately and iterated multiple times until the cross entropy loss function converges and the maximum cross entropy loss function converges. In each iteration, the adversarial perturbation feature vector generated by the adversarial sample generation network is used to update the main deep network, making it more robust to data distribution shifts and noise perturbations.

[0100] On this basis, the training effect of the credit risk identification model can be evaluated in the following ways:

[0101] A test training sample data set is input into the trained credit risk identification model, and the credit risk identification model outputs the corresponding risk prediction results based on the input test training sample data set. Performance indicators are then calculated based on the output risk prediction results, including indicators such as the area under the curve, accuracy, and recall rate, and performance evaluation is performed based on the calculated indicators.

[0102] In a second aspect, the present application provides a credit risk identification method, wherein, Figure 3 As shown, the method comprises the following steps:

[0103] S31, obtaining user characteristics of the user to be identified, and inputting the user characteristics into the target credit risk identification model;

[0104] S32, determining the overdue probability of the to-be-identified user in each credit cycle based on the user characteristics by the target credit risk identification model;

[0105] Wherein, the target credit risk identification model is a network model pre-trained by using the credit risk identification model training method as described in the first aspect.

[0106] Among them, obtaining the user characteristics of the user to be identified can be to obtain the user portrait of the user to be identified from the credit business data interface, or the historical credit record of the user to be identified. The embodiment of the present application is selected because the credit risk identification model can cope with different degrees of data distribution transfer and noise disturbance and output accurate credit risk prediction results. Based on this, the overdue probability of the user to be identified in each credit cycle can be calculated with the help of the target credit risk identification model, thereby increasing the accuracy of credit risk identification.

[0107] In addition, machine learning integration methods such as random forest, boosting method or stacked generalization can be used to combine multiple models for credit risk prediction, further improving the accuracy and robustness of the overall prediction.

[0108] In a third aspect, the present application provides a credit risk identification model training device, wherein, Figure 4 As shown, the device 40 includes:

[0109] An input module 401 is used to obtain training sample data, wherein the training sample data is feature vector data extracted based on historical credit record data;

[0110] The training module 402 is used to input each feature vector in the training sample data into the initial credit risk identification model, and the credit risk identification model calculates the risk prediction result corresponding to each piece of training sample data based on the input feature vector; wherein the initial credit risk identification model includes: a main deep network and an adversarial sample generation network; the loss function in the adversarial sample generation network is an adversarial loss function, the adversarial sample generation network is used to generate an adversarial perturbation feature vector based on the data features generated by the main deep network with a delay, and the main deep network is used to output the risk prediction result corresponding to each piece of training sample data based on the adversarial perturbation feature vector and each feature vector in the training sample data;

[0111] The parameter adjustment module 403 is used to adjust the model parameters of the credit risk identification model according to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data, until the target data difference is less than a preset data difference threshold;

[0112] The determination module 404 is used to determine the credit risk identification model corresponding to the target data difference being less than a preset data difference threshold as the target credit risk identification model.

[0113] In a fourth aspect, the present application provides a credit risk identification device, wherein: Figure 5 As shown, the device 50 comprises:

[0114] An acquisition module 501 is used to acquire user characteristics of the user to be identified and input the user characteristics into a target credit risk identification model;

[0115] An identification module 502, used for the target credit risk identification model to determine the overdue probability of the to-be-identified user in each credit cycle based on the user characteristics;

[0116] Wherein, the target credit risk identification model is a network model pre-trained by using the credit risk identification model training method as described in the first aspect.

[0117] Among them, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with the relevant laws and regulations and do not violate public order and good morals.

[0118] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0119] In a fifth aspect, the exemplary embodiments of the present application further provide an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and when the computer program is executed by the at least one processor, the electronic device performs a method according to an embodiment of the present application.

[0120] In a sixth aspect, an exemplary embodiment of the present application further provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present application.

[0121] In a seventh aspect, an exemplary embodiment of the present application further provides a computer program product, comprising a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to perform a method according to an embodiment of the present application.

[0122] refer to Figure 6 , the structural block diagram of the electronic device 600 that can be used as the server or client of the present application will now be described, which is an example of a hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent various forms of digital electronic computer equipment, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0123] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0124] A plurality of components in the electronic device 600 are connected to the I / O interface 605, including: an input unit 606, an output unit 607, a storage unit 608, and a communication unit 609. The input unit 606 may be any type of device capable of inputting information to the electronic device 600, and the input unit 606 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 607 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 608 may include, but is not limited to, a disk, an optical disk. The communication unit 609 allows the electronic device 600 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0125] The computing unit 601 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above. For example, in some embodiments, the aforementioned credit risk identification model training method and credit risk identification method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 608. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 600 via the ROM 602 and / or the communication unit 609. In some embodiments, the computing unit 601 may be configured to perform the aforementioned credit risk identification model training method and credit risk identification method in any other appropriate manner (e.g., by means of firmware).

[0126] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0127] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0128] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0131] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

Claims

1. A credit risk identification model training method, characterized in that: The method comprises: Acquire training sample data, wherein the training sample data is feature vector data extracted based on historical credit record data; Input each feature vector in the training sample data into the initial credit risk identification model, and the credit risk identification model calculates the risk prediction result corresponding to each piece of training sample data based on the input feature vector; wherein the initial credit risk identification model includes: a main deep network and an adversarial sample generation network; the loss function in the adversarial sample generation network is an adversarial loss function, the adversarial sample generation network is used to generate an adversarial perturbation feature vector based on the delayed data feature generated by the main deep network delay, and the main deep network is used to output the risk prediction result corresponding to each piece of training sample data based on the adversarial perturbation feature vector and each feature vector in the training sample data; According to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data, the model parameters of the credit risk identification model are adjusted until the target data difference is less than a preset data difference threshold; The credit risk identification model corresponding to the target data difference being less than a preset data difference threshold is determined as the target credit risk identification model.

2. The method according to claim 1, characterized in that: The main deep network includes: a first fully connected layer, a first encoding layer, and an output label probability layer; the adversarial sample generation network includes: a second fully connected layer and a second encoding layer, wherein the first fully connected layer outputs delayed data features to the second fully connected layer, so that the second fully connected layer and the second encoding layer generate adversarial perturbation feature vectors based on the input delayed data features.

3. The method according to claim 2, characterized in that The delay data feature generated by the main deep network delay is: the first fully connected layer outputs a feature vector according to a preset delay period based on the input target training sample feature vector; the adversarial sample generation network also includes: an adversarial loss layer, which is used to: Receiving the adversarial perturbation feature vector output by the second coding layer according to the target training sample feature vector, and calculating the loss of the adversarial perturbation feature vector according to a preset loss function; According to the loss of the adversarial perturbation feature vector, the network parameters of the adversarial sample generation network are adjusted until the adversarial feature vector generated by the adversarial sample generation network makes the preset loss function reach a maximum value.

4. The method according to claim 3, characterized in that The preset loss function is a maximum entropy loss function, which satisfies the following formula: in, It refers to the optimal solution selected by the i-th sample at the k-th iteration. is the first cross entropy loss function, h(θ; x) is the information entropy loss function, β is the weight of the information entropy loss function, is the Wasserstein distance function, γ is the weight of the Wasserstein distance function, where x represents the sample, Y i is the label value corresponding to the i-th sample x, θ is the network parameter of the adversarial sample generation network, It means to find the element in set X that maximizes the objective function.

5. The method according to claim 2, characterized in that: The step of adjusting the model parameters of the credit risk identification model according to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data until the target data difference is less than a preset data difference threshold comprises: A second cross entropy loss function is constructed according to the risk prediction result and the actual risk overdue result, and the parameters of the main deep network are adjusted according to the function value of the second cross entropy loss function until the second cross entropy loss function converges.

6. The method according to claim 2, characterized in that The method further comprises: The main deep network and the adversarial sample generation network are trained alternately until the target data difference is less than a preset data difference threshold.

7. A credit risk identification method, characterized in that: The method comprises: Acquire user characteristics of the user to be identified, and input the user characteristics into a target credit risk identification model, so that the target credit risk identification model determines the overdue probability of the user to be identified in each credit cycle based on the user characteristics; Wherein, the target credit risk identification model is a network model pre-trained by the credit risk identification model training method according to any one of claims 1 to 6.

8. A credit risk identification model training device, characterized in that: The device comprises: An input module, used to obtain training sample data, wherein the training sample data is feature vector data extracted based on historical credit record data; A training module, used to input each feature vector in the training sample data into an initial credit risk identification model, and the credit risk identification model calculates the risk prediction result corresponding to each piece of training sample data based on the input feature vector; wherein the initial credit risk identification model includes: a main deep network and an adversarial sample generation network; the loss function in the adversarial sample generation network is an adversarial loss function, the adversarial sample generation network is used to generate an adversarial perturbation feature vector based on the data features generated by the main deep network with a delay, and the main deep network is used to output the risk prediction result corresponding to each piece of training sample data based on the adversarial perturbation feature vector and each feature vector in the training sample data; A parameter adjustment module, used to adjust the model parameters of the credit risk identification model according to the target data difference between the risk prediction result corresponding to each piece of training sample data and the actual risk overdue result corresponding to each piece of training sample data, until the target data difference is less than a preset data difference threshold; The determination module is used to determine the credit risk identification model corresponding to the target data difference being less than a preset data difference threshold as the target credit risk identification model.

9. A credit risk identification device, characterized in that: The device comprises: An acquisition module, used to acquire user characteristics of the user to be identified, and input the user characteristics into a target credit risk identification model; An identification module, used for the target credit risk identification model to determine the overdue probability of the user to be identified in each credit cycle based on the user characteristics; Wherein, the target credit risk identification model is a network model pre-trained by the credit risk identification model training method according to any one of claims 1 to 6.

10. An electronic device, characterized in that: The electronic device comprises: Processor; and Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1-6 or 7.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-6, or 7.

Images