Credit risk prediction method based on gating feature selection and dynamic modulation mechanism

Abstract

The application discloses a credit risk prediction method based on a gating feature selection and dynamic modulation mechanism, relates to the technical field of credit risk control, and solves the technical problem that the existing technology cannot simultaneously realize the pertinence of feature selection and the adaptability of cross-feature interaction, so that the model generalization ability and risk identification stability still need to be further improved; the application comprises the following steps: gradually selecting and representing static financial features through a gating mechanism, and dynamically modulating the representation process of time sequence behavior features on the basis, so that the model can simultaneously mine local feature correlation and consider global time sequence dependency, thereby improving the prediction accuracy, generalization ability and robustness of the model under complex financial behavior data and diversified risk modes, and better meeting the application requirement of high-precision risk identification of actual financial credit risk control business.

Description

Technical Field

[0001] This invention relates to the field of credit risk control technology, specifically to a credit risk prediction method based on gating feature selection and dynamic modulation mechanism. Background Technology

[0002] In financial credit risk control scenarios, with the continuous improvement of business digitalization and online operations, financial institutions have accumulated a large amount of borrower-related data during credit approval and post-loan management. This type of data typically includes static features such as basic user attributes and credit profiles, as well as temporal features such as transaction records and lending behavior sequences, exhibiting characteristics of high dimensionality, complex structure, significant nonlinear relationships, and the coexistence of heterogeneous features. How to effectively model multi-source financial features and fully explore the potential correlations between different features has significant application value and practical significance for accurately identifying borrower default risk, improving the accuracy and timeliness of credit decisions, and reducing the bad debt ratio and operational risk of financial institutions.

[0003] However, existing financial risk control technologies still have shortcomings in multidimensional heterogeneous feature modeling. On the one hand, traditional risk assessment methods based on logistic regression, support vector machines, random forests, or gradient boosting decision trees typically rely on manually constructed features or simple feature selection mechanisms. These methods struggle to dynamically identify feature subsets that have a critical impact on risk assessment within a high-dimensional feature space, and their ability to characterize complex nonlinear relationships and conditional dependencies between features is limited. On the other hand, while some deep learning methods introduce automatic feature learning mechanisms, they often employ a method of directly concatenating or weighted fusion after modeling static and temporal features separately. This lacks effective modeling of the mutual influence relationships between different feature branches, making it difficult to dynamically adjust the expression of temporal behavioral features based on borrower static attributes, or to reverse-correct the importance of static features based on behavioral changes. In high-noise, highly heterogeneous financial data environments, these methods often struggle to simultaneously achieve targeted feature selection and adaptive cross-feature interactions. This results in the model's generalization ability and risk identification stability still needing further improvement, making it difficult to meet the practical requirements for high accuracy and robustness in complex credit risk control scenarios. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides a credit risk prediction method based on gated feature selection and dynamic modulation mechanism. This method solves the technical problem that the prior art is unable to simultaneously achieve the targeting of feature selection and the adaptability of cross-feature interaction, resulting in the need for further improvement in the model's generalization ability and risk identification stability.

[0005] A credit risk prediction method based on gating feature selection and dynamic modulation mechanism includes: training a discriminant classification network using a training set; using the trained discriminant classification network to determine whether a user defaults based on input data; the judgment process of the discriminant classification network includes: preprocessing the input data and dividing it into static data and time-series data; using a feature allocation module to perform stepwise feature extraction on the static data to obtain a static feature embedding representation; using a deep fusion model to extract features from the time-series data to obtain a time-series feature representation; using the static feature embedding representation to conditionally modulate the time-series feature representation to obtain conditional time-series features, and fusing the static feature embedding representation and the conditional time-series features through an attention mechanism as a dynamic feature representation; concatenating the static feature embedding representation and the dynamic feature representation and inputting them into a classifier module, which then performs the judgment on the user's credit default probability.

[0006] Furthermore, the generation of the dynamic feature representation includes: constructing a modulation vector for modulating the temporal feature modeling process based on the static feature embedding representation; performing feature-level modulation on the temporal feature representation using the modulation vector to generate a conditional temporal feature representation; and using an additive / multiplicative attention mechanism to model the importance of the conditional temporal feature representation at different time steps, calculating the attention weights constrained by the static semantic conditions, thereby realizing the conditional dependency modeling between static semantics and temporal dynamics to obtain the dynamic feature representation.

[0007] Furthermore, the feature allocation module includes a feature encoder, a multi-step gating decision layer, and a step-by-step output layer;

[0008] The feature encoder adopts a shared fully connected network structure, consisting of a linear mapping layer, a batch normalization layer, a GeLU nonlinear activation function, and Dropout, which is used to map the original static features to a low-dimensional latent feature space to enhance feature expressive power and suppress noisy features.

[0009] The gating decision layer is implemented using learnable, continuous gating masks, and its mathematical formula is as follows:

[0010]

[0011] Where x is the input feature, Let f be the gating weight at step t, and f be the shared feature encoder mapping function. This is the residual mask from the previous step. The function is Sigmoid; the number of gated steps is explicitly specified by the model hyperparameters, and each step uses independent gate mapping parameters. Features selected in the current step are suppressed in subsequent steps, as shown in the following formula:

[0012]

[0013] Each step outputs a low-order embedding vector. The feature representations output by each decision step are concatenated along the channel dimension, and the numerical distribution of the output features from different decision steps is unified by batch normalization, finally yielding a static feature embedding representation.

[0014] Furthermore, the deep fusion model includes a cascaded local feature extraction module based on a convolutional neural network and a sequence-time simulation module based on a bidirectional long short-term memory network. The local feature extraction module processes the input time-series data through two layers of one-dimensional convolution and max pooling, outputting a sequence with enhanced local features and compressed time dimension. The sequence-time simulation module processes the compressed sequence output by the local feature extraction module through the bidirectional long short-term memory network, outputting a hidden state sequence at each time step that integrates contextual information as a time-series feature representation.

[0015] Furthermore, the modulation vector is generated by static feature embedding through linear mapping and a sigmoid nonlinear activation function, including a bias vector for additive modulation and a gate vector for multiplicative modulation, and its generation is mathematically formulated as follows:

[0016]

[0017]

[0018] Where α is the gating vector, β is the bias vector, and s is the static feature embedding.

[0019] A storage medium storing computer instructions, characterized in that: when the computer instructions are executed, they perform the steps of the aforementioned credit risk prediction method based on gating feature selection and dynamic modulation mechanism.

[0020] A terminal includes a memory and a processor, wherein the memory stores computer instructions executable on the processor, characterized in that the processor executes the steps of the aforementioned financial credit risk prediction method based on gating feature selection and dynamic modulation mechanism when executing the computer instructions.

[0021] The beneficial effects of this invention include:

[0022] This invention introduces a gated feature selection mechanism in the process of financial credit risk prediction, which progressively filters and weights high-dimensional static financial features. This enables adaptive identification of feature subsets that have a key impact on risk judgment during model training, reducing the interference of irrelevant or redundant features on the model prediction results, thereby improving the pertinence and stability of feature representation.

[0023] Furthermore, this invention introduces a dynamic modulation mechanism between static feature modeling and temporal behavioral feature modeling, enabling the model to adaptively adjust the expression process of temporal features based on the borrower's static attribute information. This effectively characterizes the conditional dependencies between different feature branches while mining local behavioral patterns and global temporal dependencies, avoiding the problem of insufficient information utilization caused by simply splicing multi-source features in the prior art.

[0024] Furthermore, this invention combines the ability of convolutional neural networks to extract local feature associations in time-series behavioral data with the ability of bidirectional long short-term memory networks to model long-term temporal dependencies. Under the constraint of a dynamic modulation mechanism, it achieves collaborative modeling of complex financial behavior patterns. This enables the model to achieve higher classification prediction accuracy and better generalization performance when facing complex cluster labels, high noise, and non-stationary financial data, and has strong practical value and promotion significance. Attached Figure Description

[0025] Figure 1 This is the data processing flow of the credit risk prediction method based on gating feature selection and dynamic modulation mechanism involved in the embodiments of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0027] Example 1

[0028] The following is in conjunction with the appendix Figure 1 Specific embodiments of the present invention will be described in detail;

[0029] A credit risk prediction method based on gating feature selection and dynamic modulation mechanism includes: training a discriminant classification network using a training set; using the trained discriminant classification network to determine whether a user defaults based on input data; the judgment process of the discriminant classification network includes: preprocessing the input data and dividing it into static data and time-series data; using a feature allocation module to perform stepwise feature extraction on the static data to obtain a static feature embedding representation; using a deep fusion model to extract features from the time-series data to obtain a time-series feature representation; using the static feature embedding representation to conditionally modulate the time-series feature representation to obtain conditional time-series features, and fusing the static feature embedding representation and the conditional time-series features through an attention mechanism as a dynamic feature representation; concatenating the static feature embedding representation and the dynamic feature representation and inputting them into a classifier module, which then performs the judgment on the user's credit default probability.

[0030] The generation of the dynamic feature representation includes: constructing a modulation vector for modulating the temporal feature modeling process based on the static feature embedding representation; using the modulation vector to perform feature-level modulation on the temporal feature representation to generate a conditional temporal feature representation; using an additive / multiplicative attention mechanism to model the importance of the conditional temporal feature representation at different time steps, and calculating the attention weights constrained by the static semantic conditions, thereby realizing the conditional dependency modeling between static semantics and temporal dynamics to obtain the dynamic feature representation.

[0031] Specifically, the training set contains real data provided by financial lending platforms, including static data describing user profiles and time-series data describing user credit behavior. All feature data are abstracted into numerical types;

[0032] The input feature data is preprocessed and then input into the branch for processing static features and the branch for processing time-series features, respectively, according to feature type.

[0033] The preprocessing includes filtering static feature columns, imputing missing values, and performing standardization on all feature values. Static features with a missing rate greater than a set value are excluded. The missing static features are imputed by the mean, and the missing time-series features are imputed. Finally, Z-score standardization is performed on each retained feature.

[0034] The static feature allocation module is the GSFS module, which uses a gated stepwise feature allocation method to extract features from the static feature data as a potential representation of user static attributes in the risk semantic space.

[0035] The GSFS module consists of a shared feature encoder, a multi-step gating decision layer, and a step-by-step output layer.

[0036] The feature encoder employs a shared fully connected network structure, consisting of a linear mapping layer, a batch normalization layer, a GeLU nonlinear activation function, and Dropout. It is used to map the original static features to a low-dimensional latent feature space (64 dimensions) to enhance feature expressive power and suppress noisy features.

[0037] The gating decision layer is implemented using learnable, continuous gating masks, and its mathematical formula is as follows:

[0038]

[0039] Where x is the input feature, Let f be the gating weight at step t, and f be the shared feature encoder mapping function. σ is the residual mask from the previous step, and σ is the Sigmoid function.

[0040] The number of gated steps is explicitly specified as 3 by the model hyperparameters, and each step uses independent gate mapping parameters. Features selected in the current step are suppressed in subsequent steps, as shown in the following formula:

[0041]

[0042] Each step outputs a 43-dimensional low-order embedding vector. The feature representations output by each decision step are concatenated along the channel dimension, and the numerical distribution of the output features from different decision steps is unified by batch normalization, finally resulting in a 129-dimensional static feature embedding representation.

[0043] This module can control model complexity while enabling explicit feature selection and enhanced feature representation capabilities, and also possesses a certain degree of interpretability.

[0044] The deep fusion model consists of a local feature extraction module based on a convolutional neural network and a sequence time simulation module based on a bidirectional long short-term memory network connected in series.

[0045] The local feature extraction module based on the convolutional neural network (CNN) adopts a two-layer cascaded structure of one-dimensional convolution and max pooling. The convolution kernel size is 3, the number of output channels of the first layer is 128, and the number of output channels of the second layer is 256. Batch normalization and ReLU non-linear activation function are used after each convolution.

[0046] One-dimensional convolution performs a sliding window operation in the time dimension, which can model feature combinations within consecutive time steps and is suitable for capturing local change patterns in financial behavior sequences.

[0047] By using multi-layer convolution and pooling operations, the model can gradually expand its receptive field while reducing temporal resolution, thereby suppressing noise and highlighting key behavioral segments.

[0048] Financial behavior data often contains abnormal behavior patterns within a short period of time, such as sudden large transactions, continuous loan applications, or abnormal repayment operations.

[0049] Convolutional neural networks can effectively capture such short-term, high-frequency change features through local receptive field mechanisms without explicitly modeling global dependencies; while the sequence time simulation module BiLSTM of bidirectional long short-term memory networks can simultaneously utilize historical and future contextual information to model financial behavior sequences with periodic or phased characteristics (such as repayment cycles and changes in lending rhythm), making it suitable for capturing long-term dependencies.

[0050] Compared to the Transformer architecture, CNN combined with BiLSTM offers advantages such as lower parameter count and higher training stability on small to medium-sized financial and credit datasets, while not relying on large-scale pre-training corpora. Furthermore, while maintaining modeling capabilities, this architecture's computational complexity increases linearly with sequence length, better meeting the deployment requirements of practical risk control systems.

[0051] Based on static feature embedding representation, a modulation vector is constructed for the modulation time-series feature modeling process. The modulation vector is generated by static feature embedding through a linear mapping from 129 dimensions to 512 dimensions and a sigmoid nonlinear activation function. It includes a bias vector for additive modulation and a gate vector for multiplicative modulation. The mathematical formula for its generation is as follows:

[0052]

[0053]

[0054] Where α is the gating vector, β is the bias vector, and s is the static feature embedding.

[0055] Using modulation vectors, the intermediate hidden states (i.e., temporal feature representations) generated at each time step of the BiLSTM layer are additively / multiplicatively modulated at the feature level to generate conditional temporal feature representations. Additive bias is used to introduce static user profile information without altering the original temporal dynamic structure, essentially providing a personalized baseline for temporal behavior. In financial scenarios, static characteristics such as income levels and occupational attributes of different users have a long-term impact on their behavioral patterns; this bias mechanism can characterize this long-term prior. Multiplicative gating is used to recalibrate the importance of different dimensions of the temporal hidden states, enabling the model to dynamically emphasize or suppress certain behavioral patterns based on user static characteristics. For example, for users with high-risk profiles, their abnormal trading behavior should be given higher weight in risk assessment. The mathematical formula for modulation is as follows:

[0056]

[0057] in, This represents the hidden state of the BiLSTM at time step t. This modulation mechanism achieves the same behavior but with different risk implications for different user profiles.

[0058] Based on the conditional temporal feature representation, an additive attention mechanism is used to model the importance of different time steps, and the attention weights constrained by static semantic conditions are calculated. This achieves the conditional dependency modeling between static semantics and temporal dynamics, resulting in a vector representation that serves as the dynamic feature representation of the borrower's behavior sequence in the risk pattern space. The calculation process is as follows:

[0059]

[0060]

[0061]

[0062] In the formula, Indicates the first Conditionalized temporal features at each time step, with dimensions of ; Represents the learnable weight matrix; This represents the hyperbolic tangent activation function, which introduces nonlinearity, enabling the model to capture more complex dependencies; Indicates the first The unnormalized attention score at each time step reflects the importance of that time step to the final output. This represents an exponential function that converts fractions to non-negative values. Indicates the first Attention weights at each time step, satisfying This indicates the proportion of contribution of that time step to the final feature; This represents the final dynamic feature representation, with dimensions and The same is a weighted average of the conditional features of all time steps, where the weights are dynamically allocated by the attention mechanism based on the content of each time step.

[0063] The calculation of attention weights directly affects the conditional feature representation, ensuring that the weight distribution not only reflects the dynamic changes of the time series itself but also implicitly depends on static semantic information, thereby enabling adaptive selection of key time steps. Subsequently, the conditional temporal features are weighted and aggregated according to the attention weights to obtain a fixed-dimensional dynamic feature embedding representation.

[0064] The static and dynamic feature representations are concatenated and then passed through a classifier module to infer the likelihood of a user's credit default.

[0065] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A credit risk prediction method based on gating feature selection and dynamic modulation mechanism, characterized in that, include: Train a discriminative classification network using the training set; The trained discriminant classification network is used to determine whether a user has breached the contract based on the input data. The judgment process of the discriminant classification network includes: preprocessing the input data and dividing it into static data and time-series data; using a feature allocation module to perform stepwise feature extraction on the static data to obtain a static feature embedding representation; using a deep fusion model to extract features from the time-series data to obtain a time-series feature representation; using the static feature embedding representation to conditionally modulate the time-series feature representation to obtain conditional time-series features, and fusing the static feature embedding representation and the conditional time-series features as a dynamic feature representation through an attention mechanism; concatenating the static feature embedding representation and the dynamic feature representation and inputting them into the classifier module, which then performs the judgment on the user's credit default probability.

2. The credit risk prediction method based on gating feature selection and dynamic modulation mechanism according to claim 1, characterized in that, The generation of the dynamic feature representation includes: constructing a modulation vector for modulating the temporal feature modeling process based on the static feature embedding representation; using the modulation vector to perform feature-level modulation on the temporal feature representation to generate a conditional temporal feature representation; using an additive / multiplicative attention mechanism to model the importance of the conditional temporal feature representation at different time steps, and calculating the attention weights constrained by the static semantic conditions, thereby realizing the conditional dependency modeling between static semantics and temporal dynamics to obtain the dynamic feature representation.

3. The credit risk prediction method based on gating feature selection and dynamic modulation mechanism according to claim 1, characterized in that, The feature allocation module includes a feature encoder, a multi-step gating decision layer, and a step-by-step output layer; The feature encoder adopts a shared fully connected network structure, consisting of a linear mapping layer, a batch normalization layer, a GeLU nonlinear activation function, and Dropout, which is used to map the original static features to a low-dimensional latent feature space to enhance feature expressive power and suppress noisy features. The multi-step gating decision layer is implemented using learnable continuous gating masks, and its mathematical formula is as follows: Where x is the input feature, Let f be the gating weight at step t, and f be the shared feature encoder mapping function. This is the residual mask from the previous step. The function is Sigmoid; the number of gated steps is explicitly specified by the model hyperparameters, and each step uses independent gate mapping parameters. Features selected in the current step are suppressed in subsequent steps, as shown in the following formula: Each step outputs a low-order embedding vector. The feature representations output by each decision step are concatenated along the channel dimension, and the numerical distribution of the output features from different decision steps is unified by batch normalization, finally yielding a static feature embedding representation.

4. The credit risk prediction method based on gating feature selection and dynamic modulation mechanism according to claim 1, characterized in that, The deep fusion model includes a local feature extraction module based on a convolutional neural network and a sequence temporal simulation module based on a bidirectional long short-term memory network, which are connected in series. The local feature extraction module processes the input temporal data through two layers of one-dimensional convolution and max pooling to output a sequence with enhanced local features and compressed temporal dimension. The sequence temporal simulation module processes the compressed sequence output by the local feature extraction module through a bidirectional long short-term memory network to output a hidden state sequence that integrates contextual information at each time step as a temporal feature representation.

5. The credit risk prediction method based on gating feature selection and dynamic modulation mechanism according to claim 2, characterized in that, The modulation vector is generated by static feature embedding through linear mapping and a Sigmoid nonlinear activation function, including a bias vector for additive modulation and a gate vector for multiplicative modulation.

6. A storage medium storing computer instructions thereon, characterized in that: When the computer instructions are executed, they perform the steps of the credit risk prediction method based on gating feature selection and dynamic modulation mechanism according to any one of claims 1-5.

7. A terminal comprising a memory and a processor, wherein the memory stores computer instructions executable on the processor, characterized in that, When the processor executes the computer instructions, it performs the steps of the financial credit risk prediction method based on gating feature selection and dynamic modulation mechanism according to any one of claims 1-5.

Images