Credit risk dynamic feature extraction method based on hierarchical reinforcement learning

Abstract

The invention relates to the technical field of credit risk assessment, in particular to a credit risk dynamic feature extraction method based on hierarchical reinforcement learning. The layered reinforcement learning-based credit risk dynamic feature extraction method comprises the following steps of S1, obtaining credit risk data, and constructing a knowledge graph; s2, extracting candidate feature attributes, constructing an initial risk tag, and generating first-time multi-dimensional vector sample data; s3, performing dynamic feature engineering processing, and performing re-calibration and iterative updating on the initial risk label of the first-time multi-dimensional vector sample data; s4, performing spatio-temporal feature fusion processing, performing risk dynamic assessment, and calculating a risk prediction value corresponding to the time sequence feature; and S5, constructing a layered DQN framework, and dynamically outputting an interpretable risk feature set in combination with SHAP value quantification contribution verification. According to the method, high-precision and high-interpretability credit risk assessment is realized through multi-source data deep integration, self-attention driven feature engineering and time-space fusion risk quantification.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of credit risk assessment, and particularly relates to a credit risk dynamic feature extraction method based on hierarchical reinforcement learning. BACKGROUND

[0002] Credit risk feature extraction research is an important basis for market subject credit risk evaluation and early warning. In the field of credit risk assessment, traditional methods mostly rely on single data source to construct static models, which have the following significant problems: cross-department data is difficult to integrate due to format inconsistency and entity identification conflict, resulting in incomplete feature coverage and affecting the comprehensiveness of risk assessment; traditional models rely on fixed rules or historical labels and cannot capture dynamic risk evolution, so they are not real-time and adaptive; manual feature screening is easily limited by subjective experience, lacks an automated mechanism to remove redundant or low-contribution features, resulting in high model complexity and low efficiency; black-box models cannot provide clear basis for risk decision-making, which does not meet the compliance requirements of financial risk control. In addition, due to the rapid development of China's economy and society, market subject related data presents the characteristics of large increment, frequent change and high dimension, while related technical research is more focused on credit risk feature extraction for specific departments, types or fields, without considering the impact of multi-dimensional credit risk data under incremental update scenarios. Therefore, how to analyze multi-dimensional credit risk data under incremental update through technical means to identify and extract credit risk features has become a problem to be solved.

[0003] For example, the patent with the publication number CN115496357A, titled "Enterprise credit risk early warning method and system based on fusion of forum text time sequence features", selects forum text as a single data source. There may be a lot of noise in the text in the forum, such as irrelevant information, advertisements or false comments, which will affect the accuracy of sentiment analysis. Fixed division by day, week and month does not take into account the differences in forum activity of different industries or enterprises. Low-frequency forums may lead to sparse data when divided by day, while high-activity forums may lose short-term fluctuation information when divided by month. Using mutual information for feature selection may ignore the interaction between features. Adaboost is sensitive to noise data, and the labeling noise of forum text may cause Adaboost to overfit to noise data, reducing model robustness. For example, the published paper (doi:10.3969 / j.issn.1007-130X.2023.12.018), titled "Personal credit risk prediction based on multi-scale deep feature fusion", although XGBoost itself has interpretability, the deep features extracted through multi-scale convolution and attention fusion lack clear business meaning. In the field of financial risk control, the interpretability of model decision is crucial, and the paper does not provide feature importance analysis or visual explanation; the paper focuses on one-dimensional personal credit feature extraction and does not consider that market subject credit risk data is a high-dimensional data that is incrementally updated, which will involve multi-dimensional correlation analysis of multiple different attributes.

[0004] Therefore, there is still a lack of hierarchical extraction and analysis scheme for dynamic features of multi-subject credit risk. SUMMARY

[0005] The present application provides a credit risk dynamic feature extraction method based on hierarchical reinforcement learning, which realizes high-precision and high-explainability credit risk assessment through multi-source data deep integration, self-attention driven feature engineering, spatio-temporal fusion risk quantification and hierarchical decision mechanism.

[0006] To achieve the above purpose, the present application provides a credit risk dynamic feature extraction method based on hierarchical reinforcement learning, comprising the following steps:

[0007] S1, acquiring credit risk data of different fields corresponding to a plurality of sample subjects, and constructing a knowledge graph;

[0008] S2, extracting candidate feature attributes from the knowledge graph, constructing an initial risk label, and generating a first multi-dimensional vector sample data;

[0009] S3, performing dynamic feature engineering processing on the first multi-dimensional vector sample data, constructing a double-channel integrated hybrid supervised learning model based on a rule engine and a neural network, and recalibrating and iteratively updating the initial risk label of the first multi-dimensional vector sample data;

[0010] S4, performing spatiotemporal feature fusion processing on the first multi-dimensional vector sample data processed in S3 to obtain a time sequence feature, performing dynamic risk assessment on the time sequence feature, and calculating a risk prediction value corresponding to the time sequence feature;

[0011] S5, constructing a hierarchical DQN framework, based on the hierarchical DQN framework, combining SHAP value quantitative contribution verification, and dynamically outputting an interpretable risk feature set, the interpretable risk feature set including a feature attribute, a risk label and a normalized weight.

[0012] Preferably, the constructing a knowledge graph in step S1 specifically includes:

[0013] Step 1.1 data standardization and field mapping;

[0014] Step 1.2 data cleaning and missing value processing;

[0015] Step 1.3 entity alignment unique identification.

[0016] The first multi-dimensional vector sample data in step S2 is composed of multi-dimensional vectors, and the form is , wherein represents a candidate feature attribute, represents a risk label;

[0017] The extracting a candidate feature attribute from the knowledge graph and constructing an initial risk label specifically includes:

[0018] Step S2, entity attribute extraction is performed on the constructed knowledge graph node to obtain static attributes, relationship feature extraction is performed based on the edges of the knowledge graph to generate dynamic features, and the graph structure is mapped into a low-dimensional vector, the expression of the target function is:

[0019] ;

[0020] In the formula, is the embedding vector of node , is a neighbor node sampled by biased random walk, is a node set, is a mapping function, is a probability function. The initial risk label is preliminarily marked by a rule engine or constructed by historical data backtracking, and the first multi-dimensional vector sample data is generated;

[0021]

[0022] ​If part of the data has no clear label, an initial risk label is constructed by inferring the associated entity in the knowledge graph; for the first multi-dimensional vector sample data inferred by the associated entity in the knowledge graph, verification is carried out through artificial and / or semi-supervised learning.

[0023] For unlabeled data inferred by the associated entity in the knowledge graph, the inference accuracy needs to be verified, samples are randomly or strategically extracted from unlabeled data, artificial annotation is carried out by business experts, and the inference result of the model is compared, semi-supervised learning verification trains the model with a small amount of labeled data (such as historical default records), verifies the rationality of predicting unlabeled data, indirect evaluation based on cross-validation divides the labeled data set into training set and test set, simulates the unlabeled inference scene with the training set, verifies the inference logic, and checks whether the inference result conforms to business common sense or domain knowledge based on the logic verification of business rules.

[0024] Preferably, the dynamic feature engineering processing of the first multi-dimensional vector sample data in step S3 specifically includes:

[0025] Step 3.1 introduces self-attention mechanism and time series feature enhancement to input feature matrix The self-attention weight is calculated to automatically discover the correlation between key features, and the attention weight calculation expression is:

[0026] ;

[0027] In the formula, is the query matrix; is the key matrix; is the feature matrix; , is a learnable parameter; is the dimension of the key vector; T represents the transpose of the matrix; Softmax is a function for normalizing numerical vectors into probability distribution vectors;

[0028] The output weighted feature is:

[0029] ;

[0030] In the formula, is the value matrix; is the weighted feature matrix; is a learnable parameter; is the attention weight;

[0031] An attenuation window feature is generated for historical data to capture the time sequence pattern of risk evolution, and the time sequence feature construction expression is:

[0032] ;

[0033] wherein, is the feature at current time; is the decay factor, controlling the weight of historical features; is the feature at historical time; is the length of time window.

[0034] Preferably, the rule engine and neural network based in step S3 are used to build a dual-channel integrated hybrid supervised learning model, specifically:

[0035] Step 3.2: Build a rule channel expression as:

[0036]

[0037] wherein, is the rule channel, are the minimum and maximum endpoints of the decision rule interval defined by experts, respectively; is the indicator function; is the rule weight;

[0038] Build a neural network channel as:

[0039]

[0040] wherein, is the neural network channel, is the neural network weight matrix, is the hidden layer weight, is the output layer weight; is the bias term, is the hidden layer bias term, is the output layer bias term; is the linear rectification unit; is the Sigmoid activation function;

[0041] Adaptive fusion according to adaptive weight parameters:

[0042]

[0043] wherein, is the adaptive fusion function, is the adaptive weight parameter, is the rule channel, is the neural network channel, optimized through the validation set;

[0044] Using decision trees and other rules with strong interpretability and dual-channel integrated hybrid supervised learning model output comparison to ensure business logic consistency;

[0045] ​​​Since the sample data of the present application is derived from multi-agent and multi-field credit risk data, there are characteristics of cross-field non-linear correlation, long-tail distribution, and multi-source noise difference. Traditional regression models can only model linear relationships and are sensitive to noise. The mixed double-channel model of the present application prevents overfitting in sparse areas by hard constraints of the rule channel, automatically adjusts the fusion weight of the rule and neural network based on sample density, focuses on the neural network channel in data-intensive areas and the rule channel in sparse areas, and the rule channel has a noise filtering mechanism to trigger the artificial review process for contradictory data.

[0046] The recalibration and iterative update of the initial risk label of the first multi-dimensional vector sample data specifically includes:

[0047] Step 3.3 Based on the double-channel integrated mixed supervised learning model of 3.2, an improved loss function combining Focal Loss and elastic weight is constructed as the cost function of the model. Focal Loss is an improved loss function for class imbalance problems, that is, the difference between the real label distribution and the label distribution after the mixed model is trained is measured to evaluate the pros and cons of the model. The expression of the improved loss function is:

[0048] ;

[0049] In the formula, is the improved loss function, is the predicted probability of the model for sample index is the real label of sample index is the Focal Loss parameter, which reduces the weight of easy-to-classify samples, is the automatic class weight balancing, total sample size, positive sample number, and negative sample number, respectively; is the regularization coefficient; is the gradient correction term to prevent overfitting. In credit risk assessment, some features (such as "tax arrears amount") may have too much influence on model prediction, causing the model to ignore other weakly related but stable features (such as "historical repayment record"), thereby reducing the generalization ability. Adding a decision boundary regularization term prevents over-reliance on a single feature. The expression of the gradient correction term is:

[0050]

[0051] ;

[0052] In the formula, is the regularization coefficient, is the model output to input features ​​the gradient of the loss function ; the gradient correction term is the square of the L2 norm of the gradient vector, i.e., the sum of the squares of the gradients of all features; is the regularization coefficient, which controls the strength of the penalty term and needs to be optimized by tuning, is the input feature vector ; and

[0053] Step 3.4 Design a second-order optimization and intelligent optimization strategy with a dynamic learning rate, use hessian approximation update to accelerate the convergence speed of the model, the parameter update expression is:

[0054] ;

[0055] In the formula, is the parameter vector of the model at the th iteration; is the global learning rate (step size), which controls the amplitude of parameter update; is the gradient of the loss function with respect to the parameter , which indicates the direction of parameter update; is the Hessian matrix of the loss function ; is the diagonal perturbation term, is a very small constant, is the identity matrix, which is used to ensure that the matrix is invertible; is the inverse matrix of the Hessian matrix, which is used to adjust the direction and step size of gradient update.

[0056] The hierarchical learning rate is used to achieve fine-tuning at the parameter level, and the expression of the hierarchical learning rate is:

[0057] ;

[0058] In the formula, is the hierarchical learning rate, is the base learning rate; is the number of neurons in the kth layer, is the learning rate weight matrix; is the Frobenius norm of the learning rate weight matrix;

[0059] Step 3.5 Construct a dynamic update active iteration mechanism based on uncertainty sampling, which includes sample selection, preferentially selecting samples with the largest amount of annotation information, and the sample selection strategy is:

[0060] ;

[0061] In the formula, argmax selects the sample with the largest information content; is the unlabeled pool; is the labeled set; is the prediction entropy of the model for the sample , is the feature vector of the current sample; is the balance hyperparameter, adjusting the reference weight of historical uncertainty; is the predicted label of the sample, is the sample in the labeled set L, providing a historical uncertainty reference benchmark to avoid repeated selection of similar samples;

[0062] Set a label correction rule combined with model prediction and rule engine double verification to double verify controversial samples:

[0063] ;

[0064] wherein, is the corrected label; is the original label; is the prediction probability distribution of the model for the sample ; is the probability threshold; is the consistency threshold; that is, take the maximum value in the prediction probability distribution; is the nearest neighbor sample label consistency score; when the maximum prediction probability is greater than or equal to or the nearest neighbor sample label consistency score is greater than or equal to , the original label is retained, otherwise the largest prediction probability label is selected.

[0065] Step 3.6 Enhance the interpretability of the rule set, attribute the decision quantitatively, calculate the feature contribution degree by generating the feature importance score through the integral gradient, and the feature contribution degree calculation expression is:

[0066] ;

[0067] wherein, is the feature vector of the current sample, is the i-th feature in the vector, is the baseline input corresponding to the feature ; is the baseline input; is the integral gradient value of the feature ; is the model prediction function; is the interpolation parameter, controlling the path from the baseline to the sample, is the feature ; an absolute deviation from a reference state to a current state; path gradient of the model to the feature , i.e. the partial derivative of the model to .

[0068] After the complex model is converted into the interpretable rules by the rule distillation, the new rule set is output, and the rule distillation expression is:

[0069] ;

[0070] wherein, is to find the optimal rule set, i.e. the Pareto optimal of the minimum prediction error term and the rule set complexity, is the prediction output of the rule set; is the prediction value of the original complex model to the sample ; is the sample number; is the prediction error term, the L2 norm between the complex model and the rule set output; is the regularization coefficient, which controls the weight of the rule complexity in the total loss; is the complexity of the rule set, which measures the complexity of the rule set .

[0071] Preferably, the first multi-dimensional vector sample data processed in step S3 in step S4 includes the processed sample data and historical update frequency data; and the risk prediction value includes risk intensity, volatility index and pattern change degree.

[0072] Preferably, the spatio-temporal feature fusion processing and the risk dynamic evaluation on the time sequence feature in step S4 specifically include:

[0073] Step 4.1 extracts spatio-temporal features from the processed sample data, and fuses the extracted time sequence features and spatial features to obtain time sequence features;

[0074] The expression of the time sequence features is:

[0075] ;

[0076] wherein, is the time sequence feature, is the number of time steps, is the dimension of the input feature vector .

[0077] The expression of the spatial features is:

[0078] ;

[0079] wherein, For a relation graph, a relation graph node set Containing enterprise attributes, an edge set Containing transaction intensity

[0080] The feature fusion formula is:

[0081] ;

[0082] In the formula, The feature fusion is, The time sequence feature is, The relation graph is, and the ReLU is a linear correction unit; the TCN is a time convolution network extracting local patterns, and the GGN is a graph neural network extracting spatial correlation, The linear transformation weight matrix is, The bias vector is;

[0083] Step 4.2 Construction A mixed architecture, wherein The branch is used for capturing local patterns, and the expression of the branch is:

[0084] ;

[0085] In the formula, The expression of the branch is, The current time index is, The expansion factor (exponential growth) is, The convolution kernel size is, The convolution kernel learnable parameter is, The feature value of the input sequence at the time point The convolution kernel position index is; The Transformer branch models global dependence, an improved attention mechanism introduces a DTW prior, and a differentiable soft DTW algorithm is used to align the waveform of the predicted sequence with the real sequence, and the expression of the Transformer branch is:

[0086] ;

[0087] ;

[0088] In the formula, The expression of the Transformer branch is, The query matrix is; The key matrix is; The value matrix is; The dimension of the key vector is; The learnable gating parameter is,​​​​ is the pre-computed DTW similarity matrix; T denotes the transpose of the matrix; Softmax is a function that normalizes a numerical vector into a probability distribution vector;

[0089] Pre-computed by dynamic time warping;

[0090] ;

[0091] wherein, are two groups of time series; is the DTW optimal path point, is the optimal path length, is the element index within the sequence;

[0092] Integrate TCN and Transformer outputs to realize dynamic gating fusion, and calculate dynamic weights:

[0093] ;

[0094] wherein, is the dynamic weight, is the sigmoid function, is the output feature vector of the branch; is the output feature vector of the branch, is the learnable fusion weight;

[0095] The final representation is:

[0096] ;

[0097] wherein, is the final representation, denotes element-wise multiplication; is the dynamic weight, is the output feature vector of the branch; is the output feature vector of the branch;

[0098] Step 4.3 Risk dynamic assessment of time series features, calculate the risk intensity corresponding to the time series features, volatility index, pattern change degree and calculate the update frequency;

[0099] The risk intensity is obtained from the final hidden state of the hybrid architecture, then linearly transformed, and finally probability mapped, and the expression is:

[0100] ;

[0101] wherein, For the risk intensity weight matrix, the hidden state is mapped to a scalar, For the final feature vector output by the hybrid architecture, sigmoid compresses the output to the interval [0, 1], the risk probability represents the credit risk intensity at the current time, close to 1 is a high-risk state, and close to 0 is a low-risk state;

[0102] The volatility index is the standard deviation of the risk intensity within the evaluation window, measuring the fluctuation range of the risk intensity, reflecting the risk stability, and the calculation expression is:

[0103] ;

[0104] In the formula, is the risk stability, is the evaluation window length, is the mean of the risk intensity within the window, is the historical risk intensity, is the current time index, is the time index within the window;

[0105] The mode change degree is to identify the mutation of enterprise risk characteristics and capture abnormal behavior signals, and the actual calculation steps are:

[0106] Distribution estimation: first, the window data of historical risk intensity is subjected to kernel density estimation (KDE):

[0107] ;

[0108] In the formula, is the Gaussian kernel function, is the bandwidth selected according to the Silverman criterion, is the estimated probability density at point, represents the value point of the risk intensity; is the historical risk intensity, is the total number of risk intensity data points within the current evaluation window;

[0109] Numerical integration: the definition domain of is discretized into 1000 points , and the KL divergence is approximately calculated:

[0110] ;

[0111] In the formula, is the mode change degree, is the estimated probability density at the current point, is the historical The estimated probability density at point . Let be the integrand. This represents the value point representing the risk intensity. Discretized form, It is the index identifier of the discretized value range, representing the first value after uniformly dividing the domain of risk intensity. One point, , It is a numerical integral infinitesimal element, representing adjacent discrete points. and The step size between;

[0112] The business significance of pattern change:

[0113] ;

[0114] Ultimately, the degree of mode variability can be characterized as:

[0115] ;

[0116] In the formula, For pattern variability, The current window The risk intensity distribution function, It is a window into history Risk intensity distribution Indexed by the current time. To evaluate window length, KL is the Kullback-Leibler divergence, which measures the difference between two distributions. The differential symbol for continuous integrals;

[0117] The dynamic frequency calculation expression is as follows:

[0118] ;

[0119] In the formula, For dynamic frequency, Based on the update frequency; Minimum update frequency; For maximum update frequency, For risk stability, For pattern variability; , The moving average of historical fluctuations and variability. is the sensitivity coefficient for historical fluctuations and changes, which is dynamically optimized through reinforcement learning, and Clip is the range constraint function.

[0120] Since the risk features need to be quantified in step S5, the present application adds spatio-temporal fusion features in S4 to calculate risk intensity and other quantitative features, and uses dynamic risk assessment to calculate the results more accurately.

[0121] Preferably, the output of the Transformer-TCN includes at least one of risk prediction value, time series feature or gating weight as the environmental state signal or training trigger, which triggers dynamic adjustment of training frequency update when credit risk pattern changes are detected.

[0122] Preferably, the hierarchical DQN framework in step S5 includes a meta-controller and a sub-controller, the meta-controller classifies the features to be processed, and the sub-controller assigns risk labels to the classified features to be processed.

[0123] The hierarchical DQN framework includes:

[0124] Step 5.1: Construct a Hierarchical DQN framework for processing risk feature selection, determine the hierarchical architecture of the framework, which includes a meta-controller and a sub-controller, the meta-controller is used for high-level decision-making, selecting the feature category to be processed, the feature category includes one or more of basic information, financial indicators or credit history, the sub-controller is used for bottom-level decision-making, selecting specific features from the selected category and assigning risk labels;

[0125] Input data structured feature pool:

[0126] ;

[0127] ;

[0128] wherein, is the full set of features, , , is the feature category corresponding to the basic information, financial indicators or credit history, is the number of specific features corresponding to the category, is the selected feature in the full set of features with index , is the index of the full set of features, ranging from 1 to the total number of features , ensuring that each feature has a unique identifier, is the feature category corresponding to the basic information, is the feature index in the corresponding category, , is the feature category corresponding to the financial indicators, is the feature index in the corresponding category, , is the feature category corresponding to the credit history, for the feature index in the corresponding category,

[0129] Numerical standardization:

[0130] for the continuous feature standardization:

[0131]

[0132] where, is the standardized feature, is the feature mean, is the feature standard deviation, is the expectation operator, is the variance operator;

[0133] The state of the meta-controller includes the proportion of selected features, the number of unselected features, and the global risk score. The action is to select a feature category. The reward combines the cumulative reward of the sub-controller and the AUC increment. The meta-controller state representation is:

[0134]

[0135] where, is the meta-controller state; is the selected feature set; is the full feature set; , , is the number of unselected features in the category, , , is the AUC score of the current risk assessment model;

[0136] The cumulative reward of the sub-controller is passed on, combined with the increment of the final model AUC, and the reward function is:

[0137]

[0138] where, is the reward function, is the cumulative reward of the sub-controller, and are the sub-controller weight coefficients, is the immediate reward weight, is the long-term reward weight, satisfying , is the AUC score increment, where and are the AUC scores of the new and old models.

[0139] ​​​​​​The state of the sub-controller is the candidate features and the selected feature statistics of the current class, the action is to select features and assign labels, and the reward is based on feature importance and label error;

[0140] The sub-controller state represents:

[0141] ;

[0142] In the formula, is the sub-controller state, is the mean and standard deviation of the selected features, is the distribution histogram of the candidate features, is the candidate feature in the current class; is the selected feature set;

[0143] The composite action space:

[0144] ;

[0145] In the formula, is the sub-controller composite action space, is the feature with index in the basic information class, is the number of specific features in the corresponding class, is the risk label (0, 1, 2 respectively represent low, medium and high risk).

[0146] Reward function:

[0147] ;

[0148] ;

[0149] In the formula, is the reward function, is the feature index in the corresponding class; is the feature index corresponding to the model predicted label; is the feature importance weight; is the cross-entropy loss function; is the labeled risk label.

[0150] Step 5.2 Network structure and learning:

[0151] Meta-controller network:

[0152] Meta-controller action value function:

[0153] ;

[0154] In the formula, in State vector of the meta-controller; Action of the meta-controller; Network parameters of the meta-controller, Weight matrix of the reinforcement learning from input layer to hidden layer, Bias term of the reinforcement learning hidden layer, Weight matrix of the reinforcement learning from hidden layer to output layer, Bias term of the reinforcement learning output layer; ReLU is linear rectifier unit.

[0155] Sub-controller network:

[0156] The sub-controller network contains a double-head output architecture, which includes a feature head and a label head:

[0157] ;

[0158] In the formula, Probability distribution of feature selection, Probability distribution of label assignment, Feature head weight, Label head weight, Encoding weight matrix, Encoding bias vector, State vector of the sub-controller, ReLU is linear rectifier unit.

[0159] Use dynamic mask technology to mask selected features:

[0160] ;

[0161] Mask the selected features; Selected feature set; when calculating the action probability, the selected features Set to negative infinity, so that the probability after Softmax is zero, and the dynamic mask technology is realized digitally:

[0162] ;

[0163] In the formula, Original vector, Vector after applying dynamic mask, Feature in the basic information category with index ; Selected feature set. Convert the masked

[0164] to a probability distribution using the Softmax function:

[0165] ​ ;

[0166] For the application of dynamic mask vector, when = , =0, so for the selected feature, that is , its probability =0; for the unselected feature, its probability is determined by the original value .

[0167] Preferably, the dynamic output of the interpretable risk feature set in step S5 is based on the hierarchical DQN framework, which combines SHAP value quantification contribution verification, and specifically includes:

[0168] Step 5.3 training process:

[0169] Meta-controller decision:

[0170] Select the category through the ε-greedy strategy;

[0171] ;

[0172] In the formula, is the meta-controller action of policy selection, is the state vector of the meta-controller; is the selected feature category; is the action with the highest value selected, is the action value function, is the policy selection probability, is the utilization probability, is the exploration probability; Sub-controller interaction:

[0173] Use double network update to prevent overestimation, and calculate TD error for each feature selection action:

[0174]

[0175] ;

[0176] Wherein, is the discount factor, is the sub-controller immediate reward, max is the maximum value operator, is the current state of the sub-controller, is the action selected under the current state , is the network parameter that needs to be updated, is the next state, ​​In the state The action to choose from, For target network parameters;

[0177] Global parameter updates are performed by the meta-controller via policy gradient updates;

[0178] ;

[0179] In the formula, This is the state vector of the meta-controller; For the action of the meta controller; For the network parameters of the meta controller, Regarding network parameters gradient operator, For the expected total return of the strategy, In the state Down-Policy Parameterization The strategy gives the action The probability, The total reward obtained by the meta controller on the trajectory. The estimated state value for the value network. The expected value of the action-state trajectory;

[0180] Step 5.4 Evaluation and Optimization Strategies

[0181] Feature Importance Validation: The SHAP value is used to quantify the feature contribution and validate the impact of the selected features on the risk assessment model. Based on the Shapley value in game theory, it is used for feature importance evaluation.

[0182] SHAP value verification:

[0183] ;

[0184] In the formula, For SHAP value, For the complete set of features; A subset of features; For subset The model's predicted values; The index for the feature set is The selected features, For in subset Add features to the basis The model's predicted values, Features marginal contribution For calculating features Add different subsets The probability weights at that time;

[0185] Label consistency check: ROC curve comparing labeled risk and actual default events;

[0186] Course learning strategy: gradually increase the complexity of the feature pool, first train a single category and then expand;

[0187] ;

[0188] In the formula, is the training stage, is the stage index, is the feature category available in the current stage, corresponding to the basic information, financial indicators, credit history feature categories respectively.

[0189] The application adopts a hierarchical DQN framework, the meta-controller controls the risk feature category, and the sub-controller outputs specific risk features. Compared with the Q table checking method, the hierarchical DQN framework has better accuracy performance and is more suitable for high real-time requirement and multi-source data environment scenarios.

[0190] Since the application uses multi-source data, multi-source data will cause more noise impact than single source data. At this time, the hierarchical DQN framework is introduced, the feature categories are selected and distinguished through high-level decision-making, and then specific risk features are selected and output in the distinguished categories. The noise impact after classification will be relatively reduced, and the obtained explainable risk feature set is more accurate.

[0191] Step 5.5 risk feature output

[0192] Dynamic feature description:

[0193] ;

[0194] In the formula, is the dynamic feature of the explainable risk feature set, is the selected feature set, is the output feature name, is the risk label, is the normalized weight.

[0195] The application also provides a credit risk dynamic feature extraction system based on hierarchical reinforcement learning, comprising the following structure:

[0196] The acquisition module is used to acquire credit risk data corresponding to different fields of a plurality of sample subjects, and construct a knowledge graph;

[0197] The construction module is used to extract candidate feature attributes from the knowledge graph, construct an initial risk label, and generate a first multi-dimensional vector sample data;

[0198] The calibration updating module: dynamically performs feature engineering processing on the first multi-dimensional vector sample data, constructs a double-channel integrated hybrid supervised learning model based on a rule engine and a neural network, and performs recalibration and iterative updating on the initial risk label of the first multi-dimensional vector sample data;

[0199] The calculation evaluation module: performs spatio-temporal feature fusion processing on the processed first multi-dimensional vector sample data to obtain time sequence features, performs dynamic risk evaluation on the time sequence features, and calculates the risk prediction value corresponding to the time sequence features;

[0200] The extraction module is used to construct a hierarchical DQN framework, based on the hierarchical DQN framework, combined with SHAP value quantification contribution verification, dynamically outputting an interpretable risk feature set, the interpretable risk feature set including feature attributes, risk labels and normalized weights.

[0201] The above scheme of the present application has the following advantages:

[0202] (1) By integrating market supervision, tax, judicial and other different department heterogeneous data through the knowledge graph, standardizing the fields and solving the entity conflict, the traditional single data source limitation is broken through, and the comprehensiveness and accuracy of feature coverage are improved;

[0203] (2) The self-attention mechanism is introduced to automatically discover the key feature correlation, the time sequence decay window is combined to capture the risk evolution mode, the double-channel hybrid model is used to balance the business logic and data-driven prediction, and the model generalization ability is improved;

[0204] (3) TCN and Transformer are fused, DTW is used to align global time sequence dependence, risk intensity, volatility index and mode change degree are dynamically calculated, risk dynamic evolution is accurately quantified, and hierarchical DQN feature extraction model training and prediction in data incremental update scene are realized;

[0205] (4) Based on the hierarchical DQN framework, the meta-controller selects the feature category, the sub-controller allocates specific labels (low / medium / high risk), the SHAP value quantifies the feature contribution, dynamically removes redundant features, and improves the model efficiency and interpretability.

[0206] Other advantages of the present application will be described in detail in the subsequent specific embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0207] Figure 1 It is a simple schematic diagram of the credit risk dynamic feature extraction method of the present application;

[0208] Figure 2 It is a specific schematic diagram of the credit risk dynamic feature extraction method of the present application;

[0209] Figure 3A specific flow chart of S3 in the credit risk dynamic feature extraction method of the present application. DETAILED DESCRIPTION

[0210] To make the technical problems solved by the present application, technical solutions and advantages clearer, the following will be described in detail with reference to the drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0211] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0212] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connection" should be understood broadly, for example, it can be a locking connection, or a detachable connection, or an integral connection; it can be a mechanical connection, or an electrical connection; it can be a direct connection, or an indirect connection through an intermediate medium, or a communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0213] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0214] As Figure 1 , 2As shown in FIGS. 1 and 3, the present disclosure proposes a method for extracting dynamic features of credit risk based on hierarchical reinforcement learning. The method integrates credit risk data corresponding to multiple subjects such as market supervision, taxation, and justice to construct a knowledge graph, standardizes the fields, and realizes multi-source data fusion based on entity alignment and missing value interpolation. Static attributes and dynamic relationship features are extracted from the knowledge graph, and initial risk labels are generated by combining rule engines and semi-supervised learning. Dynamic feature engineering and risk representation recalibration are performed through self-attention mechanisms and dual-channel hybrid models. The risk intensity, volatility index, and pattern change degree are calculated using spatiotemporal feature fusion, and the feature selection is dynamically optimized through hierarchical reinforcement learning. The SHAP value is used to quantify the feature contribution, and the interpretable risk label is output. This method improves the accuracy of feature extraction and dynamic response capability.

[0215] Step 1: Multi-source data integration and knowledge graph construction

[0216] In specific implementation, credit risk data from multiple departments and fields (including but not limited to market supervision departments, financial supervision departments, intellectual property departments, court departments, and taxation departments) can be preprocessed in the form of a knowledge graph.

[0217] Step 1.1: Data standardization and field mapping

[0218] A multi-source data field mapping table is developed to unify field definitions and clarify the correspondence between data fields from each department and standard fields in the knowledge graph, for example:

[0219]

[0220] Data format standardization, for example:

[0221]

[0222] Step 1.2: Data cleaning and missing value processing

[0223] Redundant data is filtered, mainly including removing duplicate records and deleting invalid fields, such as de-duplication based on the unified social credit code and deletion of log-type unstructured text.

[0224] Missing values are processed by setting three interpolation strategies:

[0225] Knowledge graph-based interpolation uses associated entity information to infer missing values. For example, if an enterprise's tax data is missing, but there is an "execution case for arrears" in the judicial data, it is marked as "tax risk suspicious."

[0226] Statistical interpolation fills numerical fields with the mean or median of similar enterprises.

[0227] Mark missing states by adding a new field (e.g., is_tax_missing) to record missing information for the model to distinguish.

[0228] Step 1.3 Entity Alignment and Unique Identification

[0229] Master Data Management (MDM): Establish a unique identifier for the enterprise (such as the Unified Social Credit Code) as the core primary key, and use a fuzzy matching algorithm (such as Jaccard similarity) to align entities without a unified identifier (such as natural persons).

[0230] Step 2: Knowledge Graph Feature Extraction and Initial Label Generation. Candidate feature attributes are extracted from the knowledge graph to construct initial risk labels, generating the first multidimensional vector sample data, which includes candidate feature attributes and risk labels.

[0231] In specific implementation, the initial multidimensional vector sample data based on the knowledge graph is constructed, consisting of multidimensional vectors in the form of... ,in Represents candidate feature attributes, Indicates a risk label;

[0232] According to a specific implementation of an embodiment of this disclosure, step 2 specifically includes:

[0233] Static attributes are obtained by extracting entity attributes from the constructed knowledge graph nodes. Dynamic features are generated by extracting relation features based on the edges of the knowledge graph. Map the graph structure to a low-dimensional vector. The expression for the objective function is:

[0234] ;

[0235] In the formula, For nodes Embedded vector, To sample neighboring nodes through biased random walks, For a set of nodes, For mapping functions, Let be a probability function.

[0236] Initial risk labels are constructed using a rule engine or by backtracking historical data. If some data lacks clear labels, the first multidimensional vector sample data is generated by inferring from related entities in the knowledge graph.

[0237] For unlabeled data inferred by associated entities in the knowledge graph, it is necessary to verify its inference accuracy. Sample manual review randomly or strategically extracts samples from unlabeled data, and business experts manually annotate and compare with the inference results of the model. Semi-supervised learning verification trains the model with a small amount of labeled data (such as historical default records) to verify the rationality of the prediction of unlabeled data. Indirect evaluation based on cross-validation divides the labeled data set into training set and test set, uses the training set to simulate the unlabeled inference scene, verifies the inference logic, and checks whether the inference result conforms to business common sense or domain knowledge based on business rule logic verification.

[0238]

[0239] This example selects a multi-dimensional cross-verification of unlabeled data inference accuracy combined with technical means and business logic, and verifies it in stages. In the early stage, manual sampling review + semi-supervised learning is given priority to quickly verify the core logic, in the middle stage, cross-validation and comparative experiments are introduced to optimize the inference rules, and in the long term, through active learning and time series verification, dynamic and continuous optimization is realized. Ensure that the inference result is technically reliable and business reasonable, and provide a high-quality data basis for subsequent model training.

[0240] Step 3 Dynamic feature engineering and model iteration optimization, dynamic feature engineering processing is performed on the first multi-dimensional vector sample data, and a supervised learning model is constructed to recalibrate the risk representation of the first sample data;

[0241] Step 3.1 Dynamic feature engineering processing is performed on the first sample data, and self-attention mechanism and time series feature enhancement are introduced to the input feature matrix The self-attention weight is calculated to automatically discover the relevance between key features, and the attention weight calculation expression is:

[0242] ;

[0243] In the formula, is the query matrix; is the key matrix; is the feature matrix; , is a learnable parameter; is the dimension of the key vector; T represents the transpose of the matrix; Softmax is a function that normalizes the numerical vector to a probability distribution vector;

[0244] The output weighted feature is:

[0245] ;

[0246] In the formula, is the value matrix; is the weighted feature matrix; is a learnable parameter; is an attention weight.

[0247] An attenuation window feature is generated for historical data to capture the time-series pattern of risk evolution, and the time-series feature construction expression is:

[0248] ;

[0249] wherein, is a feature at the current time; is an attenuation factor to control the weight of historical features; is a feature at the historical time; is the length of the time window.

[0250] Step 3.2 A dual-channel integrated hybrid model based on rule engine and neural network collaborative modeling is constructed to balance business rules and data-driven prediction, and the rule channel expression is constructed as

[0251] ;

[0252] wherein, is a rule channel, are the minimum endpoint and maximum endpoint of the decision rule interval defined by experts, respectively; is an indicator function; is a rule weight;

[0253] The neural network channel is constructed as:

[0254] ;

[0255] wherein, is a neural network channel, is a neural network weight matrix, is a hidden layer weight, is an output layer weight; is a bias term, is a hidden layer bias term, is an output layer bias term; is a linear rectification unit; is a Sigmoid activation function;

[0256] ;

[0257] wherein, is an adaptive fusion function, is an adaptive weight parameter, is a rule channel, is a neural network channel, optimized through a validation set;

[0258] The decision tree is used to ensure the consistency of business logic.

[0259] Step 3.3 Based on the dual-channel hybrid model in 3.2, an improved loss function that combines Focal Loss and elastic weight is constructed as the cost function of the model. Focal Loss is an improved loss function for class imbalance problems, which measures the difference between the real label distribution and the label distribution after the model is trained to evaluate the model. The improved loss function expression is:

[0260]

[0261] Among them, is the improved loss function, is the predicted probability of the model for sample index , is the real label of sample index , is the Focal Loss parameter, which reduces the weight of easy-to-classify samples, is the automatic class weight balancing, total sample size, positive sample number, and negative sample number, respectively; is the regularization coefficient; is the gradient correction term to prevent overfitting.

[0262] In credit risk assessment, some features (such as "tax arrears amount") may have too much influence on model prediction, causing the model to ignore other weakly related but stable features (such as "historical repayment record"), thereby reducing the generalization ability. Adding a decision boundary regularization term prevents over-reliance on a single feature. The gradient correction term expression is:

[0263] ;

[0264] In the formula, is the regularization coefficient, is the gradient of the model output to the input feature , that is ; the gradient correction term is the square of the L2 norm of the gradient vector, that is, the sum of the squares of all feature gradients; is the regularization coefficient, which controls the strength of the penalty term and needs to be optimized through parameter tuning, is the dimension of the input feature vector .

[0265] Step 3.4 Design an intelligent optimization strategy with second-order optimization and dynamic learning rate, use hessian approximation update to accelerate the convergence speed of the model, the parameter update expression is:

[0266] ;

[0267] where, is the parameter vector of the model at the th iteration; is the global learning rate (step size), which controls the magnitude of parameter update; is the gradient of the loss function with respect to the parameter , which indicates the direction of parameter update; is the hessian matrix of the loss function ; is the diagonal perturbation term, is a small constant, is the identity matrix, which is used to ensure the invertibility of the matrix; is the inverse matrix of the hessian matrix, which is used to adjust the direction and step size of gradient update.

[0268] Hierarchical learning rate is used to achieve fine-tuning at the parameter level, the expression of hierarchical learning rate is:

[0269]

[0270] where, is the hierarchical learning rate, is the base learning rate; is the number of neurons in the kth layer, is the learning rate weight matrix; is the Frobenius norm of the learning rate weight matrix;

[0271] Step 3.5 Construct a dynamic update active iteration mechanism based on uncertainty sampling, which includes sample selection, preferentially labeling samples with the largest amount of information, the sample selection strategy is:

[0272] ;

[0273] where, is the optimal sample selected, argmax selects the sample with the largest amount of information; U is the unlabeled pool; L is the labeled set; is the prediction entropy of the model for the sample , is the feature vector of the current sample; is the balance hyperparameter, which adjusts the reference weight of historical uncertainty; is the predicted label of the sample, For the samples in the labeled set L, provide historical uncertainty reference benchmarks to avoid repeated selection of similar samples;

[0274] Set a label correction rule combined with model prediction and rule engine double verification to double verify controversial samples:

[0275] ;

[0276] In the formula, is the corrected label; is the original label; is the prediction probability distribution of the model for the sample ; is the probability threshold; is the consistency threshold; That is, take the maximum value in the prediction probability distribution; is the nearest neighbor sample label consistency score; when the maximum prediction probability is greater than or equal to or the nearest neighbor sample label consistency score is greater than or equal to , the original label is retained, otherwise the maximum prediction probability label is selected.

[0277] Step 3.6 Enhance the interpretability of the rule set, quantify the decision attribution, calculate the feature contribution by generating the integral gradient feature importance score, and the feature contribution calculation expression is:

[0278] ;

[0279] In the formula, is the feature vector of the current sample; is the th feature in the vector, is the baseline input corresponding to the feature ; is the baseline input; is the integral gradient value of the feature ; is the model prediction function; is the interpolation parameter, which controls the path from the baseline to the sample, is the absolute deviation of the feature from the baseline state to the current state; is the path gradient of the model for the feature , that is, the partial derivative of the model for .

[0280] After rule distillation, the complex model is converted into an interpretable rule set, and the rule distillation expression is:

[0281] ;

[0282] wherein, to find the optimal rule set, i.e., the Pareto optimal of the minimum prediction error term and rule set complexity, is the prediction output of the rule set; is the prediction value of the original complex model on the sample ; is the number of samples; is the prediction error term, the L2 norm between the complex model and the rule set output; is the regularization coefficient, which controls the weight of the rule complexity in the total loss; is the complexity of the rule set, which measures the complexity of the rule set .

[0283] On the basis of the above embodiment, step 4 specifically comprises:

[0284] Step 4 spatiotemporal feature fusion and dynamic risk assessment, fusing TCN time series features and Transformer global dependence, calculating risk intensity (probability mapping), volatility index (standard deviation) and mode change degree (KL divergence);

[0285] Step 4.1 spatiotemporal feature extraction is performed on the sample data and historical update frequency data, and the extracted time series features and spatial features are fused, and the time series feature expression is:

[0286] ;

[0287] wherein, is the time series feature, is the number of time steps, is the dimension of the input feature vector ;

[0288] The spatial feature expression is:

[0289] ;

[0290] wherein, is the relationship graph, and the node set of the relationship graph contains enterprise attributes, and the edge set contains transaction intensity;

[0291] The feature fusion formula is:

[0292] ;

[0293] wherein, is the feature fusion, is the time series feature, ReLU is linear rectifier unit; TCN is time convolution network to extract local patterns; GGN is graph neural network to extract spatial correlation, is linear transformation weight matrix, is bias vector;

[0294] Step 4.2 Construction Hybrid architecture, where Branch is used to capture local patterns, Expression of branch is:

[0295] ;

[0296] In the formula, is Expression of branch, is current time index, is exponential growth of dilated factor, is convolution kernel size, is convolution kernel learnable parameter, is feature value of input sequence at time point , is convolution kernel position index, ;

[0297] The Transformer branch models global dependencies, and the improved attention mechanism introduces DTW prior, using a differentiable soft DTW algorithm to align the predicted and real sequence waveforms. The expression of the Transformer branch is:

[0298] ;

[0299] In the formula, is the expression of the Transformer branch, is the query matrix; is the key matrix; is the value matrix; is the dimension of the key vector; is the learnable gating parameter, is the precomputed DTW similarity matrix; T represents the transpose of the matrix; Softmax is a function that normalizes a numerical vector into a probability distribution vector;

[0300] Precompute through dynamic time warping:

[0301] ;

[0302] In the formula, is the two groups of time series; ​For the DTW optimal path point, For the optimal path length, For the element index within the sequence;

[0303] Integrate the TCN and Transformer outputs to realize dynamic gating fusion and calculate dynamic weights:

[0304] ;

[0305] In the formula, is the dynamic weight, is the sigmoid function, is the output feature vector of the branch; is the output feature vector of the branch, the learnable fusion weight;

[0306] The final representation is:

[0307] ;

[0308] In the formula, is the final representation, denotes element-wise multiplication; is the dynamic weight, is the output feature vector of the branch; is the output feature vector of the branch;

[0309] Step 4.3 Dynamic risk assessment, calculate risk intensity, volatility index, pattern change degree and calculate update frequency, risk intensity is obtained from the final hidden state of the hybrid architecture, then linear transformation, finally probability mapping, expression is:

[0310] ;

[0311] In the formula, is the risk intensity weight matrix, which maps the hidden state to a scalar, is the final feature vector output by the hybrid architecture, sigmoid compresses the output to the [0, 1] interval, and the risk probability represents the credit risk intensity at the current time, close to 1 is a high-risk state, and close to 0 is a low-risk state;

[0312] The volatility index risk intensity is the standard deviation of the risk intensity within the evaluation window, which measures the volatility amplitude of the risk intensity and reflects the risk stability, and the calculation expression is:

[0313] ;

[0314] wherein, is the risk stability, is the evaluation window length, is the mean of risk intensity within the window, is the historical risk intensity, is the current time index, is the time index within the window;

[0315] The pattern change degree identifies the mutation of enterprise risk characteristics and captures abnormal behavior signals. The actual calculation steps are:

[0316] Distribution estimation: First, the historical risk intensity window data is subjected to kernel density estimation (KDE):

[0317] ;

[0318] wherein, is the Gaussian kernel function, is the bandwidth selected according to the Silverman criterion, is the estimated probability density at point, represents the value point of risk intensity; is the historical risk intensity, is the total number of risk intensity data points within the current evaluation window;

[0319] Numerical integration: The definition domain of is discretized into 1000 points , and the KL divergence is approximately calculated:

[0320] ;

[0321] wherein, is the pattern change degree, is the estimated probability density at the current point, is the estimated probability density at the historical point, is the integrand function, represents the value point of risk intensity, which is the discretized form of , is the index identifier of the discretized value domain, indicating the point after uniformly dividing the definition domain of risk intensity, , is the numerical integration infinitesimal, indicating the step length between adjacent discrete points and ;

[0322] Business significance of the pattern change degree: ​

[0323] ;

[0324] Finally, the pattern variability can be represented as:

[0325] ;

[0326] wherein, is the pattern variability, is the risk intensity distribution function of the current window is the risk intensity distribution of the historical window is the current time index, is the evaluation window length, KL is the Kullback-Leibler divergence, which measures the difference between two distributions, is the differential symbol of continuous integration; The dynamic frequency calculation expression is:

[0327]

[0328] ;

[0329] wherein, is the dynamic frequency, is the basic update frequency; is the minimum update frequency; is the maximum update frequency, is the risk stability, is the pattern variability; , is the sliding average of historical volatility and variability, is the sensitivity coefficient of historical volatility and variability, which is dynamically optimized by reinforcement learning, and Clip is the range limiting function.

[0330] The Transformer-TCN spatiotemporal modeling can update the risk intensity and volatility index in real time, enhancing the dynamic adaptability. The output of the Transformer-TCN (risk prediction value, time series feature, gating weight, etc.) can also be used as an environmental state signal or a training trigger. When the credit risk pattern changes are detected, the training frequency is adjusted dynamically.

[0331] Step 5: Hierarchical reinforcement learning feature selection, based on the hierarchical DQN framework, the meta-controller selects the feature category, the sub-controller assigns the label, and the SHAP value is used to quantify the contribution, and the dynamically output explainable risk feature set is obtained;

[0332] On the basis of the above embodiment, step 5 specifically comprises:

[0333] ​​Step 5.1. Construct the Hierarchical DQN framework for risk feature selection, determine the hierarchical architecture of the framework, which includes meta-controller and sub-controller, the meta-controller is used for high-level decision, selecting the feature category to be processed (such as basic information, financial indicators, credit history), the sub-controller is used for bottom-level decision, selecting specific features from the selected category and assigning risk labels;

[0334] Input data structured feature pool:

[0335] ;

[0336] ;

[0337] wherein, is the feature set, , , is the feature category corresponding to basic information, financial indicators, and credit history, is the number of specific features corresponding to the category, is the selected feature in the feature set with index , is the feature set index, ranging from 1 to the total number of features , ensuring that each feature has a unique identifier, is the feature category corresponding to basic information, is the feature index in the corresponding category, , is the feature category corresponding to financial indicators, is the feature index in the corresponding category, , is the feature category corresponding to credit history, is the feature index in the corresponding category, ;

[0338] Numerical standardization:

[0339] Standardize the continuous features :

[0340] ;

[0341] wherein, is the standardized feature, is the feature mean, is the feature standard deviation, is the expectation operator, is the variance operator;

[0342] ​The state of the meta-controller includes the selected feature proportion, the number of unselected features, and the global risk score, the action is to select a feature category, the reward combines the cumulative reward of the sub-controller and the AUC increment, and the meta-controller state representation:

[0343]

[0344] wherein, is the meta-controller state; is the selected feature set; is the full feature set; is the number of unselected features in the category, is the AUC score of the current risk assessment model;

[0345] The cumulative reward of the sub-controller is passed, and the increment of the final model AUC is combined, and the reward function is:

[0346]

[0347] wherein, is the reward function, is the cumulative reward of the sub-controller, and are the sub-controller weight coefficients, is the immediate reward weight, is the long-term reward weight, satisfying , is the AUC score increment, wherein and are the AUC scores of the new and old models.

[0348] The state of the sub-controller is the candidate feature and the selected feature statistics of the current category, the action is to select the feature and assign the label, and the reward is based on the feature importance and the label error

[0349] The sub-controller state representation is:

[0350]

[0351] wherein, is the sub-controller state, is the mean and standard deviation of the selected features, is the distribution histogram of the candidate features, is the candidate feature in the current category; is the selected feature set;

[0352] The composite action space is: ​​​​​​​​

[0353] ;

[0354] wherein, is the sub-controller composite action space, is the feature in the base information category with index is the number of specific features in the corresponding category, is the risk label (0, 1, 2 represent low, medium, high risk respectively).

[0355] Reward function:

[0356] ;

[0357] ;

[0358] wherein, is the reward function, is the feature index in the corresponding category; is the feature index corresponding to the model prediction label; is the feature importance weight; is the cross-entropy loss function; is the labeled risk label.

[0359] Step 5.2 Network structure and learning

[0360] Meta-controller network:

[0361] Meta-controller action value function:

[0362] ;

[0363] wherein, in is the state vector of the meta-controller; is the meta-controller action; is the meta-controller network parameter, is the weight matrix of the reinforcement learning input layer to the hidden layer, is the reinforcement learning hidden layer bias term, is the weight matrix of the reinforcement learning hidden layer to the output layer, is the reinforcement learning output layer bias term; ReLU is the linear rectification unit;

[0364] Sub-controller network:

[0365] The sub-controller network includes a double-head output architecture, which includes a feature head and a label head:

[0366] ; ​

[0367] where, the probability distribution of feature selection, the probability distribution of label assignment, is the feature head weight, is the label head weight, is the encoding weight matrix, is the encoding bias vector, is the state vector of the sub-controller, ReLU is the linear rectification unit;

[0368] Using dynamic mask technology to mask selected features:

[0369] ;

[0370] Masking selected features; is the selected feature set; when calculating the action probability, the is set to negative infinity, so that the probability after Softmax is zero, and the dynamic mask technology is realized digitally:

[0371] When calculating the action probability, the is set to negative infinity, so that the probability after Softmax is zero, and the dynamic mask technology is realized digitally:

[0372] ;

[0373] where, is the original vector, is the vector after applying dynamic mask, is the feature with index in the basic information category, is the selected feature set.

[0374] Probability distribution:

[0375] ;

[0376] is the vector after applying dynamic mask, when = , =0, so for the selected feature, i.e. , its probability =0; for the unselected feature, its probability is determined by the original value .

[0377] Step 5.3 Training process

[0378] Meta-controller decision:

[0379] Select class by ε-greedy policy

[0380] ;

[0381] where, is the meta-controller action for policy selection, is the state vector of the meta-controller; is the selected feature class; is the action with the highest selection value, is the action value function, is the policy selection probability, is the exploitation probability, is the exploration probability; Sub-controller interaction:

[0382] Use double network update to prevent overestimation, calculate TD error for each feature selection action:

[0383]

[0384] ; where,

[0385] is the discount factor, is the sub-controller immediate reward, max is the maximum operator, is the sub-controller current state, is the action selected in the current state ; is the action selected in the next state ; is the network parameter that needs to be updated, is the next state, is the action selected in the state ; is the target network parameter;

[0386] Global parameter update is updated by the meta-controller through policy gradient:

[0387] ;

[0388] where, is the state vector of the meta-controller; is the meta-controller action; is the meta-controller network parameter, is the gradient operator for the network parameter ; is the expected total return of the policy, is the policy parameterization in state ; the policy gives the action the probability, the total return obtained by the meta-controller on the trajectory, the state value estimated by the value network, the expectation of the action-state trajectory;

[0389] Step 5.4 Evaluate and optimize the policy

[0390] Feature importance verification: use SHAP values to quantify feature contributions to verify the impact of selected features on the risk assessment model, based on Shapley values in game theory for feature importance evaluation.

[0391] SHAP value verification:

[0392] ;

[0393] where, is the SHAP value, is the feature universe; is the feature subset; is the model prediction value of the subset ; is the selected feature with index in the feature universe, is the model prediction value based on the subset plus the feature ; is the marginal contribution of the feature , is the probability weight when calculating the feature joining different subsets ;

[0394] Label consistency check: compare the ROC curve of the labeled risk and the actual default event

[0395] Course learning strategy: gradually increase the complexity of the feature pool, first train a single category and then expand

[0396] ;

[0397] where, is the training stage, is the stage index, is the feature category available in the current stage, corresponding to the basic information, financial indicators, and credit history feature categories respectively.

[0398] Step 5.5 Risk feature output

[0399] Dynamic feature description:

[0400] ;

[0401] In the formula, is a dynamic feature of an interpretable risk feature set, is a selected feature set, is an output feature name, is a risk label, is a normalized weight.

[0402] Heterogeneous data conflicts are solved by a knowledge graph, comprehensiveness of features is improved, and deep integration of multi-source data is achieved; risk intensity and fluctuation are accurately quantified by combining time sequence attenuation and spatial correlation, dynamic risk capture is achieved; logical interpretability is guaranteed by a rule engine, and business and data collaboration is achieved by enhancing prediction accuracy of a neural network; a hierarchical reinforcement learning dynamically optimizes a feature set, reduces redundant interference, and realizes adaptive feature selection.

[0403] Embodiments propose a system for dynamic feature extraction of credit risk based on hierarchical reinforcement learning, which comprises:

[0404] An acquisition module is configured to acquire credit risk data of different fields corresponding to a plurality of sample subjects, and construct a knowledge graph;

[0405] A construction module is configured to extract candidate feature attributes from the knowledge graph, construct an initial risk label, and generate first multi-dimensional vector sample data;

[0406] A calibration update module is configured to perform dynamic feature engineering processing on the first multi-dimensional vector sample data, construct a double-channel integrated hybrid supervised learning model based on a rule engine and a neural network, and recalibrate and iteratively update the initial risk label of the first multi-dimensional vector sample data;

[0407] A calculation and evaluation module is configured to perform spatio-temporal feature fusion processing on the processed first multi-dimensional vector sample data, obtain time sequence features, perform risk dynamic evaluation on the time sequence features, and calculate risk prediction values corresponding to the time sequence features;

[0408] An extraction module is configured to construct a hierarchical DQN framework, verify and dynamically output an interpretable risk feature set based on the hierarchical DQN framework and SHAP value quantification contribution, wherein the interpretable risk feature set comprises feature attributes, risk labels, and normalized weights.

[0409] The above describes preferred embodiments of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should also be considered within the scope of protection of the present application.

Claims

1. A method for extracting dynamic features of credit risk based on hierarchical reinforcement learning, characterized in that, Includes the following steps: S1. Obtain credit risk data from different fields corresponding to multiple sample subjects and construct a knowledge graph; S2. Extract candidate feature attributes from the knowledge graph, construct initial risk labels, and generate the first multidimensional vector sample data; S3. Perform dynamic feature engineering on the first multidimensional vector sample data. Based on the rule engine and neural network, construct a dual-channel integrated hybrid supervised learning model to recalibrate and iteratively update the initial risk labels of the first multidimensional vector sample data. S4. Perform spatiotemporal feature fusion processing on the first multidimensional vector sample data processed in S3 to obtain time series features, perform dynamic risk assessment on the time series features, and calculate the risk prediction value corresponding to the time series features. S5. Construct a hierarchical DQN framework. Based on the hierarchical DQN framework, combine SHAP value quantification contribution verification, and dynamically output an interpretable risk feature set. The interpretable risk feature set includes feature attributes, risk labels, and normalized weights.

2. The method for extracting dynamic features of credit risk as described in claim 1, characterized in that, The construction of the knowledge graph in step S1 specifically includes: Step 1.1 Data standardization and field mapping; Step 1.2 Data cleaning and missing value handling; Step 1.3 Align entities with unique identifiers; The initial multidimensional vector sample data mentioned in step S2 consists of multidimensional vectors, in the form of... ,in Represents candidate feature attributes, Indicates a risk label; The step of extracting candidate feature attributes from the knowledge graph and constructing initial risk labels specifically includes: Step S2: Extract entity attributes from the constructed knowledge graph nodes to obtain static attributes, and extract relation features based on the edges of the knowledge graph to generate dynamic features. Initial risk labels are constructed by using a rule engine for preliminary labeling or by backtracking historical data, generating the first multidimensional vector sample data; If some data lacks explicit labels, initial risk labels are inferred from related entities in the knowledge graph; for the first multidimensional vector sample data inferred from related entities in the knowledge graph, verification is performed through manual and / or semi-supervised learning.

3. The method for extracting dynamic features of credit risk as described in claim 1, characterized in that, Step S3, which describes the dynamic feature engineering processing of the initial multidimensional vector sample data, specifically includes: Step 3.1 Introduce a self-attention mechanism and time series feature enhancement to the input feature matrix. Calculate self-attention weights to automatically discover correlations between key features.

4. The method for extracting dynamic features of credit risk as described in claim 1, characterized in that, The construction of the dual-channel integrated hybrid supervised learning model based on the rule engine and neural network in step S3 is specifically as follows: Step 3.2 Construct the rule channel expression as follows: ； In the formula, For rule channels, These are the minimum and maximum endpoints of the decision rule interval defined by the experts, respectively. For indicator functions; For rule weights; The neural network channel is constructed as follows: ； In the formula, For neural network channels, This is the weight matrix of the neural network. For hidden layer weights, These are the output layer weights; For bias terms, For hidden layer bias terms, For output layer bias terms; ReLU is a linear correction unit; Use the Sigmoid activation function; Adaptive fusion based on adaptive weight parameters: ； In the formula, For adaptive fusion function, For adaptive weight parameters, For rule channels, For neural network channels, optimization is performed using a validation set; The output of a hybrid supervised learning model that combines decision tree rules with dual-channel integration is compared to ensure consistency of business logic. The recalibration and iterative update of the initial risk labels for the first multidimensional vector sample data specifically includes: Step 3.3 Based on the dual-channel integrated hybrid supervised learning model in 3.2, an improved loss function that integrates Focal Loss and elastic weights is constructed as the cost function of the model. Focal Loss is an improved loss function scheme for the class imbalance problem, which measures the difference between the true label distribution and the label distribution after training of the hybrid model in order to evaluate the quality of the model. Step 3.4 Design an intelligent optimization strategy for second-order optimization and dynamic learning rate; Step 3.5 Construct a dynamic update proactive iteration mechanism based on uncertainty sampling; Step 3.6 Enhance the interpretability of the rule set by quantifying decision attribution and calculating feature contribution by generating feature importance scores through integral gradients.

5. The method for extracting dynamic features of credit risk as described in claim 1, characterized in that, The first multidimensional vector sample data processed in S3 in step S4 includes the processed sample data and historical update frequency data; the risk prediction value includes risk intensity, volatility index and pattern variability.

6. The method for extracting dynamic features of credit risk as described in claim 5, characterized in that, Step S4, which involves spatiotemporal feature fusion processing and dynamic risk assessment of temporal features, specifically includes: Step 4.1 Extract spatiotemporal features from the processed sample data, and fuse the extracted temporal and spatial features to obtain the temporal features; Step 4.2 Construct the Transformer-TCN hybrid architecture, where Branches are used to capture local patterns; Transformer branches model global dependencies. Step 4.3 Perform dynamic risk assessment on time series characteristics, calculate the risk intensity, volatility index, and pattern variability corresponding to the time series characteristics, and calculate the update frequency.

7. The method for extracting dynamic features of credit risk as described in claim 6, characterized in that, The output of the Transformer-TCN includes at least one of risk prediction values, time-series features, or gating weights, which serve as environmental state signals or training triggers. When a change in credit risk pattern is detected, the training frequency is dynamically adjusted and updated.

8. The method for extracting dynamic features of credit risk as described in claim 1, characterized in that, The hierarchical DQN framework described in step S5 includes a meta-controller and a sub-controller. The meta-controller classifies the features to be processed, and the sub-controller assigns risk labels to the classified features to be processed. The construction of the hierarchical DQN framework specifically includes: Step 5.1 Construct a Hierarchical DQN framework for handling risk feature selection, and determine the hierarchical architecture of the framework, which includes a meta-controller and sub-controllers. The meta-controller is used for high-level decision-making, selecting the feature category to be processed. The feature category includes one or more of basic information, financial indicators or credit history. The sub-controller is used for low-level decision-making, selecting specific features from the selected categories and assigning risk labels. Step 5.2 Determine the network structure and learning of the meta-controller and sub-controllers; Step 5.3 Train the meta-controller's decision-making and sub-controller's interaction.

9. The method for extracting dynamic features of credit risk as described in claim 1, characterized in that, Step S5, which involves quantifying the contribution using SHAP values, specifically includes: The SHAP value is used to quantify the feature contribution and verify the impact of the selected features on the risk assessment model. Based on the Shapley value in game theory, it is used for feature importance assessment. SHAP value verification: ； In the formula, For SHAP value, For the complete set of features; A subset of features; For subset The model's predicted values; The index for the feature set is The selected features, For in subset Add features to the basis The model's predicted values, Features marginal contribution For calculating features Add different subsets The probability weights at that time; The dynamic output can interpret the dynamic feature description of the risk feature set, specifically including: Explainable risk feature set dynamic feature description formula: ； In the formula, To explain the dynamic characteristics of the risk feature set, For the selected feature set, To output the feature name, As a risk label, For normalized weights.

10. A credit risk dynamic feature extraction system based on hierarchical reinforcement learning, characterized in that, Includes the following structure: Acquisition module: used to acquire credit risk data from different fields corresponding to multiple sample subjects and construct a knowledge graph; The building module is used to extract candidate feature attributes from the knowledge graph, construct initial risk labels, and generate the first multidimensional vector sample data. Calibration Update Module: Performs dynamic feature engineering on the initial multidimensional vector sample data. Based on the rule engine and neural network, it constructs a dual-channel integrated hybrid supervised learning model to recalibrate and iteratively update the initial risk labels of the initial multidimensional vector sample data. The calculation and evaluation module performs spatiotemporal feature fusion processing on the processed initial multidimensional vector sample data to obtain time-series features, performs dynamic risk assessment on the time-series features, and calculates the risk prediction value corresponding to the time-series features. The extraction module is used to construct a hierarchical DQN framework. Based on the hierarchical DQN framework, combined with SHAP value quantification contribution verification, it dynamically outputs an interpretable risk feature set, which includes feature attributes, risk labels, and normalized weights.

Images