Financial user portrait construction method based on causal discovery and two-stage attention

Abstract

The application discloses a financial user portrait construction method based on causal discovery and double-stage attention, aims to combine the financial field expert guidance opinion, excavate the association between the financial operation time series data and the user portrait through a causal discovery algorithm and a double-stage attention mechanism, and proposes a financial user portrait construction method. The application can be used for user classification, product recommendation, operation supervision and risk prevention of financial institutions. The method features include: the financial data is completed and cleaned based on a K-Nearest Neighbor (KNN) interpolation method; the financial data features are screened based on a Greedy Fast Causal Interference (GFCI) algorithm; a financial user portrait label system is constructed based on a K-Means algorithm; a deep learning model is constructed based on a double-stage attention mechanism and combined with financial field transaction operation data features, and user portrait prediction is carried out combined with financial operation time series data. Experimental result analysis shows that the application can effectively improve the financial user portrait construction accuracy.

Description

Technical Field

[0001] This invention relates to the fields of finance, causal reasoning, and deep learning, and in particular to a method for constructing financial user profiles based on causal discovery and two-stage attention. Background Technology

[0002] In modern finance, the concept of financial users has gained increasing attention. According to financial experts, financial users exhibit diverse trading characteristics; however, under normal circumstances, the trading style of the same user or institution does not change significantly in the short term. Therefore, analyzing static and dynamic financial data to construct profiles of financial users and financial institutions in the market is very helpful in reflecting changes in the trading market. It also provides financial institutions with a method to identify potential risk users. However, current research on financial user modeling is limited, and the construction of user models based on multi-dimensional characteristics is lacking.

[0003] There are many different methods for building user profiles, such as the relatively simple and intuitive pure statistical analysis method. With the emergence and development of machine learning, various industries have incorporated machine learning methods into their profile building processes. Accurate user profiles can provide institutions with more accurate regulatory data support and evaluation criteria. Furthermore, in the field of risk prevention and control, combining historical market data with reasonable user profiles can provide risk warnings. In the field of simulation, accurate user profiles can simulate real users as trading entities in a simulated environment, thereby providing simulated trading test results for the market, institutions, and individuals, reducing financial risk, and minimizing financial losses.

[0004] Since its introduction by Turing Award winner Judea Pearl, causal reasoning has been used in various fields, with the main purpose of discovering causal relationships between things. One direct approach is called causal discovery. Causal discovery algorithms can be mainly divided into three categories: constraint-based methods, score-based search methods, and score-based functional causal model methods. The GFCI algorithm combines the score-based FGESc algorithm and the constraint-based FCI algorithm. The GFCI algorithm takes a dataset of continuous variables as input and outputs a causal graph consisting of a set of factors that may contain hidden confounding elements.

[0005] The two-stage attention mechanism is based on human attention theory, which models human behavior into two attention stages. Stage 1 selection theory posits that during the preprocessing stage of external influences, the human attention mechanism filters and selects information. Stage 2 selection theory states that this filtering only occurs during the identification and classification stages. Deep learning models based on this theory can significantly improve model accuracy, filter out unimportant information, and increase task efficiency. Summary of the Invention

[0006] This invention addresses the problem of a lack of accurate user profiling methods in the financial sector, a lack of methods for dimensionality reduction of high-dimensional financial data, and a lack of methods for dynamically generating financial user profiles. This invention proposes a method for constructing user profiles in the financial sector that adapts to the dynamic changes in the financial market and solves the problem of accurate user profile prediction. The purpose of this invention is to provide a financial user profile construction method based on causal discovery and a two-stage attention mechanism. This method uses a causal discovery algorithm to calculate causal relationships and select features from financial data, employs the K-Means clustering algorithm to construct profile labels, and combines a two-stage attention mechanism with financial operation time series to predict user profiles, thereby improving the accuracy and interpretability of the financial profiles generated by the algorithm.

[0007] The specific technical solution for achieving the objective of this invention is as follows:

[0008] A method for constructing financial user profiles based on causal discovery and two-stage attention is characterized by its ability to reduce the dimensionality of high-dimensional data in the financial field, using financial transaction data to construct a financial user profile generation system, and dynamically constructing accurate financial user profiles by combining a two-stage attention model with operational time-series data. The method includes the following specific steps:

[0009] A: The original financial transaction data is cleaned and standardized using dirty data testing, outlier handling, deduplication, and KNN interpolation. The financial transaction data includes transaction data and operational data.

[0010] B: The GFCI (Growth Factor Discovery) algorithm is used to calculate causal relationships in the cleaned financial transaction data and generate a causal graph based on the data. Financial features are then screened based on the causal graph, combined with financial knowledge and expert guidance, to achieve dimensionality reduction of high-dimensional data.

[0011] C: The filtered financial transaction data is clustered in multiple dimensions using the K-Means clustering algorithm. Based on the clustering results and labels, a multi-dimensional user profile is constructed.

[0012] D: Use financial user profiles and cleaned financial operation data as training data, and build a deep learning model based on a two-stage attention mechanism. Input the training data, iteratively train the deep learning model, and update the loss function and accuracy at the same time.

[0013] E: Perform steps A and B on the financial transaction data to be predicted, and convert it into operational time series data. Input the data into the trained deep learning model, predict financial user tags through the deep learning model, and obtain a financial user profile based on the combination of these financial user tags.

[0014] Step A specifically includes:

[0015] A1: Clean financial transaction data and financial operation data, check dirty data, and use a combination of automated and manual methods to screen for redundant and outlier values. Use KNN imputation method to complete financial data for missing values.

[0016] A2: Standardize the cleaned data to transform it into dimensionless, order-of-magnitude standardized values.

[0017] Step B uses a causal discovery algorithm to reduce the dimensionality of the financial transaction data:

[0018] B1: Calculate the causal effects between features of standardized financial data using the GFCI algorithm;

[0019] B2: Construct a causal graph based on the calculated causal effects;

[0020] B3: Set a threshold and filter out causal variables whose causal effects are higher than the set threshold based on the causal effects in the causal graph as important financial transaction data features;

[0021] B4: Heuristically optimize and adjust the screened causal graph by combining financial knowledge and expert guidance. The financial knowledge and expert guidance include adding or deleting nodes in the screened causal graph and specifying or modifying the causal relationships between nodes in the causal graph.

[0022] B5: In the optimized causal graph, select the graph nodes with a degree of 0 as the dimensionality reduction result of the high-dimensional financial features.

[0023] Step C uses the K-Means algorithm to perform cluster analysis on the financial transaction data after feature selection:

[0024] C1: Statistical analysis of financial transaction data from multiple dimensions, including financial products, financial models, and transaction time;

[0025] C2: Perform K-Means clustering on the statistically analyzed data for each dimension, and sort the clustering results according to the size of the cluster centers;

[0026] C3: Transform the clustering results into corresponding financial tags, give them financial connotations, and integrate all user tags to build a financial user profile.

[0027] Step D achieves dynamic prediction of financial user profiles by constructing a deep learning model based on a two-stage attention mechanism:

[0028] D1: Convert the format of financial operation data after data cleaning and feature screening, and transform it into operation time series data using the Embedding and Masking methods.

[0029] D2: Match the operation time series data with user labels and combine them as training data, where the operation time series data is the training input data and the user labels are the training output data;

[0030] D3: Construct a deep learning model based on a two-stage attention mechanism, divide the training data into a training set and a test set in an 8:2 ratio, and input the training data into the model;

[0031] D4: Use test accuracy and loss function value as metrics to iteratively train the model until it converges.

[0032] The model building method in step D3 specifically includes:

[0033] D31: Construct the first-stage selection model, and calculate the attention ratio of each user's data by sequentially passing through the first-stage selection attention layer, the Softmax layer, and the weighted summation layer;

[0034] D32: Construct the second-stage selection model, which sequentially passes through an attention layer, an LSTM encoder, a softmax layer, a weighted summation layer, and a loop consisting of an LSTM decoder layer and a linear transformation layer. Each historical time step corresponds to a weighted summation layer.

[0035] D33: Pass the output data from step D32 through a ReLU activation layer, and set the Adam optimizer and cross-entropy loss function.

[0036] Step E enables dynamic prediction of financial user profiles, specifically including:

[0037] E1: Perform dirty data verification on the operation data of the user to be predicted, and handle redundant values, missing values ​​and outliers;

[0038] E2: Based on the feature dimensionality reduction method for financial operation data in step B, perform feature dimensionality reduction on the cleaned operation data to be predicted.

[0039] E3: Transform the data into operational time-series data and input it into the trained two-stage attention model to predict each dimension;

[0040] E4: Combine the predicted financial tags to obtain a financial user profile.

[0041] Compared with existing technologies, the advantages of this invention are as follows: It proposes a dimensionality reduction method for high-dimensional financial data based on a causal discovery algorithm and a highly universal method for automatically constructing user profiles in the financial field. It can be applied to any similar dataset in the financial field. Compared with existing data dimensionality reduction methods, since the causal discovery algorithm calculates the causal relationship between financial features and takes into account the causality between data, it can retain more effective information in the dimensionality reduction results of high-dimensional data, avoid modifying key feature data, and has stronger interpretability. At the same time, this invention proposes a dynamic construction method for financial user profiles based on a two-stage attention mechanism deep learning model. This method makes up for the lack of utilization of operational data in the financial field and shows high model accuracy and good interpretability. Attached Figure Description

[0042] Figure 1 This is a flowchart of the present invention;

[0043] Figure 2 This is a structural diagram of a deep learning model based on a two-stage attention mechanism. Detailed Implementation

[0044] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings and specific examples. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures; however, the scope of protection of the present invention is not limited to the following embodiments.

[0045] This invention proposes a method for constructing financial user profiles based on causal discovery and two-stage attention. (See reference...) Figure 1 The main process of this method can be divided into five parts: financial data cleaning, dimensionality reduction of high-dimensional financial data, financial label generation, two-stage attention model construction, and dynamic prediction of financial user profiles. Among these, the financial data cleaning and dimensionality reduction methods are applicable to most financial data. The process of constructing financial user profiles is as follows: Figure 1 As shown in the dashed box on the left, the dynamic prediction process for financial user profiles is as follows: Figure 1 The specific implementation steps are as shown in the dashed box on the right:

[0046] A: We clean the financial data. The original financial transaction data includes financial transaction data and financial operation data. First, we clean the financial transaction data by deleting redundant and outlier values ​​and filling in missing values ​​using the KNN imputation method. Then, we standardize the data for each feature dimension to ensure that the mean is 0 and the standard deviation is 1.

[0047] B: After cleaning, the data is processed using the GFCI algorithm to calculate the causal effects between financial feature dimensions, generating a causal graph for financial transaction data. A threshold is set for this graph, and edges with causal effects exceeding the threshold are filtered out, while edges with causal effects below the threshold are discarded. The remaining edges are the important financial features. This method can be combined with domain knowledge, allowing for appropriate modifications to the causal graph with the intervention of financial experts.

[0048] C: Perform multi-dimensional statistics on the dimensionality-reduced financial transaction data. First, perform statistics on the data according to different feature dimensions, such as summing up the transaction amount and number of transactions from the dimensions of transaction products and transaction modes. Then, use the K-Means algorithm to cluster the statistical data of each dimension. Assuming that the number of clusters is 3, the transaction amount cluster label can be represented as "conservative, normal, aggressive" from small to large, and the number of transactions cluster label can be represented as "low frequency, normal, high frequency" from small to large. Integrate the financial labels of all dimensions to construct a financial user profile.

[0049] D: After feature filtering, the data is transformed into One-Hot vectors and then subjected to Embedding and Masking operations to convert it into time-series data with unified time steps. The first dimension represents the user, the second dimension represents time, and the third dimension represents the One-Hot vector corresponding to the operation. The operation time-series data is matched with financial user profiles, and a deep learning model based on a two-stage attention mechanism is constructed. The operation time-series data and financial labels are used as training input and output data, respectively, and the training and test sets are divided in an 8:2 ratio. During training, test accuracy and loss function values ​​are used as model training metrics. The model is iteratively trained until the training metrics no longer change or fluctuate below a certain threshold.

[0050] E: To predict financial user profiles using operational data, prepare financial operational data for new users. Repeat step A to clean the operational data of the users to be predicted, handling redundant, missing, and outlier values. Based on the feature filtering results of the financial operational data in step B, perform corresponding feature dimensionality reduction on the cleaned operational data to be predicted. Transform the data into operational time-series data and input it into the trained two-stage attention model to predict each dimension. Merge the predicted financial tags to form the predicted financial user profile.

[0051] See Figure 2 Step D specifically includes:

[0052] D1: Construct the first-stage selection model, pass the operation time series data through the first-stage selection attention layer and the Softmax layer in sequence, and output the attention ratio of each user, that is, assign weights to the importance of each user. After assignment, input it into the weighted summation layer, sum the data of each user with the weights and output it.

[0053] D2: Construct a second-stage selection model. The output of the first-stage selection is passed sequentially through an attention layer, an LSTM encoder, a softmax layer, a weighted summation layer, and a loop consisting of an LSTM decoder layer and a linear transformation layer. The second-stage selection is similar to the attention mechanism of traditional seq2seq. The corresponding attention ratio is calculated for the operation at different time steps and different weights are assigned.

[0054] D3: The output data selected in the second stage is passed through the ReLU activation layer, and the output dimension is set to one dimension, which represents a financial label. The Adam optimizer and cross-entropy loss function are set for the deep learning model, and the model hyperparameters are adjusted according to the training results.

[0055] This invention combines causal discovery algorithms and two-stage attention to construct financial user profiles. It proposes using the GFCI algorithm combined with financial domain knowledge to reduce the dimensionality of financial data, which can efficiently filter out important features. At the same time, it uses K-Means for multi-dimensional clustering to create accurate financial user profiles. Finally, it constructs a deep learning model based on the two-stage attention mechanism, which, combined with financial operation data, can dynamically predict financial user profiles, effectively improving the accuracy and interpretability of the profile model.

Claims

1. A method for constructing financial user profiles based on causal discovery and two-stage attention, characterized in that, The method includes the following specific steps: A: The original financial transaction data is cleaned and standardized using dirty data inspection, outlier handling, deduplication, and KNN interpolation. The financial transaction data includes transaction data and operational data. B: The GFCI (Growth Factor Discovery) algorithm is used to calculate causal relationships in the cleaned financial transaction data and generate a causal graph based on the data. Financial features are then screened based on the causal graph, combined with financial knowledge and expert guidance, to achieve dimensionality reduction of high-dimensional data. C: The filtered financial transaction data is clustered in multiple dimensions using the K-Means clustering algorithm. Based on the clustering results and labels, a multi-dimensional user profile is constructed. D: Use financial user profiles and cleaned financial operation data as training data, and build a deep learning model based on a two-stage attention mechanism. Input the training data, iteratively train the deep learning model, and update the loss function and accuracy at the same time. E: Execute steps A and B on the financial transaction data to be predicted, transform it into operational time-series data, input it into the trained deep learning model, predict financial user tags through the deep learning model, and obtain a financial user profile based on the combination of these financial user tags; where: Step B specifically includes: B1: Calculate the causal effects between features of standardized financial data using the GFCI algorithm; B2: Construct a causal graph based on the calculated causal effects; B3: Set a threshold and filter out causal variables whose causal effects are higher than the set threshold based on the causal effects in the causal graph as important financial transaction data features; B4: Heuristically optimize and adjust the screened causal graph by combining financial knowledge and expert guidance. The financial knowledge and expert guidance include adding or deleting nodes in the screened causal graph and specifying or modifying the causal relationships between nodes in the causal graph. B5: In the optimized causal graph, select the graph nodes with an out-degree of 0 as the dimensionality reduction result of the high-dimensional financial features; Step D specifically includes: D1: Convert the format of financial operation data after data cleaning and feature screening, and transform it into operation time series data using the Embedding and Masking methods. D2: Match the operation time series data with user labels and combine them as training data, where the operation time series data is the training input data and the user labels are the training output data; D3: Construct a deep learning model based on a two-stage attention mechanism, divide the training data into a training set and a test set in an 8:2 ratio, and input the training data into the model; D4: Use test accuracy and loss function value as metrics to iteratively train the model until it converges; Step D3 specifically includes: D31: Construct the first-stage selection model, and calculate the attention ratio of each user's data by sequentially passing through the first-stage selection attention layer, the Softmax layer, and the weighted summation layer; D32: Construct the second-stage selection model, which sequentially passes through an attention layer, an LSTM encoder, a softmax layer, a weighted summation layer, and a loop consisting of an LSTM decoder layer and a linear transformation layer. Each historical time step corresponds to a weighted summation layer. D33: Pass the output data from step D32 through a ReLU activation layer, and set the Adam optimizer and cross-entropy loss function.

2. The financial user profile construction method based on causal discovery and two-stage attention as described in claim 1, characterized in that, Step A specifically includes: A1: Clean financial transaction data and financial operation data, check dirty data, and use a combination of automated and manual methods to screen for redundant and outlier values. Use KNN imputation method to complete financial data for missing values. A2: Standardize the cleaned data to transform it into dimensionless, order-of-magnitude standardized values.

3. The financial user profile construction method based on causal discovery and two-stage attention as described in claim 1, characterized in that, Step C specifically includes: C1: Statistical analysis of financial transaction data from multiple dimensions, including financial products, financial models, and transaction time; C2: Perform K-Means clustering on the statistically analyzed data for each dimension, and sort the clustering results according to the size of the cluster centers; C3: Transform the clustering results into corresponding financial tags, give them financial connotations, and integrate all user tags to build a financial user profile.

4. The financial user profile construction method based on causal discovery and two-stage attention as described in claim 1, characterized in that, Step E specifically includes: E1: Perform dirty data verification on the operation data of the user to be predicted, and handle redundant values, missing values ​​and outliers; E2: Based on the feature dimensionality reduction method for financial operation data in step B, perform feature dimensionality reduction on the cleaned operation data to be predicted. E3: Transform the data into operational time-series data and input it into the trained two-stage attention model to predict each dimension; E4: Combine the predicted financial tags to obtain a financial user profile.

Images