Domain data governance method and system based on artificial intelligence platform

Abstract

The invention provides a domain data management method and system based on an artificial intelligence platform, and the method comprises the steps: obtaining financial transaction data through a data warehouse, carrying out the preprocessing, and extracting transaction metadata and semantic association thereof through a natural language technology; and classifying the metadata by adopting a hierarchical clustering algorithm, and constructing a transaction data consanguinity map. And extracting time sequence and spatial features from the atlas, calculating a node interaction index by using a pre-configured data interaction prediction model, and marking a high-heat node. And extracting high-dimensional transaction association features based on the graph neural network model, and identifying abnormal transaction data. And marking an abnormal node in the blood relationship map, and tracking and positioning an origin node of abnormal data. According to the method, multiple advanced technologies are comprehensively utilized, and rapid and accurate risk identification can be realized in the face of massive financial transaction data.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and specifically relates to a field data governance method and system based on an artificial intelligence platform. Background Art

[0002] With the rapid development of information technology, a large amount of financial data is stored in the data warehouses of the Internet, banks and other institutions. The scale of financial data is growing exponentially, covering all levels from personal transactions to global markets. There are intricate correlations between these data, forming a huge data network. The update frequency of financial data is getting faster and faster, especially in the fields of high-frequency trading and real-time risk control, where data changes may occur at milliseconds. In this context, how to accurately identify transaction risks in financial data has become a key technical problem that needs to be solved urgently.

[0003] Existing technologies face huge challenges in processing such massive, complex and rapidly changing financial data. Traditional data analysis methods are often based on static data or simple time series analysis, which makes it difficult to capture the complex correlations and dynamic change characteristics between data. This leads to many deficiencies in identifying transaction risks, such as high false positive rates, frequent underreporting, and slow response speed. Especially when faced with new financial products and innovative transaction models, existing technologies often have difficulty in identifying potential risks in a timely and effective manner. Summary of the invention

[0004] The present invention provides a domain data governance method and system based on an artificial intelligence platform to solve the problem in the prior art that it is difficult to quickly and accurately identify potential data risks in massive financial data.

[0005] In a first aspect, the present invention provides a domain data governance method based on an artificial intelligence platform, the method comprising the following steps:

[0006] Obtain financial transaction data through data warehouses;

[0007] Performing data preprocessing on the financial transaction data, wherein the data preprocessing includes data cleaning processing and data repair processing;

[0008] Extracting transaction metadata and semantic associations between the transaction metadata from the financial transaction data after data preprocessing by using natural language technology, and completing metadata classification of the transaction metadata by using a hierarchical clustering algorithm;

[0009] Combining the semantic association relationship and the classified transaction metadata to construct a transaction data lineage graph of the financial transaction data, wherein the graph nodes in the transaction data lineage graph represent transaction metadata describing different types of financial transaction entities, and the graph node edges between the graph nodes represent interactive relationships between different transaction metadata;

[0010] Extracting graph temporal features and graph spatial features from the transaction data lineage graph, combining the graph temporal features and the graph spatial features, and predicting the node interaction index of all graph nodes in the transaction data lineage graph through a pre-configured data interaction prediction model, wherein the data interaction prediction model is constructed based on a graph neural network model and a multi-layer perceptron;

[0011] Marking the graph nodes whose node interaction index exceeds a preset index threshold as high-heat graph nodes;

[0012] Taking a plurality of high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction association features from the transaction data lineage graph based on a graph neural network model, and identifying abnormal transaction data in the financial transaction data based on the high-dimensional transaction association features;

[0013] Marking abnormal graph nodes in the bloodline graph of the transaction data based on the abnormal transaction data;

[0014] Taking the abnormal graph node as the tracing starting point, a data lineage tracing method is used to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph.

[0015] Optionally, the data preprocessing of the financial transaction data comprises the following steps:

[0016] For the numerical data in the financial transaction data, a hash algorithm is used to remove duplicate data in the numerical data;

[0017] For the text data in the financial transaction data, performing format standardization processing on the text data to obtain standard text data;

[0018] Using regular expressions to remove invalid data from the standard text data;

[0019] The spelling correction processing and abbreviation expansion processing of the standard text data are completed based on a preset financial data standardization dictionary and a dictionary matching method.

[0020] Optionally, extracting transaction metadata and semantic associations between the transaction metadata from the financial transaction data after data preprocessing by using natural language technology, and completing metadata classification of the transaction metadata by using a hierarchical clustering algorithm comprises the following steps:

[0021] Combining named entity recognition technology with a graph neural network model to build a financial data analysis model, the financial data analysis model is a BiLSTM-CRF-RCNN model;

[0022] Complete the annotation training of the financial data analysis model according to the pre-created financial knowledge training set and using a remote supervision method;

[0023] Inputting the preprocessed financial transaction data into the trained financial data analysis model, and identifying and extracting all transaction metadata describing financial transaction entities in the financial transaction data through the bidirectional long short-term memory network module and the conditional random field module in the financial data analysis model;

[0024] Extracting the semantic association relationship between the transaction metadata through a convolutional network module including a self-attention mechanism in the financial data analysis model;

[0025] A hierarchical clustering algorithm is used to complete metadata classification of the transaction metadata.

[0026] Optionally, the step of using a hierarchical clustering algorithm to complete the metadata classification of the transaction metadata includes the following steps:

[0027] Calculating the Gaussian distribution of the transaction metadata to obtain a high-dimensional similarity matrix of the transaction metadata in a high-dimensional space;

[0028] Initializing low-dimensional data points in a preset low-dimensional space, where the number of the low-dimensional data points is the same as the number of the transaction metadata;

[0029] Calculating the Cauchy distribution of all the low-dimensional data points to obtain a low-dimensional similarity matrix of the low-dimensional data points in the low-dimensional space;

[0030] The KL divergence is calculated by combining the high-dimensional similarity matrix and the low-dimensional similarity matrix;

[0031] Minimize the KL divergence using a gradient descent method and simultaneously optimize and update the position of the low-dimensional data point in the low-dimensional space until the KL divergence reaches a minimum value, completing the process of reducing the dimension of the transaction metadata to the low-dimensional space;

[0032] The minimized intra-class variance of the transaction metadata is calculated in the low-dimensional space, and metadata classification of the transaction metadata in the low-dimensional space is completed based on the minimized intra-class variance.

[0033] Optionally, extracting graph temporal features and graph spatial features from the transaction data lineage graph, combining the graph temporal features and the graph spatial features, and predicting the node interaction index of all the graph nodes in the transaction data lineage graph through a pre-configured data interaction prediction model comprises the following steps:

[0034] Extracting graph temporal features and graph spatial features from the transaction data lineage graph, wherein the graph temporal features include time feature representations of all graph nodes, and the graph spatial features include node spatial topological feature representations of all graph nodes, and the node spatial topological feature representations include node PageRank values, node degree numbers, node betweenness centrality, and central node connectivity;

[0035] Performing feature fusion on the graph temporal features and the graph spatial features to obtain a graph spatiotemporal fusion feature;

[0036] Input the transaction data lineage graph and the graph spatiotemporal fusion features into a pre-configured data interaction prediction model, and extract node interaction features between all graph nodes from the graph spatiotemporal fusion features through multiple gated loop modules in the data interaction prediction model;

[0037] The context information of the transaction data lineage graph is captured by the graph convolutional neural network module in the data interaction prediction model, and the node interaction features of all the graph nodes are spliced ​​into a graph comprehensive representation vector according to the context information;

[0038] Based on the comprehensive representation vector of the graph and through the multi-layer perceptron module in the data interaction prediction model, the node interaction index of all the graph nodes in the transaction data lineage graph is predicted.

[0039] Optionally, the method of using a plurality of high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction association features from the transaction data lineage graph based on a graph neural network model, and identifying abnormal transaction data in the financial transaction data according to the high-dimensional transaction association features comprises the following steps:

[0040] Counting the number of first nodes of the graph nodes and the number of second nodes of the high-heat graph nodes in the transaction data lineage graph;

[0041] Determining a graph transformation degree according to the first number of nodes and the second number of nodes, and randomly generating or removing graph node edges in the transaction data lineage graph according to the graph transformation degree to obtain a transaction data transformation graph;

[0042] Using multiple high-heat graph nodes of different entity types in the transaction data lineage graph as source graph nodes, and using nodes corresponding to the source graph nodes in the transaction data transformation graph as target graph nodes;

[0043] Combining the source graph node and the target graph node and using a dual-graph neural network model to extract high-dimensional transaction association features of the transaction data lineage graph;

[0044] Based on the high-dimensional transaction association features and using an anomaly detection algorithm, abnormal transaction data in the financial transaction data is identified.

[0045] Optionally, the step of combining the source graph node and the target graph node and extracting high-dimensional transaction association features of the transaction data lineage graph using a dual-graph neural network model comprises the following steps:

[0046] The dual-graph neural network model is used to simultaneously extract the source node features of the source graph node and the target node features of the target graph node;

[0047] Constructing a feature association relationship between the source node feature and the target node feature through a cross attention mechanism in the dual-graph neural network;

[0048] The source node features and the target node features are mapped to the same high-dimensional feature space, and the source node features and the target node features are aggregated in the high-dimensional feature space based on the feature association relationship to obtain high-dimensional transaction association features of the transaction data lineage map.

[0049] Optionally, the step of using the abnormal graph node as a tracing starting point and using a data lineage tracing method to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph comprises the following steps:

[0050] For each of the abnormal graph nodes, calculating a node anomaly score of the abnormal graph node according to the high-dimensional transaction association feature;

[0051] Using the abnormal graph node as a tracing starting point, tracing nodes in the transaction data lineage graph according to the interactive relationship;

[0052] In the node tracing process, an abnormal impact score of the traced graph node is calculated based on the node abnormality score and an iterative algorithm is used, and a plurality of most influential predecessor nodes in the traced graph node are determined according to the abnormal impact score;

[0053] Taking all the most influential predecessor nodes as tracing end points, a plurality of abnormal tracing paths between the tracing starting point and the plurality of tracing end points are formed;

[0054] The path anomaly score of each of the abnormal tracing paths is calculated in combination with the anomaly score and the anomaly impact score, the abnormal tracing path with the largest path anomaly score is used as the target abnormal tracing path, and the largest impact predecessor node in the target abnormal tracing path is used as the origin abnormality graph node of the abnormal origin of the abnormal transaction data.

[0055] In a second aspect, the present invention also provides a domain data governance system based on an artificial intelligence platform, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the domain data governance method based on the artificial intelligence platform as described in the first aspect is implemented.

[0056] In a third aspect, the present invention further provides a computer-readable storage medium having instructions stored thereon, which, when executed by a processor, configures the processor to execute the domain data governance method based on an artificial intelligence platform according to the first aspect.

[0057] The beneficial effects of the present invention are:

[0058] The present invention effectively solves the limitations of traditional methods in processing complex financial data by constructing a lineage graph of transaction data and applying an advanced graph neural network model. It can comprehensively capture the semantic associations and spatiotemporal features between data, greatly improving the accuracy and efficiency of abnormal transaction identification. The data interaction prediction model of the present invention can dynamically predict the node interaction index and realize real-time monitoring of high-risk transactions. By marking high-heat graph nodes and extracting high-dimensional transaction association features, the present invention can more accurately identify potential risks and reduce false positives and false negatives. In addition, the data lineage tracking method of the present invention can quickly locate the source of abnormal transactions, providing key support for risk management. This comprehensive and in-depth analysis method not only improves the risk control capabilities of financial institutions, but also provides a more powerful decision-making basis for regulatory authorities. The implementation of the present invention helps to build a safer and more stable financial ecosystem while effectively preventing systemic financial risks. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a flowchart of a domain data governance method based on an artificial intelligence platform in one of the implementation modes of the present application. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0061] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0062] Figure 1 FIG. 1 is a flow chart of a method for domain data governance based on an artificial intelligence platform in one embodiment. It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps in the above method may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps. Figure 1 As shown, the domain data governance method based on the artificial intelligence platform disclosed in the present invention specifically includes the following steps:

[0063] S101. Obtain financial transaction data through a data warehouse.

[0064] Among them, first of all, a centralized data warehouse is established, which adopts a distributed storage architecture to cope with the storage needs of massive data. The core of the data warehouse is to use the ETL (extraction, transformation, loading) process to extract data from various source systems. These source systems include but are not limited to the core transaction system of the bank, the securities trading platform, the third-party payment platform such as the insurance company system, the credit card transaction system, etc. In the data extraction stage, the incremental extraction method is adopted to obtain only the data added or changed after the last extraction to improve efficiency and reduce the burden of network transmission. In order to ensure the real-time nature of the data, the CDC (change data capture) technology can be used to capture the data changes of the source system in real time. In the data conversion stage, ETL tools such as Informatica, Talend or open source Apache Nifi are used to clean and convert the data. This stage unifies the format of data from different sources. Parallel loading technology is used in the data loading stage, such as using Oracle's direct path loading or Hadoop's batch import, which significantly improves the speed of data entry. The data warehouse is designed with a snowflake model, centered on the transaction fact table, and associated with customer dimensions, product dimensions, time dimensions, geographic location dimensions, etc. In order to optimize query performance, materialized views and appropriate indexes are established for frequently accessed data.

[0065] S102. Perform data preprocessing on financial transaction data.

[0066] Among them, a large amount of financial transaction data obtained is pre-processed. Data pre-processing includes data cleaning and data repair. Data cleaning and repair are key steps to ensure the accuracy of subsequent analysis. This process includes multiple sub-steps, each of which targets specific data quality issues. First, data deduplication is performed, and a hash-based method is used to quickly identify completely duplicated records. For nearly duplicated records, more complex similarity algorithms such as Jaccard similarity or edit distance are used. Then, missing values ​​are processed. For randomly missing data, multiple imputation can be used, which generates multiple possible complete data sets and then merges the results. For non-randomly missing data, a missing model needs to be established, such as using the Markov Chain Monte Carlo (MCMC) method. For data with inconsistent formats, standardization is performed. For example, regular expressions are used to unify date formats and convert different currency units into unified units. The standardization of address data can use the geocoding API to convert unstructured addresses into standard latitude and longitude coordinates.

[0067] S103. Extract transaction metadata and semantic associations between transaction metadata from the financial transaction data after data preprocessing through natural language technology, and use a hierarchical clustering algorithm to complete metadata classification of the transaction metadata.

[0068] Among them, extracting and classifying financial transaction metadata from preprocessed financial transaction data is a complex natural language processing task. This process involves multiple subtasks, including named entity recognition, relationship extraction, and metadata classification. First, named entity recognition (NER) technology is used to identify key entities in transaction data, such as product names, institution names, transaction amounts, etc. A deep learning-based BiLSTM-CRF model is adopted, in which the BiLSTM layer captures contextual information and the CRF layer ensures the consistency of annotations. In order to improve the generalization ability of the model, a pre-trained language model such as BERT is introduced as a feature extractor, and the entire model is fine-tuned for NER tasks in the financial field.

[0069] Next, use relation extraction technology to identify the relationship between entities, such as "transaction", "regulation", etc. Use the remote supervision method to automatically annotate the training data using the existing financial knowledge graph. Then use the neural network model with attention mechanism to classify the relationship. The model structure can be based on graph neural network, such as R-GCN (Relational Graph Convolutional Network). For the extracted metadata, a hierarchical clustering algorithm is used for classification. Ward's method is used, and its goal is to minimize the intra-class variance. In order to process high-dimensional data, the metadata is first mapped to a low-dimensional space using dimensionality reduction techniques such as t-SNE or UMAP, and then clustered in the low-dimensional space. In order to improve the accuracy and interpretability of classification, domain expert knowledge is introduced. A financial terminology dictionary and ontology library are established to guide entity recognition and relationship extraction. At the same time, a topic model such as LDA (latent Dirichlet allocation) is used to discover potential topics in metadata. The generation process of LDA can be expressed as:

[0070] 1. For each document d, sample the topic distribution θd~Dir(α) from the Dirichlet distribution

[0071] 2. For each topic k, sample word distribution φk~Dir(β) from Dirichlet distribution

[0072] 3. For each word wdi in document d:

[0073] a. Sample topics zdi~Multinomial(θd) from a multinomial distribution

[0074] b. Sampling words wdi~Multinomial(φzdi) from multinomial distribution

[0075] Finally, the clustering results, expert knowledge and topic model results are combined to obtain the final metadata classification. This multi-angle classification method can make full use of the semantic information and structural information of the data to improve the accuracy and reliability of classification. Through this complex natural language processing process, unstructured or semi-structured financial transaction data can be converted into structured metadata and classified according to semantic similarity, providing rich and accurate basic data for subsequent graph network construction.

[0076] S104. Construct a transaction data lineage graph of financial transaction data by combining semantic association relationships and classified transaction metadata.

[0077] Among them, a transaction data lineage graph is constructed based on the extracted and classified financial transaction metadata. This graph not only shows the relationship between financial entities, but also reflects the flow and dependency of data. The graph nodes in the transaction data lineage graph represent the transaction metadata describing different types of financial transaction entities, and the graph node edges between the graph nodes represent the interactive relationship between different transaction metadata. Specifically, different types of financial transaction entities (such as product name, institution name, indicator name, etc.) are first used as nodes of the graph. Each node is represented by a multidimensional feature vector, which contains the attribute information of the entity, such as entity type, creation time, update frequency, etc. Then, the edges between the nodes are established according to the semantic relationship between entities (such as transaction, supervision, investment, etc.). The edge is directed, indicating the direction of data flow or influence. The edge also carries a weight, indicating the strength or importance of the relationship. In order to capture complex multi-entity relationships, the concept of hyperedge is introduced, allowing one edge to connect multiple nodes. The hypergraph can be represented by a tensor H, where H[e][v] represents the association strength between the hyperedge e and the node v.

[0078] The graph construction process can be expressed as an iterative update process:

[0079] 1. Initialize the node set V and edge set E.

[0080] 2. For each new metadata record m: a. If m introduces a new entity, add it to V; b. Update the attributes of related nodes; c. Update E based on the relationship information of m.

[0081] 3. Regularly compress and optimize the graph, merge similar nodes, and delete low-weight edges.

[0082] The transaction data lineage graph constructed in this way can not only intuitively display the complex relationships between financial transaction entities, but also reflect the flow, dependency and evolution of data. This graph structure provides a strong topological foundation for subsequent anomaly detection, risk assessment and decision support.

[0083] S105. Extract graph temporal features and graph spatial features from the transaction data lineage graph, combine the graph temporal features and graph spatial features, and predict the node interaction index of all graph nodes in the transaction data lineage graph through a pre-configured data interaction prediction model.

[0084] Among them, the data interaction prediction model is built based on the graph neural network model and the multi-layer perceptron. Extracting graph temporal features and graph spatial features from the transaction data lineage graph and combining these features to use a multi-layer vector machine to predict high-heat graph nodes is a complex multi-stage process. First, extract graph temporal features. This involves analyzing snapshots of the graph at different time points. Use a dynamic graph neural network (DGNN) to capture the evolution of the graph structure over time. Next, extract graph spatial features. This includes node-level, edge-level, and graph-level features. For node-level features, various centrality indicators such as degree centrality, betweenness centrality, and eigenvector centrality are calculated. Finally, a multi-layer support vector machine (MLSVM) is used to predict high-heat graph nodes. MLSVM is a variant of SVM that processes nonlinearly separable complex data through a multi-layer structure. The training process of MLSVM is carried out layer by layer, and the output of each layer is used as the input of the next layer. The output of the last layer is the heat prediction of the node. In order to deal with the problem of class imbalance (there may be fewer high-heat nodes), oversampling techniques such as SMOTE (Synthetic Minority Over-sampling Technique) are used to increase the number of minority class samples.

[0085] S106. Mark the graph nodes whose node interaction index exceeds the preset index threshold as high-heat graph nodes.

[0086] S107. Using multiple high-heat graph nodes of different entity types as source graph nodes, extract high-dimensional transaction association features from the transaction data lineage graph based on the graph neural network model, and identify abnormal transaction data in financial transaction data based on the high-dimensional transaction association features.

[0087] First, the high heat map nodes identified in the previous step are classified according to entity type, such as financial products, trading institutions, trading accounts, etc. These high heat nodes will serve as the starting point for anomaly detection because they are often the center or source of abnormal behavior. For each entity type, the top-k nodes with the highest heat are selected as the source node set S.

[0088] Next, we use a graph neural network (GNN) to extract high-order topological features from the transaction metadata graph network. Specifically, we can use a graph attention network (GAT) as the basic model because GAT can adaptively assign different importance to different neighbor nodes. Based on these high-order topological features, an anomaly detection algorithm is used to preliminarily identify abnormal transactions. Here, we use a graph-based anomaly detection method called Graph Deviation Network (GDN). The core idea of ​​GDN is to compare the difference between the actual features of a node and the features predicted by its neighborhood. Through the above process, we can make full use of the graph structure information and effectively identify abnormal patterns in financial transaction networks. This method not only considers the characteristics of a single transaction, but also its position and relationship in the entire network, so it can capture more complex and hidden abnormal behaviors.

[0089] S108. Mark abnormal graph nodes in the transaction data lineage graph based on the abnormal transaction data.

[0090] S109. Using the abnormal graph node as the tracing starting point, a data lineage tracing method is used to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph.

[0091] First, for each abnormal node i, its abnormal degree is defined. Next, the data lineage tracing method is used to locate the origin of abnormal transactions. This process can be regarded as back propagation on the graph. Starting from each abnormal node, the incoming edges are gradually traced upward to construct a directed acyclic graph (DAG) to represent the data flow. In the tracing process, the weight of the edge and the abnormal degree of the node are used to calculate the impact score. In order to deal with possible circular dependencies, an iterative algorithm is used to calculate the impact score. In order to accelerate convergence and process large-scale graphs, graph sampling methods such as GraphSAGE can be used to approximate the impact score. While calculating the impact score, the most influential predecessor node of each node is recorded to construct an impact propagation tree. By tracing back these most influential predecessor nodes, the path from the abnormal node to the potential origin node can be found. In order to more comprehensively analyze the abnormal propagation mode, path-based abnormal propagation analysis is introduced. Considering the temporal nature of financial transactions, time window constraints are added in the tracing process. Define the time decay function. In order to distinguish different types of abnormal propagation modes, multi-dimensional abnormal vectors are introduced. Finally, based on the impact score and path analysis results, the most likely abnormal origin node set O is identified. Through this complex data lineage tracing process, not only can the potential origin of abnormal transactions be located, but also the path and pattern of abnormal propagation can be revealed. This method provides in-depth insights into the formation mechanism of financial risks, which helps financial institutions and regulators take more targeted risk prevention and control measures. At the same time, by analyzing the abnormal propagation path, key risk nodes and weak links can also be identified, providing an important basis for optimizing the transaction network structure and formulating risk management strategies.

[0092] In one embodiment, preprocessing the financial transaction data includes the following steps:

[0093] For numerical data in financial transaction data, a hash algorithm is used to remove duplicate data in the numerical data;

[0094] For text data in financial transaction data, format standardization is performed on the text data to obtain standard text data;

[0095] Use regular expressions to remove invalid data from standard text data;

[0096] Based on the preset financial data standardization dictionary and the dictionary matching method, the spelling correction and abbreviation expansion processing of standard text data are completed.

[0097] In this embodiment, each numerical data is first converted into a unique hash value, and then these hash values ​​are used to quickly identify and delete duplicates. In specific implementation, a secure hash algorithm such as SHA-256 can be used. For each value x, its hash value h(x)=SHA-256(x) is calculated. Then an empty set S is created, and all values ​​are traversed. If h(x) is not in S, x is retained and h(x) is added to S; otherwise, x is considered to be a duplicate value and deleted. In order to deal with the precision problem of floating point numbers, the values ​​can be rounded or truncated before hashing. For example, for amount data, the values ​​can be converted into integers divided into units before hashing. In addition, in order to improve efficiency, a Bloom filter can be used to quickly determine whether an element may be in the set, thereby reducing the number of hash calculations. The misjudgment rate of the Bloom filter can be estimated by the following formula: p≈(1-e^(-kn / m))^k. Where k is the number of hash functions, n is the number of elements, and m is the size of the bit array. By adjusting these parameters, a balance can be achieved between space efficiency and accuracy. This method can not only effectively remove duplicate data, but also greatly improve the efficiency of subsequent data processing, laying a good foundation for financial transaction data analysis.

[0098] Next, the text data in the financial transaction data needs to be formatted and standardized. This process involves multiple subtasks, including case unification, whitespace processing, special character standardization, etc. First, convert all text to lowercase to eliminate inconsistencies caused by case differences. The specific code can be: standardized_text = text.lower(). Then, use regular expressions to process whitespace characters, and replace multiple consecutive spaces, tabs, newlines, etc. with a single space. The specific code can be: standardized_text = re.sub(r'\s+',", standardized_text). For special characters, such as currency symbols, percent signs, etc., unified conversion is required. For example, different currency symbols are uniformly replaced with standard codes. This can be achieved by creating a mapping dictionary and traversing the replacement. The specific code can be:

[0099] forsymbol,codeincurrency_map.items():

[0100] standardized_text=standardized_text.replace(symbol,code)

[0101] In addition, you need to process the date and time formats. You can use regular expressions to identify different date formats and then convert them to the ISO8601 standard format (YYYY-MM-DD). The specific code can be:

[0102] date_pattern=r'\b(\d{1,2})[ / .-](\d{1,2})[ / .-](\d{2,4})\b'

[0103] standardized_text=re.sub(date_pattern,lambdam:f"{m.group(3).zfill(4)}-{m.gro up(2).zfill(2)}-{m.group(1).zfill(2)}",standardized_text)

[0104] For numeric formats, decimal points are used as decimal separators and commas are used as thousands separators. These standardization processes can significantly improve the consistency of text data and lay the foundation for subsequent analysis and processing. Standardized text data is easier to compare and aggregate, which can improve the effectiveness of data mining and machine learning algorithms, and is also easier for manual review and understanding.

[0105] Next, you need to use regular expressions to remove invalid data from standard text data. This process mainly targets data items that do not conform to the expected format or content to improve the overall data quality. In financial transaction data, invalid data may include malformed dates, illegal amount values, account numbers that do not conform to the specifications, etc. For each type of invalid data, a corresponding regular expression needs to be designed for matching and processing. For example, for date data, you can use the following regular expression to check the format, and the specific code can be: date_pattern = r'^(\d{4})-(\d{2})-(\d{2})$'. For amount data, you can use the following code: amount_pattern = r'^\d+(\.\d{1,2})? $'. For the account number, assuming it should be 16 digits, you can use the following code: account_pattern = r'^\d{16}$'. The process of data validation using these regular expressions can be expressed as:

[0106] validated_data=[itemforitemindataifre.match(pattern,item)]

[0107] Where pattern is the corresponding regular expression, and data is the list of data that needs to be verified.

[0108] In addition to validating the data format, you can also use regular expressions to remove special characters or unwanted content in the text. For example, to remove all non-alphanumeric characters, the specific code can be:

[0109] cleaned_text=re.sub(r'[^a-zA-Z0-9\s]',",text)

[0110] In actual applications, it may be necessary to design more complex regular expressions based on specific data characteristics and business requirements. At the same time, in order to improve efficiency, commonly used regular expressions can be pre-compiled. By using regular expressions to remove invalid data, the quality and consistency of data can be significantly improved. This not only reduces errors and deviations in subsequent analysis, but also improves the efficiency of data processing. At the same time, the use of regular expressions also provides flexibility and scalability for the data cleaning process, which can be adjusted and optimized according to the changing data characteristics and business needs.

[0111] Next, we use the dictionary matching method to complete the spelling correction and abbreviation expansion processing of the standard text data based on the preset financial data standardization dictionary. This process involves two main tasks: spelling correction and abbreviation expansion. First, we build a financial dictionary that contains correct spellings and common abbreviations and their full names. This dictionary can be represented as a mapping:

[0112] D={misspelled_word:correct_word,abbreviation:full_form}

[0113] For example: {"finacial":"financial","ROI":"Return on Investment"}

[0114] For spelling correction, an edit distance algorithm (such as Levenshtein distance) can be used to find the closest correct spelling. Given a word w, its correction process can be expressed as:

[0115] corrected_word=argmin_{vinD}Levenshtein(w,v)

[0116] Levenshtein(w,v) calculates the edit distance between w and v. To improve efficiency, a Burkhard-Keller tree (BK tree) can be used to quickly find similar words.

[0117] For abbreviation expansion, dictionary matching is used directly:

[0118] expanded_word=D.get(word,word)

[0119] Here, if word is an abbreviation in the dictionary, its full name is returned; otherwise, it remains as it is.

[0120] In actual processing, context information can be combined to improve accuracy. For example, the n-gram model is used to calculate the probability of candidate words in the current context, and the candidate word with the highest probability is selected as the correction result:

[0121] P(w|context)=count(context,w) / count(context)

[0122] corrected_word=argmax_{vincandidates}P(v|context)

[0123] In order to handle domain-specific terminology and new words, a dynamic update mechanism can be set up to regularly update the dictionary based on new data. At the same time, any uncertain situations can be marked for manual review.

[0124] In one embodiment, extracting transaction metadata and semantic associations between transaction metadata from financial transaction data after data preprocessing by natural language technology, and completing metadata classification of transaction metadata by using a hierarchical clustering algorithm includes the following steps:

[0125] Combining named entity recognition technology with graph neural network model to build a financial data analysis model, the financial data analysis model is a BiLSTM-CRF-RCNN model;

[0126] Complete the annotation training of financial data analysis models based on the pre-created financial knowledge training set and using remote supervision methods;

[0127] The preprocessed financial transaction data is input into the trained financial data analysis model, and all transaction metadata describing the financial transaction entities in the financial transaction data are identified and extracted through the bidirectional long short-term memory network module and conditional random field module in the financial data analysis model;

[0128] The semantic correlation between transaction metadata is extracted through the convolutional network module containing the self-attention mechanism in the financial data analysis model;

[0129] A hierarchical clustering algorithm is used to complete the metadata classification of transaction metadata.

[0130] In this embodiment, a financial data analysis model is constructed by combining named entity recognition technology and graph neural network model, using BiLSTM-CRF-RCNN architecture. This model consists of three main parts: bidirectional long short-term memory network (BiLSTM), conditional random field (CRF) and self-attention convolutional neural network (RCNN). The BiLSTM layer is responsible for capturing the long-term dependencies of the input sequence. It contains two LSTM networks, forward and backward, which can consider past and future context information at the same time. For the input sequence x = (x1,...,xT), the output of BiLSTM can be expressed as:

[0131] h_t=[forward_LSTM(x_t),backward_LSTM(x_t)]

[0132] The CRF layer is used to sequence the output of BiLSTM, which takes into account the dependencies between labels and improves the accuracy of prediction. The objective function of the CRF layer can be expressed as:

[0133] logP(y|x)=Σ_t(Φ(y_t,x,t)+Ψ(y_t,y_{t-1}))-logZ(x)

[0134] where Φ is the emission function, Ψ is the transfer function, and Z(x) is the normalization factor.

[0135] The RCNN part introduces the self-attention mechanism, which can capture long-distance dependencies in the sequence. The calculation formula of the self-attention mechanism is:

[0136] Attention(Q,K,V)=softmax(QK^T / √d_k)V

[0137] Where Q, K, V are query, key and value matrices respectively, and d_k is the dimension of the key.

[0138] The convolution layer is used to extract local features, and its operation can be expressed as:

[0139] C_i=f(W*x_{i:i+h-1}+b)

[0140] Where W is the convolution kernel, h is the convolution kernel size, and f is the activation function.

[0141] This complex model architecture can effectively handle entity recognition and relationship extraction tasks in financial text data, capturing complex semantic information and long-distance dependencies. It is particularly suitable for processing professional terms and complex expressions in the financial field, and can accurately identify and classify various financial entities, such as company names, financial products, transaction types, etc. At the same time, the graph neural network part of the model can further utilize the relationship information between entities to improve the overall recognition and classification performance. This method not only improves the accuracy of financial data analysis, but also can process large-scale and complex financial text data, providing strong support for subsequent risk analysis, market forecasting and other tasks.

[0142] The annotation training of the financial data analysis model is completed based on the pre-created financial knowledge training set and the remote supervision method. Remote supervision uses the existing knowledge base to automatically generate labels for a large amount of unlabeled text, overcoming the time-consuming and labor-intensive problem of manual annotation. First, it is necessary to build a knowledge base containing financial entities and relationships, such as:

[0143] KB={(entity1,relation,entity2),...}

[0144] For example, ("XX Company", "Issue", "Corporate Bonds").

[0145] Then, for each unlabeled text, if a pair of entities in the knowledge base appear in the text at the same time, it is assumed that the text expresses the relationship between the two entities in the knowledge base. The formal representation is:

[0146] If (e1, r, e2) ∈ KB and e1, e2 ∈ sentence, then label(sentence) = r

[0147] However, this method may introduce noise. In order to reduce mislabeling, a multi-instance learning strategy can be adopted. All sentences containing the same entity pair (e1, e2) are classified into a bag. As long as at least one sentence in the bag correctly expresses the relation r, the bag is considered to be a positive example of the relation r. This can be expressed as:

[0148] P(r|bag)=max(P(r|sentence)forsentenceinbag)

[0149] During training, negative log-likelihood is used as the loss function:

[0150] L=-logP(r|bag)

[0151] In order to further improve the annotation quality, an attention mechanism can be introduced to assign different weights to the sentences in the bag:

[0152] P(r|bag)=Σ_iα_iP(r|sentence_i)

[0153] Where α_i is the attention weight of sentence i, which can be learned through a neural network.

[0154] In addition, the entity type and context information can be used to filter out unreasonable annotations. For example, if the subject of the relationship "acquisition" should be "company", then the incorrect annotation of the subject as "individual" can be excluded. The advantage of this method is that it can quickly generate a large amount of labeled training data, significantly reducing the workload of manual annotation. Although the generated labels may contain some noise, the annotation quality can be greatly improved through the above strategy. This method is particularly suitable for the financial field because there is a lot of structured knowledge in the financial field that can be used as the basis for remote supervision. The model trained in this way can better understand the entities and relationships in financial texts, laying the foundation for subsequent financial data analysis tasks.

[0155] The preprocessed financial transaction data is input into the trained financial data analysis model. The BiLSTM module processes the input sequence and captures long-distance dependencies. For the input sequence x = (x1,...,xT), the forward and backward hidden states of BiLSTM are: h_t^f = LSTM_f (x_t, h_{t-1}^f); h_t^b = LSTM_b (x_t, h_{t+1}^b).

[0156] The final BiLSTM output is the concatenation of the forward and backward states: h_t = [h_t^f; h_t^b].

[0157] Next, the CRF layer uses the output of BiLSTM to perform sequence labeling. CRF takes into account the transition probability between labels and can capture the overall structure of the label sequence. The conditional probability of CRF can be expressed as:

[0158] P(y|x)=exp(Σ_t(s(x,t,y_t)+t(y_{t-1},y_t))) / Z(x)

[0159] where s(x,t,y_t) is the emission fraction, t(y_{t-1},y_t) is the transfer fraction, and Z(x) is the normalization factor.

[0160] In the decoding stage, the Viterbi algorithm is used to find the most likely tag sequence: y*=argmax_yP(y|x).

[0161] In specific implementation, the BIOES annotation scheme (B-begin, I-internal, O-external, E-end, S-single token entity) can be used to annotate financial entities. For example:

[0162] "XX / B-ORG company / E-ORG issued / O 10 billion / B-AMOUNT / E-AMOUNT corporate bonds / S-PRODUCT"

[0163] In this way, structured transaction metadata can be extracted from financial transaction data to provide a basis for subsequent analysis and decision-making. These metadata may include key information such as transaction subject, transaction object, transaction amount, transaction time, etc., which are crucial for understanding the nature and impact of financial transactions.

[0164] Next, the semantic association between transaction metadata is extracted through the convolutional network module (RCNN) containing the self-attention mechanism in the financial data analysis model. The convolutional network module (RCNN) combines the local feature extraction capability of the convolutional neural network (CNN) and the global dependency capture capability of the self-attention mechanism, and can effectively extract the complex semantic relationship between transaction metadata. First, the self-attention mechanism calculates the correlation between metadata. Given an input sequence X = (x1,...,xn), the calculation process of self-attention is as follows: Q = XW_Q, K = XW_K, V = XW_V; Attention(Q,K,V) = softmax(QK^T / √d_k)V.

[0165] Where W_Q, W_K, W_V are learnable weight matrices, and d_k is the dimension of the key vector. This step can capture long-range dependencies between metadata. Next, the convolution layer is used to extract local features. For the attention output A, the convolution operation can be expressed as: C_i=f(W*A_{i:i+h-1}+b), where W is the convolution kernel, h is the convolution kernel size, and f is the activation function (such as ReLU). In order to capture features of different scales, multiple convolution kernels of different sizes are usually used, and then the most significant features are selected through the maximum pooling operation: F_j=max(C_{1,j},C_{2,j},...,C_{n-h+1,j}). Finally, all features are concatenated to obtain the final representation: F=[F_1;F_2;...;F_m], where m is the number of convolution kernels used.

[0166] The advantages of the convolutional network module (RCNN) are: the self-attention mechanism can capture the global dependencies between metadata and is suitable for processing long-distance semantic associations. The convolution operation can extract local contextual features, which helps to understand the semantics at the phrase level. Multi-scale convolution kernels and pooling operations enable the model to capture semantic information of different granularities. In this way, the model can extract complex semantic relationships such as "Company A acquires Company B" and "The price increase of Product X affects the stock price of Company Y". These relationships can be represented by a graph structure, where the nodes are transaction metadata and the edges are the semantic relationships between them. For example, the following semantic relationship graph can be obtained:

[0167] (XX Company, Issue, Corporate Bonds)

[0168] (corporate bonds, amount, 10 billion)

[0169] (Bond issuance, impact, company stock price)

[0170] Finally, the hierarchical clustering algorithm is used to complete the metadata classification of transaction metadata. The hierarchical clustering algorithm builds a hierarchical structure of data by gradually merging or splitting clusters, which is suitable for processing complex hierarchical relationships in financial transaction data. The basic steps of hierarchical clustering are as follows: treat each metadata as an independent cluster; calculate the distance between all clusters; merge the two clusters with the closest distance; repeat the above steps until the predetermined number of clusters is reached or the stopping condition is met.

[0171] Distance calculation is a key step, and a variety of distance measurement methods can be used. For transaction metadata, you can consider using cosine similarity: cos(x,y) = (x·y) / (||x||||y||), where x and y are vector representations of metadata.

[0172] There are also many options for calculating the distance between clusters, and the commonly used ones include single linkage, complete linkage, average linkage, etc. For financial transaction data, average linkage usually provides better results because it takes into account the average situation of all elements in the cluster. The result of hierarchical clustering can be represented by a dendrogram, where each node of the tree represents a cluster and the leaf node is a single metadata. The final classification result can be determined by setting a distance threshold or specifying the number of clusters. In order to improve the clustering efficiency of large-scale data, the BIRCH algorithm can be used. BIRCH compresses data by constructing a CF tree (Clustering Feature Tree), where each node contains clustering features (N, LS, SS): N represents the number of elements in the cluster, LS represents the linear sum, and SS represents the square sum Σx^2. The main steps of the BIRCH algorithm include: traversing the data and inserting each element into the nearest leaf node; merging similar leaf nodes to reduce the size of the tree; clustering the leaf nodes of the CF tree; and fine-tuning the clustering results. The advantage of this method is that its time complexity is O(n), which is suitable for processing large-scale transaction metadata. Through hierarchical clustering, a multi-level classification structure of transaction metadata can be obtained.

[0173] In one implementation, using a hierarchical clustering algorithm to complete metadata classification of transaction metadata includes the following steps:

[0174] Calculate the Gaussian distribution of transaction metadata and obtain a high-dimensional similarity matrix of transaction metadata in a high-dimensional space;

[0175] Initialize low-dimensional data points in a preset low-dimensional space, where the number of low-dimensional data points is the same as the number of transaction metadata;

[0176] Calculate the Cauchy distribution of all low-dimensional data points to obtain the low-dimensional similarity matrix of the low-dimensional data points in the low-dimensional space;

[0177] The KL divergence is calculated by combining the high-dimensional similarity matrix and the low-dimensional similarity matrix;

[0178] Use the gradient descent method to minimize the KL divergence and simultaneously optimize and update the position of the low-dimensional data points in the low-dimensional space until the KL divergence reaches the minimum value to complete the process of reducing the dimension of the transaction metadata to the low-dimensional space;

[0179] The minimized intra-class variance of the transaction metadata is calculated in the low-dimensional space, and the metadata classification of the transaction metadata in the low-dimensional space is completed based on the minimized intra-class variance.

[0180] In this embodiment, calculating the Gaussian distribution of transaction metadata and obtaining a high-dimensional similarity matrix of transaction metadata in high-dimensional space is the first step in the data dimensionality reduction process. This step aims to capture the similarity relationship in the original high-dimensional data. For each pair of transaction metadata xi and xj, the high-dimensional similarity pij between them is calculated. The probability density function of the Gaussian distribution is used to define this similarity: pij = (exp(-||xi-xj||^2 / (2σi^2))) / (Σk≠iexp(-||xi-xk||^2 / (2σi^2))). Here, σi is adaptively selected based on the perplexity parameter. The perplexity is usually set between 5 and 50, which controls the variance of the Gaussian distribution and thus affects the calculation of similarity. Higher perplexity results in considering more distant neighbors, while lower perplexity pays more attention to close neighbors. In this way, an N×N high-dimensional similarity matrix P can be obtained, where N is the number of transaction metadata. This matrix P captures the relative distance relationship between data points in the original high-dimensional space and provides a basis for the subsequent dimensionality reduction process.

[0181] Initializing the low-dimensional data points in the preset low-dimensional space, the number of low-dimensional data points is the same as the number of transaction metadata, is the preparatory stage of the dimensionality reduction process. The purpose of this step is to create an initial low-dimensional representation for each high-dimensional transaction metadata. Usually, the dimension of the low-dimensional space is selected to be 2 or 3 for easy visualization. The initialization method has an important impact on the final result. Common methods include random initialization and principal component analysis (PCA) initialization. Random initialization simply samples from the standard normal distribution: yi~N(0,10^-4*I), where yi is the initial representation of the i-th transaction metadata in the low-dimensional space. PCA initialization uses the principal components of the data to initialize the low-dimensional points, which can speed up the convergence: yi=U[:d]xi, where U[:d] is the first d principal eigenvectors of the covariance matrix of the original data. After initialization, an N×d matrix Y is obtained, where N is the number of data points and d is the dimension of the target low-dimensional space.

[0182] Calculating the Cauchy distribution of all low-dimensional data points and obtaining the low-dimensional similarity matrix of low-dimensional data points in low-dimensional space is one of the core steps of the t-SNE algorithm. The Cauchy distribution (or t-distribution, with 1 degree of freedom) is used to calculate the similarity between data points in low-dimensional space. For any two points yi and yj in low-dimensional space, the similarity qij between them is defined as: qij=(1+||yi-yj||^2)^-1 / Σk≠l(1+||yk-yl||^2)^-1. The reason why the Cauchy distribution is used here instead of the Gaussian distribution is that the Cauchy distribution has a longer tail, which helps to better separate data clusters in low-dimensional space and reduce the "crowding problem". By calculating qij between all low-dimensional point pairs, an N×N low-dimensional similarity matrix Q can be obtained. This matrix Q reflects the relative position relationship between data points after dimensionality reduction and is the key to the subsequent optimization process. The use of the Cauchy distribution makes t-SNE particularly good at preserving the local structure in the original data while creating a more obvious cluster separation in the low-dimensional space.

[0183] Calculating the KL divergence by combining the high-dimensional similarity matrix and the low-dimensional similarity matrix is ​​a key step in evaluating the quality of dimensionality reduction. KL divergence (Kullback-Leibler divergence) is used to measure the difference between two probability distributions. Here it is used to measure the similarity of the distribution of data points in high-dimensional space and low-dimensional space. The calculation formula of KL divergence is: C = KL(P||Q) = Σi≠jpijlog(pij / qij). Here, pij and qij are the similarities between points i and j in high-dimensional and low-dimensional space, respectively. The smaller the KL divergence, the better the low-dimensional representation retains the structure of the original data. It is worth noting that KL divergence is asymmetric, and using P||Q instead of Q||P can better preserve local structure because it pays more attention to similar points in high-dimensional space. The gradient of this objective function relative to the low-dimensional point yi is calculated as: This gradient information guides the subsequent optimization process and helps adjust the positions of low-dimensional points to minimize the KL divergence.

[0184] Use the gradient descent method to minimize the KL divergence and simultaneously optimize and update the position of the low-dimensional data points in the low-dimensional space until the KL divergence reaches the minimum value to complete the process of reducing the dimension of the transaction metadata to the low-dimensional space. This is the core optimization step of the t-SNE algorithm. In specific implementation, the stochastic gradient descent method with momentum is usually used. In each iteration, the position of the low-dimensional point is updated according to the following formula: Here, η is the learning rate and α(t) is the momentum coefficient that varies over time. The learning rate usually starts at a large value (such as 100) and then gradually decreases during the iteration process. The momentum term helps to accelerate convergence and avoid local minima. The optimization process usually performs hundreds to thousands of iterations until the change in KL divergence is less than a certain threshold or the maximum number of iterations is reached. For improved efficiency, the Barnes-Hut algorithm can be used to approximate the gradient, reducing the time complexity from O(N^2) to O(NlogN). The result of this optimization process is the final position of each transaction metadata in the low-dimensional space, which retains the key structural features of the original high-dimensional data.

[0185] Calculating the minimized intra-class variance of transaction metadata in low-dimensional space and completing metadata classification of transaction metadata in low-dimensional space based on minimizing the intra-class variance is the clustering analysis step after dimensionality reduction. This step aims to group the reduced-dimensional data points to discover potential data structures and patterns. The method of minimizing intra-class variance usually uses the K-means algorithm. First, select an appropriate number of clusters K, which can be determined by the elbow rule or silhouette analysis. Then, the K-means algorithm iteratively performs the following steps:

[0186] (1) Randomly initialize K cluster centers; (2) Assign each data point to the nearest cluster center; (3) Update the center of each cluster to the average position of all points in the cluster. The objective function is to minimize the total intra-class variance: J = Σk = 1^KΣx∈Ck||x-μk||^2, where Ck is the kth cluster and μk is the center of the kth cluster. The algorithm iterates until the cluster assignment no longer changes or the maximum number of iterations is reached. In order to improve the stability of the results, K-means is usually run multiple times and the best result is selected. The advantage of this method is that it can further reveal the intrinsic structure of the data while retaining the t-SNE dimensionality reduction results, which helps to identify similar trading patterns or behaviors. The final classification results can be used for financial analysis tasks such as anomaly detection and risk assessment.

[0187] In one embodiment, extracting graph temporal features and graph spatial features from the transaction data lineage graph, combining the graph temporal features and graph spatial features, and predicting the node interaction index of all graph nodes in the transaction data lineage graph through a pre-configured data interaction prediction model includes the following steps:

[0188] Extract graph temporal features and graph spatial features from the lineage graph of transaction data. Graph temporal features include the time feature representation of all graph nodes. Graph spatial features include the node spatial topological feature representation of all graph nodes. Node spatial topological feature representation includes node PageRank value, node degree number, node betweenness centrality and central node connectivity.

[0189] The temporal features and spatial features of the graph are fused to obtain the temporal and spatial fusion features of the graph;

[0190] The transaction data lineage graph and graph spatiotemporal fusion features are input into a pre-configured data interaction prediction model, and the node interaction features between all graph nodes are extracted from the graph spatiotemporal fusion features through multiple gated loop modules in the data interaction prediction model;

[0191] The graph convolutional neural network module in the data interaction prediction model captures the contextual information of the transaction data lineage graph, and splices the node interaction features of all graph nodes into a comprehensive graph representation vector based on the contextual information;

[0192] Based on the comprehensive representation vector of the graph and through the multi-layer perceptron module in the data interaction prediction model, the node interaction index of all graph nodes in the bloodline graph of the transaction data is predicted.

[0193] In this embodiment, extracting the temporal and spatial features of the graph from the bloodline graph of transaction data is a key step in understanding the dynamic evolution and structure of the graph. The temporal features capture the pattern of node changes over time, which can be achieved through time embedding methods, such as converting timestamps into periodic features: sin(2πt / T) and cos(2πt / T), where t is the timestamp and T is the period (such as one day or one year). Spatial features reflect the importance and structural role of the node in the graph. The PageRank value is calculated as PR(u)=(1-d)+d*Σv∈N(u)PR(v) / OutDegree(v), where d is the damping factor, usually 0.85. The node degree is the number of edges directly connected to the node. Betweenness centrality indicates the degree to which a node acts as a "bridge", calculated as BC(v)=Σs≠v≠t(σst(v) / σst), where σst is the number of shortest paths from s to t, and σst(v) is the number of shortest paths passing through v. The central node connectivity measures how closely a node is connected to the center of the graph, and can be obtained by calculating the average shortest path length to all other nodes. Together, these features form a comprehensive representation of the graph, providing a rich information basis for subsequent analysis.

[0194] Fusing the temporal and spatial features of the graph to obtain the spatiotemporal fusion features of the graph is a key step in integrating multi-dimensional information. This process aims to create a comprehensive representation that captures both the temporal dynamics and spatial structure of the graph. Feature fusion can be achieved in a variety of ways, such as simple concatenation, weighted sum, or more complex neural network methods. An effective method is to use an attention mechanism for feature fusion: F = Σiαi*Fi, where Fi is the i-th feature vector (which can be temporal or spatial features), αi is the corresponding attention weight, calculated by softmax(Wi*Fi+bi). Wi and bi are learnable parameters. This method allows the model to dynamically adjust the importance of different features. Another method is to use a gating mechanism: F = z⊙Ft+(1-z)⊙Fs, where Ft is the temporal feature, Fs is the spatial feature, and z is the gating vector, calculated by sigmoid(W[Ft;Fs]+b). ⊙ represents element-wise multiplication. This method can achieve effective fusion of temporal and spatial features while retaining the original feature information. The fused feature F provides a comprehensive representation for each node, including its time evolution characteristics and structural characteristics in the graph.

[0195] Inputting the transaction data lineage graph and graph spatiotemporal fusion features into the pre-configured data interaction prediction model and extracting node interaction features through multiple gated recurrent modules is a key step in capturing complex temporal dependencies. This process uses variants of recurrent neural networks (RNNs), such as long short-term memory networks (LSTMs) or gated recurrent units (GRUs), to process temporal data. Taking GRU as an example, its update process is as follows:

[0196] resetgate:rt=σ(Wr[ht-1,xt]+br)

[0197] updategate:zt=σ(Wz[ht-1,xt]+bz)

[0198] candidatestate:ht=tanh(Wh[rt⊙ht-1,xt]+bh)

[0199] hiddenstate:ht=(1-zt)⊙ht-1+zt⊙ht

[0200] Among them, xt is the input at time t (graph spatiotemporal fusion features), ht is the hidden state, σ is the sigmoid function, and ⊙ represents elementwise multiplication. Through this mechanism, the model is able to learn long-term dependencies and capture complex interaction patterns between nodes. In order to process graph-structured data, a GRU unit can be configured for each node, and the state of the current node is updated according to the state of the neighboring nodes at each time step. This approach allows information to propagate in the graph, thereby capturing direct and indirect interactions between nodes. The stacking of multiple GRU layers can further enhance the model's ability to capture complex interaction patterns. The output of this step is an interaction feature vector for each node, which integrates the node's historical information and neighbor influences.

[0201] Capturing the contextual information of the transaction data lineage graph through the graph convolutional neural network (GCN) module in the data interaction prediction model and splicing the node interaction features of all graph nodes into a comprehensive graph representation vector is a key step in integrating local and global information. The core operation of GCN can be expressed as: in is an adjacency matrix with self-loops added and normalized, H(l) is the node feature matrix of the lth layer, W(l) is a learnable weight matrix, and σ is a nonlinear activation function (such as ReLU). This operation implements local aggregation of node features, enabling each node to comprehensively consider the information of its neighbors. Stacking multiple layers of GCN allows information to propagate over a larger range, thereby capturing a wider range of contexts. In order to enhance the expressiveness of the model, a variant of the attention mechanism can be used, such as the attention network (GAT):

[0202] eij=a(Whi,Whj); αij=softmaxi(eij); hi'=σ(ΣjαijWhj)

[0203] Where a is the attention function, usually a single-layer feedforward neural network, and αij is the attention coefficient. This mechanism allows the model to dynamically assign different importance to different neighbors. After capturing the contextual information, the features of all nodes are concatenated to form a comprehensive graph representation vector: G = [h1'; h2'; ...; hN'], where hi' is the feature vector of node i after GCN processing, and N is the total number of nodes. This comprehensive representation vector contains the global structural information of the graph and the local features of each node, providing rich input for subsequent prediction tasks. In this way, the model can simultaneously consider the individual characteristics of the node and its position and role in the entire transaction network, thereby more accurately predicting the interaction behavior between nodes.

[0204] Based on the graph comprehensive representation vector, predicting the node interaction index of all graph nodes in the transaction data lineage graph through the multi-layer perceptron (MLP) module in the data interaction prediction model is a key step to achieve the final prediction. MLP consists of multiple fully connected layers, and the calculation of each layer can be expressed as: Y = σ (WX + b), where X is the input vector, W is the weight matrix, b is the bias vector, and σ is a nonlinear activation function (such as ReLU or tanh). For the interaction index prediction task, the last layer usually uses the sigmoid function to compress the output to between 0 and 1: in is the predicted interaction index, and hL is the output of the last hidden layer. The number of MLP layers and the number of neurons in each layer need to be adjusted according to the complexity of the task and the size of the data. In order to improve the generalization ability of the model, a dropout layer can be added between each layer: hdrop = dropout(h,p), where p is the dropout probability.

[0205] The back-propagation algorithm is used during training to minimize the loss function, such as the binary cross entropy loss: Where y is the true label. In order to deal with the problem of class imbalance, a weighted loss function or oversampling technique can be used. In addition, regularization techniques (such as L2 regularization) can be used to prevent overfitting: Lreg = L + λΣ||Wi||2, where λ is the regularization coefficient. Through this MLP module, the model is able to learn complex nonlinear mappings and transform the comprehensive representation of the graph into an interaction index prediction between nodes. This prediction result can be used to identify potential high-risk transactions, predict future transaction patterns, or evaluate the importance of nodes in the transaction network. By adjusting the threshold, the continuous interaction index can be converted into a binary interaction prediction.

[0206] In one embodiment, multiple high-heat graph nodes of different entity types are used as source graph nodes, high-dimensional transaction association features are extracted from the transaction data lineage graph based on a graph neural network model, and abnormal transaction data in financial transaction data are identified based on the high-dimensional transaction association features, including the following steps:

[0207] Count the number of first nodes of the graph nodes in the bloodline graph of transaction data and the number of second nodes of the high-heat graph nodes;

[0208] Determine the degree of graph transformation according to the first node number and the second node number, and randomly generate or remove graph node edges in the transaction data lineage graph according to the degree of graph transformation to obtain a transaction data transformation graph;

[0209] Multiple high-heat graph nodes of different entity types in the transaction data lineage graph are used as source graph nodes, and the corresponding nodes of the source graph nodes in the transaction data transformation graph are used as target graph nodes;

[0210] Combining source graph nodes and target graph nodes and using a dual-graph neural network model to extract high-dimensional transaction association features from the bloodline graph of transaction data;

[0211] Based on high-dimensional transaction correlation features and using anomaly detection algorithms, abnormal transaction data in financial transaction data is identified.

[0212] In this embodiment, counting the number of first nodes of the graph nodes in the bloodline graph of transaction data and the number of second nodes of the high-heat graph nodes is a key step in understanding the graph structure. The first node refers to the node directly connected to the central node, and its number reflects the complexity of the direct connection of the graph. High-heat graph nodes refer to nodes whose transaction frequency or transaction volume is significantly higher than the average level, and are usually identified using the Z-score method: Z = (x-μ) / σ, where x is the transaction volume of the node, μ is the average transaction volume, and σ is the standard deviation. Nodes with a Z-score greater than 2 are selected as high-heat nodes. The second node refers to the node directly connected to the high-heat node, and its number reflects the influence range of the high-activity area. In specific implementation, the depth-first search (DFS) or breadth-first search (BFS) algorithm can be used to traverse the graph and count the number of nodes. For example, using BFS: starting from the central node, add it to the queue Q, and then repeat the following steps until Q is empty: (1) Take a node v from Q; (2) Count the number of unvisited neighbors of v; (3) Add the unvisited neighbors of v to Q. The time complexity of this process is O(V+E), where V is the number of nodes and E is the number of edges.

[0213] Determining the degree of graph transformation based on the number of first nodes and the number of second nodes, and randomly generating or removing graph node edges in the transaction data lineage graph to obtain a transaction data transformed graph is an important step in simulating network dynamic changes. The degree of graph transformation can be calculated by the following formula: T = α * (N1 / N) + β * (N2 / N), where N1 is the number of first nodes, N2 is the number of second nodes of high - heat nodes, N is the total number of nodes, and α and β are weight coefficients. The larger the T value, the more complex the graph structure and the more transformations are required. Based on the T value, the addition or deletion ratio of edges P = f(T) can be determined, where f can be a non - linear function, such as sigmoid: P = 1 / (1 + e^(-kT)), and k is a tuning parameter. During actual transformation, for each edge, a random number r between [0, 1] is generated. If r < P, then the transformation operation is performed: for adding an edge, two non - directly - connected nodes are randomly selected and an edge is added; for deleting an edge, an existing edge is randomly selected and deleted. To maintain the connectivity of the graph, the Union - Find algorithm can be used to ensure that the deletion operation does not cause the graph to split. This transformation method simulates the dynamic characteristics of the real - world transaction network, including the establishment of new relationships and the disappearance of old relationships. The transformed graph retains the basic structural features of the original graph while introducing randomness.

[0214] Taking multiple high - heat graph nodes of different entity types in the transaction data lineage graph as source graph nodes, and taking the corresponding nodes of the source graph nodes in the transaction data transformed graph as target graph nodes is a key step in constructing the input of a dual - graph neural network. First, the selection criteria for high - heat nodes need to be determined. The PageRank algorithm or Degree Centrality can be used to measure the importance of nodes. The calculation formula of PageRank is: PR(u) = (1 - d)+d * Σv∈N(u)PR(v) / |N(v)|, where d is the damping factor (usually 0.85), and N(u) is the set of neighbors of node u. Select the top k% of nodes with the PageRank value as high - heat nodes. To ensure a balanced representation of different entity types, top - m high - heat nodes can be selected in each entity type. The corresponding relationship between source graph nodes and target graph nodes is established through node ID or feature matching. If a certain source node does not exist in the transformed graph, the node most similar to it can be selected as a substitute, and the similarity can be calculated by the cosine similarity of node features: sim(u, v)=(fu·fv) / (||fu||*||fv||), where fu and fv are the feature vectors of nodes u and v.

[0215] Combining source graph nodes and target graph nodes and using the dual-graph neural network model to extract high-dimensional transaction association features of the transaction data lineage graph is a complex feature learning process. The dual-graph neural network model processes the original graph and the transformed graph at the same time to capture the structural differences and semantic changes between the graphs. The core of the model is the graph convolution operation, which can be expressed as: H^(l+1)= in is a normalized adjacency matrix, H^(l) is the node feature matrix of the lth layer, W^(l) is a learnable weight matrix, and σ is a nonlinear activation function. In order to capture the relationship between graphs, an attention mechanism can be used: αij=softmax(a(Whi,Whj)), where a is the attention function, hi and hj are the features of the corresponding nodes in the two graphs. Then, by aggregating the information from the two graphs: h'i=σ(W[hi||Σjαijhj]), where || represents the concatenation operation. Multiple layers of such operations can capture more complex structural relationships. Finally, a graph-level representation is obtained through a global pooling operation: hG=READOUT({hi|i∈G}), where READOUT can be a sum, average, or a more complex learnable function. The output of this process is a high-dimensional vector that encodes the structural features and semantic information of the original graph and the transformed graph. In this way, the model is able to learn the key differences before and after the graph transformation, which may indicate potential abnormal patterns.

[0216] Identifying abnormal transaction data in financial transaction data based on high-dimensional transaction association features and using anomaly detection algorithms is a key step in achieving transaction risk monitoring. Common anomaly detection algorithms include Isolation Forest, Local Outlier Factor (LOF) and One-Class SVM. Taking Isolation Forest as an example, its basic principle is that abnormal points are easier to isolate. The algorithm randomly selects a feature and splits the data at the random value of the feature, repeating this process until each point is isolated. Anomalies usually require fewer segmentation steps. The anomaly score calculation formula is: s(x,n)=2^(-E(h(x)) / c(n)), where h(x) is the average path length of point x, and c(n) is the average path length when the data set size is n. For the LOF algorithm, the ratio of the local density of each point to the local density of its k nearest neighbors is calculated, and the LOF value of the abnormal point is significantly greater than 1. The one-class SVM tries to find a hyperplane that encloses most data points in a compact area, and points that fall outside the area are considered abnormal.

[0217] In one embodiment, combining the source graph node and the target graph node and using the dual-graph neural network model to extract high-dimensional transaction association features of the transaction data lineage graph includes the following steps:

[0218] The dual-graph neural network model is used to simultaneously extract the source node features of the source graph nodes and the target node features of the target graph nodes;

[0219] The feature association relationship between source node features and target node features is constructed through the cross-attention mechanism in the dual-graph neural network;

[0220] The source node features and the target node features are mapped to the same high-dimensional feature space, and the source node features and the target node features are aggregated in the high-dimensional feature space based on the feature association relationship to obtain the high-dimensional transaction association features of the transaction data lineage map.

[0221] In this embodiment, using the dual-graph neural network model to simultaneously extract the source node features of the source graph node and the target node features of the target graph node is a process of processing two graph structures in parallel. The dual-graph neural network model contains two parallel graph neural network subnetworks, which process the source graph and the target graph respectively. Each subnetwork uses multi-layer graph convolution operations to extract node features. Taking the graph convolution network (GCN) as an example, its core operation can be expressed as: in is a normalized adjacency matrix, H^(l) is the node feature matrix of the lth layer, W^(l) is a learnable weight matrix, and σ is a nonlinear activation function (such as ReLU). In order to capture more complex graph structure information, more advanced graph neural network variants can be used, such as the Graph Attention Network (GAT). GAT introduces an attention mechanism that allows nodes to give different importance to different neighbors: eij = a(Whi, Whj), αij = softmax(eij), h'i = σ(ΣjαijWhj), where a is the attention function, usually a small neural network. By stacking multiple layers of such operations, the model can gradually expand the receptive field and capture a wider range of structural information. The output of the last layer is the required node feature. In order to maintain the correspondence between the source and target graph features, the two sub-networks share the same network structure and parameters. This parallel processing method not only improves computational efficiency, but also ensures that the source node features and the target node features are in the same feature space, laying the foundation for subsequent feature association and aggregation.

[0222] Constructing the feature association relationship between source node features and target node features through the cross-attention mechanism in the dual-graph neural network is a key information fusion step. The cross-attention mechanism allows the model to establish dynamic and adaptive associations between the source and target graphs. The core idea is to let each source node feature interact with all target node features and vice versa. The specific implementation can be achieved through the following steps: First, the source node features and target node features are used as queries (Q), keys (K), and values ​​(V) respectively: Qs = WqHs, Kt = WkHt, Vt = WvHt, where Hs and Ht are the source and target node feature matrices, respectively, and Wq, Wk, and Wv are learnable weight matrices. Then, the attention score is calculated: A = softmax(QsKt^T / √d), where d is the feature dimension. Finally, aggregate the information: H's = AVt. This process can be done in both directions, that is, the attention from the target to the source is also calculated. The output of cross attention can be expressed as: H's = f([Hs||AVt]), H't = f([Ht||A^TVs]), where f is a feedforward neural network and || represents a concatenation operation. In order to capture multi-scale associations, a multi-head attention mechanism can be used, that is, multiple sets of independent attention are calculated in parallel, and then the results are concatenated:

[0223] MultiHead(Q,K,V)=Concat(head1,...,headh)Wo

[0224] Where headi = Attention(QWi^Q, KWi^K, VWi^V). Through this mechanism, the model can automatically learn the complex correspondence between the source and target graphs, capturing the structural changes and semantic evolution before and after the graph transformation. This step can establish a dynamic association between the source and target graph nodes, capture the structural differences and semantic changes between graphs, and provide important association information for subsequent feature aggregation.

[0225] The last step of feature fusion is to map the source node features and the target node features to the same high-dimensional feature space, and aggregate the source node features and the target node features in the high-dimensional feature space based on the feature association relationship to obtain the high-dimensional transaction association features of the transaction data lineage map. First, the source node features and the target node features are projected into a common high-dimensional space using a nonlinear mapping function: Fs = φ(Hs), Ft = φ(Ht), where φ can be a multi-layer perceptron (MLP) or other complex nonlinear functions. In order to ensure the alignment of features, an alignment loss can be introduced: Lalign = ||Fs-Ft||F, where ||·||F represents the Frobenius norm. Next, feature aggregation is performed based on the feature association relationship obtained in the previous step. An effective aggregation method is to use a gating mechanism: G = sigmoid(Wg[Fs||Ft]), F = G⊙Fs+(1-G)⊙Ft, where Wg is a learnable weight matrix and ⊙ represents element-wise multiplication. This mechanism allows the model to dynamically decide how much information to retain from the source and target features.

[0226] To capture more complex interactions, multi-layer aggregation can be used: F(l+1)=MLP(F(l)||(F(l)sA(l)+F(l)t(A(l))^T)), where A(l) is the attention matrix of the lth layer. Finally, a graph-level representation is obtained through a global pooling operation: Fglobal=READOUT({Fi|i∈G}), where READOUT can be a sum, average, or a more complex learnable function. This final high-dimensional transaction association feature Fglobal encodes the structural information, semantic information of the source and target graphs, and the changing relationship between them. This step can generate a high-dimensional feature representation containing rich information, capture the key differences before and after graph transformation, and provide high-quality feature input for subsequent anomaly detection or other downstream tasks.

[0227] In one embodiment, taking the abnormal graph node as the tracing starting point, locating the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph using the data lineage tracing method includes the following steps:

[0228] For each abnormal graph node, the node anomaly score of the abnormal graph node is calculated based on the high-dimensional transaction association features;

[0229] Using the abnormal graph node as the tracing starting point, trace the nodes in the transaction data lineage graph according to the interactive relationship;

[0230] In the process of node tracing, the abnormal impact score of the traced graph node is calculated based on the node abnormality score and an iterative algorithm is used, and multiple most influential predecessor nodes in the traced graph node are determined according to the abnormal impact score;

[0231] All the most influential predecessor nodes are used as the tracing end points to form multiple abnormal tracing paths between the tracing starting point and multiple tracing end points;

[0232] The path anomaly score of each abnormal tracing path is calculated by combining the anomaly score and the anomaly impact score. The abnormal tracing path with the largest path anomaly score is taken as the target abnormal tracing path, and the largest impact predecessor node in the target abnormal tracing path is taken as the origin abnormality graph node of the abnormal origin of the abnormal transaction data.

[0233] In this embodiment, for each abnormal graph node, calculating the node anomaly score of the abnormal graph node based on the high-dimensional transaction association features is a key step in quantifying the degree of anomaly. This process uses the high-dimensional transaction association features extracted in the previous step to assign an anomaly score to each node through an anomaly detection algorithm. An effective method is to use a distance-based anomaly detection algorithm, such as the local anomaly factor (LOF). The core idea of ​​LOF is to compare the local density of a point with its neighbors. The specific calculation steps are as follows: First, for each node i, calculate the average distance of its k nearest neighbors:

[0234] lrd(i)=1 / (Σj∈Nk(i)reach-distk(i,j) / k)

[0235] Where reach-distk(i,j)=max{k-distance(j),d(i,j)}, d(i,j) is the distance between nodes i and j. Then, the LOF value is calculated: LOF(i)=Σj∈Nk(i)(lrd(j) / lrd(i)) / k. Points with LOF values ​​significantly greater than 1 are considered anomalies. In order to normalize the LOF value to the [0,1] interval, the sigmoid function can be used: score(i)=1 / (1+exp(-α(LOF(i)-β))), where α and β are adjustable parameters. Another method is to use Isolation Forest, which is based on the principle that abnormal points are more easily isolated. The anomaly score is calculated as: s(x,n)=2^(-E(h(x)) / c(n)), where h(x) is the average path length of point x, and c(n) is the average path length when the dataset size is n. These methods can effectively identify nodes that behave abnormally in high-dimensional feature spaces. The calculated anomaly score not only quantifies the degree of anomaly of each node, but also provides important starting point information for subsequent anomaly tracing. This step assigns a quantitative anomaly score to each graph node, identifies the most likely abnormal transaction node, and provides priority guidance for anomaly tracing.

[0236] Using the abnormal graph node as the starting point for tracing, tracing nodes according to the interaction relationship in the transaction data lineage graph is a key step in backtracking analysis. This process aims to find the source node or impact path that may cause the abnormality. Node tracing uses a graph traversal algorithm, such as depth-first search (DFS) or breadth-first search (BFS), starting from the abnormal node and tracing upward along the interaction relationship. DFS is suitable for finding long paths, while BFS is more suitable for finding short-distance influences. Taking BFS as an example, the algorithm steps are as follows: (1) Initialize a queue Q and add the abnormal node to Q; (2) When Q is not empty, perform the following operations: a. Take a node v from Q; b. Check all predecessor nodes of v; c. Add the unvisited predecessor nodes to Q and mark them as visited; (3) Repeat step 2 until Q is empty. In order to avoid loops and excessive tracing, a maximum tracing depth D can be set. In actual implementation, an adjacency list or adjacency matrix can be used to represent the graph structure to improve search efficiency. In addition, heuristic rules can be added to optimize the tracing process, such as preferentially tracing edges with larger transaction amounts or higher transaction frequencies. During the tracing process, the depth and path information of each accessed node is recorded, which is crucial for the subsequent abnormal impact analysis. In order to process large-scale graphs, you can consider using parallel computing technology to trace back from multiple abnormal nodes at the same time. This tracing method can effectively construct a potential abnormal propagation network and provide a basis for subsequent abnormal impact analysis. This step constructs a tracing path from the abnormal node to the potential source, identifies related nodes that may be involved in abnormal activities, and provides graph structure support for subsequent abnormal impact assessment.

[0237] In the process of node tracing, the anomaly impact score of the traced graph node is calculated based on the node anomaly score and an iterative algorithm is used. The key step to quantify the impact of anomaly propagation is to determine the multiple most influential predecessor nodes in the traced graph node according to the anomaly impact score. The anomaly impact score reflects the degree of influence of a node on downstream anomaly nodes. The calculation process uses an iterative algorithm, which is similar to the idea of ​​PageRank. At initialization, the anomaly impact score of each node is set to its node anomaly score. Then, multiple rounds of iterative updates are performed:

[0238] IS(v)=α*AS(v)+(1-α)*Σu∈N_out(v)(IS(u)*w(v,u) / Σx∈N_in(u)w(x,u))

[0239] Where IS(v) is the abnormal impact score of node v, AS(v) is the node abnormal score, N_out(v) is the set of outgoing neighbors of v, w(v,u) is the weight of the edge (v,u) (which can be the transaction amount or frequency), and α is the balance parameter (usually between 0.1 and 0.3). Iterate until the score converges or the maximum number of iterations is reached. In order to improve the computational efficiency, matrix operations can be used: IS = α*AS + (1-α)*W*IS, where W is the normalized adjacency matrix. After each round of iteration, the scores are normalized to prevent numerical overflow. After the calculation is completed, the direct predecessor nodes of each node are sorted by the abnormal impact score, and the top k are selected as the most influential predecessor nodes. The choice of k can be a fixed value or dynamically determined based on the score threshold. This method not only takes into account the abnormality of the node itself, but also its influence in the entire network, and can effectively identify entities that may be the source of anomalies or key propagation nodes.

[0240] Taking all the most influential predecessor nodes as the tracing end points and forming multiple anomaly tracing paths between the tracing start point and multiple tracing end points is a key step in constructing a complete anomaly propagation network. This process involves path search and path optimization. First, use a backtracking algorithm or dynamic programming to find all the paths from the tracing start point (anomaly node) to each tracing end point (most influential predecessor node). For each pair of start point s and end point t, a modified version of the depth-first search (DFS) algorithm can be used: (1) Start from s and traverse upward along the edges; (2) Record all nodes in the current path; (3) Stop when t is reached or the maximum depth is reached; (4) If t is reached, save the current path; (5) Backtrack and explore other possible paths. To improve efficiency, pruning techniques can be used, such as stopping the search when the path length exceeds a preset threshold. In practical applications, there may be a large number of paths, so path screening and optimization are required. One method is to use the cumulative abnormal impact score of the path to sort: PathScore = Σv∈pathIS(v)*d^(-λ), where IS(v) is the abnormal impact score of node v, d is the depth of the node in the path, and λ is the attenuation factor. This calculation method takes into account the influence of all nodes on the path, while reducing the weight of distant nodes through the depth attenuation factor. Another optimization method is to consider the diversity of the path and use the maximum edge disjoint path algorithm to select a set of non-overlapping critical paths.

[0241] The path anomaly score of each anomaly tracing path is calculated by combining the anomaly score and the anomaly impact score. The anomaly tracing path with the largest path anomaly score is taken as the target anomaly tracing path, and the most influential predecessor node in the target anomaly tracing path is taken as the origin of the abnormal transaction data. The anomaly graph node is the last key step of anomaly tracing. The calculation of the path anomaly score needs to comprehensively consider the degree of anomaly and influence of all nodes on the path. An effective calculation method is: PathScore = Σv∈path(β*AS(v)+(1-β)*IS(v))*γ^d(v), where AS(v) is the anomaly score of node v, IS(v) is its anomaly impact score, d(v) is the depth of node v in the path, β is the weight for balancing the anomaly score and the impact score (usually between 0.4 and 0.6), and γ is the depth attenuation factor (usually less than 1). This calculation method not only considers the degree of anomaly and influence of the node, but also reflects the attenuation effect of anomaly propagation through depth attenuation.

[0242] To normalize the scores of paths of different lengths, PathScore can be divided by the path length. After calculating the anomaly scores of all paths, the path with the highest score is selected as the target anomaly tracing path. On this path, the predecessor node with the highest anomaly impact score is identified as the anomaly origin node. If there are multiple nodes with very close scores, they can all be listed as potential origin nodes. In order to increase the reliability of the results, an integrated method can be used, such as considering the top-k paths and determining the final anomaly origin based on their common characteristics. This step can quantify and compare the importance of different anomaly propagation paths, identify the most likely anomaly origin nodes, and provide key information for root cause analysis and risk management of abnormal transactions. This method can not only find the source of the anomaly, but also provide a detailed path of how the anomaly propagates in the transaction network, which is of great significance for understanding and preventing financial risks.

[0243] The present invention also discloses a domain data governance system based on an artificial intelligence platform, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the domain data governance method based on the artificial intelligence platform as described in any one of the above embodiments is implemented.

[0244] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0245] Among them, the memory can be an internal storage unit of a computer device, such as a hard disk or memory of a computer device, or an external storage device of a computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the computer device, etc., and the memory can also be a combination of an internal storage unit and an external storage device of a computer device. The memory is used to store computer programs and other programs and data required by the computer device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.

[0246] The present invention also discloses a computer-readable storage medium, on which instructions are stored. When the instructions are executed by a processor, the processor is configured to execute the domain data governance method based on the artificial intelligence platform described in any one of the above embodiments.

[0247] Among them, the computer program can be stored in a machine-readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or certain middleware, etc. The machine-readable medium includes any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the machine-readable medium includes but is not limited to the above-mentioned components.

[0248] Among them, through this computer-readable storage medium, the comprehensive fault detection method for transmission lines in the above embodiment is stored in a computer-readable storage medium, and is loaded and executed on a processor to facilitate the storage and application of the above method.

[0249] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of protection of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the present application as above, which are not provided in detail for the sake of simplicity.

[0250] One or more embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A domain data governance method based on an artificial intelligence platform, characterized in that: The steps include: Obtain financial transaction data through data warehouses; Performing data preprocessing on the financial transaction data, wherein the data preprocessing includes data cleaning processing and data repair processing; Extracting transaction metadata and semantic associations between the transaction metadata from the financial transaction data after data preprocessing by using natural language technology, and completing metadata classification of the transaction metadata by using a hierarchical clustering algorithm; Combining the semantic association relationship and the classified transaction metadata to construct a transaction data lineage graph of the financial transaction data, wherein the graph nodes in the transaction data lineage graph represent transaction metadata describing different types of financial transaction entities, and the graph node edges between the graph nodes represent interactive relationships between different transaction metadata; Extracting graph temporal features and graph spatial features from the transaction data lineage graph, combining the graph temporal features and the graph spatial features, and predicting the node interaction index of all graph nodes in the transaction data lineage graph through a pre-configured data interaction prediction model, wherein the data interaction prediction model is constructed based on a graph neural network model and a multi-layer perceptron; Marking the graph nodes whose node interaction index exceeds a preset index threshold as high-heat graph nodes; Taking a plurality of high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction association features from the transaction data lineage graph based on a graph neural network model, and identifying abnormal transaction data in the financial transaction data based on the high-dimensional transaction association features; Marking abnormal graph nodes in the bloodline graph of the transaction data based on the abnormal transaction data; Taking the abnormal graph node as the tracing starting point, a data lineage tracing method is used to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph.

2. The domain data governance method based on the artificial intelligence platform according to claim 1 is characterized in that: The data preprocessing of the financial transaction data comprises the following steps: For the numerical data in the financial transaction data, a hash algorithm is used to remove duplicate data in the numerical data; For the text data in the financial transaction data, performing format standardization processing on the text data to obtain standard text data; Using regular expressions to remove invalid data from the standard text data; The spelling correction processing and abbreviation expansion processing of the standard text data are completed based on a preset financial data standardization dictionary and a dictionary matching method.

3. The domain data governance method based on the artificial intelligence platform according to claim 2 is characterized in that: The extracting of transaction metadata and semantic associations between the transaction metadata from the financial transaction data after data preprocessing by using natural language technology, and completing metadata classification of the transaction metadata by using a hierarchical clustering algorithm comprises the following steps: Combining named entity recognition technology with a graph neural network model to build a financial data analysis model, the financial data analysis model is a BiLSTM-CRF-RCNN model; Complete the annotation training of the financial data analysis model according to the pre-created financial knowledge training set and using a remote supervision method; Inputting the preprocessed financial transaction data into the trained financial data analysis model, and identifying and extracting all transaction metadata describing financial transaction entities in the financial transaction data through the bidirectional long short-term memory network module and the conditional random field module in the financial data analysis model; Extracting the semantic association relationship between the transaction metadata through a convolutional network module including a self-attention mechanism in the financial data analysis model; A hierarchical clustering algorithm is used to complete metadata classification of the transaction metadata.

4. The domain data governance method based on the artificial intelligence platform according to claim 3 is characterized in that: The metadata classification of the transaction metadata using the hierarchical clustering algorithm comprises the following steps: Calculating the Gaussian distribution of the transaction metadata to obtain a high-dimensional similarity matrix of the transaction metadata in a high-dimensional space; Initializing low-dimensional data points in a preset low-dimensional space, where the number of the low-dimensional data points is the same as the number of the transaction metadata; Calculating the Cauchy distribution of all the low-dimensional data points to obtain a low-dimensional similarity matrix of the low-dimensional data points in the low-dimensional space; The KL divergence is calculated by combining the high-dimensional similarity matrix and the low-dimensional similarity matrix; Minimize the KL divergence using a gradient descent method and simultaneously optimize and update the position of the low-dimensional data point in the low-dimensional space until the KL divergence reaches a minimum value, completing the process of reducing the dimension of the transaction metadata to the low-dimensional space; The minimized intra-class variance of the transaction metadata is calculated in the low-dimensional space, and metadata classification of the transaction metadata in the low-dimensional space is completed based on the minimized intra-class variance.

5. The domain data governance method based on the artificial intelligence platform according to claim 1 is characterized in that: The extracting of graph temporal features and graph spatial features from the transaction data lineage graph, combining the graph temporal features and the graph spatial features, and predicting the node interaction index of all the graph nodes in the transaction data lineage graph through a pre-configured data interaction prediction model comprises the following steps: Extracting graph temporal features and graph spatial features from the transaction data lineage graph, wherein the graph temporal features include time feature representations of all graph nodes, and the graph spatial features include node spatial topological feature representations of all graph nodes, and the node spatial topological feature representations include node PageRank values, node degree numbers, node betweenness centrality, and central node connectivity; Performing feature fusion on the graph temporal features and the graph spatial features to obtain a graph spatiotemporal fusion feature; Input the transaction data lineage graph and the graph spatiotemporal fusion features into a pre-configured data interaction prediction model, and extract node interaction features between all graph nodes from the graph spatiotemporal fusion features through multiple gated loop modules in the data interaction prediction model; The context information of the transaction data lineage graph is captured by the graph convolutional neural network module in the data interaction prediction model, and the node interaction features of all the graph nodes are spliced ​​into a graph comprehensive representation vector according to the context information; Based on the comprehensive representation vector of the graph and through the multi-layer perceptron module in the data interaction prediction model, the node interaction index of all the graph nodes in the transaction data lineage graph is predicted.

6. The domain data governance method based on the artificial intelligence platform according to claim 5 is characterized in that: The method of using a plurality of high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction association features from the transaction data lineage graph based on a graph neural network model, and identifying abnormal transaction data in the financial transaction data according to the high-dimensional transaction association features includes the following steps: Counting the number of first nodes of the graph nodes and the number of second nodes of the high-heat graph nodes in the transaction data lineage graph; Determining a graph transformation degree according to the first number of nodes and the second number of nodes, and randomly generating or removing graph node edges in the transaction data lineage graph according to the graph transformation degree to obtain a transaction data transformation graph; Using multiple high-heat graph nodes of different entity types in the transaction data lineage graph as source graph nodes, and using nodes corresponding to the source graph nodes in the transaction data transformation graph as target graph nodes; Combining the source graph node and the target graph node and using a dual-graph neural network model to extract high-dimensional transaction association features of the transaction data lineage graph; Based on the high-dimensional transaction association features and using an anomaly detection algorithm, abnormal transaction data in the financial transaction data is identified.

7. The domain data governance method based on the artificial intelligence platform according to claim 6 is characterized in that: The step of combining the source graph node and the target graph node and extracting the high-dimensional transaction association features of the transaction data lineage graph using a dual-graph neural network model comprises the following steps: The dual-graph neural network model is used to simultaneously extract the source node features of the source graph node and the target node features of the target graph node; Constructing a feature association relationship between the source node feature and the target node feature through a cross attention mechanism in the dual-graph neural network; The source node features and the target node features are mapped to the same high-dimensional feature space, and the source node features and the target node features are aggregated in the high-dimensional feature space based on the feature association relationship to obtain high-dimensional transaction association features of the transaction data lineage map.

8. The domain data governance method based on the artificial intelligence platform according to claim 6 is characterized in that: The method of using the abnormal graph node as the tracing starting point and using the data lineage tracing method to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph comprises the following steps: For each of the abnormal graph nodes, calculating a node anomaly score of the abnormal graph node according to the high-dimensional transaction association feature; Using the abnormal graph node as a tracing starting point, tracing nodes in the transaction data lineage graph according to the interactive relationship; In the node tracing process, an abnormal impact score of the traced graph node is calculated based on the node abnormality score and an iterative algorithm is used, and a plurality of most influential predecessor nodes in the traced graph node are determined according to the abnormal impact score; Taking all the most influential predecessor nodes as tracing end points, a plurality of abnormal tracing paths between the tracing starting point and the plurality of tracing end points are formed; The path anomaly score of each of the abnormal tracing paths is calculated in combination with the anomaly score and the anomaly impact score, the abnormal tracing path with the largest path anomaly score is used as the target abnormal tracing path, and the largest impact predecessor node in the target abnormal tracing path is used as the origin abnormality graph node of the abnormal origin of the abnormal transaction data.

9. A domain data governance system based on an artificial intelligence platform, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the domain data governance method based on the artificial intelligence platform as described in any one of claims 1 to 8.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instruction is executed by a processor, the processor is configured to execute the domain data governance method based on an artificial intelligence platform according to any one of claims 1 to 8.

Images