Domain data governance method and system based on artificial intelligence platform

Abstract

The application provides a domain data governance method and system based on an artificial intelligence platform, which obtains financial transaction data through a data warehouse, pre-processes the data, and uses natural language technology to extract transaction metadata and semantic association thereof. A hierarchical clustering algorithm is used to classify the metadata, and a transaction data blood relationship graph is constructed. Time sequence and spatial features are extracted from the graph, a pre-configured data interaction prediction model is used to calculate node interaction indexes, and high-heat nodes are marked. High-dimensional transaction association features are extracted based on a graph neural network model, and abnormal transaction data are identified. Abnormal nodes are marked in the blood relationship graph, and the origin nodes of the abnormal data are tracked and located. The application comprehensively utilizes various advanced technologies, and can realize rapid and accurate risk identification when facing massive financial transaction data.

Description

Technical Field

[0001] This invention belongs to the field of data processing technology, specifically relating to a domain data governance method and system based on an artificial intelligence platform. Background Technology

[0002] In today's era of rapid information technology development, data warehouses of internet companies and banks store vast amounts of financial data. The scale of financial data is growing exponentially, encompassing everything from individual transactions to global markets. These data points are interconnected intricately, forming a massive data network. The frequency of financial data updates is accelerating, especially in areas like high-frequency trading and real-time risk control, where changes can occur at millisecond rates. Against this backdrop, accurately identifying transaction risks within financial data has become a critical technical challenge that urgently needs to be addressed.

[0003] Existing technologies face significant challenges in processing such massive, complex, and rapidly changing financial data. Traditional data analysis methods, often based on static data or simple time-series analysis, struggle to capture the complex relationships and dynamic changes between data points. This leads to numerous shortcomings in identifying trading risks, such as high false positive rates, frequent missed detections, and slow response times. Especially when facing new financial products and innovative trading models, existing technologies often fail to identify potential risks in a timely and effective manner. Summary of the Invention

[0004] This invention provides a domain data governance method and system based on an artificial intelligence platform to solve the problem in existing technologies that make it difficult to quickly and accurately identify potential data risks in massive amounts of financial data.

[0005] In a first aspect, the present invention provides a domain data governance method based on an artificial intelligence platform, the method comprising the following steps:

[0006] Obtain financial transaction data through a data warehouse;

[0007] The financial transaction data is preprocessed, including data cleaning and data repair.

[0008] Natural language processing technology is used to extract transaction metadata and semantic relationships between the preprocessed financial transaction data, and a hierarchical clustering algorithm is used to classify the transaction metadata.

[0009] construct a transaction data bloodline graph of the financial transaction data according to the semantic association relationship and the classified transaction metadata, wherein a graph node in the transaction data bloodline graph represents transaction metadata describing different types of financial transaction entities, and a graph node edge between the graph nodes represents an interaction relationship between different transaction metadata;

[0010] extract graph temporal features and graph spatial features from the transaction data bloodline graph, combine the graph temporal features and the graph spatial features, and predict a node interaction index of all the graph nodes in the transaction data bloodline graph by using a preconfigured data interaction prediction model based on a graph neural network model and a multilayer perception mechanism;

[0011] mark a graph node with the node interaction index exceeding a preset index threshold as a high-heat graph node;

[0012] use a plurality of high-heat graph nodes of different entity types as source graph nodes, extract high-dimensional transaction association features from the transaction data bloodline graph based on a graph neural network model, and identify abnormal transaction data in the financial transaction data according to the high-dimensional transaction association features;

[0013] mark an abnormal graph node in the transaction data bloodline graph based on the abnormal transaction data;

[0014] use the abnormal graph node as a tracking starting point, and locate an origin abnormal graph node corresponding to the abnormal transaction data in the transaction data bloodline graph by using a data bloodline tracking method.

[0015] Optionally, the data preprocessing of the financial transaction data includes the following steps:

[0016] for numerical data in the financial transaction data, a hash algorithm is used to remove duplicate data in the numerical data;

[0017] for text data in the financial transaction data, format standardization processing is performed on the text data to obtain standard text data;

[0018] invalid data in the standard text data is removed by using a regular expression;

[0019] spelling correction processing and abbreviation expansion processing of the standard text data are completed based on a preconfigured financial data standardization dictionary and by using a dictionary matching method.

[0020] Optionally, the extracting transaction metadata and semantic association relationship between the transaction metadata from the pre-processed financial transaction data by natural language technology and completing metadata classification of the transaction metadata by hierarchical clustering algorithm comprises the following steps:

[0021] The financial data analysis model is constructed by combining named entity recognition technology and a graph neural network model, and the financial data analysis model is a BiLSTM-CRF-RCNN model.

[0022] The labeling and training of the financial data analysis model are completed according to a pre-created financial knowledge training set and by using a remote supervision method.

[0023] The pre-processed financial transaction data is input into the trained financial data analysis model, and all transaction metadata describing financial transaction entities in the financial transaction data are recognized and extracted by a bidirectional long short-term memory network module and a conditional random field module in the financial data analysis model.

[0024] The semantic association relationship between the transaction metadata is extracted by a convolutional network module with a self-attention mechanism in the financial data analysis model.

[0025] The metadata classification of the transaction metadata is completed by using a hierarchical clustering algorithm.

[0026] Optionally, the metadata classification of the transaction metadata by using a hierarchical clustering algorithm comprises the following steps:

[0027] The Gaussian distribution of the transaction metadata is calculated to obtain a high-dimensional similarity matrix of the transaction metadata in a high-dimensional space.

[0028] The low-dimensional data points in a preset low-dimensional space are initialized, and the number of the low-dimensional data points is the same as the number of the transaction metadata.

[0029] The Cauchy distribution of all the low-dimensional data points is calculated to obtain a low-dimensional similarity matrix of the low-dimensional data points in the low-dimensional space.

[0030] The KL divergence is calculated by combining the high-dimensional similarity matrix and the low-dimensional similarity matrix.

[0031] The KL divergence is minimized by using a gradient descent method, and the positions of the low-dimensional data points in the low-dimensional space are simultaneously optimized and updated until the KL divergence reaches a minimum value, thereby completing the dimension reduction of the transaction metadata to the low-dimensional space.

[0032] The minimum intra-class variance of the transaction metadata in the low-dimensional space is calculated, and the metadata classification of the transaction metadata in the low-dimensional space is completed based on the minimum intra-class variance.

[0033] Optionally, the step of extracting the graph temporal features and the graph spatial features from the transaction data bloodline graph, combining the graph temporal features and the graph spatial features, and predicting the node interaction indexes of all the graph nodes in the transaction data bloodline graph through a pre-configured data interaction prediction model comprises the following steps:

[0034] extracting the graph temporal features and the graph spatial features from the transaction data bloodline graph, wherein the graph temporal features comprise time feature representations of all the graph nodes, and the graph spatial features comprise node spatial topology feature representations of all the graph nodes, and the node spatial topology feature representations comprise node PageRank values, node degree quantities, node betweenness centrality, and central node connectivity;

[0035] performing feature fusion on the graph temporal features and the graph spatial features to obtain graph spatio-temporal fusion features;

[0036] inputting the transaction data bloodline graph and the graph spatio-temporal fusion features into a pre-configured data interaction prediction model, and extracting node interaction features between all the graph nodes from the graph spatio-temporal fusion features through multiple gate recurrent modules in the data interaction prediction model;

[0037] capturing context information of the transaction data bloodline graph through a graph convolutional neural network module in the data interaction prediction model, and concatenating the node interaction features of all the graph nodes into a graph comprehensive representation vector according to the context information;

[0038] predicting the node interaction indexes of all the graph nodes in the transaction data bloodline graph based on the graph comprehensive representation vector and through a multi-layer perceptron module in the data interaction prediction model.

[0039] Optionally, the step of taking multiple high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction correlation features from the transaction data bloodline graph based on a graph neural network model, and identifying abnormal transaction data in the financial transaction data according to the high-dimensional transaction correlation features comprises the following steps:

[0040] counting a first node quantity of the graph nodes and a second node quantity of the high-heat graph nodes in the transaction data bloodline graph;

[0041] determining a graph transformation degree according to the first node quantity and the second node quantity, and randomly generating or removing graph node edges in the transaction data bloodline graph according to the graph transformation degree to obtain a transaction data transformed graph;

[0042] Take a plurality of high-frequency graph nodes of different entity types in the transaction data bloodline graph as source graph nodes, and take the corresponding nodes of the source graph nodes in the transaction data transformation graph as target graph nodes;

[0043] Combine the source graph nodes and the target graph nodes and extract high-dimensional transaction correlation features of the transaction data bloodline graph using a dual-graph neural network model;

[0044] Based on the high-dimensional transaction correlation features and using an anomaly detection algorithm, identify abnormal transaction data in the financial transaction data.

[0045] Optionally, the combining the source graph nodes and the target graph nodes and extracting high-dimensional transaction correlation features of the transaction data bloodline graph using a dual-graph neural network model comprises the following steps:

[0046] Simultaneously extract source node features of the source graph nodes and target node features of the target graph nodes using a dual-graph neural network model;

[0047] Construct a feature correlation relationship between the source node features and the target node features through a cross-attention mechanism in the dual-graph neural network;

[0048] Map the source node features and the target node features to the same high-dimensional feature space, and aggregate the source node features and the target node features in the high-dimensional feature space based on the feature correlation relationship, to obtain high-dimensional transaction correlation features of the transaction data bloodline graph.

[0049] Optionally, the taking the abnormal graph nodes as tracking starting points and using a data bloodline tracking method to locate the origin abnormal graph nodes corresponding to the abnormal transaction data in the transaction data bloodline graph comprises the following steps:

[0050] For each abnormal graph node, calculate a node anomaly score of the abnormal graph node according to the high-dimensional transaction correlation features;

[0051] Take the abnormal graph nodes as tracking starting points and track nodes in the transaction data bloodline graph according to the interaction relationship;

[0052] In the node tracking process, calculate an abnormal influence score of a tracked graph node based on the node anomaly score and using an iterative algorithm, and determine a plurality of maximum influence predecessor nodes in the tracked graph node according to the abnormal influence score;

[0053] Take all the maximum influence predecessor nodes as tracking endpoints, and form a plurality of abnormal tracking paths between the tracking starting points and the tracking endpoints;

[0054] The path anomaly score of each abnormal trace path is calculated by combining the abnormal score and the abnormal influence score, the abnormal trace path with the maximum path anomaly score is taken as a target abnormal trace path, and the maximum influence predecessor node in the target abnormal trace path is taken as an origin abnormal graph node of the abnormal transaction data origin.

[0055] In a second aspect, the present application further provides a domain data governance system based on an artificial intelligence platform, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the domain data governance method based on the artificial intelligence platform as described in the first aspect when executing the computer program.

[0056] In a third aspect, the present application further provides a computer-readable storage medium having instructions stored thereon, wherein the instructions, when executed by a processor, cause the processor to be configured to perform the domain data governance method based on the artificial intelligence platform according to the first aspect.

[0057] The present application has the following beneficial effects:

[0058] The present application effectively solves the limitations of traditional methods in processing complex financial data by constructing transaction data bloodline graphs and using advanced graph neural network models. It can comprehensively capture the semantic association and spatio-temporal characteristics between data, greatly improving the accuracy and efficiency of abnormal transaction identification. The data interaction prediction model of the present application can dynamically predict node interaction indexes, realizing real-time monitoring of high-risk transactions. By marking high-heat graph nodes and extracting high-dimensional transaction association features, the present application can more accurately identify potential risks and reduce false positives and false negatives. In addition, the data bloodline tracking method of the present application can quickly locate the source of abnormal transactions, providing key support for risk management. This comprehensive and in-depth analysis method not only improves the risk control ability of financial institutions, but also provides stronger decision-making basis for regulatory departments. The implementation of the present application helps to build a safer and more stable financial ecosystem, while effectively preventing systemic financial risks. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 It is a flowchart of the domain data governance method based on an artificial intelligence platform in one of the embodiments of the present application. DETAILED DESCRIPTION

[0060] With reference to the drawings and the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly described. Obviously, the described embodiments are only some of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art are within the scope of protection of the present application.

[0061] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a category and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in a "or" relationship.

[0062] Figure 1 The flowchart of the method for domain data governance based on an artificial intelligence platform in one embodiment is shown. It should be understood that, although Figure 1 the steps in the flowchart are displayed in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps has no strict order limitation, and these steps can be executed in other orders. Moreover, Figure 1 at least part of the steps in the flowchart can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or sub-steps or stages of other steps. As Figure 1 shown, the method for domain data governance based on an artificial intelligence platform specifically includes the following steps:

[0063] S101. Obtain financial transaction data through a data warehouse.

[0064] Firstly, a centralized data warehouse is established, which adopts a distributed storage architecture to cope with massive data storage requirements. The core of the data warehouse is to extract data from various source systems using the ETL (Extract, Transform, Load) process. These source systems include but are not limited to bank core transaction systems, securities trading platforms, insurance company systems, third-party payment platforms, credit card transaction systems, etc. The data extraction stage adopts an incremental extraction method, only obtaining newly added or changed data since the last extraction, to improve efficiency and reduce network transmission burden. In order to ensure the real-time nature of the data, the CDC (Change Data Capture) technology can be used to capture data changes in the source system in real time. In the data transformation stage, ETL tools such as Informatica, Talend or open-source ApacheNifi are used for data cleaning and conversion, which unifies the formats of data from different sources. In the data loading stage, parallel loading techniques are used, such as Oracle's direct path loading or Hadoop's bulk import, which significantly improves the data loading speed. The data warehouse adopts a snowflake model design, with transaction fact tables as the center, and associated customer dimensions, product dimensions, time dimensions, geographic location dimensions, etc. In order to optimize query performance, materialized views and appropriate indexes are established for frequently accessed data.

[0065] S102. Data preprocessing is performed on the financial transaction data.

[0066] Among them, the large amount of financial transaction data obtained is preliminarily preprocessed, and the data preprocessing includes data cleaning processing and data repair processing, and the realization of data cleaning and repair is a key step to ensure the accuracy of subsequent analysis. This process includes multiple sub-steps, each of which targets specific data quality problems. First, data deduplication is performed, using a hash algorithm-based method to quickly identify completely duplicated records, and for approximately duplicated records, a more complex similarity algorithm such as Jaccard similarity or edit distance is used. Then, missing values are handled. For randomly missing data, multiple imputation (Multiple Imputation) can be used, which generates multiple complete data sets and then combines the results. For non-random missing data, a missing model needs to be established, such as using the Markov Chain Monte Carlo (MCMC) method. For data with inconsistent formats, standardization processing is performed. For example, regular expressions are used to standardize date formats and convert different currency units to a unified unit. Address data standardization can use a geographic coding API to convert unstructured addresses to standard latitude and longitude coordinates.

[0067] S103. The transaction metadata and the semantic association relationship between the transaction metadata are extracted from the data preprocessed financial transaction data through natural language technology, and the metadata classification of the transaction metadata is completed by using a hierarchical clustering algorithm.

[0068] Where extracting and classifying financial transaction metadata from pre-processed financial transaction data is a complex natural language processing task. This process involves multiple sub-tasks, including named entity recognition, relation extraction, and metadata classification. First, use named entity recognition (NER) techniques to identify key entities in transaction data, such as product names, institution names, transaction amounts, etc. Use a deep learning-based BiLSTM-CRF model, where the BiLSTM layer captures contextual information and the CRF layer ensures the consistency of labeling. To improve the generalization ability of the model, introduce a pre-trained language model such as BERT as a feature extractor, and fine-tune the entire model for the NER task in the financial domain.

[0069] Next, use relation extraction techniques to identify relationships between entities, such as "transaction," "regulation," etc. Use a remote supervision method to automatically label training data using existing financial knowledge graphs. Then use a neural network model with attention mechanism for relation classification. The model structure can be based on graph neural networks, such as R-GCN (Relational Graph Convolutional Network). For the extracted metadata, use hierarchical clustering algorithms for classification. Use Ward's method, which aims to minimize intra-class variance. To handle high-dimensional data, first use dimensionality reduction techniques such as t-SNE or UMAP to map metadata to low-dimensional space, then perform clustering in low-dimensional space. To improve the accuracy and interpretability of classification, introduce domain expert knowledge. Establish a financial terminology dictionary and ontology library to guide entity recognition and relation extraction. At the same time, use topic models such as LDA (Latent Dirichlet Allocation) to discover the underlying topics in the metadata, the generation process of LDA can be represented as:

[0070] 1. For each document d, sample the topic distribution θd ~ Dir(α) from the Dirichlet distribution

[0071] 2. For each topic k, sample the word distribution φk ~ Dir(β) from the Dirichlet distribution

[0072] 3. For each word wdi in document d:

[0073] a. Sample the topic zdi ~ Multinomial(θd) from the multinomial distribution

[0074] b. Sample the word wdi ~ Multinomial(φzdi) from the multinomial distribution

[0075] Finally, the clustering results, expert knowledge, and topic model results are integrated to obtain the final metadata classification. This multi-angle classification method can fully utilize the semantic information and structural information of the data, improving the accuracy and reliability of the classification. Through this complex natural language processing process, unstructured or semi-structured financial transaction data can be converted into structured metadata and classified according to semantic similarity, providing rich and accurate basic data for subsequent graph network construction.

[0076] S104. Construct a transaction data bloodline graph of financial transaction data in combination with the semantic association relationship and the classified transaction metadata.

[0077] Among them, the transaction data bloodline graph is constructed according to the extracted and classified financial transaction metadata. This graph not only shows the relationship between financial entities, but also reflects the flow and dependence of data. The graph nodes in the transaction data bloodline graph represent transaction metadata describing different types of financial transaction entities, and the graph node edges between the graph nodes represent the interaction between different transaction metadata. Specifically, first, different types of financial transaction entities (such as product names, institution names, index names, etc.) are taken as nodes of the graph. Each node is represented by a multi-dimensional feature vector, containing attribute information of the entity, such as entity type, creation time, update frequency, etc. Then, according to the semantic relationship between entities (such as transaction, supervision, investment, etc.), the edges between nodes are established. The edge is directed, indicating the direction of data flow or influence. The edge also has a weight, indicating the strength or importance of the relationship. In order to capture complex multi-entity relationships, the concept of hyperedge is introduced, allowing a single edge to connect multiple nodes. The hypergraph can be represented by a tensor H, where H[e][v] represents the association strength between hyperedge e and node v.

[0078] The construction process of the graph can be represented as an iterative updating process:

[0079] 1. Initialize the node set V and the edge set E.

[0080] 2. For each new metadata record m: a. If m introduces a new entity, add it to V; b. Update the attributes of related nodes; c. Update E according to the relationship information of m.

[0081] 3. Periodically perform graph compression and optimization, merge similar nodes, and delete low-weight edges.

[0082] The transaction data bloodline graph constructed in this way not only visually displays the complex relationship between financial transaction entities, but also reflects the flow, dependence, and evolution process of data. This graph structure provides a strong topological structure foundation for subsequent anomaly detection, risk assessment, and decision support.

[0083] S105. Extracting the graph temporal features and graph spatial features from the transaction data bloodline graph, combining the graph temporal features and graph spatial features, and predicting the node interaction index of all graph nodes in the transaction data bloodline graph through the pre-configured data interaction prediction model.

[0084] Wherein, the data interaction prediction model is built based on a graph neural network model and a multilayer perception mechanism. Extracting graph temporal features and graph spatial features from the transaction data bloodline graph and predicting high-heat graph nodes using a multilayer vector machine is a complex multi-stage process. First, extract the graph temporal features. This involves analyzing snapshots of the graph at different time points. Dynamic graph neural networks (DGNN) are used to capture the evolution of the graph structure over time. Next, extract the graph spatial features. This includes node-level, edge-level, and graph-level features. For node-level features, various centrality measures are calculated, such as degree centrality, betweenness centrality, and eigenvector centrality. Finally, a multi-layer support vector machine (MLSVM) is used to predict high-heat graph nodes. MLSVM is a variant of SVM that uses a multi-layer structure to handle complex data that is not linearly separable. The training process of MLSVM is carried out layer by layer, and the output of each layer is used as the input of the next layer. The output of the last layer is the heat prediction of the node. To handle the class imbalance problem (high-heat nodes may be less), oversampling techniques such as SMOTE (Synthetic Minority Over-sampling Technique) are used to increase the number of minority class samples.

[0085] S106. Marking the graph nodes with node interaction index exceeding the preset index threshold as high-heat graph nodes.

[0086] S107. Taking multiple high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction association features from the transaction data bloodline graph based on the graph neural network model, and identifying abnormal transaction data in the financial transaction data according to the high-dimensional transaction association features.

[0087] Wherein, first, the high-heat graph nodes identified in the previous step are classified according to entity types, such as financial products, transaction institutions, and transaction accounts. These high-heat nodes will serve as the starting point for anomaly detection, as they are often the center or source of abnormal behavior. For each entity type, select the top-k nodes with the highest heat as the source node set S.

[0088] Next, high-order topological features are extracted from the transaction metadata graph network using a graph neural network (GNN). Specifically, a graph attention network (GAT) can be used as the basic model because GAT can adaptively assign different importance to different neighbor nodes. Based on these high-order topological features, an anomaly detection algorithm is used to preliminarily identify abnormal transactions. Here, a graph-based anomaly detection method called GraphDeviationNetwork (GDN) is used. The core idea of GDN is to compare the difference between the actual features of a node and the features predicted by its neighborhood. Through the above process, the graph structure information can be fully utilized, and abnormal patterns in the financial transaction network can be effectively identified. This method not only considers the features of a single transaction, but also considers its position and relationship in the entire network, so it can capture more complex and hidden abnormal behaviors.

[0089] S108. Based on the abnormal transaction data, mark the abnormal graph node in the transaction data bloodline graph.

[0090] S109. Take the abnormal graph node as the starting point of tracking, and use the data bloodline tracking method to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data bloodline graph.

[0091] Among them, first, for each abnormal node i, define its abnormality degree. Next, use the data bloodline tracking method to locate the origin of abnormal transactions. This process can be seen as a reverse propagation on the graph. Starting from each abnormal node, step by step, trace back along the incoming edges, and build a directed acyclic graph (DAG) representation of the data flow direction. During the tracing process, the weight of the edge and the abnormality degree of the node are used to calculate the influence score. In order to deal with the possible existence of ring dependence, an iterative algorithm is used to calculate the influence score. In order to accelerate the convergence and deal with large-scale graphs, graph sampling methods such as GraphSAGE can be used to approximate the calculation of the influence score. While calculating the influence score, record the maximum influence predecessor node of each node, thereby constructing the influence propagation tree. By backtracking these maximum influence predecessor nodes, the path from the abnormal node to the potential origin node can be found. In order to more comprehensively analyze the abnormal propagation pattern, path-based abnormal propagation analysis is introduced. Considering the time sequence of financial transactions, a time window constraint is added during the tracing process. A time decay function is defined. In order to distinguish different types of abnormal propagation patterns, a multi-dimensional abnormality vector is introduced. Finally, based on the influence score and the path analysis results, the most likely abnormal origin node set O is identified. Through this complex data bloodline tracking process, not only can the potential origin of abnormal transactions be located, but also the path and pattern of abnormal propagation can be revealed. This method provides in-depth insight into the formation mechanism of financial risks, and helps financial institutions and regulatory authorities to take more targeted risk prevention and control measures. At the same time, by analyzing the abnormal propagation path, key risk nodes and weak links can also be identified, providing important basis for optimizing the structure of the transaction network and formulating risk management strategies.

[0092] In one embodiment, the data preprocessing of the financial transaction data includes the following steps:

[0093] For numerical data in the financial transaction data, a hash algorithm is used to remove duplicate data in the numerical data;

[0094] For text data in the financial transaction data, the text data is subjected to format standardization processing to obtain standard text data;

[0095] Invalid data in the standard text data is removed using a regular expression;

[0096] Based on a pre-set financial data standardization dictionary and using a dictionary matching method, spelling correction processing and abbreviation expansion processing of the standard text data are completed.

[0097] In this embodiment, first, each numerical data is converted into a unique hash value, and then these hash values are used to quickly identify and remove duplicates. In practice, a secure hash algorithm such as SHA-256 can be used. For each numerical value x, calculate its hash value h(x) = SHA-256(x). Then create an empty set S, and iterate through all numerical values. If h(x) is not in S, keep x and add h(x) to S; otherwise, consider x as a duplicate and delete it. To handle the precision problem of floating-point numbers, the numerical value can be rounded or truncated before hashing. For example, for amount data, the numerical value can be converted to an integer in units of cents before hashing. In addition, to improve efficiency, a Bloom filter can be used to quickly determine whether an element may be in a set, thereby reducing the number of hash calculations. The false positive rate of the Bloom filter can be estimated by the following formula: p ≈ (1 - e^(-kn / m))^k. Where k is the number of hash functions, n is the number of elements, and m is the size of the bit array. By adjusting these parameters, a balance between space efficiency and accuracy can be achieved. This method not only effectively removes duplicate data, but also greatly improves the efficiency of subsequent data processing, laying a good foundation for financial transaction data analysis.

[0098] Next, the text data in the financial transaction data needs to be standardized in format. This process involves multiple sub-tasks, including case normalization, white space processing, and special character standardization. First, convert all text to lowercase to eliminate inconsistencies caused by case differences. The specific code can be: standardized_text = text.lower(). Then, use regular expressions to process white space, replacing multiple consecutive spaces, tabs, line breaks, etc. with a single space. The specific code can be: standardized_text = re.sub(r'\s+', ", standardized_text). For special characters such as currency symbols, percent signs, etc., they need to be uniformly converted. For example, different currency symbols are replaced with standard codes. This can be achieved by creating a mapping dictionary and iterating through the replacement. The specific code can be:

[0099] for symbol, code in currency_map.items():

[0100] standardized_text = standardized_text.replace(symbol, code)

[0101] Additionally, date and time formats need to be handled. Different date formats can be identified using regular expressions and then converted to the ISO 8601 standard format (YYYY-MM-DD). The specific code can be:

[0102] date_pattern = r'\b(\d{1,2})[ / .-](\d{1,2})[ / .-](\d{2,4})\b'

[0103] standardized_text = re.sub(date_pattern, lambda m: f"{m.group(3).zfill(4)}-{m.group(2).zfill(2)}-{m.group(1).zfill(2)}", standardized_text)

[0104] For numerical formats, use a decimal point as the decimal separator and a comma as the thousand separator. Through these standardization processes, the consistency of text data can be significantly improved, laying the foundation for subsequent analysis and processing. Standardized text data is easier to compare and aggregate, which can improve the effectiveness of data mining and machine learning algorithms, and also facilitates manual review and understanding.

[0105] Next, we need to use regular expressions to remove invalid data from the standardized text data. This process mainly targets data items that do not meet the expected format or content, in order to improve the overall data quality. In financial transaction data, invalid data may include incorrectly formatted dates, illegal amount values, and non-standard account numbers. For each type of invalid data, a corresponding regular expression needs to be designed for matching and processing. For example, for date data, the following regular expression can be used to check the format: date_pattern = r'^(\d{4})-(\d{2})-(\d{2})$'. For amount data, the following code can be used: amount_pattern = r'^\d+(\.\d{1,2})?$'. For account numbers, assuming it should be 16 digits, the following code can be used: account_pattern = r'^\d{16}$'. The process of data validation using these regular expressions can be represented as:

[0106] validated_data = [item for item in data if re.match(pattern, item)]

[0107] where pattern is the corresponding regular expression and data is the list of data to be validated.

[0108] In addition to verifying data formats, regular expressions can also be used to remove special characters or unwanted content from text. For example, to remove all non-alphanumeric characters, the code can be:

[0109] cleaned_text = re.sub(r'[^a-zA-Z0-9\s]', "", text)

[0110] In practical applications, more complex regular expressions may be needed based on specific data characteristics and business requirements. Meanwhile, to improve efficiency, commonly used regular expressions can be pre-compiled. By using regular expressions to remove invalid data, the quality and consistency of data can be significantly improved. This not only reduces errors and biases in subsequent analysis, but also improves the efficiency of data processing. At the same time, the use of regular expressions also provides flexibility and scalability for data cleaning processes, which can be adjusted and optimized according to changing data characteristics and business requirements.

[0111] Next, based on the pre-set financial data standardization dictionary and using the dictionary matching method, the spelling correction and abbreviation expansion of the standard text data are completed. This process involves two main tasks: spelling correction and abbreviation expansion. First, a financial dictionary containing correct spelling and common abbreviations and their full forms is constructed. This dictionary can be represented as a mapping:

[0112] D = {misspelled_word:correct_word, abbreviation:full_form}

[0113] For example: {"finacial":"financial","ROI":"Return on Investment"}

[0114] For spelling correction, the edit distance algorithm (such as Levenshtein distance) can be used to find the closest correct spelling. Given a word w, its correction process can be represented as:

[0115] corrected_word = argmin_{vinD} Levenshtein(w, v)

[0116] where Levenshtein(w, v) calculates the edit distance between w and v. To improve efficiency, BK trees (Burkhard-Keller trees) can be used to quickly find similar words.

[0117] For abbreviation expansion, direct dictionary matching is used:

[0118] expanded_word = D.get(word, word)

[0119] Here if word is an abbreviation in the dictionary, return its full name; otherwise keep it as is.

[0120] In actual processing, the context information can be combined to improve accuracy. For example, using n-gram model to calculate the probability of candidate words in the current context, select the candidate word with the highest probability as the correction result:

[0121] P(w|context) = count(context, w) / count(context)

[0122] corrected_word = argmax_{v in candidates} P(v|context)

[0123] In order to process domain-specific terms and new words, a dynamic updating mechanism can be set up to update the dictionary regularly according to new data. At the same time, for the cases that cannot be determined, it can be marked for manual review.

[0124] In one embodiment, transaction metadata and semantic association relationships between transaction metadata are extracted from preprocessed financial transaction data by natural language technology, and hierarchical clustering algorithm is used to complete metadata classification of transaction metadata, including the following steps:

[0125] The financial data analysis model is constructed by combining named entity recognition technology and graph neural network model, and the financial data analysis model is BiLSTM-CRF-RCNN model;

[0126] According to the pre-created financial knowledge training set and using the remote supervision method to complete the annotation training of the financial data analysis model;

[0127] The preprocessed financial transaction data is input into the trained financial data analysis model, and all transaction metadata describing financial transaction entities in the financial transaction data are recognized and extracted by the bidirectional long short-term memory network module and the conditional random field module in the financial data analysis model;

[0128] The semantic association relationships between transaction metadata are extracted by the convolutional network module containing self-attention mechanism in the financial data analysis model;

[0129] The hierarchical clustering algorithm is used to complete the metadata classification of transaction metadata.

[0130] In this embodiment, a financial data analysis model is constructed by combining named entity recognition technology and graph neural network model, adopting BiLSTM-CRF-RCNN architecture. This model consists of three main parts: bidirectional long short-term memory network (BiLSTM), conditional random field (CRF) and self-attention convolutional neural network (RCNN). The BiLSTM layer is responsible for capturing the long-term dependencies of the input sequence, which contains two forward and backward LSTM networks that can consider past and future context information simultaneously. For the input sequence x = (x1,..., xT), the output of BiLSTM can be represented as:

[0131] h_t = [forward_LSTM(x_t), backward_LSTM(x_t)]

[0132] The CRF layer is used for sequence labeling of BiLSTM output, which considers the dependency between labels and improves the accuracy of prediction. The objective function of the CRF layer can be represented as:

[0133] logP(y|x) = Σ_t(Φ(y_t,x,t) + Ψ(y_t,y_{t-1})) - logZ(x)

[0134] where Φ is the emission function, Ψ is the transition function, and Z(x) is the normalization factor.

[0135] The RCNN part introduces the self-attention mechanism, which can capture long-distance dependencies in the sequence. The calculation formula of the self-attention mechanism is:

[0136] Attention(Q,K,V) = softmax(QK^T / √d_k)V

[0137] where Q, K, V are query, key and value matrices, and d_k is the dimension of the key.

[0138] The convolutional layer is used to extract local features, and its operation can be represented as:

[0139] C_i = f(W*x_{i:i+h-1} + b)

[0140] where W is the convolution kernel, h is the size of the convolution kernel, and f is the activation function.

[0141] This complex model architecture can effectively handle entity recognition and relationship extraction tasks in financial text data, capturing complex semantic information and long-range dependencies. It is particularly suitable for handling specialized terminology and complex expressions in the financial field, and can accurately identify and classify various financial entities such as company names, financial products, and transaction types. At the same time, the graph neural network part of the model can further utilize the relationship information between entities to improve the overall recognition and classification performance. This method not only improves the accuracy of financial data analysis, but also can handle large-scale and complex financial text data, providing strong support for subsequent risk analysis, market prediction, and other tasks.

[0142] According to the pre-created financial knowledge training set and using the remote supervision method to complete the labeling and training of the financial data analysis model. Remote supervision uses existing knowledge base to automatically generate labels for a large number of unlabeled texts, overcoming the time-consuming and labor-intensive problem of manual labeling. First, a knowledge base containing financial entities and relationships needs to be constructed, for example:

[0143] KB = {(entity1, relation, entity2),...}

[0144] For example, ("XX Company", "issue", "Company Bond").

[0145] Then, for each unlabeled text, if the text contains a pair of entities in the knowledge base, it is assumed that the text expresses the relationship between the pair of entities in the knowledge base. Formally represented as:

[0146] If (e1, r, e2) ∈ KB and e1, e2 ∈ sentence, then label(sentence) = r

[0147] However, this method may introduce noise. To reduce false labeling, a multi-instance learning strategy can be used. All sentences containing the same entity pair (e1, e2) are grouped into a bag (bag), and as long as at least one sentence in the bag correctly expresses the relationship r, the bag is considered a positive example of relationship r. This can be represented as:

[0148] P(r|bag) = max(P(r|sentence) for sentence in bag)

[0149] During training, the negative log-likelihood is used as the loss function:

[0150] L = -log P(r|bag)

[0151] To further improve the labeling quality, an attention mechanism can be introduced to assign different weights to the sentences in the bag:

[0152] P(r|bag) = ∑_i α_i P(r|sentence_i)

[0153] where α_i is the attention weight of sentence i, which can be learned by a neural network.

[0154] In addition, entity types and contextual information can also be used to filter unreasonable labels. For example, if the subject of the relationship "acquisition" should be "company", then the incorrect label of the subject as "individual" can be excluded. The advantage of this method is that it can quickly generate a large amount of labeled training data, significantly reducing the workload of manual labeling. Although the generated labels may contain some noise, the labeling quality can be greatly improved through the above strategies. This method is particularly suitable for the financial field because there is a lot of structured knowledge in the financial field that can be used as the basis for remote supervision. The model trained in this way can better understand the entities and relationships in financial text, laying the foundation for subsequent financial data analysis tasks.

[0155] The preprocessed financial transaction data is input into the trained financial data analysis model. The BiLSTM module processes the input sequence and captures long-distance dependencies. For the input sequence x = (x1,...,xT), the forward and backward hidden states of BiLSTM are respectively:

[0156] The final BiLSTM output is the concatenation of the forward and backward states: h_t = [h_t^f; h_t^b].

[0157] Next, the CRF layer uses the output of BiLSTM for sequence labeling. CRF considers the transition probabilities between labels and can capture the overall structure of the label sequence. The conditional probability of CRF can be represented as:

[0158] P(y|x) = exp(Σ_t(s(x,t,y_t)+t(y_{t-1},y_t)) / Z(x)

[0159] where s(x,t,y_t) is the emission score, t(y_{t-1},y_t) is the transition score, and Z(x) is the normalization factor.

[0160] In the decoding phase, the Viterbi algorithm is used to find the most likely label sequence: y* = argmax_y P(y|x).

[0161] In specific implementation, the BIOES labeling scheme (B-start, I-internal, O-external, E-end, S-single token entity) can be used to label financial entities. For example:

[0162] "XX / B-ORG company / E-ORG issued / O 10 billion / B-AMOUNT / E-AMOUNT company bonds / S-PRODUCT"

[0163] In this way, structured transaction metadata can be extracted from financial transaction data, providing a foundation for subsequent analysis and decision-making. These metadata may include transaction subjects, transaction objects, transaction amounts, transaction times, and other key information, which are crucial for understanding the nature and impact of financial transactions.

[0164] Next, the semantic association between transaction metadata is extracted through the convolutional network module (RCNN) containing self-attention mechanism in the financial data analysis model. The convolutional network module (RCNN) combines the local feature extraction capability of convolutional neural network (CNN) and the global dependency capture capability of self-attention mechanism, which can effectively extract the complex semantic relationship between transaction metadata. First, the self-attention mechanism calculates the correlation between metadata. Given the input sequence X = (x1,...,xn), the calculation process of self-attention is as follows: Q = XW_Q, K = XW_K, V = XW_V; Attention(Q,K,V) = softmax(QK^T / √d_k)V.

[0165] where W_Q, W_K, W_V are learnable weight matrices, and d_k is the dimension of the key vector. This step can capture the long-distance dependency relationship between metadata. Next, the convolutional layer is used to extract local features. For attention output A, the convolution operation can be represented as: C_i = f(W*A_{i:i+h-1}+b), where W is the convolution kernel, h is the size of the convolution kernel, and f is the activation function (such as ReLU). In order to capture features of different scales, multiple convolution kernels of different sizes are usually used, and then the most significant features are selected through the max-pooling operation: F_j = max(C_{1,j},C_{2,j},...,C_{n-h+1,j}). Finally, all features are concatenated to get the final representation: F = [F_1; F_2;...; F_m], where m is the number of convolution kernels used.

[0166] The advantages of the convolutional network module (RCNN) are as follows: the self-attention mechanism can capture global dependencies between metadata, which is suitable for processing long-distance semantic associations. The convolution operation can extract local context features, which helps to understand the semantics at the phrase level. Multi-scale convolution kernels and pooling operations enable the model to capture semantic information at different granularities. In this way, the model can extract complex semantic relationships such as "Company A acquires Company B", "Product X price rise affects Company Y stock price", etc. These relationships can be represented in a graph structure, where nodes are transaction metadata and edges are their semantic relationships. For example, the following semantic relationship graph can be obtained:

[0167] (XX company, issue, corporate bonds)

[0168] (Corporate bonds, amount, 10 billion)

[0169] (Bond issuance, impact, company stock price)

[0170] Finally, the hierarchical clustering algorithm is used to complete the metadata classification of transaction metadata. The hierarchical clustering algorithm constructs the hierarchical structure of data by gradually merging or splitting clusters, which is suitable for processing complex hierarchical relationships in financial transaction data. The basic steps of hierarchical clustering are as follows: treat each metadata as an independent cluster; calculate the distance between all clusters; merge the two clusters with the closest distance; repeat the above steps until the predetermined number of clusters is reached or the stopping condition is met.

[0171] Distance calculation is a key step, and multiple distance measurement methods can be used. For transaction metadata, cosine similarity can be considered: cos(x, y) = (x · y) / (||x|| ||y||), where x and y are vector representations of metadata.

[0172] There are also various options for calculating the distance between clusters, including single linkage, complete linkage, average linkage, etc. For financial transaction data, average linkage usually provides better results because it considers the average of all elements within the cluster. The results of hierarchical clustering can be represented by a tree diagram (Dendrogram), where each node of the tree represents a cluster, and the leaf nodes are individual data elements. The final classification result can be determined by setting a distance threshold or specifying the number of clusters. To improve the clustering efficiency of large-scale data, the BIRCH algorithm can be used. BIRCH compresses data by constructing a CF tree (Clustering Feature Tree), where each node contains clustering features (N, LS, SS): N represents the number of elements in the cluster, LS represents the linear sum, and SS represents the square sum Σx^2. The main steps of the BIRCH algorithm include: traversing the data, inserting each element into the nearest leaf node; merging similar leaf nodes to reduce the size of the tree; clustering the leaf nodes of the CF tree; and fine-tuning the clustering results. The advantage of this method is that its time complexity is O(n), making it suitable for processing large-scale transaction data. Through hierarchical clustering, a multi-level classification structure of transaction data can be obtained.

[0173] In one embodiment, the hierarchical clustering algorithm is used to complete the metadata classification of transaction data, including the following steps:

[0174] Calculate the Gaussian distribution of transaction data to obtain a high-dimensional similarity matrix of transaction data in high-dimensional space;

[0175] Initialize low-dimensional data points in a pre-set low-dimensional space, with the number of low-dimensional data points equal to the number of transaction data elements;

[0176] Calculate the Cauchy distribution of all low-dimensional data points to obtain a low-dimensional similarity matrix of low-dimensional data points in low-dimensional space;

[0177] Calculate the KL divergence based on the high-dimensional similarity matrix and the low-dimensional similarity matrix;

[0178] Use the gradient descent method to minimize the KL divergence and simultaneously optimize and update the position of low-dimensional data points in low-dimensional space until the KL divergence reaches a minimum value, completing the process of reducing transaction data to low-dimensional space;

[0179] Calculate the minimum intra-class variance of transaction data in low-dimensional space, and complete the metadata classification of transaction data in low-dimensional space based on the minimum intra-class variance.

[0180] In this embodiment, the Gaussian distribution of transaction metadata is calculated, resulting in a high-dimensional similarity matrix of transaction metadata in high-dimensional space is the first step of the data dimensionality reduction process. This step aims to capture the similarity relationship in the original high-dimensional data. For each pair of transaction metadata xi and xj, the high-dimensional similarity pij between them is calculated. The probability density function of the Gaussian distribution is used to define this similarity: pij = (exp(-||xi-xj||^2 / (2σi^2))) / (∑k≠i exp(-||xi-xk||^2 / (2σi^2))). Here, σi is adaptively selected according to the perplexity parameter. Perplexity is usually set between 5 and 50, which controls the variance of the Gaussian distribution, thus affecting the calculation of similarity. Higher perplexity will lead to considering more distant neighbors, while lower perplexity will focus more on the neighbors. In this way, an N × N high-dimensional similarity matrix P can be obtained, where N is the number of transaction metadata. This matrix P captures the relative distance relationship between data points in the original high-dimensional space, providing a basis for the subsequent dimensionality reduction process.

[0181] Initializing low-dimensional data points in a pre-set low-dimensional space, the number of low-dimensional data points is the same as the number of transaction metadata, is the preparation stage of the dimensionality reduction process. The purpose of this step is to create an initial low-dimensional representation for each high-dimensional transaction metadata. Usually, the dimension of the low-dimensional space is selected as 2 or 3 to facilitate visualization. The initialization method has a significant impact on the final result, and common methods include random initialization and principal component analysis (PCA) initialization. Random initialization simply samples from a standard normal distribution: yi ~ N(0, 10^-4*I), where yi is the initial representation of the i-th transaction metadata in the low-dimensional space. PCA initialization initializes the low-dimensional points using the principal components of the data, which can speed up the convergence: yi = U[:d]xi, where U[:d] is the first d principal eigenvectors of the original data covariance matrix. After initialization, an N × d matrix Y is obtained, where N is the number of data points and d is the dimension of the target low-dimensional space.

[0182] Computing the cosine similarity between all pairs of low-dimensional data points to obtain a low-dimensional similarity matrix in the low-dimensional space is one of the core steps of the t-SNE algorithm. The cosine similarity (or t-distribution with 1 degree of freedom) is used to compute the similarity between data points in the low-dimensional space. For any two points yi and yj in the low-dimensional space, their similarity qij is defined as: qij = (1 + ||yi - yj||^2)^-1 / Σk≠l (1 + ||yk - yl||^2)^-1. The reason for using the cosine similarity instead of the Gaussian similarity is that the cosine similarity has longer tails, which helps to better separate data clusters in the low-dimensional space and reduce the "crowding problem". By computing qij for all pairs of low-dimensional points, a N x N low-dimensional similarity matrix Q can be obtained. This matrix Q reflects the relative positional relationships between data points after dimensionality reduction and is the key to the subsequent optimization process. The use of cosine similarity makes t-SNE particularly good at preserving local structures in the original data while creating more obvious cluster separation in the low-dimensional space.

[0183] Computing the KL divergence based on the high-dimensional similarity matrix and the low-dimensional similarity matrix is a key step in evaluating the quality of dimensionality reduction. The KL divergence (Kullback-Leibler divergence) is used to measure the difference between two probability distributions, and here it is used to measure the similarity of data point distributions in high-dimensional and low-dimensional spaces. The formula for computing the KL divergence is: C = KL(P||Q) = Σi≠j pij log(pij / qij). Here, pij and qij are the similarities between points i and j in the high-dimensional and low-dimensional spaces, respectively. The smaller the KL divergence, the better the low-dimensional representation preserves the structure of the original data. It is worth noting that the KL divergence is not symmetric, and using P||Q instead of Q||P can better preserve the local structure because it places more emphasis on points that are close in the high-dimensional space. The gradient of this objective function with respect to the low-dimensional point yi is computed as: This gradient information guides the subsequent optimization process, helping to adjust the positions of the low-dimensional points to minimize the KL divergence.

[0184] The gradient descent method is used to minimize the KL divergence and simultaneously optimize the update of the positions of the low-dimensional data points in the low-dimensional space until the KL divergence reaches a minimum value, completing the process of reducing transaction metadata to a low-dimensional space. This is the core optimization step of the t-SNE algorithm. In specific implementation, the stochastic gradient descent method with momentum is usually used. In each iteration, the position of the low-dimensional point is updated according to the following formula: Here, η is the learning rate, and α(t) is the momentum coefficient that varies over time. The learning rate usually starts from a larger value (e.g., 100) and gradually decreases during the iterations. The momentum term helps to accelerate convergence and avoid local minima. The optimization process usually runs for several hundred to several thousand iterations until the change in KL divergence is less than a certain threshold or the maximum number of iterations is reached. To improve efficiency, the Barnes-Hut algorithm can be used to approximate the gradient, reducing the time complexity from O(N^2) to O(N log N). The result of this optimization process is the final position of each transaction metadata in the low-dimensional space that preserves the key structural features of the original high-dimensional data.

[0185] The minimization of the within-class variance of the transaction metadata in the low-dimensional space is a step in the clustering analysis after dimensionality reduction. This step aims to group the data points after dimensionality reduction to discover potential data structures and patterns. The method of minimizing the within-class variance usually uses the K-means algorithm. First, choose an appropriate number of clusters K, which can be determined by the elbow rule or silhouette analysis. Then, the K-means algorithm iteratively performs the following steps:

[0186] (1) Randomly initialize K cluster centers; (2) assign each data point to the nearest cluster center; (3) update the center of each cluster to be the average position of all points in that cluster. The objective function is to minimize the total within-class variance: J = Σk=1^K Σx∈Ck ||x-μk||^2, where Ck is the kth cluster and μk is the center of the kth cluster. The algorithm iterates until the cluster assignment no longer changes or reaches the maximum number of iterations. To improve the stability of the results, K-means is usually run multiple times and the best result is selected. The advantage of this method is that it can further reveal the internal structure of the data based on the t-SNE dimensionality reduction results, helping to identify similar trading patterns or behaviors. The final classification results can be used for financial analysis tasks such as anomaly detection and risk assessment.

[0187] In one embodiment, the graph temporal features and graph spatial features are extracted from the transaction data genealogy graph, the graph temporal features and graph spatial features are combined, and the node interaction index of all nodes in the transaction data genealogy graph is predicted by a pre-configured data interaction prediction model, including the following steps:

[0188] The graph temporal features and graph spatial features are extracted from the transaction data genealogy graph, the graph temporal features include the time feature representation of all graph nodes, the graph spatial features include the node spatial topology feature representation of all graph nodes, and the node spatial topology feature representation includes the node PageRank value, node degree, node betweenness centrality, and central node connectivity.

[0189] The atlas temporal feature and the atlas spatial feature are fused to obtain an atlas spatiotemporal fusion feature;

[0190] The transaction data bloodline atlas and the atlas spatiotemporal fusion feature are input into a pre-configured data interaction prediction model, and node interaction features between all nodes in the atlas are extracted from the atlas spatiotemporal fusion feature through multiple gate cycle modules in the data interaction prediction model;

[0191] Context information of the transaction data bloodline atlas is captured through a graph convolutional neural network module in the data interaction prediction model, and the node interaction features of all nodes are spliced into an atlas comprehensive representation vector according to the context information;

[0192] Based on the atlas comprehensive representation vector, a node interaction index of all nodes in the transaction data bloodline atlas is predicted through a multilayer perceptron module in the data interaction prediction model.

[0193] In this embodiment, extracting the atlas temporal feature and the spatial feature from the transaction data bloodline atlas is a key step to understand the dynamic evolution and structure of the atlas. The temporal feature captures the pattern of node changes over time, which can be achieved through time embedding methods, such as converting timestamps into periodic features: sin(2πt / T) and cos(2πt / T), where t is the timestamp and T is the period (such as a day or a year). The spatial feature reflects the importance and structural role of the node in the graph. The PageRank value calculation method is PR(u) = (1-d) + d*∑v∈N(u)PR(v) / OutDegree(v), where d is the damping factor, usually 0.85. The node degree is the number of edges directly connected to the node. The betweenness centrality represents the degree of the node as a "bridge" and is calculated as BC(v) = ∑s≠v≠t(σst(v) / σst), where σst is the number of shortest paths from s to t, and σst(v) is the number of shortest paths passing through v. The closeness centrality measures the close degree of the node to the center of the graph, which can be obtained by calculating the average shortest path length to all other nodes. These features together constitute a comprehensive representation of the atlas, providing a rich information foundation for subsequent analysis.

[0194] Fusion of the temporal and spatial features of the graph atlas is a key step in integrating multi-dimensional information. This process aims to create a comprehensive representation that captures both the temporal dynamics and spatial structure of the graph atlas. Feature fusion can be achieved through various methods, such as simple concatenation, weighted sum, or more complex neural network approaches. One effective method is to use an attention mechanism for feature fusion: F = ∑iαi*Fi, where Fi is the i-th feature vector (which can be either temporal or spatial), and αi is the corresponding attention weight, calculated by softmax(Wi*Fi+bi). Wi and bi are learnable parameters. This method allows the model to dynamically adjust the importance of different features. Another method is to use a gating mechanism: F = z⊙Ft + (1-z)⊙Fs, where Ft is the temporal feature, Fs is the spatial feature, and z is the gating vector, calculated by sigmoid(W[Ft;Fs]+b). ⊙ represents element-wise multiplication. This method enables effective fusion of temporal and spatial features while preserving the original feature information. The fused feature F provides a comprehensive representation for each node, incorporating both its temporal evolution characteristics and structural characteristics in the graph.

[0195] Inputting the transaction data blood atlas and the spatiotemporal fusion features of the graph atlas into the pre-configured data interaction prediction model, and extracting node interaction features through multiple gating cycle modules is a key step in capturing complex temporal dependencies. This process uses variants of recurrent neural networks (RNN), such as long short-term memory networks (LSTM) or gated recurrent units (GRU), to process temporal data. Taking GRU as an example, its update process is as follows:

[0196] resetgate: rt = σ(Wr[ht-1, xt] + br)

[0197] updategate: zt = σ(Wz[ht-1, xt] + bz)

[0198] candidatestate: ht = tanh(Wh[rt⊙ht-1, xt] + bh)

[0199] hiddenstate: ht = (1-zt)⊙ht-1 + zt⊙ht

[0200] Here, xt is the input at time t (the spatiotemporal fusion feature of the graph), ht is the hidden state, σ is the sigmoid function, and ⊙ represents element-wise multiplication. Through this mechanism, the model can learn long-term dependencies and capture complex interaction patterns between nodes. To handle graph-structured data, a GRU unit can be configured for each node, and the current node's state can be updated at each time step based on the states of its neighbors. This approach allows information to propagate through the graph, thereby capturing both direct and indirect interactions between nodes. Stacking multiple GRU layers can further enhance the model's ability to capture complex interaction patterns. The output of this step is the interaction feature vector for each node, which integrates the node's historical information and the influence of its neighbors.

[0201] Capturing the contextual information of the pedigree graph of transaction data through the Graph Convolutional Neural Network (GCN) module in the data interaction prediction model, and concatenating the node interaction features of all graph nodes into a graph comprehensive representation vector, is a key step in integrating local and global information. The core operation of GCN can be represented as: in It is an adjacency matrix with self-loops added and normalized, H(l) is the node feature matrix of the l-th layer, W(l) is the learnable weight matrix, and σ is a non-linear activation function (such as ReLU). This operation achieves local aggregation of node features, enabling each node to comprehensively consider the information of its neighbors. Stacking multiple layers of GCN allows information to spread over a wider range, thus capturing a broader context. To enhance the expressive power of the model, variations of the attention mechanism can be used, such as the Figure Attention Network (GAT).

[0202] eij=a(Whi,Whj); αij=softmaxi(eij); hi'=σ(ΣjαijWhj)

[0203] Here, 'a' is the attention function, typically a single-layer feedforward neural network, and αij is the attention coefficient. This mechanism allows the model to dynamically assign different importance to different neighbors. After capturing contextual information, the features of all nodes are concatenated to form a graph comprehensive representation vector: G = [h1'; h2'; ...; hN'], where hi' is the feature vector of node i after GCN processing, and N is the total number of nodes. This comprehensive representation vector contains global structural information of the graph and local features of each node, providing rich input for subsequent prediction tasks. Through this method, the model can simultaneously consider the individual characteristics of nodes and their position and role in the entire transaction network, thereby more accurately predicting the interaction behavior between nodes.

[0204] Based on the graph comprehensive representation vector, predicting the node interaction index of all graph nodes in the transaction data bloodline graph through the multilayer perception (MLP) module in the data interaction prediction model is a key step to realize the final prediction. The MLP is composed of multiple fully connected layers, and the calculation of each layer can be represented as: Y = σ (WX + b), where X is the input vector, W is the weight matrix, b is the bias vector, and σ is a nonlinear activation function (such as ReLU or tanh). For the interaction index prediction task, the last layer usually uses a sigmoid function to compress the output to between 0 and 1: where is the predicted interaction index, and hL is the output of the last hidden layer. The number of layers and the number of neurons in each layer of the MLP need to be adjusted according to the complexity of the task and the size of the data. In order to improve the generalization ability of the model, a dropout layer can be added between each layer: hdrop = dropout (h, p), where p is the dropout probability.

[0205] The backpropagation algorithm is used to minimize the loss function, such as binary cross-entropy loss, during the training process: where y is the true label. In order to deal with the class imbalance problem, a weighted loss function or oversampling technique can be used. In addition, regularization techniques such as L2 regularization can be used to prevent overfitting: Lreg = L + λΣ||Wi||2, where λ is the regularization coefficient. Through this MLP module, the model can learn complex nonlinear mappings to convert the comprehensive representation of the graph into a prediction of the interaction index between nodes. This prediction result can be used to identify potential high-risk transactions, predict future transaction patterns, or evaluate the importance of nodes in the transaction network. By adjusting the threshold, the continuous interaction index can be converted into a binary interaction prediction.

[0206] In one of the embodiments, a plurality of high-heat graph nodes of different entity types are taken as source graph nodes, high-dimensional transaction correlation features are extracted from the transaction data bloodline graph based on a graph neural network model, and abnormal transaction data in the financial transaction data is identified according to the high-dimensional transaction correlation features, including the following steps:

[0207] Statistically, the first node quantity of the graph nodes in the transaction data bloodline graph and the second node quantity of the high-heat graph nodes are counted;

[0208] According to the first node quantity and the second node quantity, the graph transformation degree is determined, and the graph nodes edges in the transaction data bloodline graph are randomly generated or removed according to the graph transformation degree to obtain a transaction data transformed graph;

[0209] Different entity types in the transaction data bloodline graph are used as source graph nodes, and the corresponding nodes in the transaction data transformation graph are used as target graph nodes.

[0210] The source graph nodes and the target graph nodes are combined, and a double graph neural network model is used to extract high-dimensional transaction correlation features of the transaction data bloodline graph.

[0211] Based on the high-dimensional transaction correlation features, an anomaly detection algorithm is used to identify abnormal transaction data in the financial transaction data.

[0212] In this embodiment, the first number of nodes and the second number of nodes of the high-heat graph nodes in the transaction data bloodline graph are key steps to understand the graph structure. The first node refers to the node directly connected to the center node, and its number reflects the direct connection complexity of the graph. The high-heat graph node refers to the node whose transaction frequency or transaction volume is significantly higher than the average level, and the Z-score method is usually used to identify it: Z=(x-μ) / σ, where x is the transaction volume of the node, μ is the average transaction volume, and σ is the standard deviation. Nodes with a Z-score greater than 2 are selected as high-heat nodes. The second node refers to the node directly connected to the high-heat node, and its number reflects the influence range of the high-activity area. In specific implementation, depth-first search (DFS) or breadth-first search (BFS) algorithm can be used to traverse the graph and count the number of nodes. For example, using BFS: start from the center node, add it to the queue Q, and then repeat the following steps until Q is empty: (1) take a node v from Q; (2) count the number of unvisited neighbors of v; (3) add the unvisited neighbors of v to Q. The time complexity of this process is O(V+E), where V is the number of nodes and E is the number of edges.

[0213] Determining the graph transformation degree according to the first node quantity and the second node quantity, and randomly generating or removing the graph node edges in the transaction data bloodline graph to obtain a transaction data transformed graph is an important step to simulate the dynamic changes of the network. The graph transformation degree can be calculated by the following formula: T = a * (N1 / N) + b * (N2 / N), wherein N1 is the first node quantity, N2 is the second node quantity of the high-heat nodes, N is the total node quantity, and a and b are weight coefficients. The greater the value of T is, the more complex the graph structure is, and more transformation is needed. Based on the value of T, the proportion of edge addition or deletion P = f(T) can be determined, wherein f can be a nonlinear function, such as sigmoid: P = 1 / (1+e^(-kT)), and k is an adjustment parameter. In actual transformation, for each edge, a random number r between 0 and 1 is generated. If r < P, a transformation operation is performed: for edge addition, two nodes not directly connected are randomly selected, and an edge is added; for edge deletion, an existing edge is randomly selected and deleted. In order to maintain the connectivity of the graph, the Union-Find algorithm can be used to ensure that the deletion operation does not cause the graph to split. This transformation method simulates the dynamic characteristics of the real transaction network, including the establishment of new relationships and the disappearance of old relationships. The transformed graph retains the basic structural characteristics of the original graph while introducing randomness.

[0214] Taking a plurality of high-heat graph nodes of different entity types in the transaction data bloodline graph as source graph nodes and taking corresponding nodes of the source graph nodes in the transaction data transformed graph as target graph nodes is a key step for constructing an input of a double-graph neural network. First, the selection criteria for high-heat nodes need to be determined, and the PageRank algorithm or the degree centrality (Degree Centrality) can be used to measure the importance of nodes. The calculation formula of PageRank is: PR(u) = (1-d) + d * Σv∈N(u) PR(v) / |N(v)|, wherein d is a damping factor (usually 0.85), and N(u) is the neighbor set of node u. The top-k% nodes with the highest PageRank values are selected as high-heat nodes. In order to ensure balanced representation of different entity types, the top-m high-heat nodes can be selected in each entity type. The correspondence between the source graph nodes and the target graph nodes is established through node ID or feature matching. If a source node does not exist in the transformed graph, the most similar node can be selected as a substitute, and the similarity can be calculated by the cosine similarity of node features: sim(u, v) = (fu·fv) / (||fu||*||fv||), wherein fu and fv are the feature vectors of nodes u and v.

[0215] Extracting high-dimensional transaction association features from the kinship graph of transaction data by combining source and target graph nodes and utilizing a dual-graph neural network model is a complex feature learning process. The dual-graph neural network model simultaneously processes the original and transformed graphs, capturing structural differences and semantic variations between them. The core of the model is the graph convolution operation, which can be represented as: H^(l+1)= in Here, H^(l) is the normalized adjacency matrix, H^(l) is the node feature matrix of the l-th layer, W^(l) is the learnable weight matrix, and σ is the non-linear activation function. To capture relationships between graphs, an attention mechanism can be used: αij = softmax(a(Whi,Whj)), where a is the attention function, and hi and hj are the features of corresponding nodes in the two graphs. Then, information from the two graphs is aggregated: h'i = σ(W[hi||Σjαijhj]), where || denotes the concatenation operation. Multiple layers of such operations can capture more complex structural relationships. Finally, a graph-level representation is obtained through global pooling: hG = READOUT({hi|i∈G}), where READOUT can be a summation, averaging, or a more complex learnable function. The output of this process is a high-dimensional vector encoding the structural features and semantic information of the original and transformed graphs. Through this method, the model can learn key differences before and after the graph transformation, which may indicate potential anomalous patterns.

[0216] Identifying anomalous transaction data in financial transaction data based on high-dimensional transaction correlation features and using anomaly detection algorithms is a crucial step in achieving transaction risk monitoring. Commonly used anomaly detection algorithms include Isolation Forest, Local Outlier Factor (LOF), and One-Class SVM. Taking Isolation Forest as an example, its basic principle is that outliers are more easily isolated. The algorithm randomly selects features and splits the data at random values ​​of those features, repeating this process until each point is isolated. Outliers typically require fewer splitting steps. The anomaly score is calculated as: s(x,n)=2^(-E(h(x)) / c(n)), where h(x) is the average path length of point x, and c(n) is the average path length when the dataset size is n. For the LOF algorithm, the ratio of the local density of each point to the local density of its k nearest neighbors is calculated; the LOF value of anomalies is significantly greater than 1. One-Class SVM attempts to find a hyperplane that encloses most data points within a compact region; points falling outside this region are considered anomalies.

[0217] In one implementation, the process of extracting high-dimensional transaction association features from the transaction data lineage graph by combining source graph nodes and target graph nodes and utilizing a dual-graph neural network model includes the following steps:

[0218] The source node features of the source graph nodes and the target node features of the target graph nodes are simultaneously extracted by using a dual-graph neural network model;

[0219] The feature association relationship between the source node features and the target node features is constructed through a cross-attention mechanism in the dual-graph neural network.

[0220] The source node features and the target node features are mapped into the same high-dimensional feature space, and the source node features and the target node features are aggregated in the high-dimensional feature space based on the feature association relationship, to obtain high-dimensional transaction association features of the transaction data bloodline graph.

[0221] In the embodiment, the simultaneous extraction of the source node features of the source graph nodes and the target node features of the target graph nodes by using the dual-graph neural network model is a process of parallel processing of two graph structures. The dual-graph neural network model includes two parallel graph neural network subnetworks, which process the source graph and the target graph, respectively. Each subnetwork uses a multi-layer graph convolution operation to extract node features. Taking a graph convolution network (GCN) as an example, the core operation can be represented as: wherein is a normalized adjacency matrix, H^(l) is a node feature matrix of the lth layer, W^(l) is a learnable weight matrix, and σ is a nonlinear activation function (such as ReLU). In order to capture more complex graph structure information, more advanced graph neural network variants can be used, such as a graph attention network (GAT). GAT introduces an attention mechanism, allowing nodes to assign different importance to different neighbors: eij=a(Whi,Whj), αij=softmax(eij), h'i=σ(ΣjαijWhj), where a is an attention function, usually a small neural network. By stacking multiple layers of such operations, the model can gradually expand the receptive field and capture more extensive structure information. The output of the last layer is the required node feature. In order to maintain the correspondence of the source graph and the target graph features, the two subnetworks share the same network structure and parameters. This parallel processing method not only improves the computational efficiency, but also ensures that the source node features and the target node features are in the same feature space, laying a foundation for subsequent feature association and aggregation.

[0222] Building the feature association between source node features and target node features through cross-attention mechanism in the dual-graph neural network is a key information fusion step. Cross-attention mechanism allows the model to establish dynamic and adaptive associations between the source graph and the target graph. The core idea is to let each source node feature interact with all target node features, and vice versa. The specific implementation can be done through the following steps: First, take the source node features and target node features as query (Q), key (K) and value (V) respectively: Qs = WqHs, Kt = WkHt, Vt = WvHt, where Hs and Ht are the source and target node feature matrices respectively, and Wq, Wk, Wv are learnable weight matrices. Then, calculate the attention score: A = softmax(QsKt^T / √d), where d is the feature dimension. Finally, aggregate information: H's = AVt. This process can be done bidirectionally, i.e. also calculate the target-to-source attention. The output of cross-attention can be represented as: H's = f([Hs||AVt]), H't = f([Ht||A^TVs]), where f is a feed-forward neural network, and || represents the concatenation operation. To capture multi-scale association relationships, a multi-head attention mechanism can be used, i.e. calculate multiple independent attention groups in parallel and then concatenate the results:

[0223] MultiHead(Q,K,V) = Concat(head1,...,headh)Wo

[0224] where headi = Attention(QWi^Q,KWi^K,VWi^V). Through this mechanism, the model can automatically learn the complex correspondence between the source graph and the target graph, capture the structural changes and semantic evolution before and after the graph transformation. This step can establish dynamic associations between the nodes of the source graph and the target graph, capture the structural differences and semantic changes between the graphs, and provide important association information for subsequent feature aggregation.

[0225] The final step in feature fusion is to map source and target node features to the same high-dimensional feature space and then aggregate these features in that high-dimensional space based on feature relationships to obtain high-dimensional transaction association features of the transaction data lineage map. First, a nonlinear mapping function is used to project source and target node features into a common high-dimensional space: Fs = φ(Hs), Ft = φ(Ht), where φ can be a multilayer perceptron (MLP) or other complex nonlinear function. To ensure feature alignment, an alignment loss can be introduced: Lalign = ||Fs - Ft||F, where ||·||F represents the Frobenius norm. Next, feature aggregation is performed based on the feature relationships obtained in the previous step. An effective aggregation method is to use a gating mechanism: G = sigmoid(Wg[Fs||Ft]), F = G⊙Fs + (1-G)⊙Ft, where Wg is a learnable weight matrix, and ⊙ represents element-wise multiplication. This mechanism allows the model to dynamically determine how much information to retain from source and target features.

[0226] To capture more complex interactions, multi-layer aggregation can be used: F(l+1)=MLP(F(l)||(F(l)sA(l)+F(l)t(A(l))^T)), where A(l) is the attention matrix of the l-th layer. Finally, a graph-level representation is obtained through a global pooling operation: Fglobal=READOUT({Fi|i∈G}), where READOUT can be a summation, averaging, or a more complex learnable function. This final high-dimensional transaction association feature Fglobal encodes the structural and semantic information of the source and target graphs, as well as the relationships between them. This step can generate a high-dimensional feature representation containing rich information, capture key differences before and after graph transformation, and provide high-quality feature input for subsequent anomaly detection or other downstream tasks.

[0227] In one implementation, using anomaly graph nodes as the starting point for tracing, the method of tracing data lineage to locate the originating anomaly graph node corresponding to the abnormal transaction data in the transaction data lineage graph includes the following steps:

[0228] For each anomalous graph node, calculate the node anomalous score based on the high-dimensional transaction association features;

[0229] Using abnormal graph nodes as the starting point for tracing, nodes in the transaction data lineage graph are traced according to their interaction relationships.

[0230] During the node tracing process, the abnormal impact score of the traced graph node is calculated based on the node abnormality score and an iterative algorithm, and multiple maximum impact predecessor nodes in the traced graph node are determined according to the abnormal impact score.

[0231] all the maximum impact predecessor nodes as the tracking endpoints, forming multiple abnormal tracing paths between the tracking starting point and the multiple tracking endpoints;

[0232] The path abnormal score of each abnormal tracing path is calculated in combination with the abnormal score and the abnormal impact score, the abnormal tracing path with the maximum path abnormal score is taken as a target abnormal tracing path, and the maximum impact predecessor node in the target abnormal tracing path is taken as an origin abnormal graph node of an abnormal transaction data origin.

[0233] In the embodiment, for each abnormal graph node, calculating the node abnormal score of the abnormal graph node according to the high-dimensional transaction correlation feature is a key step of quantifying the abnormal degree. This process uses the high-dimensional transaction correlation feature extracted in the previous step to assign an abnormal score to each node through an anomaly detection algorithm. An effective method is to use a distance-based anomaly detection algorithm such as Local Outlier Factor (LOF). The core idea of LOF is to compare the local density of a point with its neighbors. The specific calculation steps are as follows: first, for each node i, calculate the average distance of its k-neighbors:

[0234] lrd(i) = 1 / (∑j∈Nk(i)reach-distk(i,j) / k)

[0235] where reach-distk(i,j) = max{k-distance(j), d(i,j)}, d(i,j) is the distance between nodes i and j. Then, the LOF value is calculated: LOF(i) = ∑j∈Nk(i)(lrd(j) / lrd(i)) / k. Points with LOF values significantly greater than 1 are considered abnormal. To normalize the LOF value to the [0, 1] interval, a sigmoid function can be used: score(i) = 1 / (1 + exp(-a(LOF(i)-b))), where a and b are adjustable parameters. Another method is to use Isolation Forest, which is based on the principle that abnormal points are more easily isolated. The abnormal score is calculated as: s(x, n) = 2^(-E(h(x)) / c(n)), where h(x) is the average path length of point x, and c(n) is the average path length when the data set size is n. These methods can effectively identify nodes that are abnormal in high-dimensional feature space. The calculated abnormal score not only quantifies the abnormal degree of each node, but also provides important starting point information for subsequent abnormal tracing. This step assigns a quantified abnormal score to each graph node, identifies the most likely abnormal transaction node, and provides priority guidance for abnormal tracing.

[0236] Tracing back from the abnormal graph nodes as the starting point in the transaction data bloodline graph according to the interaction relationship is a key step of backtracking analysis. This process aims to find the source nodes or influence paths that may cause abnormalities. Node tracing uses graph traversal algorithms such as Depth-First Search (DFS) or Breadth-First Search (BFS) to trace back from the abnormal nodes along the interaction relationship. DFS is suitable for finding long paths, while BFS is more suitable for finding short-distance influences. Taking BFS as an example, the algorithm steps are as follows: (1) initialize a queue Q and add the abnormal node to Q; (2) when Q is not empty, perform the following operations: a. take a node v from Q; b. check all predecessor nodes of v; c. add unvisited predecessor nodes to Q and mark them as visited; (3) repeat step 2 until Q is empty. To avoid loops and excessive tracing, a maximum tracing depth D can be set. In actual implementation, an adjacency list or adjacency matrix can be used to represent the graph structure to improve lookup efficiency. In addition, heuristic rules can be added to optimize the tracing process, such as prioritizing edges with larger transaction amounts or higher transaction frequencies. During the tracing process, record the depth and path information of each visited node, which is crucial for subsequent abnormal influence analysis. To handle large-scale graphs, consider using parallel computing techniques and starting tracing from multiple abnormal nodes. This tracing method can effectively construct potential abnormal propagation networks and provide a foundation for subsequent abnormal influence analysis. This step constructs the tracing path from abnormal nodes to potential sources, identifies related nodes that may participate in abnormal activities, and provides graph structure support for subsequent abnormal influence evaluation.

[0237] In the node tracing process, based on the node abnormal score and using an iterative algorithm to calculate the abnormal influence score of the traced graph nodes, and determining the multiple maximum influence predecessor nodes in the traced graph nodes according to the abnormal influence score is a key step of quantifying abnormal propagation influence. The abnormal influence score reflects the influence degree of a node on downstream abnormal nodes. The calculation process uses an iterative algorithm similar to the idea of PageRank. Initialization sets the abnormal influence score of each node to its node abnormal score. Then, multiple rounds of iteration are updated:

[0238] IS(v) = a * AS(v) + (1-a) * Σu∈N_out(v) (IS(u) * w(v,u) / Σx∈N_in(u) w(x,u))

[0239] where IS(v) is the abnormal influence score of node v, AS(v) is the node abnormal score, N_out(v) is the out-neighborhood set of v, w(v, u) is the weight of edge (v, u) (which can be transaction amount or frequency), a is the balance parameter (usually between 0.1 and 0.3). Iteration until the score converges or reaches the maximum number of iterations. To improve computational efficiency, matrix operations can be used: IS = a * AS + (1 - a) * W * IS, where W is the normalized adjacency matrix. After each iteration, the scores are normalized to prevent numerical overflow. After the calculation is complete, the direct predecessor nodes of each node are sorted by abnormal influence score, and the top k are selected as the maximum influence predecessor nodes. The choice of k can be a fixed value or dynamically determined based on a score threshold. This method not only considers the abnormality of the node itself, but also considers its influence in the entire network, and can effectively identify those entities that may be the source of abnormality or key propagation nodes.

[0240] All maximum influence predecessor nodes are used as tracking endpoints, and multiple abnormal tracing paths between the tracking starting point and multiple tracking endpoints are formed, which is a key step in building a complete abnormal propagation network. This process involves path search and path optimization. First, use backtracking algorithms or dynamic programming to find all paths from the tracking starting point (abnormal node) to each tracking endpoint (maximum influence predecessor node). For each pair of starting point s and endpoint t, a modified version of the depth-first search (DFS) algorithm can be used: (1) start from s and traverse upwards along the edges; (2) record all nodes in the current path; (3) stop when reaching t or reaching the maximum depth; (4) if t is reached, save the current path; (5) backtrack and explore other possible paths. To improve efficiency, pruning techniques can be used, such as stopping the search when the path length exceeds a pre-set threshold. In practical applications, there may be a large number of paths, so path screening and optimization are needed. One method is to use the cumulative abnormal influence score of the path to sort: PathScore = Σv∈pathIS(v) * d^(-λ), where IS(v) is the abnormal influence score of node v, d is the depth of the node in the path, and λ is the decay factor. This calculation method takes into account the influence of all nodes on the path, while reducing the weight of distant nodes through the depth decay factor. Another optimization method is to consider the diversity of the path, using the maximum edge disjoint path algorithm to select a set of key paths that do not overlap with each other.

[0241] The path anomaly score of each abnormal trace path is calculated by combining the abnormal score and the abnormal influence score, the abnormal trace path with the largest path anomaly score is taken as the target abnormal trace path, and the maximum influence predecessor node in the target abnormal trace path is taken as the origin abnormal graph node of the abnormal transaction data origin. The calculation of the path anomaly score needs to comprehensively consider the abnormal degree and influence of all nodes on the path. An effective calculation method is: PathScore =∑v∈path(β*AS(v)+(1-β)*IS(v))*γ^d(v), wherein AS(v) is the abnormal score of the node v, IS(v) is the abnormal influence score thereof, d(v) is the depth of the node v in the path, β is a weight for balancing the abnormal score and the influence score (usually between 0.4 and 0.6), and γ is a depth attenuation factor (usually less than 1). This calculation method not only considers the abnormal degree and influence of the node, but also reflects the attenuation effect of abnormal propagation through depth attenuation.

[0242] In order to normalize the scores of paths with different lengths, the PathScore can be divided by the path length. After calculating the abnormal scores of all paths, the path with the highest score is selected as the target abnormal trace path. On this path, the predecessor node with the highest abnormal influence score is identified as the abnormal origin node. If there are multiple nodes with scores very close to each other, they can all be listed as potential origin nodes. In order to increase the reliability of the results, an integrated method can be used, such as considering the top-k paths and determining the final abnormal origin based on their common characteristics. This step can quantify and compare the importance of different abnormal propagation paths, identify the most likely abnormal origin node, and provide key information for the root analysis and risk management of abnormal transactions. This method not only finds the source of the abnormality, but also provides detailed paths of how the abnormality propagates in the transaction network, which is of great significance for understanding and preventing financial risks.

[0243] The application further discloses a field data governance system based on an artificial intelligence platform, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the field data governance method based on the artificial intelligence platform as described in any one of the above embodiments when executing the computer program.

[0244] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, and the application does not limit this.

[0245] The memory can be an internal storage unit of the computer device, for example, a hard disk or a memory of the computer device, or an external storage device of the computer device, for example, a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD), or a flash card (FC) equipped on the computer device, or a combination of the internal storage unit and the external storage device of the computer device.

[0246] The application further discloses a computer readable storage medium, which stores instructions, and the instructions enable a processor to be configured to perform the method for domain data governance based on an artificial intelligence platform when the instructions are executed by the processor.

[0247] The computer program can be stored in a machine readable medium, and the computer program includes computer program code, which can be in a form of source code, object code, an executable file, or some intermediate form, etc. The machine readable medium includes any entity or device, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier wave signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the machine readable medium includes but is not limited to the above-mentioned elements.

[0248] The computer readable storage medium stores the power transmission line comprehensive fault detection method in the above embodiment, and is loaded and executed on the processor, so as to facilitate storage and application of the method.

[0249] It should be understood by those skilled in the art that the above discussion of any embodiment is only exemplary and is not intended to limit the protection scope of the application to these examples; under the idea of the application, the above embodiments or technical features in different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of one or more embodiments of the application as described above. In order to be brief, they are not provided in details.

[0250] One or more embodiments of the application are intended to cover all such alternatives, modifications and variations falling within the broad scope of the application. Therefore, any omission, modification, equivalent replacement, improvement, etc. made in the spirit and principle of one or more embodiments of the application should be included in the protection scope of the application.

Claims

1. A domain data governance method based on an artificial intelligence platform, characterized in that, Includes the following steps: Obtain financial transaction data through a data warehouse; The financial transaction data is preprocessed, including data cleaning and data repair. Natural language processing technology is used to extract transaction metadata and semantic relationships between the preprocessed financial transaction data, and a hierarchical clustering algorithm is used to classify the transaction metadata. By combining the semantic associations and the classified transaction metadata, a transaction data lineage graph of the financial transaction data is constructed. The graph nodes in the transaction data lineage graph represent transaction metadata describing different types of financial transaction entities, and the graph node edges between the graph nodes represent the interaction relationships between different transaction metadata. Extract temporal and spatial features of the kinship map from the transaction data kinship map; The temporal features and spatial features of the map are fused to obtain the spatiotemporal fusion features of the map. The transaction data lineage map and the spatiotemporal fusion features of the map are input into a pre-configured data interaction prediction model. Multiple gated loop modules in the data interaction prediction model extract the node interaction features between all the map nodes from the spatiotemporal fusion features of the map. The graph convolutional neural network module in the data interaction prediction model captures the contextual information of the pedigree graph of the transaction data, and concatenates the node interaction features of all the graph nodes into a graph comprehensive representation vector based on the contextual information. Based on the graph comprehensive representation vector and through the multilayer perceptron module in the data interaction prediction model, the node interaction index of all graph nodes in the transaction data lineage graph is predicted. Graph nodes whose node interaction index exceeds a preset index threshold are marked as high-heat graph nodes. Using multiple high-heat graph nodes of different entity types as source graph nodes, high-dimensional transaction association features are extracted from the transaction data lineage graph based on a graph neural network model, and abnormal transaction data in the financial transaction data are identified based on the high-dimensional transaction association features. Based on the abnormal transaction data, mark the abnormal graph nodes in the transaction data lineage graph; Using the abnormal graph node as the starting point for tracing, the data lineage tracing method is used to locate the origin abnormal graph node corresponding to the abnormal transaction data in the transaction data lineage graph.

2. The domain data governance method based on an artificial intelligence platform according to claim 1, characterized in that, The data preprocessing of the financial transaction data includes the following steps: For the numerical data in the financial transaction data, a hash algorithm is used to remove duplicate data from the numerical data; For the text data in the financial transaction data, the text data is standardized to obtain standard text data; Use regular expressions to remove invalid data from the standard text data; Based on a pre-defined standardized dictionary of financial data, the spelling correction and abbreviation expansion of the standard text data are completed using a dictionary matching method.

3. The domain data governance method based on an artificial intelligence platform according to claim 2, characterized in that, The process of extracting transaction metadata and semantic relationships between transaction metadata from the preprocessed financial transaction data using natural language processing techniques, and classifying the transaction metadata using a hierarchical clustering algorithm, includes the following steps: A financial data analysis model is constructed by combining named entity recognition technology and graph neural network model. The financial data analysis model is a BiLSTM-CRF-RCNN model. The financial data analysis model is labeled and trained using a pre-created financial knowledge training set and a remote supervision method. The preprocessed financial transaction data is input into the trained financial data analysis model. The bidirectional long short-term memory network module and the conditional random field module in the financial data analysis model are used to identify and extract all transaction metadata describing the financial transaction entity from the financial transaction data. The semantic relationships between the transaction metadata are extracted using a convolutional network module containing a self-attention mechanism in the financial data analysis model. The transaction metadata was classified using a hierarchical clustering algorithm.

4. The domain data governance method based on an artificial intelligence platform according to claim 3, characterized in that, The process of classifying the transaction metadata using a hierarchical clustering algorithm includes the following steps: Calculate the Gaussian distribution of the transaction metadata to obtain the high-dimensional similarity matrix of the transaction metadata in a high-dimensional space; Initialize low-dimensional data points in a preset low-dimensional space, wherein the number of low-dimensional data points is the same as the number of transaction metadata. Calculate the Cauchy distribution of all the low-dimensional data points to obtain the low-dimensional similarity matrix of the low-dimensional data points in the low-dimensional space; The KL divergence is calculated by combining the high-dimensional similarity matrix and the low-dimensional similarity matrix. The gradient descent method is used to minimize the KL divergence and the position of the low-dimensional data points in the low-dimensional space is simultaneously optimized and updated until the KL divergence reaches the minimum value, thus completing the process of reducing the transaction metadata to the low-dimensional space. The minimum intra-class variance of the transaction metadata is calculated in the low-dimensional space, and the metadata classification of the transaction metadata in the low-dimensional space is completed based on the minimum intra-class variance.

5. The domain data governance method based on an artificial intelligence platform according to claim 1, characterized in that, The step of using multiple high-heat graph nodes of different entity types as source graph nodes, extracting high-dimensional transaction association features from the transaction data lineage graph based on a graph neural network model, and identifying abnormal transaction data in the financial transaction data based on the high-dimensional transaction association features includes the following steps: Count the number of first nodes and the number of second nodes of the high-popularity graph nodes in the kinship graph of the transaction data; The degree of graph transformation is determined based on the number of the first node and the number of the second node, and graph node edges in the transaction data lineage graph are randomly generated or removed based on the degree of graph transformation to obtain the transaction data transformation graph; Multiple high-heat graph nodes of different entity types in the transaction data lineage graph are used as source graph nodes, and the nodes corresponding to the source graph nodes in the transaction data transformation graph are used as target graph nodes. By combining the source graph nodes and the target graph nodes and using a dual-graph neural network model, high-dimensional transaction association features of the transaction data lineage graph are extracted. Based on the high-dimensional transaction correlation features, anomaly detection algorithms are used to identify abnormal transaction data in the financial transaction data.

6. The domain data governance method based on an artificial intelligence platform according to claim 5, characterized in that, The step of combining the source graph nodes and the target graph nodes and using a dual-graph neural network model to extract high-dimensional transaction association features of the transaction data lineage graph includes the following steps: The source node features of the source graph nodes and the target node features of the target graph nodes are extracted simultaneously using a dual-graph neural network model. The feature association relationship between the source node features and the target node features is constructed through the cross-attention mechanism in the dual-graph neural network. The source node features and the target node features are mapped to the same high-dimensional feature space, and the source node features and the target node features are aggregated in the high-dimensional feature space based on the feature association relationship to obtain the high-dimensional transaction association features of the transaction data lineage map.

7. The domain data governance method based on an artificial intelligence platform according to claim 5, characterized in that, The step of using the anomaly graph node as the starting point for tracing and employing a data lineage tracing method to locate the origin anomaly graph node corresponding to the anomaly transaction data in the transaction data lineage graph includes the following steps: For each of the aforementioned abnormal graph nodes, the node abnormality score of the abnormal graph node is calculated based on the high-dimensional transaction association features; Using the abnormal graph nodes as the starting point for tracing, node tracing is performed in the transaction data lineage graph according to the interaction relationships. During the node tracing process, the abnormal impact score of the traced graph node is calculated based on the node abnormality score and using an iterative algorithm, and multiple maximum impact predecessor nodes among the traced graph nodes are determined according to the abnormal impact score. By taking all the most influential predecessor nodes as the tracking endpoints, multiple anomaly tracing paths are formed between the tracking start point and multiple tracking endpoints; The path anomaly score of each anomaly tracing path is calculated by combining the anomaly score and the anomaly impact score. The anomaly tracing path with the largest path anomaly score is taken as the target anomaly tracing path, and the predecessor node with the largest impact in the target anomaly tracing path is taken as the origin anomaly graph node of the anomaly origin of the anomaly ...

8. A domain data governance system based on an artificial intelligence platform, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the domain data governance method based on an artificial intelligence platform as described in any one of claims 1 to 7.

9. A computer-readable storage medium storing instructions thereon, characterized in that, When executed by a processor, the instruction causes the processor to be configured to perform the domain data governance method based on an artificial intelligence platform according to any one of claims 1 to 7.

Images