Graph kernel attention network time sequence knowledge graph embedding method

Abstract

The invention relates to a graph kernel attention network time sequence knowledge graph embedding method, which comprises the following steps: taking a dishonesty entity as a node, taking a dishonesty relationship as an edge, calculating a weight based on the dishonesty relationship and a timestamp, and constructing a directed graph; extracting feature representation of nodes in the directed graph by using a graph attention network; performing correlation analysis on the feature representation of the nodes by using a twin neural network, and extracting similar relation features; the attention enhancement mechanism calculates a global dishonesty entity relationship weight based on dishonesty relationship and similar relationship characteristics; mapping the feature representation of the node to a high-dimensional space through a domain graph kernel calculation kernel function; calculating a feature entropy value of the updated weight through linear structure entropy; obtaining a high-dimensional embedded vector of the node by combining a domain graph kernel processing result and a feature entropy value; carrying out dimensionality reduction on the high-dimensional embedded vector of the node by adopting a t-SNE method; and fusing the plurality of timestamps and the high-dimensional embedded vectors of the dimension reduction of the dishonesty entity corresponding to the dishonesty relationship to obtain embedded vector representation.

Description

Technical Field

[0001] The present application relates to the technical field of knowledge graph embedding, and in particular to a method for embedding a temporal knowledge graph using a graph core attention network. Background Art

[0002] At present, dishonest information is the default information generated by individuals, enterprises or organizations, which can be used to evaluate their credit status. The time series knowledge graph adds timestamps (h, r, t, τ) on the basis of triples. The time series knowledge graph based on dishonest information can more accurately capture the dynamic evolution of the relationship between dishonest subjects and show the dynamic interaction of dishonest information.

[0003] The existing knowledge graph embedding technology for dishonest information mainly has the following problems: 1. Static modeling problem: In order to simplify the modeling process, the knowledge graph is assumed to be static data, ignoring the dynamic interaction, update or evolution of the relationship between the untrustworthy entities. As a result, the model cannot capture the dynamic nature of the relationship between the untrustworthy entities in a timely manner, resulting in the embedding results being inaccurate and incomplete, affecting the performance of downstream tasks.

[0004] 2. The structural weight problem mainly focuses on the global structure of the knowledge graph and assigns the same weight to all triples. This makes it difficult to accurately reflect the closeness of the relationship and the degree of semantic association between different dishonest entities, making the embedding lack of prominence and distinctiveness.

[0005] 3. The problem of embedding vector smoothness. The vector representation of adjacent nodes in the knowledge graph in the embedding space has a certain smoothness property, but the complexity of the graph structure and the limitations of the embedding method make the embedding result not smooth enough, making it difficult to capture the correlation between the defaulting entity and the defaulting information.

[0006] 4. The embedding method lacks specificity. The existing embedding method lacks the feature highlighting and extraction of dishonest information, resulting in the embedding results being insufficiently sensitive to dishonest information, affecting the downstream performance in dishonest prediction tasks. Summary of the invention

[0007] Based on this, it is necessary to provide a graph core attention network temporal knowledge graph embedding method, which includes: S1: Obtain original dishonest information, extract dishonest entities, dishonest relations and timestamps in the original dishonest information to construct a quaternary, and construct a dishonest time series knowledge graph with the quaternary as the basic unit; S2: Take the dishonest entities in the dishonest time series knowledge graph as nodes and the dishonest relations as edges, and calculate weights based on the dishonest relations and timestamps to construct a directed graph; use the graph attention network to extract the feature representation of the nodes in the directed graph; apply the twin neural network to perform correlation analysis on the feature representation of the nodes and extract similarity relationship features; enhance the attention mechanism to calculate the global dishonest entity relationship weight based on the dishonest relations and similarity relationship features; S3: Map the feature representation of the node to a high-dimensional space by calculating the kernel function through the domain graph kernel; update the weight based on the global default entity relationship weight, and calculate the feature entropy value of the updated weight through the linear structural entropy; combine the domain graph kernel processing result and the feature entropy value to obtain the high-dimensional embedding vector of the node; S4: using the t-SNE method to reduce the dimension of the high-dimensional embedding vector of the node; by weighted summation, the high-dimensional embedding vectors of the untrustworthy entities corresponding to multiple timestamps and untrustworthy relationships are fused to obtain an embedding vector representation.

[0008] Preferably, the process of constructing a knowledge graph of dishonesty time series includes: Step 1: Preprocessing the original dishonesty information; Step 2: Extract the dishonest entities in the pre-processed original dishonest information through entity recognition technology, extract the dishonest relations in the pre-processed original dishonest information through relationship extraction technology, and extract the timestamp in combination with the context; Step 3: The defaulting entity, the defaulting relationship and the timestamp are used to form the quaternary group, which is recorded as (h, r, t, τ), where h represents the defaulting head entity, r represents the defaulting relationship, t represents the defaulting tail entity, and τ represents the timestamp; Step 4: Repeat steps 2-3 until all the pre-processed original default information is traversed to obtain multiple quadruple groups; integrate the multiple quadruple groups into the default time series knowledge graph.

[0009] Preferably, the preprocessing of the original default information includes: cleaning the original default information, unifying the capitalization, and splitting the sentences in the original default information into words or phrases by word segmentation technology; the cleaning includes removing noise and stop words.

[0010] Preferably, the calculating weight based on the dishonest relationship and the timestamp comprises: converting the timestamp into a time vector, and calculating the weight based on the dishonest relationship and the time vector through a residual network; The weight calculation formula includes: ; in, represents the weight of the edge between node i and node j; ReLU activation function. Represents the first convolutional layer of the residual network; Represents the second convolutional layer of the residual network; represents the edge connecting node i and node j; represents the timestamp between node i and node j; b represents the first training parameter; Represents a join operation.

[0011] Preferably, the method of extracting feature representations of nodes in a directed graph using a graph attention network includes: Step 1: The weight calculation target node and its directed neighbor nodes interact with each other using the following formula: ; in, represents a single-layer feed-forward neural network with LeakyReLU activation function; represents the weight of the edge between node i and node j; represents the jth directed neighbor node of node i; represents the second training parameter; represents the i-th node; Indicates that in the graph attention network l The interaction characteristics of node j to node i in the layer; Step 2: Calculate the attention coefficient between nodes based on the interaction features. The calculation formula is: ; in, Indicates that in the graph attention network l The attention coefficient between node i and node j in the layer; ReLU activation function. represents the kth node; N represents the number of nodes; Express The function is normalized and transposed; Step 3: Update the feature representation of the directed neighbor nodes based on the target node, the attention coefficient between nodes, and the weight. The update formula is: ; in, Indicates that in the graph attention network l+ Feature representation of node j in layer 1; represents a neural network layer; Step 4: Use a multi-head attention mechanism to aggregate the multiple feature representations to obtain the final node feature representation. The calculation formula is: ); in, Represented in the graph attention network l+ The feature representation of the final node j in layer 1; b represents the first training parameter; represents the connection operation; M represents the total number of attention heads; m represents the attention head; V represents the number of layers of the graph attention network.

[0012] Preferably, the step of using a twin neural network to perform correlation analysis on feature representations of nodes and extracting similarity relationship features comprises: Step 1: construct a feature vector pair based on the edge between nodes and the feature representation of the nodes; input the feature vector pair into a multi-layer perceptron to extract the high-level features of the two feature representations respectively. The calculation formula is: ; ; in, The high-level features representing the feature representation of the i-th node; represents the feature representation of the i-th node; The high-level features representing the feature representation of the j-th node; represents the feature representation of the jth node; represents a multi-layer perceptron; Step 2: Calculate the similarity between the high-level features of the two feature vectors using the Manhattan distance. The calculation formula is: ; in, express , The similarity between the high-level features of two feature vectors; Representing high-level features of feature vectors The kth component of ; Representing high-level features of feature vectors The kth component of ; d represents the dimension of high-level features; Step 3: Calculate the contrast loss function based on the similarity, and the calculation formula is: ; in, represents the contrast loss function; N' represents the number of feature vector pairs; It represents the label of the nth pair of feature vectors, and its value is 1 or 0, 1 represents similarity, and 0 represents dissimilarity; q represents the threshold hyperparameter; Indicates similarity; Step 4: Minimize the contrast loss function through the back propagation algorithm and the gradient descent optimization algorithm, and update the parameters of the twin neural network; input the node feature representation into the trained twin neural network, and output the similarity relationship feature.

[0013] Preferably, the enhanced attention mechanism calculates the global dishonest entity relationship weight based on the dishonest relationship and similarity relationship features, including: Step 1: Calculate the enhanced attention coefficient based on the similarity relationship feature, and the calculation formula is: ; in, Indicates the enhanced attention factor; represents the initial enhanced attention coefficient; Indicates similarity relationship characteristics; Define the initial relation embedding matrix , represents the number of dishonest relationships; F represents the dimension of the initial features embedded in each relationship; Step 2: The types of dishonest relationships include direct relationships, indirect relationships, and both direct and indirect relationships; the direct relationship matrix is ​​recorded as ; The indirect relationship matrix is ​​recorded as ; The matrix with both direct and indirect relationships is recorded as ;in, Represents the attention coefficient between node i and node j; Represents the corresponding row of the initial relation embedding matrix; It means sum; For any type of relationship, the linear transformation formula is: ; in, Represents any type of relationship matrix after linear transformation; Indicates about The weight matrix of represents the weight matrix about F; represents any type of relationship matrix; Step 3: Weight matrix Perform a linear transformation, and then use the ReLU activation function to get the absolute attention value of each relationship. The calculation formula is: ; in, represents the absolute attention value; represents the ReLU activation function; b represents the first training parameter; Step 4: Use the Softmax layer to normalize the absolute attention value of each relationship to obtain the global default entity relationship weight, which is calculated as: ; in, Represents the global default entity relationship weight.

[0014] Preferably, the process of obtaining the high-dimensional embedding vector of the node includes: Step 1: Update the weight based on the global default entity relationship weight, and obtain the feature representation of each node; Step 2: For each node, combine the feature representations of the nodes to obtain the adjacency matrix, aggregate the feature representations of the nodes associated with it, and obtain the aggregation matrix. The calculation formula is: ; ; in, Indicates l +1 layer aggregation matrix; A represents the adjacency matrix; represents a nonlinear activation function; represents the diagonal node degree matrix; I represents the identity matrix; Represents the features of the lth layer; represents the weight of the edge between node i and node j; Step 3: Use the domain graph kernel calculation kernel function to map the node aggregation matrix to a high-dimensional space to obtain the domain graph kernel processing result. The calculation formula is: ; in, Indicates the domain graph kernel processing result; represents the domain of node i; represents the domain of node j; represents the basic kernel function; Step 4: Use linear structural entropy to calculate the characteristic entropy value of the updated weight. The calculation formula is: ; in, represents the characteristic entropy value; Represents the weight of the edge between node i and node j after the update; Represents any type of relationship matrix after linear transformation; Step 5: Combine the domain graph kernel processing results and feature entropy values ​​to obtain the high-dimensional embedding vector of the node. The calculation formula is: ; in, Represents the high-dimensional embedding vector of the i-th node; represents a mapping function; represents the domain graph kernel processing result of the i-th node; Represents the characteristic entropy value of the i-th node.

[0015] Preferably, it also includes training the overall embedding process; the training includes training using an energy function or an Adam optimizer, and a soft margin loss; Using the energy function as the first training loss, and optimizing the overall embedding process by minimizing the first training loss; A soft margin loss is used as the second training loss, and the learning rate is dynamically adjusted through the Adam optimizer until the overall embedding process converges.

[0016] Preferably, the first training loss expression is: ; in, represents the first training loss; h represents the default head entity; r represents the default relation; t represents the default tail entity; t represents the timestamp; G represents the sampling set; G' represents the negative sampling set; h' represents the negative sample of the dishonest head entity; r' represents the negative sample of the dishonest relationship; t' represents the negative sample of the dishonest tail entity; the Negative samples representing timestamps; Indicates with t The difference function expressed by the L1 norm between h+r and t; Indicates with the The dissimilarity function expressed by the L1 norm between h'+r' and t'; represents the marginal hyperparameter; The second training loss expression is: ; ; in, represents the second training loss; represents the score function.

[0017] Beneficial effects: This method can better capture the characteristics of the relationship between dishonest entities changing over time, highlight the latest and most important knowledge, make the embedding vector more focused on the current important dishonest information, and provide downstream tasks with more time-series and dynamic feature representations. At the same time, the use of graph attention networks and twin neural networks can effectively capture the characteristics and associations between dishonest entities, and the enhanced attention mechanism highlights the current important dishonest information. The domain graph kernel method captures complex time dependencies and generates high-quality embedding vectors. Using t-SNE to reduce the high-dimensional embedding vector to a low-dimensional vector space can not only retain the key information in the knowledge graph, but also reduce the data processing and computational complexity of downstream tasks, and improve the accuracy and interpretability of subsequent tasks such as entity alignment, link prediction, and reasoning completion. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 This is a flowchart of the graph core attention network temporal knowledge graph embedding method of an embodiment of the present application.

[0020] Figure 2 A schematic diagram of relationship types according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0022] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0023] like Figure 1 As shown, this embodiment provides a method for embedding a time series knowledge graph based on a graph core attention network based on dishonesty information, the method comprising: S1: Obtain original dishonest information, extract dishonest entities, dishonest relations and timestamps in the original dishonest information to construct a quadruple, and construct a dishonest time series knowledge graph using the quadruple as a basic unit.

[0024] Specifically, the process of constructing the knowledge graph of dishonesty time series includes: Step 1: Preprocessing the original dishonesty information; Step 2: Extract the dishonest entities in the pre-processed original dishonest information through entity recognition technology, extract the dishonest relations in the pre-processed original dishonest information through relationship extraction technology, and extract the timestamp in combination with the context; In this embodiment, since the data involved are only text data and the date formats are all in common formats, the timestamp can be extracted by using the dateparser library in Python or simple natural language processing technology.

[0025] Step 3: The defaulting entity, the defaulting relationship and the timestamp are used to form the quaternary group, which is recorded as (h, r, t, τ), where h represents the defaulting head entity, r represents the defaulting relationship, t represents the defaulting tail entity, and τ represents the timestamp; Step 4: Repeat steps 2-3 until all the pre-processed original default information is traversed to obtain multiple quadruple groups; integrate the multiple quadruple groups into the default time series knowledge graph.

[0026] Furthermore, the preprocessing of the original untrustworthy information includes: cleaning the original untrustworthy information, unifying the capitalization, and splitting the sentences in the original untrustworthy information into words or phrases by word segmentation technology; the cleaning includes removing noise and stop words.

[0027] S2: Construct a directed graph with the untrustworthy entities in the untrustworthy time series knowledge graph as nodes and the untrustworthy relationships as edges, and calculate weights based on the untrustworthy relationships and timestamps; use the graph attention network to extract the feature representations of the nodes in the directed graph; apply the twin neural network to perform correlation analysis on the feature representations of the nodes and extract similarity relationship features; enhance the attention mechanism to calculate the global untrustworthy entity relationship weights based on the untrustworthy relationship and similarity relationship features.

[0028] Specifically, the calculating weight based on the dishonest relationship and the timestamp includes: converting the timestamp into a time vector, and calculating the weight based on the dishonest relationship and the time vector through a residual network; The weight calculation formula includes: ; in, represents the weight of the edge between node i and node j; ReLU activation function. Represents the first convolutional layer of the residual network; Represents the second convolutional layer of the residual network; represents the edge connecting node i and node j; represents the timestamp between node i and node j; b represents the first training parameter; Represents a join operation.

[0029] Furthermore, the method of extracting feature representations of nodes in a directed graph using a graph attention network includes: Step 1: The weight calculation target node and its directed neighbor nodes interact with each other using the following formula: ; in, represents a single-layer feed-forward neural network with LeakyReLU activation function; represents the weight of the edge between node i and node j; represents the jth directed neighbor node of node i; represents the second training parameter; represents the i-th node; Indicates that in the graph attention network l The interaction characteristics of node j to node i in the layer; Step 2: Calculate the attention coefficient between nodes based on the interaction features. The calculation formula is: ; in, Indicates that in the graph attention network l The attention coefficient between node i and node j in the layer; ReLU activation function. represents the kth node; N represents the number of nodes; Express The function is normalized and transposed to ensure the scale consistency of the neural network output and optimize the feature expression; Step 3: Update the feature representation of the directed neighbor nodes based on the target node, the attention coefficient between nodes, and the weight. The update formula is: ; in, Indicates that in the graph attention network l+ Feature representation of node j in layer 1; represents a neural network layer; Step 4: Use a multi-head attention mechanism to aggregate the multiple feature representations to obtain the final node feature representation. The calculation formula is: ; in, Represented in the graph attention network l+ The feature representation of the final node j in layer 1; b represents the first training parameter; represents the connection operation; M represents the total number of attention heads; m represents the attention head; V represents the number of layers of the graph attention network.

[0030] Furthermore, the twin neural network is used to perform correlation analysis on the feature representation of the nodes and extract similar relationship features, including: Step 1: construct a feature vector pair based on the edge between nodes and the feature representation of the nodes; input the feature vector pair into a multi-layer perceptron to extract the high-level features of the two feature representations respectively. The calculation formula is: ; ; in, The high-level features representing the feature representation of the i-th node; represents the feature representation of the i-th node; The high-level features representing the feature representation of the j-th node; represents the feature representation of the jth node; represents a multi-layer perceptron; Step 2: Calculate the similarity between the high-level features of the two feature vectors using the Manhattan distance. The calculation formula is: ; in, express , The similarity between the high-level features of two feature vectors; Representing high-level features of feature vectors The kth component of ; Representing high-level features of feature vectors The kth component of ; d represents the dimension of high-level features; Step 3: Calculate the contrast loss function based on the similarity, and the calculation formula is: ; in, represents the contrast loss function; N' represents the number of feature vector pairs; It represents the label of the nth pair of feature vectors, and its value is 1 or 0, 1 represents similarity, and 0 represents dissimilarity; q represents the threshold hyperparameter; Indicates similarity; Step 4: Minimize the contrast loss function through the back propagation algorithm and the gradient descent optimization algorithm, and update the parameters of the twin neural network; input the node feature representation into the trained twin neural network, and output the similarity relationship feature.

[0031] Furthermore, the enhanced attention mechanism calculates the global dishonest entity relationship weight based on the dishonest relationship and similarity relationship features, including: Step 1: Calculate the enhanced attention coefficient based on the similarity relationship feature, and the calculation formula is: ; in, Indicates the enhanced attention factor; represents the initial enhanced attention coefficient; Indicates similarity relationship characteristics; Define the initial relation embedding matrix , represents the number of dishonest relationships; F represents the dimension of the initial features embedded in each relationship; Step 2: If Figure 2 As shown in the figure, the types of dishonest relationships include direct relationships, indirect relationships, and both direct and indirect relationships; the direct relationship matrix is ​​recorded as ; The indirect relationship matrix is ​​recorded as ; The matrix with both direct and indirect relationships is recorded as ;in, Represents the attention coefficient between node i and node j; Represents the corresponding row of the initial relation embedding matrix; It means sum; For any type of relationship, the linear transformation formula is: ; in, Represents any type of relationship matrix after linear transformation; Indicates about The weight matrix of represents the weight matrix about F; represents any type of relationship matrix; Step 3: Weight matrix Perform a linear transformation, and then use the ReLU activation function to get the absolute attention value of each relationship. The calculation formula is: ; in, represents the absolute attention value; represents the ReLU activation function; b represents the first training parameter; Step 4: Use the Softmax layer to normalize the absolute attention value of each relationship to obtain the global default entity relationship weight, which is calculated as: ; in, Represents the global default entity relationship weight.

[0032] The graph attention network is used to assign different attention weights to different relationship paths and time attributes, thereby extracting vector representations with domain characteristics, and the twin neural network is used to perform correlation analysis on the features to further improve the quality and accuracy of feature representation. Based on the correlation analysis, the enhanced attention mechanism is applied to assign corresponding attention weights to the correlation analysis results, thereby highlighting the latest and most important information on dishonesty.

[0033] S3: Map the feature representation of the node to the high-dimensional Hilbert space through the domain graph kernel calculation kernel function; update the weight based on the global default entity relationship weight, and calculate the characteristic entropy value of the updated weight through linear structural entropy; combine the domain graph kernel processing result and the characteristic entropy value to obtain the high-dimensional embedding vector of the node.

[0034] Specifically, the process of obtaining the high-dimensional embedding vector of a node includes: Step 1: Update the weight based on the global default entity relationship weight, and obtain the feature representation of each node; Step 2: For each node, combine the feature representations of the nodes to obtain the adjacency matrix, aggregate the feature representations of the nodes associated with it, and obtain the aggregation matrix. The calculation formula is: ; ; in, Indicates l +1 layer aggregation matrix; A represents the adjacency matrix; represents a nonlinear activation function; represents the diagonal node degree matrix; I represents the identity matrix; Represents the features of the lth layer; represents the weight of the edge between node i and node j; Step 3: Use the domain graph kernel calculation kernel function to map the node aggregation matrix to the high-dimensional Hilbert space to obtain the domain graph kernel processing result. The calculation formula is: ; in, Indicates the domain graph kernel processing result; represents the domain of node i; represents the domain of node j; represents the basic kernel function; Step 4: Use linear structural entropy to calculate the characteristic entropy value of the updated weight. The calculation formula is: ; in, represents the characteristic entropy value; Represents the weight of the edge between node i and node j after the update; Represents any type of relationship matrix after linear transformation; Step 5: Combine the domain graph kernel processing results and feature entropy values ​​to obtain the high-dimensional embedding vector of the node. The calculation formula is: ; in, Represents the high-dimensional embedding vector of the i-th node; represents a mapping function; represents the domain graph kernel processing result of the i-th node; Represents the characteristic entropy value of the i-th node.

[0035] Through the graph kernel method, it is mapped to the high-dimensional Hilbert space, and the graph kernel is used to judge the isomorphism of the dishonest information and find the isomorphism coefficients of adjacent nodes. By measuring the linear structural entropy between time points, the change of dishonest information over time is understood, and finally a high-dimensional embedding vector of dynamic dishonest information is obtained to improve the embedding quality and solve the smoothing problem of the embedded vector.

[0036] S4: using the t-SNE method to reduce the dimension of the high-dimensional embedding vector of the node; by weighted summation, the high-dimensional embedding vectors of the untrustworthy entities corresponding to multiple timestamps and untrustworthy relationships are fused to obtain an embedding vector representation.

[0037] The adopting the t-SNE method to reduce the dimension of the high-dimensional embedding vector of the node includes: From the previous step, we know that represents the high-dimensional embedding vector of the i-th node, let , n represents the number of nodes, and the Gaussian kernel function is used to calculate the similarity between each high-dimensional embedding vector: ; in, Indicates that in high-dimensional space, high-dimensional embedding vector and The similarity between Represents the corresponding high-dimensional embedding vector Gaussian distribution variance; Represents the high-dimensional embedding vector of the k-th node.

[0038] Then randomly initialize the position of each data point in the low-dimensional space , use t distribution to calculate the similarity matrix between data points in low-dimensional space, the calculation formula is: ; in, Represents the similarity matrix between data point i and data point j in low-dimensional space; Represents the location of data point i in low-dimensional space; Represents the location of data point j in low-dimensional space; Represents the location of data point k in low-dimensional space; Use gradient descent to minimize the KL divergence between the high-dimensional and low-dimensional similarity distributions to optimize the position of data points in the low-dimensional space, so that similar data points in the high-dimensional space still maintain a close distance in the low-dimensional space. The calculation method is: ; After optimization, the high-dimensional embedding vector in the high-dimensional space The position mapped to the low-dimensional space ; Next, the high-dimensional embedding vectors of the dishonest entities corresponding to multiple timestamps and dishonest relationships are fused by weighted summation to form a unified embedding vector representation; the calculation formula is: ; in, represents the embedding vector representation; N represents the number of nodes; Represents the global default entity relationship weight.

[0039] The t-SNE method is used to reduce the dimensionality of high-dimensional embedding vectors, while retaining the time-based domain feature loss information, reducing the computational complexity and improving data processing efficiency.

[0040] In this embodiment, the method further includes training the overall embedding process; the training includes training using an energy function or an Adam optimizer, and a soft margin loss; The energy function is used as the first training loss, and the overall embedding process is optimized by minimizing the first training loss; the first training loss expression is: ; in, represents the first training loss; h represents the default head entity; r represents the default relation; t represents the default tail entity; t represents the timestamp; G represents the sampling set; G' represents the negative sampling set; h' represents the negative sample of the dishonest head entity; r' represents the negative sample of the dishonest relationship; t' represents the negative sample of the dishonest tail entity; the Negative samples representing timestamps; Indicates with t The difference function expressed by the L1 norm between h+r and t; Indicates with the The dissimilarity function expressed by the L1 norm between h'+r' and t'; represents the marginal hyperparameter; Soft margin loss is used as the second training loss, and the learning rate is dynamically adjusted through the Adam optimizer until the overall embedding process converges. The second training loss expression is: ; ; in, represents the second training loss; represents the score function.

[0041] The graph core attention network temporal knowledge graph embedding method provided in this embodiment can better capture the characteristics of the relationship between the untrustworthy entities changing over time, highlight the latest and most important knowledge, make the embedding vector more focused on the current important untrustworthy information, and provide downstream tasks with more time-series and dynamic feature representations. At the same time, the use of graph attention networks and twin neural networks can effectively capture the characteristics and associations between untrustworthy entities, enhance the attention mechanism to highlight the current important untrustworthy information, and the domain graph core method captures complex time dependencies and generates high-quality embedding vectors. Using t-SNE to reduce the high-dimensional embedding vector to a low-dimensional vector space can not only retain the key information in the knowledge graph, but also reduce the data processing and computational complexity of downstream tasks, and improve the accuracy and interpretability of subsequent tasks such as entity alignment, link prediction, and reasoning completion.

[0042] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0043] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent application shall be subject to the attached claims.

Claims

1. A graph core attention network temporal knowledge graph embedding method, characterized in that: include: S1: Obtain original dishonest information, extract dishonest entities, dishonest relations and timestamps in the original dishonest information to construct a quaternary, and construct a dishonest time series knowledge graph with the quaternary as the basic unit; S2: Take the dishonest entities in the dishonest time series knowledge graph as nodes and the dishonest relations as edges, and calculate weights based on the dishonest relations and timestamps to construct a directed graph; use the graph attention network to extract the feature representation of the nodes in the directed graph; apply the twin neural network to perform correlation analysis on the feature representation of the nodes and extract similar relationship features; The enhanced attention mechanism calculates the global default entity relationship weight based on the default relationship and similarity relationship features; S3: Map the feature representation of the node to a high-dimensional space by computing the kernel function through the domain graph kernel; The weight is updated based on the global default entity relationship weight, and the characteristic entropy value of the updated weight is calculated by linear structural entropy; the high-dimensional embedding vector of the node is obtained by combining the domain graph kernel processing result and the characteristic entropy value; S4: using the t-SNE method to reduce the dimension of the high-dimensional embedding vector of the node; by weighted summation, the high-dimensional embedding vectors of the untrustworthy entities corresponding to multiple timestamps and untrustworthy relationships are fused to obtain an embedding vector representation.

2. The graph core attention network temporal knowledge graph embedding method according to claim 1 is characterized in that: The process of constructing the knowledge graph of dishonesty time series includes: Step 1: Preprocessing the original dishonesty information; Step 2: Extract the dishonest entities in the pre-processed original dishonest information through entity recognition technology, extract the dishonest relations in the pre-processed original dishonest information through relationship extraction technology, and extract the timestamp in combination with the context; Step 3: The defaulting entity, the defaulting relationship and the timestamp are used to form the quaternary group, which is recorded as (h, r, t, τ), where h represents the defaulting head entity, r represents the defaulting relationship, t represents the defaulting tail entity, and τ represents the timestamp; Step 4: Repeat steps 2-3 until all the pre-processed original default information is traversed to obtain multiple quadruple groups; integrate the multiple quadruple groups into the default time series knowledge graph.

3. The graph core attention network temporal knowledge graph embedding method according to claim 2 is characterized in that: The preprocessing of the original default information includes: cleaning the original default information, unifying the capitalization, and splitting the sentences in the original default information into words or phrases by word segmentation technology; the cleaning includes removing noise and stop words.

4. The graph core attention network temporal knowledge graph embedding method according to claim 2 is characterized in that: The calculating weight based on the dishonest relationship and the timestamp includes: converting the timestamp into a time vector, and calculating the weight based on the dishonest relationship and the time vector through a residual network; The weight calculation formula includes: ； in, represents the weight of the edge between node i and node j; ReLU activation function. Represents the first convolutional layer of the residual network; Represents the second convolutional layer of the residual network; represents the edge connecting node i and node j; represents the timestamp between node i and node j; b represents the first training parameter; Represents a join operation.

5. The graph core attention network temporal knowledge graph embedding method according to claim 4 is characterized in that: The method of extracting feature representation of nodes in a directed graph by using a graph attention network includes: Step 1: The weight calculation target node and its directed neighbor nodes interact with each other using the following formula: ； in, represents a single-layer feed-forward neural network with LeakyReLU activation function; represents the weight of the edge between node i and node j; represents the jth directed neighbor node of node i; represents the second training parameter; represents the i-th node; Indicates that in the graph attention network l The interaction characteristics of node j to node i in the layer; Step 2: Calculate the attention coefficient between nodes based on the interaction features. The calculation formula is: ； in, Indicates that in the graph attention network l The attention coefficient between node i and node j in the layer; ReLU activation function. represents the kth node; N represents the number of nodes; Express The function is normalized and transposed; Step 3: Update the feature representation of the directed neighbor nodes based on the target node, the attention coefficient between nodes, and the weight. The update formula is: ； in, Indicates that in the graph attention network l+ Feature representation of node j in layer 1; represents a neural network layer; Step 4: Use a multi-head attention mechanism to aggregate the multiple feature representations to obtain the final node feature representation. The calculation formula is: )； in, Represented in the graph attention network l+ The feature representation of the final node j in layer 1; b represents the first training parameter; represents the connection operation; M represents the total number of attention heads; m represents the attention head; V represents the number of layers of the graph attention network.

6. The graph core attention network temporal knowledge graph embedding method according to claim 5 is characterized in that: The twin neural network is used to perform correlation analysis on the feature representation of the nodes and extract similarity relationship features, including: Step 1: construct a feature vector pair based on the edge between nodes and the feature representation of the nodes; input the feature vector pair into a multi-layer perceptron to extract the high-level features of the two feature representations respectively. The calculation formula is: ； ； in, The high-level features representing the feature representation of the i-th node; represents the feature representation of the i-th node; The high-level features representing the feature representation of the j-th node; represents the feature representation of the jth node; represents a multi-layer perceptron; Step 2: Calculate the similarity between the high-level features of the two feature vectors using the Manhattan distance. The calculation formula is: ； in, express , The similarity between the high-level features of two feature vectors; Representing high-level features of feature vectors The kth component of ; Representing high-level features of feature vectors The kth component of ; d represents the dimension of high-level features; Step 3: Calculate the contrast loss function based on the similarity, and the calculation formula is: ； in, represents the contrast loss function; N' represents the number of feature vector pairs; It represents the label of the nth pair of feature vectors, and its value is 1 or 0, 1 represents similarity, and 0 represents dissimilarity; q represents the threshold hyperparameter; Indicates similarity; Step 4: Minimize the contrast loss function through the back propagation algorithm and the gradient descent optimization algorithm, and update the parameters of the twin neural network; input the node feature representation into the trained twin neural network, and output the similarity relationship feature.

7. The graph core attention network temporal knowledge graph embedding method according to claim 6 is characterized in that: The enhanced attention mechanism calculates the global default entity relationship weight based on the default relationship and similarity relationship features, including: Step 1: Calculate the enhanced attention coefficient based on the similarity relationship feature, and the calculation formula is: ； in, Indicates the enhanced attention factor; represents the initial enhanced attention coefficient; Indicates similarity relationship characteristics; Define the initial relation embedding matrix , represents the number of dishonest relationships; F represents the dimension of the initial features embedded in each relationship; Step 2: The types of dishonest relationships include direct relationships, indirect relationships, and both direct and indirect relationships; the direct relationship matrix is ​​recorded as ; The indirect relationship matrix is ​​recorded as ; The matrix with both direct and indirect relationships is recorded as ;in, Represents the attention coefficient between node i and node j; represents the corresponding row of the initial relation embedding matrix; It means sum; For any type of relationship, the linear transformation formula is: ； in, Represents any type of relationship matrix after linear transformation; Indicates about The weight matrix of represents the weight matrix about F; represents any type of relationship matrix; Step 3: Weight matrix Perform a linear transformation, and then use the ReLU activation function to get the absolute attention value of each relationship. The calculation formula is: ； in, represents the absolute attention value; represents the ReLU activation function; b represents the first training parameter; Step 4: Use the Softmax layer to normalize the absolute attention value of each relationship to obtain the global default entity relationship weight, which is calculated as: ； in, Represents the global default entity relationship weight.

8. The graph core attention network temporal knowledge graph embedding method according to claim 1 is characterized in that: The process of obtaining the high-dimensional embedding vector of a node includes: Step 1: Update the weight based on the global default entity relationship weight, and obtain the feature representation of each node; Step 2: For each node, combine the feature representations of the nodes to obtain the adjacency matrix, aggregate the feature representations of the nodes associated with it, and obtain the aggregation matrix. The calculation formula is: ； ； in, Indicates l +1 layer aggregation matrix; A represents the adjacency matrix; represents a nonlinear activation function; represents the diagonal node degree matrix; I represents the identity matrix; Represents the features of the lth layer; represents the weight of the edge between node i and node j; Step 3: Use the domain graph kernel calculation kernel function to map the node aggregation matrix to a high-dimensional space to obtain the domain graph kernel processing result. The calculation formula is: ； in, Indicates the domain graph kernel processing result; represents the domain of node i; represents the domain of node j; represents the basic kernel function; Step 4: Use linear structural entropy to calculate the characteristic entropy value of the updated weight. The calculation formula is: ； in, represents the characteristic entropy value; Represents the weight of the edge between node i and node j after the update; Represents any type of relationship matrix after linear transformation; Step 5: Combine the domain graph kernel processing results and feature entropy values ​​to obtain the high-dimensional embedding vector of the node. The calculation formula is: ； in, Represents the high-dimensional embedding vector of the i-th node; represents a mapping function; represents the domain graph kernel processing result of the i-th node; Represents the characteristic entropy value of the i-th node.

9. The graph core attention network temporal knowledge graph embedding method according to claim 1 is characterized in that: Also included is a training overall embedding process; the training includes training using an energy function or an Adam optimizer and a soft margin loss; Using the energy function as the first training loss, and optimizing the overall embedding process by minimizing the first training loss; A soft margin loss is used as the second training loss, and the learning rate is dynamically adjusted through the Adam optimizer until the overall embedding process converges.

10. The graph core attention network temporal knowledge graph embedding method according to claim 9 is characterized in that: The first training loss expression is: ； in, represents the first training loss; h represents the default head entity; r represents the default relation; t represents the default tail entity; τ represents the timestamp; G represents the sampling set; G' represents the negative sampling set; h' represents the negative sample of the dishonest head entity; r' represents the negative sample of the dishonest relationship; t' represents the negative sample of the dishonest tail entity; τ' Represents negative samples of timestamps; Indicates with τ The difference function expressed by the L1 norm between h+r and t; Indicates with τ' The dissimilarity function expressed by the L1 norm between h'+r' and t'; represents the marginal hyperparameter; The second training loss expression is: ； ； in, represents the second training loss; represents the score function.

Images