Time sequence knowledge graph completion method, system and equipment based on mixed attention

Through the time sequence knowledge graph completion method of mixed attention, combined with the linear time representation model and the mixed self-attention mechanism, the problem of existing models ignoring historical information is solved, and the accuracy and stability of knowledge graph completion is improved. It is suitable for Various completion tasks.

CN119005318AInactive Publication Date: 2024-11-22CETC DIGITAL INTELLIGENCE TECH (BEIJING) CO LTD

Patent Information

Application Number
CN202411129015.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-16
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing timing knowledge graph completion model only focuses on the entity and relationship characteristics of the current time in the inference process, ignoring historical information, so it is impossible to accurately predict the complete information.

Method used

Using a time series knowledge graph completion method based on mixed attention, by constructing a linear time representation model and a mixed self-attention mechanism, we capture the dynamic changes of entities at different time points and the interaction between time and structure, and aggregate historical information to improve Complete accuracy.

Benefits of technology

Effectively introducing the timeliness of knowledge into vector operations of entities and relationships, improving the accuracy and stability of knowledge graph completion, suitable for a variety of completion tasks, and reducing model complexity.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119005318A_ABST
    Figure CN119005318A_ABST
Patent Text Reader

Abstract

The invention discloses a time sequence knowledge graph completion method based on mixed attention, and the method comprises the steps: carrying out the initial definition of a time sequence knowledge graph; constructing a linear time representation model, and obtaining vector representation of the knowledge graph data set and vector representation of entities / relationships / time through the linear time representation model; historical interaction records of each entity under different timestamps are collected to form a historical information database, vectors of the knowledge graph data set are embedded, and a hybrid self-attention mechanism is obtained through the vectors of the embedded knowledge graph data set; and aggregating historical information through a hybrid self-attention mechanism to obtain a time sequence knowledge graph completion model. According to the method, through a mixed attention mechanism based on time and space, attention can more accurately learn entity historical information and complement the relation of tasks from two modes of semantics and time sequence.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of data mining technology, and in particular to a method, system and device for completing a temporal knowledge graph based on hybrid attention. Background Art

[0002] A knowledge graph is a large-scale structured semantic knowledge base that describes concepts and their relationships in the physical world in symbolic form. The basic unit of a knowledge graph is a "head entity-relationship-tail entity" triple. Entities are connected to each other through relationships to form a network of knowledge structures. In a knowledge graph, each node represents an entity, and each edge represents a relationship between entities, which can be directional or non-directional. Knowledge graphs are often missing a lot of content due to difficulties in construction, incomplete data, and domain limitations.

[0003] Knowledge graph completion is one of the effective means to improve the missing content of knowledge graphs. Its goal is to infer the missing information of entities in the graph based on the existing information in the knowledge graph. The most typical completion task is to predict the tail entity based on the provided head entity and relationship, that is, to predict the tail entity through the head entity and relationship. Existing knowledge graph completion models usually adopt a representation learning-based method, that is, mapping the entities and relationships in the knowledge graph into a low-dimensional vector space, and using vector operations to infer unknown information. However, traditional knowledge graph completion models ignore the timeliness of knowledge, such as ignoring that employees' occupations will change over time according to their career development, which will cause the model to give wrong inference results. Considering this shortcoming, the temporal knowledge graph completion model introduces the timeliness of knowledge, that is, the timestamp of the knowledge triple, into the learning and inference of the knowledge graph structure to improve the completion effect. However, the existing temporal knowledge graph completion models usually use two types of methods to realize the temporal structure learning and completion of the knowledge graph.

[0004] The first type of method is based on path search, which obtains prediction-related subgraphs by setting time constraints and heuristic rules. This type of method trains a neural network-based path searcher to guide the search direction and target on the knowledge graph, and determines whether the path end point has been reached and the completion task has been completed. However, this type of model usually requires path traversal in the knowledge graph, so the training complexity on a knowledge graph with rich connections increases exponentially, and also causes the reasoning speed to drop significantly. And due to the introduction of timing information, the decision space of the path search model increases exponentially, which also greatly increases the difficulty of finding the correct path.

[0005] The second type of method is based on the operation of entity, relationship, and time representation vectors in the knowledge graph in low-dimensional space, such as translation model, bilinear model, tensor decomposition model, etc. The core problem of this type of model is how to effectively introduce time into the reasoning process of the model, and proposes models including time-dependent entity / relationship representation learning, temporal graph evolution, and temporal rule discovery to achieve completion. Compared with path-based methods, this type of method can use the learned implicit rules and patterns in vector space to achieve graph completion, with the characteristics of low model complexity, high reasoning and completion efficiency, and easier embedding of time information. However, existing methods need to use specific vector operations to learn the features between entity / relationship / time vectors, such as four-dimensional tensor decomposition, hyperplane mapping, space rotation and other geometric transformations, and most of these feature operations are suitable for the extraction of specific features and cannot be extended to other types of features. In addition, during the reasoning process, the above models only focus on the entity and relationship features of the current time, ignoring their historical information, and therefore cannot accurately predict the completion information. Summary of the invention

[0006] One of the purposes of the present invention is to provide a temporal knowledge graph completion method based on hybrid attention, so as to solve the problem that the feature operation only focuses on the entity and relationship features of the current time during the reasoning process and ignores the historical information, so the completed information cannot be accurately predicted.

[0007] The present invention is implemented through the following technical scheme, a temporal knowledge graph completion method based on hybrid attention, comprising the following steps: S100, initially defining the temporal knowledge graph, the initial definition including the data format, type and completion task of the temporal knowledge graph, collecting knowledge graph data containing timestamps, and organizing the knowledge graph data into triples of q=(s, r, o, t), wherein s is the head entity, o is the tail entity, and r is the relationship; t is the timestamp, indicating the validity period of the triple; S200, constructing a linear time representation model, and obtaining a vector representation of the knowledge graph data set through the linear time representation model; vector representation of entity / relationship / time; S300, collecting historical interaction records of each entity at different timestamps to form a historical information database, embedding the vector of the knowledge graph data set, and obtaining a hybrid self-attention mechanism through the vector of the embedded knowledge graph data set; S400, aggregating historical information through the hybrid self-attention mechanism to obtain a temporal knowledge graph completion model.

[0008] It should be noted that based on the traditional knowledge graph representation learning model, entities, relations and time are mapped into a low-dimensional vector space, and the completion task is constructed as a specific operation in the low-dimensional vector space. Compared with the path-based method, this type of method has lower complexity and higher model flexibility, and can be easily extended to different completion tasks (such as head entity completion, tail entity completion, relationship completion, and time completion, etc.). The core goal of this type of model is how to effectively introduce the timeliness of knowledge into the vector operation of entities / relations and improve the accuracy of completion. A more common approach is to add the representation vector of time as a new dimension to the original entity / relationship operation. However, existing methods need to use specific vector operations to learn the features between entity / relationship / time vectors, such as four-dimensional tensor decomposition, hyperplane mapping, space rotation and other geometric transformations, and most of these operations are suitable for the extraction of specific features and cannot be extended to other types of features. In addition, during the reasoning process, the above model only focuses on the entity and relationship features of the current time, but ignores their historical information, so it cannot accurately predict the completion information.

[0009] Furthermore, the knowledge graph data can be integrated into a knowledge graph dataset, and the knowledge graph dataset is represented as G=(E,R,T), where G is the knowledge graph dataset, E is the real set, R is the relationship set, and T is the time set contained in the graph.

[0010] The triples are used to complete the temporal knowledge graph, and the completion includes predicting the missing tail entity by combining the knowledge graph dataset (s, r, ?, t), and / or predicting the missing head entity by combining the knowledge graph dataset (?, r, o, t).

[0011] The predicted missing tail entity is represented as,

[0012] The predicted missing head entity is represented as,

[0013] in, Represents the candidate tail entity that minimizes f(*) Represents the candidate head entity that minimizes f(*) s,r,o,t∈C d is a vector of entities, relations, and time; d is the dimension of the vector.

[0014] Further, constructing a linear time representation model may also include, S210, defining a vector representation model of time, treating the timestamp as an independent time segment in the existing time vector model; S220, assigning an independent vector to each timestamp; S230, adding a time smoothing constraint to ensure the order of time, and constructing a linear time representation model; the linear time representation model includes setting a reference time vector t0 and an incremental time vector Δ t , the representation vector of any time can be obtained by Get, where t i is the representation vector of any time; t0 is the reference time vector; Δ t is the incremental time vector; i is t i and t0.

[0015] Furthermore, aggregating historical information may include capturing dynamic changes of entities at different time points, combining time and structure information, capturing the interaction between time and structure, designing a hybrid self-attention mechanism by capturing the interaction between time and structure, and completing the aggregation of historical information through the designed hybrid self-attention mechanism.

[0016] Furthermore, the hybrid self-attention mechanism includes,

[0017] Attention is defined as α(v1,v2,h)=h T σ(Wv1+Uv2),

[0018] Among them, σ(*) is the activation function, h, W, U are the learning parameters, α(v1, v2, h) is the attention function, and h T is the transpose of h, v1 and v2 are the representation vectors of input entity 1 and input entity 2.

[0019] The attention combines the structural information of both temporal and spatial modalities.

[0020] In the temporal mode, the attention measures the influence of historical information at different times on the current completion target, that is,

[0021]

[0022] in, For s to s i The attention score at time t, α(s,s i ,t) is the attention function, s,s i , t are the entity, the entity’s neighbor, and the corresponding timestamp, exp is the natural base, N s represents the neighbor set of entity s, s j ∈N sis a neighbor entity of s.

[0023] In the spatial modality, the attention measures the influence of different types of historical information on the current completion target, that is,

[0024] in, For s to s i The attention score on the relation r, α(s,s i ,r) is the attention function, s,s i , r are the entity, the neighbor of the entity, and the corresponding relationship type, exp is the natural base, N s represents the neighbor set of entity s, s j ∈N s is a neighbor entity of s.

[0025] The temporal modal attention and the spatial modal attention are mixed to obtain a hybrid self-attention mechanism.

[0026]

[0027] in, is the final entity representation vector, s is the original entity representation vector, is the neighbor information based on time aggregation, Represents neighbor information based on spatial aggregation.

[0028] Furthermore, the data in the historical information database may be standardized. The standardized processing may be performed by using Z-score standardization. The Z-score standardization processing includes:

[0029]

[0030] Among them, Z is the data output after standardization, dimensionless; x is the data in the historical information database, dimensionless; μ is the mean of the data in all historical information databases, dimensionless; σ is the standard deviation of the data, dimensionless; M is the total number of data set samples in the historical information database, dimensionless; x i is the data value of the i-th sample in the historical information database, dimensionless.

[0031] It should be noted that the historical information database contains a large number of historical interaction records of various entities at different timestamps. At the same time, due to the limitations of actual conditions, various types of noise data will inevitably be generated when collecting historical interaction records, and these noise data need to be eliminated. If vector feature operations are performed without processing these data, it is likely to cause errors in the final hybrid self-attention mechanism results, or the correct hybrid self-attention mechanism cannot be formed. Therefore, by standardizing the data in the historical information database, the consistency of information in the historical information database can be effectively improved, and the Z-score standardization is applicable to a wide range of data types. For such historical data with timestamps, its relative position and degree of deviation can be accurately judged, thereby identifying and eliminating abnormal and noise data.

[0032] Furthermore, the completion method may also include a model training step, defining the loss function and learning process of the model by a hybrid self-attention mechanism representation, and obtaining negative samples by adopting a time-series negative sampling-based method, thereby improving the training effect of the model.

[0033] Furthermore, the negative samples can be obtained based on the time series negative sampling method as follows:

[0034]

[0035] Among them, D - is the negative sample set, (s - ,r,o,t) are randomly obtained negative samples, where s - is a randomly sampled head entity, D is a set of triples in the knowledge graph, and similarly, (s, r, o - ,t) is a randomly sampled negative sample, where o - is a randomly sampled tail entity.

[0036] According to the knowledge graph triple information and the negative samples obtained by sampling, the learning objective function is defined as a double boundary loss function.

[0037] loss=∑q∈D,q - ∈D - {[f(q)-γ1] + +∈[γ2-f(q - )] +},

[0038] Among them, γ1,γ2 are the boundary constants of positive and negative samples, ∈ is the compromise constant, f(q) is the score of a triple, [*] + is a positive function, defined as:

[0039]

[0040] Specifically, f(q) as a fraction of a triple can support multiple definitions. The specific triple fraction selected by the present invention is f(q)=|s+r-o+t|.

[0041] On the other hand, the present invention provides a temporal knowledge graph completion system based on hybrid attention, including an initial definition unit, a linear time representation model construction unit, a hybrid self-attention mechanism generation unit, a historical information aggregation unit and a model training unit.

[0042] Among them, the initial definition unit is configured to perform an initial definition of the time series knowledge graph, and the initial definition includes the data format, type and completion task of the time series knowledge graph, and at the same time collects knowledge graph data containing timestamps, and organizes the knowledge graph data into triples of q = (s, r, o, t), where s is the head entity, o is the tail entity, r is the relationship; t is the timestamp, indicating the validity period of the triple.

[0043] The linear time representation model construction unit is connected to the initial definition unit and is configured to construct a linear time representation model, and obtain the vector representation of the knowledge graph dataset through the linear time representation model.

[0044] Specifically, it includes: defining a vector representation model of time, treating the timestamp as an independent time segment in the existing time vector model; assigning an independent vector to each timestamp; adding a time smoothing constraint to ensure the order of time, and constructing a linear time representation model; the linear time representation model includes setting a reference time vector t0 and an incremental time vector Δ t , the representation vector of any time can be obtained by Get, where t i is the representation vector of any time; t0 is the reference time vector; Δ t is the incremental time vector; i is t i and t0.

[0045] The hybrid self-attention mechanism generation unit is connected to the linear time representation model construction unit, and is configured to collect the historical interaction records of each entity at different timestamps to form a historical information database, embed the vector of the knowledge graph dataset, and obtain the hybrid self-attention mechanism through the vector embedded in the knowledge graph dataset.

[0046] The historical information aggregation unit is connected to the hybrid self-attention mechanism generation unit, and is configured to aggregate historical information through the hybrid self-attention mechanism to obtain a temporal knowledge graph completion model.

[0047] The model training unit is connected to the historical information aggregation unit and is configured to define the loss function and learning process of the model through a hybrid self-attention mechanism representation, and to obtain negative samples by adopting a time-series negative sampling-based method, thereby improving the training effect of the model.

[0048] Furthermore, the negative samples can be obtained based on the time series negative sampling method as follows:

[0049]

[0050] Among them, D - is the negative sample set, (s - ,r,o,t) are randomly obtained negative samples, where s - is a randomly sampled head entity, D is a set of triples in the knowledge graph, and similarly, (s, r, o - ,t) is a randomly sampled negative sample, where o - is a randomly sampled tail entity.

[0051] According to the knowledge graph triple information and the negative samples obtained by sampling, the learning objective function is defined as a double boundary loss function.

[0052] loss=∑q∈D,q - ∈D - {[f(q)-γ1] + +∈[γ2-f(q - )] +},

[0053] Where q is a triple, q - is a negative sample, γ1,γ2 are the boundary constants of positive and negative samples, ∈ is a compromise constant. f(q) is the score of a triple, [*] + is a positive function, defined as:

[0054]

[0055] On the other hand, the present invention provides a computer device, which may include: a processor; a memory storing a computer program, and when the computer program is executed by the processor, the temporal knowledge graph completion method based on hybrid attention as described above is implemented.

[0056] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the temporal knowledge graph completion method based on hybrid attention as described above.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] 1. The present invention assigns an independent vector to each timestamp and integrates the timing constraints in the existing method into the time vector. A vector representation of any moment is constructed through a reference time vector and an incremental time vector. The aggregate utilization of historical data is achieved through the vector representation of timestamps.

[0059] 2. The present invention adopts a hybrid attention mechanism based on time and space, which can more accurately learn the connection between entity historical information and completion tasks from two modalities: semantics and timing.

[0060] 3. The present invention realizes the standardization of historical data by standardizing the history in the historical information database to facilitate the smooth mixing of the subsequent self-attention mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the embodiments of the present invention, constitute a part of this application, and do not constitute a limitation of the embodiments of the present invention. In the drawings:

[0062] Figure 1 A flowchart provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0064] Example 1

[0065] Figure 1 A flow chart of this embodiment is shown, which provides a method for completing a temporal knowledge graph based on hybrid attention.

[0066] Step 1: Initial definition step, initially define the time series knowledge graph, which includes the data format, type and completion task of the time series knowledge graph, collect the knowledge graph data containing timestamps, and organize the knowledge graph data into triples of q = (s, r, o, t), where s is the head entity, o is the tail entity, r is the relationship; t is the timestamp, indicating the validity period of the triple.

[0067] The temporal knowledge graph is completed by triples. Specifically, the completion may include predicting the missing tail entity by combining the knowledge graph dataset (s, r, ?, t), and / or predicting the missing head entity by combining the knowledge graph dataset (?, r, o, t).

[0068] The predicted missing tail entity is represented as,

[0069] The predicted missing head entity is represented as, in, Represents the candidate tail entity that minimizes f(*) Represents the candidate head entity that minimizes f(*) ; s,r,o,t∈C d is a vector of entities, relations, and time; d is the dimension of the vector.

[0070] Step 2: Define a linear time representation model. The linear time representation model is used to obtain the vector representation of the knowledge graph dataset and the vector representation of entities / relationships / time.

[0071] Specifically, it can include the following sub-steps:

[0072] 1) Define a vector representation model for time, and treat the timestamps as independent time segments in the existing time vector model.

[0073] 2) Assign a separate vector to each timestamp.

[0074] 3) Adding time smoothing constraints to ensure the order of time, and constructing a linear time representation model; the linear time representation model includes setting a reference time vector t0 and an incremental time vector Δ t , the representation vector of any time can be obtained by Get, where t i is the representation vector of any time; t0 is the reference time vector; Δ t is the incremental time vector; i is t i and t0.

[0075] It should be noted that in this step, an independent vector is assigned to each timestamp, and the timing constraints in the existing method are integrated into the time vector. Then, a vector representation of any moment is constructed through a baseline time vector and an incremental time vector, and finally the historical data is aggregated and utilized through the vector representation of the timestamp.

[0076] Step 3: Collect the historical interaction records of each entity at different timestamps to form a historical information database, embed the vector of the knowledge graph dataset, and obtain the hybrid self-attention mechanism through the vector embedded in the knowledge graph dataset.

[0077] Specifically, aggregating historical information may include capturing the dynamic changes of entities at different time points, combining time and structure information, capturing the interaction between time and structure, designing a hybrid self-attention mechanism by capturing the interaction between time and structure, and completing the aggregation of historical information through the designed hybrid self-attention mechanism.

[0078] The hybrid self-attention mechanism can include,

[0079] Define attention as, α(v1,v2,h)=h T σ(Wv1+Uv2),

[0080] Among them, σ(*) is the activation function, h, W, U are the learning parameters, α(v1, v2, h) is the attention function, and h T is the transpose of h, v1 and v2 are the representation vectors of input entity 1 and entity 2.

[0081] It can be seen that the attention defined in this embodiment combines the structural information of both temporal and spatial modalities.

[0082] First, in terms of time modality, the attention measures the influence of historical information at different times on the current completion target, that is,

[0083]

[0084] in, For s to s i The attention score at time t, α(s,s i ,t) is the attention function, s,s i , t are the entity, the entity’s neighbor, and the corresponding timestamp, exp is the natural base, N s represents the neighbor set of entity s, s j ∈N s is a neighbor entity of s.

[0085] Secondly, in the spatial modality, the attention measures the influence of different types of historical information on the current completion target, that is,

[0086] in, For s to s i The attention score on the relation r, α(s,s i ,r) is the attention function, s,s i , r are the entity, the neighbor of the entity, and the corresponding relationship type, exp is the natural base, N s represents the neighbor set of entity s, s j ∈N s is a neighbor entity of s.

[0087] Mixing temporal modal attention and spatial modal attention can obtain a hybrid self-attention mechanism.

[0088]

[0089] in, is the final entity representation vector, s is the original entity representation vector, is the neighbor information based on time aggregation, Represents neighbor information based on spatial aggregation.

[0090] In this step, after collecting the historical interaction records of each entity at different timestamps, a historical information database is formed. However, the data in the historical database may not all have the same scale. In order to mix the subsequent temporal modal attention and spatial modal attention, in this embodiment, the data in the historical information database is standardized, and the standard score of each data point in the historical information database is calculated to ensure that all historical information data have similar scales. The reason why the data in the historical information database is standardized in this embodiment is that the historical information database contains a large number of historical interaction records of each entity at different timestamps. At the same time, due to the limitations of actual conditions, various types of noise data will inevitably be generated when collecting historical interaction records, and these noise data need to be eliminated. If the vector feature operation is performed without processing these data, it is likely to cause the final hybrid self-attention mechanism result to be wrong, or the correct hybrid self-attention mechanism cannot be formed. Therefore, by standardizing the data in the historical information database, the consistency of the information in the historical information database can be effectively improved, and the Z-score standardization is applicable to a wide range of data types. For such historical data with timestamps, its relative position and degree of deviation can be accurately judged, thereby identifying and eliminating abnormal and noise data.

[0091] Specifically, the standardization process used in this embodiment is Z-score standardization, and the Z-score standardization process includes:

[0092]

[0093] Among them, Z is the data output after standardization, dimensionless; x is the data in the historical information database, dimensionless; μ is the mean of the data in all historical information databases, dimensionless; σ is the standard deviation of the data, dimensionless; M is the total number of data set samples in the historical information database, dimensionless; x i is the data value of the i-th sample in the historical information database, dimensionless.

[0094] By combining the techniques in step 2 and step 3, it is possible to implement the task of temporal knowledge graph completion while taking background information into consideration and being applicable to multiple graph inference and extrapolation predictions, thereby improving the accuracy and stability of knowledge graph completion.

[0095] Step 4: Aggregate historical information through the hybrid self-attention mechanism to obtain the temporal knowledge graph completion model.

[0096] Step 5: Model training step, define the model's loss function and learning process through a hybrid self-attention mechanism representation, and obtain negative samples by adopting a time-series negative sampling method to improve the model's training effect.

[0097] Specifically, the model training steps include obtaining negative samples based on time series negative sampling:

[0098]

[0099] Among them, D - is the negative sample set, (s - ,r,o,t) are randomly obtained negative samples, where s - is a randomly sampled head entity, D is a set of triples in the knowledge graph, and similarly, (s, r, o - ,t) is a randomly sampled negative sample, where o - is a randomly sampled tail entity.

[0100] According to the knowledge graph triple information and the negative samples obtained by sampling, the learning objective function is defined as a double boundary loss function.

[0101] loss=∑q∈D,q - ∈D - {[f(q)-γ1] + +∈[γ2-f(q - )] +},

[0102] Where q is a triple, q - is a negative sample, γ1,γ2 are the boundary constants of positive and negative samples, ∈ is a compromise constant. f(q) is the score of a triple, [*] + is a positive function, defined as:

[0103] Specifically, f(q) as a fraction of a triple can support multiple definitions. In this embodiment, the specific triple fraction selected is f(q)=|s+r-o+t|.

[0104] Example 2

[0105] This embodiment provides a temporal knowledge graph completion system based on hybrid attention, which includes an initial definition unit, a linear time representation model construction unit, a hybrid self-attention mechanism generation unit, a historical information aggregation unit and a model training unit.

[0106] Among them, the initial definition unit is configured to perform an initial definition of the time series knowledge graph, and the initial definition includes the data format, type and completion task of the time series knowledge graph, and at the same time collects knowledge graph data containing timestamps, and organizes the knowledge graph data into triples of q = (s, r, o, t), where s is the head entity, o is the tail entity, r is the relationship; t is the timestamp, indicating the validity period of the triple.

[0107] The linear time representation model construction unit is connected to the initial definition unit and is configured to construct a linear time representation model. The vector representation of the knowledge graph dataset is obtained through the linear time representation model, specifically including: defining a vector representation model of time, treating the timestamp as an independent time segment in the existing time vector model; and assigning an independent vector to each timestamp.

[0108] By adding time smoothing constraints to ensure the time sequence, a linear time representation model is constructed; the linear time representation model includes a reference time vector t0 and an incremental time vector Δ t , the representation vector of any time can be obtained by Get, where t i is the representation vector of any time; t0 is the reference time vector; Δ t is the incremental time vector; i is t i and t0; the hybrid self-attention mechanism generation unit is connected to the linear time representation model construction unit, and is configured to collect historical interaction records of each entity at different timestamps, embed the vector of the knowledge graph dataset, and obtain the hybrid self-attention mechanism through the vector embedded in the knowledge graph dataset.

[0109] The historical information aggregation unit is connected to the hybrid self-attention mechanism generation unit, and is configured to aggregate historical information through the hybrid self-attention mechanism to obtain a temporal knowledge graph completion model; the model training unit is connected to the historical information aggregation unit, and is configured to define the loss function and learning process of the model through the hybrid self-attention mechanism representation, and obtain negative samples by adopting a method based on temporal negative sampling, thereby improving the training effect of the model.

[0110] The method of obtaining negative samples based on time series negative sampling is:

[0111] Among them, D -is the negative sample set, (s - ,r,o,t) are randomly obtained negative samples, where s - is a randomly sampled head entity, D is a set of triples in the knowledge graph, and similarly, (s, r, o - ,t) is a randomly sampled negative sample, where o - is a randomly sampled tail entity.

[0112] According to the triple information of the knowledge graph and the negative samples obtained by sampling, the learning objective function is defined as a double-boundary loss function, loss = ∑q∈D,q - ∈D - {[f(q)-γ1] + +∈[γ2-f(q - )] +},

[0113] Where q is a triple, q - is a negative sample, γ1,γ2 are the boundary constants of positive and negative samples, ∈ is a compromise constant. f(q) is the score of a triple, [*] + is a positive function, defined as:

[0114] Example 3

[0115] This exemplary embodiment provides a computer device, including:

[0116] at least one processor;

[0117] A memory storing program instructions, wherein the program instructions are configured to be executed by the at least one processor

[0118] Execution, the program instructions include instructions for executing the temporal knowledge graph completion method based on hybrid attention described in Example 1.

[0119] Example 4

[0120] This embodiment provides a computer-readable storage medium.

[0121] The storage medium stores a computer program. When the computer program instructions are executed by the processor, the temporal knowledge graph completion method based on hybrid attention as described in Example 1 is implemented.

[0122] The computer-readable storage medium may be any data storage device that stores data that can be read by a computer system. For example, examples of computer-readable storage media may include: read-only memory, random access memory, read-only optical disk, magnetic tape, floppy disk, optical data storage device, and carrier wave (such as data transmission via the Internet via a wired or wireless transmission path).

[0123] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A temporal knowledge graph completion method based on hybrid attention, characterized in that: The completion method includes: S100, initially defining the time series knowledge graph, the initial definition includes the data format, type and completion task of the time series knowledge graph, collecting the knowledge graph data containing timestamps, and organizing the knowledge graph data into triples of q = (s, r, o, t), where s is the head entity, o is the tail entity, r is the relationship, and t is the timestamp, indicating the validity period of the triple; S200, constructing a linear time representation model, and obtaining a vector representation of a knowledge graph dataset and a vector representation of entities / relationships / time through the linear time representation model; S300, collecting historical interaction records of each entity at different timestamps to form a historical information database, embedding the vector of the knowledge graph dataset, and obtaining a hybrid self-attention mechanism through the vector embedded in the knowledge graph dataset; S400, aggregates historical information through a hybrid self-attention mechanism to obtain a temporal knowledge graph completion model.

2. The method for completing temporal knowledge graph based on hybrid attention according to claim 1, characterized in that: Integrate the knowledge graph data into a knowledge graph dataset, where the knowledge graph dataset is represented by G = (E, R, T), where G is the knowledge graph dataset, E is a real set, R is a relationship set, and T is a time set contained in the graph; The triples are used to complete the temporal knowledge graph, and the completion includes predicting the missing tail entity by combining the knowledge graph dataset (s, r, ?, t), and / or predicting the missing head entity by combining the knowledge graph dataset (?, r, o, t); The predicted missing tail entity is represented as, The predicted missing head entity is represented as, in, Represents the candidate tail entity that minimizes f(*) Represents the candidate head entity that minimizes f(*) s,r,o,t∈C d is a vector of entities, relations, and time; d is the dimension of the vector.

3. The method for completing temporal knowledge graph based on hybrid attention according to claim 1, characterized in that: The constructing of the linear time representation model further comprises: S210, defining a vector representation model of time, wherein the existing time vector model regards timestamps as independent time segments; S220, assigning an independent vector to each timestamp; S230, adding a time smoothing constraint to ensure the time sequence, and constructing a linear time representation model; the linear time representation model includes setting a reference time vector t0 and an incremental time vector Δ t , The representation vector for any time can be obtained by get, Among them, t i is the representation vector of any time; t0 is the reference time vector; Δ t is the incremental time vector; i is t i and t0.

4. The method for completing temporal knowledge graph based on hybrid attention according to claim 1, characterized in that: The aggregated historical information includes: Capture the dynamic changes of entities at different time points, combine time and structure information, capture the interaction between time and structure, design a hybrid self-attention mechanism by capturing the interaction between time and structure, and complete the aggregation of historical information through the designed hybrid self-attention mechanism.

5. The method for completing temporal knowledge graph based on hybrid attention according to claim 4, characterized in that: The hybrid self-attention mechanism includes: Attention is defined as α(v1,v2,h)=h T σ(Wv1+Uv2), Among them, σ(*) is the activation function, h, W, U are the learning parameters, α(v1, v2, h) is the attention function, and h T is the transpose of h, v1 and v2 are the representation vectors of input entity 1 and input entity 2; The attention combines the structural information of both temporal and spatial modalities; In the temporal mode, the attention measures the influence of historical information at different times on the current completion target, that is, in, For s to s i The attention score at time t, α(s,s i ,t) is the attention function, s is the entity, s i is the neighbor of the entity, t is the corresponding timestamp, exp is the natural base, N s represents the neighbor set of entity s, s j ∈N s is a neighbor entity of s; In the spatial modality, the attention measures the influence of different types of historical information on the current completion target, that is, in, For s to s i The attention score on the relation r, α(s,s i ,r) is the attention function, s is the entity, s i is the neighbor of the entity, r is the corresponding relationship type, exp is the natural base, N s represents the neighbor set of entity s, s j ∈N s is a neighbor entity of s; The temporal modal attention and the spatial modal attention are mixed to obtain a hybrid self-attention mechanism. in, is the final entity representation vector, s is the original entity representation vector, is the neighbor information based on time aggregation, Represents neighbor information based on spatial aggregation.

6. The method for completing temporal knowledge graph based on hybrid attention according to claim 1, characterized in that: The completion method also includes a model training step, which defines the loss function and learning process of the model by expressing the hybrid self-attention mechanism, and obtains negative samples by adopting a method based on time series negative sampling, thereby improving the training effect of the model.

7. The method for completing a temporal knowledge graph based on hybrid attention according to claim 6, characterized in that: The method of obtaining negative samples based on time series negative sampling is: Among them, D - is the negative sample set, (s - ,r,o,t) are randomly obtained negative samples, where s - is a randomly sampled head entity, D is a set of triples in the knowledge graph, and similarly, (s, r, o - ,t) is a randomly sampled negative sample, where o - is the randomly sampled tail entity; According to the knowledge graph triple information and the negative samples obtained by sampling, the learning objective function is defined as a double boundary loss function. loss=∑q∈D,q - ∈D - {[f(q)-γ1] + +∈[γ2-f(q - )] + }, Among them, γ1,γ2 are the boundary constants of positive and negative samples, ∈ is the compromise constant, f(q) is the score of a triple, [*] + is a positive function, defined as:

8. A temporal knowledge graph completion system based on hybrid attention, characterized in that: The completion system comprises: Initial definition unit, linear time representation model construction unit, hybrid self-attention mechanism generation unit, historical information aggregation unit and model training unit, among which, The initial definition unit is configured to initially define the time series knowledge graph, the initial definition includes the data format, type and completion task of the time series knowledge graph, and collects the knowledge graph data containing timestamps, and organizes the knowledge graph data into triples of q=(s, r, o, t), where s is the head entity, o is the tail entity, r is the relationship; t is the timestamp, indicating the validity period of the triple; The linear time representation model construction unit is connected to the initial definition unit and is configured to construct a linear time representation model. The vector representation of the knowledge graph dataset is obtained through the linear time representation model, which specifically includes: defining a vector representation model of time, treating the timestamp as an independent time segment in the existing time vector model; assigning an independent vector to each timestamp; adding a time smoothing constraint to ensure the order of time, and constructing a linear time representation model; the linear time representation model includes setting a reference time vector t0 and an incremental time vector Δ t , the representation vector of any time can be obtained by Get, where t i is the representation vector of any time; t0 is the reference time vector; Δ t is the incremental time vector; i is t i The time difference between t and t0; The hybrid self-attention mechanism generation unit is connected to the linear time representation model construction unit, and is configured to collect historical interaction records of each entity at different timestamps to form a historical information database, embed the vector of the knowledge graph dataset, and obtain the hybrid self-attention mechanism through the vector embedded in the knowledge graph dataset; The historical information aggregation unit is connected to the hybrid self-attention mechanism generation unit and is configured to aggregate historical information through the hybrid self-attention mechanism to obtain a temporal knowledge graph completion model; The model training unit is connected to the historical information aggregation unit and is configured to define the loss function and learning process of the model through a hybrid self-attention mechanism representation, and obtain negative samples by adopting a time-series negative sampling method, thereby improving the training effect of the model; The method of obtaining negative samples based on time series negative sampling is: Among them, D - is the negative sample set, (s - ,r,o,t) are randomly obtained negative samples, where s - is a randomly sampled head entity, D is a set of triples in the knowledge graph, and similarly, (s, r, o - ,t) is a randomly sampled negative sample, where o - is the randomly sampled tail entity; According to the knowledge graph triple information and the negative samples obtained by sampling, the learning objective function is defined as a double boundary loss function. loss=∑q∈D,q - ∈D - {[f(q)-γ1] + +∈[γ2-f(q - )] + }, where q is a triple, q - is a negative sample, γ1,γ2 are the boundary constants of positive and negative samples, ∈ is a compromise constant. f(q) is the score of a triple, [*] + is a positive function, defined as:

9. A computer device, characterized in that: The computer device comprises: processor; A memory storing a computer program, which, when executed by a processor, implements the temporal knowledge graph completion method based on hybrid attention as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the temporal knowledge graph completion method based on hybrid attention as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Time sequence knowledge graph completion method and system based on multi-dimensional rotation

    CN117808086A

  • Recursive timing knowledge graph completion method and apparatus

    WO2022052374A1

Cited By

  • Multi-modal knowledge graph completion method based on composite operation

    CN119250076A

  • Information recommendation method and device based on time sequence knowledge graph

    CN120821837A