Knowledge graph embedding representation method and related equipment

Abstract

The embodiment of the invention discloses a knowledge graph embedding representation method and related equipment. The method comprises the steps that N related entities of each entity in M entities of a target knowledge graph and K concepts corresponding to each related entity can be obtained from a preset knowledge base; then determining semantic relevancy between each entity and each related entity of the entity, and determining a first entity embedding representation of each related entity according to the corresponding K concepts; and then modeling the embedded representation of the entity / relationship according to the first entity embedded representation and the semantic relevancy, and training a model in combination with an attention mechanism and a preset model training method to obtain the embedded representation of the entity / relationship. By adopting the embodiment of the invention, the background content of the entity can be captured, the semantic extension of the entity is realized, and the representation capability of the embedded representation model under the complex relationship between the entities and the accuracy and comprehensiveness of knowledge graph completion are improved.

Description

[0001] This application is a divisional application. The application number of the original application is 201910583845.0, and the original application date is June 29, 2019. The entire contents of the original application are incorporated into this application by reference. Technical Field

[0002] The present application relates to the field of information processing, and in particular to a knowledge graph embedding representation method and related equipment. Background Art

[0003] Knowledge graph is a highly structured information representation that can be used to describe the relationships between various entities in the real world. Entities are objective and distinguishable things, such as names of people, places, and movie titles. A typical knowledge graph consists of a large number of [head entity, entity relationship, tail entity] triples, each of which represents a fact. Figure 1 As shown in the figure, the fact triples included in the knowledge graph are [Jay Chou, blood type, O type], [Jay Chou, nationality, Han nationality], [Secret, producer, Jiang Zhiqiang], etc. At present, there are many large-scale, open-domain knowledge graphs, such as Freebase and WordNet, but they are still far from complete. However, the completeness of the knowledge graph determines its application value. In order to improve the knowledge graph and improve its completeness, the existing knowledge graph can be embedded and represented first, and then the knowledge graph can be completed based on the embedded representation of entities / relationships. However, the existing knowledge graph embedding representation and completion methods are limited by the structural sparsity of the graph on the one hand, and the external information features used are easily affected by the size of the text corpus on the other hand, resulting in unsatisfactory completion of the knowledge graph. Summary of the invention

[0004] The embodiments of the present application provide a knowledge graph embedding representation method and related equipment, which can realize the semantic extension of entities, thereby improving the representation ability of complex relationships between entities in the knowledge graph, as well as the accuracy and comprehensiveness of knowledge graph completion.

[0005] In a first aspect, an embodiment of the present application provides a knowledge graph completion method, comprising: first obtaining M entities in a target knowledge graph, wherein the M entities include entity 1, entity 2, ..., entity M, and M is an integer greater than 1; then obtaining N related entities of entity m among the M entities, and K concepts corresponding to related entity n among the N related entities from a preset knowledge base, wherein the N related entities include related entity 1, related entity 2, ..., related entity N, wherein N and K are integers not less than 1, m = 1, 2, 3, ..., M and n = 1, 2, 3 ..., N, and entity m is placed between the N related entities of the entity, and between the related entity n and the corresponding K concepts; secondly, the semantic relevance between each entity in the M entities and each related entity of the entity is determined, and the first entity embedding representation of each related entity is determined according to the corresponding K concepts; finally, based on the first entity embedding representation and the semantic relevance, the embedding representation of the entity relationship between M entities and M entities is modeled to obtain the embedding representation model, and the embedding representation model is trained to obtain the second entity embedding representation of each entity and the relational embedding representation of the entity relationship. By modeling the entity / relationship embedding representation in the knowledge graph through a two-layer information fusion mechanism such as entity-related entity-related entity, the semantic extension of the entity can be effectively realized, thereby improving the completion effect of the knowledge graph.

[0006] In a possible design, each of the K concepts of the related entity n can be vectorized to obtain the word vector of each concept; the word vectors of the K concepts of the related entity n are averaged and summed to obtain the first entity embedding representation of the related entity n, n = 1, 2, 3..., N. Representing the related entity by the word vector of the concept is equivalent to the first information fusion from the concept to the related entity, preparing for the second information fusion from the related entity to the entity.

[0007] In another possible design, the unigram text embedding representation corresponding to each entity can be determined based on the semantic relevance and the first entity embedding representation of N related entities; the common related entities of every two entities in M ​​entities can be determined based on the N related entities; the bigram text embedding representation corresponding to every two entities can be determined based on the semantic relevance and the first entity embedding representation of the common related entities; the embedding representation model can be determined based on the unigram text embedding representation and the bigram text embedding representation. The unigram text embedding representation is equivalent to the vectorized representation of the content of the aligned text of the entity, which is used to capture the background information of the entity. The bigram text embedding representation is equivalent to the vectorized representation of the content intersection of the aligned texts corresponding to the two entities, which changes with the change of the entity and is used to model the relationship, so that the embedding representation of complex relationships such as one-to-many, many-to-one and many-to-many can be realized.

[0008] In another possible design, the unigram text embedding representation and the bigram text embedding representation can be mapped to the same vector space to obtain semantically enhanced unigram text embedding representation and bigram text embedding representation; an embedding representation model is established based on the semantically enhanced unigram text embedding representation and the semantically enhanced bigram text embedding representation. Since the unigram text embedding representation corresponding to a single entity and the bigram text embedding representation corresponding to two entities are usually not in the same vector space, the computational complexity is increased. In order to overcome this defect, the two can be mapped to the same vector space.

[0009] In another possible design, the semantic relevance can be used as the first weight coefficient of each related entity; and the first entity embedding representations of N related entities are weighted and summed according to the first weight coefficient to obtain a unigram text embedding representation. Among them, the semantic relevance can reflect the degree of association between the entity and the related entities to a certain extent. Therefore, using the semantic relevance as a weight coefficient can improve the accuracy of the tendency of the semantic expression of the entity after information fusion.

[0010] In another possible design, the minimum semantic relevance between the common related entity and the semantic relevance between each two entities is used as the second weight coefficient of the common related entity; the first entity embedding representation of the common related entity is weighted and summed according to the second weight coefficient to obtain the binary text embedding representation. The binary text embedding representation is equivalent to the vectorized representation of the content intersection of the aligned texts corresponding to the two entities. The minimum semantic relevance can improve the accuracy of the content intersection, thereby improving the effectiveness and accuracy of the binary text embedding representation.

[0011] In another possible design, the loss function of the embedding representation model is determined; the embedding representation model is trained according to a preset training method to minimize the function value of the loss function, thereby obtaining the second entity embedding representation and the relationship embedding representation. The loss function represents the Euclidean distance between the sum vector between the head entity and the relationship of the known fact triple and the tail entity. Therefore, minimizing the function value of the loss function can make the sum vector closest to the tail entity, thereby realizing the embedding representation of the knowledge graph based on the TransE framework.

[0012] In another possible design, the function value of the loss function is associated with the embedding representation of each entity and entity relationship, and the unigram text embedding representation; therefore, the embedding representation of each entity and entity relationship can be initialized first to obtain the initial entity embedding representation and the initial relationship embedding representation; then the first weight coefficient is updated according to the attention mechanism to update the unigram text embedding representation, and the initial entity embedding representation and the initial relationship embedding representation are iteratively updated according to the training method. Among them, the weight coefficients of the relevant entities in the unigram text embedding representation can be continuously learned through the attention mechanism, thereby continuously improving the accuracy of the background content of each entity captured. Therefore, updating the initial entity embedding representation and the initial relationship embedding representation in combination with the updated unigram text embedding representation can effectively improve the usefulness of the final entity and relationship embedding representation for knowledge graph completion.

[0013] In another possible design, the target knowledge graph includes known fact triples, which include two entities among M entities and one entity relationship. Therefore, after obtaining the second entity embedding representation of each entity and the relational embedding representation of the entity relationship, the entity relationship included in the known fact triple can be replaced with other entity relationships between N entities, or one entity included in the known fact triple can be replaced with other entities among N entities to obtain a predicted fact triple. The recommendation score of the predicted fact triple is determined based on the second entity embedding representation of the entity in the predicted fact triple and the relational embedding representation of the entity relationship. Then, the predicted fact triple is added to the target knowledge graph based on the recommendation score. The knowledge coverage of the target knowledge graph can be improved, thereby improving the use value of the knowledge graph.

[0014] In the second aspect, an embodiment of the present application provides an embedded representation device for a knowledge graph, which is configured to implement the method and functions performed by the embedded representation device for the knowledge graph in the first aspect above, and is implemented by hardware / software, and its hardware / software includes units corresponding to the above functions.

[0015] In a third aspect, an embodiment of the present application provides an embedded representation device for a knowledge graph, comprising: a processor, a memory, and a communication bus, wherein the communication bus is used to realize connection and communication between the processor and the memory, and the processor executes a program stored in the memory to implement the steps in a method for embedded representation of a knowledge graph provided in the first aspect above.

[0016] In a possible design, the knowledge graph embedding representation device provided in the embodiment of the present application may include a module for executing the behavior of the knowledge graph completion device in the above method design. The module may be software and / or hardware.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the methods in the above aspects.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the above-mentioned methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0020] Figure 1 It is a structural schematic diagram of a knowledge graph provided by the background technology;

[0021] Figure 2 It is a structural diagram of an application software system provided by an embodiment of the present application;

[0022] Figure 3 It is a flowchart of a knowledge graph embedding representation method provided in an embodiment of the present application;

[0023] Figure 4 It is a flowchart of a knowledge graph embedding representation method provided by another embodiment of the present application;

[0024] Figure 5 It is a schematic diagram of a knowledge graph completion effect provided in an embodiment of the present application;

[0025] Figure 6 It is a structural schematic diagram of a knowledge graph embedding representation device provided in an embodiment of the present application;

[0026] Figure 7 It is a structural diagram of a knowledge graph embedding representation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0028] Please refer to Figure 2 , Figure 2It is a structural diagram of an application software system provided by an embodiment of the present application. As shown in the figure, the application software system includes a knowledge graph completion module, a knowledge graph storage module, a query interface and a knowledge graph service module. Among them, the knowledge graph completion module may include an entity / relationship embedding representation unit and an entity / relationship prediction unit. The knowledge graph service module can provide services such as intelligent search, intelligent question and answer, and intelligent recommendation based on the knowledge graph stored in the knowledge graph storage module. In this system, the knowledge graph completion module can receive a text corpus and a known knowledge graph input from the outside world, and complete the known knowledge graph according to the preset knowledge graph completion method and text corpus, that is, add new fact triples to the known knowledge graph. Among them, the entity / relationship embedding representation unit can embed entities and entity relationships in the knowledge graph, wherein the entities and relationships in the knowledge graph are text or other forms that cannot be operated, and the embedded representation means mapping the semantic information of each entity and each entity relationship to a multidimensional vector space, expressed as a vector. The entity / relationship prediction unit can infer new fact triples based on the obtained vectors and add the new fact triples to the known knowledge graph. The knowledge graph storage module can store the completed known knowledge graph. The knowledge graph service module can apply the knowledge graph stored in the knowledge graph storage module to tasks in various fields through the query interface. For example, information matching the keywords entered by the user is queried from the stored completed known knowledge graph and displayed to the user.

[0029] At present, the knowledge graph completion methods used by the knowledge graph completion module may include: (1) Methods based on structural information - inferring new triples from existing fact triples in the knowledge graph, such as TransE and TransR models. In actual application, it is found that this method is often easily limited by the structural sparsity of the graph, and cannot effectively embed the complex entity relationships (one-to-many, many-to-one relationships) in the knowledge graph to be completed, resulting in poor knowledge graph completion effect. (2) Methods based on information fusion - fusing external information (i.e., text corpus) to extract new entities and new fact triples. However, this method usually only uses the features of co-occurring words, and this feature is often easily limited by the size of the corpus, resulting in certain errors in the knowledge graph completion results. In order to solve the problem of unsatisfactory knowledge graph completion effect, the embodiments of the present application provide the following knowledge graph embedding representation method.

[0030] Please refer to Figure 3 , Figure 3 This is a flow chart of a knowledge graph embedding representation method provided in an embodiment of the present application. The method includes but is not limited to the following steps:

[0031] S301, obtain M entities in the target knowledge graph.

[0032] In the specific implementation, the knowledge graph can be regarded as a network graph containing multiple nodes, where multiple nodes can be connected to each other, each node represents an entity, and the edge connecting two nodes represents the relationship between the two connected entities. Where M is an integer not less than 1, and the M entities include entity 1, entity 2, ..., entity M. The target knowledge graph can be any knowledge graph that needs to be embedded and completed. For example, Figure 1 As shown, entities such as "Jay Chou", "Tamkang High School", "Taiwan" and "Han nationality" can be obtained from the target knowledge graph.

[0033] S302, obtaining N related entities of entity m among M entities and K concepts corresponding to related entity n among N related entities from a preset knowledge base.

[0034] In the specific implementation, N and K are both integers not less than 1, and the N related entities include related entity 1, related entity 2, ..., related entity M, and m = 1, 2, 3, ..., M, and n = 1, 2, 3, ..., N. The knowledge base includes a large number of texts and pages. First, each entity in the target knowledge graph can be automatically linked to the text in the knowledge base using, but not limited to, entity linking technology, and the related entities of the entity are obtained. For an entity in the target knowledge graph, the related entity refers to an entity that is semantically related to the entity, or an entity that is contextually related to the entity. For example, "Zhang Yimou" and "The Flowers of War". Among them, the available entity linking technologies include AIDA technology, Doctagger technology, and LINDEN technology. Then, the related entity can be linked to a page in the knowledge base. After removing the punctuation marks and stop words on the page, the concept corresponding to the related entity can be obtained from the page. Among them, the wiki tool can be used, but not limited to, to automatically identify all concepts in the page, and then extract names and place names from the identified concepts as the concepts corresponding to the related entity. For example, if the related entity is "David", the corresponding page that "David" links to is usually a page that introduces the basic information of David, including that David was born in Hawaii, USA, graduated from Harvard University, and his wife is Michelle. Then, the place names "USA", "Hawaii" and "Harvard University" and the name "Michelle" can be extracted from the page as the four concepts corresponding to the related entity "David". In the field of knowledge bases, concepts are slightly broader than entities. In most cases, concepts can be directly regarded as entities, and entities can be directly regarded as concepts. However, there is no unified standard for whether to distinguish between concepts and entities and how to distinguish between them in different knowledge bases.

[0035] S303, determining the semantic relevance between each entity in the M entities and each related entity of the entity, and determining a first entity embedding representation of each related entity according to the corresponding K concepts.

[0036] In the specific implementation, on the one hand, we can first determine the i-th entity e in the target knowledge graph i The actual total number of N related entities is E1, and e i The jth related entity The actual total number of corresponding K concepts is E2. Then, based on E1 and E2, the entity e can be calculated according to formula (1): i Related entities The semantic relevance between ij .

[0037]

[0038] , where W is the total number of entities contained in the preset knowledge base. E1∩E2 means e i E1 related entities and The number of entities and concepts with the same text content in E2 concepts. i There are three related entities: "China", "Huaxia" and "Ancient Civilization". If there is a concept of "China", then i and They each have related entities and concepts whose text content is "China", that is, E1∩E2 is 1. Among them, min(a,b) means finding the minimum value of a and b, and max(a,b) means finding the maximum value of a and b.

[0039] It should be noted that in step S302, R related entities of each entity can usually be obtained through entity linking technology, where R is greater than N. Therefore, the N related entities mentioned above can be selected from the R related entities according to the semantic relevance. For example, the R related entities can be sorted in order from high to low in semantic relevance, and then the top N related entities can be selected as the above N related entities. All related entities whose semantic relevance among the R related entities is greater than a preset threshold can also be used as the above N related entities.

[0040] On the other hand, a word vector generation model (such as the word2vec model) can be used to vectorize each of the K concepts to obtain the word vector of each concept, and then the word vectors of each concept are averaged and summed, and the average summation result is used as the first entity embedding representation of the corresponding related entity.

[0041] For example: The word vector set composed of the word vectors of the corresponding K concepts is Where μ is a G-dimensional row vector, and the size of G can be set according to the actual scenario and / or the scale of the knowledge graph. The first entity embedding representation It can be calculated according to formula (2).

[0042]

[0043] S304: Model the embedding representations of the M entities and the entity relationships between the M entities according to the first entity embedding representation and the semantic relevance to obtain an embedding representation model.

[0044] In the specific implementation, the entity e in the target knowledge graph can be i As the central entity, e i The M related entities are as well as The first entity embedding representations are The modeling steps of the embedding representation model include:

[0045] (1) According to And the semantic relevance y between each related entity and the central entity ij , computing center entity e i The corresponding unary text embedding representation n(e i ), where the semantic relevance can be used as the first weight coefficient of each related entity, and then the first weight coefficient is used to Perform weighted summation and get

[0046]

[0047] , the coefficients in the above formula are Used to normalize the first weight coefficient. Among them, the one-element text embedding representation can be regarded as the central entity e i The linked text is a vectorized representation of the content of the text where the related entity is located.

[0048] (2) Based on the N related entities of each entity, determine the common related entities between every two entities. Among them, the common related entities of two entities may be one or more, or there may be no common related entities. For example, the related entities of the entity "Zhang Yimou" include "return", "hero", and "my father and mother". The related entities of the entity "Gong Li" include "return" and "Farewell My Concubine", so the common related entity of "Zhang Yimou" and "Gong Li" is "return". Then, based on the semantic relevance of each entity in every two entities to the common related entity, and the first entity embedding representation of the common related entity, determine the binary text embedding representation corresponding to every two entities. The binary text embedding representation can be regarded as two central entities e i The vectorized representation of the content intersection of the linked text. The minimum semantic relevance between the common related entity and each two entities can be used as the second weight coefficient of the common related entity. The common related entities are then weighted and summed according to the second weight coefficient, and the result of the weighted sum is used as the binary text embedding representation. For example, entity e i and e j The commonly related entities include Then i and e j The corresponding bigram text embedding representation n(e i ,e j )for

[0049]

[0050] Among them, y ik and jk are respectively the first entity embedding representation Corresponding related entities With e i and e j The semantic relevance between them. min(y ik ,y jk ) is The second weight coefficient of , 1 / Z is used to normalize the second weight coefficient. Therefore,

[0051]

[0052] It should be noted that when e i and e j When there is no common related entity, n(e i ,e j ) is set to the zero vector.

[0053] (3) Determine the embedding representation model based on the unigram text embedding representation and the bigram text embedding representation. Specifically, based on the existing knowledge graph embedding representation model - TransE model, the unigram text embedding representation and the bigram text embedding representation can be mapped to the same vector space to obtain the semantically enhanced unigram text embedding representation and the bigram text embedding representation. Then, the embedding representation model is established based on the semantically enhanced unigram text embedding representation and the bigram text embedding representation. Considering that the embedding representation model involves both the embedding representation of entities and the embedding representation of relationships, the modeling process can be explained from the perspective of fact triples. Specifically, for the known fact triple [h, r, t] in the target knowledge graph, according to the above two steps (1) and (2), the unigram text embedding representations corresponding to h and t are n(h) and n(t), respectively, and the bigram text embedding representation n(h, t) corresponding to h and t can be obtained. Then, according to the TransE model, n(h), n(t) and n(h, t) are mapped to obtain,

[0054]

[0055] Where A and B are the predetermined entity mapping matrix and relationship mapping matrix. h, t and r are the model parameters corresponding to h, t and r in the TransE model. That is, the semantically enhanced unigram text embedding representation corresponding to n(h) and n(t), and That is the semantically enhanced binary text embedding representation corresponding to n(h,t).

[0056] Then, we can continue to use the modeling idea of ​​the TransE model to and Based on this, the embedding representation model of the target knowledge graph is modeled as

[0057]

[0058] In order to enhance the robustness of the entity / relation embedding representation of the model, the components can be regularized so that ||h||2≤1, ||t||2≤1, ||r||2≤1, ||n(h)*A||2≤1, ||n(t)*A||2≤1, and ||n(h,t)*B||2≤1. Among them, ‖‖2 represents the second norm.

[0059] It should be noted that, as shown in formula (8), for different head entities h and / or tail entities t, The loss function of the traditional TransE model is f′(h, t, r) = ||h+rt||2. Therefore, compared with the traditional TransE model, the embedding representation model shown in formula (9) provided in the embodiment of the present application can handle complex relationships of one-to-many, many-to-one and many-to-many. The specific reason is that for different h and t, the loss function of the traditional TransE model is f′(h, t, r) = ||h+rt||2. (i.e., entity relationships) have different representations, while in f′(h, t, r), r does not change with h and t. In addition to the TransE model, other frameworks of graph embedding representation models can also be used, such as TransR, TransH, etc. Among them, TransE, TransR and TransH are Trans series models. The basic idea of ​​the Trans series model is to make h+r as equal to t as possible by continuously adjusting the model parameters h, t and r corresponding to h, r and t, that is, h+r≈t. The only difference is that the loss functions (model functions) of multiple models are different.

[0060] S305, training the embedding representation model to obtain a second entity embedding representation of each entity and a relational embedding representation of the entity relationship.

[0061] In a specific implementation, the loss function of the embedding representation model can be determined first. Based on the basic idea of ​​the TransE model, the loss function of the embedding representation model shown in formula (9) provided in the entity example of this application can be determined as:

[0062] L=∑ (h,r,t)∈S ∑ (h′,r′,t′)∈S′ max(0,f(h,t,r)+λ-f(h′,r′,t′)) (10)

[0063] , where λ is a hyperparameter greater than 0, S is the set of correct triples consisting of known fact triples in the target knowledge graph, and S′ is the set of incorrect triples consisting of incorrect fact triples constructed based on known fact triples. For example, [Secret that cannot be told, producer, Jiang Zhiqiang] is a known fact triple, then the incorrect fact triple [Secret that cannot be told, producer, Jay Chou] can be constructed based on the known fact triple.

[0064] Then, the embedding representation model is trained according to a preset training method to minimize the function value of the loss function, thereby obtaining a second entity embedding representation and a relationship embedding representation. The model may be trained according to, but not limited to, a gradient descent method, that is, the function value of the loss function is minimized, and the model parameters h, t, and r are iteratively updated according to the gradient descent method until the function value of the loss function converges or the number of iterative updates is greater than a preset number. Then, h and t obtained from the last update are used as the corresponding entity embedding representations of h and t, and r is used as the relationship embedding representation of r.

[0065] In an embodiment of the present application, M entities in the target knowledge graph can be first obtained; then N related entities of entity m in the M entities and K concepts corresponding to related entity n in the N related entities are obtained from a preset knowledge base, where m = 1, 2, 3, ..., M, n = 1, 2, 3, ..., N; then the semantic relevance between each entity in the M entities and each related entity of the entity is determined, and the first entity embedding representation of each related entity is determined according to the corresponding K concepts; finally, based on the first entity embedding representation and the semantic relevance, the embedding representation of the entity relationship between M entities and M entities is modeled to obtain an embedding representation model, and the embedding representation model is trained to obtain the second entity embedding representation of each entity and the relational embedding representation of the entity relationship. On the basis of the TransE model, the semantically extended embedding representation of entities and entity relationships can be realized through two-layer information fusion of entity-related entity-related entity, so that the final embedded representation model can effectively handle complex relationships such as one-to-many, many-to-one and many-to-many in the knowledge graph.

[0066] Please refer to Figure 4 , Figure 4 This is a flow chart of a knowledge graph embedding representation method provided by another embodiment of the present application. The method includes but is not limited to the following steps:

[0067] S401, obtaining M entities in the target knowledge graph. This step is the same as S301 in the previous embodiment, and will not be described in detail.

[0068] S402, obtaining N related entities of entity m among M entities and K concepts corresponding to related entity n among N related entities from a preset knowledge base. This step is the same as S302 in the previous embodiment, and will not be described in detail.

[0069] S403, determining the semantic relevance between each entity in the M entities and each related entity of the entity, and determining the first entity embedding representation of each related entity according to the corresponding K concepts. This step is the same as S303 in the previous embodiment, and will not be repeated here.

[0070] S404: Based on the first entity embedding representation and the semantic relevance, the embedding representation of the M entities and the entity relationships between the M entities is modeled to obtain an embedding representation model. This step is the same as S304 in the previous embodiment, and this step will not be repeated.

[0071] S405, determining a loss function of the embedding representation model.

[0072] In a specific implementation, the loss function of the embedding representation model can be determined as a function as shown in formula (10). Combining formulas (6)-(9), it can be seen that the function value of the loss function is not only associated with the embedding representations h, t of entities h and t in the target knowledge graph, and the embedding representation r of entity relationship r, but also with the unigram text embedding representations n(h), n(t) corresponding to h and t, and the bigram text embedding representation n(h, t) corresponding to h and t.

[0073] S406, initializing the embedding representation of each entity and entity relationship to obtain an initial entity embedding representation and an initial relationship embedding representation.

[0074] In a specific implementation, h, t and r can be initialized arbitrarily but are not limited to it. For example, each dimension of h, t and r can be randomly set to a value between 0 and 1. And after h, t and r are initialized, their moduli need to be normalized.

[0075] S407, updating the first weight coefficient according to the attention mechanism to update the unary text embedding representation, and iteratively updating the initial entity embedding representation and the initial relationship embedding representation according to a preset training method, thereby training the embedding representation model to obtain the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship.

[0076] In a specific implementation, on the one hand, updating the first weight coefficient according to the attention mechanism to update the unigram text embedding representation includes:

[0077] First, according to the first weight coefficient y ij , calculate β ij ,

[0078]

[0079] Here, tanh represents the inverse tangent function. V, b, and ω are all parameters learned by the attention mechanism. Then, according to β ij Update the first weight coefficient to obtain the updated first weight coefficient α ij ,

[0080]

[0081] In formula (10), exp represents an exponential function with the natural constant e=2.71828 as the base.

[0082] In the process of training the embedding representation model, the attention mechanism will be executed simultaneously to learn the importance of each relevant entity in representing the text content of the corresponding text, and update the weight of each relevant entity in the unigram text embedding representation of the corresponding text according to the result of each learning, that is, update the parameter in formula (11) V, b, and ω. So β ij The value of is constantly updated during the model training process, so that α ij The value is also constantly updated.

[0083] For example, the related entities corresponding to the entity "Zhang Yimou" include "return" and "hero". In the aligned text corresponding to "Zhang Yimou" that mainly introduces the realistic themes directed by Zhang Yimou, the attention mechanism can gradually learn that the weight of "return" will be greater than the weight of "hero".

[0084] On the other hand, the initial entity embedding representation of each entity and the initial relationship embedding representation of each entity relationship can be iteratively updated according to a preset model training method (such as gradient descent).

[0085] In summary, the essence of embedding representation model training is to continuously update the unigram text embedding representation n(h), n(t), and the embedding representations h, t, and r of entities and entity relationships with the purpose of minimizing the function value of the loss function until the loss function converges or the number of iterative updates exceeds the preset number. Then, h and t obtained from the last update are used as the second entity embedding representation of h and t, and r obtained from the last update is used as the relationship embedding representation of r.

[0086] Optionally, after obtaining the second entity embedding representation of each entity in the target knowledge graph and the relational representation of each entity relationship, the knowledge graph can be completed based on the embedding representation, that is, new fact triples can be added to the knowledge graph. Specifically, the following steps may be included:

[0087] (1) Replace the entity relationship included in the known fact triples in the target knowledge graph with other entity relationships included in the knowledge graph, or replace one entity included in the known fact triples with other entities included in the knowledge graph to obtain predicted fact triples.

[0088] For example, Figure 1As shown in the figure, the knowledge graph includes the known fact triple [Jay Chou, nationality, Han nationality]. Then, one of the entities "Jay Chou" can be replaced with another entity "Jiang Zhiqiang" in the knowledge graph to obtain the predicted fact triple [Jiang Zhiqiang, nationality, Han nationality]. Similarly, "Han nationality" can be replaced with "Taiwan" to obtain another predicted fact triple [Jay Chou, nationality, Taiwan].

[0089] (2) Determine the recommendation score of the predicted fact triplet based on the second entity embedding representation of the entity in the predicted fact triplet and the relational embedding representation of the entity relationship. The recommendation score can be used to measure the prediction accuracy of each predicted fact triplet, and can also be regarded as the probability that the predicted fact triplet is an actual fact triplet. Among them, the model function of the embedding representation model of the entity / entity relationship (such as formula (9)) can be used as the score function of the model, and then the second entity embedding representation of the entity in the predicted fact triplet and the relational embedding representation of the entity relationship are substituted into the score function for calculation, and the recommendation score of the preset fact triplet is determined according to the calculated function value. Among them, in the TransE framework, since the wrong fact triplet is compared with the correct fact triplet, the recommendation score of the wrong fact triplet is higher than that of the correct fact triplet. and The distance between them is farther, so substitute it into the score function The function value calculated is larger than the correct fact triple. In this case, in order to comply with the general recommendation logic, the preset maximum score that can be achieved, that is, the difference between the full score of the recommendation score (such as 1 point, 10 points, 100 points, etc.) and the function value of f(h, t, r) can be used as the recommendation score.

[0090] (3) According to the recommendation score, the predicted fact triple is added to the target knowledge graph. The recommendation score of each predicted fact triple can be compared with a preset threshold, and the predicted fact triple with a recommendation score greater than the preset threshold can be added to the target knowledge graph, but is not limited to being added to the target knowledge graph. The preset threshold can be 0.8, 8, or 80, etc.

[0091] For example: Figure 1 The knowledge graph shown in Figure 2 is based on the score function The predicted fact triples [Jiang Zhiqiang, nationality, Han] and [Jay Chou, nationality, Taiwan] are 0.85 and 0.34, and then because 0.85 is greater than 0.8 and 0.34 is less than 0.8, [Jiang Zhiqiang, nationality, Han] is added to the knowledge graph, and the following is obtained: Figure 5The completed knowledge graph is shown in the figure. As shown in the figure, before completion, there is no relationship between the entities "Jiang Zhiqiang" and "Han nationality" in the target knowledge graph. Through the embedded representation of entities / relationships, it can be inferred that there is an entity relationship "nationality" between them. That is, through the embedded representation of entities / relationships, the implicit entity relationship in the knowledge graph in addition to the existing entity relationship can be inferred.

[0092] Optionally, multiple predicted fact triples may be sorted first according to the recommendation scores, wherein the multiple predicted fact triples may be sorted in descending order of the recommendation scores, but are not limited to. Then the predicted fact triples ranked in the first Q positions are added to the target knowledge graph, where Q is an integer not less than 1. The actual size of Q may be determined based on the total number of predicted fact triples. For example, if the total number of predicted fact triples is 10, then Q = 10*20% = 2 may be used.

[0093] In an embodiment of the present application, firstly, M entities in the target knowledge graph are obtained; then, N related entities of entity m in the M entities and K concepts corresponding to related entity n in the N related entities are obtained from a preset knowledge base, where m=1,2,3,…,M, n=1,2,3,…,N; then, the semantic relevance between each entity and each related entity of the entity is determined, and the first entity embedding representation of each related entity is determined according to the corresponding K concepts; finally, according to the first entity embedding representation and the semantic relevance, the embedding representation of the entity relationship between M entities and M entities is modeled to obtain an embedding representation model, and finally, the first weight coefficient is iteratively updated according to the attention mechanism to update the unary text embedding representation, and at the same time, the embedding representation of the entity and the entity relationship is iteratively updated according to the preset model training method to train the embedding representation model, so as to obtain the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship. The attention mechanism can further improve the ability to capture related entity features in the aligned text, thereby further improving the embedding representation effect of the entity / relationship, and the accuracy and comprehensiveness of the completion of the target knowledge graph.

[0094] Please refer to Figure 6 , Figure 6 : is a schematic diagram of the structure of a knowledge graph embedding representation device provided in an embodiment of the present application. As shown in the figure, the device in the embodiment of the present application includes:

[0095] The information acquisition module 601 is used to acquire M entities in the target knowledge graph, where the M entities include entity 1, entity 2, ..., entity M, where M is an integer greater than 1;

[0096] The entity alignment module 602 is used to obtain N related entities of M entities m and K concepts corresponding to related entity n among the N related entities from a preset knowledge base, where the N related entities include related entity 1, related entity 2, ..., related entity N, where N and K are integers not less than 1, m = 1, 2, 3, ..., M and n = 1, 2, 3 ..., N, and there is semantic correlation between entity m and the N related entities, and between related entity n and the K concepts;

[0097] A text embedding representation module 603 is used to determine the semantic relevance between each entity in the M entities and each related entity of the entity, and to determine a first entity embedding representation of each related entity according to the corresponding K concepts;

[0098] An entity / relationship modeling module 604 is used to model the embedding representations of the M entities and the entity relationships between the M entities according to the first entity embedding representation and the semantic relevance, so as to obtain an embedding representation model;

[0099] The entity / relationship modeling module 604 is also used to train the embedding representation model to obtain the second entity embedding representation of each entity in the M entities and the relationship embedding representation of the entity relationship.

[0100] Optionally, the text embedding representation module 603 is also used to vectorize each of the K concepts corresponding to the related entity n to obtain a word vector for each concept; and average the word vectors of the K concepts to obtain a first entity embedding representation of the related entity n.

[0101] Optionally, the entity / relationship modeling module 604 is also used to determine the unigram text embedding representation corresponding to each entity based on the semantic relevance and the first entity embedding representation of N related entities; determine the common related entities of every two entities in the M entities based on the N related entities; determine the bigram text embedding representation corresponding to each two entities based on the semantic relevance and the first entity embedding representation of the common related entities; and determine the embedding representation model based on the unigram text embedding representation and the bigram text embedding representation.

[0102] Optionally, the entity / relationship modeling module 604 is also used to map the unigram text embedding representation and the bigram text embedding representation to the same vector space to obtain semantically enhanced unigram text embedding representation and bigram text embedding representation; and establish an embedding representation model based on the semantically enhanced unigram text embedding representation and the semantically enhanced bigram text embedding representation.

[0103] Optionally, the entity / relationship modeling module 604 is further used to use the semantic relevance as the first weight coefficient of each related entity; perform weighted summation on the first entity embedding representations of N related entities according to the first weight coefficient to obtain a unary text embedding representation.

[0104] Optionally, the entity / relationship modeling module 604 is also used to take the minimum semantic relevance between the common related entity and every two entities as the second weight coefficient of the common related entity; and perform weighted summation on the first entity embedding representations of the common related entities according to the second weight coefficient to obtain a binary text embedding representation.

[0105] Optionally, the entity / relationship modeling module 604 is also used to determine the loss function of the embedding representation model; train the embedding representation model according to a preset training method to minimize the function value of the loss function, thereby obtaining a second entity embedding representation and a relationship embedding representation.

[0106] The function value of the loss function is associated with the embedding representation of each entity and the entity relationship, and the unary text embedding representation;

[0107] Optionally, the entity / relationship modeling module 604 is further used to: initialize the embedding representation of each entity and entity relationship to obtain an initial entity embedding representation and an initial relationship embedding representation;

[0108] Optionally, the knowledge graph embedding representation device in the embodiment of the present application further includes an attention calculation module, which is used to update the first weight coefficient according to the attention mechanism to update the unary text embedding representation;

[0109] The entity / relationship modeling module 604 is also used to iteratively update the initial entity embedding representation and the initial relationship embedding representation according to the training method based on the updated unigram text embedding representation.

[0110] The target knowledge graph includes known fact triples, and the known fact triples include two entities among the M entities and an entity relationship;

[0111] The embedding representation device of the knowledge graph in the embodiment of the present application also includes a graph completion module, which is used to replace the entity relationship included in the known fact triple with other entity relationships between N entities, or replace one entity included in the known fact triple with other entities among the N entities, to obtain a predicted fact triple; determine the recommendation score of the predicted fact triple based on the second entity embedding representation of the entity in the predicted fact triple and the relational embedding representation of the entity relationship; and add the predicted fact triple to the target knowledge graph based on the recommendation score.

[0112] It should be noted that the implementation of each module can also refer to Figure 3 and Figure 4 The corresponding description of the method embodiment shown executes the method and functions performed by the embedding representation device of the knowledge graph in the above embodiment.

[0113] Please continue to refer to Figure 7 , Figure 7 701 is a schematic diagram of the structure of a knowledge graph embedding representation device provided in an embodiment of the present application. As shown in the figure, the knowledge graph embedding representation device may include: at least one processor 701, at least one transceiver 702, at least one memory 703 and at least one communication bus 704. Of course, in some embodiments, the processor and the memory may also be integrated together.

[0114] Among them, the processor 701 can be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit, a field programmable gate array or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the contents disclosed in this application. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a digital signal processor and a microprocessor, and so on. The communication bus 704 can be a peripheral component interconnect standard PCI bus or an extended industrial standard structure EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used to represent it, but it does not mean that there is only one bus or one type of bus. The communication bus 704 is used to realize the connection and communication between these components. Among them, the transceiver 702 of the device in the embodiment of the present application is used to communicate with other network elements. The memory 703 may include volatile memory, such as non-volatile dynamic random access memory (NVRAM), phase change random access memory (Phase Change RAM, PRAM), magnetoresistive random access memory (MRAM), etc., and may also include non-volatile memory, such as at least one disk storage device, electronically erasable programmable read-only memory (Electrically Erasable ProgrammableRead-Only Memory, EEPROM), flash memory devices, such as NOR flash memory or NAND flash memory, semiconductor devices, such as solid state disk (SSD), etc. The memory 703 may also be optionally at least one storage device located away from the aforementioned processor 701. A set of program codes is stored in the memory 703, and the processor 701 can optionally execute the program stored in the memory 703:

[0115] Obtain M entities in the target knowledge graph, where the M entities include entity 1, entity 2, ..., entity M, and M is an integer greater than 1;

[0116] Acquire N related entities of entity m among the M entities, and K concepts corresponding to related entity n among the N related entities from a preset knowledge base, wherein the N related entities include related entity 1, related entity 2, ..., related entity N, wherein N and K are integers not less than 1, m=1,2,3,...,M and n=1,2,3...,N, and there is semantic correlation between the entity m and the N related entities, and between the related entity n and the K concepts;

[0117] Determine a semantic relevance between each entity in the M entities and each related entity of the entity, and determine a first entity embedding representation of each related entity according to the corresponding K concepts;

[0118] Modeling the embedding representations of the M entities and the entity relationships between the M entities according to the first entity embedding representation and the semantic relevance to obtain an embedding representation model;

[0119] The embedding representation model is trained to obtain a second entity embedding representation of each entity and a relational embedding representation of the entity relationship.

[0120] Optionally, the processor 701 is further configured to perform the following operations:

[0121] Performing vectorization processing on each of the K concepts corresponding to the related entity n to obtain a word vector for each concept;

[0122] The word vectors of the K concepts corresponding to the related entity n are averaged and summed to obtain a first entity embedding representation of the related entity n.

[0123] Optionally, the processor 701 is further configured to perform the following operations:

[0124] Determine a unary text embedding representation corresponding to each entity according to the semantic relevance and the first entity embedding representations of the N related entities;

[0125] Determine, according to the N related entities, a common related entity of every two entities in the M entities;

[0126] Determine, according to the semantic relevance and the first entity embedding representation of the commonly related entities, a bigram text embedding representation corresponding to each two entities;

[0127] The embedding representation model is determined according to the unigram text embedding representation and the bigram text embedding representation.

[0128] Optionally, the processor 701 is further configured to perform the following operations:

[0129] Mapping the unigram text embedding representation and the bigram text embedding representation to the same vector space to obtain semantically enhanced unigram text embedding representation and bigram text embedding representation;

[0130] The embedding representation model is established according to the semantically enhanced unigram text embedding representation and the semantically enhanced bigram text embedding representation.

[0131] Optionally, the processor 701 is further configured to perform the following operations:

[0132] Using the semantic relevance as a first weight coefficient of each related entity;

[0133] The first entity embedding representations of the N related entities are weightedly summed according to the first weight coefficient to obtain the unary text embedding representation.

[0134] Optionally, the processor 701 is further configured to perform the following operations:

[0135] Using the minimum semantic relevance between the common related entity and each of the two entities as a second weight coefficient of the common related entity;

[0136] The first entity embedding representations of the commonly related entities are weightedly summed according to the second weight coefficient to obtain the binary text embedding representation.

[0137] Optionally, the processor 701 is further configured to perform the following operations:

[0138] Determining a loss function of the embedding representation model;

[0139] The embedding representation model is trained according to a preset training method to minimize the function value of the loss function, thereby obtaining the second entity embedding representation and the relationship embedding representation.

[0140] Optionally, the function value is associated with the embedding representation of each entity and the entity relationship, and the unary text embedding representation;

[0141] The processor 701 is further configured to perform the following operations:

[0142] Initializing the embedding representation of each entity and the entity relationship to obtain an initial entity embedding representation and an initial relationship embedding representation;

[0143] The first weight coefficient is updated according to the attention mechanism to update the unary text embedding representation, and the initial entity embedding representation and the initial relationship embedding representation are iteratively updated according to the training method.

[0144] Optionally, the target knowledge graph includes a known fact triple, and the known fact triple includes two entities of the M entities and an entity relationship;

[0145] The processor 701 is further configured to perform the following operations:

[0146] Replacing the entity relationship included in the known fact triple with other entity relationships between the N entities, or replacing one entity included in the known fact triple with other entities among the N entities, to obtain a predicted fact triple;

[0147] Determining a recommendation score for the predicted fact triple according to a second entity embedding representation of an entity in the predicted fact triple and a relation embedding representation of an entity relation;

[0148] According to the recommendation score, the predicted fact triple is added to the target knowledge graph.

[0149] Furthermore, the processor can also cooperate with the memory and the transceiver to perform the operations of the embedding representation device of the knowledge graph in the above-mentioned application embodiment.

[0150] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable base stations. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0151] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of this application in detail. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this application shall be included in the protection scope of this application.

Claims

1. A knowledge graph embedding representation method, characterized in that: The method comprises: Obtain M entities in the target knowledge graph, where the M entities include entity 1, entity 2, ..., entity M, and M is an integer greater than 1; Obtain N related entities of entity m among the M entities, and K concepts corresponding to related entity n among the N related entities, wherein the N related entities include related entity 1, related entity 2, ..., related entity N, wherein N and K are integers not less than 1, m=1,2,3,...,M and n=1,2,3...,N, and the entity m is semantically related to the N related entities, and the related entity n is semantically related to the K concepts; Determine a semantic relevance between each entity in the M entities and each related entity of the entity, and determine a first entity embedding representation of each related entity according to the corresponding K concepts; Modeling the embedding representations of the M entities and the entity relationships between the M entities according to the first entity embedding representation and the semantic relevance to obtain an embedding representation model; Based on the embedding representation model, a second entity embedding representation of each entity and a relational embedding representation of the entity relationship are obtained.

2. The method according to claim 1, characterized in that Determining the first entity embedding representation of each related entity according to the corresponding K concepts includes: Performing vectorization processing on each of the K concepts corresponding to the related entity n to obtain a word vector for each concept; The word vectors of the K concepts corresponding to the related entity n are averaged and summed to obtain a first entity embedding representation of the related entity n.

3. The method according to claim 1, characterized in that The step of modeling the embedding representations of the M entities and the entity relationships between the M entities according to the first entity embedding representation and the semantic relevance to obtain the embedding representation model comprises: Determine a unary text embedding representation corresponding to each entity according to the semantic relevance and the first entity embedding representations of the N related entities; Determine, according to the N related entities, a common related entity of every two entities in the M entities; Determine, according to the semantic relevance and the first entity embedding representation of the commonly related entities, a bigram text embedding representation corresponding to each two entities; The embedding representation model is established according to the unigram text embedding representation and the bigram text embedding representation.

4. The method according to claim 3, characterized in that The establishing the embedding representation model according to the unary text embedding representation and the bigram text embedding representation comprises: Mapping the unigram text embedding representation and the bigram text embedding representation to the same vector space to obtain a semantically enhanced unigram text embedding representation and a semantically enhanced bigram text embedding representation; The embedding representation model is established according to the semantically enhanced unigram text embedding representation and the semantically enhanced bigram text embedding representation.

5. The method according to claim 3 or 4, characterized in that Determining the unary text embedding representation corresponding to each entity according to the semantic relevance and the first entity embedding representation of the N related entities comprises: Using the semantic relevance as a first weight coefficient of each related entity; The first entity embedding representations of the N related entities are weightedly summed according to the first weight coefficient to obtain the unary text embedding representation.

6. The method according to any one of claims 3 to 5, characterized in that: Determining the bigram text embedding representation corresponding to each two entities according to the semantic relevance and the first entity embedding representation of the commonly related entity comprises: Using the minimum semantic relevance between the common related entity and each of the two entities as a second weight coefficient of the common related entity; The first entity embedding representations of the commonly related entities are weightedly summed according to the second weight coefficient to obtain the binary text embedding representation.

7. The method according to any one of claims 1 to 6, characterized in that: The obtaining, based on the embedding representation model, the second entity embedding representation of each entity and the relational embedding representation of the entity relationship comprises: Determining a loss function of the embedding representation model; The embedding representation model is trained according to a preset training method to minimize the function value of the loss function, thereby obtaining the second entity embedding representation and the relationship embedding representation.

8. The method according to claim 7, characterized in that The function value is associated with the embedding representation of each entity and the entity relationship, and the unary text embedding representation; The step of training the embedding representation model according to a preset training method to minimize the function value of the loss function, thereby obtaining the second entity embedding representation and the relationship embedding representation includes: Initializing the embedding representation of each entity and the entity relationship to obtain an initial entity embedding representation and an initial relationship embedding representation; The first weight coefficient is iteratively updated according to the attention mechanism to update the unary text embedding representation, and the initial entity embedding representation and the initial relationship embedding representation are iteratively updated according to the training method.

9. The method according to any one of claims 1 to 8, characterized in that: The target knowledge graph includes a known fact triple, and the known fact triple includes two entities among the M entities and an entity relationship; After obtaining the second entity embedding representation of each entity and the relationship embedding representation of the entity relationship based on the embedding representation model, the method further includes: Replacing the entity relationship included in the known fact triple with other entity relationships between the N entities, or replacing one entity included in the known fact triple with other entities among the N entities, to obtain a predicted fact triple; Determining a recommendation score for the predicted fact triple according to a second entity embedding representation of an entity in the predicted fact triple and a relation embedding representation of an entity relation; According to the recommendation score, the predicted fact triple is added to the target knowledge graph.

10. A system for obtaining an embedded representation of a knowledge graph, characterized in that: include: A memory, a communication bus and a processor, wherein the memory is used to store program codes, and the processor is used to call the program codes to execute the method according to any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 9.

12. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 9.

Images