Method and system for constructing multi-relational knowledge base

Abstract

Disclosed are a method and system for constructing a multivariate relational knowledge base. The method includes: constructing a multivariate relational model for tuples in a multivariate relational knowledge base, and establishing a corresponding scoring function and loss function; training the multivariate relational model based on the scoring function and loss function using training samples obtained from the multivariate relational knowledge base; predicting and completing missing entities in tuples with missing entities in the multivariate relational knowledge base based on the trained multivariate relational model, and constructing a complete multivariate relational knowledge base based on each tuple with completed missing entities.

Description

Technical Field

[0001] This application relates to knowledge graph technology in the field of artificial intelligence, and more specifically, to a method and system for constructing a multi-relational knowledge base. Background Art

[0002] With the rapid development of information network technologies, the amount of data content on various information networks is experiencing explosive growth. This content is often large-scale, heterogeneous, and loosely organized, posing challenges to effective access to information and knowledge. A knowledge graph (KG) is a knowledge base on a semantic network that uses visualization techniques to describe knowledge resources and their carriers, and to mine, analyze, construct, map, and display knowledge and its interconnections.

[0003] Knowledge graphs, a specialized graphical structure with entities as nodes and relations as directed edges, have recently attracted considerable interest. In a typical knowledge graph, each edge is represented as a triple (s, r, o) of the form (head entity, relation, tail entity), indicating that two entities s (the head entity) and o (the tail entity) are connected by a relation r. For example, (NewYork, isLocatedIn, USA) indicates that New York is located in the USA. Numerous large-scale knowledge graphs have been developed over the past few decades, such as WordNet, Freebase, DBpedia, and YAGO. They have enabled improvements in various downstream applications, such as structured search, question answering, and entity recommendation.

[0004] However, existing knowledge graph processing methods are typically limited to two dimensions, such as a ternary relationship between three elements like (relationship, entity 1, entity 2), but such expressions are sometimes very limited. For example, a general purchase behavior can be a ternary relationship involving a buyer, an item, and a salesperson, but sports event awards usually involve a quaternary relationship such as an athlete, a team, an award, and a season. However, most traditional knowledge graph processing methods cannot be directly extended to multi-dimensional relationships. For modeling multi-dimensional knowledge bases (KBs), existing methods cannot provide a sufficiently expressive and efficient modeling solution.

[0005] In addition, most existing works focus on modeling at the "relationship" level, while ignoring the role of "roles" in multi-relationships. Figure 1In the figure, three multi-tuple relation facts are shown as an example (i.e., the relation facts corresponding to the triple (WasGovernorOf, California, Schwarzenegger), the quadruple (PlayCharacterIn, Schwarzenegger, T-800, Terminator2: Judgment Day), and the quadruple (DirectorAwardFor, Terminator2: Judgment Day, James Cameron, Saturn Award)). Each relationship is represented as multiple roles based on the arity of the tuple to which it belongs. Each role represents the semantics between the relationship and the corresponding entity, and different relationships can share the same role (for example, the role MOVIE is shared between the relationships PlayCharacterIn and DirectorAwardFor). Roles can identify multiple semantics of relationships, which implies semantic relevance in multi-tuple relations. In addition, the interactions between relationships and entities are refined into role-based interactions between roles and related entities, which makes it possible to further determine the rationality of multi-tuple relation facts. Semantically related relationships can share similar role information and can therefore be represented as common shared roles. Such a role can further confirm the legitimacy of the fact of multiple relationships from the following two aspects:

[0006] First, roles can distinguish multiple semantics of entities. For example, Figure 1 As shown in , the roles GOVERNOR and ACTOR can clearly identify the political semantics and star semantics of the entity Schwarzenegger).

[0007] Second, roles have various interaction patterns with related entities, e.g. Figure 1 As shown in , for PlayCharacterIn, the characters ACTOR and CHARACTER have strong interactions with their corresponding entities. However, since the knowledge base provides rich knowledge related to MOVIE and Schwarzenegger together, the character MOVIE has stronger interactions and associations with Schwarzenegger than the associated entity Terminator 2.

[0008] The importance of roles (especially semantic relevance) to multi-relational knowledge bases has been neglected in existing works, and even if roles are considered, role-based interactions are often ignored.

[0009] Therefore, a model for processing multi-relation knowledge base is needed that has sufficient expressive power while capturing the implicit semantic relationships between multi-relationships. Summary of the Invention

[0010] According to an embodiment of the present invention, a method for constructing a multivariate relational knowledge base is provided, the method comprising: constructing a multivariate relational model for a tuple in a multivariate relational knowledge base, and establishing a corresponding scoring function and a loss function, wherein each tuple in the multivariate relational knowledge base consists of a relation and multiple entities, the corresponding semantics between the one relation and each entity in the multiple entities are indicated by a role, and the corresponding observation facts between each role of the one relation and the corresponding entity in the multiple entities constitute a set of observation facts corresponding to the tuple, wherein the scoring function is used to indicate the association between the role and the entity in the set of observation facts corresponding to each tuple; based on the scoring function and the loss function, the multivariate relational model is trained using training samples obtained from the multivariate relational knowledge base, wherein the training samples are a set of observation facts corresponding to tuples in the multivariate relational knowledge base that do not have missing entities; based on the trained multivariate relational model, the missing entities in the tuples with missing entities in the multivariate relational knowledge base are predicted and completed, and a complete multivariate relational knowledge base is constructed based on each tuple with completed missing entities.

[0011] Optionally, the step of establishing a scoring function may include: establishing a latent role space including a predetermined number of latent role basis vectors, and determining a role embedding vector for each role based on the latent role basis vectors; mapping each entity to a predetermined number of semantic embedding vectors for semantics; determining a role sensitive pattern matrix for each role, wherein the role sensitive pattern matrix indicates the interaction intensity between the role and the semantic embedding vectors of each entity; establishing the scoring function based on the role embedding vector, the semantic embedding vector and the role sensitive pattern matrix, wherein the role embedding vector of each role is represented by a combination of the latent role basis vectors, and wherein the role sensitive pattern matrix of each role is established based on a basic matrix associated with the latent role basis vectors.

[0012] Optionally, the scoring function can be expressed as:

[0013]

[0014] in,

[0015]

[0016]

[0017] and k′=1, 2, ..., K,

[0018] in, Indicates that the number of elements in the multi-relation knowledge base is (a r +1), r indicates the relation in the tuple and r∈relational domain e i Indicates the i-th entity in the tuple and e i ∈ entity domain ε, is the set of roles of relation r, is the set of observation facts corresponding to the multi-tuple, a r ≥2 and a r is a positive integer, is the i-th role of relation r The embedding vector of It's a role The corresponding role weight vector of the embedding vector, Indicates that entity e i The matrix of m semantic embedding vectors, It's a role The role-sensitive pattern matrix and its j-th row Indicates the multi-tuple The jth entity e in j The interaction strength of the m semantic embedding vectors, is the kth latent role basis vector in the latent role space and is the basic matrix associated with the kth potential role basic vector, Φ is a function normalized by a temperature parameter and T u is the character weight vector The temperature parameter T P Is for the role sensitive mode matrix where d indicates the embedding vector dimension of the character, K is the size of the preset latent character space and indicates the number of latent character basis vectors, and m is a predetermined value.

[0019] Alternatively, the loss function can be expressed as:

[0020]

[0021] in, represents the set of ar group negative samples x constructed for positive samples x:=rol7r:e1,…,rolarr:ear, φ is the estimated score function, Represents the entity e in the entity domain ε and the entity e in the positive sample i A different entity, Represents a replacement positive sample The set of negative samples obtained by the qth entity in , 1≤q≤ar And q is a positive integer.

[0022] Optionally, the step of training the multivariate relational model using training samples obtained from the multivariate relational knowledge base may include: initializing model parameters of the multivariate relational model; and performing at least one iterative update operation to update the model parameters in the multivariate relational model based on the initialized model parameters. Each iterative update operation may include: selecting a predetermined number of training samples as a positive sample set and setting an initial loss parameter; performing an iterative loss update operation for each positive sample in the positive sample set to update the loss parameter; and updating the model parameters based on a loss gradient of the loss parameter. Each iterative loss update operation may include: constructing a negative sample set for the current positive sample in the positive sample set; calculating a role embedding vector and a role-sensitive pattern matrix based on the current positive sample; and calculating a current loss for the current positive sample and a negative sample set constructed for the current positive sample using the determined scoring function and loss function based on the calculated role embedding vector and role-sensitive pattern matrix. The loss parameter obtained for the current iterative loss update operation is determined based on the calculated current loss and the loss parameter obtained from the previous iterative loss update operation.

[0023] Optionally, the step of predicting and completing missing entities in a multi-tuple with missing entities in the multi-tuple relational knowledge base based on a trained multi-tuple relational model may include: predicting an entity in the multi-tuple relational knowledge base that is most likely to be the missing entity in the multi-tuple with missing entities based on the scoring function; and using the predicted entity to complete the missing entity in the multi-tuple with missing entities.

[0024] According to another embodiment of the present invention, a multi-relation knowledge base construction system is provided, the system comprising: a model construction device configured to construct a multi-relation model for a multi-relation knowledge base, and establish a corresponding scoring function and a loss function, wherein each multi-relation knowledge base is composed of a relationship and multiple entities, the corresponding semantics between the relationship and each entity in the multiple entities are indicated by roles, and the corresponding observation facts between each role of the relationship and the corresponding entity in the multiple entities constitute a set of observation facts corresponding to the multi-relation knowledge base, wherein the scoring function is used to indicate The method comprises the following steps: showing the association between roles and entities in the set of observation facts corresponding to each multi-tuple; a training device is configured to train the multi-tuple relationship model based on the evaluation function and the loss function using training samples obtained from the multi-tuple relationship knowledge base, wherein the training samples are the set of observation facts corresponding to the multi-tuples in the multi-tuple relationship knowledge base that do not have missing entities; and a knowledge base construction device is configured to predict and complete the missing entities in the multi-tuples with missing entities in the multi-tuple relationship knowledge base based on the trained multi-tuple relationship model, and to construct a complete multi-tuple relationship knowledge base based on the various multi-tuples with completed missing entities.

[0025] Optionally, the model building device can be configured to establish a scoring function based on the following operations: establishing a latent role space including a predetermined number of latent role basis vectors, and determining a role embedding vector for each role based on the latent role basis vectors; mapping each entity to a predetermined number of semantic embedding vectors for semantics; determining a role sensitive pattern matrix for each role, wherein the role sensitive pattern matrix indicates the interaction intensity between the role and the semantic embedding vectors of each entity; establishing the scoring function based on the role embedding vector, the semantic embedding vector and the role sensitive pattern matrix, wherein the role embedding vector of each role is represented by a combination of the latent role basis vectors, and wherein the role sensitive pattern matrix of each role is established based on a basic matrix associated with the latent role basis vectors.

[0026] Optionally, the scoring function can be expressed as:

[0027]

[0028] in,

[0029]

[0030]

[0031] and k′=1, 2, ..., K,

[0032] in, Indicates that the number of elements in the multi-relation knowledge base is (a r +1), r indicates the relation in the tuple and r∈relational domain e i Indicates the i-th entity in the tuple and e i ∈ entity domain ε, is the set of roles of relation r, is the set of observation facts corresponding to the multi-tuple, a r ≥2 and a r is a positive integer, is the i-th role of relation r The embedding vector of It's a role The corresponding role weight vector of the embedding vector, Indicates that entity e i The matrix of m semantic embedding vectors, It's a role The role-sensitive pattern matrix and its j-th row Indicates the multi-tuple The jth entity e in j The interaction strength of the m semantic embedding vectors, is the kth latent role basis vector in the latent role space and is the basic matrix associated with the kth potential role basic vector, Φ is a function normalized by a temperature parameter and T u is the character weight vector The temperature parameter T P Is for the role sensitive mode matrix where d indicates the embedding vector dimension of the character, K is the size of the preset latent character space and indicates the number of latent character basis vectors, and m is a predetermined value.

[0033] Alternatively, the loss function can be expressed as:

[0034]

[0035] in, Indicates that for positive samples Constructed a r Group negative samples The set of φ is the scoring function, Represents the entity e in the entity domain ε and the entity e in the positive sample i A different entity, Represents a replacement positive sample The set of negative samples obtained by the qth entity in , 1≤q≤a r And q is a positive integer.

[0036] Optionally, the training device is configured to train the multivariate relationship model by the following operations: initializing the model parameters of the multivariate relationship model; based on the initialized model parameters, performing at least one iterative update operation to update the model parameters in the multivariate relationship model. Each iterative update operation may include: selecting a predetermined number of training samples as a positive sample set and setting an initial loss parameter; for each positive sample in the positive sample set, performing an iterative loss update operation to update the loss parameter; and updating the model parameters according to the loss gradient of the loss parameter. Each iterative loss update operation may include: constructing a negative sample set of the current positive sample in the positive sample set; based on the current positive sample, calculating a role embedding vector and a role sensitive pattern matrix; based on the calculated role embedding vector and the role sensitive pattern matrix, using the determined scoring function and loss function to calculate the current loss for the current positive sample and the negative sample set constructed for the current positive sample, and determining the loss parameter obtained in the current iterative loss update operation based on the calculated current loss and the loss parameter obtained in the previous iterative loss update operation.

[0037] Optionally, the knowledge base construction device can be configured to: predict the entity in the multi-relation knowledge base that is most likely to be the missing entity in the multi-tuple with missing entities based on the scoring function; and use the predicted entity to complete the missing entity in the multi-tuple with missing entities.

[0038] According to another embodiment of the present invention, a computer-readable storage medium storing instructions is provided, wherein when the instructions are executed by at least one computing device, the at least one computing device is prompted to execute the aforementioned multi-relationship knowledge base construction method.

[0039] According to another embodiment of the present invention, a system is provided, comprising at least one computing device and at least one storage device storing instructions, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to execute the aforementioned multi-relational knowledge base construction method.

[0040] Beneficial effects

[0041] By applying the method and system for constructing a multivariate relational knowledge base according to an exemplary embodiment of the present invention, it is possible to establish a multivariate relational model for the multivariate relational knowledge base that has sufficient expressive power and is capable of capturing the implicit semantic relationships between multivariate relations. This allows accurate prediction of missing entities in multivariate groups within the knowledge base, thereby completing and constructing a complete multivariate relational knowledge base. Furthermore, the method and system for constructing a multivariate relational knowledge base according to an exemplary embodiment of the present invention can also simultaneously handle the learning problem of mixed metadata, further expanding the model's flexibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] These and / or other aspects and advantages of the present invention will become more apparent and more readily understood from the following detailed description of embodiments of the present invention in conjunction with the accompanying drawings, in which:

[0043] Figure 1 is a diagram illustrating an exemplary multi-relationship fact;

[0044] Figure 2 is a block diagram illustrating a multi-relationship knowledge base construction system according to an exemplary embodiment of the present disclosure;

[0045] Figure 3 is a flowchart illustrating a method for constructing a multi-relationship knowledge base according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the present invention, exemplary embodiments of the present invention are further described in detail below with reference to the accompanying drawings and specific embodiments.

[0047] Before starting the description of the present invention, for ease of understanding, some parameter expressions used in this application will first be explained:

[0048] Multi-relational knowledge base - short for a knowledge graph consisting of multi-tuples with multi-relationships.

[0049] Scalars are denoted by lowercase letters, vectors are denoted by lowercase bold letters, and matrices are denoted by uppercase bold letters.

[0050] a(i) represents the i-th element in vector a, A(i, j) represents the (i, j)-th element in matrix A, and A(i,:) represents the i-th row in matrix A.

[0051] n e , n r - respectively represent the number of entities e and relations r in the multi-relational knowledge base, n e =|ε|,

[0052] ε-represents the relation domain for relation r (i.e., ) and the entity domain for entity e (i.e., e∈ε).

[0053] <·> - represents a multilinear dot product, and <a1,a2,...,a n >=∑ i a1(i)a2(i)…a n (i).

[0054] Furthermore, in the context of the present disclosure, parameters having the same expression form have the same definition.

[0055] Figure 2 is a block diagram illustrating a multi-relationship knowledge base construction system 200 according to an exemplary embodiment of the present disclosure.

[0056] Reference Figure 2 According to an exemplary embodiment of the present disclosure, a multi-relationship knowledge base construction system 200 includes: a model construction device 210 , a training device 220 , and a knowledge base construction device 230 .

[0057] According to an exemplary embodiment of the present invention, the model-building device 210 may construct a multivariate relationship model for each tuple in a multivariate relationship knowledge base and establish a corresponding scoring function and loss function. Subsequently, the training device 220 may train the multivariate relationship model using training samples obtained from the multivariate relationship knowledge base based on the scoring function and loss function.

[0058] In an exemplary embodiment of the present invention, each tuple in the multivariate relational knowledge base may be composed of a relation and multiple entities, the corresponding semantics between the relation and each of the multiple entities may be indicated by a role, and the corresponding observation facts between each role of the relation and the corresponding entity of the multiple entities constitute a set of observation facts corresponding to the tuple. The training samples of the multivariate relational model may be a set of observation facts corresponding to tuples in the multivariate relational knowledge base that do not have missing entities, and the scoring function may be used to indicate the association between the roles and entities in the set of observation facts corresponding to each tuple.

[0059] In an exemplary embodiment of the present invention, the multi-relation knowledge base may include mixed metadata, that is, multi-tuples with different numerals. In the following description, for ease of explanation, the numerals (a r +1) Take this as an example to illustrate, here, a r Can refer to the number of entities in the tuple, a r ≥2 and a r is a positive integer, r can indicate the relationship in the tuple and e i can indicate the i-th entity in the tuple and e i ∈Entity Domain ε. In an exemplary embodiment of the present invention, the set of roles of a relation r can be represented by To express, The set of observation facts corresponding to the tuple may be indicated.

[0060] Since semantically related relationships may share the same role information, the model building device 210 may establish a set of K potential role basis vectors The potential role space (here, K indicates the size of the preset potential role space and K<<n r , d represents the embedding vector dimension of the role, is the kth latent character basis vector in the latent character space and 1≤k≤K), and the character embedding vector of each character is determined based on the latent character basis vector. In an exemplary embodiment of the present invention, the character embedding vector of each character can be represented by a combination of the latent character basis vectors, as shown in the following expression (1):

[0061]

[0062] In the above expression, is the i-th role of relation r The embedding vector of It's a role The corresponding role weight vector of the embedding vector. Φ is the temperature parameter T u right The normalization function is shown in the following expression (2):

[0063]

[0064] In addition, the model building device 210 can i Mapped to a predetermined number (e.g., m) of semantic embedding vectors for semantics, thereby obtaining the entity e i The matrix of m semantic embedding vectors Here, the entity e i Each semantic embedding vector forms a matrix E i A column.

[0065] In addition, the model building device 210 may also determine a role-sensitive pattern matrix for each role. In an exemplary embodiment of the present invention, the role-sensitive pattern matrix may indicate the interaction strength between the role and the semantic embedding vector of each entity. Here, the role-sensitive pattern matrix for each role is established based on the basic matrix associated with the potential role basic vector, as shown in the following expression (3):

[0066]

[0067] In expression (3), It's a role The role-sensitive pattern matrix and its j-th row Directives and Plural Groups The jth entity e in j The interaction strength of the m semantic embedding vectors, is the basic matrix associated with the kth potential role basic vector, Φ is the temperature parameter T P right The row normalization function is shown in the following expression (4):

[0068]

[0069] After determining the role embedding vector, the semantic embedding vector, and the role-sensitive pattern matrix as described above, the model building device 210 may establish the scoring function based on the role embedding vector, the semantic embedding vector, and the role-sensitive pattern matrix, as shown in the following expression (5):

[0070]

[0071] Afterwards, the model building device 210 may build a loss function of the multivariate relationship model based on the above-mentioned scoring function, as shown in the following expression (6):

[0072]

[0073] in,

[0074] Indicates that for positive samples Constructed a r Group negative samples The set of φ is the scoring function, Represents the entity e in the entity domain ε and the entity e in the positive sample i Another entity that is different (i.e., to replace e i to constitute the entity of negative samples), Represents a replacement positive sample The set of negative samples obtained by the qth entity in , 1≤q≤a rAnd q is a positive integer.

[0075] It should be understood that the specific forms of the above-mentioned loss function and scoring function are only examples, and this application is not limited to this. Various other forms of scoring functions and loss functions can also be used or set, as long as they can achieve the essential intention of the scoring function and loss function of this application.

[0076] The training device 220 can train the multivariate relationship model based on the above-mentioned scoring function and loss function to find model parameters that satisfy the following expression (7):

[0077]

[0078] In an exemplary embodiment of the present invention, the training device 220 may use various algorithms (eg, stochastic gradient descent, proximal gradient descent, alternating direction multipliers, etc.) to train the multivariate relationship model.

[0079] As just an example, the training device 220 may train the multivariate relationship model according to Algorithm 1 shown below.

[0080]

[0081]

[0082] As shown in the above algorithm 1, when training the multi-relationship model, the training device 220 may first initialize the model parameters of the multi-relationship model (for example, randomly initialize). Here, the model parameters may include the embedding vector E for each entity e∈ε, ... Role Weight as well as and

[0083] Afterwards, the training device 220 may perform at least one iterative update operation (line 1 to line 10 in Algorithm 1) based on the initialized model parameters to update the learnable model parameters in the multivariate relational model (i.e., the aforementioned model parameters {E i}, ), thereby achieving the training of the model. Here, the number of iterative update operations (i.e., n epoch ) can be a preset number of times (for example, a number preset based on experience).

[0084] In an exemplary embodiment of the present invention, each iterative update operation (line 2 to line 9 in Algorithm 1) may include the following processing:

[0085] A) From the training set Select (eg, sample) a predetermined number (ie, m b ) as the positive sample set (i.e., the set ), and the initial loss parameters can also be set at this time For example, set it to 0 (line 2 in Algorithm 1);

[0086] B) For each positive sample in the positive sample set, perform an iterative loss update operation (lines 3-8 in Algorithm 1) to update the loss parameters;

[0087] C) Loss gradient according to the loss parameter To update the learnable model parameters (line 9 in Algorithm 1). Specifically, the loss gradient can be first calculated based on the loss parameter Gradient back propagation is then used to update the learnable model parameters. Here, the method of calculating the loss gradient and using the gradient to update the model parameters is known to those skilled in the art (for example, a gradient descent algorithm can be used), so for the sake of simplicity it will not be described in detail.

[0088] Here, each iterative loss update operation may include:

[0089] Construct a negative sample set of the current positive sample in the positive sample set (line 4 in Algorithm 1);

[0090] Based on the current positive sample, use the above expressions (1) and (3) to calculate the role embedding vector and role sensitive pattern matrix respectively (lines 5-6 in Algorithm 1);

[0091] Based on the calculated embedding vector and role-sensitive pattern matrix, the scoring function and the loss function are used to calculate the current loss for the current positive sample and the set of negative samples constructed for the current positive sample, and the loss parameter obtained by the current iterative loss update operation is determined based on the calculated current loss and the loss parameter obtained by the last iterative loss update operation (i.e., the last iterative loss update operation performed on another positive sample in the positive sample set) (line 7 in Algorithm 1).

[0092] As described above, Algorithm 1 shows an example of using a mini-batch method to train a multivariate relationship model according to an exemplary embodiment of the present invention. However, it should be understood that the present application is not limited to this, and various other algorithms (for example, stochastic gradient descent, proximal gradient descent, alternating direction method of multipliers, etc.) can also be used to train the multivariate relationship model.

[0093] After completing the model training, the completion device 220 may predict and complete the missing entities in each tuple with missing entities in the multi-tuple relationship knowledge base based on the trained multi-tuple relationship model.

[0094] Specifically, the knowledge base construction device 230 can predict the entity in the multi-relation knowledge base that is most likely to be the missing entity in the multi-tuple with missing entities based on the aforementioned scoring function, and then use the predicted entity to complete the missing entity in the multi-tuple with missing entities.

[0095] That is, for example, the knowledge base construction device 230 may attempt to use each entity in the multi-relational knowledge base to complete the missing entity in the multi-tuple with missing entities, and then use the scoring function to calculate the score of such a multi-tuple, and determine the entity used when the score is the highest as the entity ultimately used to complete the multi-tuple with missing entities.

[0096] Afterwards, the knowledge base construction device 230 may construct a complete multi-tuple relationship knowledge base based on each multi-tuple that completes the missing entity.

[0097] According to an exemplary embodiment of the present invention, the multi-tuple relation model can not only complete the tuple when one entity is missing in the tuple, but also complete the tuple when more than one entity is missing in the tuple.

[0098] Figure 3 is a flowchart illustrating a method for constructing a multi-relationship knowledge base according to an exemplary embodiment of the present disclosure.

[0099] Reference Figure 3In step S310, the model building device 210 can build a multivariate relationship model for the multivariate relationship knowledge base, and establish a corresponding scoring function and loss function, wherein each multivariate relationship knowledge base is composed of a relationship and multiple entities, the corresponding semantics between the one relationship and each entity in the multiple entities are indicated by a role, and the corresponding observation facts between each role of the one relationship and the corresponding entity in the multiple entities constitute an observation fact set corresponding to the multivariate relationship, wherein the scoring function is used to indicate the correlation between the role and the entity in the observation fact set corresponding to each multivariate relationship.

[0100] Then, in step S320, the multivariate relational model can be trained by the training device 220 based on the scoring function and the loss function using training samples obtained from the multivariate relational knowledge base, wherein the training samples are a set of observed facts corresponding to multivariate groups in the multivariate relational knowledge base that do not have missing entities.

[0101] Finally, in step S330, the knowledge base construction device 230 can predict and complete the missing entities in the multivariate relationship knowledge base based on the trained multivariate relationship model, and construct a complete multivariate relationship knowledge base based on each multivariate relationship with completed missing entities.

[0102] The above has been combined Figure 2 The specific processes of the detailed operations performed by the above-mentioned components of the multi-relationship knowledge base construction system 200 according to the exemplary embodiment of the present disclosure have been described in detail, and therefore will not be repeated here for the sake of brevity.

[0103] In addition, the multi-relationship knowledge base construction method according to the exemplary embodiment of the present disclosure can also use the multi-relationship knowledge base obtained in step S330 to train a machine learning model to obtain a target machine learning model for performing at least one of relationship retrieval, semantic retrieval, intelligent retrieval, intelligent recommendation, intelligent question and answer, personalized recommendation, and content distribution, and the target machine learning model can be used to perform prediction tasks, wherein the prediction tasks include at least one of relationship retrieval, semantic retrieval, intelligent retrieval, intelligent recommendation, intelligent question and answer, personalized recommendation, and content distribution.

[0104] That is to say, the multi-relationship knowledge base construction method and system of the exemplary embodiments of the present disclosure can be applied to various fields, such as relationship retrieval, semantic retrieval, intelligent retrieval, intelligent recommendation, intelligent question and answer, personalized recommendation, anti-fraud, content distribution, etc.

[0105] Just as an example, among the various application scenarios of the multi-relationship knowledge base construction method and system according to the exemplary embodiments of the present disclosure, for example, for retrieval (such as relationship retrieval, semantic retrieval, intelligence, etc.), the relationship between them or the corresponding other entity can be retrieved by entering multiple keywords.

[0106] For example, for intelligent question answering, if you input "Which team won the 2018 World Cup?", "France" will be accurately returned, thus truly understanding the user's intention through the knowledge base.

[0107] For example, for anti-fraud, when the borrower's (entity) information is added to the knowledge base, the existence of fraud risk can be determined by reading the relationship between the borrower and other people in the knowledge base, or whether the information they share is consistent.

[0108] For example, for intelligent recommendations (e.g., personalized recommendations), similar content can be recommended to entities with similar relationships. For example, for (Zhang San is a student of A High School Track and Field Team) (i.e., Zhang San is a student of A High School Track and Field Team), recommendations can be made to Zhang San based on information about other students of A High School in the knowledge base.

[0109] By applying the method and system for constructing a multivariate relational knowledge base according to an exemplary embodiment of the present invention, it is possible to establish a multivariate relational model for the multivariate relational knowledge base that has sufficient expressive power and can capture the implicit semantic relationships between multivariate relations. This allows for the accurate and rapid completion of missing entities in the multivariate relational knowledge base, thereby constructing a complete multivariate relational knowledge base. This knowledge base can then be further utilized to achieve more accurate predictions, recommendations, and other effects. Furthermore, the method and system for constructing a multivariate relational knowledge base according to an exemplary embodiment of the present invention can also simultaneously handle the learning problem of mixed metadata, further expanding the model's scope of application.

[0110] The above has been referred to Figures 1 to 3 A method and system for constructing a multivariate relational knowledge base according to an exemplary embodiment of the present disclosure are described. However, it should be understood that the devices and systems shown in the accompanying drawings can be configured as software, hardware, firmware, or any combination of the above items to perform specific functions. For example, these systems and devices may correspond to dedicated integrated circuits, pure software codes, or modules that combine software and hardware. In addition, one or more functions implemented by these systems or devices may also be uniformly performed by components in physical entity devices (e.g., processors, clients, or servers, etc.).

[0111] In addition, the above method can be implemented by instructions recorded on a computer-readable storage medium. For example, according to an exemplary embodiment of the present application, a computer-readable storage medium storing instructions can be provided, wherein, when the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the following steps: construct a multivariate relationship model for the multivariate groups in the multivariate relationship knowledge base, and establish corresponding scoring functions and loss functions, wherein each multivariate group in the multivariate relationship knowledge base is composed of a relationship and multiple entities, the corresponding semantics between the one relationship and each entity in the multiple entities is indicated by a role, and each role of the one relationship corresponds to the corresponding one in the multiple entities. The corresponding observation facts between entities constitute a set of observation facts corresponding to the tuple, wherein the scoring function is used to indicate the association between the roles and entities in the set of observation facts corresponding to each tuple; based on the scoring function and the loss function, the multivariate relationship model is trained using training samples obtained from the multivariate relationship knowledge base, wherein the training samples are sets of observation facts corresponding to tuples in the multivariate relationship knowledge base that do not have missing entities; based on the trained multivariate relationship model, the missing entities in the tuples with missing entities in the multivariate relationship knowledge base are predicted and completed, and a complete multivariate relationship knowledge base is constructed based on each tuple with completed missing entities.

[0112] The instructions stored in the computer-readable storage medium can be executed in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. It should be noted that the instructions can also be used to perform additional steps in addition to the above steps or perform more specific processing when performing the above steps. The contents of these additional steps and further processing have been referred to in the present invention. Figures 1 to 3 It is mentioned in the description of the related methods, so it will not be repeated here to avoid repetition.

[0113] It should be noted that the multi-relational knowledge base construction system according to the exemplary embodiment of the present disclosure can completely rely on the operation of computer programs or instructions to realize the corresponding functions, that is, each device corresponds to each step in the functional architecture of the computer program, so that the entire system is called through a special software package (for example, lib library) to realize the corresponding function.

[0114] On the other hand, when Figure 2 When the systems and devices shown are implemented in software, firmware, middleware, or microcode, the program code or code segments for performing the corresponding operations may be stored in a computer-readable medium such as a storage medium, so that at least one processor or at least one computing device may perform the corresponding operations by reading and running the corresponding program code or code segments.

[0115] For example, according to an exemplary embodiment of the present application, a system comprising at least one computing device and at least one storage device storing instructions may be provided, wherein the instructions, when executed by the at least one computing device, prompt the at least one computing device to perform the following steps: construct a multivariate relationship model for a multivariate relationship group in a multivariate relationship knowledge base, and establish a corresponding scoring function and a loss function, wherein each multivariate relationship group in the multivariate relationship knowledge base is composed of a relationship and multiple entities, the corresponding semantics between the relationship and each entity in the multiple entities is indicated by a role, and the corresponding semantics between each role of the relationship and the corresponding entity in the multiple entities are indicated by a role. The observed facts constitute a set of observed facts corresponding to the multivariate group, wherein the score function is used to indicate the association between the roles and entities in the set of observed facts corresponding to each multivariate group; based on the score function and the loss function, the multivariate relationship model is trained using training samples obtained from the multivariate relationship knowledge base, wherein the training samples are a set of observed facts corresponding to the multivariate groups without missing entities in the multivariate relationship knowledge base; based on the trained multivariate relationship model, the missing entities in the multivariate groups with missing entities in the multivariate relationship knowledge base are predicted and completed, and a complete multivariate relationship knowledge base is constructed based on each multivariate group with completed missing entities.

[0116] Specifically, the above system can be deployed in a server or client, or on a node in a distributed network environment. In addition, the system can be a PC, a tablet device, a personal digital assistant, a smart phone, a web application, or other device capable of executing the above instruction set. In addition, the system can also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.). In addition, all components of the system can be connected to each other via a bus and / or a network.

[0117] Here, the system is not necessarily a single system, but may be any collection of devices or circuits that can execute the above instructions (or instruction sets) individually or in combination. The system may also be part of an integrated control system or system manager, or may be configured as a portable electronic device that is interconnected with a local or remote (e.g., via wireless transmission) interface.

[0118] In the system, the at least one computing device may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, the at least one computing device may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, or the like. The computing device may run instructions or code stored in one of the storage devices, wherein the storage device may also store data. Instructions and data may also be sent and received over a network via a network interface device, wherein the network interface device may employ any known transmission protocol.

[0119] The storage device may be integrated with the computing device, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. Furthermore, the storage device may comprise a standalone device, such as an external disk drive, a storage array, or any other storage device usable by a database system. The storage device and the computing device may be operatively coupled or may be in communication with each other, for example, via an I / O port, a network connection, or the like, such that the computing device can access instructions stored in the storage device.

[0120] While the exemplary embodiments of the present application have been described above, it should be understood that the above description is merely illustrative and not exhaustive, and the present application is not limited to the disclosed exemplary embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the present application. Therefore, the scope of protection of the present application should be determined by the scope of the claims.

Claims

1. A prediction method, comprising: Constructing a multivariate relational model for a multivariate relational knowledge base, and establishing a corresponding scoring function and loss function, wherein each multivariate relational knowledge base is composed of a relation and multiple entities, the corresponding semantics between the one relation and each entity in the multiple entities are indicated by a role, and the corresponding observation facts between each role of the one relation and the corresponding entity in the multiple entities constitute a set of observation facts corresponding to the multivariate relation, wherein the scoring function is used to indicate the association between the role and the entity in the set of observation facts corresponding to each multivariate relation; Based on the scoring function and the loss function, the multivariate relation model is trained using training samples obtained from the multivariate relation knowledge base, wherein the training samples are a set of observation facts corresponding to multivariate groups without missing entities in the multivariate relation knowledge base; Predicting and completing missing entities in tuples having missing entities in the tuples of the tuples based on the trained tuple model, and constructing a complete tuple knowledge base based on the tuples with completed missing entities; Using a complete multi-relational knowledge base to train a machine learning model to obtain a target machine learning model, and using the target machine learning model to perform a prediction task, wherein the prediction task includes at least one of relational retrieval, semantic retrieval, intelligent retrieval, intelligent recommendation, intelligent question-answering, and personalized recommendation; For relational retrieval, semantic retrieval, or intelligent retrieval, multiple keywords are input into the target machine learning model, and the target machine learning model retrieves the relationship between the multiple keywords or retrieves another entity corresponding to the multiple keywords from a complete multi-relational knowledge base; For intelligent question answering, a question is input into the target machine learning model, and the target machine learning model retrieves the answer to the question from the complete multi-relationship knowledge base; or, the borrower's information is input into the target machine learning model, and the target machine learning model retrieves whether the borrower has a fraud risk from the complete multi-relationship knowledge base; For intelligent recommendation or personalized recommendation, the information of the person to be recommended is input into the target machine learning model, and the target machine learning model retrieves recommendation information for the person to be recommended from the complete multi-relationship knowledge base.

2. The method according to claim 1, wherein The steps to build the scoring function include: establishing a latent character space including a predetermined number of latent character basis vectors, and determining a character embedding vector for each character based on the latent character basis vectors; Mapping each entity to a predetermined number of semantic embedding vectors for semantics; Determine a role-sensitive pattern matrix for each role, where the role-sensitive pattern matrix indicates the interaction strength between the role and the semantic embedding vectors of each entity; The scoring function is established based on the role embedding vector, the semantic embedding vector and the role sensitive pattern matrix, Wherein, the role embedding vector of each role is represented by a combination of the latent role basis vectors, and The role-sensitive pattern matrix of each role is established based on the basic matrix associated with the potential role basic vector.

3. The method according to claim 2, wherein: The scoring function is expressed as: ， in, ， ， i =1, 2, ... , , ， , k =1, 2, ... , K and =1, 2, ... , K , in, The number of elements in the multi-relation knowledge base is ( +1) of the tuple, Indicates the relationship in the tuple and , Indicates the first i entities and Entity Domain , It's a relationship A collection of characters, Observational fact domain is the set of observation facts corresponding to the multi-tuple, ≥2 and is a positive integer, It's a relationship r No. i Characters The embedding vector of It's a role The corresponding role weight vector of the embedding vector, Indicated by the entity The matrix of m semantic embedding vectors, It's a role The role-sensitive pattern matrix and its j-th row Indicates the multi-tuple The jth entity in The interaction strength of the m semantic embedding vectors, is the first k latent role basis vectors and It is the same as the k The basic matrix associated with the potential role basis vectors, is a function normalized by the temperature parameter and is the character weight vector The temperature parameters and Is for the role sensitive mode matrix The temperature parameters, Where d indicates the embedding vector dimension of the character, K is the size of a preset latent character space and indicates the number of latent character basis vectors, and m is a predetermined value.

4. The method according to claim 3, wherein: The loss function is expressed as: ， in, , which means for positive samples Constructed Group negative samples A collection of is the scoring function, Representing entity domain The entity in the positive sample A different entity, Represents a replacement positive sample The q The set of negative samples obtained by entities, 1≤ q ≤ and q Is a positive integer.

5. The method according to claim 4, wherein: The step of training the multivariate relationship model using training samples obtained from the multivariate relationship knowledge base comprises: Initialize the model parameters of the multivariate relationship model; Based on the initialized model parameters, performing at least one iterative update operation to update the model parameters in the multivariate relational model, Each iterative update operation includes: Select a predetermined number of training samples as the positive sample set and set the initial loss parameter; For each positive sample in the positive sample set, perform an iterative loss update operation to update the loss parameter; Update the model parameters according to the loss gradient of the loss parameters, Among them, each iterative loss update operation includes: Construct a negative sample set of the current positive sample in the positive sample set; Based on the current positive sample, calculate the role embedding vector and role sensitive pattern matrix; Based on the calculated role embedding vector and role sensitive pattern matrix, the determined scoring function and loss function are used to calculate the current loss of the current positive sample and the set of negative samples constructed for the current positive sample, and the loss parameters obtained by the current iterative loss update operation are determined based on the calculated current loss and the loss parameters obtained by the previous iterative loss update operation.

6. The method of claim 1, wherein: The steps of predicting and completing missing entities in a multi-tuple with missing entities in the multi-tuple relational knowledge base based on the trained multi-tuple relational model include: predicting, based on the scoring function, an entity in the multi-relational knowledge base that is most likely to be a missing entity in the multi-tuple with missing entities; The predicted entities are used to complete missing entities in the multi-tuple having missing entities.

7. A prediction system, comprising: A model building device is configured to build a multivariate relationship model for a multivariate relationship group in a multivariate relationship knowledge base, and establish a corresponding scoring function and a loss function, wherein each multivariate relationship in the multivariate relationship knowledge base is composed of a relationship and multiple entities, the corresponding semantics between the relationship and each entity in the multiple entities are indicated by roles, and the corresponding observation facts between each role of the relationship and the corresponding entity in the multiple entities constitute an observation fact set corresponding to the multivariate relationship, wherein the scoring function is used to indicate the association between the role and the entity in the observation fact set corresponding to each multivariate relationship; A training device configured to train the multivariate relationship model using training samples obtained from the multivariate relationship knowledge base based on the scoring function and the loss function, wherein the training samples are a set of observation facts corresponding to multivariate groups in the multivariate relationship knowledge base that do not have missing entities; A knowledge base construction device configured to predict and complete missing entities in tuples with missing entities in the multi-tuple relation knowledge base based on the trained multi-tuple relation model, and to construct a complete multi-tuple relation knowledge base based on each tuple with completed missing entities; The complete multi-relation knowledge base is used to train a machine learning model to obtain a target machine learning model, and the target machine learning model is used to perform a prediction task, wherein the prediction task includes at least one of relational retrieval, semantic retrieval, intelligent retrieval, intelligent recommendation, intelligent question answering, and personalized recommendation; For relational retrieval, semantic retrieval, or intelligent retrieval, multiple keywords are input into the target machine learning model, and the target machine learning model retrieves the relationship between the multiple keywords or retrieves another entity corresponding to the multiple keywords from a complete multi-relational knowledge base; For intelligent question answering, a question is input into the target machine learning model, and the target machine learning model retrieves the answer to the question from the complete multi-relationship knowledge base; or, the borrower's information is input into the target machine learning model, and the target machine learning model retrieves whether the borrower has a fraud risk from the complete multi-relationship knowledge base; For intelligent recommendation or personalized recommendation, the information of the person to be recommended is input into the target machine learning model, and the target machine learning model retrieves recommendation information for the person to be recommended from the complete multi-relationship knowledge base.

8. The system of claim 7, wherein: The model building device is configured to establish a scoring function based on the following operations: establishing a latent character space including a predetermined number of latent character basis vectors, and determining a character embedding vector for each character based on the latent character basis vectors; Mapping each entity to a predetermined number of semantic embedding vectors for semantics; Determine a role-sensitive pattern matrix for each role, where the role-sensitive pattern matrix indicates the interaction strength between the role and the semantic embedding vectors of each entity; The scoring function is established based on the role embedding vector, the semantic embedding vector and the role sensitive pattern matrix, Wherein, the role embedding vector of each role is represented by a combination of the latent role basis vectors, and The role-sensitive pattern matrix of each role is established based on the basic matrix associated with the potential role basic vector.

9. The system of claim 8, wherein: The scoring function is expressed as: ， in, ， ， i =1, 2, ... , , ， , k =1, 2, ... , K and =1, 2, ... , K , in, The number of elements in the multi-relation knowledge base is ( +1) of the tuple, Indicates the relationship in the tuple and , Indicates the first i entities and Entity Domain , It's a relationship A collection of characters, Observational fact domain is the set of observation facts corresponding to the multi-tuple, ≥2 and is a positive integer, It's a relationship r No. i Characters The embedding vector of It's a role The corresponding role weight vector of the embedding vector, Indicated by the entity The matrix of m semantic embedding vectors, It's a role The role-sensitive pattern matrix and its j-th row Indicates the multi-tuple The jth entity in The interaction strength of the m semantic embedding vectors, is the first k latent role basis vectors and It is the same as the k The basic matrix associated with the potential role basis vectors, is a function normalized by the temperature parameter and is the character weight vector The temperature parameters and Is for the role sensitive mode matrix The temperature parameters, Where d indicates the embedding vector dimension of the character, K is the size of a preset latent character space and indicates the number of latent character basis vectors, and m is a predetermined value.

10. The system of claim 9, wherein: The loss function is expressed as: ， in, , which means for positive samples Constructed Group negative samples A collection of is the scoring function, Representing entity domain The entity in the positive sample A different entity, Represents a replacement positive sample The q The set of negative samples obtained by entities, 1≤ q ≤ and q Is a positive integer.

11. The system of claim 10, wherein: The training device is configured to train the multivariate relationship model by: Initialize the model parameters of the multivariate relationship model; Based on the initialized model parameters, performing at least one iterative update operation to update the model parameters in the multivariate relational model, Each iterative update operation includes: Select a predetermined number of training samples as the positive sample set and set the initial loss parameter; For each positive sample in the positive sample set, perform an iterative loss update operation to update the loss parameter; Update the model parameters according to the loss gradient of the loss parameters, Among them, each iterative loss update operation includes: Construct a negative sample set of the current positive sample in the positive sample set; Based on the current positive sample, calculate the role embedding vector and role sensitive pattern matrix; Based on the calculated role embedding vector and role sensitive pattern matrix, the determined scoring function and loss function are used to calculate the current loss of the current positive sample and the set of negative samples constructed for the current positive sample, and the loss parameters obtained by the current iterative loss update operation are determined based on the calculated current loss and the loss parameters obtained by the previous iterative loss update operation.

12. The system of claim 7, wherein: The knowledge base construction device is configured as follows: predicting, based on the scoring function, an entity in the multi-relational knowledge base that is most likely to be a missing entity in the multi-tuple with missing entities; The predicted entities are used to complete missing entities in the multi-tuple having missing entities.

13. A computer-readable storage medium storing instructions, wherein: When the instructions are executed by at least one computing device, the at least one computing device is prompted to perform the method according to any one of claims 1 to 6.

14. A system comprising at least one computing device and at least one storage device storing instructions, wherein: When the instructions are executed by the at least one computing device, the instructions cause the at least one computing device to perform the method according to any one of claims 1 to 6.

Images